Adding Tests

We will now add tests for the models and the views and a few functional tests in the tests package. Tests ensure that an application works, and that it continues to work when changes are made in the future.

Test harness

The project came bootstrapped with some tests and a basic harness. These are located in the tests package at the top-level of the project. It is a common practice to put tests into a tests package alongside the application package, especially as projects grow in size and complexity. A useful convention is for each module in the application to contain a corresponding module in the tests package. The test module would have the same name with the prefix test_.

The harness consists of the following setup:

  • pytest.ini - controls basic pytest configuration, including where to find the tests. We have configured pytest to search for tests in the application package and in the tests package.

  • .coveragerc - controls coverage config. In our setup, it works with the pytest-cov plugin that we use via the --cov options to the pytest command.

  • testing.ini - a mirror of development.ini and production.ini that contains settings used for executing the test suite. Most importantly, it contains the database connection information used by tests that require the database.

  • tests_require in setup.py - controls the dependencies installed when testing. When the list is changed, it is necessary to re-run $VENV/bin/pip install -e ".[testing]" to ensure the new dependencies are installed.

  • tests/conftest.py - the core fixtures available throughout our tests. The fixtures are explained in more detail in the following sections. Open tests/conftest.py and follow along.

Session-scoped test fixtures

  • app_settings - the settings dict parsed from the testing.ini file that would normally be passed by pserve into your app's main function.

  • app - the Pyramid WSGI application, implementing the pyramid.interfaces.IRouter interface. Most commonly this would be used for functional tests.

Per-test fixtures

  • tm - a transaction.TransactionManager object controlling a transaction lifecycle. Generally other fixtures would join to the tm fixture to control their lifecycle and ensure they are aborted at the end of the test.

  • testapp - a webtest.TestApp instance wrapping the app and is used to sending requests into the application and return full response objects that can be inspected. The testapp is able to mutate the request environ such that the tm fixture is injected and used by any code that touches request.tm. This should join the request.root ZODB model to the transaction manager as well, to enable rolling back changes to the database. The testapp maintains a cookiejar, so it can be used to share state across requests, as well as the transaction database connection.

  • app_request - a pyramid.request.Request object that can be used for more lightweight tests versus the full testapp. The app_request can be passed to view functions and other code that need a fully functional request object.

  • dummy_request - a pyramid.testing.DummyRequest object that is very lightweight. This is a great object to pass to view functions that have minimal side-effects as it will be fast and simple.

Unit tests

We can test individual APIs within our codebase to ensure they fulfill the expected contract that the rest of the application expects. For example, we will test the password hashing features we added to tutorial.security and the rest of our models.

Create tests/test_models.py such that it appears as follows:

 1from tutorial import models
 2
 3def test_page_model():
 4    instance = models.Page(data='some data')
 5    assert instance.data == 'some data'
 6
 7def test_wiki_model():
 8    wiki = models.Wiki()
 9    assert wiki.__parent__ is None
10    assert wiki.__name__ is None
11
12def test_appmaker():
13    root = {}
14    models.appmaker(root)
15    assert root['app_root']['FrontPage'].data == 'This is the front page'
16
17def test_password_hashing():
18    from tutorial.security import hash_password, check_password
19
20    password = 'secretpassword'
21    hashed_password = hash_password(password)
22    assert check_password(hashed_password, password)
23    assert not check_password(hashed_password, 'attackerpassword')
24    assert not check_password(None, password)

Integration tests

We can directly execute the view code, bypassing Pyramid and testing just the code that we have written. These tests use dummy requests that we will prepare appropriately to set the conditions each view expects.

Update tests/test_views.py such that it appears as follows:

 1from pyramid import testing
 2
 3
 4class Test_view_wiki:
 5    def test_it_redirects_to_front_page(self):
 6        from tutorial.views.default import view_wiki
 7        context = testing.DummyResource()
 8        request = testing.DummyRequest()
 9        response = view_wiki(context, request)
10        assert response.location == 'http://example.com/FrontPage'
11
12class Test_view_page:
13    def _callFUT(self, context, request):
14        from tutorial.views.default import view_page
15        return view_page(context, request)
16
17    def test_it(self):
18        wiki = testing.DummyResource()
19        wiki['IDoExist'] = testing.DummyResource()
20        context = testing.DummyResource(data='Hello CruelWorld IDoExist')
21        context.__parent__ = wiki
22        context.__name__ = 'thepage'
23        request = testing.DummyRequest()
24        info = self._callFUT(context, request)
25        assert info['page'] == context
26        assert info['page_text'] == (
27            '<div class="document">\n'
28            '<p>Hello <a href="http://example.com/add_page/CruelWorld">'
29            'CruelWorld</a> '
30            '<a href="http://example.com/IDoExist/">'
31            'IDoExist</a>'
32            '</p>\n</div>\n')
33        assert info['edit_url'] == 'http://example.com/thepage/edit_page'
34
35
36class Test_add_page:
37    def _callFUT(self, context, request):
38        from tutorial.views.default import add_page
39        return add_page(context, request)
40
41    def test_it_notsubmitted(self):
42        context = testing.DummyResource()
43        request = testing.DummyRequest()
44        request.subpath = ['AnotherPage']
45        info = self._callFUT(context, request)
46        assert info['page'].data == ''
47        assert info['save_url'] == request.resource_url(
48            context, 'add_page', 'AnotherPage')
49
50    def test_it_submitted(self):
51        context = testing.DummyResource()
52        request = testing.DummyRequest({
53            'form.submitted': True,
54            'body': 'Hello yo!',
55        })
56        request.subpath = ['AnotherPage']
57        self._callFUT(context, request)
58        page = context['AnotherPage']
59        assert page.data == 'Hello yo!'
60        assert page.__name__ == 'AnotherPage'
61        assert page.__parent__ == context
62
63class Test_edit_page:
64    def _callFUT(self, context, request):
65        from tutorial.views.default import edit_page
66        return edit_page(context, request)
67
68    def test_it_notsubmitted(self):
69        context = testing.DummyResource()
70        request = testing.DummyRequest()
71        info = self._callFUT(context, request)
72        assert info['page'] == context
73        assert info['save_url'] == request.resource_url(context, 'edit_page')
74
75    def test_it_submitted(self):
76        context = testing.DummyResource()
77        request = testing.DummyRequest({
78            'form.submitted': True,
79            'body': 'Hello yo!',
80        })
81        response = self._callFUT(context, request)
82        assert response.location == 'http://example.com/'
83        assert context.data == 'Hello yo!'

Functional tests

We will test the whole application, covering security aspects that are not tested in the unit and integration tests, like logging in, logging out, checking that the basic user cannot edit pages that it did not create, but that the editor user can, and so on.

Update tests/test_functional.py such that it appears as follows:

 1viewer_login = (
 2    '/login?login=viewer&password=viewer'
 3    '&came_from=FrontPage&form.submitted=Login'
 4)
 5viewer_wrong_login = (
 6    '/login?login=viewer&password=incorrect'
 7    '&came_from=FrontPage&form.submitted=Login'
 8)
 9editor_login = (
10    '/login?login=editor&password=editor'
11    '&came_from=FrontPage&form.submitted=Login'
12)
13
14def test_root(testapp):
15    res = testapp.get('/', status=303)
16    assert res.location == 'http://example.com/FrontPage'
17
18def test_FrontPage(testapp):
19    res = testapp.get('/FrontPage', status=200)
20    assert b'FrontPage' in res.body
21
22def test_missing_page(testapp):
23    res = testapp.get('/SomePage', status=404)
24    assert b'Not Found' in res.body
25
26def test_referrer_is_login(testapp):
27    res = testapp.get('/login', status=200)
28    assert b'name="came_from" value="/"' in res.body
29
30def test_successful_log_in(testapp):
31    res = testapp.get(viewer_login, status=303)
32    assert res.location == 'http://example.com/FrontPage'
33
34def test_failed_log_in(testapp):
35    res = testapp.get(viewer_wrong_login, status=400)
36    assert b'login' in res.body
37
38def test_logout_link_present_when_logged_in(testapp):
39    res = testapp.get(viewer_login, status=303)
40    res = testapp.get('/FrontPage', status=200)
41    assert b'Logout' in res.body
42
43def test_logout_link_not_present_after_logged_out(testapp):
44    res = testapp.get(viewer_login, status=303)
45    res = testapp.get('/FrontPage', status=200)
46    res = testapp.get('/logout', status=303)
47    assert b'Logout' not in res.body
48
49def test_anonymous_user_cannot_edit(testapp):
50    res = testapp.get('/FrontPage/edit_page', status=200)
51    assert b'Login' in res.body
52
53def test_anonymous_user_cannot_add(testapp):
54    res = testapp.get('/add_page/NewPage', status=200)
55    assert b'Login' in res.body
56
57def test_viewer_user_cannot_edit(testapp):
58    res = testapp.get(viewer_login, status=303)
59    res = testapp.get('/FrontPage/edit_page', status=200)
60    assert b'Login' in res.body
61
62def test_viewer_user_cannot_add(testapp):
63    res = testapp.get(viewer_login, status=303)
64    res = testapp.get('/add_page/NewPage', status=200)
65    assert b'Login' in res.body
66
67def test_editors_member_user_can_edit(testapp):
68    res = testapp.get(editor_login, status=303)
69    res = testapp.get('/FrontPage/edit_page', status=200)
70    assert b'Editing' in res.body
71
72def test_editors_member_user_can_add(testapp):
73    res = testapp.get(editor_login, status=303)
74    res = testapp.get('/add_page/NewPage', status=200)
75    assert b'Editing' in res.body
76
77def test_editors_member_user_can_view(testapp):
78    res = testapp.get(editor_login, status=303)
79    res = testapp.get('/FrontPage', status=200)
80    assert b'FrontPage' in res.body

Running the tests

We can run these tests by using pytest similarly to how we did in Run the tests. Courtesy of the cookiecutter, our testing dependencies have already been satisfied. pytest and coverage have already been configured. We can jump right to running tests.

On Unix:

$VENV/bin/pytest -q

On Windows:

%VENV%\Scripts\pytest -q

The expected result should look like the following:

.........................
25 passed in 3.87 seconds