Add preliminary automated tests #1

3wordchant · 2021-07-19T21:09:26Z

3wordchant commented

2021-07-19 21:09:26 +00:00

As discussed in #services:cyberia.club — automated tests could make it safer to contribute code, easier to review new code, and much easier to refactor, or upgrade Python dependencies.

To run tests:

create a Postgres database called capsulflask_test
run python -m unittest

Architecture

I tried to make the absolute minimal changes to be able to override settings in tests – possible alternative approaches include accepting an argument to create_app() to define which env file to load, or adding conditional logic to create_app() to pre-load specific settings before running load_dotenv() – but allowing env vars to override dotenv vars seemed cleanest. (Thanks @forest for improving on this approach)

Creating test databases

One outstanding question is how to initialise/reinitialise the test database.

Currently, the tests rely on the existence of a capsulflask_test database on localhost, accessible by the postgres user with password dev.

I create this manually using:
docker exec -it d1702306f409 psql -U postgres createdb -O postgres capsulflask_test;

In between test runs, you can either drop and recreate that database, or manually clear data using:

docker exec -it d1702306f409 psql -U postgres capsulflask_test -c "DELETE FROM vms; DELETE FROM login_tokens; DELETE FROM ssh_public_keys; DELETE FROM api_tokens; DELETE FROM accounts;

Test coverage

This tests the "landing" (public) pages, login, capsul index and creation. I didn't add automated coverage reporting yet, unclear if that seems useful.

As discussed in #services:cyberia.club — automated tests could make it safer to contribute code, easier to review new code, and much easier to refactor, or upgrade Python dependencies. To run tests: 1. create a Postgres database called `capsulflask_test` 2. run `python -m unittest` ## Architecture I tried to make the absolute minimal changes to be able to override settings in tests – possible alternative approaches include accepting an argument to create_app() to define which env file to load, or adding conditional logic to create_app() to pre-load specific settings before running load_dotenv() – but allowing env vars to override dotenv vars seemed cleanest. (Thanks @forest for improving on this approach) ## Creating test databases One outstanding question is how to initialise/reinitialise the test database. Currently, the tests rely on the existence of a capsulflask_test database on localhost, accessible by the postgres user with password dev. I create this manually using: ` docker exec -it d1702306f409 psql -U postgres createdb -O postgres capsulflask_test;` In between test runs, you can either drop and recreate that database, or manually clear data using: `docker exec -it d1702306f409 psql -U postgres capsulflask_test -c "DELETE FROM vms; DELETE FROM login_tokens; DELETE FROM ssh_public_keys; DELETE FROM api_tokens; DELETE FROM accounts;` ## Test coverage This tests the "landing" (public) pages, login, capsul index and creation. I didn't add automated coverage reporting yet, unclear if that seems useful.

3wordchant added 5 commits 2021-07-19 21:09:27 +00:00

665e087bd4 Basic testing using flask-testing

This commit makes it possible to override settings during tests, by
switching capsulflask/__init__.py to a "create_app" pattern, and using
`dotenv_values` instead of `load_dotenv`.

The create_app() method returns a Flask app instance, to give
more control over when to initialise the app. This allows setting
environment variables in test files.

Then, use dotenv_values to override loaded .env variables with ones from
the environment, so that tests can set `POSTGRES_CONNECTION_PARAMETERS`
and `SPOKE_MODEL` (possibly others in future..).

Inital tests for the "landing" pages, and login / activation, are
included.

4c7caa1a38 Initial console tests

NB capsul create isn't working properly, see #83

9823bda4a2 Add SSH key tests

7b9e9debab Disable VM creation check for the moment

0268e4c97d ensure that app is defined in app.py to fix login link logging issues.

also shuffled some things around for cleanliness

decentral1se approved these changes 2021-07-20 21:09:08 +00:00

decentral1se left a comment

LGTM! Super nice to get some tests in.

FYI, I will note that Pytest is the de facto standard for testing which is often picked up instead of good 'ol nosetest. It has a plugin in https://pytest-flask.readthedocs.io/en/latest/ and there is a rich ecosystem of plugins in general in https://docs.pytest.org/en/latest/reference/plugin_list.html. For example, there is a plugin to help with setting up + wiping the database between tests: https://pypi.org/project/pytest-postgresql/.

LGTM! Super nice to get some tests in. FYI, I will note that Pytest is the de facto standard for testing which is often picked up instead of good 'ol nosetest. It has a plugin in https://pytest-flask.readthedocs.io/en/latest/ and there is a rich ecosystem of plugins in general in https://docs.pytest.org/en/latest/reference/plugin_list.html. For example, there is a plugin to help with setting up + wiping the database between tests: https://pypi.org/project/pytest-postgresql/.

3wordchant commented

2021-07-20 21:54:30 +00:00

Ha, I kept flip flopping between unittest and pytest and ended up going with unittest (via flask-testing) copypastad from a previous project.

Do you think it's better to go with pytest straight from the start, if we're going to? i.e. is it worth merging this as-is or should we switch first?

Ha, I kept flip flopping between `unittest` and `pytest` and ended up going with `unittest` (via flask-testing) copypastad from a previous project. Do you think it's better to go with pytest straight from the start, if we're going to? i.e. is it worth merging this as-is or should we switch first?

decentral1se commented

2021-07-20 22:09:47 +00:00

You can get Pytest to run your Nosetests so I think you can just merge this as-is. See https://docs.pytest.org/en/6.2.x/nose.html for more.

👀 1

This pull request has changes conflicting with the target branch.

capsulflask/__init__.py

You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.

git checkout -b tests master

git pull origin tests

Step 2:

Merge the changes and update on Gitea.

git checkout master

git merge --no-ff tests

git push origin master

Sign in to join this conversation.

No reviewers

No Label

No Milestone

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: 3wordchant/capsul-flask#1