docs: automation for int suite

docs/abra/hack.md

@@ -54,11 +54,42 @@ go test ./pkg/recipe -v -run TestGetVersionLabelLocalDoesNotUseTimeoutLabel
 
 ## Integration tests
 
-### Install dependencies
+### Running on the CI server
 
-We use [`bats`](https://bats-core.readthedocs.io/en/stable/), you can install
-the required dependencies with the following. You also need a working
-installation of Docker and Go (not covered in this section).
+Based on
+[R020](https://docs.coopcloud.tech/federation/resolutions/passed/020/), we have
+automated running the integration test suite. Here's the TLDR;
+
+* We have a donated CI server (tysm `@mirsal` 💝) standing at the ready,
+  `int.coopcloud.tech`.
+* We run the entire integration suite nightly via our Drone CI/CD configuration [here](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/.drone.yml) (see "`name: integration test`" stanza)
+
+What follows is a listing of how this was achieved so that we can collectivise
+the maintenance.
+
+On the server, we have:
+
+* Created an `abra` user with `docker` permissions
+* Ran `apt install bats bats-file bats-assert bats-support jq make git golang-1.21 wget bash`
+* Installed `bats-core` from source, following the instructions below
+* Docker was already installed on the machine, so nothing to do there
+* `docker login` with the `thecoopcloud` details so we don't get rate limited
+
+The drone configuration was wired up as follows:
+
+* Generated a SSH key and put the public key part in `~/.ssh/authorize_keys`
+* Added that public key part as a "deploy key" in the abra repo (so we can do `ssh://` git remote pulls)
+* Added the private key part as a Drone secret which is available in build so that the build can SSH over to the server to run commands. That was done like so: `drone secret add --repository coop-cloud/abra --name abra_int_private_key --data @id_ed25519`
+
+Please ask `@decentral1se` or on the Matrix channels for SSH access to the machine.
+
+### Running them locally
+
+#### Install dependencies
+
+We use [`bats`](https://bats-core.readthedocs.io/en/stable/) to run the tests.
+You can install the required dependencies with the following. You also need a
+working installation of Docker and Go >= 1.16 (not covered in this section).
 
 ```
 apt install bats-file bats-assert bats-support jq make git
@@ -75,12 +106,14 @@ cd bats-core
 sudo ./install.sh /usr/local
 ```
 
-### Setup Test Server
+#### Setup Test Server
 
-For many tests an actual server is needed, where apps can be deployed. You can
-either use a local one or a remote test server.
+For some tests an actual server is needed, where apps can be deployed. You can
+either use a local one or a remote test server. There is also a way to run or
+skip tests that require a remote server. This is covered below in the
+[filtering tests](#filter-tests_1) section.
 
-#### With remote test server
+##### Remote swarm
 
 ```
 export ABRA_TEST_DOMAIN="test.example.com"
@@ -89,14 +122,9 @@ export ABRA_DIR="$HOME/.abra_test"
 
 `ABRA_TEST_DOMAIN` should also have a DNS A record for `*.test.example.com`
 which points to the same server so that the test suite can deploy apps freely.
-It's advised that you re-use the same server and therefore the same Traefik
-deployment for running your integration tests. The test suite does not deploy
-Traefik for you. Then you'll have more stable results.
+The test suite does not deploy Traefik for you.
 
-You probably don't want to run the entire test suite though, it takes a while.
-Try the following for starters.
-
-#### With local swarm
+##### Local swarm
 
 When running the test suite localy you need a running docker swarm setup:
 
@@ -128,10 +156,11 @@ bats -Tp tests/integration/autocomplete.bats
 
 ### Tagging tests
 
-When a test actually deploys something to a server, we tag it with the following:
+When a test actually deploys something, we tag it as "slow". When the test
+requires public DNS, we use "dns". There may be more tags we write more tests.
 
 ```
-# bats test_tags=slow
+# bats test_tags=slow,dns
 @test "..." {
   ...
 }
@@ -166,14 +195,17 @@ bats -Tp tests/integration --filter "validate app argument"
 You can filter on tags.
 
 ```
-bats -Tp tests/integration --filter-tags "\!slow" # only fast tests
-bats -Tp tests/integration --filter-tags "slow"   # only slow tests
+bats -Tp tests/integration --filter-tags \!slow      # only fast tests
+bats -Tp tests/integration --filter-tags slow        # only slow tests
+bats -Tp tests/integration --filter-tags slow,\!dns  # slow but no DNS tests
 ```
 
 You can also only run the previously failed tests.
 
 ```
-bats -TP tests/integration --filter-status failed
+mkdir -p tests/integration/.bats/run-logs
+bats -Tp tests/integration                        # run tests
+bats -Tp tests/integration --filter-status failed # re-run only failed
 ```
 
 ### Debug tests