docs: Resolution 012 Budget 006

Thanks @V

Dockerfile

@@ -1,4 +1,4 @@
-FROM squidfunk/mkdocs-material:9.1.19
+FROM squidfunk/mkdocs-material:9.2.8
 
 EXPOSE 8000
 

docs/abra/hack.md

@@ -20,7 +20,47 @@ Our [Drone CI configuration](https://git.coopcloud.tech/coop-cloud/abra/src/bran
 
 Please use the [conventional commit format](https://www.conventionalcommits.org/en/v1.0.0/) for your commits so we can automate our change log.
 
-### Using the `abra` public API
+## Integration tests
+
+We use [`bats`](https://bats-core.readthedocs.io/en/stable/), you can install
+the required dependencies with the following. You also need a working
+installation of Docker and Go.
+
+```
+apt install bats bats-file bats-assert bats-support jq make git
+```
+
+Then you can run the integration test suite with the following:
+
+```
+export ABRA_TEST_DOMAIN="test.example.com"
+export ABRA_DIR="$HOME/.abra_test"
+bats tests/integration
+```
+
+`ABRA_TEST_DOMAIN` should also have a DNS A record for `*.test.example.com`
+which points to the same server so that the test suite can deploy apps freely.
+It's advised that you re-use the same server and therefore the same Traefik
+deployment for running your integration tests. Then you'll have more stable
+results.
+
+If you're hacking on the tests, you can speed things up with the following.
+
+```
+export ABRA_SKIP_TEARDOWN=1
+```
+
+This will avoid nuking `$ABRA_DIR` for each test which avoids the costly `git
+clone` of the catalogue on each test run. For the actual CI testing, we
+probably want this clean slate on each test to avoid confusion.
+
+You can filter on test names to run specific kinds of tests.
+
+```
+bats tests/integration --filter "validate app argument"
+```
+
+## Using the `abra` public API
 
 Warning, there is currently no stability promise for the `abra` public API! Most of the internals are exposed in order to allow a free hand for developers to try build stuff. If people start to build things then we can start the discussion on what is useful to have open/closed and keep stable etc. Please let us know if you depend on the APIs!
 
@@ -61,7 +101,7 @@ Some tools that are making use of the API so far are:
 
 * [`kadabra`](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/cmd/kadabra/main.go)
 
-### Cross-compiling
+## Cross-compiling
 
 If there's no official release for the architecture you use, you can cross-compile `abra` very easily. Clone the source code from [here](https://git.coopcloud.tech/coop-cloud/abra) and then:
 
@@ -70,6 +110,18 @@ If there's no official release for the architecture you use, you can cross-compi
 - run `git checkout <tag>`, where `<tag>` is the latest version
 - run `GOOS=<os> GOARCH=<arch> [GOARM=<arm>] make build`. You only have to use `GOARM` if you're building for ARM, this specifies the ARM version (5,6,7 etc). See [this](https://go.dev/doc/install/source#environment) for a list of all supported OS'es and architectures.
 
+## Building in Docker
+
+If you are living under a curse of constant Go environment problems, it might be easier to build `abra` using Docker:
+
+```
+sudo setenforce 0  # SELinux probably won't allow Docker to access files
+docker run -it -v $PWD:/abra golang:1.19.6 bash
+cd /abra
+. .envrc
+git config --global --add safe.directory /abra  # work around funky file permissions
+make build
+```
 
 ## Release management
 

docs/abra/upgrade.md

@@ -18,9 +18,29 @@ abra upgrade --rc
 
 ## Migration guides
 
-### `0.6.x-beta` -> `0.7.x-beta`
+> General release notes are [here](https://git.coopcloud.tech/coop-cloud/abra/releases/)
 
-> General release notes are [here](https://git.coopcloud.tech/coop-cloud/abra/releases/tag/0.7.0-beta)
+### `0.7.x-beta` -> `0.8.x-beta`
+
+> Currently still a release candidate! Please help test 😌
+
+- We now have an `--offline` flag instead of relying on internal logic to try
+  to decide when offline/online works best. It's up to you! A lot of `abra`
+  operations require network access, so it is not really truly "offline". The
+  logic prefers local filesystem access when this flag is passed. E.g. if there
+  is a local copy of the catalogue, then don't `git pull`.
+
+- There is a new linting rule for catching invalid tags in recipe versions.
+  This is an seemingly unavoidable issue that requires some maintenance work.
+  If you run into the error, here's some
+  [docs](https://docs.coopcloud.tech/maintainers/handbook/#r014-invalid-lightweight-tag)
+  to help work through it.
+
+- `~/.abra/catalogue` is now *only* updated via `git pull`. You may need to
+  `cd ~/.abra/catalogue && git checkout .` to get `abra` to stop complaining about
+  unstaged changes.
+
+### `0.6.x-beta` -> `0.7.x-beta`
 
 - **ALERTA, ALERTA**, security related issue: all `$domain.env` env vars are now exposed to the deployment via the `app` service container. Each `FOO=BAR` is exported within the context of the container. If you have any privately committed secrets in your `.env` files, please migrate them to the `secrets: ...` configuration in the recipe. This change was made to facilitate tooling which can support auto-upgrading of apps in a deployment.
 

docs/federation/resolutions/in-progress/012.md

@@ -0,0 +1,31 @@
+---
+title: "Resolution 012: Budget 006: Abra integration test suite - 2023-09-08"
+---
+
+- Deadline: 2023-09-22
+- Size: Large
+
+### Summary
+
+Recent attempts to continue developing features and fixes have lead to several regressions in Abra. It's more difficult to make progress and it's unclear if changes will create more bugs. Abra has always had bugs and we've relied on user testing to smoke those out.
+
+However, we're reaching a scale where it's very difficult to test all changes with just user testing. Also, there is only so far you can go with user testing before bug fatigue and frustration is the norm.
+
+It's time to build a robust Abra integration test suite which can help us stop regressions from happening and rely less on user testing. This will help us move forward with larger changes in Abra which we can then test and have confidence that the base functionality is not broken.
+
+### Details (Budget 006)
+
+References so far:
+- [3wc & myself (d1) have had a planning meeting](https://pad.autonomic.zone/kdLrPXMSSb2TZezCBhdYtw?edit)
+- [The first PR and proof of concept has landed in Abra](https://git.coopcloud.tech/coop-cloud/abra/pulls/347)
+- [Initial documentation has been written](https://docs.coopcloud.tech/abra/hack/#integration-tests)
+
+With some further experimentation, I'm relatively confident that this approach will allow us to implement an integration test suite which covers the majority of the Abra functionality. It's *a lot* of work. I'm estimating this to come in at 30 hours of work.
+
+**Budget amount**: 600 EUR (30 hrs * 20 EUR/hr)
+
+**Who will implement this**: decentral1se
+
+**When will the money be spent**: Before the end of October 2023.
+
+**What is the money for**: Implementing an Abra integration test suite.

docs/federation/resolutions/passed/010.md

@@ -25,7 +25,7 @@ If the fix is urgent and things need to move faster, please state so on the tick
 
 ### Details (Budget 004)
 
-This budget stands open and available for use until the common fund buffer is reached. Please see Resolution 009 for further details on the buffer.
+This budget stands open and available for use until the common fund buffer is reached. Please see [Resolution 009](https://docs.coopcloud.tech/federation/resolutions/passed/009/) for further details on the buffer.
 
 Please see [Resolution 003](https://docs.coopcloud.tech/federation/resolutions/passed/003/) for details on the hourly rate for this work.
 

docs/federation/resolutions/passed/011.md

@@ -1,5 +1,5 @@
 ---
-title: "Resolution 008: Budget 005: Backup improvements - 2023-07-23"
+title: "Resolution 011: Budget 005: Backup improvements - 2023-07-23"
 ---
 
 - Deadline: 2022-08-06
@@ -7,7 +7,7 @@ title: "Resolution 008: Budget 005: Backup improvements - 2023-07-23"
 
 ### Summary
 
-Agree Budget 005, for up to 10 hours (€200) to implement changes to the Co-op Cloud backup system, including `abra`, and `backup-bot-two`. 
+Agree Budget 005, for up to 10 hours (€200) to implement changes to the Co-op Cloud backup system, including `abra`, and `backup-bot-two`.
 
 More information, including background to the changes, and breakdown of the work, can be found here: https://pad.local-it.org/4chDYXkBQZywXKj0zs_Ymg#
 

docs/maintainers/handbook.md

@@ -646,3 +646,74 @@ export APP_ENTRYPOINT_VERSION=v5
 ```
 
 You should be able to deploy this overriden configuration now.
+
+## Linting rules
+
+### R014: "invalid lightweight tag"
+
+This is an issue related to the way Git/`go-git` handle Git tags internally. We
+need to use "annotated tags" and not "lightweight tags" for our recipe versions
+tags. Otherwise, `abra` has a hard time parsing what is going on.
+
+The `R01O4` linting error happens because the recipe in question has a
+lightweight tag. This needs to be replaced. This is a manual process. Here's a
+practical example with the Gitea recipe when we had this issue.
+
+You can validate what kind of tag is which by running the following:
+
+```
+git for-each-ref refs/tags
+734045872a57d795cd54b1992a1753893a4934f1 tag    refs/tags/1.0.0+1.14.5-rootless
+b2cefa5ccf2f2f77dae54cf6c304cccecb3547ca tag    refs/tags/1.1.0+1.15.0-rootless
+6d669112d8caafcdcf4eb1485f2d6afdb54a8e30 tag    refs/tags/1.1.1+1.15.3-rootless
+64761ad187cc7a3984a37dd9abd4fa16979f97b9 tag    refs/tags/1.1.2+1.15.6-rootless
+1ccb1cb6a63a08eebf6ba5508b676eaaccba7ed8 tag    refs/tags/1.1.3+1.15.10-rootless
+b86e1f6dfef3c464b16736274b3cd95f8978f66b tag    refs/tags/1.2.0+1.16.3-rootless
+b1d22f3c39ca768a4efa1a0b9b9f780268c924b3 tag    refs/tags/1.2.1+1.16.8-rootless
+85a45aa749427822a73ef62b6362d57bae1a61af tag    refs/tags/1.3.0+1.17.2-rootless
+f35689989c0b57575b8362e1252476d8133dc961 commit refs/tags/1.3.1+1.17.3-rootless
+df015fae592fca7728a3f0835217e110da4dbafc tag    refs/tags/2.0.0+1.18.0-rootless
+71920adb0c25a59f7678894e39f1a705f0ad08dd tag    refs/tags/2.0.1+1.18.2-rootless
+1ab9a96922341c8e54bdb6d60850630cce4b9587 tag    refs/tags/2.1.0+1.18.5-rootless
+1e612d84a2ad7c9beb7aa064701a520c7e91eecc commit refs/tags/2.1.2+1.19.3-rootless
+0bee99615a8bbd534a66a315ee088af3124e054b tag    refs/tags/2.2.0+1.19.3-rootless
+699378f53501b2d5079fa62cc7f8e79930da7540 tag    refs/tags/2.3.0+1.20.1-rootless
+c0dc5f82930d875c0a6e29abc016b4f6a53b83dd tag    refs/tags/2.3.1+1.20.1-rootless
+```
+
+Where `f35689989c0b57575b8362e1252476d8133dc961` &
+`1e612d84a2ad7c9beb7aa064701a520c7e91eecc` need to be removed ("commit"). We
+will deal with `refs/tags/1.3.1+1.17.3-rootless` in this example.
+
+```
+# find the tag hash
+git show 1.3.1+1.17.3-rootless
+commit f35689989c0b57575b8362e1252476d8133dc961 (tag: 1.3.1+1.17.3-rootless)
+Merge: af97db8 1d4dc8e
+Author: decentral1se <decentral1se@noreply.git.coopcloud.tech>
+Date:   Sun Nov 13 21:54:01 2022 +0000
+
+    Merge pull request 'Adding Oauth2 options and up on versions' (#29) from javielico/gitea:master into master
+
+    Reviewed-on: https://git.coopcloud.tech/coop-cloud/gitea/pulls/29
+
+# delete the tag locally / remotely
+git tag -d 1.3.1+1.17.3-rootless
+git push origin 1.3.1+1.17.3-rootless --delete
+
+# re-tag, this time with `-a` (annotated)
+git checkout f35689989c0b57575b8362e1252476d8133dc961
+git tag -a 1.3.1+1.17.3-rootless
+
+# push new tag
+git checkout master # might be main on other recipes!
+git push origin master --tags
+
+# check everything works
+git for-each-ref refs/tags | grep 1.3.1+1.17.3-rootless
+964f1680000fbba6daa520aa8d533a53ad151ab8 tag refs/tags/1.3.1+1.17.3-rootless
+```
+
+That's it! Spread the word, use `-a` when tagging recipe versions manually! Or
+just use `abra` which should handle this issue automagically for you in all
+cases 🎉

mkdocs.yml

@@ -7,13 +7,15 @@ use_directory_urls: true
 theme:
   name: material
   features:
+    - content.action.edit
     - navigation.expand
+    - navigation.indexes
     - navigation.instant
+    - navigation.path
     - navigation.sections
     - navigation.tabs
     - navigation.tabs.sticky
-    - navigation.indexes
-    - content.action.edit
+    - navigation.tracking
   palette:
     primary: light pink
     accent: purple
@@ -83,10 +85,14 @@ nav:
           - federation/resolutions/passed/004.md
           - federation/resolutions/passed/005.md
           - federation/resolutions/passed/006.md
+          - federation/resolutions/passed/007.md
+          - federation/resolutions/passed/008.md
+          - federation/resolutions/passed/009.md
+          - federation/resolutions/passed/010.md
+          - federation/resolutions/passed/011.md
         - "In progress":
           - federation/resolutions/in-progress/index.md
-          - federation/resolutions/in-progress/009.md
-          - federation/resolutions/in-progress/010.md
+          - federation/resolutions/in-progress/012.md
         - "Draft":
           - federation/resolutions/drafts/index.md
       - "Finance": federation/finance.md

requirements.txt

@@ -1,4 +1,4 @@
-mkdocs-awesome-pages-plugin==2.9.1
+mkdocs-awesome-pages-plugin==2.9.2
 mkdocs-material-extensions==1.1.1
-mkdocs-material==9.1.19
-mkdocs==1.4.3
+mkdocs-material==9.2.8
+mkdocs==1.5.2