feat: finally write those upgrade docs

docs/maintainers/upgrade.md

@@ -2,14 +2,78 @@
 title: How to Upgrade a Recipe
 ---
 
+!!! warning "The recipe config commons is people!"
+
+    This documentation is all about helping you make recipe configuration
+    changes that won't break other peoples upgrade deployments :heart: There
+    are some specific maintainer responsibilities which are not exactly
+    obvious. This page is all about making them simple and clear to understand.
+    We hope it's clear and if not, please send documentation patches!
+
 ## Updating versions in the `abra.sh`
 
-`#TODO`
+If you update a config in a recipe, you also need to update the associated config version.
+
+For example, in the [Traefik](https://git.coopcloud.tech/coop-cloud/traefik)
+recipe, we have the following in
+[`compose.yml`](https://git.coopcloud.tech/coop-cloud/traefik/src/branch/master/compose.yml#L104-L108):
+
+```yaml
+configs:
+  traefik_yml:
+    name: ${STACK_NAME}_traefik_yml_${TRAEFIK_YML_VERSION}
+    file: traefik.yml.tmpl
+    template_driver: golang
+```
+And in the [`abra.sh`](https://git.coopcloud.tech/coop-cloud/traefik/src/commit/9a46c85735701780dc8f0717a4e6cab4969420fc/abra.sh#L1), we have:
+
+```bash
+export TRAEFIK_YML_VERSION=v29
+```
+
+If you update teh `traefik.yml.tmpl` with new changes, you **must** increment
+`TRAEFIK_YML_VERSION`. Otherwise, other Co-op Cloud operators will see an
+obscure error message when they try to deploy your new version.
 
 ## Backwards compatible environment variable changes
 
-`#TODO`
+If you add a new environment variable to the `.env.sample` then you create
+manual upgrade work for other Co-op Cloud operators. Depending on the
+functionality that the environment variable enables or how the recipe is
+already configured, it is possible to provide a default which maintains
+backwards compatibility. There is hard and fast rule for this and it depends on
+the recipe and context.
+
+A typical way to do this, is to provide a default in the `environment` stanza.
+For example, providing a default of `false` for `NEW_ENV_VARS` so Co-op Cloud
+operators do not need to set `NEW_ENV_VAR=false` in their app `.env` file.
+
+```yaml
+environment:
+    - NEW_ENV_VAR=${NEW_ENV_VAR:-false}
+```
+
+Sometimes it is not appropriate to provide a default and it is important that
+other Co-op Cloud operators make an informed choice about the setting of an
+environment variable. In this case, please consider leaving a note about your
+new environment variables in the release notes (see below). The release of this
+change should then be considered a break change and require a major release
+version.
+
+`abra` will also warn Co-op Cloud operators about missing environment variables
+that are present in the recipe `.env.sample` but not in their app `.env` file.
 
 ## Creating new release notes
 
-`#TODO`
+Help other Co-op Cloud operators understand what they need to take care of when
+doing an upgrade deployment by creating some [release
+notes](/maintainers/handbook/#how-do-i-write-version-release-notes) about your
+specific changes.
+
+This is crucial when there are breaking changes, data migrations, work-arounds,
+hacks etc. present in the new changes. `abra` will show what you write when
+others run `abra app upgrade`.
+
+Good release notes are proven to save hours of pain and misery, so put
+solidarity into practice and write some excellent release notes for your fellow
+Co-op Cloud operators!