feat(upgrade): add pre-upgrade-hooks that run before an app is replaced

fix(hooks): retry a command whose replaced container is still around
fix(secrets): read a secret without asking for a terminal
2026-10-06 11:39:03 +02:00 · 2026-09-22 13:40:54 +02:00 · 2026-09-22 12:46:33 +02:00 · 2026-09-22 01:09:02 +02:00 · 2026-09-21 23:53:47 +02:00 · 2026-09-21 23:15:59 +02:00
17 changed files with 2113 additions and 531 deletions
@@ -35,7 +35,7 @@ RUN apt-get update \

 COPY --from=abra /out/abra /usr/local/bin/abra
 COPY --from=deps /opt/alakazam /opt/alakazam
-COPY alakazam.py combine.yml /opt/alakazam/
+COPY alakazam.py /opt/alakazam/
 RUN printf '#!/bin/sh\nexec /opt/alakazam/bin/python /opt/alakazam/alakazam.py "$@"\n' \
  > /usr/local/bin/alakazam \
  && chmod +x /usr/local/bin/alakazam
@@ -227,12 +227,7 @@ Configuration files support templating with Jinja2 and global variables, facilit
 1. **`alaka.yml`**/**`alaka-*.yml`**: Contains global configurations for a hierarchical layer, affecting all subordinate apps.
 2. **`example.com.yml`** (`<your-domain>.yml`): A minimalist configuration for each instance, specifying the apps to be installed, the instance domain and optionally instance specific configurations.
    - an **instance** means a collection of apps that are integrated with each other (one server / vm can contain multiple instances)
-3. **`combine.yml`**: Manages configurations required to integrate multiple apps with each other.
-    - This configuration should not be touched by the operator.
-    - At the moment it's part of the `alakazam` repository.
-    - In future it should be split into the recipes repositories and maintained by the recipe maintainer.
-        - We started to move each app entry as `alaconnect.yml` into the respective repositories.
-4. **`config-sets.yml`**: Optional file placed at the `root` path. Defines named sets of per-app configurations that can be activated or deactivated per instance or group, avoiding duplication across instance files. See [examples/config-sets.yml](./examples/config-sets.yml).
+3. **`config-sets.yml`**/**`config-sets-*.yml`**: Optional files at the `root` path. They define named sets of per-app configurations, either switched on per instance or group, or applying by themselves as an integration between apps. See [Config-Sets](#config-sets).

 ### Global Settings

@@ -259,7 +254,7 @@ uptime_kuma:

 ### App Configuration

-`alaka.yml`/`alaka-*.yml`, `example.com.yml` and `combine.yml`/`alaconnect.yml` contain a similar configuration structure.
+`alaka.yml`/`alaka-*.yml`, `example.com.yml` and the config-sets contain a similar configuration structure.
 For each app/recipe the following `<app_configurations>` can be used:

 - **`uncomment`/`comment`**: Manages the active status of lines in configuration files, such as uncommenting needed variables .
@@ -267,6 +262,13 @@ For each app/recipe the following `<app_configurations>` can be used:
    - this is useful for env variables that are used multiple times like `COMPOSE_FILE`
 - **`env`**: Sets values for environment variables.
 - **`*-hooks`**: Specifies `abra.sh` commands or local scripts to run at specific stages.
+    - **`pre-upgrade-hooks`**: commands that should be run before each upgrade, while the old
+      containers are still running
+        - they are a precondition rather than an addition, so unlike every other kind they run
+          without `--execute-hooks`; `--skip-pre-upgrade-hooks` leaves them out
+        - a failing one costs that app its upgrade and nothing else, the run then ends non-zero
+        - the command comes from the recipe version the configuration asks for, so it has to exist
+          there and has to work against the app that is still running the old one
    - **`initial-hooks`**: commands for initialisation
    - **`deploy-hooks`**: commands that should be run after each deployment
    - **`upgrade-hooks`**: commands that should be run after each upgrade
@@ -274,16 +276,47 @@ For each app/recipe the following `<app_configurations>` can be used:
 - **`dependency`**: Names the apps that have to be set up before this one. See [Dependencies](#dependencies).
 - **`secrets`**: Inserts specific values (i.E. smtp passwords) into secrets; future updates will support encrypted file usage.
 - **`secret-hooks`**: Run `abra.sh` commands locally or local scripts for secrets that need to be generated.
- **`subdomain`**: Specifies the subdomain scheme for individual recipes and apps. (not available in `combine.yml`/`alaconnect.yml`)
+- **`shared_secrets`**: Shares a secret between two apps, as `<source_app>: {<target_secret>: <source_secret>}`.
+- **`subdomain`**: Specifies the subdomain scheme for individual recipes and apps. (not available in config-sets)
    - i.e. `cloud.example.com` for nextcloud
- **`version`**: Controls the recipe version to deploy; if unspecified, the latest version is used. (not available in `combine.yml`/`alaconnect.yml`)
+- **`version`**: Controls the recipe version to deploy; if unspecified, the latest version is used. (not available in config-sets)
    - a commit can be pinned with `<release>@<commit>`, for example `12.0.2+2026.5.2@be9ebb3`
    - only the commit is deployed; the release in front of the `@` is there so that a dependency bot can follow the recipe's tags and offer patch updates

-The `combine.yml`/`alaconnect.yml` configuration additionally contains:
+### Config-Sets

- **`shared_secrets`**: Specifies secret sharing between apps.
-     - `<source_secret_name>:<target_secret_name>`
+A config-set is a named bundle of per-app configuration, defined in `config-sets.yml` or any `config-sets-*.yml` at the `root` path. The files are merged in the order `config-sets.yml`, then the `config-sets-*.yml` alphabetically, the same precedence the `alaka*.yml` of a directory have.
+
+A set applies in one of two ways.
+
+**Switched on per instance or group**, via `CONFIG-SETS` in any `alaka.yml` or `example.com.yml`:
+
+```yaml
+CONFIG-SETS:
+    bbb: true
+```
+
+This is for bundles of per-app configuration that an operator turns on deliberately, a feature for instance. See [examples/config-sets.yml](./examples/config-sets.yml).
+
+**By itself**, when it carries `apply-when` and every app named there is part of the instance:
+
+```yaml
+integrate-authentik-nextcloud:
+    apply-when: [authentik, nextcloud]
+    authentik:
+        uncomment: [compose.nextcloud.yml]
+    nextcloud:
+        uncomment: [compose.authentik.yml]
+        shared_secrets:
+            authentik:
+                nextcloud_secret: authentik_secret
+```
+
+That is what an integration between two apps is: configuration belonging to both of them that is pointless unless both are there. `apply-when` names the condition rather than deriving it from the configured apps, so an integration can configure only one side: `[matrix-synapse, traefik]` while configuring `traefik` alone. An instance can still switch such a set off with `CONFIG-SETS: {integrate-authentik-nextcloud: false}`.
+
+Merge priority: automatic sets < group < sets switched on explicitly < instance. An integration is the lowest layer of all, so a group configuration can adjust it — the precedence the `combine.yml` it replaces had. Its list entries land behind the ones an app configures for itself, which is what keeps an integration hook such as `app set_authentik` running after the `app core_install` that sets the app up in the first place.
+
+[examples/config-sets-combine.yml](./examples/config-sets-combine.yml) holds the integrations alakazam used to ship as `combine.yml`. Copy it into your `root` path to keep them.

 ### Dependencies

@@ -346,22 +379,9 @@ These configurations are designed to modularize and simplify the management of a
        <app_configurations>
    ```

-2. **`combine.yml`**
-    - This file orchestrates the integration between multiple apps within the same instance (`example.com.yml`), applying `<target_app_configurations>` to ensure seamless functionality between connected applications.
-    - The configuration for each app combination is structured as follows, again `<target_app_configurations>` is following the guidelines set in the [App Configuration](#app-configuration) section:
-
-    ```
-    <target_app_recipe>:
-        <source_app_recipe>:
-            <target_app_configurations>
-    ```
-
-    - In future each `<source_app_recipe>` entry will be placed in the `alaconnect.yml` inside the `<target_app_recipe>` folder.
-
-3. **`config-sets.yml`**
-    - Optional file at the `root` path that defines named sets of per-app configurations. Each top-level key is a config-set name; its value is a map of `<app_recipe>` to `<app_configurations>`.
-    - Activate or disable sets per instance or group via `CONFIG-SETS` in any `alaka.yml` or `example.com.yml`. Merge priority: group < config-set < instance.
-    - See [examples/config-sets.yml](./examples/config-sets.yml).
+2. **`config-sets.yml`**/**`config-sets-*.yml`**
+    - Optional files at the `root` path. Each top-level key is a config-set name; its value is a map of `<app_recipe>` to `<app_configurations>`.
+    - See [Config-Sets](#config-sets).


 ### Templating Configurations
@@ -1,302 +0,0 @@
-authentik:
-    nextcloud:
-        uncomment:
-            - compose.nextcloud.yml
-            - NEXTCLOUD_DOMAIN
-            - SECRET_NEXTCLOUD_ID_VERSION
-            - SECRET_NEXTCLOUD_SECRET_VERSION
-            - nextcloud.png
-    wordpress:
-        uncomment:
-            - compose.wordpress.yml
-            - WORDPRESS_DOMAIN
-            - WORDPRESS_GROUP
-            - SECRET_WORDPRESS_ID_VERSION
-            - SECRET_WORDPRESS_SECRET_VERSION
-            - wordpress.png
-    matrix-synapse:
-        uncomment:
-            - compose.matrix.yml
-            - ELEMENT_DOMAIN
-            - MATRIX_DOMAIN
-            - SECRET_MATRIX_ID_VERSION
-            - SECRET_MATRIX_SECRET_VERSION
-            - matrix.svg
-        secrets:
-            matrix_id: matrix
-    wekan:
-        uncomment:
-            - compose.wekan.yml
-            - WEKAN_DOMAIN
-            - SECRET_WEKAN_ID_VERSION
-            - SECRET_WEKAN_SECRET_VERSION
-            - wekan.png
-        secrets:
-            wekan_id: wekan
-    vikunja:
-        uncomment:
-            - compose.vikunja.yml
-            - VIKUNJA_DOMAIN
-            - SECRET_VIKUNJA_ID_VERSION
-            - SECRET_VIKUNJA_SECRET_VERSION
-            - vikunja.svg
-        secrets:
-            vikunja_id: vikunja
-    kimai:
-        uncomment:
-            - compose.kimai.yml
-            - KIMAI_DOMAIN
-            - SECRET_KIMAI_ID_VERSION
-            - SECRET_KIMAI_SECRET_VERSION
-            - kimai_logo.png
-            - KIMAI_GROUP
-    zammad:
-        uncomment:
-            - compose.zammad.yml
-            - ZAMMAD_DOMAIN
-            - zammad.svg
-    monitoring-ng:
-        uncomment:
-            - compose.monitoring.yml
-            - MONITORING_DOMAIN
-            - SECRET_MONITORING_ID_VERSION
-            - SECRET_MONITORING_SECRET_VERSION
-            - monitoring.svg
-    outline:
-        uncomment:
-            - compose.outline.yml
-            - OUTLINE_DOMAIN
-            - SECRET_OUTLINE_ID_VERSION
-            - SECRET_OUTLINE_SECRET_VERSION
-            - outline.png
-        secrets:
-            outline_id: outline
-    rallly:
-        uncomment:  
-            - compose.rallly.yml
-            - RALLLY_DOMAIN
-            - SECRET_RALLLY_ID_VERSION
-            - SECRET_RALLLY_SECRET_VERSION
-            - rallly.png
-        secrets:
-            rallly_id: rallly
-    hedgedoc:
-        uncomment:  
-            - compose.hedgedoc.yml
-            - HEDGEDOC_DOMAIN
-            - SECRET_HEDGEDOC_ID_VERSION
-            - SECRET_HEDGEDOC_SECRET_VERSION
-            - hedgedoc.png
-        secrets:
-            hedgedoc_id: hedgedoc
-    pretix:
-        env:
-            APPLICATIONS: 
-                Pretix:
-                    url: https://pretix.example.com/control/
-                    group:
-            EXTRA_ICONS:
-                Pretix: ~/.abra/recipes/authentik/icons/pretix.svg
-    vaultwarden:
-        env:
-            APPLICATIONS: 
-                Vaultwarden:
-                    url: https://vaultwarden.example.com/
-                    group:
-            EXTRA_ICONS:
-                Vaultwarden: ~/.abra/recipes/authentik/icons/vaultwarden.svg
-    mila:
-        uncomment:
-            - compose.mila.yml
-            - MILA_DOMAIN
-            - MILA_GROUP
-            - SECRET_MILA_ID_VERSION
-            - SECRET_MILA_SECRET_VERSION
-            - mila.svg
-        secrets:
-            mila_id: mila
-kimai:
-    authentik:
-        uncomment:
-            - SSO_ENABLED
-            - SSO_PROVIDER_URL
-            - SSO_SAML_URL
-            - SSO_LOGOUT_URL
-        secret_hooks:
-            - insert_authentik_certificate
-        dependency: [authentik]
-zammad:
-    authentik:
-        uncomment:
-            - SSO_PROVIDER_DOMAIN
-            - IDP_SSO_TARGET_URL
-            - IDP_SLO_SERVICE_URL
-        initial-hooks:
-            - local enable_authentik_sso
-        dependency: [authentik]
-nextcloud:
-    authentik:
-        uncomment:
-            - compose.authentik.yml
-            - AUTHENTIK_USER_PREFIX
-            - AUTHENTIK_DOMAIN
-            - SECRET_AUTHENTIK_SECRET_VERSION
-            - SECRET_AUTHENTIK_ID_VERSION
-        initial-hooks:
-            - app set_authentik
-        shared_secrets:
-            nextcloud_secret: authentik_secret
-            nextcloud_id: authentik_id
-    onlyoffice:
-        uncomment:
-            - compose.onlyoffice.yml
-            - ONLYOFFICE_URL
-            - SECRET_ONLYOFFICE_JWT_VERSION
-        initial-hooks:
-            - app install_onlyoffice
-    collabora:
-        uncomment:
-            - COLLABORA_URL
-        initial-hooks:
-            - app install_collabora
-onlyoffice:
-    nextcloud:
-        uncomment:
-            - compose.jwt.yml
-            - SECRET_JWT_SECRET_VERSION
-        shared_secrets:
-            onlyoffice_jwt: jwt_secret
-outline:
-    authentik:
-        env: 
-            OIDC_CLIENT_ID: outline
-            OIDC_AUTH_URI: https://authentik.example.com/application/o/authorize/
-            OIDC_TOKEN_URI: https://authentik.example.com/application/o/token/
-            OIDC_USERINFO_URI: https://authentik.example.com/application/o/userinfo/
-            OIDC_DISPLAY_NAME: "Authentik"
-        uncomment:
-            - compose.oidc.yml
-            - OIDC_ENABLED
-            - OIDC_USERNAME_CLAIM
-            - OIDC_SCOPES
-            - SECRET_OIDC_CLIENT_SECRET_VERSION
-        shared_secrets:
-            outline_secret: oidc_client_secret
-wordpress:
-    authentik:
-        uncomment:
-            - compose.authentik.yml
-            - AUTHENTIK_DOMAIN
-            - SECRET_AUTHENTIK_SECRET_VERSION
-            - SECRET_AUTHENTIK_ID_VERSION
-            - LOGIN_TYPE
-        initial-hooks:
-            - app set_authentik
-        shared_secrets:
-            wordpress_secret: authentik_secret
-            wordpress_id: authentik_id
-vikunja:
-    authentik:
-        env:
-            OAUTH_NAME: authentik
-            OAUTH_URL: https://authentik.example.com/application/o/vikunja/
-            OAUTH_LOGOUT_URL: https://authentik.example.com/application/o/vikunja/end-session/
-            # TODO: set CLIENT_ID as secret
-            OAUTH_CLIENT_ID: vikunja
-        uncomment:
-            - compose.oauth.yml
-            - OAUTH_ENABLED
-            - SECRET_OAUTH_SECRET_VERSION
-        shared_secrets:
-            #vikunja_id: oauth_id
-            vikunja_secret: oauth_secret
-matrix-synapse:
-    authentik:
-        env:
-            KEYCLOAK_ID: authentik
-            KEYCLOAK_NAME: sso
-            KEYCLOAK_URL: https://authentik.example.com/application/o/matrix/
-            # TODO: correct client domain?
-            KEYCLOAK_CLIENT_DOMAIN: https://element-web.example.com
-            KEYCLOAK_ALLOW_EXISTING_USERS: "true"
-            # TODO: set CLIENT_ID as secret
-            KEYCLOAK_CLIENT_ID: matrix
-        uncomment:
-            - compose.keycloak.yml
-            - KEYCLOAK_ENABLED
-            - KEYCLOAK_CLIENT_ID
-            - SECRET_KEYCLOAK_CLIENT_SECRET_VERSION
-        shared_secrets:
-            matrix_secret: keycloak_client_secret
-        dependency: [authentik]
-traefik:
-    matrix-synapse:
-        uncomment:
-            - compose.matrix.yml
-            - MATRIX_FEDERATION_ENABLED
-rallly:
-    authentik:
-        env:
-            OIDC_NAME: "Authentik"
-            OIDC_DISCOVERY_URL: "https://authentik.example.com/application/o/rallly/.well-known/openid-configuration"
-            OIDC_ISSUER_URL: "https://authentik.example.com/application/o/rallly/"
-            OIDC_CLIENT_ID: rallly
-        uncomment: 
-            - compose.oidc.yml
-            - SECRET_OIDC_CLIENT_SECRET_VERSION
-            - OIDC_ENABLED
-        shared_secrets:
-            rallly_secret: oidc_client_secret
-wekan:
-    authentik:
-        env: 
-            OAUTH2_ENABLED: "true"
-            OAUTH2_SERVER_URL: https://authentik.example.com
-            # TODO: set CLIENT_ID as secret
-            OAUTH2_CLIENT_ID: wekan
-        uncomment:
-            - OAUTH2_LOGIN_STYLE
-            - OAUTH2_AUTH_ENDPOINT
-            - OAUTH2_USERINFO_ENDPOINT
-            - OAUTH2_TOKEN_ENDPOINT
-            - OAUTH2_REQUEST_PERMISSIONS
-            - OAUTH2_ID_MAP
-            - OAUTH2_USERNAME_MAP
-            - OAUTH2_FULLNAME_MAP
-            - OAUTH2_EMAIL_MAP
-            - PROPAGATE_OIDC_DATA
-            - OIDC_REDIRECTION_ENABLED
-        shared_secrets:
-            wekan_secret: oauth2_secret
-hedgedoc:
-    authentik:
-        env:
-            CMD_OAUTH2_USER_PROFILE_URL: https://authentik.example.com/application/o/userinfo/
-            CMD_OAUTH2_USER_PROFILE_USERNAME_ATTR: preferred_username
-            CMD_OAUTH2_USER_PROFILE_DISPLAY_NAME_ATTR: name
-            CMD_OAUTH2_USER_PROFILE_EMAIL_ATTR: email
-            CMD_OAUTH2_TOKEN_URL: https://authentik.example.com/application/o/token/
-            CMD_OAUTH2_AUTHORIZATION_URL: https://authentik.example.com/application/o/authorize/
-            # TODO: set CLIENT_ID as secret
-            CMD_OAUTH2_CLIENT_ID: hedgedoc
-            CMD_OAUTH2_PROVIDERNAME: Authentik
-        uncomment: 
-            - compose.oauth.yml
-            - SECRET_OAUTH_KEY_VERSION
-        shared_secrets:
-            hedgedoc_secret: oauth_key
-        dependency: [authentik]
-mila:
-    authentik:
-        env:
-            OIDC_CLIENT_ID: mila
-            OIDC_BASE_URL: https://authentik.example.com/application/o/mila
-            OIDC_REDIRECT_URI: https://mila.example.com/auth/user/oidc/callback
-            OIDC_ADMIN_GROUP_NAME: mv_admin
-            OIDC_GROUPS_CLAIM: groups
-            OIDC_ONLY: "true"
-        uncomment:
-            - compose.oidc.yml
-            - SECRET_OIDC_CLIENT_SECRET_VERSION
-        shared_secrets:
-            mila_secret: oidc_client_secret
@@ -0,0 +1,395 @@
+# Integrations, converted from the combine.yml that alakazam used to ship.
+# A set applies as soon as every app of its 'apply-when' is part of an instance.
+# Copy this file into your alakazam root path, next to alaka-versions.yml.
+
+integrate-authentik-nextcloud:
+    apply-when:
+      - authentik
+      - nextcloud
+    authentik:
+        uncomment:
+          - compose.nextcloud.yml
+          - NEXTCLOUD_DOMAIN
+          - SECRET_NEXTCLOUD_ID_VERSION
+          - SECRET_NEXTCLOUD_SECRET_VERSION
+          - nextcloud.png
+    nextcloud:
+        uncomment:
+          - compose.authentik.yml
+          - AUTHENTIK_USER_PREFIX
+          - AUTHENTIK_DOMAIN
+          - SECRET_AUTHENTIK_SECRET_VERSION
+          - SECRET_AUTHENTIK_ID_VERSION
+        initial-hooks:
+          - app set_authentik
+        shared_secrets:
+            authentik:
+                nextcloud_secret: authentik_secret
+                nextcloud_id: authentik_id
+integrate-authentik-wordpress:
+    apply-when:
+      - authentik
+      - wordpress
+    authentik:
+        uncomment:
+          - compose.wordpress.yml
+          - WORDPRESS_DOMAIN
+          - WORDPRESS_GROUP
+          - SECRET_WORDPRESS_ID_VERSION
+          - SECRET_WORDPRESS_SECRET_VERSION
+          - wordpress.png
+    wordpress:
+        uncomment:
+          - compose.authentik.yml
+          - AUTHENTIK_DOMAIN
+          - SECRET_AUTHENTIK_SECRET_VERSION
+          - SECRET_AUTHENTIK_ID_VERSION
+          - LOGIN_TYPE
+        initial-hooks:
+          - app set_authentik
+        shared_secrets:
+            authentik:
+                wordpress_secret: authentik_secret
+                wordpress_id: authentik_id
+integrate-authentik-matrix-synapse:
+    apply-when:
+      - authentik
+      - matrix-synapse
+    authentik:
+        uncomment:
+          - compose.matrix.yml
+          - ELEMENT_DOMAIN
+          - MATRIX_DOMAIN
+          - SECRET_MATRIX_ID_VERSION
+          - SECRET_MATRIX_SECRET_VERSION
+          - matrix.svg
+        secrets:
+            matrix_id: matrix
+    matrix-synapse:
+        env:
+            KEYCLOAK_ID: authentik
+            KEYCLOAK_NAME: sso
+            KEYCLOAK_URL: https://authentik.example.com/application/o/matrix/
+            # TODO: correct client domain?
+            KEYCLOAK_CLIENT_DOMAIN: https://element-web.example.com
+            KEYCLOAK_ALLOW_EXISTING_USERS: "true"
+            # TODO: set CLIENT_ID as secret
+            KEYCLOAK_CLIENT_ID: matrix
+        uncomment:
+          - compose.keycloak.yml
+          - KEYCLOAK_ENABLED
+          - KEYCLOAK_CLIENT_ID
+          - SECRET_KEYCLOAK_CLIENT_SECRET_VERSION
+        shared_secrets:
+            authentik:
+                matrix_secret: keycloak_client_secret
+        dependency: [authentik]
+integrate-authentik-wekan:
+    apply-when:
+      - authentik
+      - wekan
+    authentik:
+        uncomment:
+          - compose.wekan.yml
+          - WEKAN_DOMAIN
+          - SECRET_WEKAN_ID_VERSION
+          - SECRET_WEKAN_SECRET_VERSION
+          - wekan.png
+        secrets:
+            wekan_id: wekan
+    wekan:
+        env:
+            OAUTH2_ENABLED: "true"
+            OAUTH2_SERVER_URL: https://authentik.example.com
+            # TODO: set CLIENT_ID as secret
+            OAUTH2_CLIENT_ID: wekan
+        uncomment:
+          - OAUTH2_LOGIN_STYLE
+          - OAUTH2_AUTH_ENDPOINT
+          - OAUTH2_USERINFO_ENDPOINT
+          - OAUTH2_TOKEN_ENDPOINT
+          - OAUTH2_REQUEST_PERMISSIONS
+          - OAUTH2_ID_MAP
+          - OAUTH2_USERNAME_MAP
+          - OAUTH2_FULLNAME_MAP
+          - OAUTH2_EMAIL_MAP
+          - PROPAGATE_OIDC_DATA
+          - OIDC_REDIRECTION_ENABLED
+        shared_secrets:
+            authentik:
+                wekan_secret: oauth2_secret
+integrate-authentik-vikunja:
+    apply-when:
+      - authentik
+      - vikunja
+    authentik:
+        uncomment:
+          - compose.vikunja.yml
+          - VIKUNJA_DOMAIN
+          - SECRET_VIKUNJA_ID_VERSION
+          - SECRET_VIKUNJA_SECRET_VERSION
+          - vikunja.svg
+        secrets:
+            vikunja_id: vikunja
+    vikunja:
+        env:
+            OAUTH_NAME: authentik
+            OAUTH_URL: https://authentik.example.com/application/o/vikunja/
+            OAUTH_LOGOUT_URL: https://authentik.example.com/application/o/vikunja/end-session/
+            # TODO: set CLIENT_ID as secret
+            OAUTH_CLIENT_ID: vikunja
+        uncomment:
+          - compose.oauth.yml
+          - OAUTH_ENABLED
+          - SECRET_OAUTH_SECRET_VERSION
+        shared_secrets:
+            #vikunja_id: oauth_id
+            authentik:
+                vikunja_secret: oauth_secret
+integrate-authentik-kimai:
+    apply-when:
+      - authentik
+      - kimai
+    authentik:
+        uncomment:
+          - compose.kimai.yml
+          - KIMAI_DOMAIN
+          - SECRET_KIMAI_ID_VERSION
+          - SECRET_KIMAI_SECRET_VERSION
+          - kimai_logo.png
+          - KIMAI_GROUP
+    kimai:
+        uncomment:
+          - SSO_ENABLED
+          - SSO_PROVIDER_URL
+          - SSO_SAML_URL
+          - SSO_LOGOUT_URL
+          - SSO_ADMIN_GROUP_NAME
+        secret_hooks:
+          - insert_authentik_certificate
+        dependency: [authentik]
+integrate-authentik-zammad:
+    apply-when:
+      - authentik
+      - zammad
+    authentik:
+        uncomment:
+          - compose.zammad.yml
+          - ZAMMAD_DOMAIN
+          - zammad.svg
+    zammad:
+        uncomment:
+          - SSO_PROVIDER_DOMAIN
+          - IDP_SSO_TARGET_URL
+          - IDP_SLO_SERVICE_URL
+        initial-hooks:
+          - local enable_authentik_sso
+        dependency: [authentik]
+integrate-authentik-monitoring-ng:
+    apply-when:
+      - authentik
+      - monitoring-ng
+    authentik:
+        uncomment:
+          - compose.monitoring.yml
+          - MONITORING_DOMAIN
+          - SECRET_MONITORING_ID_VERSION
+          - SECRET_MONITORING_SECRET_VERSION
+          - monitoring.svg
+        secrets:
+            monitoring_id: monitoring
+    monitoring-ng:
+        env:
+            GF_SERVER_ROOT_URL: https://monitoring-ng.example.com
+            OIDC_CLIENT_ID: monitoring
+            OIDC_AUTH_URL: https://authentik.example.com/application/o/authorize/
+            OIDC_API_URL: https://authentik.example.com/application/o/userinfo/
+            OIDC_TOKEN_URL: https://authentik.example.com/application/o/token/
+        uncomment:
+          - compose.prometheus.yml
+          - PROMETHEUS_RETENTION_TIME
+          - compose.loki.yml
+          - LOKI_RETENTION_PERIOD
+          - LOKI_STORAGE_FILESYSTEM
+          - compose.grafana.yml
+          - OIDC_ENABLED
+          - SECRET_GRAFANA_ADMIN_PASSWORD_VERSION
+          - SECRET_GRAFANA_OIDC_CLIENT_SECRET_VERSION
+          - SECRET_GRAFANA_SMTP_PASSWORD_VERSION
+        shared_secrets:
+            authentik:
+                monitoring_secret: grafana_oidc_client_secret
+        dependency: [authentik]
+integrate-authentik-outline:
+    apply-when:
+      - authentik
+      - outline
+    authentik:
+        uncomment:
+          - compose.outline.yml
+          - OUTLINE_DOMAIN
+          - SECRET_OUTLINE_ID_VERSION
+          - SECRET_OUTLINE_SECRET_VERSION
+          - outline.png
+        secrets:
+            outline_id: outline
+    outline:
+        env:
+            OIDC_CLIENT_ID: outline
+            OIDC_AUTH_URI: https://authentik.example.com/application/o/authorize/
+            OIDC_TOKEN_URI: https://authentik.example.com/application/o/token/
+            OIDC_USERINFO_URI: https://authentik.example.com/application/o/userinfo/
+            OIDC_DISPLAY_NAME: "Authentik"
+        uncomment:
+          - compose.oidc.yml
+          - OIDC_ENABLED
+          - OIDC_USERNAME_CLAIM
+          - OIDC_SCOPES
+          - SECRET_OIDC_CLIENT_SECRET_VERSION
+        shared_secrets:
+            authentik:
+                outline_secret: oidc_client_secret
+integrate-authentik-rallly:
+    apply-when:
+      - authentik
+      - rallly
+    authentik:
+        uncomment:
+          - compose.rallly.yml
+          - RALLLY_DOMAIN
+          - SECRET_RALLLY_ID_VERSION
+          - SECRET_RALLLY_SECRET_VERSION
+          - rallly.png
+        secrets:
+            rallly_id: rallly
+    rallly:
+        env:
+            OIDC_NAME: "Authentik"
+            OIDC_DISCOVERY_URL: "https://authentik.example.com/application/o/rallly/.well-known/openid-configuration"
+            OIDC_ISSUER_URL: "https://authentik.example.com/application/o/rallly/"
+            OIDC_CLIENT_ID: rallly
+        uncomment:
+          - compose.oidc.yml
+          - SECRET_OIDC_CLIENT_SECRET_VERSION
+          - OIDC_ENABLED
+        shared_secrets:
+            authentik:
+                rallly_secret: oidc_client_secret
+integrate-authentik-hedgedoc:
+    apply-when:
+      - authentik
+      - hedgedoc
+    authentik:
+        uncomment:
+          - compose.hedgedoc.yml
+          - HEDGEDOC_DOMAIN
+          - SECRET_HEDGEDOC_ID_VERSION
+          - SECRET_HEDGEDOC_SECRET_VERSION
+          - hedgedoc.png
+        secrets:
+            hedgedoc_id: hedgedoc
+    hedgedoc:
+        env:
+            CMD_OAUTH2_USER_PROFILE_URL: https://authentik.example.com/application/o/userinfo/
+            CMD_OAUTH2_USER_PROFILE_USERNAME_ATTR: preferred_username
+            CMD_OAUTH2_USER_PROFILE_DISPLAY_NAME_ATTR: name
+            CMD_OAUTH2_USER_PROFILE_EMAIL_ATTR: email
+            CMD_OAUTH2_TOKEN_URL: https://authentik.example.com/application/o/token/
+            CMD_OAUTH2_AUTHORIZATION_URL: https://authentik.example.com/application/o/authorize/
+            # TODO: set CLIENT_ID as secret
+            CMD_OAUTH2_CLIENT_ID: hedgedoc
+            CMD_OAUTH2_PROVIDERNAME: Authentik
+        uncomment:
+          - compose.oauth.yml
+          - SECRET_OAUTH_KEY_VERSION
+        shared_secrets:
+            authentik:
+                hedgedoc_secret: oauth_key
+        dependency: [authentik]
+integrate-authentik-pretix:
+    apply-when:
+      - authentik
+      - pretix
+    authentik:
+        env:
+            APPLICATIONS:
+                Pretix:
+                    url: https://pretix.example.com/control/
+                    group:
+            EXTRA_ICONS:
+                Pretix: ~/.abra/recipes/authentik/icons/pretix.svg
+integrate-authentik-vaultwarden:
+    apply-when:
+      - authentik
+      - vaultwarden
+    authentik:
+        env:
+            APPLICATIONS:
+                Vaultwarden:
+                    url: https://vaultwarden.example.com/
+                    group:
+            EXTRA_ICONS:
+                Vaultwarden: ~/.abra/recipes/authentik/icons/vaultwarden.svg
+integrate-authentik-mila:
+    apply-when:
+      - authentik
+      - mila
+    authentik:
+        uncomment:
+          - compose.mila.yml
+          - MILA_DOMAIN
+          - MILA_GROUP
+          - SECRET_MILA_ID_VERSION
+          - SECRET_MILA_SECRET_VERSION
+          - mila.svg
+        secrets:
+            mila_id: mila
+    mila:
+        env:
+            OIDC_CLIENT_ID: mila
+            OIDC_BASE_URL: https://authentik.example.com/application/o/mila
+            OIDC_REDIRECT_URI: https://mila.example.com/auth/user/oidc/callback
+            OIDC_ADMIN_GROUP_NAME: mv_admin
+            OIDC_GROUPS_CLAIM: groups
+            OIDC_ONLY: "true"
+        uncomment:
+          - compose.oidc.yml
+          - SECRET_OIDC_CLIENT_SECRET_VERSION
+        shared_secrets:
+            authentik:
+                mila_secret: oidc_client_secret
+integrate-nextcloud-onlyoffice:
+    apply-when:
+      - nextcloud
+      - onlyoffice
+    nextcloud:
+        uncomment:
+          - compose.onlyoffice.yml
+          - ONLYOFFICE_URL
+          - SECRET_ONLYOFFICE_JWT_VERSION
+        initial-hooks:
+          - app install_onlyoffice
+    onlyoffice:
+        uncomment:
+          - compose.jwt.yml
+          - SECRET_JWT_SECRET_VERSION
+        shared_secrets:
+            nextcloud:
+                onlyoffice_jwt: jwt_secret
+integrate-collabora-nextcloud:
+    apply-when:
+      - collabora
+      - nextcloud
+    nextcloud:
+        uncomment:
+          - COLLABORA_URL
+        initial-hooks:
+          - app install_collabora
+integrate-matrix-synapse-traefik:
+    apply-when:
+      - matrix-synapse
+      - traefik
+    traefik:
+        uncomment:
+          - compose.matrix.yml
+          - MATRIX_FEDERATION_ENABLED
@@ -7,4 +7,4 @@ python-dotenv==1.0.0
 icecream==2.1.3
 packaging==24.0 
 GitPython==3.1.43
-uptime_kuma_api==1.2.1
+uptime-kuma-api2==2.7.0
@@ -0,0 +1,191 @@
+"""Tests for config-sets, in particular the ones that apply by themselves."""
+
+import logging
+import os
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+from alakazam import get_config_set_app_configs, merge_instance_configs, read_config_sets
+
+INTEGRATION = {
+    "integrate-authentik-nextcloud": {
+        "apply-when": ["authentik", "nextcloud"],
+        "authentik": {"uncomment": ["compose.nextcloud.yml"]},
+        "nextcloud": {"uncomment": ["compose.authentik.yml"],
+                      "shared_secrets": {"authentik": {"nextcloud_secret": "authentik_secret"}}},
+    }
+}
+
+
+def apply(config_sets, instance_apps, active=None):
+    """The effective configuration, with the automatic sets below the explicit ones."""
+    automatic, explicit = get_config_set_app_configs(active or {}, config_sets, instance_apps)
+    return alakazam.merge_dict(automatic, explicit)
+
+
+class TestApplyWhen:
+    def test_it_applies_when_every_app_is_there(self):
+        result = apply(INTEGRATION, ["authentik", "nextcloud", "traefik"])
+        assert result["authentik"]["uncomment"] == ["compose.nextcloud.yml"]
+        assert result["nextcloud"]["shared_secrets"] == {"authentik": {"nextcloud_secret": "authentik_secret"}}
+
+    def test_it_stays_out_when_one_app_is_missing(self):
+        assert apply(INTEGRATION, ["authentik", "traefik"]) == {}
+
+    def test_apply_when_is_not_passed_on_as_an_app(self):
+        """It is a condition, not configuration, and would end up in an .env otherwise."""
+        assert "apply-when" not in apply(INTEGRATION, ["authentik", "nextcloud"])
+
+    def test_an_instance_can_switch_it_off(self):
+        assert apply(INTEGRATION, ["authentik", "nextcloud"],
+                     active={"integrate-authentik-nextcloud": False}) == {}
+
+    def test_a_one_sided_integration_configures_only_one_app(self):
+        """4 of the 17 pairs in the shipped combine.yml were one-sided."""
+        sets = {"integrate-matrix-synapse-traefik": {
+            "apply-when": ["matrix-synapse", "traefik"],
+            "traefik": {"uncomment": ["compose.matrix.yml"]}}}
+        assert list(apply(sets, ["matrix-synapse", "traefik"])) == ["traefik"]
+        assert apply(sets, ["traefik"]) == {}
+
+
+class TestExplicitSets:
+    SET = {"bbb": {"authentik": {"env": {"A": "b"}}}}
+
+    def test_an_enabled_set_applies(self):
+        assert apply(self.SET, ["authentik"], active={"bbb": True})["authentik"]["env"] == {"A": "b"}
+
+    def test_a_set_that_is_not_enabled_does_not_apply(self):
+        assert apply(self.SET, ["authentik"]) == {}
+
+    def test_an_unknown_name_is_reported(self, caplog):
+        with caplog.at_level(logging.WARNING):
+            apply(self.SET, ["authentik"], active={"typo": True})
+        assert "'typo' is enabled but not defined" in caplog.text
+
+    def test_an_explicit_set_refines_an_automatic_one(self):
+        """Automatic integrations are the base, an explicit set may still override them."""
+        sets = dict(INTEGRATION)
+        sets["override"] = {"authentik": {"env": {"SOURCE": "explicit"}}}
+        result = apply(sets, ["authentik", "nextcloud"], active={"override": True})
+        assert result["authentik"]["env"] == {"SOURCE": "explicit"}
+        assert result["authentik"]["uncomment"] == ["compose.nextcloud.yml"]
+
+
+class TestReadConfigSets:
+    def write(self, root, name, text):
+        (root / name).write_text(text)
+
+    def test_the_base_file_is_read(self, tmp_path):
+        self.write(tmp_path, "config-sets.yml", "bbb:\n    authentik:\n        env:\n            A: b\n")
+        assert read_config_sets(tmp_path)["bbb"]["authentik"]["env"] == {"A": "b"}
+
+    def test_further_files_are_merged(self, tmp_path):
+        self.write(tmp_path, "config-sets.yml", "bbb:\n    authentik: {}\n")
+        self.write(tmp_path, "config-sets-authentik.yml", "integrate:\n    apply-when: [a, b]\n")
+        assert sorted(read_config_sets(tmp_path)) == ["bbb", "integrate"]
+
+    def test_the_base_file_is_merged_first(self, tmp_path):
+        self.write(tmp_path, "config-sets.yml", "bbb:\n    authentik:\n        env:\n            A: base\n")
+        self.write(tmp_path, "config-sets-later.yml", "bbb:\n    authentik:\n        env:\n            A: later\n")
+        assert read_config_sets(tmp_path)["bbb"]["authentik"]["env"]["A"] == "later"
+
+    def test_a_root_without_config_sets(self, tmp_path):
+        assert read_config_sets(tmp_path) == {}
+
+    def test_a_key_defined_twice_is_reported(self, tmp_path, caplog):
+        self.write(tmp_path, "config-sets.yml", "bbb:\n    authentik:\n        env:\n            A: base\n")
+        self.write(tmp_path, "config-sets-later.yml", "bbb:\n    authentik:\n        env:\n            A: later\n")
+        with caplog.at_level(logging.WARNING):
+            read_config_sets(tmp_path)
+        assert "bbb.authentik.env.A" in caplog.text
+
+
+class TestIntegrationReachesTheMergedConfig:
+    """The instance config is what every command reads, the integration has to arrive there."""
+
+    def merge(self, apps):
+        return merge_instance_configs({}, "example.com", {app: None for app in apps}, INTEGRATION)
+
+    def test_both_apps_get_their_side(self):
+        merged = self.merge(["authentik", "nextcloud"])
+        assert merged["authentik"]["uncomment"] == ["compose.nextcloud.yml"]
+        assert merged["nextcloud"]["uncomment"] == ["compose.authentik.yml"]
+
+    def test_a_lone_app_gets_nothing(self):
+        assert "uncomment" not in self.merge(["authentik"])["authentik"]
+
+
+class TestUnknownAppKeys:
+    """A misspelt app key drops its configuration, which is only visible if it is reported."""
+
+    def test_a_typo_in_an_automatic_set_is_reported(self, caplog):
+        sets = {"integrate": {"apply-when": ["authentik", "nextcloud"],
+                              "nextcoud": {"env": {"A": "b"}}}}
+        with caplog.at_level(logging.WARNING):
+            merge_instance_configs({}, "example.com", {"authentik": None, "nextcloud": None}, sets)
+        assert "'nextcoud' is not listed" in caplog.text
+
+    def test_a_typo_in_an_explicit_set_is_reported(self, caplog):
+        sets = {"bbb": {"nextcoud": {"env": {"A": "b"}}}}
+        with caplog.at_level(logging.WARNING):
+            merge_instance_configs({}, "example.com",
+                                   {"nextcloud": None, "CONFIG-SETS": {"bbb": True}}, sets)
+        assert "'nextcoud' is not listed" in caplog.text
+
+    def test_a_correct_app_key_is_not_reported(self, caplog):
+        sets = {"integrate": {"apply-when": ["authentik", "nextcloud"],
+                              "nextcloud": {"env": {"A": "b"}}}}
+        with caplog.at_level(logging.WARNING):
+            merge_instance_configs({}, "example.com", {"authentik": None, "nextcloud": None}, sets)
+        assert caplog.text == ""
+
+
+class TestPrecedence:
+    """An integration is the lowest layer, as combine.yml was before it."""
+
+    def test_the_group_configuration_beats_an_integration(self):
+        group = {"nextcloud": {"env": {"SOURCE": "group"}}}
+        sets = {"integrate": {"apply-when": ["authentik", "nextcloud"],
+                              "nextcloud": {"env": {"SOURCE": "integration"}}}}
+        merged = merge_instance_configs(group, "example.com", {"authentik": None, "nextcloud": None}, sets)
+        assert merged["nextcloud"]["env"]["SOURCE"] == "group"
+
+    def test_the_instance_beats_everything(self):
+        group = {"nextcloud": {"env": {"SOURCE": "group"}}}
+        sets = {"integrate": {"apply-when": ["authentik", "nextcloud"],
+                              "nextcloud": {"env": {"SOURCE": "integration"}}}}
+        merged = merge_instance_configs(
+            group, "example.com", {"authentik": None, "nextcloud": {"env": {"SOURCE": "instance"}}}, sets)
+        assert merged["nextcloud"]["env"]["SOURCE"] == "instance"
+
+    def test_an_integration_still_adds_what_nobody_else_sets(self):
+        group = {"nextcloud": {"env": {"OTHER": "group"}}}
+        sets = {"integrate": {"apply-when": ["authentik", "nextcloud"],
+                              "nextcloud": {"env": {"SOURCE": "integration"}}}}
+        merged = merge_instance_configs(group, "example.com", {"authentik": None, "nextcloud": None}, sets)
+        assert merged["nextcloud"]["env"] == {"SOURCE": "integration", "OTHER": "group"}
+
+
+class TestHookOrder:
+    """An integration adds to an app that already configures itself, and must come after it."""
+
+    SETS = {"integrate-authentik-wordpress": {
+        "apply-when": ["authentik", "wordpress"],
+        "wordpress": {"initial-hooks": ["app set_authentik"]}}}
+
+    def test_the_integration_hook_runs_last(self):
+        """set_authentik configures SSO in a WordPress that core_install has to create first."""
+        group = {"wordpress": {"initial-hooks": ["app core_install", "app enable_auto_updates"]}}
+        merged = merge_instance_configs(group, "example.com", {"authentik": None, "wordpress": None}, self.SETS)
+        assert merged["wordpress"]["initial-hooks"] == [
+            "app core_install", "app enable_auto_updates", "app set_authentik"]
+
+    def test_an_instance_hook_also_comes_first(self):
+        instance = {"authentik": None, "wordpress": {"initial-hooks": ["app from_instance"]}}
+        merged = merge_instance_configs({}, "example.com", instance, self.SETS)
+        assert merged["wordpress"]["initial-hooks"] == ["app from_instance", "app set_authentik"]
@@ -0,0 +1,132 @@
+"""Tests for what a deploy run does: deploy everything, wait for all of it, then hook."""
+
+import os
+import sys
+
+import pytest
+from click.testing import CliRunner
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+from alakazam import deploy_apps
+
+CONFIG = {"a.org": {
+    "nextcloud": {"app_domain": "cloud.a.org", "server": "a.org", "version": "2.0.0"},
+    "wekan": {"app_domain": "boards.a.org", "server": "a.org", "version": "2.0.0"},
+}}
+
+
+@pytest.fixture
+def steps(monkeypatch):
+    """Records what a deploy run does, in order, against an abra that answers every call."""
+    steps = []
+    monkeypatch.setattr(alakazam, "INSTANCE_CONFIGS", CONFIG)
+    monkeypatch.setattr(alakazam, "sleep", lambda seconds: steps.append(f"sleep:{seconds}"))
+    monkeypatch.setattr(alakazam, "execute_cmds",
+                        lambda config, **kw: steps.append(f"hook:{config['app_domain']}"))
+    monkeypatch.setattr(alakazam, "get_apps", lambda recipes, **kw: {
+        "a.org": [[app, config["app_domain"]] for app, config in CONFIG["a.org"].items()
+                  if not recipes or app in recipes]})
+    monkeypatch.setattr(alakazam, "get_apps_by_deployment", lambda recipes, deployed=True: {
+        "a.org": [[app, config["app_domain"]] + (["2.0.0"] if deployed else [])
+                  for app, config in CONFIG["a.org"].items() if not recipes or app in recipes]})
+
+    def abra(*args, **kwargs):
+        if args[:2] == ("app", "ps"):
+            steps.append(f"ps:{args[2]}")
+            return {"app": {"service": "app", "status": "healthy", "version": "2.0.0",
+                            "chaos": "false", "state": "running", "image": "img"}}
+        steps.append(" ".join(a for a in args if a))
+        return ""
+
+    monkeypatch.setattr(alakazam, "abra", abra)
+    return steps
+
+
+def selected(steps, prefix):
+    return [step for step in steps if step.startswith(prefix)]
+
+
+def run(args):
+    result = CliRunner().invoke(alakazam.deploy, ["-n"] + args, standalone_mode=False)
+    assert result.exit_code == 0, result.exception
+    return result
+
+
+class TestDeployHooks:
+    def test_the_hooks_do_not_switch_on_converge_checks(self, steps):
+        """Waiting for each app in turn is what the separate flag is for."""
+        run(["-e"])
+        assert all("--no-converge-checks" in s for s in selected(steps, "app deploy"))
+
+    def test_converge_checks_come_from_their_own_flag_only(self, steps):
+        run(["-c"])
+        assert not any("--no-converge-checks" in s for s in selected(steps, "app deploy"))
+
+    def test_every_app_is_deployed_before_the_first_hook(self, steps):
+        run(["-e"])
+        deploys = selected(steps, "app deploy")
+        assert max(steps.index(s) for s in deploys) < steps.index("hook:cloud.a.org")
+
+    def test_the_hooks_run_after_the_wait(self, steps):
+        """A hook would otherwise reach a container the deployment has not brought up yet."""
+        run(["-e"])
+        assert steps.index("ps:cloud.a.org") < steps.index("hook:cloud.a.org")
+        assert steps.index("ps:boards.a.org") < steps.index("hook:boards.a.org")
+
+    def test_the_health_is_only_read_after_the_services_settled(self, steps):
+        """A replaced service still reports its old container, which has been healthy all along."""
+        run(["-e"])
+        assert steps.index(f"sleep:{alakazam.PS_WAIT_SETTLE}") < steps.index("ps:cloud.a.org")
+        assert max(steps.index(s) for s in selected(steps, "app deploy")) \
+            < steps.index(f"sleep:{alakazam.PS_WAIT_SETTLE}")
+
+    def test_without_the_flag_nothing_is_hooked(self, steps):
+        run([])
+        assert selected(steps, "hook:") == []
+
+    def test_the_wait_happens_without_the_flag_too(self, steps):
+        """It is what tells a deploy run whether the apps came up at all."""
+        run([])
+        assert selected(steps, "ps:") == ["ps:cloud.a.org", "ps:boards.a.org"]
+
+    def test_a_filtered_run_waits_for_its_own_apps_only(self, steps):
+        run(["-e", "-r", "wekan"])
+        assert selected(steps, "ps:") == ["ps:boards.a.org"]
+        assert selected(steps, "hook:") == ["hook:boards.a.org"]
+
+
+class TestDeployAppsWithoutWaiting:
+    """setup arranges its own waiting, per dependency level and once for the whole environment."""
+
+    def test_the_wait_can_be_opted_out_of(self, steps):
+        deploy_apps({"a.org": [["nextcloud", "cloud.a.org"]]}, wait=False)
+        assert selected(steps, "ps:") == []
+        assert selected(steps, "sleep:") == []
+
+    def test_the_hooks_still_run_without_a_wait(self, steps):
+        deploy_apps({"a.org": [["nextcloud", "cloud.a.org"]]}, execute_hooks=True, wait=False)
+        assert selected(steps, "hook:") == ["hook:cloud.a.org"]
+
+    def test_nothing_to_deploy_skips_the_wait(self, steps):
+        """Waiting on a run that did not happen would report on unrelated apps."""
+        deploy_apps({})
+        assert steps == []
+
+
+class TestSettleOption:
+    def test_the_delay_can_be_set(self, steps):
+        run(["--settle", "5"])
+        assert "sleep:5" in steps
+        assert f"sleep:{alakazam.PS_WAIT_SETTLE}" not in steps
+
+    def test_zero_skips_the_delay_without_skipping_the_wait(self, steps):
+        """A converge-checked run has waited already, the health still has to be confirmed."""
+        run(["--settle", "0"])
+        assert selected(steps, "sleep:") == []
+        assert selected(steps, "ps:") == ["ps:cloud.a.org", "ps:boards.a.org"]
+
+    def test_the_default_is_used_when_it_is_not_given(self, steps):
+        run([])
+        assert f"sleep:{alakazam.PS_WAIT_SETTLE}" in steps
@@ -0,0 +1,144 @@
+"""Tests for how update_configs() applies a config's env/comment/uncomment keys to a .env file."""
+
+import logging
+import os
+import sys
+
+from dotenv import dotenv_values
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from alakazam import format_env_value, set_extending_key, update_configs
+
+
+class TestUnknownConfigKeys:
+    def test_an_unknown_key_never_reaches_the_env_file(self, tmp_path):
+        """A promotion mechanism sets 'hold: <reason>' on a recipe, which must pass through untouched."""
+        env = tmp_path / "app.env"
+        env.write_text("#SMTP_HOST=mail.example.com\n")
+        update_configs(env, {"hold": "waiting for the upstream fix", "env": {"SMTP_HOST": "mail.a.org"}})
+        content = env.read_text()
+        assert "SMTP_HOST=mail.a.org" in content
+        assert "hold" not in content
+        assert "waiting for the upstream fix" not in content
+
+    def test_an_unknown_key_alone_changes_nothing(self, tmp_path):
+        env = tmp_path / "app.env"
+        env.write_text("#SMTP_HOST=mail.example.com\n")
+        update_configs(env, {"hold": "waiting for the upstream fix"})
+        assert env.read_text() == "#SMTP_HOST=mail.example.com\n"
+
+
+class TestExtendingEnvKeys:
+    """
+    Some env keys are repeated on multiple lines, relying on bash accumulation
+    ("COMPOSE_FILE=\"$COMPOSE_FILE:compose.mariadb.yml\"") to build up a value.
+    A config that sets such a key should extend that accumulation, not replace
+    or uncomment every line sharing the key.
+    """
+
+    ACCUMULATING_ENV = (
+        'COMPOSE_FILE="compose.yml"\n'
+        'COMPOSE_FILE="$COMPOSE_FILE:compose.mariadb.yml"\n'
+        '# COMPOSE_FILE="$COMPOSE_FILE:compose.onlyoffice.yml"\n'
+    )
+
+    def test_a_key_sharing_a_prefix_is_not_matched_by_substring(self, tmp_path):
+        """COMPOSE must not be treated as matching lines whose key is COMPOSE_FILE."""
+        env = tmp_path / "app.env"
+        env.write_text(self.ACCUMULATING_ENV)
+        update_configs(env, {"env": {"COMPOSE": "compose.custom.yml"}})  # not self-referencing
+        lines = env.read_text().splitlines()
+        assert lines[:-1] == self.ACCUMULATING_ENV.splitlines()  # every original line untouched
+        assert lines[-1] == 'COMPOSE=compose.custom.yml'  # set as its own, unrelated key
+
+    def test_a_self_referencing_value_is_appended_without_touching_other_lines(self, tmp_path):
+        env = tmp_path / "app.env"
+        env.write_text(self.ACCUMULATING_ENV)
+        update_configs(env, {"env": {"COMPOSE_FILE": "$COMPOSE_FILE:../../customoverride.yml"}})
+        lines = env.read_text().splitlines()
+        assert lines[:-1] == self.ACCUMULATING_ENV.splitlines()  # every original line untouched
+        assert lines[-1] == 'COMPOSE_FILE="$COMPOSE_FILE:../../customoverride.yml"'  # appended
+
+    def test_setting_the_same_self_referencing_value_twice_is_idempotent(self, tmp_path):
+        env = tmp_path / "app.env"
+        env.write_text(self.ACCUMULATING_ENV)
+        config = {"env": {"COMPOSE_FILE": "$COMPOSE_FILE:../../customoverride.yml"}}
+        update_configs(env, config)
+        update_configs(env, config)
+        lines = env.read_text().splitlines()
+        assert lines.count('COMPOSE_FILE="$COMPOSE_FILE:../../customoverride.yml"') == 1
+
+    def test_a_self_referencing_value_matching_an_existing_commented_line_uncomments_it_in_place(self, tmp_path):
+        env = tmp_path / "app.env"
+        env.write_text(self.ACCUMULATING_ENV)
+        update_configs(env, {"env": {"COMPOSE_FILE": "$COMPOSE_FILE:compose.onlyoffice.yml"}})
+        lines = env.read_text().splitlines()
+        assert lines[:-1] == self.ACCUMULATING_ENV.splitlines()[:-1]  # untouched, incl. the already-active line
+        assert lines[-1] == 'COMPOSE_FILE="$COMPOSE_FILE:compose.onlyoffice.yml"'  # uncommented in place
+        assert len(lines) == len(self.ACCUMULATING_ENV.splitlines())  # no new line appended
+
+
+class TestQuotedEnvValues:
+    """
+    A value keeps the quotes of its YAML scalar, and jinja substitutes globals that carry quotes of
+    their own. Quotes of the same style then end the line early where it is read back, so which
+    style a configuration happens to use must not change the value that arrives in the .env file.
+    """
+
+    def test_both_quoting_styles_produce_the_same_line(self, tmp_path):
+        env = tmp_path / "app.env"
+        env.write_text("")
+        update_configs(env, {"env": {
+            "DOUBLE": '"["turn:a.org:443?transport=tcp"]"',
+            "SINGLE": '\'["turn:a.org:443?transport=tcp"]\'',
+        }})
+        assert env.read_text().splitlines() == [
+            'DOUBLE=\'["turn:a.org:443?transport=tcp"]\'',
+            'SINGLE=\'["turn:a.org:443?transport=tcp"]\'',
+        ]
+
+    def test_an_inner_quote_does_not_end_the_line(self, tmp_path):
+        """Otherwise the parser stops after "[" and reads the rest of the line as further keys."""
+        env = tmp_path / "app.env"
+        env.write_text("")
+        update_configs(env, {"env": {"TURN_URIS": '"["turn:a.org:443"]"'}})
+        assert dotenv_values(env) == {"TURN_URIS": '["turn:a.org:443"]'}
+
+    def test_the_other_style_is_preferred_over_escaping(self):
+        """It needs no escape at all, so it survives a reader that resolves none."""
+        assert format_env_value('"["a"]"') == '\'["a"]\''
+        assert format_env_value("'it's here'") == '"it\'s here"'
+
+    def test_a_value_that_expands_keeps_its_style_and_escapes(self):
+        """Single quotes would stop abra from expanding the "$", which changes what the app gets."""
+        assert format_env_value('"a "b" $VAR"') == '"a \\"b\\" $VAR"'
+
+    def test_an_already_escaped_quote_is_not_escaped_again(self):
+        assert format_env_value(r'"a \" b"') == r'"a \" b"'
+
+    def test_a_value_holding_both_styles_is_reported(self, caplog):
+        """Neither style can hold it, single quotes have no escape in abra's parser."""
+        with caplog.at_level(logging.WARNING):
+            format_env_value('\'it\'s "quoted"\'')
+        assert "single quote" in caplog.text
+
+    def test_a_value_without_quotes_is_written_unchanged(self):
+        assert format_env_value("plain value") == "plain value"
+
+    def test_a_non_string_passes_through(self):
+        assert format_env_value(8080) == 8080
+
+    def test_a_quoted_extending_value_is_not_wrapped_twice(self, tmp_path):
+        """Two layers of quotes read back as an empty string, which drops every compose file."""
+        env = tmp_path / "app.env"
+        env.write_text("")
+        set_extending_key(env, "COMPOSE_FILE", '"$COMPOSE_FILE:compose.mariadb.yml"')
+        assert dotenv_values(env)["COMPOSE_FILE"] == "$COMPOSE_FILE:compose.mariadb.yml"
+
+    def test_an_unquoted_extending_value_is_still_quoted_for_expansion(self, tmp_path):
+        """The quotes have to be double ones, abra expands the "$" only inside those."""
+        env = tmp_path / "app.env"
+        env.write_text("")
+        set_extending_key(env, "COMPOSE_FILE", "$COMPOSE_FILE:compose.mariadb.yml")
+        assert env.read_text() == 'COMPOSE_FILE="$COMPOSE_FILE:compose.mariadb.yml"\n'
@@ -0,0 +1,33 @@
+"""Tests for substitute_jinja_variable()'s recursion into dicts, lists, and scalars."""
+
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from alakazam import substitute_jinja_variable
+
+
+class TestSubstituteJinjaVariable:
+    def test_a_scalar_in_a_dict_substitutes(self):
+        config = {"nextcloud": {"env": {"DOMAIN": "{{ domain }}"}}}
+        substitute_jinja_variable(config, {"domain": "nc.example.com"})
+        assert config["nextcloud"]["env"]["DOMAIN"] == "nc.example.com"
+
+    def test_a_scalar_in_a_list_substitutes(self):
+        config = {"nextcloud": {"initial-hooks": [
+            "app install_apps",
+            "app set_app_config files default_quota {{ quota }}",
+            "app run_occ",
+        ]}}
+        substitute_jinja_variable(config, {"quota": "0"})
+        hooks = config["nextcloud"]["initial-hooks"]
+        assert isinstance(hooks, list) and len(hooks) == 3
+        assert hooks[0] == "app install_apps"
+        assert "0" in hooks[1]
+        assert hooks[2] == "app run_occ"
+
+    def test_a_scalar_in_a_list_without_a_template_is_left_untouched(self):
+        config = {"nextcloud": {"initial-hooks": ["app run_occ 'app:install groupfolders'"]}}
+        substitute_jinja_variable(config, {"quota": "0"})
+        assert config["nextcloud"]["initial-hooks"] == ["app run_occ 'app:install groupfolders'"]
@@ -0,0 +1,68 @@
+"""Tests for reading the newest recipe version out of what 'abra recipe versions -m' writes."""
+
+import json
+import os
+import sys
+
+import click
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+from alakazam import get_latest_version, parse_machine_output
+
+# one document per recipe version, each listing the images of that version, as abra writes it
+GRIST_VERSIONS = (
+    '[{"name":"gristlabs/grist-oss","service":"app","tag":"1.7.19","version":"3.6.4+1.7.19"},'
+    '{"name":"redis","service":"redis","tag":"8.10-alpine","version":"3.6.4+1.7.19"}]\n'
+    '[{"name":"gristlabs/grist-oss","service":"app","tag":"1.7.18","version":"3.6.3+1.7.18"},'
+    '{"name":"redis","service":"redis","tag":"8.10-alpine","version":"3.6.3+1.7.18"}]\n'
+    '[{"name":"gristlabs/grist","service":"app","tag":"1.1.12","version":"1.0.0+1.1.12"},'
+    '{"name":"postgres","service":"db","tag":"13","version":"1.0.0+1.1.12"}]\n'
+)
+
+
+class TestParseMachineOutput:
+    def test_several_documents_are_read_as_a_list(self):
+        """A plain json.loads() stops after the first line with "Extra data"."""
+        assert len(parse_machine_output(GRIST_VERSIONS)) == 3
+
+    def test_a_single_document_stays_one_document(self):
+        assert parse_machine_output('{"a.org": {"apps": []}}') == {"a.org": {"apps": []}}
+
+    def test_a_single_list_stays_one_list(self):
+        assert parse_machine_output('[{"name": "db_password"}]') == [{"name": "db_password"}]
+
+    def test_blank_lines_are_skipped(self):
+        assert parse_machine_output('{"a": 1}\n\n{"b": 2}\n') == [{"a": 1}, {"b": 2}]
+
+    def test_output_that_is_no_json_at_all_still_raises(self):
+        """Swallowing it would turn a broken abra call into an empty but plausible answer."""
+        with pytest.raises(json.JSONDecodeError):
+            parse_machine_output("FATA something went wrong")
+
+
+class TestGetLatestVersion:
+    def install(self, monkeypatch, output):
+        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: parse_machine_output(output))
+
+    def test_the_newest_version_is_taken(self, monkeypatch):
+        self.install(monkeypatch, GRIST_VERSIONS)
+        assert get_latest_version("grist") == "3.6.4+1.7.19"
+
+    def test_the_order_of_the_documents_does_not_matter(self, monkeypatch):
+        self.install(monkeypatch, "\n".join(reversed(GRIST_VERSIONS.strip().splitlines())))
+        assert get_latest_version("grist") == "3.6.4+1.7.19"
+
+    def test_a_single_flat_document_is_read_too(self, monkeypatch):
+        """Which is what abra writes for a recipe that has exactly one version."""
+        self.install(monkeypatch, '[{"name":"redis","tag":"8.10","version":"1.0.0+1.1.12"}]')
+        assert get_latest_version("grist") == "1.0.0+1.1.12"
+
+    def test_a_recipe_without_versions_is_reported(self, monkeypatch):
+        """Reaching for the newest of nothing would end in an IndexError with no hint in it."""
+        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: {})
+        with pytest.raises(click.ClickException) as excinfo:
+            get_latest_version("grist")
+        assert "no versions for grist" in excinfo.value.message
@@ -168,7 +168,8 @@ class TestInstanceDomainThreading:
        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: "")
        calls = []
        monkeypatch.setattr(alakazam, "execute_cmds", lambda app_config, **k: calls.append(k))
-        deploy_apps({"a.org": [["authentik", "login.a.org"]]}, execute_hooks=True)
+        # the health check is not what this is about, and it would need a whole abra to answer
+        deploy_apps({"a.org": [["authentik", "login.a.org"]]}, execute_hooks=True, wait=False)
        [kwargs] = calls
        assert kwargs["deploy"] is True
        assert kwargs["instance_domain"] == "a.org"
@@ -109,9 +109,11 @@ class TestRelevantConfigPaths:
        assert config_root / "group" / "alaka-versions.yml" in paths
        assert config_root / "group" / "example.com.yml" in paths

-    def test_covers_the_connection_configuration(self, config_root):
+    def test_covers_every_config_set_file(self, config_root):
+        (config_root / "config-sets-authentik.yml").write_text("integrate:\n    apply-when: [a]\n")
        paths = get_relevant_config_paths(config_root, config_root / "group", [])
-        assert alakazam.Path(alakazam.COMBINE_PATH) in paths
+        assert config_root / "config-sets.yml" in paths
+        assert config_root / "config-sets-authentik.yml" in paths

    def test_a_single_instance_file_pulls_in_its_ancestors(self, config_root):
        paths = get_relevant_config_paths(config_root, config_root / "group" / "example.com.yml", [])
@@ -1,5 +1,6 @@
-"""Tests for which abra failures may be retried, and which configuration keys are ignored."""
+"""Tests for which abra failures may be retried."""

+import logging
 import os
 import sys

@@ -9,7 +10,7 @@ import pytest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

 import alakazam
-from alakazam import abra, is_connection_error, update_configs
+from alakazam import abra, get_retry_reason, is_connection_error


 class FakeProcess:
@@ -21,19 +22,22 @@ class FakeProcess:

 UNREACHABLE = b"FATA error during connect: kex_exchange_identification: Connection closed by remote host"
 REJECTED = b"FATA login.a.org is still deployed"
+AMBIGUOUS = (b"WARN ambiguous container list received, prompting for input\n"
+             b"? which container are you looking for?\nFATA EOF")
+
+
+@pytest.fixture
+def runs(monkeypatch):
+    """Hands out a prepared result per abra call and records what was slept between them."""
+    calls = {"results": [], "slept": []}
+    monkeypatch.setattr(alakazam, "sleep", calls["slept"].append)
+    monkeypatch.setattr(alakazam.subprocess, "run", lambda cmd, capture_output: calls["results"].pop(0))
+    return calls


 class TestRetryLoop:
    """A server that drops the ssh handshake is usually throttling, so the retries have to spread out."""

-    @pytest.fixture
-    def runs(self, monkeypatch):
-        calls = {"results": [], "slept": []}
-        monkeypatch.setattr(alakazam, "sleep", calls["slept"].append)
-        monkeypatch.setattr(alakazam.subprocess, "run",
-                            lambda cmd, capture_output: calls["results"].pop(0))
-        return calls
-
    def test_a_successful_command_does_not_retry(self, runs):
        runs["results"] = [FakeProcess(0)]
        abra("app", "ls")
@@ -85,19 +89,33 @@ class TestConnectionErrors:
        assert not is_connection_error(output)


-class TestUnknownConfigKeys:
-    def test_an_unknown_key_never_reaches_the_env_file(self, tmp_path):
-        """A promotion mechanism sets 'hold: <reason>' on a recipe, which must pass through untouched."""
-        env = tmp_path / "app.env"
-        env.write_text("#SMTP_HOST=mail.example.com\n")
-        update_configs(env, {"hold": "waiting for the upstream fix", "env": {"SMTP_HOST": "mail.a.org"}})
-        content = env.read_text()
-        assert "SMTP_HOST=mail.a.org" in content
-        assert "hold" not in content
-        assert "waiting for the upstream fix" not in content
+class TestAmbiguousContainer:
+    """
+    An app that was just updated keeps its replaced container for a while, and abra then cannot
+    tell which of the two a command is meant for. It asks, and a run without a terminal ends on EOF.
+    """

-    def test_an_unknown_key_alone_changes_nothing(self, tmp_path):
-        env = tmp_path / "app.env"
-        env.write_text("#SMTP_HOST=mail.example.com\n")
-        update_configs(env, {"hold": "waiting for the upstream fix"})
-        assert env.read_text() == "#SMTP_HOST=mail.example.com\n"
+    @pytest.mark.parametrize("output", [
+        "WARN ambiguous container list received, prompting for input\nFATA EOF",
+        "FATA expected 1 container but found 2: files_app.1.pxo3 (created About a minute ago)",
+    ])
+    def test_it_is_worth_another_attempt(self, output):
+        assert get_retry_reason(output) == ("found more than one container, "
+                                            "the replaced one has not gone away yet")
+
+    def test_an_unreachable_server_keeps_its_own_reason(self):
+        assert get_retry_reason("kex_exchange_identification") == "failed to reach the server"
+
+    def test_a_rejected_operation_is_worth_none(self):
+        assert get_retry_reason("FATA login.a.org is still deployed") is None
+
+    def test_the_command_is_repeated_until_one_container_is_left(self, runs):
+        runs["results"] = [FakeProcess(1, AMBIGUOUS), FakeProcess(1, AMBIGUOUS), FakeProcess(0)]
+        abra("app", "cmd", "-T", "files.a.org", "app", "--", "run_occ", "upgrade")
+        assert runs["slept"] == [alakazam.ABRA_RETRY_DELAY, alakazam.ABRA_RETRY_DELAY * 2]
+
+    def test_the_reason_is_reported(self, runs, caplog):
+        runs["results"] = [FakeProcess(1, AMBIGUOUS), FakeProcess(0)]
+        with caplog.at_level(logging.WARNING):
+            abra("app", "cmd", "-T", "files.a.org", "app", "--", "run_occ", "upgrade")
+        assert "the replaced one has not gone away yet" in caplog.text
@@ -0,0 +1,85 @@
+"""Tests for reading a secret out of a running app, and for what happens when that fails."""
+
+import logging
+import os
+import subprocess
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+from alakazam import get_running_containers, get_secret, run_over_ssh
+
+
+class TestGetRunningContainers:
+    def test_the_containers_are_returned(self, monkeypatch):
+        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: [{"service": "app"}])
+        assert get_running_containers("cloud.a.org") == [{"service": "app"}]
+
+    def test_an_app_that_is_not_deployed_has_none(self, monkeypatch):
+        def abra(*args, **kwargs):
+            raise RuntimeError("abra -o app ps -m cloud.a.org\n STDERR: FATA cloud.a.org is not deployed?")
+
+        monkeypatch.setattr(alakazam, "abra", abra)
+        assert get_running_containers("cloud.a.org") == []
+
+    def test_any_other_failure_is_raised(self, monkeypatch):
+        """Reading an unreachable server as "not running" would skip the exchange without a word."""
+        def abra(*args, **kwargs):
+            raise RuntimeError("abra -o app ps -m cloud.a.org\n STDERR: ssh: connect to host")
+
+        monkeypatch.setattr(alakazam, "abra", abra)
+        with pytest.raises(RuntimeError, match="connect to host"):
+            get_running_containers("cloud.a.org")
+
+
+class TestGetSecret:
+    def test_the_run_asks_for_no_terminal(self, monkeypatch):
+        """A ci run has no terminal to give, and one would rewrite the line endings of the value."""
+        calls = []
+        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: calls.append(a) or "s3cret")
+        assert get_secret("cloud.a.org", "db_password", ["app"]) == "s3cret"
+        assert calls[0][:4] == ("app", "run", "-t", "cloud.a.org")
+
+    def test_the_next_container_is_tried(self, monkeypatch):
+        def abra(*args, **kwargs):
+            if args[4] == "db":
+                raise RuntimeError("FATA no such container")
+            return "s3cret"
+
+        monkeypatch.setattr(alakazam, "abra", abra)
+        assert get_secret("cloud.a.org", "db_password", ["db", "app"]) == "s3cret"
+
+
+class TestRunOverSsh:
+    def result(self, returncode=0, stdout="", stderr=""):
+        return subprocess.CompletedProcess(["ssh"], returncode, stdout, stderr)
+
+    def test_the_output_is_returned_unchanged(self, monkeypatch):
+        """A trailing newline can be part of a secret, so nothing is stripped."""
+        monkeypatch.setattr(subprocess, "run", lambda *a, **k: self.result(stdout="s3cret\n"))
+        assert run_over_ssh("a.org", "cat x", "the read") == "s3cret\n"
+
+    def test_a_failing_command_is_raised(self, monkeypatch):
+        """It exited 1 and wrote nothing, which read as an empty secret before."""
+        monkeypatch.setattr(subprocess, "run",
+                            lambda *a, **k: self.result(returncode=1, stderr="Permission denied"))
+        with pytest.raises(RuntimeError) as excinfo:
+            run_over_ssh("a.org", "cat x", "the read")
+        assert "exited with code 1" in str(excinfo.value)
+        assert "Permission denied" in str(excinfo.value)
+
+    def test_a_silent_failure_is_still_raised(self, monkeypatch):
+        monkeypatch.setattr(subprocess, "run", lambda *a, **k: self.result(returncode=255))
+        with pytest.raises(RuntimeError, match="no output"):
+            run_over_ssh("a.org", "cat x", "the read")
+
+    def test_stderr_on_a_successful_command_is_reported(self, monkeypatch, caplog):
+        """ssh writes its own warnings there, and they explain an answer that looks wrong."""
+        monkeypatch.setattr(subprocess, "run",
+                            lambda *a, **k: self.result(stdout="s3cret", stderr="Pseudo-terminal will not be allocated"))
+        with caplog.at_level(logging.WARNING):
+            assert run_over_ssh("a.org", "cat x", "the read") == "s3cret"
+        assert "Pseudo-terminal" in caplog.text
@@ -0,0 +1,388 @@
+"""Tests for what an upgrade run does: deploy everything, wait for all of it, then hook."""
+
+import logging
+import os
+import sys
+
+import click
+import pytest
+from click.testing import CliRunner
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+
+# only nextcloud carries a pre-upgrade hook, so that "not configured" is distinguishable from
+# "configured but not run"
+CONFIG = {"a.org": {
+    "nextcloud": {"app_domain": "cloud.a.org", "server": "a.org", "version": "2.0.0",
+                  "pre-upgrade-hooks": ["db update_mongo_version"],
+                  "upgrade-hooks": ["app occ upgrade"]},
+    "wekan": {"app_domain": "boards.a.org", "server": "a.org", "version": "2.0.0",
+              "upgrade-hooks": ["app ready"]},
+}}
+
+# kept before any fixture replaces it, for the tests that check what a dry run actually prints
+REAL_EXECUTE_CMDS = alakazam.execute_cmds
+
+
+@pytest.fixture
+def versions():
+    """The version each app is deployed at, a test changes it to steer what needs upgrading."""
+    return {"nextcloud": "1.0.0", "wekan": "1.0.0"}
+
+
+@pytest.fixture
+def steps(monkeypatch, versions):
+    """Records what an upgrade run does, in order, against an abra that answers every call."""
+    steps = []
+    monkeypatch.setattr(alakazam, "INSTANCE_CONFIGS", CONFIG)
+    monkeypatch.setattr(alakazam, "sleep", lambda seconds: steps.append(f"sleep:{seconds}"))
+    monkeypatch.setattr(alakazam, "get_missing_secrets", lambda domain: [])
+    def execute_cmds(config, dry_run=False, **kw):
+        """
+        Labels a hook call by kind, and keeps quiet for an app that configures none of that kind.
+
+        selected() matches on a prefix, so the pre-upgrade label needs one of its own rather than a
+        suffix: "hook:pre:..." would be picked up by every assertion on "hook:".
+        """
+        kind, key = ("pre-hook", "pre-upgrade-hooks") if kw.get("pre_upgrade") else ("hook", "upgrade-hooks")
+        if not config.get(key):
+            return
+        steps.append(f"{'show:' if dry_run else ''}{kind}:{config['app_domain']}")
+
+    monkeypatch.setattr(alakazam, "execute_cmds", execute_cmds)
+
+    # the recipe filter has to be honoured here, it is what the wait narrows itself down with
+    def apps_by_deployment(recipes, deployed=True):
+        return {"a.org": [[app, CONFIG["a.org"][app]["app_domain"], version]
+                          for app, version in versions.items() if not recipes or app in recipes]}
+
+    def abra(*args, **kwargs):
+        if args[:2] == ("app", "ps"):
+            steps.append(f"ps:{args[2]}")
+            return {"app": {"service": "app", "status": "healthy", "version": "2.0.0",
+                            "chaos": "false", "state": "running", "image": "img"}}
+        # the planning phase asks for the release notes with the same command plus -r
+        if "-r" not in args:
+            steps.append(" ".join(a for a in args if a))
+        return ""
+
+    monkeypatch.setattr(alakazam, "get_apps_by_deployment", apps_by_deployment)
+    monkeypatch.setattr(alakazam, "abra", abra)
+    return steps
+
+
+def selected(steps, prefix):
+    return [step for step in steps if step.startswith(prefix)]
+
+
+def run(args, **kwargs):
+    result = CliRunner().invoke(alakazam.upgrade, args, standalone_mode=False, **kwargs)
+    assert result.exit_code == 0, result.exception
+    return result
+
+
+class TestUpgradeHooks:
+    def test_the_hooks_do_not_switch_on_converge_checks(self, steps):
+        """Waiting for each app in turn is what the separate flag is for."""
+        run(["-n", "-e"])
+        assert all("--no-converge-checks" in s for s in selected(steps, "app upgrade"))
+
+    def test_converge_checks_come_from_their_own_flag_only(self, steps):
+        run(["-n", "-c"])
+        assert not any("--no-converge-checks" in s for s in selected(steps, "app upgrade"))
+
+    def test_every_app_is_deployed_before_the_first_hook(self, steps):
+        run(["-n", "-e"])
+        deploys = selected(steps, "app upgrade")
+        assert max(steps.index(s) for s in deploys) < steps.index("hook:cloud.a.org")
+
+    def test_the_hooks_run_after_the_wait(self, steps):
+        """A hook would otherwise reach a container the upgrade has not brought up again yet."""
+        run(["-n", "-e"])
+        assert steps.index("ps:cloud.a.org") < steps.index("hook:cloud.a.org")
+        assert steps.index("ps:boards.a.org") < steps.index("hook:boards.a.org")
+
+    def test_the_health_is_only_read_after_the_services_settled(self, steps):
+        """A replaced service still reports its old container, which has been healthy all along."""
+        run(["-n", "-e"])
+        settle = steps.index(f"sleep:{alakazam.PS_WAIT_SETTLE}")
+        assert max(steps.index(s) for s in selected(steps, "app upgrade")) < settle
+        assert settle < steps.index("ps:cloud.a.org")
+
+    def test_without_the_flag_nothing_is_hooked(self, steps):
+        run(["-n"])
+        assert selected(steps, "hook:") == []
+
+    def test_the_wait_happens_without_the_flag_too(self, steps):
+        """It is what tells an upgrade run whether the apps came back at all."""
+        run(["-n"])
+        assert selected(steps, "ps:") == ["ps:cloud.a.org", "ps:boards.a.org"]
+
+    def test_a_filtered_run_waits_for_its_own_apps_only(self, steps):
+        run(["-n", "-e", "-r", "wekan"])
+        assert selected(steps, "ps:") == ["ps:boards.a.org"]
+        assert selected(steps, "hook:") == ["hook:boards.a.org"]
+
+    def test_an_app_already_at_its_version_is_neither_waited_for_nor_hooked(self, steps, versions):
+        versions["nextcloud"] = "2.0.0"
+        run(["-n", "-e"])
+        assert selected(steps, "ps:") == ["ps:boards.a.org"]
+        assert selected(steps, "hook:") == ["hook:boards.a.org"]
+
+    def test_an_app_with_an_unreadable_version_is_neither_waited_for_nor_hooked(self, steps, versions):
+        versions["nextcloud"] = "whatever"
+        run(["-n", "-e"])
+        assert selected(steps, "ps:") == ["ps:boards.a.org"]
+        assert selected(steps, "hook:") == ["hook:boards.a.org"]
+
+    def test_nothing_to_upgrade_skips_the_wait_entirely(self, steps, versions):
+        """Waiting on a run that did not happen would report on unrelated apps."""
+        versions.update({"nextcloud": "2.0.0", "wekan": "2.0.0"})
+        run(["-n", "-e"])
+        assert steps == []
+
+
+class TestDryRun:
+    """A dry run prints what would happen. Touching the server at all makes the flag pointless."""
+
+    def executed(self, steps):
+        return [step for step in steps if not step.startswith("show:")]
+
+    def test_an_interactive_dry_run_upgrades_nothing(self, steps):
+        """It still shows the release notes, so it asks a question that must not start anything."""
+        run(["--dry-run", "-e"], input="\nYES\n")
+        assert self.executed(steps) == []
+
+    def test_a_non_interactive_dry_run_upgrades_nothing(self, steps):
+        run(["-n", "--dry-run", "-e"])
+        assert self.executed(steps) == []
+
+    def test_an_interactive_dry_run_asks_nothing_about_upgrading(self, steps):
+        assert "Do you really want to upgrade" not in run(["--dry-run"], input="\n").output
+
+    def test_the_version_jump_is_shown_and_labelled(self, steps):
+        output = run(["--dry-run"], input="\n").output
+        assert "from" in output and "to" in output
+        assert "1.0.0" in output and "2.0.0" in output
+
+    def test_the_hooks_are_shown_when_they_would_run(self, steps, monkeypatch):
+        monkeypatch.setattr(alakazam, "execute_cmds", REAL_EXECUTE_CMDS)
+        assert "Run 'app occ upgrade' in cloud.a.org" in run(["--dry-run", "-e"], input="\n").output
+
+    def test_the_hooks_stay_out_without_the_flag(self, steps, monkeypatch):
+        """They would not run either, so showing them would misreport the run."""
+        monkeypatch.setattr(alakazam, "execute_cmds", REAL_EXECUTE_CMDS)
+        assert "app occ upgrade" not in run(["--dry-run"], input="\n").output
+
+    def test_a_shown_hook_is_not_executed(self, steps):
+        run(["--dry-run", "-e"], input="\n")
+        assert selected(steps, "show:") == ["show:pre-hook:cloud.a.org",
+                                            "show:hook:cloud.a.org", "show:hook:boards.a.org"]
+        assert selected(steps, "hook:") == []
+        assert selected(steps, "pre-hook:") == []
+
+
+class TestSettleOption:
+    def test_the_delay_can_be_set(self, steps):
+        run(["-n", "--settle", "5"])
+        assert "sleep:5" in steps
+        assert f"sleep:{alakazam.PS_WAIT_SETTLE}" not in steps
+
+    def test_zero_skips_the_delay_without_skipping_the_wait(self, steps):
+        """A converge-checked run has waited already, the health still has to be confirmed."""
+        run(["-n", "--settle", "0"])
+        assert selected(steps, "sleep:") == []
+        assert selected(steps, "ps:") == ["ps:cloud.a.org", "ps:boards.a.org"]
+
+
+class TestAFailingApp:
+    """A server that refuses one app must not take the apps behind it down with it."""
+
+    @pytest.fixture
+    def failing(self, monkeypatch, steps):
+        """Lets the upgrade of cloud.a.org fail the way abra reports a rejected deployment."""
+        original = alakazam.abra
+
+        def abra(*args, **kwargs):
+            # the planning phase asks for the release notes with the same command plus -r
+            if args[:2] == ("app", "upgrade") and "-r" not in args and "cloud.a.org" in args:
+                raise RuntimeError("abra -o app upgrade -n cloud.a.org 2.0.0 \n STDERR: FATA "
+                                   "failed to update cloud_a_org_db: context deadline exceeded")
+            return original(*args, **kwargs)
+
+        monkeypatch.setattr(alakazam, "abra", abra)
+        return steps
+
+    def invoke(self, args):
+        return CliRunner().invoke(alakazam.upgrade, ["-n"] + args, standalone_mode=False)
+
+    def test_the_other_apps_are_still_upgraded(self, failing):
+        self.invoke(["-e"])
+        assert "app upgrade -n --no-converge-checks boards.a.org 2.0.0" in failing
+
+    def test_the_run_ends_on_the_failure(self, failing):
+        result = self.invoke(["-e"])
+        assert isinstance(result.exception, click.ClickException)
+        assert "1 of 2 app(s) failed to upgrade: cloud.a.org" in result.exception.message
+
+    def test_the_failing_app_is_neither_waited_for_nor_hooked(self, failing):
+        """Its state is unknown, so a hook against it would run on whatever is left there."""
+        self.invoke(["-e"])
+        assert selected(failing, "ps:") == ["ps:boards.a.org"]
+        assert selected(failing, "hook:") == ["hook:boards.a.org"]
+
+    def test_the_reason_is_reported(self, failing, caplog):
+        with caplog.at_level(logging.ERROR):
+            self.invoke(["-e"])
+        assert "context deadline exceeded" in caplog.text
+
+    def test_every_app_failing_skips_the_wait(self, steps, monkeypatch):
+        def abra(*args, **kwargs):
+            if args[:2] == ("app", "upgrade") and "-r" not in args:
+                raise RuntimeError("FATA failed to update: context deadline exceeded")
+            return ""
+
+        monkeypatch.setattr(alakazam, "abra", abra)
+        result = self.invoke(["-e"])
+        assert isinstance(result.exception, click.ClickException)
+        assert "2 of 2 app(s) failed" in result.exception.message
+        assert selected(steps, "ps:") == []
+
+    def test_a_failing_wait_does_not_hide_the_failing_app(self, failing, monkeypatch):
+        """One failure covering up another is how a broken app slips through a green pipeline."""
+        monkeypatch.setattr(alakazam, "wait_for_healthy_apps", lambda *a, **k: (_ for _ in ()).throw(
+            click.ClickException("1 service(s) still not healthy after 900s")))
+        message = self.invoke(["-e"]).exception.message
+        assert "cloud.a.org" in message
+        assert "still not healthy" in message
+
+    def test_the_hooks_stay_out_when_the_wait_failed(self, failing, monkeypatch):
+        monkeypatch.setattr(alakazam, "wait_for_healthy_apps", lambda *a, **k: (_ for _ in ()).throw(
+            click.ClickException("1 service(s) still not healthy after 900s")))
+        self.invoke(["-e"])
+        assert selected(failing, "hook:") == []
+
+
+class TestPreUpgradeHooks:
+    """
+    A pre-upgrade hook prepares an app while its old containers still run, which is the only moment
+    some migrations have: mongo refuses to start on data whose feature compatibility version is two
+    majors behind the binary, so the version has to be raised before the binary is replaced.
+    """
+
+    def test_it_runs_before_that_app_is_deployed(self, steps):
+        run(["-n"])
+        upgrade = next(s for s in steps if "app upgrade" in s and "cloud.a.org" in s)
+        assert steps.index("pre-hook:cloud.a.org") < steps.index(upgrade)
+
+    def test_it_runs_before_the_undeploy_of_a_redeploy(self, steps):
+        """After an undeploy there is nothing left for the hook to read or change."""
+        run(["-n", "--redeploy"])
+        undeploy = next(s for s in steps if "app undeploy" in s and "cloud.a.org" in s)
+        assert steps.index("pre-hook:cloud.a.org") < steps.index(undeploy)
+
+    def test_it_runs_without_execute_hooks(self, steps):
+        """Unlike the other kinds: skipping a precondition does not skip work, it breaks the app."""
+        run(["-n"])
+        assert selected(steps, "pre-hook:") == ["pre-hook:cloud.a.org"]
+        assert selected(steps, "hook:") == []
+
+    def test_an_app_that_configures_none_gets_none(self, steps):
+        run(["-n"])
+        assert "pre-hook:boards.a.org" not in steps
+
+    def test_an_app_already_at_its_version_gets_none(self, steps, versions):
+        versions["nextcloud"] = "2.0.0"
+        run(["-n"])
+        assert selected(steps, "pre-hook:") == []
+
+    def test_the_flag_leaves_them_out(self, steps):
+        run(["-n", "--skip-pre-upgrade-hooks"])
+        assert selected(steps, "pre-hook:") == []
+        assert any("app upgrade" in s for s in steps)
+
+    def test_a_dry_run_shows_them_and_runs_nothing(self, steps):
+        run(["--dry-run"], input="\n")
+        assert selected(steps, "show:") == ["show:pre-hook:cloud.a.org"]
+        assert selected(steps, "pre-hook:") == []
+        assert not any("app upgrade" in s and "-r" not in s for s in steps)
+
+    def test_a_dry_run_honours_the_skip(self, steps):
+        run(["--dry-run", "--skip-pre-upgrade-hooks"], input="\n")
+        assert selected(steps, "show:") == []
+
+
+class TestAFailingPreUpgradeHook:
+    """It is a precondition, so it costs its own app the upgrade and no other app anything."""
+
+    @pytest.fixture
+    def failing(self, monkeypatch, steps):
+        def execute_cmds(config, dry_run=False, **kw):
+            if kw.get("pre_upgrade"):
+                if not config.get("pre-upgrade-hooks"):
+                    return
+                # what execute_cmds(strict=True) raises for a command the recipe does not have
+                raise click.ClickException(
+                    "command 'db update_mongo_version' failed for cloud.a.org: "
+                    "FATA no db function in abra.sh")
+            steps.append(f"hook:{config['app_domain']}")
+
+        monkeypatch.setattr(alakazam, "execute_cmds", execute_cmds)
+        return steps
+
+    def invoke(self, args):
+        return CliRunner().invoke(alakazam.upgrade, ["-n"] + args, standalone_mode=False)
+
+    def test_that_app_is_not_upgraded(self, failing):
+        self.invoke([])
+        assert not any("app upgrade" in s and "cloud.a.org" in s for s in failing)
+
+    def test_the_other_app_still_is(self, failing):
+        self.invoke([])
+        assert any("app upgrade" in s and "boards.a.org" in s for s in failing)
+
+    def test_the_run_ends_naming_the_app(self, failing):
+        result = self.invoke([])
+        assert isinstance(result.exception, click.ClickException)
+        assert "1 of 2 app(s) failed to upgrade: cloud.a.org" in result.exception.message
+
+    def test_the_failing_app_is_neither_waited_for_nor_hooked(self, failing):
+        self.invoke(["-e"])
+        assert selected(failing, "ps:") == ["ps:boards.a.org"]
+        assert selected(failing, "hook:") == ["hook:boards.a.org"]
+
+    def test_the_reason_is_reported(self, failing, caplog):
+        with caplog.at_level(logging.ERROR):
+            self.invoke([])
+        assert "no db function in abra.sh" in caplog.text
+
+
+class TestDryRunNamesTheHookKind:
+    """
+    A real run has "deploy <domain>" between the kinds; a dry run prints them back to back, so the
+    kind has to be named or a pre-upgrade hook is indistinguishable from an identical later one.
+    """
+
+    @pytest.fixture
+    def output(self, steps, monkeypatch):
+        monkeypatch.setattr(alakazam, "execute_cmds", REAL_EXECUTE_CMDS)
+        return run(["--dry-run", "-e"], input="\n").output
+
+    def test_both_kinds_are_named(self, output):
+        assert "pre-upgrade-hooks for cloud.a.org:" in output
+        assert "upgrade-hooks for cloud.a.org:" in output
+
+    def test_the_pre_upgrade_kind_comes_first(self, output):
+        assert output.index("pre-upgrade-hooks for cloud.a.org:") < output.index("upgrade-hooks for")
+
+    def test_an_app_without_the_kind_does_not_announce_it(self, output):
+        assert "pre-upgrade-hooks for boards.a.org:" not in output
+
+    def test_a_real_run_names_no_kind(self, steps, monkeypatch):
+        """There the deployment output separates them, and the label would only be noise."""
+        monkeypatch.setattr(alakazam, "execute_cmds", REAL_EXECUTE_CMDS)
+        output = run(["-n", "-e"]).output
+        assert "upgrade-hooks for" not in output
+        assert "Run 'db update_mongo_version' in cloud.a.org:db" in output
@@ -0,0 +1,88 @@
+"""Tests for filling in secrets a recipe added, before the upgrade that would fail on them."""
+
+import os
+import sys
+
+import pytest
+from click.testing import CliRunner
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import alakazam
+from alakazam import get_missing_secrets
+
+CONFIG = {"a.org": {"nextcloud": {"app_domain": "cloud.a.org", "server": "a.org", "version": "2.0.0"}}}
+
+
+class TestGetMissingSecrets:
+    def install(self, monkeypatch, stored):
+        monkeypatch.setattr(alakazam, "abra", lambda *a, **k: stored)
+
+    def test_a_complete_app_reports_nothing(self, monkeypatch):
+        self.install(monkeypatch, [{"name": "db_password", "created on server": "true"}])
+        assert get_missing_secrets("cloud.a.org") == []
+
+    def test_the_missing_names_are_returned(self, monkeypatch):
+        self.install(monkeypatch, [
+            {"name": "db_password", "created on server": "true"},
+            {"name": "oidc_secret", "created on server": "false"},
+        ])
+        assert get_missing_secrets("cloud.a.org") == ["oidc_secret"]
+
+    def test_an_app_without_any_secrets(self, monkeypatch):
+        """abra answers with an empty document rather than a list when nothing is stored."""
+        self.install(monkeypatch, {})
+        assert get_missing_secrets("cloud.a.org") == []
+
+
+class TestUpgradeFillsThemIn:
+    @pytest.fixture
+    def steps(self, monkeypatch):
+        steps = []
+        monkeypatch.setattr(alakazam, "INSTANCE_CONFIGS", CONFIG)
+        monkeypatch.setattr(alakazam, "sleep", lambda s: None)
+        monkeypatch.setattr(alakazam, "get_apps_by_deployment",
+                            lambda recipes, deployed=True: {"a.org": [["nextcloud", "cloud.a.org", "1.0.0"]]})
+        monkeypatch.setattr(alakazam, "create_secrets",
+                            lambda recipes, **kw: steps.append(f"secrets:{recipes[0]}"))
+        return steps
+
+    def install_abra(self, monkeypatch, steps, missing):
+        def abra(*args, **kwargs):
+            if args[:3] == ("app", "secret", "ls"):
+                return [{"name": n, "created on server": str(n not in missing).lower()}
+                        for n in ("db_password", "oidc_secret")]
+            if args[:2] == ("app", "ps"):
+                return {"app": {"service": "app", "status": "healthy", "version": "2.0.0",
+                                "chaos": "false", "state": "running", "image": "img"}}
+            steps.append(" ".join(a for a in args if a))
+            return ""
+        monkeypatch.setattr(alakazam, "abra", abra)
+
+    def invoke(self):
+        return CliRunner().invoke(alakazam.upgrade, ["-n"], standalone_mode=False)
+
+    def test_a_missing_secret_is_created_before_the_upgrade(self, monkeypatch, steps):
+        self.install_abra(monkeypatch, steps, missing={"oidc_secret"})
+        result = self.invoke()
+        assert result.exit_code == 0, result.exception
+        # the planning phase fetches release notes with the same command plus -r, skip that one
+        upgrade = next(i for i, s in enumerate(steps)
+                       if s.startswith("app upgrade") and " -r " not in s)
+        assert steps.index("secrets:nextcloud") < upgrade
+
+    def test_the_missing_name_is_reported(self, monkeypatch, steps):
+        """Silently repairing secrets would hide which one the recipe added."""
+        self.install_abra(monkeypatch, steps, missing={"oidc_secret"})
+        assert "missing 1 secret(s): oidc_secret" in self.invoke().output
+
+    def test_a_complete_app_does_not_run_the_secrets(self, monkeypatch, steps):
+        self.install_abra(monkeypatch, steps, missing=set())
+        self.invoke()
+        assert not any(s.startswith("secrets:") for s in steps)
+
+    def test_a_dry_run_creates_nothing(self, monkeypatch, steps):
+        """The check sits behind the confirmation, so --dry-run must not reach it."""
+        self.install_abra(monkeypatch, steps, missing={"oidc_secret"})
+        CliRunner().invoke(alakazam.upgrade, ["-n", "--dry-run"], standalone_mode=False)
+        assert not any(s.startswith("secrets:") for s in steps)
Author	SHA1	Message	Date
moritz	bc4ee485a6	feat(upgrade): add pre-upgrade-hooks that run before an app is replaced continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-10-06 11:39:03 +02:00
moritz	34dcd1d0c1	fix(hooks): retry a command whose replaced container is still around continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-22 13:40:54 +02:00
moritz	aced2b4162	fix(secrets): read a secret without asking for a terminal continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-22 12:46:33 +02:00
moritz	8a0e9d72c1	fix(upgrade): let the run continue past an app the server refuses continuous-integration/drone/push Build is passing Details	2026-09-22 01:09:02 +02:00
moritz	e043c09f01	fix(versions): read the json abra writes per recipe version continuous-integration/drone/push Build is passing Details	2026-09-21 23:53:47 +02:00
moritz	805dbd5099	feat(deploy): run the hooks after one wait for all apps continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-21 23:15:59 +02:00
moritz	7b4b3fb408	fix(upgrade): make --dry-run report instead of upgrading	2026-09-21 22:58:27 +02:00
moritz	ece7e3f6c1	feat(upgrade): run the hooks after one wait for all apps	2026-09-21 22:58:27 +02:00
moritz	4c52ddcac7	fix(env): keep an env value holding quotes parsable continuous-integration/drone/push Build is passing Details	2026-09-16 20:09:15 +02:00
moritz	f1507653f4	fix(ls): let --domains answer nothing instead of raising continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-15 13:26:45 +02:00
moritz	147f463134	increase recipe fetch time continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-15 03:32:22 +02:00
moritz	78eb98a7bf	fix(uptime): switch to a client that supports uptime kuma 2 continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-14 22:00:59 +02:00
moritz	f3e283ee99	Merge pull request 'fix(env, jinja) self referencing env values and template substitution in lists' (#15 ) from eCommons/alakazam:fix/env-repeated-keys-and-jinja-lists into main continuous-integration/drone/push Build is passing Details Reviewed-on: #15	2026-09-14 11:55:24 +00:00
moritz	3200514165	Merge pull request 'feat(config)!: replace combine.yml with auto-applying config-sets #12 ' (#14 ) from config-set-combine into main continuous-integration/drone/push Build is passing Details Reviewed-on: #14	2026-09-14 11:32:35 +00:00
moritz	d74895bd01	fix(config): report a misspelt app key in an automatic config-set continuous-integration/drone/push Build is passing Details continuous-integration/drone/pr Build is passing Details	2026-09-14 13:31:06 +02:00
dannygroenewegen	17abd1f02b	fix(jinja): recurse into lists when substituting template variables continuous-integration/drone/pr Build is passing Details substitute_jinja_variable() only recursed into dict values; a list value fell into the scalar branch, where str(value) stringified the whole list	2026-09-11 17:05:25 +02:00
dannygroenewegen	ba23f2c8e0	fix(env): correctly set self referencing env values dotenv.set_key() rewrites every line sharing a key, breaking recipes that legitimately repeat a key (e.g. COMPOSE_FILE) to accumulate a value via bash expansion. Values that reference their own key (e.g. "$COMPOSE_FILE:extra.yml") are now routed to a new set_extending_key(), which only touches the one matching line by uncommenting or appending it, leaving every other line sharing that key untouched.	2026-09-11 16:25:52 +02:00
moritz	cab4b1573c	feat(config)!: replace combine.yml with auto-applying config-sets continuous-integration/drone/push Build is passing Details continuous-integration/drone/pr Build is passing Details continuous-integration/drone/tag Build is passing Details BREAKING CHANGE: the integrations alakazam shipped as combine.yml are gone from the repository. Copy examples/config-sets-combine.yml into the root path to keep them, where they can be pinned and adjusted along with the versions. An integration is now a config-set carrying 'apply-when', which names the apps whose presence makes it apply. Naming the condition rather than deriving it from the configured apps is what lets a set configure only one side, as four of the seventeen pairs in combine.yml did.	2026-09-09 23:16:04 +02:00
moritz	df1c50174f	fix(combine): use monitoring-ng 1.x names for the grafana stack continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-09 21:24:04 +02:00
moritz	e91abd5a60	feat(upgrade): create the secrets a new recipe version added continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is passing Details	2026-09-09 16:46:37 +02:00
moritz	9d0d6f1f3a	fix(combine): uncomment kimai's SSO_ADMIN_GROUP_NAME continuous-integration/drone/push Build is passing Details	2026-09-08 15:21:14 +02:00
moritz	b7ab146acc	feat(combine): wire monitoring-ng's grafana stack to authentik continuous-integration/drone/push Build is passing Details continuous-integration/drone/tag Build is failing Details	2026-09-08 14:29:11 +02:00