readme

README.md

@@ -1,3 +1,169 @@
 # Alakazam
 
 Proof-of-concept meta-configuration app-connector abra wrapper.
+
+## Advantages
+
+The following problems should be solved with this approach:
+
+- have global configuration that apply on every configuration:
+    - avoid copy pasta errors from reusing the same configuration for multiple instances
+    - avoid forgetting important configurations
+- reduce manual configuration overhead for connecting multiple apps
+    - exchange domains
+    - share secrets
+    - set specific env vars
+- have a minimal configuration file for each instance
+    - it should only contain parameters that differ from the defaults and the global configuration
+- set a global subdomain convention
+    - each app will always have the same subdomain
+- set up an instance with multiple connected apps in one run
+- Automatic updates of the env configurations 
+    - avoid manual env configuration at all
+
+## Configuration
+
+### Concept
+
+This concept contains the following configuration files.
+Each configuration can be templated with jinja2 and global variables.
+
+1. `defaults.yml`
+    - It contains configurations for each app that are applied on each instance
+    - i.E. smtp config, language...
+    - This configuration is for the operator to avoid copy pasta errors 
+2. `domains.yml`
+    - It contains a mapping for each app to a subdomain
+    - i.e. `nextcloud: cloud.example.com`
+    - This configuration is for the operator to keep the same naming convention
+3. `config.yml`
+    - a minimalist configuration per instance
+    - contains at least:
+        - the domain (and the server, if they differ)
+        - the apps to be installed
+    - can optionally contain instance specific configurations for each app
+4. `combine.yml`
+    - contains the configuration steps that are required to combine apps with each other
+        - the env variables that should be uncommented
+        - specific values for env variables
+        - secrets that should be shared with each other
+        - commands that should be executed
+    - This configuration should not be touched by the operator
+        - It should be either split into the recipes repositories and maintained by the recipe maintainer or it should be maintained by the alakazam maintainer
+
+### App Configuration `<app_configurations>`
+
+`defaults.yml`, `config.yml` and `combine.yml` contain a similar structure to configure the individual apps.
+For each app the following configuration steps can be used:
+
+- `uncomment`
+    - uncomment each matching line
+    - this is useful for env variables that are used multiple times like `COMPOSE_FILE`
+- `env`
+    - set the value for env variables
+- `execute`
+    - `abra.sh` commands that should be executed after deployment
+- `secrets`
+    - insert a specific value into a secret
+    - if the secrets can not be generated, i.e. smtp passwords
+
+The `combine.yml` configuration additionally contains
+
+- `shared_secrets`
+     - `<source_secret_name>:<target_secret_name>`
+     - a mapping to define which secrets should be shared between two apps
+
+Each configuration file can have a `GLOBALS` section, that cat be used to template the configuration with global values.
+
+```
+GLOBALS:
+    smtp_user: noreply
+    smtp_domain: example.org
+authentik:
+    env:
+        AUTHENTIK_EMAIL__USERNAME: "{{smtp_user}}@{{smtp_domain}}"
+nextcloud:
+    env: 
+        MAIL_FROM_ADDRESS: "{{smtp_user}}"
+```
+
+### Configuration Structure
+
+1. `defaults.yml`
+    - there is an entry for each `<app_recipe>` that should have global configuration
+    - the `<app_configuration>` is configured according to "App Configuration" above
+
+```
+<app_recipe>:
+    <app_configurations>
+```
+
+2. `domains.yml`
+
+```
+`<app_recipe>`: <subdomain>.example.com
+```
+
+3. `config.yml`
+
+```
+domain: <instance_domain>
+server: <instance_server>
+apps:
+    <app_recipe>:
+        <app_configurations>
+```
+
+4. `combine.yml`
+    - For each `<target_app_recipe>` there is a `<target_app_configurations>` that is only applied if the `config.yml` also contains the `<source_app_recipe>`, to combine these two apps.
+
+```
+<target_app_recipe>:
+    <source_app_recipe>:
+        <target_app_configurations>
+```
+
+
+## Process
+
+Creating a new instance
+
+1. The config.yml is created by the operator to define the domain and required apps
+2. `./alakazam.py init-server`
+    - installs `traefik` and `backup-bot-two`
+3. `./alakazam.py setup-apps`
+    1. the env files for each app are created
+    2. the `defaults.yml` configs/secrets are applied on the env files
+    3. the `config.yml` configs/secrets are applied on the env files
+    4. the `combine.yml` configs/secrets are applied on the env files
+        - the shared_secrets are generated
+        `<source_app_recipe>.example.com` is replaced according to `domains.yml` in the `<target_app` env file
+    5. all the remaining secrets are generated
+4. `./alakazam.py deploy-apps`
+    - each app is deployed
+    - the `execute` commands from `defaults.yml`, `configs.yml` and `combine.yml` are executed
+
+Step 3. (`./alakazam.py setup-apps`) should be executed every time the config is changed or the recipe env file is updated. It regenerates the env files and creates all the necessary secrets. 
+The env files itself shouldn't be edited manually anymore.
+
+## Conceptual Questions
+
+Please note any ideas and arguments you have: https://pad.local-it.org/dB0FsiomRCOCPs7rS40YSQ
+
+- Possible abra integration
+    - An independent wrapper for abra, that uses abras `--machine` output
+    - A go binary that is build on top of the abra library
+    - An abra command like `abra compose`
+
+- Should we move completely away from .env files?
+    - This would skip the translation step into env files
+    - It reduces the complexity!
+    - Recipes couldn't be run with `docker compose` only anymore
+        - Is this still possible?
+
+- Where should the `combine.yml` be maintained?
+    - Separated config for each recipe?
+        - better configuration versioning for different recipe versions
+        - harder to maintain the recipes: requires knowledge about the connectable apps
+    - Global config?
+