# Alakazam

Alakazam is a meta-configuration app-connector and an abra wrapper, designed as a proof-of-concept to simplify the management of environment configuration files across multiple instances.

> [!WARNING]
> This software is in an experimental stage and has to be used with caution, expect things to break. 
>
> At the moment it is used in production for a [very tailored use case](https://wiki.local-it.cloud/s/kollicloud-wiki/doc/installation-wVp2LfBbg7).
> We try to mainline features of this project to `abra` and make the tool more accessible by everyone.
> Any feedback is highly appreciated.

## Problem Statement


Managing numerous environment (env) files can be prone to errors such as:

- Frequent copy-pasting mistakes.
- Difficulty maintaining an overview of configuration differences across instances.
- Laborious manual processes required for updating env files, particularly when underlying recipe env configurations change.
- Challenges in defining default configurations that should consistently apply across all apps and all instances.
- Manual effort required for app integration (e.g., SSO), including configuring environment files and exchanging secrets.

## Advantages of Alakazam

- **Global Configuration**: Prevents errors related to manual copying and pasting and ensures no critical configurations are overlooked.
- **Reduced Manual Configuration**: Facilitates the connection between multiple applications (i.E. SSO) through domain exchanges, secret sharing, and specific environmental variables.
- **Minimal Configuration Files**: Each instance configuration only contains parameters that deviate from the defaults or global configurations, simplifying management.
- **Standardized Subdomain Conventions**: Ensures consistent subdomain use across apps.
- **Simplified Setup**: Allows for the setup of an instance with multiple connected apps in one run.
- **Automatic Env Configuration Updates**: Eliminates manual env configuration.
- **Automatic Batch Updates**: update all apps of a group of instances at once.
- **Hierarchical Configuration Structure**: Supports configuration inheritance for ease of managing large-scale configurations.

## How does it work

`alakazam` serves as a wrapper for `abra`, automating the deployment process for multiple apps.
It utilizes YAML configuration files to generate the necessary .env files and manage secret exchanges between applications.
It supports the deployment or updating of multiple apps simultaneously and can execute post-deploy hooks.
All configurations are hierarchically structured in YAML files, which can inherit properties from the layers above.

## Tutorial

### Install alakazam

To install Alakazam, follow these steps to clone the repository and set up the executable.
Replace `apt` with your package manager of choice.

```bash
sudo apt update
sudo apt install curl git python3.11-venv
# Install golang
curl -sS https://webi.sh/golang | sh; \
source ~/.config/envman/PATH.env
# clone alakazam
git clone --recursive https://git.coopcloud.tech/moritz/alakazam.git
# install alakazam
cd alakazam
ln -s $PWD/alakazam.sh ~/.local/bin/alakazam
# Ensure `~/.local/bin` is in your `$PATH`
alakazam install
```

Create a global `~/.config/alakazam.yml` that contains at least the `root` path, see [Global Settings](#global-settings)

### Create a new instance:

To set up a new instance with Alakazam, begin by specifying the required applications in `example.com.yml`. The name of this file determines the domain for the instance.
Here's an example configuration:

```yaml
traefik:
backup-bot-two:
authentik:
nextcloud:
onlyoffice:
wordpress:
vikunja:
matrix-synapse:
element-web:
rallly:
```

Ensure that DNS records are set up for all your applications. The simplest approach is to create a `CNAME` record for the wildcard domain `*.example.com` to cover all subdomains used by the apps.

For a more comprehensive configuration, refer to the example in [./examples](./examples), which can also serve as a template for single-instance configurations (`example.com.yml`).

Steps to initialize the instance:

1. **Generate .env Files**: `alakazam example.com.yml config`
2. **Insert Secrets**: `alakazam example.com.yml secrets`
3. **Deploy Applications**: `alakazam example.com.yml deploy -e` 
    - the `-e` flag executes post-deployment hooks after each deployment

### Updating All Instances

To update all instances:

1. **Update Environment Files**: `alakazam example.com.yml config` to refresh .env files. It's a good practice to keep these files under version control (e.g., in a Git repository at `~/.abra/example.com`) and review changes with `git diff` before proceeding.
2. **Upgrade Applications**: `alakazam example.com.yml upgrade` to update all applications.

### Run CMDs

Escaping can be akward:
```
alakazam example.com.yml cmd -c 'app run_occ ''"config:app:get sociallogin custom_providers"''' -r nextcloud
```


## Configuration

### Instance Configuration

Configuration files support templating with Jinja2 and global variables, facilitating dynamic adjustments. The primary types of configuration files include:

1. **`alaka.yml`**/**`alaka-*.yml`**: Contains global configurations for a hierarchical layer, affecting all subordinate apps.
2. **`example.com.yml`** (`<your-domain>.yml`): A minimalist configuration for each instance, specifying the apps to be installed, the instance domain and optionally instance specific configurations.
    - an **instance** means a collection of apps that are integrated with each other (one server / vm can contain multiple instances)
3. **`combine.yml`**: Manages configurations required to integrate multiple apps with each other.
    - This configuration should not be touched by the operator.
    - At the moment it's part of the `alakazam` repository.
    - In future it should be split into the recipes repositories and maintained by the recipe maintainer.
        - We started to move each app entry as `alaconnect.yml` into the respective repositories.
4. **`config-sets.yml`**: Optional file placed at the `root` path. Defines named sets of per-app configurations that can be activated or deactivated per instance or group, avoiding duplication across instance files. See [examples/config-sets.yml](./examples/config-sets.yml).

### Global Settings

**`~/.config/alakazam.yml`**: The global settings for alakazam.
It must contain the `root` path to all instance configurations. This path contains all the `alaka.yml`,`alaka-*.yml` and `example.com.yml` files.
Further it can contain settings for an Uptime Kuma instance.

```
root: ~/root/path/to/my/instance/configurations
uptime_kuma:
    url: https://status.example.com
    user: <username>
    password: <password>
    parameter:
        interval: 300
        retryInterval: 360
        timeout: 48
        maxretries: 5
        ignoreTls: False
        notificationIDList: [15, 16]
        expiryNotification: True
```


### App Configuration

`alaka.yml`/`alaka-*.yml`, `example.com.yml` and `combine.yml`/`alaconnect.yml` contain a similar configuration structure.
For each app/recipe the following `<app_configurations>` can be used:

- **`uncomment`/`comment`**: Manages the active status of lines in configuration files, such as uncommenting needed variables .
    - it matches against parts of the line (i.E. `compose.smtp.yml`)
    - this is useful for env variables that are used multiple times like `COMPOSE_FILE`
- **`env`**: Sets values for environment variables.
- **`*-hooks`**: Specifies `abra.sh` commands or local scripts to run at specific stages.
    - **`initial-hooks`**: commands for initialisation
    - **`deploy-hooks`**: commands that should be run after each deployment
    - **`upgrade-hooks`**: commands that should be run after each upgrade
- **`secrets`**: Inserts specific values (i.E. smtp passwords) into secrets; future updates will support encrypted file usage.
- **`secret-hooks`**: Run `abra.sh` commands locally or local scripts for secrets that need to be generated.
- **`subdomain`**: Specifies the subdomain scheme for individual recipes and apps. (not available in `combine.yml`/`alaconnect.yml`)
    - i.e. `cloud.example.com` for nextcloud
- **`version`**: Controls the recipe version to deploy; if unspecified, the latest version is used. (not available in `combine.yml`/`alaconnect.yml`)

The `combine.yml`/`alaconnect.yml` configuration additionally contains:

- **`shared_secrets`**: Specifies secret sharing between apps.
     - `<source_secret_name>:<target_secret_name>`

#### \*-Hooks Command Formats

**Abra command** — runs an abra.sh command inside a container:

```yaml
initial-hooks:
  - app set_default_quota
```

**Local script** — runs a script on the local machine (for custom actions that aren't generic enough to be implemented as abra.sh commands):

```yaml
initial-hooks:
  - script ./scripts/script.sh arg1
```

Relative paths resolve from the `root` path. The script receives `ALAKAZAM_APP_DOMAIN`, `ALAKAZAM_APP_SERVER`, and `ALAKAZAM_INSTANCE_DOMAIN` as environment variables.

### Configuration Structure

Configuration can be simplified into a single `example.com.yml` or expanded into multiple layered `alaka.yml`/`alaka-*.yml` files for complex deployments. This allows for easy maintenance of multiple instances or groups.
These configurations are designed to modularize and simplify the management of app settings and integrations, making it easier to maintain and scale your deployments.

1. **`alaka.yml`/`alaka-*.yml` and `example.com.yml`**
    - **`alaka.yml`/`alaka-*.yml`**: Contain global app configurations (`<app_recipe>`) which are inherited by all configurations within its hierarchical layer. This ensures consistent settings across multiple applications. All `alaka-*.yml` files inside a directory are merged into one configuration. Avoid overrides between `alaka-*.yml` inside of one layer.
    - **`example.com.yml`**: Specifies which apps should be deployed (`<app_recipe>` entries), focusing on the individual needs of each instance.
    - Configurations within these files are structured as follows, with specific `<app_configuration>` settings detailed in the [App Configuration](#app-configuration) section:

    ```
    <app_recipe>:
        <app_configurations>
    ```

2. **`combine.yml`**
    - This file orchestrates the integration between multiple apps within the same instance (`example.com.yml`), applying `<target_app_configurations>` to ensure seamless functionality between connected applications.
    - The configuration for each app combination is structured as follows, again `<target_app_configurations>` is following the guidelines set in the [App Configuration](#app-configuration) section:

    ```
    <target_app_recipe>:
        <source_app_recipe>:
            <target_app_configurations>
    ```

    - In future each `<source_app_recipe>` entry will be placed in the `alaconnect.yml` inside the `<target_app_recipe>` folder.

3. **`config-sets.yml`**
    - Optional file at the `root` path that defines named sets of per-app configurations. Each top-level key is a config-set name; its value is a map of `<app_recipe>` to `<app_configurations>`.
    - Activate or disable sets per instance or group via `CONFIG-SETS` in any `alaka.yml` or `example.com.yml`. Merge priority: group < config-set < instance.
    - See [examples/config-sets.yml](./examples/config-sets.yml).


### Templating Configurations

Each configuration file can include a `GLOBALS` section that allows for templating with global variables, making it easier to manage repetitive values across different configurations. Here is how you can define and use these global variables:

```yaml
GLOBALS:
    server: example-server.com
    smtp_user: noreply
    smtp_domain: example.org
authentik:
    env:
        AUTHENTIK_EMAIL__USERNAME: "{{smtp_user}}@{{smtp_domain}}"
nextcloud:
    env: 
        MAIL_FROM_ADDRESS: "{{smtp_user}}"
```

- **`server`**: Specifies the server where the instance will be deployed. If not provided, the instance domain is automatically taken from the `example.com.yml` file name.

## Inheriting Configuration Structure

Alakazam supports a flexible inheritance model for configuration management, allowing for both simplicity and complexity depending on your deployment needs:

- **Simple Configuration**: For straightforward setups, you can manage everything within a single `example.com.yml` file, placing all necessary configurations directly in this document.
- **Multiple Similar Instances**: To manage multiple similar instances efficiently, common application configurations can be centralized in a global `alaka.yml`. This method reduces duplication by allowing shared settings across multiple instances. The global `alaka.yml` can also be split into multiple `alaka-*.yml` files which are merged together.
- **Grouped Instances**: For more complex setups, instances can be organized into folders, each with its own `alaka.yml`/`alaka-*.yml`. This structure allows for group-specific configurations, making it easier to manage settings at different levels of your infrastructure hierarchy.
- **Inherited Configurations**: In even more complex scenarios, each group can inherit configurations from the levels above it. This cascading setup ensures that changes at a higher level can be automatically applied to all subordinate groups and instances, maintaining consistency and ease of updates across your entire environment.

This hierarchical approach to configuration is designed to scale with your system’s complexity and can be visualized through the following example diagram:

![File Structure](./config_file_structure.png)

![Configuration Structure](./config_structure.png)