180 lines
10 KiB
Markdown
180 lines
10 KiB
Markdown
# Alakazam
|
||
|
||
Alakazam is a meta-configuration app-connector and an abra wrapper, designed as a proof-of-concept to simplify the management of environment configuration files across multiple instances.
|
||
|
||
## Problem Statement
|
||
|
||
|
||
Managing numerous environment (env) files can be prone to errors such as:
|
||
|
||
- Frequent copy-pasting mistakes.
|
||
- Difficulty maintaining an overview of configuration differences across instances.
|
||
- Laborious manual processes required for updating env files, particularly when underlying recipe env configurations change.
|
||
- Challenges in defining default configurations that should consistently apply across all apps and all instances.
|
||
- Manual effort required for app integration (e.g., SSO), including configuring environment files and exchanging secrets.
|
||
|
||
## Advantages of Alakazam
|
||
|
||
- **Global Configuration**: Prevents errors related to manual copying and pasting and ensures no critical configurations are overlooked.
|
||
- **Reduced Manual Configuration**: Facilitates the connection between multiple applications (i.E. SSO) through domain exchanges, secret sharing, and specific environmental variables.
|
||
- **Minimal Configuration Files**: Each instance configuration only contains parameters that deviate from the defaults or global configurations, simplifying management.
|
||
- **Standardized Subdomain Conventions**: Ensures consistent subdomain use across apps.
|
||
- **Simplified Setup**: Allows for the setup of an instance with multiple connected apps in one run.
|
||
- **Automatic Env Configuration Updates**: Eliminates manual env configuration.
|
||
- **Automatic Batch Updates**: update all apps of a group of instances at once.
|
||
- **Hierarchical Configuration Structure**: Supports configuration inheritance for ease of managing large-scale configurations.
|
||
|
||
## How does it work
|
||
|
||
`alakazam` serves as a wrapper for `abra`, automating the deployment process for multiple apps.
|
||
It utilizes YAML configuration files to generate the necessary .env files and manage secret exchanges between applications.
|
||
It supports the deployment or updating of multiple apps simultaneously and can execute post-deploy hooks.
|
||
All configurations are hierarchically structured in YAML files, which can inherit properties from the layers above.
|
||
|
||
## Configuration
|
||
|
||
### Concept
|
||
|
||
Configuration files support templating with Jinja2 and global variables, facilitating dynamic adjustments. The primary types of configuration files include:
|
||
|
||
1. **`alaka.yml`**/**`alaka-*.yml`**: Contains global configurations for a hierarchical layer, affecting all subordinate apps.
|
||
2. **`example.com.yml`** (`<your-domain>.yml`): A minimalist configuration for each instance, specifying the apps to be installed, the instance domain and optionally instance specific configurations.
|
||
- an **instance** means a collection of apps that are integrated with each other (one server / vm can contain multiple instances)
|
||
3. **`combine.yml`**: Manages configurations required to integrate multiple apps with each other.
|
||
- This configuration should not be touched by the operator.
|
||
- At the moment it's part of the `alakazam` repository.
|
||
- In future it should be split into the recipes repositories and maintained by the recipe maintainer.
|
||
- We started to move each app entry as `alaconnect.yml` into the respective repositories.
|
||
|
||
### App Configuration
|
||
|
||
`alaka.yml`/`alaka-*.yml`, `example.com.yml` and `combine.yml`/`alaconnect.yml` contain a similar configuration structure.
|
||
For each app/recipe the following `<app_configurations>` can be used:
|
||
|
||
- **`uncomment`/`comment`**: Manages the active status of lines in configuration files, such as uncommenting needed variables .
|
||
- it matches against parts of the line (i.E. `compose.smtp.yml`)
|
||
- this is useful for env variables that are used multiple times like `COMPOSE_FILE`
|
||
- **`env`**: Sets values for environment variables.
|
||
- **`execute`**: Specifies `abra.sh` commands to run post-deployment.
|
||
- **`secrets`**: Inserts specific values (i.E. smtp passwords) into secrets; future updates will support encrypted file usage.
|
||
- **`subdomain`**: Specifies the subdomain scheme for individual recipes and apps. (not available in `combine.yml`/`alaconnect.yml`)
|
||
- i.e. `cloud.example.com` for nextcloud
|
||
- **`version`**: Controls the recipe version to deploy; if unspecified, the latest version is used. (not available in `combine.yml`/`alaconnect.yml`)
|
||
|
||
The `combine.yml`/`alaconnect.yml` configuration additionally contains:
|
||
|
||
- **`shared_secrets`**: Specifies secret sharing between apps.
|
||
- `<source_secret_name>:<target_secret_name>`
|
||
|
||
### Configuration Structure
|
||
|
||
Configuration can be simplified into a single `example.com.yml` or expanded into multiple layered `alaka.yml`/`alaka-*.yml` files for complex deployments. This allows for easy maintenance of multiple instances or groups.
|
||
These configurations are designed to modularize and simplify the management of app settings and integrations, making it easier to maintain and scale your deployments.
|
||
|
||
1. **`alaka.yml`/`alaka-*.yml` and `example.com.yml`**
|
||
- **`alaka.yml`/`alaka-*.yml`**: Contain global app configurations (`<app_recipe>`) which are inherited by all configurations within its hierarchical layer. This ensures consistent settings across multiple applications. All `alaka-*.yml` files inside a directory are merged into one configuration. Avoid overrides between `alaka-*.yml` inside of one layer.
|
||
- **`example.com.yml`**: Specifies which apps should be deployed (`<app_recipe>` entries), focusing on the individual needs of each instance.
|
||
- Configurations within these files are structured as follows, with specific `<app_configuration>` settings detailed in the [App Configuration](#app-configuration) section:
|
||
|
||
```
|
||
<app_recipe>:
|
||
<app_configurations>
|
||
```
|
||
|
||
2. **`combine.yml`**
|
||
- This file orchestrates the integration between multiple apps within the same instance (`example.com.yml`), applying `<target_app_configurations>` to ensure seamless functionality between connected applications.
|
||
- The configuration for each app combination is structured as follows, again `<target_app_configurations>` is following the guidelines set in the [App Configuration](#app-configuration) section:
|
||
|
||
```
|
||
<target_app_recipe>:
|
||
<source_app_recipe>:
|
||
<target_app_configurations>
|
||
```
|
||
|
||
- In future each `<source_app_recipe>` entry will be placed in the `alaconnect.yml` inside the `<target_app_recipe>` folder.
|
||
|
||
|
||
### Templating Configurations
|
||
|
||
Each configuration file can include a `GLOBALS` section that allows for templating with global variables, making it easier to manage repetitive values across different configurations. Here is how you can define and use these global variables:
|
||
|
||
```yaml
|
||
GLOBALS:
|
||
server: example-server.com
|
||
smtp_user: noreply
|
||
smtp_domain: example.org
|
||
authentik:
|
||
env:
|
||
AUTHENTIK_EMAIL__USERNAME: "{{smtp_user}}@{{smtp_domain}}"
|
||
nextcloud:
|
||
env:
|
||
MAIL_FROM_ADDRESS: "{{smtp_user}}"
|
||
```
|
||
|
||
- **`server`**: Specifies the server where the instance will be deployed. If not provided, the instance domain is automatically taken from the `example.com.yml` file name.
|
||
|
||
## Inheriting Configuration Structure
|
||
|
||
Alakazam supports a flexible inheritance model for configuration management, allowing for both simplicity and complexity depending on your deployment needs:
|
||
|
||
- **Simple Configuration**: For straightforward setups, you can manage everything within a single `example.com.yml` file, placing all necessary configurations directly in this document.
|
||
- **Multiple Similar Instances**: To manage multiple similar instances efficiently, common application configurations can be centralized in a global `alaka.yml`. This method reduces duplication by allowing shared settings across multiple instances. The global `alaka.yml` can also be split into multiple `alaka-*.yml` files which are merged together.
|
||
- **Grouped Instances**: For more complex setups, instances can be organized into folders, each with its own `alaka.yml`/`alaka-*.yml`. This structure allows for group-specific configurations, making it easier to manage settings at different levels of your infrastructure hierarchy.
|
||
- **Inherited Configurations**: In even more complex scenarios, each group can inherit configurations from the levels above it. This cascading setup ensures that changes at a higher level can be automatically applied to all subordinate groups and instances, maintaining consistency and ease of updates across your entire environment.
|
||
|
||
This hierarchical approach to configuration is designed to scale with your system’s complexity and can be visualized through the following example diagram:
|
||
|
||
|
||
|
||
|
||
|
||
|
||
## Tutorial
|
||
|
||
### Install alakazam
|
||
|
||
To install Alakazam, follow these steps to clone the repository and set up the executable:
|
||
|
||
```bash
|
||
git clone https://git.coopcloud.tech/moritz/alakazam.git
|
||
cd alakazam
|
||
ln -s $PWD/alakazam.sh ~/.local/bin/alakazam
|
||
# Ensure `~/.local/bin` is in your `$PATH`
|
||
```
|
||
|
||
## Create a new instance:
|
||
|
||
To set up a new instance with Alakazam, begin by specifying the required applications in `example.com.yml`. The name of this file determines the domain for the instance.
|
||
Here's an example configuration:
|
||
|
||
```yaml
|
||
traefik:
|
||
backup-bot-two:
|
||
authentik:
|
||
nextcloud:
|
||
onlyoffice:
|
||
wordpress:
|
||
vikunja:
|
||
matrix-synapse:
|
||
element-web:
|
||
rallly:
|
||
```
|
||
|
||
Ensure that DNS records are set up for all your applications. The simplest approach is to create a `CNAME` record for the wildcard domain `*.example.com` to cover all subdomains used by the apps.
|
||
|
||
For a more comprehensive configuration, refer to the example in [./examples](./examples), which can also serve as a template for single-instance configurations (`example.com.yml`).
|
||
|
||
Steps to initialize the instance:
|
||
|
||
1. **Generate .env Files**: `./alakazam -p example.com.yml config`
|
||
2. **Insert Secrets**: `./alakazam -p example.com.yml secrets`
|
||
3. **Deploy Applications**: `./alakazam -p example.com.yml deploy -r`
|
||
- the `-r` flag executes post-deployment hooks after each deployment
|
||
|
||
### Updating All Instances
|
||
|
||
To update all instances:
|
||
|
||
1. **Update Environment Files**: `alakazam -p example.com.yml config` to refresh .env files. It's a good practice to keep these files under version control (e.g., in a Git repository at `~/.abra/example.com`) and review changes with `git diff` before proceeding.
|
||
2. **Upgrade Applications**: `./alakazam -p example.com.yml upgrade` to update all applications.
|