abra@c65be64e7d | ||
examples | ||
.gitmodules | ||
alakazam.py | ||
alakazam.sh | ||
combine.yml | ||
config_file_structure.png | ||
config_structure.drawio | ||
config_structure.png | ||
README.md | ||
requirements.txt |
Alakazam
Alakazam is a meta-configuration app-connector and an abra wrapper, designed as a proof-of-concept to simplify the management of environment configuration files across multiple instances.
Problem Statement
Managing numerous environment (env) files can be prone to errors such as:
- Frequent copy-pasting mistakes.
- Difficulty maintaining an overview of configuration differences across instances.
- Laborious manual processes required for updating env files, particularly when underlying recipe env configurations change.
- Challenges in defining default configurations that should consistently apply across all apps and all instances.
- Manual effort required for app integration (e.g., SSO), including configuring environment files and exchanging secrets.
Advantages of Alakazam
- Global Configuration: Prevents errors related to manual copying and pasting and ensures no critical configurations are overlooked.
- Reduced Manual Configuration: Facilitates the connection between multiple applications (i.E. SSO) through domain exchanges, secret sharing, and specific environmental variables.
- Minimal Configuration Files: Each instance configuration only contains parameters that deviate from the defaults or global configurations, simplifying management.
- Standardized Subdomain Conventions: Ensures consistent subdomain use across apps.
- Simplified Setup: Allows for the setup of an instance with multiple connected apps in one run.
- Automatic Env Configuration Updates: Eliminates manual env configuration.
- Automatic Batch Updates: update all apps of a group of instances at once.
- Hierarchical Configuration Structure: Supports configuration inheritance for ease of managing large-scale configurations.
How does it work
alakazam
serves as a wrapper for abra
, automating the deployment process for multiple apps.
It utilizes YAML configuration files to generate the necessary .env files and manage secret exchanges between applications.
It supports the deployment or updating of multiple apps simultaneously and can execute post-deploy hooks.
All configurations are hierarchically structured in YAML files, which can inherit properties from the layers above.
Configuration
Instance Configuration
Configuration files support templating with Jinja2 and global variables, facilitating dynamic adjustments. The primary types of configuration files include:
alaka.yml
/alaka-*.yml
: Contains global configurations for a hierarchical layer, affecting all subordinate apps.example.com.yml
(<your-domain>.yml
): A minimalist configuration for each instance, specifying the apps to be installed, the instance domain and optionally instance specific configurations.- an instance means a collection of apps that are integrated with each other (one server / vm can contain multiple instances)
combine.yml
: Manages configurations required to integrate multiple apps with each other.- This configuration should not be touched by the operator.
- At the moment it's part of the
alakazam
repository. - In future it should be split into the recipes repositories and maintained by the recipe maintainer.
- We started to move each app entry as
alaconnect.yml
into the respective repositories.
- We started to move each app entry as
Global Settings
~/.config/alakazam.yml
: The global settings for alakazam.
It must contain the root
path to all instance configurations. This path contains all the alaka.yml
,alaka-*.yml
and example.com.yml
files.
Further it can contain settings for an Uptime Kuma instance.
root: ~/root/path/to/my/instance/configurations
uptime_kuma:
url: https://status.example.com
user: <username>
password: <password>
parameter:
interval: 300
retryInterval: 360
timeout: 48
maxretries: 5
ignoreTls: False
notificationIDList: [15, 16]
expiryNotification: True
App Configuration
alaka.yml
/alaka-*.yml
, example.com.yml
and combine.yml
/alaconnect.yml
contain a similar configuration structure.
For each app/recipe the following <app_configurations>
can be used:
uncomment
/comment
: Manages the active status of lines in configuration files, such as uncommenting needed variables .- it matches against parts of the line (i.E.
compose.smtp.yml
) - this is useful for env variables that are used multiple times like
COMPOSE_FILE
- it matches against parts of the line (i.E.
env
: Sets values for environment variables.*-hooks
: Specifiesabra.sh
commands to run at specific stages.initial-hooks
: commands for initialisationdeploy-hooks
: commands that should be run after each deploymentupgrade-hooks
: commands that should be run after each upgrade
secrets
: Inserts specific values (i.E. smtp passwords) into secrets; future updates will support encrypted file usage.secret-hooks
: Runabra.sh
commands locally for secrets that need to be generated.subdomain
: Specifies the subdomain scheme for individual recipes and apps. (not available incombine.yml
/alaconnect.yml
)- i.e.
cloud.example.com
for nextcloud
- i.e.
version
: Controls the recipe version to deploy; if unspecified, the latest version is used. (not available incombine.yml
/alaconnect.yml
)
The combine.yml
/alaconnect.yml
configuration additionally contains:
shared_secrets
: Specifies secret sharing between apps.<source_secret_name>:<target_secret_name>
Configuration Structure
Configuration can be simplified into a single example.com.yml
or expanded into multiple layered alaka.yml
/alaka-*.yml
files for complex deployments. This allows for easy maintenance of multiple instances or groups.
These configurations are designed to modularize and simplify the management of app settings and integrations, making it easier to maintain and scale your deployments.
-
alaka.yml
/alaka-*.yml
andexample.com.yml
alaka.yml
/alaka-*.yml
: Contain global app configurations (<app_recipe>
) which are inherited by all configurations within its hierarchical layer. This ensures consistent settings across multiple applications. Allalaka-*.yml
files inside a directory are merged into one configuration. Avoid overrides betweenalaka-*.yml
inside of one layer.example.com.yml
: Specifies which apps should be deployed (<app_recipe>
entries), focusing on the individual needs of each instance.- Configurations within these files are structured as follows, with specific
<app_configuration>
settings detailed in the App Configuration section:
<app_recipe>: <app_configurations>
-
combine.yml
- This file orchestrates the integration between multiple apps within the same instance (
example.com.yml
), applying<target_app_configurations>
to ensure seamless functionality between connected applications. - The configuration for each app combination is structured as follows, again
<target_app_configurations>
is following the guidelines set in the App Configuration section:
<target_app_recipe>: <source_app_recipe>: <target_app_configurations>
- In future each
<source_app_recipe>
entry will be placed in thealaconnect.yml
inside the<target_app_recipe>
folder.
- This file orchestrates the integration between multiple apps within the same instance (
Templating Configurations
Each configuration file can include a GLOBALS
section that allows for templating with global variables, making it easier to manage repetitive values across different configurations. Here is how you can define and use these global variables:
GLOBALS:
server: example-server.com
smtp_user: noreply
smtp_domain: example.org
authentik:
env:
AUTHENTIK_EMAIL__USERNAME: "{{smtp_user}}@{{smtp_domain}}"
nextcloud:
env:
MAIL_FROM_ADDRESS: "{{smtp_user}}"
server
: Specifies the server where the instance will be deployed. If not provided, the instance domain is automatically taken from theexample.com.yml
file name.
Inheriting Configuration Structure
Alakazam supports a flexible inheritance model for configuration management, allowing for both simplicity and complexity depending on your deployment needs:
- Simple Configuration: For straightforward setups, you can manage everything within a single
example.com.yml
file, placing all necessary configurations directly in this document. - Multiple Similar Instances: To manage multiple similar instances efficiently, common application configurations can be centralized in a global
alaka.yml
. This method reduces duplication by allowing shared settings across multiple instances. The globalalaka.yml
can also be split into multiplealaka-*.yml
files which are merged together. - Grouped Instances: For more complex setups, instances can be organized into folders, each with its own
alaka.yml
/alaka-*.yml
. This structure allows for group-specific configurations, making it easier to manage settings at different levels of your infrastructure hierarchy. - Inherited Configurations: In even more complex scenarios, each group can inherit configurations from the levels above it. This cascading setup ensures that changes at a higher level can be automatically applied to all subordinate groups and instances, maintaining consistency and ease of updates across your entire environment.
This hierarchical approach to configuration is designed to scale with your system’s complexity and can be visualized through the following example diagram:
Tutorial
Install alakazam
To install Alakazam, follow these steps to clone the repository and set up the executable:
git clone https://git.coopcloud.tech/moritz/alakazam.git
cd alakazam
ln -s $PWD/alakazam.sh ~/.local/bin/alakazam
# Ensure `~/.local/bin` is in your `$PATH`
Create a global ~/.config/alakazam.yml
that contains at least the root
path, see Global Settings
Create a new instance:
To set up a new instance with Alakazam, begin by specifying the required applications in example.com.yml
. The name of this file determines the domain for the instance.
Here's an example configuration:
traefik:
backup-bot-two:
authentik:
nextcloud:
onlyoffice:
wordpress:
vikunja:
matrix-synapse:
element-web:
rallly:
Ensure that DNS records are set up for all your applications. The simplest approach is to create a CNAME
record for the wildcard domain *.example.com
to cover all subdomains used by the apps.
For a more comprehensive configuration, refer to the example in ./examples, which can also serve as a template for single-instance configurations (example.com.yml
).
Steps to initialize the instance:
- Generate .env Files:
alakazam example.com.yml config
- Insert Secrets:
alakazam example.com.yml secrets
- Deploy Applications:
alakazam example.com.yml deploy -e
- the
-e
flag executes post-deployment hooks after each deployment
- the
Updating All Instances
To update all instances:
- Update Environment Files:
alakazam example.com.yml config
to refresh .env files. It's a good practice to keep these files under version control (e.g., in a Git repository at~/.abra/example.com
) and review changes withgit diff
before proceeding. - Upgrade Applications:
alakazam example.com.yml upgrade
to update all applications.