devydave/alakazam
0
0
Fork 0
forked from moritz/alakazam
Code Pull Requests Activity

breaking change: use ~/.config/alakazam.yml for setting root path and make -g parameter a fixed argument

Browse Source
This commit is contained in:
Moritz
2024-06-03 20:41:35 +02:00
parent 330d38da3a
commit 50ccd9a87d
2 changed files with 94 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
README.md
View File

@ -33,7 +33,7 @@ All configurations are hierarchically structured in YAML files, which can inheri
## Configuration
### Concept
### Instance Configuration
Configuration files support templating with Jinja2 and global variables, facilitating dynamic adjustments. The primary types of configuration files include:
@ -46,6 +46,29 @@ Configuration files support templating with Jinja2 and global variables, facilit
- In future it should be split into the recipes repositories and maintained by the recipe maintainer.
- We started to move each app entry as `alaconnect.yml` into the respective repositories.
### Global Settings
**`~/.config/alakazam.yml`**: The global settings for alakazam.
It must contain the `root` path to all instance configurations. This path contains all the `alaka.yml`,`alaka-*.yml` and `example.com.yml` files.
Further it can contain settings for an Uptime Kuma instance.
```
root: ~/root/path/to/my/instance/configurations
uptime_kuma:
url: https://status.example.com
user: <username>
password: <password>
parameter:
interval: 300
retryInterval: 360
timeout: 48
maxretries: 5
ignoreTls: False
notificationIDList: [15, 16]
expiryNotification: True
```
### App Configuration
`alaka.yml`/`alaka-*.yml`, `example.com.yml` and `combine.yml`/`alaconnect.yml` contain a similar configuration structure.
@ -142,6 +165,8 @@ ln -s $PWD/alakazam.sh ~/.local/bin/alakazam
# Ensure `~/.local/bin` is in your `$PATH`
```
Create a global `~/.config/alakazam.yml` that contains at least the `root` path, see [Global Settings](#global-settings)
## Create a new instance:
To set up a new instance with Alakazam, begin by specifying the required applications in `example.com.yml`. The name of this file determines the domain for the instance.
@ -166,14 +191,14 @@ For a more comprehensive configuration, refer to the example in [./examples](./e
Steps to initialize the instance:
1. **Generate .env Files**: `./alakazam -p example.com.yml config`
2. **Insert Secrets**: `./alakazam -p example.com.yml secrets`
3. **Deploy Applications**: `./alakazam -p example.com.yml deploy -r`
1. **Generate .env Files**: `./alakazam example.com.yml config`
2. **Insert Secrets**: `./alakazam example.com.yml secrets`
3. **Deploy Applications**: `./alakazam example.com.yml deploy -r`
- the `-r` flag executes post-deployment hooks after each deployment
### Updating All Instances
To update all instances:
1. **Update Environment Files**: `alakazam -p example.com.yml config` to refresh .env files. It's a good practice to keep these files under version control (e.g., in a Git repository at `~/.abra/example.com`) and review changes with `git diff` before proceeding.
2. **Upgrade Applications**: `./alakazam -p example.com.yml upgrade` to update all applications.
1. **Update Environment Files**: `alakazam example.com.yml config` to refresh .env files. It's a good practice to keep these files under version control (e.g., in a Git repository at `~/.abra/example.com`) and review changes with `git diff` before proceeding.
2. **Upgrade Applications**: `./alakazam example.com.yml upgrade` to update all applications.

107
alakazam.py
View File

@ -21,7 +21,7 @@ from ruamel.yaml.nodes import ScalarNode
from packaging import version
COMBINE_PATH = os.path.dirname(os.path.realpath(__file__)) + "/combine.yml"
# CONFIGS: dict: contains all app organized by recipe names and instance domains
# INSTANCE_CONFIGS: dict: contains all app organized by recipe names and instance domains
# The structure of the dictionary is as follows:
# {
# "instance_domain": {
@ -33,7 +33,23 @@ COMBINE_PATH = os.path.dirname(os.path.realpath(__file__)) + "/combine.yml"
# },
# ...
# },
CONFIGS = {}
INSTANCE_CONFIGS = {}
# SETTINGS: dics: contains global settings for alakazam
#{
# 'root': <path that contains all the instance config files>,
# 'uptime_kuma':
# {
# "url": ...,
# "user": ...,
# "password": ...,
# "paramter": {...},
# ...
# },
# ...
#}
SETTINGS = {}
SETTINGS_PATH = "~/.config/alakazam.yml"
class MySafeConstructor(SafeConstructor):
@ -139,21 +155,20 @@ def merge_dict(dict1: Dict[Any, Any], dict2: Dict[Any, Any], reverse_list_order:
return merged_dict
def merge_all_group_configs(config_path: str) -> Dict[str, Dict[str, Any]]:
def merge_all_group_configs(root_path: Path) -> Dict[str, Dict[str, Any]]:
"""
Recursively merges 'alaka.yml/alaka-*.yml' files within a specified directory into a single comprehensive configuration dictionary.
Configurations at higher directory levels get inherited and potentially overridden by configurations in lower directory levels.
Configurations at higher directory levels get inherited and potentially overridden by configurations in lower directory levels. The merged configurations at each recursion level are accessible by their directory path relative to the root path.
Args:
dir_path (str): The path to the directory containing hierarchical 'alaka.yml/alaka-*.yml' configuration files. The directory should follow an organizational structure where each subdirectory can contain 'alaka.yml/alaka-*.yml' files that inherits and potentially overrides settings from its parent directory's 'alaka.yml/alaka-*.yml' files.
root_path (str): The path to the directory containing hierarchical 'alaka.yml/alaka-*.yml' configuration files. The directory should follow an organizational structure where each subdirectory can contain 'alaka.yml/alaka-*.yml' files that inherits and potentially overrides settings from its parent directory's 'alaka.yml/alaka-*.yml' files.
Returns:
dict: A dictionary representing the merged configurations from all the 'alaka.yml/alaka-*.yml' files found in the directory hierarchy.
The dictionary's keys are the paths of the directories relative to the root directory specified by 'dir_path', indicating the source of the configurations. Each key maps to its respective merged configuration dictionary, which includes all inherited and overridden settings from higher-level directories down to the specified directory.
The dictionary's keys are the paths of the directories relative to the root directory specified by 'root_path', indicating the source of the configurations. Each key maps to its respective merged configuration dictionary, which includes all inherited and overridden settings from higher-level directories down to the specified directory.
"""
dir_path = Path(config_path).absolute()
merged_configs = {}
for root, _, files in os.walk(dir_path):
for root, _, files in os.walk(root_path):
no_config_found = True
for file in files:
if re.match(r'^alaka(-.*)?\.ya?ml$', file):
@ -247,7 +262,7 @@ def map_subdomain(recipe: str, instance_domain: str, app_config: Dict[str, Any])
return domain
def get_merged_instance_configs(config_path: str, group_configs: Dict[str, Any]) -> Dict[str, Dict[str, Any]]:
def get_merged_instance_configs(config_path: Path, group_configs: Dict[str, Any]) -> Dict[str, Dict[str, Any]]:
"""
Traverse a directory structure to read and merge all YAML configuration files for each instance.
This function supports a hierarchical configuration approach by aggregating configurations across directories.
@ -259,15 +274,14 @@ def get_merged_instance_configs(config_path: str, group_configs: Dict[str, Any])
Returns:
dict: A dictionary with domains as keys and their respective merged configurations as values.
"""
_config_path = Path(config_path).absolute()
if _config_path.is_file():
parent_path = os.path.dirname(_config_path)
instance_config = read_config(str(_config_path))
domain = _config_path.name.removesuffix('.yml').removesuffix('.yaml')
if config_path.is_file():
parent_path = os.path.dirname(config_path)
instance_config = read_config(str(config_path))
domain = config_path.name.removesuffix('.yml').removesuffix('.yaml')
merged_config = merge_instance_configs(group_configs[parent_path], domain, instance_config)
return {domain: merged_config}
instances = {}
for root, _, files in os.walk(Path(_config_path)):
for root, _, files in os.walk(config_path):
for file in files:
# This pattern matches for files of the format "<domain>.yml" or "<domain>.yaml"
pattern = r"^(?:[A-Za-z0-9](?:[A-Za-z0-9\-]{0,61}[A-Za-z0-9])?\.)+[A-Za-z]{2,6}(?:\.yaml|\.yml)$"
@ -722,27 +736,32 @@ def execute_cmds(app_config: Dict[str, Any]) -> None:
print(abra("app", "cmd", domain, container, *cmd, ignore_error=True))
@click.group()
@click.option('-l', '--log', 'loglevel')
@click.option('-g', '--group_path', 'group_path')
@click.option('-c', '--config_path', 'config_path', default=".")
def cli(loglevel: str, group_path: str, config_path: str) -> None:
@click.group(context_settings={"help_option_names": ['-h', '--help']})
@click.option('-l', '--log', 'loglevel', help='Desired logging level ("debug", "info", "warning", "error", "critical")')
@click.argument('group_path')
def cli(loglevel: str, group_path: str) -> None:
"""
Command-line interface setup function for the Alakazam application. It configures logging levels, loads configuration files, and merges configuration settings from specified paths.
This function is the entry point for CLI commands provided by the Alakazam tool.
Alakazam is a meta-configuration app-connector and an abra wrapper, designed as a proof-of-concept to simplify the management of environment configuration files across multiple instances.
Args:
loglevel (str): Desired logging level ("debug", "info", "warning", "error", "critical").
group_path (str): Path to the directory containing the desired configuration group.
config_path (str): Path to the root directory containing configuration files.
GROUP_PATH: path to the directory that contains a group of instance configurations or the instance configurations itself.
"""
global CONFIGS
all_group_configs = merge_all_group_configs(config_path)
if not Path(group_path).exists():
logging.error(f"{group_path} does not exists! Are you in the correct directory?")
global INSTANCE_CONFIGS
global SETTINGS
SETTINGS = read_config(SETTINGS_PATH)
if not (root_path:= SETTINGS.get('root')):
root_path = os.getcwd()
logging.warning(f"There is no 'root' path defined in '{SETTINGS_PATH}', use current path '{root_path}'instead")
_group_path = Path(group_path).expanduser().absolute()
_root_path = Path(root_path).expanduser().absolute()
if not _group_path.exists():
logging.error(f"{_group_path} does not exists! Are you in the correct directory?")
exit(1)
instance_configs = get_merged_instance_configs(group_path, all_group_configs)
CONFIGS = merge_connection_configs(instance_configs)
if not str(_group_path).startswith(str(_root_path)):
logging.error(f"{_root_path} does not contain {_group_path}?")
exit(1)
all_group_configs = merge_all_group_configs(_root_path)
instance_configs = get_merged_instance_configs(_group_path, all_group_configs)
INSTANCE_CONFIGS = merge_connection_configs(instance_configs)
if loglevel:
numeric_level = getattr(logging, loglevel.upper(), None)
if not isinstance(numeric_level, int):
@ -759,10 +778,10 @@ def show_config(apps: List[str]) -> None:
Args:
apps (list): A list of application names for which the instance configurations should be shown.
"""
filtered_configs = CONFIGS
filtered_configs = INSTANCE_CONFIGS
if apps:
filtered_configs = {server: {app: app_config for app, app_config in app_configs.items() if app in apps}
for server, app_configs in CONFIGS.items()}
for server, app_configs in INSTANCE_CONFIGS.items()}
print(json.dumps(filtered_configs, indent=2))
@ -776,7 +795,7 @@ def config(apps: List[str]) -> None:
Args:
apps (list): A list of application names for which .env files need to be generated or updated.
"""
for instance, instance_config in CONFIGS.items():
for instance, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = []
for app in apps:
@ -810,7 +829,7 @@ def secrets(apps: List[str]) -> None:
Args:
apps (list): A list of application names for which secrets are to be generated and managed.
"""
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
instance_apps = instance_config.keys()
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
@ -838,7 +857,7 @@ def get_deployed_apps(apps: List[str]) -> Dict[str, str]:
"""
deployed_apps = {}
processed_server = []
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
else:
@ -881,7 +900,7 @@ def deploy(apps: List[str], run_cmds: bool, force: bool, converge_checks: bool)
return
for instance, apps_to_deploy in instance_apps.items():
for app, domain in apps_to_deploy:
app_config = get_value(CONFIGS, instance, app)
app_config = get_value(INSTANCE_CONFIGS, instance, app)
version = app_config.get('version')
if not version:
version = 'latest'
@ -926,7 +945,7 @@ def upgrade(apps: List[str], run_cmds: bool, dry_run: bool) -> None:
upgrade_apps = []
for app_details in deployed_apps:
app, domain, deployed_version = app_details
app_config = get_value(CONFIGS, instance, app)
app_config = get_value(INSTANCE_CONFIGS, instance, app)
upgrade_version = app_config.get('version')
if not upgrade_version:
upgrade_version = get_latest_version(app)
@ -981,7 +1000,7 @@ def undeploy(apps: List[str]) -> None:
if input(f"Do you really want to undeploy these apps? Type YES: ") != "YES":
return
deployed_domains = get_deployed_apps(apps)
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
else:
@ -1009,7 +1028,7 @@ def cmds(apps: List[str]) -> None:
None
"""
deployed_domains = get_deployed_apps(apps)
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
else:
@ -1033,7 +1052,7 @@ def list_cmds(apps: List[str]) -> None:
Args:
apps (list): List of application names to list post-deployment commands for.
"""
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
else:
@ -1119,7 +1138,7 @@ def get_apps(apps: Optional[List[str]] = None) -> Dict[str, List[List]]:
dict: Dictionary containing instances as keys and lists of lists [app name, domain] as values.
"""
instance_apps = {}
for instance, instance_config in CONFIGS.items():
for instance, instance_config in INSTANCE_CONFIGS.items():
instance_app_domains = []
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
@ -1145,7 +1164,7 @@ def get_server(apps: Optional[List[str]] = None) -> Set[str]:
Set: Set containing the server domains.
"""
server = set()
for _, instance_config in CONFIGS.items():
for _, instance_config in INSTANCE_CONFIGS.items():
if apps:
selected_apps = [app for app in apps if app in instance_config.keys()]
else: