Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 40db4e79f7 | |||
| 1d054aa2e8 | |||
| e7f979c6fa | |||
| 9b90b76fd0 | |||
| 15545b10ec | |||
| e95dd23504 | |||
| a2ecb767aa | |||
| 81c2a43128 |
+166
@@ -0,0 +1,166 @@
|
||||
# For Maintainers
|
||||
|
||||
From the perspective of the recipe maintainer, backup/restore is just more
|
||||
`deploy: ...` labels. Tools can read these labels and then perform the
|
||||
backup/restore logic.
|
||||
|
||||
## Tools
|
||||
|
||||
Two of the current "blessed" options are, which both implement the [backupbot specification](link to spec)
|
||||
|
||||
- [`backup-bot-two`](https://git.coopcloud.tech/coop-cloud/backup-bot-two)
|
||||
- [`abra`](https://git.coopcloud.tech/coop-cloud/abra)
|
||||
|
||||
### `backup-bot-two`
|
||||
|
||||
`backup-bot-two` is a recipe which gets deployed on the server, it can perform automatic backups and uses restic.
|
||||
Please see the [`README.md`](https://git.coopcloud.tech/coop-cloud/backup-bot-two#backupbot-ii) for the full docs.
|
||||
|
||||
### `abra`
|
||||
|
||||
`abra` will read labels and store backups in `~/.abra/backups/...` .
|
||||
It also provides an integration for `backup-bot-two`.
|
||||
|
||||
## Backup
|
||||
|
||||
### How to Configure backups
|
||||
|
||||
Unless otherwise stated all labels should be added to the main service (which should be named `app`).
|
||||
|
||||
1. Enable backups for the recipe:
|
||||
You need to enable backups for the recipe by adding the following deploy label:
|
||||
|
||||
```
|
||||
backupbot.backup=true
|
||||
```
|
||||
|
||||
2. Decide wich volumes should be backed up:
|
||||
By default all volumes will be backed up. To disable a certain volume you can add the following deploy label:
|
||||
|
||||
```
|
||||
backupbot.backup.volumes.{volume_name}=false
|
||||
```
|
||||
|
||||
3. Decide which path should be backed up on each volume
|
||||
By default all files get backed up for a volume. To only include certain paths you can add the following deploy label:
|
||||
|
||||
```
|
||||
backupbot.backup.volumes.{volume_name}.path=/mypath1/foo,/mypath2/bar
|
||||
```
|
||||
|
||||
Note: You can include multiple paths by providing a comma seperated list
|
||||
Note: All paths are specified relativ to the volume root
|
||||
|
||||
4. Run commands before the backup
|
||||
For certain services like a database it is not reccomend to just backup files, because the backup might end up in a corrupted state. Instead it is reccomended to make a database dump. You can run arbitrary commands in any container before the files are backed up.
|
||||
To do this add the following deploy label to the service on which you want the command being run:
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook=mysqldump -u root -pghost ghost --tab /var/lib/foo
|
||||
```
|
||||
|
||||
5. Run commands after the backup
|
||||
Sometimes you want to clean up after the backup. You can run arbitrary commands in any container after the files were backed up.
|
||||
To do this add the following deploy label to the service on which you want the command being run:
|
||||
|
||||
```
|
||||
backupbot.backup.post-hook=rm -rf /var/lib/mysql-files/*
|
||||
```
|
||||
|
||||
### Testing the backup
|
||||
|
||||
To test that your backup is configured correctly you can deploy the recipe you are working on in a test app either [locally](link to local server deployment) or on a test server.
|
||||
|
||||
After the deployment is succesfull run the backup and inspect its content
|
||||
|
||||
```
|
||||
abra app backup myrecipe.example.com
|
||||
tar -tf ~/.abra/backups/mybackup
|
||||
```
|
||||
|
||||
TODO: this is not complete yet
|
||||
|
||||
## Restore
|
||||
|
||||
When restoring an app, it takes the files from a backup and copies them to their correct location.
|
||||
In the case of restoring database tables, you can use the `pre-hook` & `post-hook` commands to run the insertion logic.
|
||||
|
||||
## Pre and Post hooks
|
||||
|
||||
To back up some services correctly it involves more than just copying a few files from one location to another. Some services already have specific backup tools that allow taking a coherent snapshot of its data like `mysqldump`.
|
||||
The pre and post hooks can be used to prepare the files which should get backed up and clean up afterwards.
|
||||
|
||||
Here are some examples:
|
||||
|
||||
### Example 1: Execute simple command
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: "echo 'foo' > /path/to/volume/bar.txt
|
||||
```
|
||||
|
||||
### Example 2: Access environment variable
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: "cat $${POSTGRES_PASSWORD_FILE}"
|
||||
```
|
||||
|
||||
### Example 3: Access secret
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: "cat /var/run/secrets/mysupersecret"
|
||||
```
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: 'mysqldump -p"$$(cat /run/secrets/mysupersecret)" mydatabase'
|
||||
```
|
||||
|
||||
### Example 4: Complex script
|
||||
|
||||
Sometimes the logic to backup up a service can get quite complex. In that case it might be easier to add a script (via mount or config) inside the container and call that from the pre and post hook:
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: "/scripts/my-pre-backup-scripts"
|
||||
backupbot.backup.post-hook: "/scripts/my-post-backup-scripts"
|
||||
```
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
### Mariadb
|
||||
|
||||
```
|
||||
services:
|
||||
db:
|
||||
image: mariadb
|
||||
volumes:
|
||||
- "mariadb:/var/lib/mysql"
|
||||
deploy:
|
||||
labels:
|
||||
backupbot.backup: "true"
|
||||
backupbot.backup.pre-hook: "sh -c 'mariadb-dump --single-transaction -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress | gzip > /var/lib/mysql/dump.sql.gz'"
|
||||
backupbot.backup.volume.mariadb.path: "dump.sql.gz"
|
||||
backupbot.backup.post-hook: "rm -f /var/lib/mysql/dump.sql.gz"
|
||||
backupbot.restore.post-hook: "sh -c 'gzip -d /var/lib/mysql/dump.sql.gz && mariadb -u root -p\"$$(cat /run/secrets/db_root_password)\" wordpress < /var/lib/mysql/dump.sql && rm -f /var/lib/mysql/dump.sql'"
|
||||
```
|
||||
|
||||
### Postgres
|
||||
|
||||
```
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
db:
|
||||
image: "postgres"
|
||||
volumes:
|
||||
- "postgres:/var/lib/postgresql/data"
|
||||
secrets:
|
||||
- db_password
|
||||
deploy:
|
||||
labels:
|
||||
backupbot.backup: "true"
|
||||
backupbot.backup.pre-hook: "PGPASSWORD=$$(cat $${POSTGRES_PASSWORD_FILE}) pg_dump -U $${POSTGRES_USER} $${POSTGRES_DB} > /var/lib/postgresql/data/backup.sql"
|
||||
backupbot.backup.post-hook: "rm -rf /var/lib/postgresql/data/backup.sql"
|
||||
backupbot.backup.volume.postgres.path: "backup.sql"
|
||||
|
||||
volumes:
|
||||
postgres:
|
||||
```
|
||||
@@ -0,0 +1,136 @@
|
||||
# Specification
|
||||
|
||||
## Summary
|
||||
|
||||
Creating automated backups of docker swarm services is an often needed task. This specification describes how backups can be configured via [service labels](https://docs.docker.com/compose/compose-file/compose-file-v3/#labels-1) in a standardised way.
|
||||
|
||||
## Requirements
|
||||
|
||||
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this specification are to be interpreted as described in [RFC-2119](https://datatracker.ietf.org/doc/html/rfc2119).
|
||||
|
||||
## Backup
|
||||
|
||||
To enable backups for a docker stack, the `backupbot.backup=true` label MUST be set on one of its services. The label MUST NOT be set multiple times for a docker stack. Otherwise the implementation MUST show an error. The label SHOULD be declared on the main service.
|
||||
|
||||
### Volumes and paths
|
||||
|
||||
By default all volumes MUST be backed up. A volume MUST be excluded from the backup when `backupbot.backup.volumes.{volume_name}=false` is set, where `{volume_name}` is the name of the volume.
|
||||
By default all files MUST be backed up on a volume. `backupbot.backup.volumes.{volume_name}.path` MAY be set to limit the paths for that volume. The value MUST be a valid path relative to the volume root. It MAY contain multiple paths which get separated by a comma. When the label is set only the given paths MUST be backed up.
|
||||
|
||||
### Pre/Post Hooks
|
||||
|
||||
A `backupbot.backup.pre-hook` and `backupbot.backup.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after backing up files.
|
||||
There is no guaranteed order in which different hooks MUST be executed.
|
||||
|
||||
TODO: escaping
|
||||
|
||||
### Output
|
||||
|
||||
A backup implementation SHOULD provide the backup of one or multiple stacks in a `.tar.gz` format. In that case each volume MUST be in `/var/lib/docker/volumes/{stack_name}_{volume_name}`, where `{stack_name}` is the name of the docker stack and `{volume_name}` is the name of each volume that got backed up.
|
||||
|
||||
## Restore
|
||||
|
||||
By default all files MUST be restored into their volume. A volume or path MAY be excluded from restoring. When restoring a backup from a `.tar.gz` it expects the directory layout as described in the [backup output](#output) section.
|
||||
|
||||
### Pre/Post Hooks
|
||||
|
||||
A `backupbot.restore.pre-hook` and `backupbot.restore.post-hook` MAY be set on a service. When set the command MUST be executed inside the running container of the service before/after restoring the files.
|
||||
There is no guaranteed order in which different hooks MUST be executed.
|
||||
|
||||
## Labels
|
||||
|
||||
### `backupbot.backup`
|
||||
|
||||
**Type:** boolean
|
||||
**Default:** false
|
||||
**Description:**
|
||||
Enables backups for this compose stack. The label should be added to the main service of the compose stack.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.backup: true
|
||||
```
|
||||
|
||||
### `backupbot.backup.volumes.{volume_name}`
|
||||
|
||||
**Type:** boolean
|
||||
**Default:** true
|
||||
**Description:** When set to false the volume is excluded from backups.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.backup.volumes.{volume_name}: false
|
||||
```
|
||||
|
||||
### `backupbot.backup.volumes.{volume_name}.path`
|
||||
|
||||
**Type:** string
|
||||
**Default:** ""
|
||||
**Description:**
|
||||
A comma seperated list of paths. When one or more paths are set, it only backs up those on the given volume instead of the whole volume.
|
||||
|
||||
**Example 1:**
|
||||
|
||||
```
|
||||
backupbot.backup.volumes.{volume_name}.path: '/var/lib/mariadb/dump.sql.gz'
|
||||
```
|
||||
|
||||
**Example 2:**
|
||||
```
|
||||
backupbot.backup.volumes.{volume_name}.path: '/var/lib/myapp/foo,/var/lib/myapp/bar'
|
||||
```
|
||||
|
||||
### `backupbot.backup.pre-hook`
|
||||
|
||||
**Type:** string
|
||||
**Default:** ""
|
||||
**Description:**
|
||||
A command, that gets executed before the files are backed up.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.backup.pre-hook: 'mysqldump -u root -p"$(cat /run/secrets/db_root_password)" -f /volume_path/dump.db'
|
||||
```
|
||||
|
||||
### `backupbot.backup.post-hook`
|
||||
|
||||
**Type:** string
|
||||
**Default:** ""
|
||||
**Description:**
|
||||
A command, that gets executed after the files are backed up.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.backup.post-hook: "rm -rf /volume_path/dump.db"
|
||||
```
|
||||
|
||||
### `backupbot.restore.pre-hook`
|
||||
|
||||
**Type:** string
|
||||
**Default:** ""
|
||||
**Description:**
|
||||
A command, that gets executed before the files are restored.
|
||||
Note, that there is no guaranteed order in which multiple hooks get executed.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.restore.pre-hook: "lock db"
|
||||
```
|
||||
|
||||
### `backupbot.restore.post-hook`
|
||||
|
||||
**Type:** string
|
||||
**Default:** ""
|
||||
**Description:**
|
||||
A command, that gets executed after the files are restored.
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
backupbot.restore.post-hook: "sqldump dump.sql && unlock db && rm dump.sql"
|
||||
```
|
||||
@@ -15,8 +15,6 @@ Definitely set up autocomplete or you'll be sad :sob: `abra` supports `bash`,
|
||||
|
||||
```
|
||||
$ abra autocomplete bash
|
||||
# Restart your terminal or load autocompletion in place
|
||||
$ source /etc/bash_completion.d/abra
|
||||
```
|
||||
|
||||
|
||||
|
||||
+20
-54
@@ -54,44 +54,11 @@ go test ./pkg/recipe -v -run TestGetVersionLabelLocalDoesNotUseTimeoutLabel
|
||||
|
||||
## Integration tests
|
||||
|
||||
### Running on the CI server
|
||||
### Install dependencies
|
||||
|
||||
Based on
|
||||
[R020](https://docs.coopcloud.tech/federation/resolutions/passed/020/), we have
|
||||
automated running the integration test suite. Here's the TLDR;
|
||||
|
||||
* We have a donated CI server (tysm `@mirsal` 💝) standing at the ready,
|
||||
`int.coopcloud.tech`.
|
||||
* We run the entire integration suite nightly via our Drone CI/CD configuration [here](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/.drone.yml) (see "`name: integration test`" stanza)
|
||||
* Here is the script that is run on the remote server: [`run-ci-int`](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/tests/run-ci-int)
|
||||
|
||||
What follows is a listing of how this was achieved so that we can collectivise
|
||||
the maintenance.
|
||||
|
||||
On the server, we have:
|
||||
|
||||
* Created an `abra` user with `docker` permissions
|
||||
* Ran `apt install bats bats-file bats-assert bats-support jq make git golang-1.21 wget bash`
|
||||
* Installed `bats-core` from source, following the instructions below
|
||||
* Docker was already installed on the machine, so nothing to do there
|
||||
* `docker login` with the `thecoopcloud` details so we don't get rate limited
|
||||
|
||||
The drone configuration was wired up as follows:
|
||||
|
||||
* Generated a SSH key and put the public key part in `~/.ssh/authorize_keys`
|
||||
* Added that public key part as a "deploy key" in the abra repo (so we can do `ssh://` git remote pulls)
|
||||
* Added the private key part as a Drone secret which is available in build so that the build can SSH over to the server to run commands. That was done like so: `drone secret add --repository coop-cloud/abra --name abra_int_private_key --data @id_ed25519`
|
||||
* In order to specify a cron timing, you need to create it with the Drone CLI: `drone cron add "coop-cloud/abra" "integration" @daily --branch main`
|
||||
|
||||
Please ask `@decentral1se` or on the Matrix channels for SSH access to the machine.
|
||||
|
||||
### Running them locally
|
||||
|
||||
#### Install dependencies
|
||||
|
||||
We use [`bats`](https://bats-core.readthedocs.io/en/stable/) to run the tests.
|
||||
You can install the required dependencies with the following. You also need a
|
||||
working installation of Docker and Go >= 1.16 (not covered in this section).
|
||||
We use [`bats`](https://bats-core.readthedocs.io/en/stable/), you can install
|
||||
the required dependencies with the following. You also need a working
|
||||
installation of Docker and Go (not covered in this section).
|
||||
|
||||
```
|
||||
apt install bats-file bats-assert bats-support jq make git
|
||||
@@ -108,14 +75,12 @@ cd bats-core
|
||||
sudo ./install.sh /usr/local
|
||||
```
|
||||
|
||||
#### Setup Test Server
|
||||
### Setup Test Server
|
||||
|
||||
For some tests an actual server is needed, where apps can be deployed. You can
|
||||
either use a local one or a remote test server. There is also a way to run or
|
||||
skip tests that require a remote server. This is covered below in the
|
||||
[filtering tests](#filter-tests_1) section.
|
||||
For many tests an actual server is needed, where apps can be deployed. You can
|
||||
either use a local one or a remote test server.
|
||||
|
||||
##### Remote swarm
|
||||
#### With remote test server
|
||||
|
||||
```
|
||||
export ABRA_TEST_DOMAIN="test.example.com"
|
||||
@@ -124,9 +89,14 @@ export ABRA_DIR="$HOME/.abra_test"
|
||||
|
||||
`ABRA_TEST_DOMAIN` should also have a DNS A record for `*.test.example.com`
|
||||
which points to the same server so that the test suite can deploy apps freely.
|
||||
The test suite does not deploy Traefik for you.
|
||||
It's advised that you re-use the same server and therefore the same Traefik
|
||||
deployment for running your integration tests. The test suite does not deploy
|
||||
Traefik for you. Then you'll have more stable results.
|
||||
|
||||
##### Local swarm
|
||||
You probably don't want to run the entire test suite though, it takes a while.
|
||||
Try the following for starters.
|
||||
|
||||
#### With local swarm
|
||||
|
||||
When running the test suite localy you need a running docker swarm setup:
|
||||
|
||||
@@ -158,11 +128,10 @@ bats -Tp tests/integration/autocomplete.bats
|
||||
|
||||
### Tagging tests
|
||||
|
||||
When a test actually deploys something, we tag it as "slow". When the test
|
||||
requires public DNS, we use "dns". There may be more tags we write more tests.
|
||||
When a test actually deploys something to a server, we tag it with the following:
|
||||
|
||||
```
|
||||
# bats test_tags=slow,dns
|
||||
# bats test_tags=slow
|
||||
@test "..." {
|
||||
...
|
||||
}
|
||||
@@ -197,17 +166,14 @@ bats -Tp tests/integration --filter "validate app argument"
|
||||
You can filter on tags.
|
||||
|
||||
```
|
||||
bats -Tp tests/integration --filter-tags \!slow # only fast tests
|
||||
bats -Tp tests/integration --filter-tags slow # only slow tests
|
||||
bats -Tp tests/integration --filter-tags slow,\!dns # slow but no DNS tests
|
||||
bats -Tp tests/integration --filter-tags "\!slow" # only fast tests
|
||||
bats -Tp tests/integration --filter-tags "slow" # only slow tests
|
||||
```
|
||||
|
||||
You can also only run the previously failed tests.
|
||||
|
||||
```
|
||||
mkdir -p tests/integration/.bats/run-logs
|
||||
bats -Tp tests/integration # run tests
|
||||
bats -Tp tests/integration --filter-status failed # re-run only failed
|
||||
bats -TP tests/integration --filter-status failed
|
||||
```
|
||||
|
||||
### Debug tests
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
---
|
||||
title: 2024-04-17
|
||||
---
|
||||
|
||||
## Meta
|
||||
|
||||
* Poll: https://poll.local-it.org/invite/Q828kjlYLNwW
|
||||
* Call: https://talk.local-it.org/rooms/nyy-z5y-yrh-sc2/join
|
||||
* Present: Local IT (moritz), EOTL (BaseBuilder, blu), BeWater(d1), Autonomic (Lai), Klasse & Methode (p4u1)
|
||||
|
||||
## Agenda
|
||||
|
||||
### First
|
||||
|
||||
* Fixed monthly Federation meeting (3rd Mon, etc) `@basebuilder`
|
||||
* Project re-organisation (recipes, tools, fedi repos) `@d1`
|
||||
* Backup specification `@p4u1`
|
||||
|
||||
### The Rest
|
||||
|
||||
* Non-Federation tasks specific bounty / funding `@basebuilder`
|
||||
* Website and docs work to better showcase federation - `@kawaiipunk`
|
||||
* https://git.coopcloud.tech/coop-cloud/organising/milestone/43
|
||||
* Recipe maintainence proposal - `@kawaiipunk`
|
||||
* "Hacking velocity = slow & money" (RE: recent fedi orga chat) `@d1`
|
||||
* Continuing budget 001 for meeting attendance, resolution 004 technically only covered 6 months to oct 2023 `@3wc` (but I won't be there)
|
||||
|
||||
## Notes
|
||||
|
||||
### Fixed monthly Federation meeting (3rd Mon, etc)
|
||||
|
||||
Talked about it couple of times, back and forth.
|
||||
- People who want to do regular can do that
|
||||
- Other people can do polled meeting
|
||||
- Poll every month is time consuming
|
||||
- Timezones is an issue
|
||||
|
||||
Poll options for meeting
|
||||
1. fix time/date every month
|
||||
1. fixed time/date with timezone wraparound (can be merged with 1. :)
|
||||
1. flexible every month (poll)
|
||||
1. fixed week with poll (day of week, crab.fit)
|
||||
|
||||
> crab.fit - software with heatmap of availability
|
||||
|
||||
### Project re-organisation (recipes, tools, fedi repos)
|
||||
|
||||
Problem: All projects are under one organisation (coop-cloud). Abra has to do a lot of work to figure out what is a recipe repo and what not. This got fixed but made recipe generation really slow
|
||||
|
||||
Proposal: 3 Organisations in gitea:
|
||||
- Recipes
|
||||
- Tools
|
||||
- Projects
|
||||
|
||||
What to look out for:
|
||||
- Redirects (mainly for recipes)
|
||||
- SSH will break though -> could make a migration script for that?
|
||||
|
||||
https://git.coopcloud.tech/coop-cloud/organising/milestone/45
|
||||
https://git.coopcloud.tech/coop-cloud/organising/issues/569
|
||||
|
||||
Maybe "tools" / "projects" not needed, only "recipes" / "other".
|
||||
|
||||
### Backup Specification
|
||||
|
||||
Needing to write operators and matainers guide
|
||||
|
||||
- [ ] should abra implement backup and restore or only provide an integration?
|
||||
- [ ] should we add a specification version?
|
||||
|
||||
## Next Meeting
|
||||
|
||||
* Who: ???
|
||||
@@ -1,57 +0,0 @@
|
||||
---
|
||||
title: "Resolution 021"
|
||||
---
|
||||
|
||||
- Topic: Budget 011: Migrate to Cobra
|
||||
- Date: 18-07-2024
|
||||
- Deadline: 31-07-2024
|
||||
- Size: Large
|
||||
|
||||
### Summary
|
||||
|
||||
Migrate away from our current command-line dependency so `abra` usage is more predictable. The goal is to maintain feature parity with no breaking changes. The main advantage that we will get is robust and flexible handling of flags/arguments which don't depend on forcing a specific order (see [`#581`](https://git.coopcloud.tech/coop-cloud/organising/issues/581)). There are other bonuses such as built-in support for auto-completion, better handling of example usage, improved support for global flags (`--debug`) and manpage support.
|
||||
|
||||
### Details (Budget 011)
|
||||
|
||||
#### The problem
|
||||
|
||||
The current help output of `abra app deploy` is as follows:
|
||||
|
||||
`abra app deploy [command options] <domain> [<version>]`
|
||||
|
||||
However, it is possible to do both of the following:
|
||||
|
||||
```
|
||||
abra app deploy --chaos example.org # "before" style
|
||||
abra app deploy example.org --chaos # "after" style
|
||||
```
|
||||
|
||||
However, `abra app cmd` is broken if you try to use the "after" style:
|
||||
|
||||
```
|
||||
abra app cmd <domain> <function> --local -- <args>
|
||||
```
|
||||
|
||||
This results in `<recipe> doesn't have a --local function` which is a bug in the `abra` code. It tries to read the position of the arguments but `--local` is included as an argument. The bug in `abra` is due to a bug in `urfave/cli` - "after" style options appear as arguments 😱
|
||||
|
||||
The only way to use `abra app cmd` right now is using the "before" style:
|
||||
|
||||
```
|
||||
abra app cmd --local <domain> <function> -- <args>
|
||||
```
|
||||
|
||||
This means that some commands allow both "after" and "before" style and some only allow "before" style. This is a source of confusion, raised issues and frustration.
|
||||
|
||||
#### The solution
|
||||
|
||||
[Several](https://git.coopcloud.tech/coop-cloud/abra/pulls/404) [attempts](https://git.coopcloud.tech/coop-cloud/abra/pulls/435) have been made to upgrade `urfave/cli` to fix this behaviour. However, as it turns out, it is **highly unlikely** that they will fix this upstream: [`urfave/cli#1950`](https://github.com/urfave/cli/issues/1950) [`urfave/cli#1928`](https://github.com/urfave/cli/pull/1928) (and even this proposal does not really include the desired robust flexible handling we need).
|
||||
|
||||
`@decentral1se` has done a spike to confirm that [`cobra`](https://cobra.dev) handles flexible handling of arguments/flags. Those reading this proposal and wishing to try it out for themselves can take [Hugo](https://gohugo.io/) for a spin (it uses `cobra` as the underlying command-line library).
|
||||
|
||||
This tool is well maintained and used by several large projects such as Hugo and Kubernetes. The library matches all functionality we require.
|
||||
|
||||
#### Budget
|
||||
|
||||
`@decentral1se` can carry out this work.
|
||||
|
||||
Proposed budget of 15 hrs: `15 hrs * 20 = 300 EUR`
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: "Resolution 020"
|
||||
title: "Resolution 022"
|
||||
---
|
||||
|
||||
- Topic: Budget 10: Abra integration suite automation
|
||||
@@ -480,23 +480,3 @@ abra app secret insert localhost ssl_cert v1 "$(cat localhost.crt)"
|
||||
abra app secret insert localhost ssl_key v1 "$(cat localhost.key)"
|
||||
```
|
||||
4. Re-deploy `traefik` with `--force` and voila!
|
||||
|
||||
## Remote recipes
|
||||
|
||||
!!! warning "This is only available in the currently unreleased version of `abra`"
|
||||
|
||||
Please see [this issue](https://git.coopcloud.tech/coop-cloud/organising/issues/583) to track current progress towards a release. All feedback and testing are welcome on this new feature. The design is not finalised yet.
|
||||
|
||||
It is possible to specify a remote recipe in your `.env` file:
|
||||
|
||||
```
|
||||
RECIPE=mygit.org/myorg/cool-recipe.git:1.3.12
|
||||
```
|
||||
|
||||
Where `1.3.12` is an optional pinned version. When `abra` runs a deployment, it
|
||||
will fetch the remote recipe and create a directory for it under `$ABRA_DIR`
|
||||
(typically `~/.abra`):
|
||||
|
||||
```
|
||||
$ABRA_DIR/recipes/mygit_org_myorg_cool-recipe
|
||||
```
|
||||
|
||||
+23
-41
@@ -13,6 +13,13 @@ In order to deploy an app you need two things:
|
||||
|
||||
This tutorial tries to help you make choices about which server and which DNS setup you need to run a _Co-op Cloud_ deployment but it does not go into great depth about how to set up a new server.
|
||||
|
||||
??? question "Can `abra` help automate this?"
|
||||
|
||||
Our `abra` tool can help bootstrap new servers & configure DNS records for
|
||||
you. We'll skip that for now since we're just getting started. For more on
|
||||
these topics after you finish the tutorial see the [operators
|
||||
handbook](/operators/handbook).
|
||||
|
||||
### Server setup
|
||||
|
||||
Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see [this FAQ entry](/intro/faq/#isnt-running-everything-in-containers-inefficient)).
|
||||
@@ -32,25 +39,15 @@ You need to keep port `:80` and `:443` free on your server for web proxying to y
|
||||
Docker.
|
||||
|
||||
```
|
||||
# ssh into your server
|
||||
ssh <server-domain>
|
||||
|
||||
# docker install convenience script
|
||||
wget -O- https://get.docker.com | bash
|
||||
|
||||
# add user to docker group
|
||||
sudo usermod -aG docker $USER
|
||||
|
||||
# exit and re-login to load the group
|
||||
exit
|
||||
ssh <server-domain>
|
||||
|
||||
# back on the server, setup swarm
|
||||
# setup swarm
|
||||
docker swarm init
|
||||
docker network create -d overlay proxy
|
||||
|
||||
# now you can exit and start using abra
|
||||
exit
|
||||
```
|
||||
|
||||
??? question "Do you support multiple web proxies?"
|
||||
@@ -92,8 +89,7 @@ abra -h
|
||||
```
|
||||
|
||||
You may need to add the `~/.local/bin/` directory to your `$PATH` variable, in
|
||||
order to run the executable. Also, run this line into your terminal so
|
||||
you have immediate access to `abra` on the current terminal.
|
||||
order to run the executable.
|
||||
|
||||
```bash
|
||||
export PATH=$PATH:$HOME/.local/bin
|
||||
@@ -107,38 +103,14 @@ If you run into issues during installation, [please report a ticket](https://git
|
||||
|
||||
### Add your server
|
||||
|
||||
Now you can connect `abra` with your server. You must have a working SSH configuration for your server before you can proceed. That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:. See the [`abra` SSH troubleshooting](/abra/trouble/#ssh-connection-issues) for a working SSH configuration example.
|
||||
|
||||
??? warning "Beware of SSH dragons :dragon_face:"
|
||||
|
||||
Under the hood `abra` uses plain 'ol `ssh` and aims to make use of your
|
||||
existing SSH configurations in `~/.ssh/config` and interfaces with your
|
||||
running `ssh-agent` for password protected secret key files.
|
||||
|
||||
Running `server add` with `-d` or `--debug` should help you debug what is
|
||||
going on under the hood. `ssh -v ...` should also help. If you're running
|
||||
into SSH connection issues with `abra` take a moment to read [this
|
||||
troubleshooting entry](/abra/trouble/#ssh-connection-issues).
|
||||
Now you can connect `abra` with your server. You should have a working SSH configuration before you can do this (e.g. a matching `Host <server-domain>` entry in `~/.ssh/config` with the correct SSH connection details). That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:.
|
||||
|
||||
```bash
|
||||
ssh <server-domain> # make sure it works
|
||||
abra server add <server-domain>
|
||||
```
|
||||
|
||||
It is important to note that `<server-domain>` here is a publicy accessible domain name which points to your server IP address. `abra` does make sure this is the case and this is done to avoid issues with HTTPS certificate rate limiting.
|
||||
|
||||
??? warning "Can I use arbitrary server names?"
|
||||
|
||||
Yes, this is possible. You need to pass `-D` to `server add` and ensure
|
||||
that your `Host ...` entry in your SSH configuration includes the name.
|
||||
So, for example:
|
||||
|
||||
Host example.com example
|
||||
...
|
||||
|
||||
And then:
|
||||
|
||||
abra server add -D example
|
||||
It is important to note that `<domain>` here is a publicy accessible domain name which points to your server IP address. `abra` does make sure this is the case and this is done to avoid issues with HTTPS certificate rate limiting.
|
||||
|
||||
You will now have a new `~/.abra/` folder on your local file system which stores all the configuration of your Co-op Cloud instance.
|
||||
|
||||
@@ -148,10 +120,20 @@ By now `abra` should have registered this server as managed. To confirm this run
|
||||
abra server ls
|
||||
```
|
||||
|
||||
??? warning "Beware of SSH dragons :dragon_face:"
|
||||
|
||||
Under the hood `abra` uses plain 'ol `ssh` and aims to make use of your
|
||||
existing SSH configurations in `~/.ssh/config` and interfaces with your
|
||||
running `ssh-agent` for password protected secret key files.
|
||||
|
||||
Running `server add` with `-d` or `--debug` should help you debug what is going
|
||||
on under the hood. If you're running into SSH connection issues with `abra`
|
||||
take a moment to read [this troubleshooting
|
||||
entry](/abra/trouble/#ssh-connection-issues).
|
||||
|
||||
??? question "How do I share my configs in `~/.abra`?"
|
||||
|
||||
It's possible and quite easy, for more see [this handbook
|
||||
entry](/operators/handbook/#understanding-app-and-server-configuration).
|
||||
It's possible and quite easy, for more see [this handbook entry](/operators/handbook/#understanding-app-and-server-configuration).
|
||||
|
||||
### Web proxy setup
|
||||
|
||||
|
||||
+2
-4
@@ -127,17 +127,15 @@ nav:
|
||||
- federation/resolutions/passed/017.md
|
||||
- federation/resolutions/passed/018.md
|
||||
- federation/resolutions/passed/019.md
|
||||
- federation/resolutions/passed/020.md
|
||||
- "In Progress":
|
||||
- federation/resolutions/in-progress/013.md
|
||||
- federation/resolutions/in-progress/021.md
|
||||
- federation/resolutions/in-progress/022.md
|
||||
- "Minutes":
|
||||
- federation/minutes/index.md
|
||||
- "Recently":
|
||||
- federation/minutes/2024-04-17.md
|
||||
- federation/minutes/2024-03-29.md
|
||||
- "Archive":
|
||||
- federation/minutes/2024-02-01.md
|
||||
- "Archive":
|
||||
- federation/minutes/2022-03-03.md
|
||||
- federation/minutes/2023-05-03.md
|
||||
- "Digital Tools": federation/tools.md
|
||||
|
||||
Reference in New Issue
Block a user