flancian/docs.coopcloud.tech
0
0
Fork 0
forked from toolshed/docs.coopcloud.tech
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Move coop cloud content under "uncategorised"

Browse Source
This commit is contained in:
Luke Murphy
2020-12-28 16:59:05 +01:00
parent 72f5a7c113
commit 3c18ba4514
10 changed files with 260 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
docs/index.md
View File

@ -2,55 +2,4 @@
title: Welcome
---
Coöp Cloud (working title; abbreviated CoCl) is a container-based, platform-agnostic, free software registry for small service providers.
- Production-ready apps in minutes: Wordpress & Nextcloud instances,
mail-servers, and more.
- Simple Docker-based framework for continuous deployment of your custom apps.
[List of CoCl apps](https://codimd.autonomic.zone/s/HyNtOhwrv){: .md-button }
HOWTOs:
- [Deploy an HTML site](howto/html-website.md)
- [CoCl-ise an app](howto/convert-app.md)
!!! warning "A note about ARM"
Not all applications currently support all ARM computers (like PINE64 and
Raspberry Pi): e.g. [Traefik will work on ARMv6 & ARM64][traefik], [the
official MariaDB app doesn't work on any ARM boards][mariadb]. It might be possible
to use third-party ARM images with separate `compose.yml` files, but we
haven't exlplored this rabbit-hole much yet.
## Definitions
CoCl is a **philosophy**.
- To get started, you create a **server**, e.g. a new VPS, including
your local configuration to connect to it...
- then you install an **app** like wordpress or nextcloud, which is made up of multiple **services**...
- .. and configure your app.
## Technical description
Software-wise, CoöpCloud is:
- [`coop-cloud`](https://git.autonomic.zone/coop-cloud/), a collection of Docker "swarm mode" configurations for popular web apps
- [`abra`](https://git.autonomic.zone/autonomic-cooperative/abra), a simple tool for Docker swarm management
- a recommended default set of stacks:
- Traefik for SSL & routing
- `postfix-relay` for outgoing email
## Principles / features:
- Security by default
- Secret storage using `docker secret` (["What makes it secure?"](https://github.com/BretFisher/ama/issues/86))
- Automatic SSL using Traefik & LetsEncrypt
- Zero-downtime deployments (for apps with healthchecks defined)
- Continuous integration testing using Drone and our [`stack-ssh-deploy`](https://git.autonomic.zone/coop-cloud/stack-ssh-deploy) plugin
- Low maintenance overhead:
- Automatic tracking of upstream Docker images using `renovate-bot`
- Avoiding custom Docker images as far as possible
[traefik]: https://hub.docker.com/_/traefik?tab=tags
[mariadb]: https://hub.docker.com/_/mariadb?tab=tags
W.I.P

0
docs/comparisons.md → docs/uncategorised/comparisons.md
View File

26
docs/uncategorised/coop-cloud/index.md Normal file
View File

@ -0,0 +1,26 @@
# About Coop Cloud
Coöp Cloud is our system for deploying and updating free software applications.
- 🔧 [Our own Coöp Cloud app configurations](https://git.autonomic.zone/autonomic-cooperative/coop-cloud-apps)
- 🍎 [Supported application list](https://codimd.autonomic.zone/49--AK0GQDWoxMq6-9ngTQ)
- 👁 [Coöp cloud overview](https://codimd.autonomic.zone/_M81xUukTCiBK96pgyC3DQ#) (portal to other CodiMD pads, including marketing copy)
We're currently using Coöp Cloud for some of our internal infrastructure:
- git.autonomic.zone
- drone.autonomic.zone
- id.autonomic.zone
- traefik.autonomic.zone
(all running on [`autonomic-swarm`](/servers/autonomic-swarm.md)), plus as many clients as we can, starting with:
- drone.neuronic-swarm.autonomic.zone (for [Neuronic Games](/clients/neuronic-games), running on [`neuronic-swarm`](/servers/neuronic-swarm.md))
- tankie.wiki (for Rebellious Data), running on `autonomic-swarm`
- wiki.jones.iww.org.uk (for IWW wiki), running on `iww-jones`
Relevant HOWTOs:
- [Set up a new Docker swarm box](setting-up-new-docker-swarm-box.md)
- [Working with Docker swarm](working-with-docker-swarm.md)
- [Logging with systemd-journald](logging-with-systemd-journald.md)

48
docs/uncategorised/coop-cloud/logging.md Normal file
View File

@ -0,0 +1,48 @@
# Logging with systemd-journald
The default Docker swarm logging driver is `json-file`, which is nice but
whenever a container is kiled or replaced, the logs are lost. This doesn't help
us when we want to look back and try to diagnose what hapened.
So, we change the default logging driver to use the systemd based journal
logging system. This is configured in the `/etc/systemd/journal.conf` and in
the `/etc/docker/daemon.json`. In practice, it means that logs are persistently
stored after containers go away, they are rotated and can be analysed later on.
`/etc/docker/daemon.json`:
```
{
"log-driver": "journald",
"log-opts": {
"labels":"com.docker.swarm.service.name"
}
}
```
`/etc/systemd/journal.conf`:
```
[Journal]
Storage=persistent
SystemMaxUse=5G
MaxFileSec=1month
```
There is also the [official docker
documentation](https://docs.docker.com/config/containers/logging/journald/) on
the journald logging driver.
Some useful commands:
- `journalctl -f`
- `journalctl CONTAINER_NAME=gitea_gitea.1.jxn9r85el63pdz42ykjnmh792 -f`
- `journalctl COM_DOCKER_SWARM_SERVICE_NAME=gitea_gitea --since="2020-09-18 13:00:00" --until="2020-09-18 13:01:00"`
- `journalctl CONTAINER_ID=$(docker ps -qf name=gitea_gitea) -f`
Also, for more system wide analysis stuff:
- `journalctl --disk-usage`
- `du -sh /var/log/journal/*`
- `man journalctl` / `man systemd-journald` / `man journald.conf`

19
docs/uncategorised/coop-cloud/newswarm.md Normal file
View File

@ -0,0 +1,19 @@
# Setting up a new Docker swarm box
Create and provision a new VPS with Docker installed.
The easiest way of doing this is using [`actions/newhetzner.yml` in our `infrastructure` repository](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/actions/newhetzner.yml).
Make sure you have `infrastructure` cloned and set up [according to the instructions](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/README.md), then run `ansible-playbook actions/newhetzner.yml` and answer the questions.
1. Add the server to your `~/.ssh/config` file (you'll at least need to specify `Port 222`).
2. Add the suggested entry to [`inventories/inventory`](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/inventories)
3. Create a new `servers/<client>/<server>.yml` file, based on `servers/neuronic/swarm.yml` (including the `swarm.single-node` role)
4. Run `ansible-playbook servers/<client>/<server>.yml` to install Docker
The easiest way forwards from here is to install `abra` and use it to set up remote context and initialise the swarm:
1. `curl -fsSL https://install.abra.autonomic.zone | bash`
2. `abra context create swarm.client.tld yourusername 222`
3. `abra context init swarm.client.tld`
4. `abra context use swarm.client.tld`

97
docs/uncategorised/coop-cloud/working.md Normal file
View File

@ -0,0 +1,97 @@
# Working with Docker Swarm
## Set up remote context
You can use `docker context` to run Docker command-line commands and have them
point to the Docker API end-point on a remote host.
This means you can run commands locally and control the remote swarm easily
(e.g. you run `docker ps` and instead of seeing container on your `localhost`
you see them on `swarm.autonomic.zone`). This allows to do remote deployments
manually, filter logs, clean-up containers etc.
!!! note "This is optional!"
If you like, you can SSH to a swarm server, [install `docker-compose`](https://docs.docker.com/compose/install/#install-compose-on-linux-systems), and run normal Docker commands instead.
Here are the 3 steps to set this up.
1. Create the remote docker context locally.
```bash
# .envrc.sample
export PASSWORD_STORE_DIR=$(pwd)/../infrastructure/credentials/password-store
```
```bash
$ cp .envrc.sample .envrc
$ direnv allow # ensure password store works
$ mkdir -vp ~/.docker/swarm.autonomic.zone && \
pass show docker/swarm.autonomic.zone/ca.pem > ~/.docker/swarm.autonomic.zone/ca.pem && \
pass show docker/swarm.autonomic.zone/cert.pem > ~/.docker/swarm.autonomic.zone/cert.pem && \
pass show docker/swarm.autonomic.zone/key.pem > ~/.docker/swarm.autonomic.zone/key.pem
$ docker context create swarm.autonomic.zone --docker \
"host=tcp://swarm.autonomic.zone:2376,ca=$HOME/.docker/swarm.autonomic.zone/ca.pem,cert=$HOME/.docker/swarm.autonomic.zone/cert.pem,key=$HOME/.docker/swarm.autonomic.zone/key.pem"
$ docker context use swarm.autonomic.zone
```
2. Deploy the application to the remote docker context.
(Assuming you're in, say, the [git.autonomic.zone](https://git.autonomic.zone/autonomic-cooperative/git.autonomic.zone) repository)
```bash
$ docker stack ls
$ docker stack deploy -c compose.yml gitea
```
You can track logs via `docker service logs gitea_gitea`.
3. Switch back to your local context.
```
$ docker context use default
```
## Useful concepts & commands
Each app is a **stack**, e.g. `drone` (`docker stack ls`), which creates one or
more **services**, e.g. `drone_drone` (`docker service ls`), each of which has one or more
**containers** e.g. `drone_drone.1.czq919syweq23x07whj38pb96` (`docker container ls`). All of this is defined in a `docker-compose.yml` file.
Containers are built from **images**, e.g. `nginx:stable`, optionally using a
`Dockerfile` to add extra commands or resources.
### Secrets
Most apps will need secret values (like API keys), which Docker can store securely using `docker secret`.
As a failsafe, and to help debugging, we also store secrets in `pass`.
You can generate a password, store it to Docker, and save it to `pass` in one
step using something like this:
```
pwgen -n 32 1 | tee \
>(docker secret create "APP_SECRET_v1" -)
>(pass insert -m hosts/HOSTNAME/APP/SECRET)
```
Use `docker secrets ls` to see the names of all secrets defined in the current
context, and `docker secrets rm <NAME>` to remove one if you need to reset it.
## Troubleshooting
If a service is trying to start, but you don't see anything in `docker service logs ...`, then try `docker service ps --no-trunc`, which will show you errors
during container initialisation.
If you still don't see anything there, log into the swarm server and check the
Docker logs:
```
sudo journalctl -u docker.service | tail -n 50
```
## Investigating persistent journald logs
See [systemd-journald
docs](https://docs.autonomic.zone/coop-cloud/logging-with-systemd-journald/)
for more information on the systemd journal logging setup.

12
docs/uncategorised/coop-cloud/wp.md Normal file
View File

@ -0,0 +1,12 @@
# Running wp-cli commands
Here is an example how to drop into a shell and run `wp` commands. Just change the relevant details for your project.
```bash
export CONTAINER_ID=$(DOCKER_CONTEXT=swarm.autonomic.zone docker container ls -f 'Name=boycott-turkey_net_app' --format '{{ .ID }}'); DOCKER_CONTEXT=swarm.autonomic.zone docker run -it --volumes-from "$CONTAINER_ID" --network "container:$CONTAINER_ID" wordpress:cli
```
```bash
export CONTAINER_ID=$(DOCKER_CONTEXT=swarm.autonomic.zone docker container ls -f 'Name=boycott-turkey_net_app' --format '{{ .ID }}'); DOCKER_CONTEXT=swarm.autonomic.zone docker run -it --volumes-from "$CONTAINER_ID" --network "container:$CONTAINER_ID" wordpress:cli user create
usage: wp user create <user-login> <user-email> [--role=<role>] [--user_pass=<password>] [--user_registered=<yyyy-mm-dd-hh-ii-ss>] [--display_name=<name>] [--user_nicename=<nice_name>] [--user_url=<url>] [--nickname=<nickname>] [--first_name=<first_name>] [--last_name=<last_name>] [--description=<description>] [--rich_editing=<rich_editing>] [--send-email] [--porcelain]
```

0
docs/howto/convert-app.md → docs/uncategorised/howto/convert-app.md
View File

0
docs/howto/html-website.md → docs/uncategorised/howto/html-website.md
View File

57
docs/uncategorised/index.md Normal file
View File

@ -0,0 +1,57 @@
---
title: Welcome
---
Coöp Cloud (working title; abbreviated CoCl) is a container-based, platform-agnostic, free software registry for small service providers.
- Production-ready apps in minutes: Wordpress & Nextcloud instances,
mail-servers, and more.
- Simple Docker-based framework for continuous deployment of your custom apps.
[List of CoCl apps](https://codimd.autonomic.zone/s/HyNtOhwrv){: .md-button }
HOWTOs:
- [Deploy an HTML site](howto/html-website.md)
- [CoCl-ise an app](howto/convert-app.md)
!!! warning "A note about ARM"
Not all applications currently support all ARM computers (like PINE64 and
Raspberry Pi): e.g. [Traefik will work on ARMv6 & ARM64][traefik], [the
official MariaDB app doesn't work on any ARM boards][mariadb]. It might be possible
to use third-party ARM images with separate `compose.yml` files, but we
haven't exlplored this rabbit-hole much yet.
## Definitions
CoCl is a **philosophy**.
- To get started, you create a **server**, e.g. a new VPS, including
your local configuration to connect to it...
- then you install an **app** like wordpress or nextcloud, which is made up of multiple **services**...
- .. and configure your app.
## Technical description
Software-wise, CoöpCloud is:
- [`coop-cloud`](https://git.autonomic.zone/coop-cloud/), a collection of Docker "swarm mode" configurations for popular web apps
- [`abra`](https://git.autonomic.zone/autonomic-cooperative/abra), a simple tool for Docker swarm management
- a recommended default set of stacks:
- Traefik for SSL & routing
- `postfix-relay` for outgoing email
## Principles / features:
- Security by default
- Secret storage using `docker secret` (["What makes it secure?"](https://github.com/BretFisher/ama/issues/86))
- Automatic SSL using Traefik & LetsEncrypt
- Zero-downtime deployments (for apps with healthchecks defined)
- Continuous integration testing using Drone and our [`stack-ssh-deploy`](https://git.autonomic.zone/coop-cloud/stack-ssh-deploy) plugin
- Low maintenance overhead:
- Automatic tracking of upstream Docker images using `renovate-bot`
- Avoiding custom Docker images as far as possible
[traefik]: https://hub.docker.com/_/traefik?tab=tags
[mariadb]: https://hub.docker.com/_/mariadb?tab=tags