Move uncategorised out of docs for now

uncategorised/coop-cloud/index.md

@@ -0,0 +1,26 @@
+# About Coop Cloud
+
+Coöp Cloud is our system for deploying and updating free software applications.
+
+- 🔧 [Our own Coöp Cloud app configurations](https://git.autonomic.zone/autonomic-cooperative/coop-cloud-apps)
+- 🍎 [Supported application list](https://codimd.autonomic.zone/49--AK0GQDWoxMq6-9ngTQ)
+- 👁 [Coöp cloud overview](https://codimd.autonomic.zone/_M81xUukTCiBK96pgyC3DQ#) (portal to other CodiMD pads, including marketing copy)
+
+We're currently using Coöp Cloud for some of our internal infrastructure:
+
+- git.autonomic.zone
+- drone.autonomic.zone
+- id.autonomic.zone
+- traefik.autonomic.zone
+
+(all running on [`autonomic-swarm`](/servers/autonomic-swarm.md)), plus as many clients as we can, starting with:
+
+- drone.neuronic-swarm.autonomic.zone (for [Neuronic Games](/clients/neuronic-games), running on [`neuronic-swarm`](/servers/neuronic-swarm.md))
+- tankie.wiki (for Rebellious Data), running on `autonomic-swarm`
+- wiki.jones.iww.org.uk (for IWW wiki), running on `iww-jones`
+
+Relevant HOWTOs:
+
+- [Set up a new Docker swarm box](newswarm.md)
+- [Working with Docker swarm](working.md)
+- [Logging with systemd-journald](logging.md)

uncategorised/coop-cloud/logging.md

@@ -0,0 +1,48 @@
+# Logging with systemd-journald
+
+The default Docker swarm logging driver is `json-file`, which is nice but
+whenever a container is kiled or replaced, the logs are lost. This doesn't help
+us when we want to look back and try to diagnose what hapened.
+
+So, we change the default logging driver to use the systemd based journal
+logging system. This is configured in the `/etc/systemd/journal.conf` and in
+the `/etc/docker/daemon.json`. In practice, it means that logs are persistently
+stored after containers go away, they are rotated and can be analysed later on.
+
+`/etc/docker/daemon.json`:
+
+```
+{
+    "log-driver": "journald",
+    "log-opts": {
+      "labels":"com.docker.swarm.service.name"
+    }
+}
+```
+
+`/etc/systemd/journal.conf`:
+
+```
+[Journal]
+Storage=persistent
+SystemMaxUse=5G
+MaxFileSec=1month
+
+```
+
+There is also the [official docker
+documentation](https://docs.docker.com/config/containers/logging/journald/) on
+the journald logging driver.
+
+Some useful commands:
+
+- `journalctl -f`
+- `journalctl CONTAINER_NAME=gitea_gitea.1.jxn9r85el63pdz42ykjnmh792 -f`
+- `journalctl COM_DOCKER_SWARM_SERVICE_NAME=gitea_gitea --since="2020-09-18 13:00:00" --until="2020-09-18 13:01:00"`
+- `journalctl CONTAINER_ID=$(docker ps -qf name=gitea_gitea) -f`
+
+Also, for more system wide analysis stuff:
+
+- `journalctl --disk-usage`
+- `du -sh /var/log/journal/*`
+- `man journalctl` / `man systemd-journald` / `man journald.conf`

uncategorised/coop-cloud/newswarm.md

@@ -0,0 +1,19 @@
+# Setting up a new Docker swarm box
+
+Create and provision a new VPS with Docker installed.
+
+The easiest way of doing this is using [`actions/newhetzner.yml` in our `infrastructure` repository](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/actions/newhetzner.yml).
+
+Make sure you have `infrastructure` cloned and set up [according to the instructions](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/README.md), then run `ansible-playbook actions/newhetzner.yml` and answer the questions.
+
+1. Add the server to your `~/.ssh/config` file (you'll at least need to specify `Port 222`).
+2. Add the suggested entry to [`inventories/inventory`](https://git.autonomic.zone/autonomic-cooperative/infrastructure/src/branch/master/inventories)
+3. Create a new `servers/<client>/<server>.yml` file, based on `servers/neuronic/swarm.yml` (including the `swarm.single-node` role)
+4. Run `ansible-playbook servers/<client>/<server>.yml` to install Docker
+
+The easiest way forwards from here is to install `abra` and use it to set up remote context and initialise the swarm:
+
+1. `curl -fsSL https://install.abra.autonomic.zone | bash`
+2. `abra context create swarm.client.tld yourusername 222`
+3. `abra context init swarm.client.tld`
+4. `abra context use swarm.client.tld`

uncategorised/coop-cloud/working.md

@@ -0,0 +1,97 @@
+# Working with Docker Swarm
+
+## Set up remote context
+
+You can use `docker context` to run Docker command-line commands and have them
+point to the Docker API end-point on a remote host.
+
+This means you can run commands locally and control the remote swarm easily
+(e.g. you run `docker ps` and instead of seeing container on your `localhost`
+you see them on `swarm.autonomic.zone`). This allows to do remote deployments
+manually, filter logs, clean-up containers etc.
+
+!!! note "This is optional!"
+If you like, you can SSH to a swarm server, [install `docker-compose`](https://docs.docker.com/compose/install/#install-compose-on-linux-systems), and run normal Docker commands instead.
+
+Here are the 3 steps to set this up.
+
+1. Create the remote docker context locally.
+
+   ```bash
+   # .envrc.sample
+   export PASSWORD_STORE_DIR=$(pwd)/../infrastructure/credentials/password-store
+   ```
+
+   ```bash
+   $ cp .envrc.sample .envrc
+   $ direnv allow  # ensure password store works
+   $ mkdir -vp ~/.docker/swarm.autonomic.zone && \
+     pass show docker/swarm.autonomic.zone/ca.pem > ~/.docker/swarm.autonomic.zone/ca.pem && \
+     pass show docker/swarm.autonomic.zone/cert.pem > ~/.docker/swarm.autonomic.zone/cert.pem && \
+     pass show docker/swarm.autonomic.zone/key.pem > ~/.docker/swarm.autonomic.zone/key.pem
+   $ docker context create swarm.autonomic.zone --docker \
+     "host=tcp://swarm.autonomic.zone:2376,ca=$HOME/.docker/swarm.autonomic.zone/ca.pem,cert=$HOME/.docker/swarm.autonomic.zone/cert.pem,key=$HOME/.docker/swarm.autonomic.zone/key.pem"
+   $ docker context use swarm.autonomic.zone
+   ```
+
+2. Deploy the application to the remote docker context.
+
+(Assuming you're in, say, the [git.autonomic.zone](https://git.autonomic.zone/autonomic-cooperative/git.autonomic.zone) repository)
+
+```bash
+$ docker stack ls
+$ docker stack deploy -c compose.yml gitea
+```
+
+You can track logs via `docker service logs gitea_gitea`.
+
+3. Switch back to your local context.
+
+```
+$ docker context use default
+```
+
+## Useful concepts & commands
+
+Each app is a **stack**, e.g. `drone` (`docker stack ls`), which creates one or
+more **services**, e.g. `drone_drone` (`docker service ls`), each of which has one or more
+**containers** e.g. `drone_drone.1.czq919syweq23x07whj38pb96` (`docker container ls`). All of this is defined in a `docker-compose.yml` file.
+
+Containers are built from **images**, e.g. `nginx:stable`, optionally using a
+`Dockerfile` to add extra commands or resources.
+
+### Secrets
+
+Most apps will need secret values (like API keys), which Docker can store securely using `docker secret`.
+
+As a failsafe, and to help debugging, we also store secrets in `pass`.
+
+You can generate a password, store it to Docker, and save it to `pass` in one
+step using something like this:
+
+```
+pwgen -n 32 1 | tee \
+  >(docker secret create "APP_SECRET_v1" -)
+  >(pass insert -m hosts/HOSTNAME/APP/SECRET)
+```
+
+Use `docker secrets ls` to see the names of all secrets defined in the current
+context, and `docker secrets rm <NAME>` to remove one if you need to reset it.
+
+## Troubleshooting
+
+If a service is trying to start, but you don't see anything in `docker service logs ...`, then try `docker service ps --no-trunc`, which will show you errors
+during container initialisation.
+
+If you still don't see anything there, log into the swarm server and check the
+Docker logs:
+
+```
+sudo journalctl -u docker.service | tail -n 50
+```
+
+## Investigating persistent journald logs
+
+See [systemd-journald
+docs](https://docs.autonomic.zone/coop-cloud/logging-with-systemd-journald/)
+for more information on the systemd journal logging setup.

uncategorised/coop-cloud/wp.md

@@ -0,0 +1,12 @@
+# Running wp-cli commands
+
+Here is an example how to drop into a shell and run `wp` commands. Just change the relevant details for your project.
+
+```bash
+export CONTAINER_ID=$(DOCKER_CONTEXT=swarm.autonomic.zone docker container ls -f 'Name=boycott-turkey_net_app' --format '{{ .ID }}'); DOCKER_CONTEXT=swarm.autonomic.zone docker run -it --volumes-from "$CONTAINER_ID" --network "container:$CONTAINER_ID" wordpress:cli
+```
+
+```bash
+export CONTAINER_ID=$(DOCKER_CONTEXT=swarm.autonomic.zone docker container ls -f 'Name=boycott-turkey_net_app' --format '{{ .ID }}'); DOCKER_CONTEXT=swarm.autonomic.zone docker run -it --volumes-from "$CONTAINER_ID" --network "container:$CONTAINER_ID" wordpress:cli user create
+usage: wp user create <user-login> <user-email> [--role=<role>] [--user_pass=<password>] [--user_registered=<yyyy-mm-dd-hh-ii-ss>] [--display_name=<name>] [--user_nicename=<nice_name>] [--user_url=<url>] [--nickname=<nickname>] [--first_name=<first_name>] [--last_name=<last_name>] [--description=<description>] [--rich_editing=<rich_editing>] [--send-email] [--porcelain]
+```