daniel/docs.coopcloud.tech
0
0
Fork 0
forked from toolshed/docs.coopcloud.tech
Code Pull Requests Activity

Fiddle with folder names

Browse Source
This commit is contained in:
3wc
2024-12-19 08:12:47 -05:00
parent c648f67cef
commit 11764163d1
9 changed files with 1586 additions and 1586 deletions
Download Patch File Download Diff File Expand all files Collapse all files

307
docs/hosters/tutorial.md
View File

@ -1,94 +1,279 @@
---
title: New maintainers tutorial
title: New Operators Tutorial
---
## Package your first recipe
This tutorial assumes you understand the [frequently asked questions](/intro/faq/) as well as [the moving parts](/intro/strategy/) of the technical problems _Co-op Cloud_ solves. If yes, proceed :smile:
### Overview
## Deploy your first app
Packaging a recipe is basically knowing a bag of about 20 tricks. Once you learn them, there is nothing more to learn. It can seem daunting at first but it's simple and easy to do once you know the tricks.
In order to deploy an app you need two things:
The nice thing about packaging is that only one person has to do it and then we all benefit. We've seen that over time, the core of the configuration doesn't really change. New options and versions might come but the config remains quite stable. This is good since it means that your packaging work stays relevant and useful for other maintainers & operators as time goes on.
1. a server with SSH access and a public IP address
2. a domain name pointing to that server
Depending on your familiarity with recipes, it might be worth reading [how a recipe is structured](/maintainers/handbook/#how-is-a-recipe-structured) and making clear you understand [what a recipe is](/glossary/#recipe) before continuing.
This tutorial tries to help you make choices about which server and which DNS setup you need to run a _Co-op Cloud_ deployment but it does not go into great depth about how to set up a new server.
### Making a plan
### Server setup
The ideal scenario is when the upstream project provides both the packaged image and a compose configuration which we can build from. If you're in luck, you'll typically find a `Dockerfile` and a `docker-compose.yml` file in the root of the upstream Git repository for the app.
Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see [this FAQ entry](/intro/faq/#isnt-running-everything-in-containers-inefficient)).
- **Tired**: Write your own image and compose file from scratch :sleeping:
- **Wired**: Use someone else's image (& maybe compose file) :smirk_cat:
- **Inspired**: Upstream image, someone else's compose file :exploding_head:
- **On fire**: Upstream image, upstream compose file :fire:
We will deploy a new Nextcloud instance in this guide, so you will only need 1GB of RAM according to [their documentation](https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html). You may also be interested in this [FAQ entry](/intro/faq/#arent-containers-horrible-from-a-security-perspective) if you are curious about security in the context of containers.
### Writing / adapting the `compose.yml`
Most Co-op Cloud deployments have been run on Debian machines so far. Some experiments have been done on single board computers & servers with low resource capacities.
Let's take a practical example, [Matomo web analytics](https://matomo.org/). We'll be making a Docker "swarm-mode" `compose.yml` file.
You need to keep port `:80` and `:443` free on your server for web proxying to your apps. Typically, you don't need to keep any other ports free as the core web proxy ([Traefik](https://traefik.io)) keeps all app ports internal to its network. Sometimes however, you need to expose an app port when you need to use a transport which would perform better or more reliably without proxying.
Luckily, Matomo already has an example compose file in their repository. Like a lot of compose files, it's intended for use with `docker-compose`, instead of "swarm mode", but it should be a good start.
`abra` has support for creating servers (`abra server new`) but that is a more advanced automation feature which is covered in the [handbook](/operators/handbook). For this tutorial, we'll focus on the basics. Assuming you've managed to create a testing VPS with some `$hosting_provider`, you'll need to install Docker, add your user to the Docker group & setup swarm mode:
First, let's create a directory with the files we need:
!!! warning "You may need to log in/out"
When running `usermod ...`, you may need to (depending on your system) log
in and out again of your shell session to get the required permissions for
Docker.
```
abra recipe new matomo
cd ~/.abra/recipes/matomo
# ssh into your server
ssh <server-domain>
# docker install convenience script
wget -O- https://get.docker.com | bash
# add user to docker group
sudo usermod -aG docker $USER
# exit and re-login to load the group
exit
ssh <server-domain>
# back on the server, setup swarm
docker swarm init
docker network create -d overlay proxy
# now you can exit and start using abra
exit
```
Then, let's download and edit the `docker-compose.yml` file:
??? question "Do you support multiple web proxies?"
```
mkdir matomo && cd matomo
wget https://raw.githubusercontent.com/matomo-org/docker/master/.examples/apache/docker-compose.yml -O compose.yml
We do not know if it is feasible and convenient to set things up on an existing server with another web proxy which uses ports `:80` & `:443`. We'd happily receive reports and documentation on how to do this if you manage to set it up!
### DNS setup
You'll need two A records, one to point to the server itself and another to support sub-domains for the apps. You can then support an app hosted on your root domain (e.g. `example.com`) and other apps on sub-domains (e.g. `foo.example.com`, `bar.example.com`).
Your entries in your DNS provider setup might look like the following.
@ 1800 IN A 116.203.211.204
*. 1800 IN A 116.203.211.204
Where `116.203.211.204` can be replaced with the IP address of your server.
??? question "How do I know my DNS is working?"
You can use a tool like `dig` on the command-line to check if your server has the necessary DNS records set up. Something like `dig +short <domain>` should show the IP address of your server if things are working.
### Install `abra`
Now we can install [`abra`](/abra) locally on your machine and hook it up to
your server. We support a script-based installation method ([script source](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer)):
```bash
curl https://install.abra.coopcloud.tech | bash
```
Open the `compose.yml` in your favourite editor and have a gander &#129442;. There are a few things we're looking for, but some immediate changes could be:
The installer will verify the downloaded binary checksum. If you prefer, you can
[manually verify](/abra/install/#manual-verification) the binary, and then
manally place it in one the directories in your `$PATH` variable. To validate
that everything is working try listing the `--help` command or `-h` to view
output:
1. Let's bump the version to `3.8`, to make sure we can use all the latest swarm coolness.
2. We load environment variables separately via [`abra`](/abra/), so we'll strip out `env_file`.
3. The `/var/www/html` volume definition on L21 is a bit overzealous; it means a copy of Matomo will be stored separately per app instance, which is a waste of space in most cases. We'll narrow it down according to the documentation. The developers have been nice enough to suggest `logs` and `config` volumes instead, which is a decent start.
4. The MySQL passwords are sent as variables which is fine for basic use, but if we replace them with Docker secrets we can keep them out of our env files if we want to publish those more widely.
5. The MariaDB service doesn't need to be exposed to the internet, so we can define an `internal` network for it to communicate with Matomo.
6. Lastly, we want to use `deploy.labels` and remove the `ports:` definition, to tell Traefik to forward requests to Matomo based on hostname and generate an SSL certificate.
The resulting `compose.yml` is available [here](https://git.autonomic.zone/coop-cloud/matomo/src/branch/main/compose.yml).
### Updating the `.env.sample`
Open the `.env.sample` file and add the following
```
DB_PASSWORD_VERSION=v1
DB_ROOT_PASSWORD_VERSION=v1
```bash
abra -h
```
The resulting `.env.sample` is available [here](https://git.coopcloud.tech/coop-cloud/matomo/src/branch/main/.env.sample)
You may need to add the `~/.local/bin/` directory to your `$PATH` variable, in
order to run the executable. Also, run this line into your terminal so
you have immediate access to `abra` on the current terminal.
### Test deployment
!!! note "Running Co-op Cloud server required!"
The rest of this guide assumes you have a Co-op Cloud server going -- we'll use `swarm.example.com`, but replace it with your own server address. Head over to [the operators tutorial](/operators/tutorial) if you need help setting one up.
Now, we're ready to create a testing instance of Matomo:
```
abra app new matomo --secrets \
--domain matomo.swarm.example.com \
--server swarm.example.com
```bash
export PATH=$PATH:$HOME/.local/bin
```
Depending on whether you defined any extra environment variables -- we didn't so
far, in this example -- you might want to run `abra app config swarm.example.com`
to check the configuration.
If you run into issues during installation, [please report a ticket](https://git.coopcloud.tech/coop-cloud/organising/issues/new) :pray: Once you're all set up, we **highly** recommend configuring command-line auto-completion for `abra`. See `abra autocomplete -h` for more on how to do this.
Otherwise, or once you've done that, go ahead and deploy the app:
??? question "Can I install `abra` on my server?"
```
abra app deploy swarm.example.com
Yes, this is possible. However, the instructions for this setup are different. For more info see [this handbook entry](/operators/handbook/#running-abra-server-side).
### Add your server
Now you can connect `abra` with your server. You must have a working SSH configuration for your server before you can proceed. That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:. See the [`abra` SSH troubleshooting](/abra/trouble/#ssh-connection-issues) for a working SSH configuration example.
??? warning "Beware of SSH dragons :dragon_face:"
Under the hood `abra` uses plain 'ol `ssh` and aims to make use of your
existing SSH configurations in `~/.ssh/config` and interfaces with your
running `ssh-agent` for password protected secret key files.
Running `server add` with `-d` or `--debug` should help you debug what is
going on under the hood. `ssh -v ...` should also help. If you're running
into SSH connection issues with `abra` take a moment to read [this
troubleshooting entry](/abra/trouble/#ssh-connection-issues).
```bash
ssh <server-domain> # make sure it works
abra server add <server-domain>
```
Then, open the `DOMAIN` you configured (you might need to wait a while for Traefik to generate SSL certificates) to finish the set-up. Luckily, this container is (mostly) configurable via environment variables, if we want to auto-generate the configuration we can use a `config` and / or a custom `entrypoint` (see [`coop-cloud/mediawiki`](https://git.autonomic.zone/coop-cloud/mediawiki) for examples of both).
It is important to note that `<server-domain>` here is a publicy accessible domain name which points to your server IP address. `abra` does make sure this is the case and this is done to avoid issues with HTTPS certificate rate limiting.
### Finishing up
??? warning "Can I use arbitrary server names?"
You've probably got more questions, check out the [packaging handbook](/maintainers/handbook)!
Yes, this is possible. You need to pass `-D` to `server add` and ensure
that your `Host ...` entry in your SSH configuration includes the name.
So, for example:
Host example.com example
...
And then:
abra server add -D example
You will now have a new `~/.abra/` folder on your local file system which stores all the configuration of your Co-op Cloud instance.
By now `abra` should have registered this server as managed. To confirm this run:
```
abra server ls
```
??? question "How do I share my configs in `~/.abra`?"
It's possible and quite easy, for more see [this handbook
entry](/operators/handbook/#understanding-app-and-server-configuration).
### Web proxy setup
In order to have your Co-op cloud deployment serve the public internet, we need to install the core web proxy, [Traefik](https://doc.traefik.io/traefik/).
Traefik is the main entrypoint for all web requests (e.g. like NGINX) and
supports automatic SSL certificate configuration and other quality-of-life
features which make deploying libre apps more enjoyable.
**1. To get started, you'll need to create a new app:**
```bash
abra app new traefik
```
Choose your newly registered server and specify a domain name. By default `abra`
will suggest `<app-name>.server.org` or prompt you with a list of servers.
**2. Configure this new `traefix` app**
You will want to take a look at your generated configuration and tweak the `LETS_ENCRYPT_EMAIL` value. You can do that by running `abra app config`:
```bash
abra app config <traefik-domain>
```
Every app you deploy will have one of these `.env` files, which contains
variables which will be injected into app configurations when deployed. These
files exist at relevantly named path:
```bash
~/.abra/servers/<domain>/<traefik-domain>.env
```
Variables starting with `#` are optional, others are required. Some things to
consider here is that by default our *Traefik* recipe exposes the metric
dashboard unauthenticated on the public internet at the URL `<traefik-domain>`
it is deployed to, which is not ideal. You can disable this with:
```
DASHBOARD_ENABLED=false
```
**3. Now it is time to deploy your app:**
```
abra app deploy <traefik-domain>
```
Voila. Abracadabra :magic_wand: your first app is deployed :sparkles:
### Deploy Nextcloud
And now we can deploy apps. Let's create a new Nextcloud app.
```bash
abra app new nextcloud -S
```
The `-S` or `--secrets` flag is used to generate secrets for the app: database connection password, root password and admin password.
??? warning "Beware of password dragons :dragon:"
Take care, these secrets are only shown once on the terminal so make sure to take note of them! `abra` makes use of the [Docker secrets](/operators/handbook/#managing-secret-data) mechanism to ship these secrets securely to the server and store them as encrypted data. Only the apps themselves have access to the values from here on, they're placed in `/run/secrets` on the container file system.
Then we can deploy Nextcloud:
```bash
abra app deploy <nextcloud-domain>
```
`abra app deploy` will wait nearly a minute for an app to deploy until it times out and shows some helpful commands for how to debug what is going on. If things don't come up in time, try running the following:
```
abra app ps -w <nextcloud-domain> # status check
abra app logs <nextcloud-domain> # logs trailing
abra app errors -w <nextcloud-domain> # error catcher
```
Your new `traefik` instance will detect that a new app is coming up and generate SSL certificates for it. You can see what `traefik` is up to using the same commands above but replacing `<netcloud-domain>` with the `<traefik-domain>` you chose earlier (`abra app ls` will remind you what domains you chose :grinning:).
### Upgrade Nextcloud
To upgrade an app manually to the newest available version run:
```bash
abra app upgrade <nextcloud-domain>
```
### Automatic Upgrades
`kadabra` the auto-updater is still under development, use it with care and don't use it in production environments. To setup the auto-updater copy the `kadabra` binary to the server and configure a cronjob for regular app upgrades. The following script will configure ssmtp for email notifications and setup a cronjob. This cronjob checks daily for new app versions, notifies if any kind of update is available and upgrades all apps to the latest patch/minor version.
```bash
apt install ssmtp
cat > /etc/ssmtp/ssmtp.conf << EOF
mailhub=$MAIL_SERVER:587
hostname=$MAIL_DOMAIN
AuthUser=$USER
AuthPass=$PASSWORD
FromLineOverride=yes
UseSTARTTLS=yes
EOF
cat > /etc/cron.d/abra_updater << EOF
MAILTO=admin@example.com
MAILFROM=noreply@example.com
0 6 * * * root ~/kadabra notify --major
30 4 * * * root ~/kadabra upgrade --all
EOF
```
Add `ENABLE_AUTO_UPDATE=true` to the env config (`abra app config <app name>`) to enable the auto-updater for a specific app.
## Finishing up
Hopefully you got something running! Well done! The [operators handbook](/operators/handbook) would probably be the next place to go check out if you're looking for more help. Especially on topics of ongoing maintenance.
If not, please [get in touch](/intro/contact) or [raise a ticket](https://git.coopcloud.tech/coop-cloud/organising/issues/new/choose) and we'll try to help out. We want our operator onboarding to be as smooth as possible, so we do appreciate any feedback we receive.