11 KiB
| title |
|---|
| New operators tutorial |
The moving parts
Co-op Cloud is made up of a few simple, composable pieces. The system does not rely on any one specific implementation: each part may be replaced and/or extended as needed.
We want to build a reliable and long-term sustainable project and that means allowing for different implementations, open formats and a diverse project organisation.
Here are the main technical concepts listed below, once you grok this, you grok the moving parts of the project.
Libre software apps
Libre software apps are tools, websites & clients that you may already use in your daily life: Nextcloud, Jitsi, Mediawiki, Rocket.chat and many more! These are tools that are created by volunteer communities who use free software licenses in order to build up the public software commons and offer more digital alternatives to proprietary systems.
The communities who develop these softwares also publish them using containers. For example, here is the Nextcloud hub.docker.com account which allows end-users to quickly deploy a new Nextcloud instance. There is a growing consensus in the free software community that containers are a useful and time saving format for distribution.
Learn more about why we use containers in the FAQ section.
The recipe packaging format
However, just having a container of an app is often not enough. The work required to deploy that app in a "production ready" setup is still too time intensive and often involves a duplication of effort. Each service provider needs to deal with the same problems: stable versioning, backup plan, secret management, upgrade plan, monitoring and the list goes on.
Individual free software projects can't take on all this responsibility. They provide the containers as is, in a secure and ready-to-go manner but it is up to service providers to worry about how the app is deployed.
Therefore, Co-op Cloud proposes a packaging format, which we refer to as a recipe, that describes the entire production state of the app in a single place. This format uses the existing standards based compose specification.
This is a file format which is most commonly used by the Docker compose tool but Co-op Cloud does not require the use of Docker compose itself. Furthermore, as described below, we also don't rely on the actual Docker CLI itself either. We do however use a lot of the underlying libraries. We're happily docker & docker-compose CLI independent! Learn more about why we use the compose specification in the FAQ section.
Each recipe that Co-op cloud provides is described using the compose specification and makes use of the upstream project published container. This is the core of our approach to working with the ecosystem of free software communities. We want to maximise the chances of sharing work, knowledge and build solidarity through concrete co-operation.
Container orchestrator
Once we have our app packaged as a recipe, we need a deployment environment. Production deployments are typically expected to support a number of features which give hosters and end-users guarantees for uptime, stability and scale.
The Co-op cloud makes use of Docker swarm as a deployment environment. It offers an approriate feature set which allows us to support zero-down time upgrades, seamless app rollbacks, automatic deploy failure handling, scaling, hybrid cloud setups and maintain a decentralised design.
Learn more about why we use Docker swarm in the FAQ section.
Command-line tool
Finally, with an app and deployment environment, we need a tool to read the recipe package format and actually deploy it to that environment. For this, we have developed and published the abra command-line tool.
abra aims at providing a simple command-line interface for managing your own Co-op Cloud. You can bootstrap machines with the required tools, create new apps, deploy them, back them up, restore them and so on. abra is written in Go and uses a lot of the libraries that the docker and docker-compose CLIs use but does not rely on them directly.
abra is our flagship command-line client but it does not need to be the only client. abra was designed in such a way that is complements a workflow which can still be done completely manually. If Co-op Cloud goes away tomorrow, our configuration commons would still be useful and usable.
Deploy your first app
In order to deploy an app you need two things:
- a server (e.g. Hetzner VPS), with 1. SSH access and 2. a public IP address
- a DNS provider (e.g. Gandi)
Create a server
Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see this FAQ entry).
We will deploy a new Nextcloud instance in this guide, so you will only need 1GB of RAM according to their documentation. You may also be interested in this FAQ entry if you are curious about security in the context of containers.
Wire up DNS
Typically, you'll need two A records, one to point to the VPS itself and another to support sub-domains for the apps. You can then support an app hosted on your root domain (e.g. example.com) and other apps on sub-domains (e.g. foo.example.com, bar.example.com). At time of writing, it is not possible to support multiple apps sharing the same domain.
Your entries in your DNS provider setup might look like the following.
@ 1800 IN A 116.203.211.204
*. 1800 IN A 116.203.211.204
Where 116.203.211.204 can be replaced with the IP address of your server.
Bootstrap abra
You have two options for setting up your command-line client. You can install abra on your local development machine or remotely on the actual server. Please see
Once your DNS and Docker daemon are up, you can install abra locally on your developer machine and hook it up to your server.
Firstly, install abra locally.
curl https://install.abra.autonomic.zone | bash
The source for this script is here.
the installer will verify the checksum. If you want to download abra yourself, you can grab it from the releases page along with the checksums.txt file. If you decide to do this, please run:
grep $(sha256sum abra_[version]_[platform]) checksums.txt > /dev/null && echo "checksum OK"
if "checksum OK" appears in your terminal - you're good to go! otherwise, you have downloaded a corrupted file.
You may need to add the ~/.local/bin/ directory with your $PATH in order to run the executable.
export PATH=$PATH:$HOME/.local/bin
abra -h # check it works
Now you can connect abra with your new server.
abra server add example.com
Where example.com is replaced with your server DNS name.
!!! note "About SSH"
`abra` uses Docker's built-in SSH support to make a secure connection to a
remote Docker daemon, to deploy and manage apps from your local development
machine.
If you need to specify a non-standard port, and/or different username, for SSH,
add them as extra arguments:
```bash
abra server add -p example.com username 2222
```
The -p or --provision flag means that abra will initialise the new single-host swarm on your server.
You will now have a new ~/.abra/ folder on your local file system which stores all the configuration of your Co-op Cloud instance. You can easily share this as a git repository with others.
Deploy Traefik
In order to have your Co-op cloud installation automagically provision SSL certificates, we will first install Traefik. This tool is the main entrypoint for all web requests (e.g. like NGINX) and supports automatic SSL certificate configuration and other quality-of-life features which make deploying libre apps more enjoyable.
abra app new traefik
You will want to take a look at your generated configuration and tweak the LETS_ENCRYPT_EMAIL value:
abra app config traefik
Every app you deploy will have one of these .env files, which contains variables which will be injected into app configurations when deployed. Variables starting with # are optional, others are required.
abra app deploy traefik
Deploy Nextcloud
And now we can deploy apps.
Let's create a new Nextcloud app.
abra app new nextcloud
And we need to generate secrets for the app: database connection password, root password and admin password.
abra app secret generate --all nextcloud
!!! warning
Take care, these secrets are only shown once on the terminal so make sure
to take note of them! `abra` makes use of the [Docker secrets](/secrets/)
mechanism to ship these secrets securely to the server and store them as
encrypted data.
Then we can deploy the Nextcloud.
abra app deploy nextcloud
We can watch to see that things come up correctly.
abra app ps nextcloud # status check
abra app logs nextcloud # logs watch
!!! note
Since Nextcloud takes some time to come up live, you can run the `ps`
command under `watch` like so.
```bash
watch abra app ps nextcloud
```
And you can wait until you see that all containers have the "Running" state.
Your traefik instance will detect that a new app is coming up and generate SSL certificates for it.