--- title: New operators tutorial --- ## The moving parts Co-op Cloud is made up of a few simple, composable pieces. The system does not rely on any one specific implementation: each part may be replaced and/or extended as needed. We want to build a resilient and long-term sustainable project and that means allowing for different implementations, open formats and a diverse project organisation. Here are the main technical concepts listed below, once you [grok](https://en.wikipedia.org/wiki/Grok) this, you grok the moving parts of the entire project. You can then move on to [deploying your first app](/operators/tutorial/#deploy-your-first-app). ### Libre software apps Libre software apps are tools, websites & software clients that you may already use in your daily life: [Nextcloud], [Jitsi], [Mediawiki], [Rocket.chat] and [many more]! These are tools that are created by volunteer communities who use [free software licenses] in order to build up the public software commons and offer more digital alternatives to [proprietary systems]. The communities who develop these softwares also publish them using [containers]. For example, here is the [Nextcloud hub.docker.com account] which allows end-users to quickly deploy a new Nextcloud instance. There is a growing consensus in the free software community that containers are a useful and time saving format for distribution. !!! question "Why did you choose to use containers?" Learn more [in the FAQ section](/intro/faq/#why-containers). [nextcloud]: https://nextcloud.com [jitsi]: https://jitsi.org [mediawiki]: https://mediawiki.org [rocket.chat]: https://rocket.chat [many more]: /recipes/ [free software licenses]: https://www.gnu.org/philosophy/free-sw.html [nextcloud hub.docker.com account]: https://hub.docker.com/_/nextcloud [proprietary systems]: https://en.wikipedia.org/wiki/Proprietary_software [containers]: https://www.docker.com/resources/what-container ### The recipe packaging format However, just having a container of an app is often not enough. The work required to deploy that app in a "production ready" setup is still too time intensive and often involves a duplication of effort. Each service provider needs to deal with the same problems: stable versioning, backup plan, secret management, upgrade plan, monitoring and the list goes on. Individual free software projects can't take on all this responsibility. They provide the containers as is, in a secure and ready-to-go manner but it is up to service providers to worry about how the app is deployed. Therefore, Co-op Cloud proposes a packaging format, which we refer to as a recipe, that describes the entire production state of the app in a single place. This format uses the existing [standards based compose specification]. This is a file format which is most commonly used by the [Docker compose] tool but Co-op Cloud **does not** require the use of Docker compose itself. Furthermore, as described below, we also don't rely on the actual Docker CLI itself either. We do however use a lot of the underlying libraries. !!! question "Why did you choose to use the compose specificiation?" Learn more [in the FAQ section](/intro/faq/#why-use-the-compose-specification). [Each recipe] that Co-op cloud provides is described using the compose specification and makes use of the upstream project published container when possible (sometimes they don't publish one!). This is the core of our approach to working with the ecosystem of free software communities. We want to maximise the chances of sharing work, knowledge and build solidarity through concrete co-operation. [standards based compose specification]: https://compose-spec.io [docker compose]: https://docs.docker.com/compose/ [each recipe]: /recipes/ ### Container orchestrator Once we have our app packaged as a recipe, we need a deployment environment (e.g. a server & something to keep the containers running). Production deployments are typically expected to support a number of features which give hosters and end-users guarantees for stability. The Co-op cloud makes use of [Docker swarm] as a deployment environment. It offers an approriate feature set which allows us to support zero-down time upgrades, seamless app rollbacks, automatic deploy failure handling, scaling, hybrid cloud setups and maintain a decentralised design. !!! question "Why did you choose to use Docker Swarm?" Learn more [in the FAQ section](/intro/faq/#why-docker-swarm). [docker swarm]: https://docs.docker.com/engine/swarm/ ### Command-line tool Finally, we need a tool to read the recipe package format and actually deploy the app. For this, we have developed and published the [abra] command-line tool. `abra` aims at providing a simple command-line interface for managing your own Co-op Cloud. You can bootstrap machines with the required tools, create new apps and deploy them. `abra` is written in [Go](https://go.dev/) and uses a lot of the libraries that the `docker` and `docker-compose` CLIs use but does not rely on those interfaces directly. `abra` is our flagship command-line client but it does not need to be the only client. `abra` was designed in such a way that it complements a workflow which can still be done completely manually. If Co-op Cloud goes away tomorrow, our configuration commons would still be useful and usable. [abra]: /abra/ ## Deploy your first app In order to deploy an app you need two things: 1. a server with SSH access and a public IP address 2. a domain name pointing to that server The tutorial tries to help you make choices about which server and which DNS setup you need to run a Co-op Cloud deployment but it does not go into great depth about how to set up a new server. !!! question "Can `abra` help automate this?" `abra` can help bootstrap new servers & configure DNS records for you. We'll skip that for now since we're just getting started. See the [operators handbook](/operators/handbook) for more on these topics after you finish the tutorial. ### Server setup Co-op Cloud has itself near zero system requirements. You only need to worry about the system resource usage of your apps and the overhead of running containers with the docker runtime (often negligible. If you want to know more, see [this FAQ entry](/faq/#isnt-running-everything-in-containers-inefficient)). We will deploy a new Nextcloud instance in this guide, so you will only need 1GB of RAM according to [their documentation](https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html). You may also be interested in this [FAQ entry](/faq/#arent-containers-horrible-from-a-security-perspective) if you are curious about security in the context of containers. Most Co-op Cloud deployments have been run on Debian machines so far. Some experiments have been done on single board computers & servers with low resource capacities. You need to keep port `:80` and `:443` free on your server for web proxying to your apps. Typically, you don't need to keep any other ports free as the core web proxy ([Traefik](https://traefik.io)) keeps all app ports internal to its network. Sometimes however, you need to expose an app port when you need to use a transport which would perform better or more reliably without proxying. `abra` has support for both creating servers (`abra server new`) & provisioning them (passing `--provision` to `abra server add`) but those are more advanced automation options which are covered in the [handbook](/operators/handbook). For this tutorial, we'll focus on the basics. Assuming you've managed to create a testing VPS with some `$hosting_provider`, you'll need to install Docker, add your user to the Docker group & setup swarm mode: ``` # docker install convenience script wget -O- https://get.docker.com | bash # add user to docker group sudo usermod -aG docker $USER # setup swarm docker swarm init docker network create -d overlay proxy ``` !!! question "Do you support multiple web proxies?" We do not know if it is feasible and convenient to set things up on an existing server with another web proxy which uses ports `:80` & `:443`. We'd happily receive reports and documentation on how to do this if you manage to set it up! ### DNS setup You'll need two A records, one to point to the server itself and another to support sub-domains for the apps. You can then support an app hosted on your root domain (e.g. `example.com`) and other apps on sub-domains (e.g. `foo.example.com`, `bar.example.com`). Your entries in your DNS provider setup might look like the following. @ 1800 IN A 116.203.211.204 *. 1800 IN A 116.203.211.204 Where `116.203.211.204` can be replaced with the IP address of your server. !!! question "How do I know my DNS is working?" You can use a tool like `dig` on the command-line to check if your server has the necessary DNS records set up. Something like `dig +short ` should show the IP address of your server if things are working. ### Command-line setup #### Install `abra` Now we can install [`abra`](/abra) locally on your machine and hook it up to your server. We support a script-based installation method (script source [here](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer)): ```bash curl https://install.abra.coopcloud.tech | bash ``` The installer will verify the downloaded binary checksum. You may need to add the `~/.local/bin/` directory with your `$PATH` in order to run the executable. You can validate that everything is in working order by listing the default help output: ```bash export PATH=$PATH:$HOME/.local/bin abra -h # check it works ``` If you run into issues during installation, [please report a ticket](https://git.coopcloud.tech/coop-cloud/abra/issues/new) :pray: Once you're all set up, we **highly** recommend configuring command-line auto-completion for `abra`. See `abra autocomplete -h` for more on how to do this. !!! question "Can I install `abra` on my server?" Yes, this is possible, see [this handbook entry](/operators/handbook/#running-abra-server-side) for more. The instructions for setup are a little different however. #### Add your server Now you can connect `abra` with your server. You need to have a working SSH configuration before you can do this. That means you can run `ssh ` on your command-line and everything Works :tm:. ```bash abra server add -p ``` The `-p` or `--provision` flag means that `abra` will install Docker and initialise the [new single-host swarm](https://docs.docker.com/engine/swarm/key-concepts/) on your server. If you've already followed the steps in [the server setup](/operators/tutorial/#server-setup) step, then `abra` should not need to do any work. It is important to note that `` here is a publicy accessible domain name which points to your server IP address. `abra` does make sure this is the case and this is done to avoid issues with HTTPS certificate rate limiting. You will now have a new `~/.abra/` folder on your local file system which stores all the configuration of your Co-op Cloud instance. `abra` should now register this server as managed in your server listing: ``` abra server ls ``` !!! warning "Beware of SSH dragons" `abra` uses plain 'ol SSH under the hood and aims to make use of your existing SSH configurations in `~/.ssh/config` and interfaces with your running `ssh-agent` for password protected secret key files. The `server add` command listed above assumes that that you make SSH connections on port 22 using your current username. If that is not he case, pass the new values as positional arguments. See `abra server add -h` for more on this. abra server add -p Running `server add` with `-d/--debug` should help you debug what is going on under the hood. It's best to take a moment to read [this troubleshooting entry](/abra/trouble/#ssh-connection-issues) if you're running into SSH connection issues with `abra`. !!! question "How do I share my configs in `~/.abra`?" It's possible and quite easy, see [this handbook entry](/operators/handbook/#understanding-app-and-server-configuration) for more. ### Web proxy setup In order to have your Co-op cloud deployment serve the public internet, we need to install the core web proxy, [Traefik](https://doc.traefik.io/traefik/). Traefik is the main entrypoint for all web requests (e.g. like NGINX) and supports automatic SSL certificate configuration and other quality-of-life features which make deploying libre apps more enjoyable. To get started, you'll need to create a new app: ```bash abra app new traefik ``` Choose your newly registered server and specify a domain name. You will want to take a look at your generated configuration and tweak the `LETS_ENCRYPT_EMAIL` value. You can do that by running `abra app config`: ```bash abra app config traefik ``` Every app you deploy will have one of these `.env` files, which contains variables which will be injected into app configurations when deployed. Variables starting with `#` are optional, others are required. Now it is time to deploy: ``` abra app deploy ``` ### Deploy Nextcloud And now we can deploy apps. Let's create a new Nextcloud app. ```bash abra app new nextcloud -S ``` The `-S` or `--secrets` flag is used to generate secrets for the app: database connection password, root password and admin password. !!! warning "Beware of password dragons" Take care, these secrets are only shown once on the terminal so make sure to take note of them! `abra` makes use of the [Docker secrets](/operators/handbook/#managing-secret-data) mechanism to ship these secrets securely to the server and store them as encrypted data. Only the apps themselves have access to the values from here on, they're placed in `/run/secrets` on the container file system. Then we can deploy Nextcloud: ```bash abra app deploy ``` `abra app deploy` will wait nearly a minute for an app to deploy until it times out and shows some helpful commands for how to debug what is going on. If things don't come up in time, try running the following: ``` abra app ps -w # status check abra app logs # logs trailing abra app errors -w # error catcher ``` Your new `traefik` instance will detect that a new app is coming up and generate SSL certificates for it. You can see what `traefik` is up to using the same commands above but replacing `` with the `` you chose earlier (`abra app ls` will remind you what domains you chose :grinning:). ## Finishing up Hopefully you got something running! Well done! The [operators handbook](/operators/handbook) would probably be the next place to go check out if you're looking for more help. Especially on topics of ongoing maintenance. If not, please [get in touch](/intro/contact) or [raise a ticket](https://git.coopcloud.tech/coop-cloud/abra/issues/new) and we'll try to help out. We want our operator onboarding to be as smooth as possible, so we do appreciate any feedback we receive.