migrate The Moving Parts to the Intro area, stylize #248

Merged
basebuilder merged 4 commits from improve-operators into main 2024-02-21 16:27:04 +00:00
6 changed files with 165 additions and 120 deletions

View File

@ -18,6 +18,24 @@ curl https://install.abra.coopcloud.tech | bash
curl https://install.abra.coopcloud.tech | bash -s -- --rc
```
## Manual verification
You can download the `abra` binary yourself from the [releases
page](https://git.coopcloud.tech/coop-cloud/abra/releases) along with the
`checksums.txt` file and verify it's integrity with the following command.
```bash
sha256sum -c checksums.txt --ignore-missing
```
If you see a line starting with `abra_...` which matches the filename you downloaded and it ends with `OK` - you're good to go!
```
abra_X.X.X-beta_linux_x86_64: OK
```
Otherwise, you downloaded a corrupted file and you should re-download it.
## Compile from source
Follow the guide [here](https://docs.coopcloud.tech/abra/hack/)

View File

@ -4,8 +4,16 @@ title: Quick start
There are a few ways to get started, here are some entrypoints listed below:
- If you're new around here and you'd like to learn how to deploy apps with `abra`, then a good place to start is the [new operators tutorial](/operators/tutorial). If you've already deployed some apps and would like to learn how to maintain them, then the [operators handbook](/operators/handbook) is the right place.
<div class="grid cards" markdown>
- If you're installing `abra` so you can do recipe packaging, take a look at the [new maintainers tutorial](/maintainers/tutorial). `abra` can help you check the quality of the recipe you've packaged and help you publish it to the public recipe catalogue. Then others can deploy your configuration :rocket:
- __Operators__
If you're new around here and you'd like to learn how to deploy apps with `abra`, then a good place to start is the [new operators tutorial](/operators/tutorial). If you've already deployed some apps and would like to learn how to maintain them, then the [operators handbook](/operators/handbook) is the right place.
- __Maintainers__
If you're installing `abra` so you can do recipe packaging, take a look at the [new maintainers tutorial](/maintainers/tutorial). `abra` can help you check the quality of the recipe you've packaged and help you publish it to the public recipe catalogue. Then others can deploy your configuration :rocket:
</div>
If you run into any issues, please see the [troubleshooting page](/abra/trouble) :bomb:

View File

@ -1,19 +1,105 @@
---
title: Project strategy
title: Project Strategy
---
!!! note "Yes, we are blog"
From our experiences working and organising as Autonomic, the tech co-op who [initiated Co-op Cloud](https://autonomic.zone/blog/co-op-cloud/), we know that the progressive tech movement lack reliable and cost-effective technical means for providing a sustainable alternative to _Big Tech_© services which are marketed as "[cloud computing](https://en.wikipedia.org/wiki/Cloud_computing)".
Some leading thoughts are outlined in the [project launch blog post](https://autonomic.zone/blog/co-op-cloud/) also.
From our experiences working and organising as Autonomic, the tech co-op who initiated Co-op Cloud, we know that the progressive tech movement lack reliable and cost-effective technical means for providing an alternative to “Big Tech” cloud services.
## Technological Saviors?
The urgency for providing an alternative comes out of the understanding that the concentration of our digital lives within the private sphere of corporate providers (e.g. [GAFAM](https://degooglisons-internet.org/en/)) represents a loss of freedom due to the threat to our privacy and self-determination through surveillance and monopolisation.
As a movement, we cannot compete with corporate providers in terms of cost and scale. Their network effects and available capital means that no one project, product or organisation can create the required shift to a more widespread public interest technology.
Technology alone will not save us. Simply deploying libre software is not enough.
> Technology alone will not save us
>
> Simply deploying libre software is not enough.
Our strategy is to mutualise our resources to facilitate this shift. Co-op Cloud is an attempt to create a new shared resource - an open and democratically managed, open standards based, copyleft licensed, libre software infrastructure project.
Our strategy is to mutualise our resources to facilitate this shift. _Co-op Cloud_ is an attempt to create a new shared resource - an open and democratically managed, open standards based, copyleft licensed, libre software infrastructure project.
From this base, we can focus on the urgent and necessary social organising work that goes beyond the technical question.
## The Moving Parts
_Co-op Cloud_ is made up of a few simple, composable pieces. The system does not rely on any one specific implementation: each part may be replaced and/or extended as needed. We want to build a resilient and long-term sustainable project and that means allowing for different implementations, open formats and a diverse project organisation. Here are the main technical concepts listed below,
``` mermaid
graph LR
A[Libre Software\n Apps] --> B{Recipe Packaging};
B --> C[CLI Tool];
C --> D[Container\n Orchestrator];
basebuilder marked this conversation as resolved Outdated

So Cool!

If you skip straight to orchestrator from the recipe packaging, do you use plain 'ol Docker to deploy? Seems to me like maybe we don't want to emphasise this in the initial presentation of the system? It's all technically possible but I'd keep it simple here and just do "package, CLI, orchestrator"?

So Cool! If you skip straight to orchestrator from the recipe packaging, do you use plain 'ol Docker to deploy? Seems to me like maybe we don't want to emphasise this in the initial presentation of the system? It's all technically possible but I'd keep it simple here and just do "package, CLI, orchestrator"?

This makes total sense, thanks for the feedback 👍 I updated it to the following:

Moving Parts Flow Chart

This makes total sense, thanks for the feedback 👍 I updated it to the following: ![Moving Parts Flow Chart](/attachments/9c54e55b-021b-48ef-ad53-70619c073842)
```
Once you [grok](https://en.wikipedia.org/wiki/Grok) this, you grok the moving parts of the entire project. You can then move on to [deploying your first app](/operators/tutorial/#deploy-your-first-app).
### Libre Software Apps
Libre software apps are tools- they take the shape of websites, mobile apps, and software clients that you may already use in your daily life, for example...
<div class="grid cards" markdown>
- :simple-nextcloud: __Nextcloud__
- :simple-jitsi: __Jitsi__
- :simple-wikimediacommons: __Mediawiki__
- :fontawesome-solid-rocket: __Rocket.chat__
</div>
...and many more. These apps are also often referred to as _open-Source_ or _Free-Software_. These are tools that are created by volunteer communities who use [free software licenses] in order to build up the public software commons and offer more digital alternatives to [proprietary systems].
The communities who develop these softwares also publish them using [containers]. For example, here is the [Nextcloud hub.docker.com account] which allows end-users to quickly deploy a new Nextcloud instance.
There is a growing consensus in the free software community that containers are a useful and time saving format for distribution.
!!! question "Why did you choose to use containers?"
Learn more [in the FAQ section](/intro/faq/#why-containers).
[free software licenses]: https://www.gnu.org/philosophy/free-sw.html
[nextcloud hub.docker.com account]: https://hub.docker.com/_/nextcloud
[proprietary systems]: https://en.wikipedia.org/wiki/Proprietary_software
[containers]: https://www.docker.com/resources/what-container
### Recipe Packaging Format
However, just having a container of an app is often not enough. The work required to deploy that app in a "production ready" setup is still too time intensive and often involves a duplication of effort.
Each service provider needs to deal with the same problems: stable versioning, backup plan, secret management, upgrade plan, monitoring and the list goes on.
Individual free software projects can't take on all this responsibility. They provide the containers as is, in a secure and ready-to-go manner but it is up to service providers to worry about how the app is deployed.
Therefore, Co-op Cloud proposes a packaging format, which we refer to as a recipe, that describes the entire production state of the app in a single place. This format uses the existing [standards based compose specification].
This is a file format which is most commonly used by the [Docker compose] tool but Co-op Cloud **does not** require the use of Docker compose itself. Furthermore, as described below, we also don't rely on the actual Docker CLI itself either. We do however use a lot of the underlying libraries.
!!! question "Why did you choose to use the compose specificiation?"
Learn more [in the FAQ section](/intro/faq/#why-use-the-compose-specification).
[Each recipe] that Co-op cloud provides is described using the compose specification and makes use of the upstream project published container when possible (sometimes they don't publish one!).
This is the core of our approach to working with the ecosystem of free software communities. We want to maximise the chances of sharing work, knowledge and build solidarity through concrete co-operation.
[standards based compose specification]: https://compose-spec.io
[docker compose]: https://docs.docker.com/compose/
[each recipe]: /recipes/
### Container Orchestrator
Once we have our app packaged as a recipe, we need a deployment environment (e.g. a server & something to keep the containers running). Production deployments are typically expected to support a number of features which give hosters and end-users guarantees for stability.
The Co-op cloud makes use of [Docker swarm] as a deployment environment. It offers an approriate feature set which allows us to support zero-down time upgrades, seamless app rollbacks, automatic deploy failure handling, scaling, hybrid cloud setups and maintain a decentralised design.
!!! question "Why did you choose to use Docker Swarm?"
Learn more [in the FAQ section](/intro/faq/#why-docker-swarm).
[docker swarm]: https://docs.docker.com/engine/swarm/
### Command-line tool
Finally, we need a tool to read the recipe package format and actually deploy the app. For this, we have developed and published the [abra] command-line tool.
`abra` aims at providing a simple command-line interface for managing your own Co-op Cloud. You can bootstrap machines with the required tools, create new apps and deploy them. `abra` is written in [Go](https://go.dev/) and uses a lot of the libraries that the `docker` and `docker-compose` CLIs use but does not rely on those interfaces directly.
`abra` is our flagship command-line client but it does not need to be the only client. `abra` was designed in such a way that it complements a workflow which can still be done completely manually. If Co-op Cloud goes away tomorrow, our configuration commons would still be useful and usable.
[abra]: /abra/

View File

@ -205,18 +205,6 @@ At time of writing (Jan 2022), we think there is a limitation in our design whic
This may be possible to overcome if someone really needs it, we encourage people to investigate. We've found that often there are limitations in the actual software which don't support this anyway and several of the current operators simply use a new domain per app.
## Validating `abra` binary checksums
You can download `abra` yourself from the [releases page](https://git.coopcloud.tech/coop-cloud/abra/releases) along with the `checksums.txt` file.
```bash
grep $(sha256sum abra_[version]_[platform]) checksums.txt > /dev/null && echo "checksum OK"
```
If "checksum OK" appears in your terminal - you're good to go!
Otherwise, you have downloaded a corrupted file.
## Creating a new server
`abra server new` can create servers if you have an account with a supported 3rd party integration. We currently support [Servers.coop](https://servers.coop) & [Hetzner](https://hetzner.com). The process of creating a new server usually goes like this:

View File

@ -2,82 +2,7 @@
title: New Operators Tutorial
---
## The moving parts
Co-op Cloud is made up of a few simple, composable pieces. The system does not rely on any one specific implementation: each part may be replaced and/or extended as needed.
We want to build a resilient and long-term sustainable project and that means allowing for different implementations, open formats and a diverse project organisation.
Here are the main technical concepts listed below, once you [grok](https://en.wikipedia.org/wiki/Grok) this, you grok the moving parts of the entire project. You can then move on to [deploying your first app](/operators/tutorial/#deploy-your-first-app).
### Libre software apps
Libre software apps are tools, websites & software clients that you may already use in your daily life: [Nextcloud], [Jitsi], [Mediawiki], [Rocket.chat] and [many more]!
These are tools that are created by volunteer communities who use [free software licenses] in order to build up the public software commons and offer more digital alternatives to [proprietary systems].
The communities who develop these softwares also publish them using [containers]. For example, here is the [Nextcloud hub.docker.com account] which allows end-users to quickly deploy a new Nextcloud instance.
There is a growing consensus in the free software community that containers are a useful and time saving format for distribution.
!!! question "Why did you choose to use containers?"
Learn more [in the FAQ section](/intro/faq/#why-containers).
[nextcloud]: https://nextcloud.com
[jitsi]: https://jitsi.org
[mediawiki]: https://mediawiki.org
[rocket.chat]: https://rocket.chat
[many more]: /recipes/
[free software licenses]: https://www.gnu.org/philosophy/free-sw.html
[nextcloud hub.docker.com account]: https://hub.docker.com/_/nextcloud
[proprietary systems]: https://en.wikipedia.org/wiki/Proprietary_software
[containers]: https://www.docker.com/resources/what-container
### The recipe packaging format
However, just having a container of an app is often not enough. The work required to deploy that app in a "production ready" setup is still too time intensive and often involves a duplication of effort.
Each service provider needs to deal with the same problems: stable versioning, backup plan, secret management, upgrade plan, monitoring and the list goes on.
Individual free software projects can't take on all this responsibility. They provide the containers as is, in a secure and ready-to-go manner but it is up to service providers to worry about how the app is deployed.
Therefore, Co-op Cloud proposes a packaging format, which we refer to as a recipe, that describes the entire production state of the app in a single place. This format uses the existing [standards based compose specification].
This is a file format which is most commonly used by the [Docker compose] tool but Co-op Cloud **does not** require the use of Docker compose itself. Furthermore, as described below, we also don't rely on the actual Docker CLI itself either. We do however use a lot of the underlying libraries.
!!! question "Why did you choose to use the compose specificiation?"
Learn more [in the FAQ section](/intro/faq/#why-use-the-compose-specification).
[Each recipe] that Co-op cloud provides is described using the compose specification and makes use of the upstream project published container when possible (sometimes they don't publish one!).
This is the core of our approach to working with the ecosystem of free software communities. We want to maximise the chances of sharing work, knowledge and build solidarity through concrete co-operation.
[standards based compose specification]: https://compose-spec.io
[docker compose]: https://docs.docker.com/compose/
[each recipe]: /recipes/
### Container orchestrator
Once we have our app packaged as a recipe, we need a deployment environment (e.g. a server & something to keep the containers running). Production deployments are typically expected to support a number of features which give hosters and end-users guarantees for stability.
The Co-op cloud makes use of [Docker swarm] as a deployment environment. It offers an approriate feature set which allows us to support zero-down time upgrades, seamless app rollbacks, automatic deploy failure handling, scaling, hybrid cloud setups and maintain a decentralised design.
!!! question "Why did you choose to use Docker Swarm?"
Learn more [in the FAQ section](/intro/faq/#why-docker-swarm).
[docker swarm]: https://docs.docker.com/engine/swarm/
### Command-line tool
Finally, we need a tool to read the recipe package format and actually deploy the app. For this, we have developed and published the [abra] command-line tool.
`abra` aims at providing a simple command-line interface for managing your own Co-op Cloud. You can bootstrap machines with the required tools, create new apps and deploy them. `abra` is written in [Go](https://go.dev/) and uses a lot of the libraries that the `docker` and `docker-compose` CLIs use but does not rely on those interfaces directly.
`abra` is our flagship command-line client but it does not need to be the only client. `abra` was designed in such a way that it complements a workflow which can still be done completely manually. If Co-op Cloud goes away tomorrow, our configuration commons would still be useful and usable.
[abra]: /abra/
This tutorial assumes you understand the [frequently asked questions](/intro/faq/) as well as [the moving parts](/intro/strategy/) of the technical problems _Co-op Cloud_ solves. If yes, proceed :smile:
## Deploy your first app
@ -86,11 +11,14 @@ In order to deploy an app you need two things:
1. a server with SSH access and a public IP address
2. a domain name pointing to that server
The tutorial tries to help you make choices about which server and which DNS setup you need to run a Co-op Cloud deployment but it does not go into great depth about how to set up a new server.
This tutorial tries to help you make choices about which server and which DNS setup you need to run a _Co-op Cloud_ deployment but it does not go into great depth about how to set up a new server.
!!! question "Can `abra` help automate this?"
??? question "Can `abra` help automate this?"
`abra` can help bootstrap new servers & configure DNS records for you. We'll skip that for now since we're just getting started. See the [operators handbook](/operators/handbook) for more on these topics after you finish the tutorial.
Our `abra` tool can help bootstrap new servers & configure DNS records for
you. We'll skip that for now since we're just getting started. For more on
these topics after you finish the tutorial see the [operators
handbook](/operators/handbook).
### Server setup
@ -116,7 +44,7 @@ docker swarm init
docker network create -d overlay proxy
```
!!! question "Do you support multiple web proxies?"
??? question "Do you support multiple web proxies?"
We do not know if it is feasible and convenient to set things up on an existing server with another web proxy which uses ports `:80` & `:443`. We'd happily receive reports and documentation on how to do this if you manage to set it up!
@ -131,36 +59,43 @@ Your entries in your DNS provider setup might look like the following.
Where `116.203.211.204` can be replaced with the IP address of your server.
!!! question "How do I know my DNS is working?"
??? question "How do I know my DNS is working?"
You can use a tool like `dig` on the command-line to check if your server has the necessary DNS records set up. Something like `dig +short <domain>` should show the IP address of your server if things are working.
### Command-line setup
### Install `abra`
#### Install `abra`
Now we can install [`abra`](/abra) locally on your machine and hook it up to your server.
We support a script-based installation method (script source [here](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer)):
Now we can install [`abra`](/abra) locally on your machine and hook it up to
your server. We support a script-based installation method ([script source](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer)):
```bash
curl https://install.abra.coopcloud.tech | bash
```
The installer will verify the downloaded binary checksum. You may need to add the `~/.local/bin/` directory with your `$PATH` in order to run the executable. You can validate that everything is in working order by listing the default help output:
The installer will verify the downloaded binary checksum. If you prefer, you can
[manually verify](/abra/install/#manual-verification) the binary, and then
manally place it in one the directories in your `$PATH` variable. To validate
that everything is working try listing the `--help` command or `-h` to view
output:
```bash
abra -h
```
You may need to add the `~/.local/bin/` directory to your `$PATH` variable, in
order to run the executable.
```bash
export PATH=$PATH:$HOME/.local/bin
abra -h # check it works
```
If you run into issues during installation, [please report a ticket](https://git.coopcloud.tech/coop-cloud/abra/issues/new) :pray: Once you're all set up, we **highly** recommend configuring command-line auto-completion for `abra`. See `abra autocomplete -h` for more on how to do this.
!!! question "Can I install `abra` on my server?"
??? question "Can I install `abra` on my server?"
Yes, this is possible, see [this handbook entry](/operators/handbook/#running-abra-server-side) for more. The instructions for setup are a little different however.
Yes, this is possible. However, the instructions for this setup are different. For more info see [this handbook entry](/operators/handbook/#running-abra-server-side).
#### Add your server
### Add your server
Now you can connect `abra` with your server. You should have a working SSH configuration before you can do this (e.g. a matching `Host <server-domain>` entry in `~/.ssh/config` with the correct SSH connection details). That means you can run `ssh <server-domain>` on your command-line and everything Works :tm:.
@ -173,21 +108,26 @@ It is important to note that `<domain>` here is a publicy accessible domain name
You will now have a new `~/.abra/` folder on your local file system which stores all the configuration of your Co-op Cloud instance.
`abra` should now register this server as managed in your server listing:
By now `abra` should have registered this server as managed. To confirm this run:
```
abra server ls
```
!!! warning "Beware of SSH dragons"
??? warning "Beware of SSH dragons :dragon_face:"
`abra` uses plain 'ol SSH under the hood and aims to make use of your existing SSH configurations in `~/.ssh/config` and interfaces with your running `ssh-agent` for password protected secret key files.
Under the hood `abra` uses plain 'ol `ssh` and aims to make use of your
existing SSH configurations in `~/.ssh/config` and interfaces with your
running `ssh-agent` for password protected secret key files.
Running `server add` with `-d/--debug` should help you debug what is going on under the hood. It's best to take a moment to read [this troubleshooting entry](/abra/trouble/#ssh-connection-issues) if you're running into SSH connection issues with `abra`.
Running `server add` with `-d` or `--debug` should help you debug what is going
on under the hood. If you're running into SSH connection issues with `abra`
take a moment to read [this troubleshooting
entry](/abra/trouble/#ssh-connection-issues).
!!! question "How do I share my configs in `~/.abra`?"
??? question "How do I share my configs in `~/.abra`?"
It's possible and quite easy, see [this handbook entry](/operators/handbook/#understanding-app-and-server-configuration) for more.
It's possible and quite easy, for more see [this handbook entry](/operators/handbook/#understanding-app-and-server-configuration).
### Web proxy setup
@ -227,7 +167,7 @@ abra app new nextcloud -S
The `-S` or `--secrets` flag is used to generate secrets for the app: database connection password, root password and admin password.
!!! warning "Beware of password dragons"
??? warning "Beware of password dragons :dragon:"
Take care, these secrets are only shown once on the terminal so make sure to take note of them! `abra` makes use of the [Docker secrets](/operators/handbook/#managing-secret-data) mechanism to ship these secrets securely to the server and store them as encrypted data. Only the apps themselves have access to the values from here on, they're placed in `/run/secrets` on the container file system.

View File

@ -48,6 +48,11 @@ markdown_extensions:
- pymdownx.superfences
- pymdownx.tabbed
- pymdownx.tilde
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
nav:
- "Introduction":