toolshed/docs.coopcloud.tech
3
2
Fork 21
Code Issues 8 Pull Requests Packages Releases Activity

Integrate VPN/multi-node install notes #289

New Issue
Open
opened 2025-10-13 19:23:42 +00:00 by decentral1se · 1 comment
decentral1se commented 2025-10-13 19:23:42 +00:00
Owner
Copy Link

Much thanks to Riviera! If anyone has time to integrate into a "setups" page, that'd be cool!

Just driving by and posting so we don't lose it in the mists of matrix threads...

title: Multi Node Coopcloud Install
subject: Technical documentation
date: October 13 2025

Getting Set Up

This text documents a method for setting up a "multi-node" coopcloud
installation using a VPN. The setup could look like this in dot
language (run it through
neato).

graph multi_node {
    overlap=voronoi
    overlap_scaling=8
	vps [label="example.coop"]
	laptop1 [label="laptop1.example.coop"]
	laptop2 [label="laptop2.example.coop"]
	laptopN [label="laptopn.example.coop"]
	vps -- {laptop1 laptop2 laptopN} [label="vpn"]
}

Advantages of this method

  1. Only one, public IP address is needed.
  2. It's scalable and web-secure
  3. It can make use of machines that are otherwise gathering dust

Idiosyncrasies of this method + wishlist

FATA cannot use [server] and --local together

abra server add ... uses ssh under the hood, and it does not make
sense to ssh into to a local server. Still, it would be useful if, for
example, a (configurable) domain name could be set when adding a local
server. This would might also speed up the process of deploying new
apps on the default server because the default option would be 'app.example.coop', rather than 'app.default'.

Docker swarm chaos

Where is the data actually?

Firstly, it appears the docker swarm has the ability to move
containers between nodes (and does so
automatically/dynamically). Thus, the traefik.example.coop app can
run on the VPS with the socket-proxy running on laptop1. If laptop
one drops off the VPN temporarily, this container will be moved
elsewhere. I don't know how docker makes these decisions, but it keeps
things running.

Secondly, given ssh access, any node can abra server add any other
node and run sites from another machine. More testing is needed, but
it seems that one traefik instance in the swarm is enough (as long as
the app is running on the VPS entrypoint?). I need to test further
it's routing knowledge. For example, if laptop1 deploys an app on
laptop2 will the traefik instance pick up on this?

In terms of the coopcloud interface, there is limited data
synchronisation across the vpn because the abra instances are
disconnected. In other words, abra on the VPS doesn't know what abra
is doing on laptop1. At the moment, I see this as a possible
opportunity insofar that it could be a privacy-promoting
feature. However, it would be nifty if the abra instances were
synchronised with one another somehow.

The managerial quorum

To launch apps remotely, abra requires that each node is a docker
swarm manager. And the docker swarm relies on a quorum of managers to
be active. This is interesting from a social point of view as it binds
resilience to solidarity. It is also anti-hierarchical in that every
node has managerial status and the 'leader role' seems not to matter
too much.

Instructions

Step One: Install a VPN on bare metal

Some examples of VPN software that can be self-hosted includes:
- headscale
- nebula
- tinc

Personally, I used tinc, but the setup will probably work with any
self-hosted vpn. Pick a VPN and install it on bare metal on each
machine which will participate in the network.

Step Two: Configure /etc/hosts

From man hosts:

This file is a simple text file that associates IP addresses with
hostnames, one line per IP address. For each host a single
line should be present with the following information:
IP_address canonical_hostname [aliases...]

This is super nifty (and needed) to configure the VPN-based coopcloud
install. The content of /etc/hosts needs to reflect the topology of
the VPN.

Here are two examples. They're almost identical on each host.

/etc/hosts on the VPS
---

127.0.0.1	localhost

# The following lines are desirable for IPv6 capable hosts
::1  		localhost ip6-localhost ip6-loopback
ff02::1		ip6-allnodes
ff02::2		ip6-allrouters

# static IP of VPS
123.456.789.101	example.coop

# vpn
10.0.55.1  example.coop # vpn ip of vps
10.0.55.11 laptop1.example.coop recipe.laptop1.example.coop latop1
10.0.55.21 laptop2.example.coop app.laptop2.example.coop  laptop2

/etc/hosts on laptop2
---

127.0.0.1	localhost
127.0.1.1	laptop2.example.coop laptop2

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

# static IP of VPS
123.456.789.101	example.coop

# vpn
10.0.55.1	example.coop # vpn ip of vps
10.0.55.21 	laptop2.example.coop laptop2

Step three install Docker

Execute the following commands from a terminal on laptop0 (e.g. your
personal computer). This is what worked for me.

$ ssh <ip_addr_of_vps>
$ wget -O- https://get.docker.com | bash
$ sudo docker run hello-world
$ sudo docker swarm init --advertise-addr <vpn_ip_of_vps>
$ sudo docker network create --driver overlay --attachable --scope swarm proxy
$ sudo docker swarm join-token manager # note the output
$ exit

$ ssh <ip_addr_of_laptop1> # laptop1 is in the swarm, laptop0 is not
$ wget -O- https://get.docker.com | bash
$ sudo docker run hello-world
$ sudo usermod -aG docker $USER
$ sudo docker swarm join --token SWMTKN-1-reallyLongTkn <vpn_ip_of_vps>:2377
$ sudo docker node ls # to check all nodes are reachable managers

I recommend starting off with at least three nodes. That way, it's possible to turn one of them off without losing a quorum of managers leading to unexpected behaviour. Make sure to get the <vpn_ip_of_vps> correct in the penultimate command line!

Step four: install Abra (on as many nodes as you want to)

Run this command on the VPS, laptop1, laptop2 etc...

$ curl https://install.abra.coopcloud.tech | bash

and see https://docs.coopcloud.tech/operators/tutorial/#install-abra for further information!

Much thanks to Riviera! If anyone has time to integrate into a "setups" page, that'd be cool! Just driving by and posting so we don't lose it in the mists of matrix threads... > --- > title: Multi Node Coopcloud Install > subject: Technical documentation > date: October 13 2025 > --- > > # Getting Set Up > > This text documents a method for setting up a "multi-node" coopcloud > installation using a VPN. The setup could look like this in dot > language (run it through > [neato](https://graphviz.org/docs/layout/neato)). > > ```dot > graph multi_node { > overlap=voronoi > overlap_scaling=8 > vps [label="example.coop"] > laptop1 [label="laptop1.example.coop"] > laptop2 [label="laptop2.example.coop"] > laptopN [label="laptopn.example.coop"] > vps -- {laptop1 laptop2 laptopN} [label="vpn"] > } > ``` > > ## Advantages of this method > > 1. Only one, public IP address is needed. > 2. It's scalable and web-secure > 3. It can make use of machines that are otherwise gathering dust > > ## Idiosyncrasies of this method + wishlist > > ### FATA cannot use [server] and --local together > `abra server add ...` uses ssh under the hood, and it does not make > sense to ssh into to a local server. Still, it would be useful if, for > example, a (configurable) domain name could be set when adding a local > server. This would might also speed up the process of deploying new > apps on the default server because the default option would be 'app.example.coop', rather than 'app.default'. > > ### Docker swarm chaos > #### Where is the data actually? > > Firstly, it appears the docker swarm has the ability to move > containers between nodes (and does so > automatically/dynamically). Thus, the traefik.example.coop *app* can > run on the VPS with the *socket-proxy* running on laptop1. If laptop > one drops off the VPN temporarily, this container will be moved > elsewhere. I don't know how docker makes these decisions, but it keeps > things running. > > Secondly, given ssh access, any node can `abra server add` any other > node and run sites from another machine. More testing is needed, but > it seems that one traefik instance in the swarm is enough (as long as > the app is running on the VPS entrypoint?). I need to test further > it's routing knowledge. For example, if laptop1 deploys an app on > laptop2 will the traefik instance pick up on this? > > In terms of the coopcloud interface, there is limited data > synchronisation across the vpn because the abra instances are > disconnected. In other words, abra on the VPS doesn't know what abra > is doing on laptop1. At the moment, I see this as a possible > opportunity insofar that it could be a privacy-promoting > feature. However, it would be nifty if the abra instances were > synchronised with one another somehow. > > #### The managerial quorum > > To launch apps remotely, abra requires that each node is a docker > swarm manager. And the docker swarm relies on a quorum of managers to > be active. This is interesting from a social point of view as it binds > resilience to solidarity. It is also anti-hierarchical in that every > node has managerial status and the 'leader role' seems not to matter > too much. > > # Instructions > > ## Step One: Install a VPN on bare metal > Some examples of VPN software that can be self-hosted includes: > - [headscale](https://headscale.net) > - [nebula](https://github.com/slackhq/nebula) > - [tinc](https://tinc-vpn.org) > > Personally, I used tinc, but the setup will probably work with any > self-hosted vpn. Pick a VPN and install it on bare metal on each > machine which will participate in the network. > > ## Step Two: Configure `/etc/hosts` > > From `man hosts`: > > > This file is a simple text file that associates IP addresses with > > hostnames, one line per IP address. For each host a single > > line should be present with the following information: > > IP_address canonical_hostname [aliases...] > > This is super nifty (and needed) to configure the VPN-based coopcloud > install. The content of `/etc/hosts` needs to reflect the topology of > the VPN. > > Here are two examples. They're almost identical on each host. > > ``` > /etc/hosts on the VPS > --- > > 127.0.0.1 localhost > > # The following lines are desirable for IPv6 capable hosts > ::1 localhost ip6-localhost ip6-loopback > ff02::1 ip6-allnodes > ff02::2 ip6-allrouters > > # static IP of VPS > 123.456.789.101 example.coop > > # vpn > 10.0.55.1 example.coop # vpn ip of vps > 10.0.55.11 laptop1.example.coop recipe.laptop1.example.coop latop1 > 10.0.55.21 laptop2.example.coop app.laptop2.example.coop laptop2 > ``` > > > ``` > /etc/hosts on laptop2 > --- > > 127.0.0.1 localhost > 127.0.1.1 laptop2.example.coop laptop2 > > # The following lines are desirable for IPv6 capable hosts > ::1 localhost ip6-localhost ip6-loopback > ff02::1 ip6-allnodes > ff02::2 ip6-allrouters > > # static IP of VPS > 123.456.789.101 example.coop > > # vpn > 10.0.55.1 example.coop # vpn ip of vps > 10.0.55.21 laptop2.example.coop laptop2 > ``` > > ## Step three install Docker > > Execute the following commands from a terminal on laptop0 (e.g. your > personal computer). This is what worked for me. > > ```console > $ ssh <ip_addr_of_vps> > $ wget -O- https://get.docker.com | bash > $ sudo docker run hello-world > $ sudo docker swarm init --advertise-addr <vpn_ip_of_vps> > $ sudo docker network create --driver overlay --attachable --scope swarm proxy > $ sudo docker swarm join-token manager # note the output > $ exit > > $ ssh <ip_addr_of_laptop1> # laptop1 is in the swarm, laptop0 is not > $ wget -O- https://get.docker.com | bash > $ sudo docker run hello-world > $ sudo usermod -aG docker $USER > $ sudo docker swarm join --token SWMTKN-1-reallyLongTkn <vpn_ip_of_vps>:2377 > $ sudo docker node ls # to check all nodes are reachable managers > ``` > > I recommend starting off with at least three nodes. That way, it's possible to turn one of them off without losing a quorum of managers leading to unexpected behaviour. Make sure to get the <vpn_ip_of_vps> correct in the penultimate command line! > > ## Step four: install Abra (on as many nodes as you want to) > > Run this command on the VPS, laptop1, laptop2 etc... > > ```console > $ curl https://install.abra.coopcloud.tech | bash > ``` > > and see https://docs.coopcloud.tech/operators/tutorial/#install-abra for further information!
decentral1se commented 2025-10-17 17:22:37 +00:00
Author
Owner
Copy Link

I looked into a bit and it doesn't seem to be so complicated. I think this issue with apps being reassigned into random places can be handled with docker service update <recipe_laptopN_app> --constraint-add node.hostname==.

> I looked into a bit and it doesn't seem to be so complicated. I think this issue with apps being reassigned into random places can be handled with docker service update <recipe_laptopN_app> --constraint-add node.hostname==<laptopN>.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 Apfelwurm carla cas coopcloud cyrnel decentral1se fauno iexos jmakdah2 joe-irving kawaiipunk knoflook kolaente lambdabundesverband moritz p4u1 pharaohgraphy renovate-bot ripclap simon sixsmith trav val yksflip
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/docs.coopcloud.tech#289
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?