Add Selfhosted Infrastructure Any% post

content/blog/selfhosted-infrastructure-any%.md

@@ -0,0 +1,187 @@
+
+---
+title: "Selfhosted Infrastructure Any%"
+date: 2024-03-30T16:00:00+00:00
+draft: false
+image: "/img/speedrun.jpg"
+---
+
+A Co-op Cloud speedrun attempt livestream! Brought to you by by libre software.
+
+<!--more-->
+
+It'd been so long since [the last Co-op Cloud demo video](https://asciinema.org/a/405836) that, since then, our core command-line tool `abra` had been completely rewritten (from Bash to Golang), and the entire Co-op Cloud project moved from an in-house Autonomic production to a participatory, democractic [Co-op Cloud federation](https://docs.coopcloud.tech/federation/). Definitely time for an update!
+
+So, Co-op Cloud nerd @3wc picked up the fine tradition of [speedrunning](https://en.wikipedia.org/wiki/Speedrunning) for a "Selfhosted Infrastructure Any%" livestream – attempting to beat their personal best time of 46m17s to deploy chat (Matrix / Fluffychat), microblogging (Hometown), and file-/calendar-sharing (Nextcloud) behind single sign-on (Keycloak).
+
+<iframe title="Selfhosted Infrastructure Any%" width="560" height="315" src="https://tv.undersco.re/videos/embed/0044ff41-bf84-4b53-a6e0-4f7d7ce38622" frameborder="0" allowfullscreen="" sandbox="allow-same-origin allow-scripts allow-popups"></iframe>
+
+Before we dive into the write-up, mega-thanks to audience member `@yala` for putting together an excellent write-up during the stream, including some insightful questions. And, of course, to everyone who came along, and everyone who helped signal-boost the announcement beforehand!
+
+# Concept
+
+The integrated set-up of Keycloak/Matrix/Hometown/Nextcloud is a running start for a small-to-medium group looking for alternatives to corporate spyware. It comprises solid alternatives to Dropbox/Google Drive, Twitter, and Slack/Discord/Whatsapp. And, the entire stack can be hosted on a cheap server (or even an old laptop or desktop computer), and the services can be accessed using reasonably slick mobile and desktop apps.
+
+Single sign-on means that in some ways, this set-up is even more usable, even for non-technical users, than the corporate equivalents. And, highlighting where Co-op Cloud can help, getting this combination of tools running manually without Co-op Cloud (that is, following the separate set-up instructions for each platform) is a non-trivial technical challenge.
+
+The choice of apps is primarily geared towards working groups of 3-200 people, and the focus is on internal coördination platforms rather than publication.
+
+## Preparation
+
+To try and simulate an experience of setting up these apps for the first time, and as a nod to the purist conditions popular with videogame speedruns, the demo starts with a blank Debian Docker image (`docker run debian`) and a blank Debian 12 server. (All commands below are run inside that container, unless specified otherwise).
+
+To avoid needing to screenshare DNS and server configuration, there are DNS records for `speedrun.coopcloud.tech` and `*.speedrun.coopcloud.tech` set up, and a pre-generated SSH key has permission to log into the server as root.
+
+## Howto / script
+
+### 1 Local setup
+
+```
+apt update && apt install vim curl wget openssh-client -y
+export EDITOR=vim
+```
+
+(The recommended installation instructions for `abra` require both `curl` and `wget` for some reason. An issue has been opened to figure that out: [`#591`](https://git.coopcloud.tech/coop-cloud/organising/issues/591))
+
+The next step is to copy-paste this from the [`abra` installation instructions](https://docs.coopcloud.tech/operators/tutorial/#install-abra):
+
+```
+curl https://install.abra.coopcloud.tech | bash
+```
+
+> An audience member commented that `curl X | bash` is not amazing security, and indeed downloading [the latest `abra` release binary](https://git.coopcloud.tech/coop-cloud/abra/releases) manually, and comparing the file checksums, would be more secure – patches to [`docs.coopcloud.tech`](https://git.coopcloud.tech/coop-cloud/docs.coopcloud.tech/) extremely welcome! See more on [`#593`](https://git.coopcloud.tech/coop-cloud/organising/issues/593).
+
+Then, running `abra autocomplete bash` shows the necessary commands to set us up some autocompletion for `abra` commands.
+
+### 2. Server set-up
+
+The next step, on the virtual server, is to run the ["Server set-up" section](https://docs.coopcloud.tech/operators/tutorial/#server-setup) of the Co-op Cloud docs:
+
+```
+ssh speedrun.coopcloud.tech
+# docker install convenience script
+wget -O- https://get.docker.com | bash
+
+# setup swarm
+docker swarm init
+docker network create -d overlay proxy
+```
+
+(Skipping adding the current user to the `docker` group, because `root` already has permission to access the Docker daemon)
+
+> As above, `wget ... | bash` is a bit wacky, this is the Docker-recommended installation method but there are others – patches welcome!
+
+Lastly, back on the local machine, `abra app server add speedrun.coopcloud.tech`
+
+### 3. Traefik
+
+```
+abra app new traefik
+abra app config traefik.speedrun.coopcloud.tech
+abra app deploy traefik.speedrun.coopcloud.tech -c -n
+```
+
+(`-c` to skip waiting for the container to come up, and `-n` to skip the confirmation prompt)
+
+![screenshot of Traefik dashboard](/img/speedrun-traefik.webp)
+
+### 4. Keycloak
+
+```
+abra app new keycloak --secrets
+abra app config keycloak.speedrun.coopcloud.tech
+abra app deploy keycloak.speedrun.coopcloud.tech -c -n
+```
+
+Then the Keycloak interface is available at `https://keycloak.speedrun.coopcloud.tech`, necessary to set up the OpenID Connect "clients" for single sign-on for the other apps.
+
+![screenshot of Keycloak login page](/img/speedrun-keycloak.webp)
+
+### 5. Matrix
+
+```
+abra app new matrix-synapse
+```
+
+Meanwhile, by this point Keycloak is alive, and ready to create a new "client". Doing so requires specifying a "client ID", and generates a new "client secret", both of which get added:
+
+```
+abra app config matrix-synapse.speedrun.coopcloud.tech --secrets
+# fill in SSO details
+abra app secret insert matrix-synapse.speedrun.coopcloud.tech keycloak_client_secret v1 (value)
+abra app deploy matrix-synapse.speedrun.coopcloud.tech -n -c
+```
+
+![screenshot of a terminal showing the above abra commands](/img/speedrun-matrix.webp)
+
+### 6. Fluffychat
+
+
+```
+abra app new fluffychat
+abra app config fluffychat.speedrun.coopcloud.tech -n -c
+```
+
+![screenshot of fluffychat login page](/img/speedrun-fluffychat.webp)
+
+### 7. Hometown
+
+```bash
+abra app new hometown
+```
+
+Again, a new Keycloak "client" is needed, giving us a client secret and client ID.
+
+Additionally (at least, for now), it's necessary to do some slightly cursed hacking on the server:
+
+```bash
+docker run -it git.coopcloud.tech/coop-cloud-chaos-patchs/hometown:v4.0.15-hometown-1.1.1  bash
+/usr/local/bin/docker-entrypoint.sh bundle exec rake secret  # once for otp_secret
+/usr/local/bin/docker-entrypoint.sh bundle exec rake secret  # then a second time for secret_key
+/usr/local/bin/docker-entrypoint.sh bundle exec rake mastodon:webpush:generate_vapid_key
+```
+
+Then manually insert the secrets, generate the remaining ones, and edit in the `VAPID_PUBLIC_KEY` and the other SSO details:
+
+```bash
+abra app secret insert hometown.speedrun.coopcloud.tech vapid_private_key v1 <value>
+abra app secret insert hometown.speedrun.coopcloud.tech otp_secret v1 <value>
+abra app secret insert hometown.speedrun.coopcloud.tech secret_key v1 <value>
+abra app config hometown.speedrun.coopcloud.tech
+# set OIDC_* settings, and VAPID_PUBLIC_KEY
+```
+
+Now, finally, the app can be deployed!
+
+```bash
+abra app deploy hometown.speedrun.coopcloud.tech -n -c
+```
+
+![screenshot of the livestream, showing the speedrun timer, a quite-bad webcam
+still, a terminal window with the above abra commands, and a chat log](/img/speedrun-hometown.webp)
+
+### 8. Nextcloud
+
+```bash
+abra app new nextcloud --secrets
+abra app deploy nextcloud.speedrun.coopcloud.tech
+```
+
+Annoyingly, Nextcloud needs stuff done in the web interface, and in the PHP config file, for SSO. So, with another new Keycloak client set up, the next step is logging into https://nextcloud.speedrun.coopcloud.tech, going to "Apps", installing "OIDC login by pulsejet", and then editing the config file:
+
+```
+abra app run nextcloud.speedrun.coopcloud.tech
+apt update && apt install vim
+vim config/config.php
+# fill in from https://git.coopcloud.tech/coop-cloud/nextcloud#how-do-i-integrate-with-keycloak-sso
+```
+
+(in the demo, this was done from the server, but the above approach is probably conceptually simpler)
+
+![screenshot of nextcloud login page](/img/speedrun-nextcloud.webp)
+
+## Closing thoughts
+
+Even though no records got broken, this was fun to make, and definitely a better way to prepare for an in-person demo than silent solo-hacking. It's also nice to get a little "this is how Co-op Cloud is currently looking" demo video out of it.
+
+If you have ideas for future streams, hit us up on Mastodon ([@coopcloud@social.coop](https://social.coop/@coopcloud/)) or dive into our Matrix ([#coopcloud:autonomic.zone](https://matrix.to/#/#coopcloud:autonomic.zone)). Otherwise, follow us directly on owncast ([`@cast@cast.coopcloud.tech`](https://cast.coopcloud.tech/)) and/or peertube ([@coopcloud@tv.undersco.re](https://tv.undersco.re/a/coopcloud/)) to hear about future streams and videos when they happen!