rix/docs.coopcloud.tech
0
0
Fork 0
forked from toolshed/docs.coopcloud.tech
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: add restack text

Browse Source
This commit is contained in:
decentral1se
2026-11-04 11:33:26 +01:00
parent 4d5e74864a
commit 91c93fb1b4
2 changed files with 282 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/federation/funding/nlnet-restack.md
+281
View File
@@ -0,0 +1,281 @@
---
title: NLNet Restack
---
# NLNet Restack
> Status: **Not submitted, missed the deadline, will submit in January 2027** ([Link to open call](https://nlnet.nl/propose/))
# Select a fund
## Which of our funds are you applying for?
Restack.
# Project overview
## Proposal title
Scaling the Co-op Cloud Software Commons.
## Project website(s) / repositories
- [coopcloud.tech](https://coopcloud.tech)
- [docs.coopcloud.tech](https://docs.coopcloud.tech)
- [recipes.coopcloud.tech](https://recipes.coopcloud.tech)
- [`git.coopcloud.tech/toolshed/abra`](https://git.coopcloud.tech/toolshed/abra)
- [`git.coopcloud.tech/toolshed/alakazam`](https://git.coopcloud.tech/toolshed/alakazam)
- [`toolshed/recipes-catalogue-json`](https://git.coopcloud.tech/toolshed/recipes-catalogue-json), [`toolshed/almanac`](https://git.coopcloud.tech/toolshed/almanac)
## Please summarise the proposed project and its expected results in a few lines. (max. 1000 characters)
Co-op Cloud is a software commons, international federation and broad solidarity network which works towards a vision of democratic technology for everyone.
We focus here on the software commons: a suite of tools and shared configs which simplify the deployment and maintenance of free software apps. Co-op Cloud implements a "pragmatic restack" where the daily needs of small-scale hosting collectives are the central driving force.
This approach has lead to considerable decentralised growth and international adoption with an estimated 50-100 active operators, 20-40 hosting collectives, thousands of end-users, 21 federation members and successful commercial projects built on top of Co-op Cloud.
This growth pushes our implementation to its limits, revealing issues in implementation (maintenance, technical debt), performance, missing distributed collaboration and operational features. In this project, we aim to overcome these limitations and continue scaling the Co-op Cloud software commons.
# Proposed effort
## How much money do you request for your project? (In Euros. Between € 5 000 and € 50 000 for a first grant.)
47880 EUR.
## Please provide a breakdown clarifying how the requested budget would be used. Which tasks will you undertake, and (roughly) how much time/effort will each take? What hourly/daily/monthly rate did you use? Will there be any expenses? (max. 4000 characters)
We propose an hourly rate of 60 EUR per hour with a total of 798 dev hours (breakdown below per heading). We anticipate no expenses. Please see attached work.pdf for more.
**Maintenance & Technical Debt** (233 hours)
- Fix the oustanding (as of Nov 2026) 21 bugs in Abra reported by operators. These issues represent daily friction for operators and make deployment, maintenance and collaboration work more difficult to carry out.
- Upstream our patches to Docker CLI and upgrade to the latest Docker libraries in Abra so that we maintain parity with upstream developments.
- Migrate from the incomplete xgettext-go to the official upstream Golang xgettext.
- Migrate to the new compose-go standard library in Abra so that we maintain parity with one of the central open standards which we rely on. This decouples our sole reliance on the Docker Swarm runtimes (out-of-date) implementation of the Compose Specification.
**Performance** (100 hours)
- The Co-op Cloud catalogue is a single JSON file which is becoming a performance bottleneck for Abra. We will convert catalogue generation and querying into a lightweight API service which always provides fast results regardless of quantity of app configs which integrates with Abra.
- Abra development follows the "make it work, make it right, make it fast" principle. We are now in the "make it fast" phase. Abra requires profiling to understand where it is doing double work and unecessary bookkeeping. The goal is to return to a high speed CLI experience.
**Distributed Collaboration & Operations** (450 hours)
- Co-op Cloud currently "takes over" an existing server, imposing a single web proxy choice. Operators need proxy agnosticism, supporting several popular proxies seamlessly (e.g. Traefik, Nginx, Caddy).
- App configs rely on a ".env" file format (FOO=BAR). This format was chosen for interoperability but with the increase of config options, these files become difficult to manage. We will migrate to a YAML configuration for maintainers which brings new organisation benefits while keeping the .env file for operators.
- When maintainers upgrade app configs, changes must be propagated to operator configs (e.g. a new config knob for database perf). This is currently a manual process. We will implement an automatic upgrade in Abra with a change overview for operators.
- With the development of the new Docker Secrets Engine, it is possible to decouple the secret injection of Abra from the Docker Swarm runtime. Implementing secret engine support in Abra would reduce our reliance on the Docker runtime overall due to its pluggable interface with independent secret management tools.
- Due to increasing numbers of app config maintainers, there are more pull request review workflows happening on git.coopcloud.tech. The Abra app config upgrade and release functionality does not conveniently support a pull request workflow. We will implement this workflow support in Abra by allowing Abra to produce and adapt to diffs which are designed for a review, approve and apply workflow.
- Co-op Cloud operators currently sync their app config changes manually using Git (e.g. deployment version). We will implement a peer-to-peer sync in Abra which allows operators to inform each other about deployment and maintenance operations directly on the command-line. We aim to implement this using p2panda-go (Iroh under the hood).
- Alakazam is a longrunning prototype used by KolliCloud (free software hosting platform built on Co-op Cloud) which implements a wrapper of Abra and supports the deployment of multiple apps in a single "stack" (e.g. deploy Nextcloud, BigBlueButton and Rauthy SSO with shared config). It has reached maturity and been adopted by the Co-op Cloud federation as an official community tool. There is a desire to bring the feature set of Alakazam back to Abra, ending the "wrapper of a wrapper" prototype implementation and giving all Co-op Cloud operators first-class support for seamless "stack" deployments without needing an additional tool.
## Please compare your own project with other relevant efforts (both existing and historical). What is new, more thorough or otherwise different? Have you considered contributing to any of the others? (max. 4000 characters)
The Co-op Cloud implementation is based upon lessons learnt while participating in the Cloudron, YunoHost and Ansible projects (Co-op Cloud maintainers were active Ansible, Molecule and YunoHost maintainers in the past).
Cloudron repackages apps in a custom format, duplicating work and became source-available. YunoHost relies too closely on the distribution cycle of Debian and does not make use of containerisation which tends towards "house of cards" server installs which must work around the OS (e.g. managing custom PHP versions outside the Debian distribution cycle).
Ansible proposes a model of re-use which is not reflected in reality, resulting in the same basic roles being developed over and over again. Solutions for role testing such as Molecule and Ansible Collections have not truly alleviated the situation. We believe Ansible is not flexible enough to allow for deployment specifics and broad config collaboration amongst multiple hosting collectives.
Kubernetes helm charts overlaps in focus but Co-op Cloud participants have long rejected Kubernetes (including distros which try to simplify k8s) as overkill and unsuitable for the needs, capacities and means of small-scale hosting collectives. Charts also duplicates work with a custom packaging format.
Nix and NixOS as a deploy and config management platform has seen minor adoption in the Co-op Cloud community but ultimately lacks the focus on day 1 and 2 operations that Abra and Alakazam bring to Co-op Cloud operators. Nix proposes once more a custom app packaging format which require duplicated effort or the complexity of a "compose2nix" layer. It has a steep learning curve and a significant "lock in" factor, making it difficult to share tools outside of the Nix ecosystem.
Uncloud is an upcoming solution to "docker compose in production" but also duplicates work with a custom app packaging format. We are watching closely how Uncloud rejects the declarative approach of the Swarm runtime in favour of a imperative model. Uncloud emerged in 2026, 6 years after the start of Co-op Cloud.
We maintain a comparisons list (docs.coopcloud.tech/intro/comparisons) to establish the position of Co-op Cloud in this fast moving field of deployment and config management. We still believe Co-op Cloud offers a unique software solution for the problem at hand.
- We re-use upstream packaging efforts of free software developers and meet them where they are at: the compose standard. All app configs form our config commons which can be re-used by any tool which consumes the compose standard (e.g. Guix ports have been attempted).
- We realise real "should to shoulder" collaboration on free software app configs with the flexibility of the compose standard. A hosting collective of 2 hackers and another of 200 hackers can work in the same app config without crashing into each other with each deploying a config which is radically customised for their specific situation. Their respective app config maintenance work always contributes back to each other and the greater software commons.
- One of the major strengths of Co-op Cloud is its relative familiarity and simplicity for (new) operators in comparison to other tools. Abra follows the Docker CLI design closely and there is limited indirection and abstraction: it's images and compose files underneath and that's it.
- The Co-op Cloud federation is a democratically run international (semi-formal) organisation of 21 small-scale hosting collectives which is open for all to participate and serves as the engine of continuity for the software commons. This offers a level of confidence for the Co-op Cloud project because it is not reliant on a single "benevolant dictator" or hosting collective and it is resilient due to the decentralised governance model: we do not need to repeat the A/I situation - if one collective is taken down, another will take its place because the underlying structure is not disrupted: the software commons, federation and solidarity network.
## What technical challenges do you expect during the project? (max. 4000 characters)
Co-op Cloud faces an immense challenge in overcoming the limitations of the "pragmatic restack" approach and the resulting path dependency: the reliance on the Docker Swarm runtime, an increasingly underdeveloped software for container orchestration. This is not an immediate emergency but remains something we work towards resolving in this project work and beyond.
An estimated 20-40 hosting collectives now rely on Co-op Cloud and we cannot deliver a breaking platform migration such as a new container orchestration engine which would disrupt their daily operations. We have analysed this dependency (docs.coopcloud.tech/abra/swarm) and see a path forward: we slowly strip away our dependency on specific parts of Swarm (e.g secret management, templating and finally orchestration) and swap them out with resilient alternatives, migrating our entire ecosystem forwards in a backwards compatible manner, leaving nobody behind.
One concrete example is the development of the new Docker Secrets Engine which will allow us to implement a plugin architecture for secrets delivery based on the choice of secret management tool of the operators themselves. This implementation represents a challenge for us because it is the first time we will undertake such an ecosystem-wide migration. This is included in the proposed work items of this proposal.
The Co-op Cloud federation represents a diverse spectrum of use cases, some demanding highly constrained deployments (offline-first, single-board computers, intermittent electricity) and others requiring high throughput (increasing userbases, heavier and more powerful free software apps, specialised deployments and collaboration features). The development of fixes and new features must pass through this collective consensus, improving and becoming the exact feature scope which benefits all participants, including end-users. This involves many technical discussions and redesigns to reach the final result.
Co-op Cloud faces a new technical challenge in recent times. The project was developed for small-scale hosting collectives that already "know what they're doing" but the pragmatic accessibility of our tools has increasingly attracted new system administrators and emerging hosting collectives, some of whom have never even used the command-line before. This is a new use case which we must bring into our design thinking and implementation of our tools and documentation writing.
Co-op Cloud tools have also become "officially" international since the Sepal Fund application from the Abya Yala Network in which we delivered a Spanish translation for Abra with full command-line coverage (options, docs, command output, etc.) and integration with a self-hosted Weblate deployment.
This has been a great success but demands more internationalisation. This process challenges the limits of most free software projects approach to internationalisation. We have been forced to rely on an incomplete implementation of xgettext which we must migrate away from and even one of the most popular CLI Golang libraries (spf13/cobra) requires patching to deliver translated option flags. This requires us to make upstream contributions which are often hard to implement and deliver due to how deep reaching they are in the upstream implementation (implementing i18n in a codebase touches nearly everything).
## What is the ecosystem around your own project, and how will you engage relevant actors? For example, does your project depend on other projects/people? And who would be the main users? (max. 2000 characters)
The Co-op Cloud federation and solidarity network remains the primary ecosystem in which our project takes place. We congregate online in our Matrix community, publish through the Fediverse, share community updates on the Co-op Cloud blog (coopcloud.tech/blog) and by participating at events, doing workshops and presentations. End-users are included through their interaction with federation members on the local level. Federation members are currently based in Latin America, Australia, North America and Europe.
Based on a usage survey conducted in October 2026, over 70% of Co-op Cloud participants offer paid free software services to customers. 2/3 of those are "small", serving less than 10 customers and 1/3 are "medium to large", serving 101-1000 customers. Here are some highlights of the most significant federation member projects:
- KolliCloud, a free software hosting platform built by Local-IT on top of Co-op Cloud serves, amongst others, Germany’s largest queer youth support organisation queersupport.de.
- Escuela Común, a hands-on self-organised school for emerging indigenous hosting collectives based in Latin America has seen 21 instances of Co-op Cloud deployed and managed by communities themselves with the emergence of 2 public media networks based on Peertube in Argentina and El Salvador.
- Merri-bek Tech, recently featured in ABC News as "a grassroots plan to reclaim the internet from 'big tech'" approaches climate resiliency from a "neighbourhood first" approach, incorporating Co-op Cloud as it's foundation in LoRes Mesh while building resilience features alongisde with peer-to-peer systems such as p2panda.
- Popular free software projects such as Bonfire, Karrot and Lauti have joined the federation and adopted Co-op Cloud as one of their foundational deployment options. This has lead to broader usage of Co-op Cloud in the case of more visible deployments such as jacobin.social (Bonfire hosting via CoQuest, also a federation member).
# Background
## What makes you the right person(s) for this effort? Do you have relevant experience in this or other projects/organisations? And if so, can you tell us a bit about your contributions? (max. 2000 characters)
(Redacted)
## Does (or did) the project have other funding sources? (max. 1000 characters)
Co-op Cloud has received 32,986 EUR from the European Cultural Fund in 2022.
Co-op Cloud received 9,240 GBP from a private funder in 2023.
Co-op Cloud receives up to three years of financial support (around 5000 USD) via The Sepal Fund since 2025 when the Abya Yala Network was accepted as a grantee. Sutty is a Co-op Cloud federation member and founding member of the Abya Yala Network.
Local-IT, a Co-op Cloud federation member based in Germany has received 540,000 EUR in public and civic funding to build KolliCloud, a free software hosting platform built on Co-op Cloud. This funding was managed and allocated by Local-IT themselves with the implementation benefitting the entire Co-op Cloud software commons.
The Co-op Cloud federation has developed a membership model which funds 5-10 hours of software development and maintenance work a month. Member fees have also funded social processes such as organising the community "walk-in" Kite-flying hours and a year long position for a financial administrator for the federation.
The Co-op Cloud federation receives donations via Open Collective for providing public internet services (e.g. git.coopcloud.tech which provides also free CI/CD) to members of our community who are not members of the federation.
# Attachments
## Attachments are not needed. Please ensure the proposal is self-contained. But in case you have supplementary material you wish to share (e.g. spreadsheets, diagrams, endorsements), you can do so here. (As HTML, PDF, ODF, images or plain text. Max 50 MB in total.)
### `work.pdf`
Links to specific work items, more context and a brief budget explanation.
#### Budget
Estimation system used:
- Small: 5-20 dev hours
- Medium: 20-40 dev hours
- Large: 40-80 dev hours
Times and totals are included below. Final totals are included at the end.
#### 1) Maintenance & Technical Debt
> Fix the oustanding (as of Nov 2026) 21 bugs in Abra reported by operators. These issues represent daily friction for operators and make deployment, maintenance and collaboration work more difficult to carry out.
All outstanding 21 `abra` bugs as of November 4 2026: [link](https://git.coopcloud.tech/toolshed/abra/issues?q=&type=all&state=open&labels=330&milestone=0&assignee=&poster=).
We estimate an average of 8 hours per bug, small. 8 hours * 21 = 168 hours.
> Upstream our patches to Docker CLI and upgrade to the latest Docker libraries in Abra so that we maintain parity with upstream developments.
Here is our [`docker-cli` patch](https://git.coopcloud.tech/toolshed/abra/src/commit/550c9aa48fcdb587e03d43cceb261018095ff6ed/go.mod#L155) which we would like to upstream. The change has to do with allowing multiple protocols ingress on a single port. This is a known issue in upstream and we believe it needs more social pressure applied to be merged and released.
We estimate 20 hours, medium.
> Migrate from the incomplete xgettext-go to the official xgettext Golang bindings.
Here is the specific issue which deals with the need to migrate to upstream `xgettext` for a complete native implementation which supports Go: [link](https://git.coopcloud.tech/toolshed/abra/issues/688).
We estimate 30 hours, medium.
> Migrate to the new compose-go standard library in Abra so that we maintain parity with one of the central open standards which we rely on. This decouples our sole reliance on the Docker Swarm runtimes (out-of-date) implementation of the Compose Specification.
Here is the issue which outlines the limitations we face by not migrating away from the (out-of-date) Docker Swarm implementation of the Compose standard: [link](https://git.coopcloud.tech/toolshed/abra/issues/492).
We estimate 30 hours, medium.
**TOTAL**: 248 hours
#### 2) Performance
> The Co-op Cloud catalogue is a single JSON file which is becoming a performance bottleneck for Abra. We will convert catalogue generation and querying into a lightweight API service which always provides fast results regardless of quantity of app configs which integrates with Abra.
Here is the general outline of an implementation which we would like to bring forward: [link](https://git.coopcloud.tech/toolshed/almanac).
We estimate 60 hours, large.
- Abra development follows the "make it work, make it right, make it fast" principle. We are now in the "make it fast" phase. Abra requires profiling to understand where it is doing double work and unecessary bookkeeping. The goal is to return to a high speed CLI experience.
This work requires investigation and the establishment of what is feasible to fix within this work item.
We estimate 40 hours, medium.
**TOTAL**: 100 hours
#### 3) Distributed Collaboration and Operations
> Co-op Cloud currently "takes over" an existing server, imposing a single web proxy choice. Operators need proxy agnosticism, supporting several popular proxies seamlessly (e.g. Traefik, Nginx, Caddy).
Proxy agnosticism: [link](https://git.coopcloud.tech/toolshed/organising/issues/686), Related issue: [`#388`](https://git.coopcloud.tech/toolshed/organising/issues/388)
We estimate 60 hours, large.
> App configs rely on a ".env" file format (FOO=BAR). This format was chosen for interoperability but with the increase of config options, these files become difficult to manage. We will migrate to a YAML configuration for maintainers which brings new organisation benefits while keeping the .env file for operators.
The new YAML format for free software app config maintainers: [link](https://git.coopcloud.tech/toolshed/organising/issues/695)
We estimate 60 hours, large.
> When maintainers upgrade app configs, changes must be propagated to operator configs (e.g. a new config knob for database perf). This is currently a manual process. We will implement an automatic upgrade in Abra with a change overview for operators.
Propagating app config changes to operators automatically: [link](https://git.coopcloud.tech/toolshed/abra/issues/504)
We estimate 40 hours, medium.
> With the development of the new Docker Secrets Engine, it is possible to decouple the secret injection of Abra from the Docker Swarm runtime. Implementing secret engine support in Abra would reduce our reliance on the Docker runtime overall due to its pluggable interface with independent secret management tools.
No issue has been created for this beyond a growing consensus in the "Research & Development" community chat the secrets engine migration could be our first step to begin the migrating away from Docker Swarm. This work item requires working closely with federation members to understand our first plugin implementation and backwards compatible migration path forward within the app configs. The Pass CLI tool is commonly used by federation members for secret management and represents a simple target for this item.
We estimate 80 hours, large.
> Due to increasing numbers of app config maintainers, there are more pull request review workflows happening on git.coopcloud.tech. The Abra app config upgrade and release functionality does not conveniently support a pull request workflow. We will implement this workflow support in Abra by allowing Abra to produce and adapt to diffs which are designed for a review, approve and apply workflow.
Here is the issue: [link](https://git.coopcloud.tech/toolshed/abra/issues/806)
We estimate 30 hours, medium.
> Co-op Cloud operators currently sync their app config changes manually using Git (e.g. deployment version). We will implement a peer-to-peer sync in Abra which allows operators to inform each other about deployment and maintenance operations directly on the command-line. We aim to implement this using p2panda-go (Iroh under the hood).
This is a longrunning thread over the years which has recently been resurrected: [link](https://git.coopcloud.tech/toolshed/abra/issues/809). This discussion is not finished and does not represent the "third way" of moving away from Git as data store and using p2panda as peer-to-peer sync strategy.
We estimate 80 hours, large.
> Alakazam is a longrunning prototype used by KolliCloud (free software hosting platform built on Co-op Cloud) which implements a wrapper of Abra and supports the deployment of multiple apps in a single "stack" (e.g. deploy Nextcloud, BigBlueButton and Rauthy SSO with shared config). It has reached maturity and been adopted by the Co-op Cloud federation as an official community tool. There is a desire to bring the feature set of Alakazam back to Abra, ending the "wrapper of a wrapper" prototype implementation and giving all Co-op Cloud operators first-class support for seamless "stack" deployments without needing an additional tool.
This effort has been decided by on federation members in [`R037`](https://docs.coopcloud.tech/federation/resolutions/passed/037/). The exact feature set needs to be drawn up and a design for how these features will be merged into Abra will also need to be published and agreed upon. The Alakazam feature set is stable since years, so we believe this should not be a moving target.
We estimate 100 hours, large.
**TOTAL**: 450 hours
#### Final totals
* 248 + 100 + 450 = 798 total hours
* 798 * 60 EUR hourly rate = 47880 EUR
### `cv.pdf`
The CV of the submitter.
# AI disclosure
## Did you use generative AI in writing this proposal?
No.
## Does your project build on an existing codebase that already includes, or expects to gain, a significant amount of code generated by AI?
No.
mkdocs.yml
+1
View File
@@ -167,6 +167,7 @@ nav:
- federation/funding/user-operated-internet.md
- federation/funding/sovereign-tech-fund.md
- federation/funding/nlnet-codesupply.md
- federation/funding/nlnet-restack.md
- "Abra":
- abra/index.md
- "Install": abra/install.md