WIP: feat: use compose-go #816

2026-04-02T15:46:59Z

decentral1se commented

2026-04-02 15:46:59 +00:00

An absolutely diabolical rabbit hole that I wouldn't wish on my worst enemy.

Docker never implemented it and so we were stuck using their custom solution for loading compose config files. This led to #492. compose-go has a better API which supports our various use-cases (loading the compose files via a loaded app, loaded recipe or just directly from the FS) and actually implements the compose spec, so it's a big win!

ToDo

Unfuck final TODOs
Fix unit tests
Fix integration tests
- bail if recipe lint errors and no --chaos
- does not crash when docker image has no tag
- detect no volumes to remove
- remove volumes
- list volumes
- remove no volumes
- remove single volume
- remove single volume doesn't delete similar name
- recipe lint
- release patch bump
- release new recipe: development release
- push during release fails
- release, fail, release: works
- upgrade patch
- upgrade minor
- upgrade and commit
WTF API surface for parsing compose configs: in app context, recipe context, no context
Manual testing
Document migration notes

Notes

The version: ... linting rule is going away because it's deprecated
We may see a regression in obscure parsing error messages due to this change

An absolutely diabolical rabbit hole that I wouldn't wish on my worst enemy. Docker [never implemented it](https://github.com/docker/cli/pull/4863) and so we were stuck using their custom solution for loading compose config files. This led to https://git.coopcloud.tech/toolshed/abra/issues/492. `compose-go` has a better API which supports our various use-cases (loading the compose files via a loaded app, loaded recipe or just directly from the FS) and actually implements the compose spec, so it's a big win! ### ToDo - [x] Unfuck final TODOs - [x] Fix unit tests - [x] Fix integration tests - [x] `bail if recipe lint errors and no --chaos` - [x] `does not crash when docker image has no tag` - [x] `detect no volumes to remove` - [x] `remove volumes` - [x] `list volumes` - [x] `remove no volumes` - [x] `remove single volume` - [x] `remove single volume doesn't delete similar name` - [x] `recipe lint` - [x] `release patch bump` - [x] `release new recipe: development release` - [x] `push during release fails` - [x] `release, fail, release: works` - [x] `upgrade patch` - [x] `upgrade minor` - [x] `upgrade and commit` - [ ] WTF API surface for parsing compose configs: in app context, recipe context, no context - [ ] Manual testing - [ ] Document migration notes ### Notes * The `version: ...` linting rule is going away because it's deprecated * We may see a regression in obscure parsing error messages due to this change

😆 1

decentral1se changed title from ~~WIP: feat: use compose-go for compose parsing~~ to WIP: feat: use compose-go

2026-04-04 16:51:13 +00:00

decentral1se force-pushed fix/492 from 7bd9d667a0 to f2e813e2e3

2026-04-04 16:51:16 +00:00

Compare

decentral1se force-pushed fix/492 from f2e813e2e3 to 5eea459bde

2026-04-04 17:20:03 +00:00

Compare

decentral1se force-pushed fix/492 from 5eea459bde to 5cf42a8104

2026-04-04 17:37:19 +00:00

Compare

decentral1se force-pushed fix/492 from 5cf42a8104 to 94624bb16d

2026-04-04 17:39:32 +00:00

Compare

decentral1se referenced this pull request from coop-cloud/traefik

2026-07-06 17:28:21 +00:00

WIP: feat: adds iocaine config #114

devydave added 17 commits 2026-08-13 23:02:37 +00:00

feat: adds local vm setup that can reach each other

continuous-integration/drone/push Build is passing

Details

4e42d4fb9e

feat: adds running integrations test suite

continuous-integration/drone/push Build is passing

Details

continuous-integration/drone/pr Build is passing

Details

9bd8f4962a

refactor: remove traefik deployment because it is not needed 0c0e16914a

refactor: only export ssh auth socket when not existing beee7bd3f2

refactor: better name for inspect function b5e22790be

refactor: better variable name 38d13eae73

refactor: adjusting hosts generation

continuous-integration/drone/push Build is passing

Details

5e0dac6508

refactor: simplify config and remove hom-manager dependency

continuous-integration/drone/push Build is passing

Details

1ccbb31b70

feat: adds host module

continuous-integration/drone/push Build is passing

Details

5766a25144

fix: remove personal terminal choice

continuous-integration/drone/push Build is passing

Details

a8c8df8604

refactor: make path editable without nix and impure 02cbf7c014

docs: add setup documentation

continuous-integration/drone/push Build is passing

Details

bb3c7576f8

docs: formatting

continuous-integration/drone/push Build is passing

Details

21dccc895d

docs: more formatting and typo

continuous-integration/drone/push Build is passing

Details

a28cc7afce

merge 4c72dcb49c

chore: mod vendor 0e5b42da17

fix: deploy labels nil check

continuous-integration/drone/push Build is failing

Details

8c72890a63

devydave added 2 commits 2026-08-14 11:02:52 +00:00

fix: prefixes volumes with stack name instead of default fallback for deployments 38f7720bd6

fix: updating labels and tags

continuous-integration/drone/push Build is failing

Details

527813c924

devydave commented

2026-08-14 11:05:36 +00:00

@decentral1se I think the integration tests are fixed.
What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD

@decentral1se I think the integration tests are fixed. What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD

decentral1se commented

2026-08-14 11:59:56 +00:00

@devydave amazing, well done!

What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD

It means to try to unify somehow these "we need to parse compose configs" interface.

We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. appPkg.GetAppComposeConfig(composeFiles, app.Env)) 2) we have a recipe (e.g. recipe.GetComposeConfig()) 3) we have neither and we just need to read the file (e.g. loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})).

This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves?

This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers.

@devydave amazing, well done! > What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD It means to try to unify somehow these "we need to parse compose configs" interface. We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. `appPkg.GetAppComposeConfig(composeFiles, app.Env)`) 2) we have a recipe (e.g. `recipe.GetComposeConfig()`) 3) we have neither and we just need to read the file (e.g. `loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})`). This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves? This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers.

devydave commented

2026-08-14 13:54:53 +00:00

@devydave amazing, well done!

What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD

It means to try to unify somehow these "we need to parse compose configs" interface.

We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. appPkg.GetAppComposeConfig(composeFiles, app.Env)) 2) we have a recipe (e.g. recipe.GetComposeConfig()) 3) we have neither and we just need to read the file (e.g. loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})).

This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves?

This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers.

I agree with that, while fixing the remaining tests it was quite confusing having multiple places where compose files are parsed.
I will take a look, if I can add unify the parsing, that can be simply adjusted by the caller.

> @devydave amazing, well done! > > > What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD > > It means to try to unify somehow these "we need to parse compose configs" interface. > > We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. `appPkg.GetAppComposeConfig(composeFiles, app.Env)`) 2) we have a recipe (e.g. `recipe.GetComposeConfig()`) 3) we have neither and we just need to read the file (e.g. `loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})`). > > This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves? > > This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers. I agree with that, while fixing the remaining tests it was quite confusing having multiple places where compose files are parsed. I will take a look, if I can add unify the parsing, that can be simply adjusted by the caller.

🚀 1

devydave added 5 commits 2026-08-14 16:14:48 +00:00

refactor: adjust replace to replaceall and log file name that will be written 8eba47a3e1

refactor: simplify code d01c5e7828

docs: explains reading stackname 2733b20d42

chore: update translations 16a789fd64

chore: adds missing translation deps

continuous-integration/drone/push Build is passing

Details

82a5dc35cc

devydave commented

2026-08-14 16:23:28 +00:00

@devydave amazing, well done!

What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD

It means to try to unify somehow these "we need to parse compose configs" interface.

We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. appPkg.GetAppComposeConfig(composeFiles, app.Env)) 2) we have a recipe (e.g. recipe.GetComposeConfig()) 3) we have neither and we just need to read the file (e.g. loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})).

This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves?

This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers.

I agree with that, while fixing the remaining tests it was quite confusing having multiple places where compose files are parsed.
I will take a look, if I can add unify the parsing, that can be simply adjusted by the caller.

After looking into it, the loading pattern of compose file is messy overall, I think it is the best to move it to a separate issue.

> > @devydave amazing, well done! > > > > > What does "WTF API surface for parsing compose configs: in app context, recipe context, no context" mean xD > > > > It means to try to unify somehow these "we need to parse compose configs" interface. > > > > We have 3 contexts with data in memory when loading compose configs: 1) we have an app (e.g. `appPkg.GetAppComposeConfig(composeFiles, app.Env)`) 2) we have a recipe (e.g. `recipe.GetComposeConfig()`) 3) we have neither and we just need to read the file (e.g. `loader.LoadCompose(loader.LoadConf{ComposeFiles: composeFiles})`). > > > > This is relevant because some completion happens of env vars, or some merging of config files, etc. etc. Can this API surface be made more understandable for our future selves? > > > > This could be a good internal refactor to carry out now while the issue is burning because it is exactly what made this migration so hilariously difficult. I could understand if you want to skip it however. It's up to you. Most of abra was cobbled together moving super fast, so even if this doesn't get done now, it's nice to mark an issue for a future refactor. It will most likely matter for future maintainers. > > I agree with that, while fixing the remaining tests it was quite confusing having multiple places where compose files are parsed. > I will take a look, if I can add unify the parsing, that can be simply adjusted by the caller. After looking into it, the loading pattern of compose file is messy overall, I think it is the best to move it to a separate issue.

👍 1

decentral1se commented

2026-08-15 09:37:08 +00:00

Great, go ahead and feel free to merge stuff, take the lead, open issues, etc. You don't need to ask.

For releaseing abra when you feel like things are stable, there are these docs:

https://docs.coopcloud.tech/abra/hack/#release-management

Thanks!

Great, go ahead and feel free to merge stuff, take the lead, open issues, etc. You don't need to ask. For releaseing abra when you feel like things are stable, there are these docs: > https://docs.coopcloud.tech/abra/hack/#release-management Thanks!

devydave referenced this pull request from coop-cloud/traefik

2026-08-17 14:37:44 +00:00

WIP: feat: adds iocaine config #114

devydave referenced this pull request from coop-cloud/forgejo-runner

2026-08-17 14:49:42 +00:00

Remove global port binding 44847 #38

All checks were successful

continuous-integration/drone/push Build is passing

Details

This pull request has changes conflicting with the target branch.