Collective maintenance often means hidden app upgrades #434

Closed
opened 2023-03-16 15:56:24 +00:00 by decentral1se · 11 comments
Owner
  • worker 1 deploys app with version a
  • recipe maintainers upgrade app to version b
  • worker 2 undeploys app
  • worker re-deploys app and gets latest version b, not version a
  • hidden upgrade

Hinted at in #338 but the penny finally dropped. The collective mainteannce case is actually pretty unstable. We've been silently upgrading apps for months probably 😆

- worker 1 deploys app with version a - recipe maintainers upgrade app to version b - worker 2 undeploys app - worker re-deploys app and gets latest version b, not version a - hidden upgrade Hinted at in https://git.coopcloud.tech/coop-cloud/organising/issues/338 but the penny finally dropped. The collective mainteannce case is actually pretty unstable. We've been silently upgrading apps for months probably 😆
decentral1se added the
bug
label 2023-03-16 15:56:24 +00:00
Owner

I have spent a good deal of time thinking about how to deal with this such that redeploys always use the same version unless otherwise specified, and I think I have a pretty good idea about how to do it.

We should record the version of the recipe in the .env file since this is the only state we store that's not in the docker demon. This would also allow us to start defining the shape of how to describe source/recipe/version information for adding additional/better sources (I've been thinking about this a while too).

I feel like what we should do is replace the TYPE environment variable with something like COOP_RECIPE, and add version information. We can it only change the COOP_RECIPE if a abra app upgrade is performed (or the user updates it manually).

I think the syntax should be something like source:type:version, (eg ccf:gitea:2.0.1+1.18.2-rootless) but we need to have some sort of semantics for parsing these and having parts missing for user convenience. We should also just leave out the source for now but keep it in mind for future.

Perhaps --chaos would imply using the current checked-out version, and update the version field to something like local-shorthash so it could be redeployed with that version (given your local repository has that hash).

For bug compatibility, a TYPE variable could specify the most current version of the recipe, but as soon as it's deployed with the current version a COOP_RECIPE should be added to the config.

I have spent a good deal of time thinking about how to deal with this such that redeploys always use the same version unless otherwise specified, and I think I have a pretty good idea about how to do it. We should record the version of the recipe in the .env file since this is the only state we store that's not in the docker demon. This would also allow us to start defining the shape of how to describe source/recipe/version information for adding additional/better sources (I've been thinking about this a while too). I feel like what we should do is replace the `TYPE` environment variable with something like `COOP_RECIPE`, and add version information. We can it only change the `COOP_RECIPE` if a `abra app upgrade` is performed (or the user updates it manually). I think the syntax should be something like `source:type:version`, (eg `ccf:gitea:2.0.1+1.18.2-rootless`) but we need to have some sort of semantics for parsing these and having parts missing for user convenience. We should also just leave out the source for now but keep it in mind for future. Perhaps `--chaos` would imply using the current checked-out version, and update the version field to something like `local-shorthash` so it could be redeployed with that version (given your local repository has that hash). For bug compatibility, a TYPE variable could specify the most current version of the recipe, but as soon as it's deployed with the current version a COOP_RECIPE should be added to the config.
Author
Owner

Nice @cas!

Does the ccf in the ccf:gitea:2.0.1+1.18.2-rootless do something? My assumption is that source leaves the door open for other recipes catalogues or? Should that correspond to a URL or something?

TYPE is currently a static string set in the .env files to mark the recipe name. COOP_RECIPE would be a dynamic string? What would be the default string for new recipes? I guess we could do a search/replace on recipe new for latest catalogue version.

There is potential for a bikeshed on COOP_RECIPE since there is more than the recipe name contained in it. Even more than the version in your proposal. It's more like a RECIPE_SPEC 😄

We could have abra support TYPE/COOP_RECIPE in parallel while things switch over. This will cost us some time to implement. So, I guess we can come up with a proposal to raise awareness about it to the rest of the federation.

--chaos shorthash logic soundds good.

Nice @cas! Does the `ccf` in the `ccf:gitea:2.0.1+1.18.2-rootless` do something? My assumption is that `source` leaves the door open for other recipes catalogues or? Should that correspond to a URL or something? `TYPE` is currently a static string set in the `.env` files to mark the recipe name. `COOP_RECIPE` would be a dynamic string? What would be the default string for new recipes? I guess we could do a search/replace on `recipe new` for latest catalogue version. There is potential for a bikeshed on `COOP_RECIPE` since there is more than the recipe name contained in it. Even more than the version in your proposal. It's more like a `RECIPE_SPEC` 😄 We could have `abra` support `TYPE`/`COOP_RECIPE` in parallel while things switch over. This will cost us some time to implement. So, I guess we can come up with a proposal to raise awareness about it to the rest of the federation. `--chaos` shorthash logic soundds good.
Owner

To be a little clearer:

My idea is that the ccf is the "source". It should be left out for the time being. The source would be basically a label attached to a recipe catalog URL. The idea being that at some point we will want to support an arbitrary collection of sources.

TYPE should be changed into a dynamic string created from the COOP_RECIPE which would be effectively set by the maintainer of the recipe (so that any given deployment would be stuck to a particular version unless intentionally upgraded).

I think that supporting both as a transitional stage is important. I think my outline as to how deployments could be transitioned to the recipe specifier would work but it should perhaps warn the user and prompt for the pinning/upgrade step to happen (so if your app deployment has a TYPE specifier, it would say "Hay, we're moving to COOP_RECIPE so that things don't get upgraded unexpectedly on deploy, is it ok to pin your deployment at <VERSION>? You can upgrade later with upgrade".

To be a little clearer: My idea is that the `ccf` is the "source". It should be left out for the time being. The source would be basically a label attached to a recipe catalog URL. The idea being that at some point we will want to support an arbitrary collection of sources. `TYPE` should be changed into a dynamic string created from the `COOP_RECIPE` which would be effectively set by the maintainer of the recipe (so that any given deployment would be stuck to a particular version unless intentionally upgraded). I think that supporting both as a transitional stage is important. I think my outline as to how deployments could be transitioned to the recipe specifier would work but it should perhaps warn the user and prompt for the pinning/upgrade step to happen (so if your app deployment has a TYPE specifier, it would say "Hay, we're moving to COOP_RECIPE so that things don't get upgraded unexpectedly on deploy, is it ok to pin your deployment at `<VERSION>`? You can upgrade later with `upgrade`".
Author
Owner

@cas I'm fine with including the source for now but we might wanna sketch out how it is used first to avoid more breaking changes. The need for decentralised catalogue support is coming up again and again. Maybe ccf is just a key in ~/.abra/catalogue/recipes.json or a filename ~/.abra/catalogue/ccf.json 🤷‍♀️

@cas I'm fine with including the source for now but we might wanna sketch out how it is used first to avoid more breaking changes. The need for decentralised catalogue support is coming up again and again. Maybe `ccf` is just a key in `~/.abra/catalogue/recipes.json` or a filename `~/.abra/catalogue/ccf.json` 🤷‍♀️
decentral1se added the
abra
label 2023-06-08 09:19:24 +00:00
decentral1se added this to the Medium/large enhancements project 2023-06-08 09:25:53 +00:00
Author
Owner

FYI @cas I'm gonna try bring this to discussion in the coming fedi meeting (date poll coming another day for date/time, see matrix for more). Would you be up for working on this in some capacity? I'd be down to help either in implementation or testing against an implementation. I'd love to see us make progress here because it's such a huge issue.

FYI @cas I'm gonna try bring this to discussion in the coming fedi meeting (date poll coming another day for date/time, see matrix for more). Would you be up for working on this in some capacity? I'd be down to help either in implementation or testing against an implementation. I'd love to see us make progress here because it's such a huge issue.
Owner

FYI @cas I'm gonna try bring this to discussion in the coming fedi meeting (date poll coming another day for date/time, see matrix for more). Would you be up for working on this in some capacity? I'd be down to help either in implementation or testing against an implementation. I'd love to see us make progress here because it's such a huge issue.

I'm 1000% up for working on this, since it bites us every time i have to do maintenance or whatever it's just real annoying.

> FYI @cas I'm gonna try bring this to discussion in the coming fedi meeting (date poll coming another day for date/time, see matrix for more). Would you be up for working on this in some capacity? I'd be down to help either in implementation or testing against an implementation. I'd love to see us make progress here because it's such a huge issue. I'm 1000% up for working on this, since it bites us every time i have to do maintenance or whatever it's just real annoying.
Member

This would also solve #440 updating only the .env file using abra app deploy -f without changing the deployed recipe version.
I can think of issues if the local COOP_RECIPE version tag and the deployed version differs for some reasons. There should be a check if both versions are equal and if they differ set the COOP_RECIPE to the deployed version.

This would also solve https://git.coopcloud.tech/coop-cloud/organising/issues/440 updating only the .env file using `abra app deploy -f` without changing the deployed recipe version. I can think of issues if the local `COOP_RECIPE` version tag and the deployed version differs for some reasons. There should be a check if both versions are equal and if they differ set the `COOP_RECIPE` to the deployed version.
decentral1se self-assigned this 2023-10-04 13:29:46 +00:00
Author
Owner

I'm starting to think some part of #467 is a prerequisite for this. Because, yeh, if we're updating a COOP_RECIPE but not syncing it to a remote (and syncing it on other machines of fellow workers), then it's next to useless?

I'm gonna try to draw up a proposal for this because it's getting kinda convoluted in my brain 😆

I'm starting to think some part of https://git.coopcloud.tech/coop-cloud/organising/issues/467 is a prerequisite for this. Because, yeh, if we're updating a `COOP_RECIPE` but not syncing it to a remote (and syncing it on other machines of fellow workers), then it's next to useless? I'm gonna try to draw up a proposal for this because it's getting kinda convoluted in my brain 😆
decentral1se modified the project from Medium/large enhancements to (deleted) 2023-10-06 09:27:58 +00:00
Owner

I agree having the data stored somewhere would be useful. i haven't thought about thisl for a bit but my assumption is that the operators would be handling that pathway on their own anyway so that would take care of synchronizing for them. I don't think it's next to useless, because it still constrains the way abra will act, but I see the argument that we should be providing a mechanism for obtaining that information if not directly from the server in question, then somewhere.

I agree having the data stored somewhere would be useful. i haven't thought about thisl for a bit but my assumption is that the operators would be handling that pathway on their own anyway so that would take care of synchronizing for them. I don't think it's next to useless, because it still constrains the way abra will act, but I see the argument that we should be providing a mechanism for obtaining that information if not directly from the server in question, then somewhere.
Author
Owner

@cas Sorry, that was poorly worded on my part. I don't know why I used that phrasing ("useless") 🤔 Certainly not.

So, yes, that's it. For the individual operator, the COOP_RECIPE will help them pin deploys. But without sync, it will not help a team pin deploys.

Once I realised that, I ended up on #467 and then #267 (comment) which are all good ideas.

I think we need to have a collective design discussion on this one whenever the next meet happens.

@cas Sorry, that was poorly worded on my part. I don't know why I used that phrasing ("useless") 🤔 Certainly not. So, yes, that's it. For the individual operator, the `COOP_RECIPE` will help them pin deploys. But without sync, it will not help a team pin deploys. Once I realised that, I ended up on https://git.coopcloud.tech/coop-cloud/organising/issues/467 and then https://git.coopcloud.tech/coop-cloud/organising/issues/267#issuecomment-18267 which are all good ideas. I think we need to have a collective design discussion on this one whenever the next meet happens.
Author
Owner

Merging into #467 👍

Merging into https://git.coopcloud.tech/coop-cloud/organising/issues/467 👍
Sign in to join this conversation.
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: coop-cloud/organising#434
No description provided.