Smoother Support for hacking on recipes #606

New Issue

p4u1 · 2024-04-08T10:04:44Z

p4u1 commented

2024-04-08 10:04:44 +00:00

Hacking on recipes as an operator is a task, that occurs quite regularly. Currently the concept of "take the local git repo for that recipe as it is and don't do any magic with it" is supported with the --chaos flag. This works in general but has some inconveniences:

While hacking on a recipe you have to provide the --chaos flag for every command
Say I want to hack on the wordpress recipe for my deployment A. But I also have a wordpress deployment B, which I can't just continue deploying normally
When multiple people work on one app there is no way in telling the other person, that I'm hackin on a recipe automatically. Instead I have to say to them: "hey comrade, I'm currently hacking on the nextcloud recipe. Please don't deploy it right now, otherwise we break it again"

To mitigate these problems I propose to have a new abra app sub command:

abra app hack mywordpress.example.com

This will do the following things:

Clone/Copy the wordpress recipe into a new folder which is specific for the app I'm hacking on. For Example:

~/.abra/recipes/wordpress-mywordpress.example.com

The abra app hack command should output that directory, to easily navigate there.

It should be possible to give a fixed location with a --path flag:

abra app hack --path ~/.abra/recipes/wordpress2 mywordpress.example.com

In that case it looks for a git repository in that location. Only when no git repository is found it will do the cloning/copying

Modify the .env file for that app to include the local path of the git repository. Something like this:
Before:

RECIPE=wordpress:1.3

After:

# RECIPE=wordpress:1.3
RECIPE=~/.abra/recipes/wordpress-mywordpress.example.com:chaos

After done hacking you either manually remove the chaos line in the .env file or you call

abra app hack --done mywordpress.example.com

This should more or less solve the issues described above:

Issue 1: You now only have to indicate once that you want to start hacking on a recipe and once for ending
Issue 2: The hacked recipe is now specific to the app, that I'm working on (while still possible to use the same hacked recipe for all deployments with the --path flag
Issue 3: If I hack on a recipe and commit the .env file with the chaos modification, my comrades will know not to touch this recipe (have to think some more on how to better prevent accidents here)

Hacking on recipes as an operator is a task, that occurs quite regularly. Currently the concept of "take the local git repo for that recipe as it is and don't do any magic with it" is supported with the `--chaos` flag. This works in general but has some inconveniences: 1. While hacking on a recipe you have to provide the `--chaos` flag for every command 2. Say I want to hack on the wordpress recipe for my deployment A. But I also have a wordpress deployment B, which I can't just continue deploying normally 3. When multiple people work on one app there is no way in telling the other person, that I'm hackin on a recipe automatically. Instead I have to say to them: "hey comrade, I'm currently hacking on the nextcloud recipe. Please don't deploy it right now, otherwise we break it again" To mitigate these problems I propose to have a new abra app sub command: ``` abra app hack mywordpress.example.com ``` This will do the following things: 1. Clone/Copy the wordpress recipe into a new folder which is specific for the app I'm hacking on. For Example: ``` ~/.abra/recipes/wordpress-mywordpress.example.com ``` The `abra app hack` command should output that directory, to easily navigate there. It should be possible to give a fixed location with a `--path` flag: ``` abra app hack --path ~/.abra/recipes/wordpress2 mywordpress.example.com ``` In that case it looks for a git repository in that location. Only when no git repository is found it will do the cloning/copying 2. Modify the `.env` file for that app to include the local path of the git repository. Something like this: _Before:_ ``` RECIPE=wordpress:1.3 ``` _After:_ ``` # RECIPE=wordpress:1.3 RECIPE=~/.abra/recipes/wordpress-mywordpress.example.com:chaos ``` After done hacking you either manually remove the chaos line in the `.env` file or you call ``` abra app hack --done mywordpress.example.com ``` This should more or less solve the issues described above: - Issue 1: You now only have to indicate once that you want to start hacking on a recipe and once for ending - Issue 2: The hacked recipe is now specific to the app, that I'm working on (while still possible to use the same hacked recipe for all deployments with the `--path flag` - Issue 3: If I hack on a recipe and commit the `.env` file with the chaos modification, my comrades will know not to touch this recipe (have to think some more on how to better prevent accidents here)

👍 1

p4u1 added the

abra

label 2024-04-08 10:04:44 +00:00

basebuilder commented

2024-04-08 11:09:51 +00:00

Great proposal @p4u1 thank you!! I totally support this concept 👍

decentral1se commented

2024-04-08 13:18:14 +00:00

For others, please note, RECIPE=wordpress:1.3 is a syntax based on a WIP proposal described in coop-cloud/organising#533 (comment)

Question: What does :chaos do in RECIPE=~/.abra/recipes/wordpress-mywordpress.example.com:chaos? To check, I assume this means "always use latest commit" because you might want a RECIPE=~/... which uses the default behaviour (checkout latest tag or HEAD)?

I think issue 3 is a broader discussion which was attempted to be addressed by R013 (which needs a re-write /cc @3wordchant @wolcen). If you commit locally your "hack"/"chaos" marker to the .env, you need to sync that. If you're friends are also hacking at the same time, they should get a "oh no someone is hacking" when they try to deploy and the .env files sync.

I think there are several things which will be enabled by the RECIPE=... syntax proposal @p4u1 and we should aim for that first. Then we should try to make a plan what use-cases to work on after, towards actually stable hacking/deploy ops in Co-op Cloud. So many moving parts...

Thanks for raising & prototyping this UI 👏

> *For others, please note, `RECIPE=wordpress:1.3` is a syntax based on a WIP proposal described in https://git.coopcloud.tech/coop-cloud/organising/issues/533#issuecomment-19038* Question: What does `:chaos` do in `RECIPE=~/.abra/recipes/wordpress-mywordpress.example.com:chaos`? To check, I assume this means "always use latest commit" because you might want a `RECIPE=~/...` which uses the default behaviour (checkout latest tag or `HEAD`)? I think issue 3 is a broader discussion which was attempted to be addressed by [R013](https://docs.coopcloud.tech/federation/resolutions/in-progress/013/) (which needs a re-write /cc @3wordchant @wolcen). If you commit locally your "hack"/"chaos" marker to the `.env`, you need to sync that. If you're friends are also hacking at the same time, they should get a "oh no someone is hacking" when they try to deploy and the `.env` files sync. I think there are several things which will be enabled by the `RECIPE=...` syntax proposal @p4u1 and we should aim for that first. Then we should try to make a plan what use-cases to work on after, towards actually stable hacking/deploy ops in Co-op Cloud. So many moving parts... Thanks for raising & prototyping this UI 👏

3wordchant commented

2024-04-11 03:44:18 +00:00

@p4u1 thanks so much for proposing, awesome efforts as usual 🙏

While hacking on a recipe you have to provide the --chaos flag for every command

Say I want to hack on the wordpress recipe for my deployment A. But I also have a wordpress deployment B, which I can't just continue deploying normally

When multiple people work on one app there is no way in telling the other person, that I'm hackin on a recipe automatically. Instead I have to say to them: "hey comrade, I'm currently hacking on the nextcloud recipe. Please don't deploy it right now, otherwise we break it again"

These are all certainly annoying problems which I also run into regularly.

My questions to you are: could all 3 problems be solved adequately by Resolution 013's OPERATOR_SYNC_VERSION=00acabacab? And do you agree that there are advantages to doing it that way? I am certainly open to this proposed if your answer to either question is "no".

Example

I want to hack on the Wordpress recipe, for example to add sqlite support.

abra recipe fetch wordpress
cd ~/.abra/recipes/wordpress
git checkout -b feat/sqlite
# hackety hack
git commit -m "Add sqlite support"
# commit ID 00acabacab
abra app sync wordpress.example.com --chaos

Now ~/.abra/servers/example.com/wordpress.example.com.env contains OPERATOR_SYNC_VERSION=00acabacab.

If anyone runs abra app deploy wordpress.example.com, the same 00acabacab commit (on the feat/sqlite branch) is deployed. --chaos is not required.

If anyone (including me) runs abra app upgrade wordpress.example.com, there is a warning that the upgrade will move away from a chaos version.

Deployments of wordpress2.example.com are unaffected, and can be upgraded normally without warning.

Advantages

Other collaborators can still redeploy wordpress.example.com.

The deployed version of wordpress.example.com is stored in version control.

Disadvantages / mitigations

The one downside I can think of is that it could be annoying that if I deploy / upgrade wordpress2.example.com, it will switch ~/.abra/recipes/wordpress away from the feat/sqlite branch to a checkout of a specific tag on main. My suggested mitigation for this (bearing in mind that no data loss would ever occur here) is to document that git-worktree is recommended for long-term recipe hacking.

@p4u1 thanks so much for proposing, awesome efforts as usual 🙏 > 1. While hacking on a recipe you have to provide the --chaos flag for every command > 2. Say I want to hack on the wordpress recipe for my deployment A. But I also have a wordpress deployment B, which I can't just continue deploying normally > 3. When multiple people work on one app there is no way in telling the other person, that I'm hackin on a recipe automatically. Instead I have to say to them: "hey comrade, I'm currently hacking on the nextcloud recipe. Please don't deploy it right now, otherwise we break it again" These are all certainly annoying problems which I also run into regularly. My questions to you are: could all 3 problems be solved adequately by Resolution 013's `OPERATOR_SYNC_VERSION=00acabacab`? And do you agree that there are advantages to doing it that way? I am certainly open to this proposed if your answer to either question is "no". ## Example I want to hack on the Wordpress recipe, for example to add sqlite support. ``` abra recipe fetch wordpress cd ~/.abra/recipes/wordpress git checkout -b feat/sqlite # hackety hack git commit -m "Add sqlite support" # commit ID 00acabacab abra app sync wordpress.example.com --chaos ``` Now `~/.abra/servers/example.com/wordpress.example.com.env` contains `OPERATOR_SYNC_VERSION=00acabacab`. If anyone runs `abra app deploy wordpress.example.com`, the same `00acabacab` commit (on the `feat/sqlite` branch) is deployed. `--chaos` is not required. If anyone (including me) runs `abra app upgrade wordpress.example.com`, there is a warning that the upgrade will move away from a chaos version. Deployments of `wordpress2.example.com` are unaffected, and can be upgraded normally without warning. ## Advantages Other collaborators can still redeploy `wordpress.example.com`. The deployed version of `wordpress.example.com` is stored in version control. ## Disadvantages / mitigations The one downside I can think of is that it could be annoying that if I deploy / upgrade `wordpress2.example.com`, it will switch `~/.abra/recipes/wordpress` away from the `feat/sqlite` branch to a checkout of a specific tag on `main`. My suggested mitigation for this (bearing in mind that no data loss would ever occur here) is to document that [`git-worktree`](https://git-scm.com/docs/git-worktree) is recommended for long-term recipe hacking.

decentral1se commented

2024-10-31 19:18:29 +00:00

We're further along now with #533.

coop-cloud/organising#467 is still outstanding though.

We're further along now with https://git.coopcloud.tech/toolshed/organising/issues/533. https://git.coopcloud.tech/coop-cloud/organising/issues/467 is still outstanding though.

decentral1se commented

2025-08-29 10:25:30 +00:00

I believe this is resolved with the 2 following approaches which have been in testing for months now but not really well documented. You can use the abra.yml and "remote recipes" to have a collaborative, per-project, recipe-hacking-friendly setup now.

I think a lot of these "tips & tricks" for smoother workflows while hacking/collaborating with others could be better understood and we'll have to see we can bring this to the forefront with the documentation. I am hoping we will make progress on #665 in the near future.

Gonna close for now. Feel free to re-open or open specific tickets for improvements (e.g. toolshed/abra#468) or missing parts. Thanks for all the input 💝

I believe this is resolved with the 2 following approaches which have been in testing for months now but not really well documented. You can use the `abra.yml` and "remote recipes" to have a collaborative, per-project, recipe-hacking-friendly setup now. * https://docs.coopcloud.tech/operators/handbook/#configure-abra-with-abrayml * https://docs.coopcloud.tech/operators/handbook/#remote-recipes I think a lot of these "tips & tricks" for smoother workflows while hacking/collaborating with others could be better understood and we'll have to see we can bring this to the forefront with the documentation. I am hoping we will make progress on https://git.coopcloud.tech/toolshed/organising/issues/665 in the near future. Gonna close for now. Feel free to re-open or open specific tickets for improvements (e.g. https://git.coopcloud.tech/toolshed/abra/issues/468) or missing parts. Thanks for all the input 💝

❤️ 1

decentral1se closed this issue

2025-08-29 10:25:30 +00:00

Sign in to join this conversation.

4 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/organising#606