toolshed/organising
toolshed/organising
8
1
Fork 0
Code Issues 120 Pull Requests Releases Activity

Operator collaboration #467

New Issue
Open
opened 2023-07-19 16:32:47 +00:00 by yksflip · 20 comments
yksflip commented 2023-07-19 16:32:47 +00:00
Member
Copy Link

Hey,

As mentioned in the sharing abra - you're on your own! section, sharing server folders via git repos is a manual process.

This is very error prone and I forget it all the time. Ofc we could write some Makefile or anything to supplement this, but I think this would be nice for abra to have.

Also we tried to keep a manual Changelog for each server wich is very tedious! Couldn't abra also do that with a good git history?

here the rough idea:

  • abra server add foo
    introduces a git repo url parameter

  • abra app new foo
    git add / commit / push the new .env file

  • abra app config
    git add / commit / push the modified .env file

  • abra app deploy
    git commit --allow-empty -m "some useful informations about the deployment here?"

  • abra server changelog
    could show a git log with all the changes from a server

  • abra app changelog
    shows the git log of a individual app

  • maybe some command to leave a custom message

and absolutely git pull all the time.

this relates to #267 #271

Hey, As mentioned in the [_sharing abra - you're on your own!_ section]( https://docs.coopcloud.tech/operators/handbook/#sharing-abra), sharing server folders via git repos is a manual process. This is very error prone and I forget it all the time. Ofc we could write some Makefile or anything to supplement this, but I think this would be nice for abra to have. Also we tried to keep a manual Changelog for each server wich is very tedious! Couldn't abra also do that with a good git history? here the rough idea: * `abra server add foo` introduces a _git repo url_ parameter * `abra app new foo` git add / commit / push the new .env file * `abra app config` git add / commit / push the modified .env file * `abra app deploy` `git commit --allow-empty -m "some useful informations about the deployment here?"` * `abra server changelog` could show a `git log` with all the changes from a server * `abra app changelog` shows the `git log` of a individual app * maybe some command to leave a custom message and absolutely `git pull` all the time. this relates to #267 #271
yksflip added the
enhancement
abra
 labels 2023-07-19 16:33:40 +00:00
decentral1se commented 2023-07-19 22:26:40 +00:00
Owner
Copy Link

Wow, yeh, that could be incredibly useful for collectively doing stuff. I can just imagine all the go-git bugs now 😆 For the abra server add foo: introduces a git repo url parameter, i'm wondering how to support the "all the server files in one repo" use case which is maybe more of what coop-cloud/organising#267 is going for. Love the git command design so far!

Wow, yeh, that could be incredibly useful for collectively doing stuff. I can just imagine all the `go-git` bugs now 😆 For the `abra server add foo: introduces a git repo url parameter`, i'm wondering how to support the "all the server files in one repo" use case which is maybe more of what https://git.coopcloud.tech/coop-cloud/organising/issues/267 is going for. Love the git command design so far!
decentral1se added this to the (deleted) project 2023-07-19 22:26:53 +00:00
decentral1se commented 2023-07-25 11:23:48 +00:00
Owner
Copy Link

Yeh just not having a way to remember what was previously deployed is such a drag. I forget all the time instantly after deploying something and then am lost in the entropy. Idk where this complaint should go but I think solving just that, whether with Git or some other way via ideas in https://docs.coopcloud.tech/operators/handbook/#sharing-abra would be amazing.

Yeh just not having a way to remember what was previously deployed is such a drag. I forget all the time instantly after deploying something and then am lost in the entropy. Idk where this complaint should go but I think solving just that, whether with Git or some other way via ideas in https://docs.coopcloud.tech/operators/handbook/#sharing-abra would be amazing.
decentral1se commented 2023-10-05 12:58:43 +00:00
Owner
Copy Link

coop-cloud/organising#434 (comment)

https://git.coopcloud.tech/coop-cloud/organising/issues/434#issuecomment-18253
decentral1se self-assigned this 2023-10-05 12:59:01 +00:00
decentral1se changed title from operator collaboration / syncing the server directory / server changelog to Operator collaboration 2023-10-05 12:59:20 +00:00
decentral1se commented 2023-10-05 13:08:05 +00:00
Owner
Copy Link

and absolutely git pull all the time

@yksflip how will we deal with merge conflicts? Just bail out gracefully and be like "ok, now you're on your own again until you solve this shit"?

I've got a proposal WIP for this https://pad.riseup.net/p/VewCbdPI-_RWR7ooe6v0-keep

> and absolutely git pull all the time @yksflip how will we deal with merge conflicts? Just bail out gracefully and be like "ok, now you're on your own again until you solve this shit"? I've got a proposal WIP for this https://pad.riseup.net/p/VewCbdPI-_RWR7ooe6v0-keep
yksflip commented 2023-10-05 19:37:54 +00:00
Author
Member
Copy Link

hmm, I guess so. I mean hopefully there wont be many conflicts as we perform git pull all the time? But stuff will go wrong and I think then it's the best to let the operator fix things and not assume anything.
Proposal looks great! nice you're tackling this :)

hmm, I guess so. I mean hopefully there wont be many conflicts as we perform `git pull` all the time? But stuff will go wrong and I think then it's the best to let the operator fix things and not assume anything. Proposal looks great! nice you're tackling this :)
🎉 1
decentral1se commented 2023-10-05 21:44:03 +00:00
Owner
Copy Link

A wild doubt appears coop-cloud/organising#267 (comment)

A wild doubt appears https://git.coopcloud.tech/coop-cloud/organising/issues/267#issuecomment-18267
decentral1se modified the project from (deleted) to (deleted) 2023-10-06 09:27:58 +00:00
moritz commented 2023-10-09 11:22:48 +00:00
Member
Copy Link

  • abra app new foo
    git add / commit / push the new .env file

  • abra app config
    git add / commit / push the modified .env file

There should be a flag to deactivate this functionality to avoid loads of pushed commits while tinkering with configs. I would prefer to only commit a finished config instead of committing each little config changes (many undos and redos). And maybe an abra server commit command if you edit files manually.

> * `abra app new foo` > git add / commit / push the new .env file > > * `abra app config` > git add / commit / push the modified .env file There should be a flag to deactivate this functionality to avoid loads of pushed commits while tinkering with configs. I would prefer to only commit a finished config instead of committing each little config changes (many undos and redos). And maybe an `abra server commit` command if you edit files manually.
cas commented 2023-10-09 16:08:32 +00:00
Member
Copy Link

If adding the repo url is at server add time, and it is optional, and it can be added to the server after the server has been created without one (by either just git init in the server's directory, or putting it into a git repository, or with an abra command), and commits/pushes are on demand rather than cooked into other operations without an additional flag to those operations (so that tinkering, the setup loop, and other things don't create a ton of commits), I'm in favor of this mechanic. I could see warning if an operation would not commit/push the changes to the env file or if it's been N hours since a commit/push has been done and there are changes.

I think a bonus point would be to also create a remote on the server in question (again, at the operators discretion) so that abra could discover not only the canonical remote for that server but also the configurations for that server at abra add server time (for example if I switched my computer but forgot to bring my abra config).

If adding the repo url is at server add time, *and* it is optional, *and* it can be added to the server after the server has been created without one (by either just git init in the server's directory, or putting it into a git repository, or with an abra command), *and* commits/pushes are on demand rather than cooked into other operations without an additional flag to those operations (so that tinkering, the setup loop, and other things don't create a ton of commits), I'm in favor of this mechanic. I could see warning if an operation would not commit/push the changes to the env file or if it's been N hours since a commit/push has been done and there are changes. I think a bonus point would be to also create a remote on the server in question (again, at the operators discretion) so that abra could discover not only the canonical remote for that server but also the configurations for that server at `abra add server` time (for example if I switched my computer but forgot to bring my abra config).
👍 3
decentral1se commented 2023-10-09 20:01:10 +00:00
Owner
Copy Link

Yeh, nice. The more explicit invocation of syncing could be a way to go. Still need to think this through...

I think a bonus point would be to also create a remote on the server in question (again, at the operators discretion) so that abra could discover not only the canonical remote for that server but also the configurations for that server at abra add server time (for example if I switched my computer but forgot to bring my abra config).

Super interesting idea! I would love to have a way to bootstrap a shared SSH config into a developer machine (imagine a Include ~/.abra/ssh/config which is appended to the ~/.ssh/config?) and having a way to grab server configurations (one or all) could potentially really speed up onboarding to get access...

Yeh, nice. The more explicit invocation of syncing could be a way to go. Still need to think this through... > I think a bonus point would be to also create a remote on the server in question (again, at the operators discretion) so that abra could discover not only the canonical remote for that server but also the configurations for that server at abra add server time (for example if I switched my computer but forgot to bring my abra config). Super interesting idea! I would **love** to have a way to bootstrap a shared SSH config into a developer machine (imagine a `Include ~/.abra/ssh/config` which is appended to the `~/.ssh/config`?) and having a way to grab server configurations (one or all) could potentially really speed up onboarding to get access...
👍 1
decentral1se commented 2023-10-13 09:22:33 +00:00
Owner
Copy Link

@cas @moritz @yksflip @3wordchant et. al here is a design proposal:

https://pad.riseup.net/p/IebZQkpe3OOpYyVT8f1j-keep

Feedback welcome!

I went for a trimmed down version, focusing on syncing the version of the app for the deploy/upgrade/rollback commands.

It isn't on demand as you mentioned @cas as I fear folks might forget to sync the versions manually. The compromise is that there is much less syncing going on than in the original design above. Sync of other details could certainly be on demand in future.

Can move towards a resolution/budget if consensus starts to emerge 🌞

UPDATE: https://docs.coopcloud.tech/federation/resolutions/in-progress/013/

@cas @moritz @yksflip @3wordchant et. al here is a design proposal: > https://pad.riseup.net/p/IebZQkpe3OOpYyVT8f1j-keep Feedback welcome! I went for a trimmed down version, focusing on syncing the version of the app for the `deploy`/`upgrade`/`rollback` commands. It isn't on demand as you mentioned @cas as I fear folks might forget to sync the versions manually. The compromise is that there is much less syncing going on than in the original design above. Sync of other details could certainly be on demand in future. Can move towards a resolution/budget if consensus starts to emerge 🌞 **UPDATE**: https://docs.coopcloud.tech/federation/resolutions/in-progress/013/
yksflip commented 2023-10-13 09:35:21 +00:00
Author
Member
Copy Link

great! I think this should tackle the overall pain points :)

great! I think this should tackle the overall pain points :)
🚀 1
decentral1se removed their assignment 2023-11-27 10:43:28 +00:00
decentral1se removed this from the (deleted) project 2023-11-27 10:43:31 +00:00
decentral1se commented 2023-11-27 10:44:22 +00:00
Owner
Copy Link

The proposal is still under consideration, there seems to be some concerns around it still. Polling for support in the federation channel. Please also see latest minutes: https://pad.autonomic.zone/dKJl5gKVStiV1yGmK-ya9Q?both

The proposal is still under consideration, there seems to be some concerns around it still. Polling for support in the federation channel. Please also see latest minutes: https://pad.autonomic.zone/dKJl5gKVStiV1yGmK-ya9Q?both
decentral1se commented 2024-07-09 09:52:32 +00:00
Owner
Copy Link

Order of operations coming together: coop-cloud/organising#541 (comment)

Order of operations coming together: https://git.coopcloud.tech/coop-cloud/organising/issues/541#issuecomment-21040
decentral1se commented 2024-07-18 22:22:54 +00:00
Owner
Copy Link

I think coop-cloud/organising#467 (comment) sounds the most sensible for a first implementation. It's fully opt-in.

abra app sync as the UI perhaps, which would:

  1. check for a abra.yml
  2. check for stanza that looks something like (?):
---
sync:
  enabled: true
  url: https://mygit.org/myorg/apps.git
  1. show diff, on yes, commit, pull --rebase & push changes

We need to somehow figure out how to initialise this repository on first use. annnnd we need to deal with merge conflicts. we have a new concern: not only just unstaged changes in the recipe but now in the .env file! a simple first stab at it could bail on merge conflicts with a diff and ask the user to resolve it.

I think https://git.coopcloud.tech/coop-cloud/organising/issues/467#issuecomment-18369 sounds the most sensible for a first implementation. It's fully opt-in. `abra app sync` as the UI perhaps, which would: 1. check for a `abra.yml` 2. check for stanza that looks something like (?): ```yaml --- sync: enabled: true url: https://mygit.org/myorg/apps.git ``` 3. show diff, on yes, commit, pull --rebase & push changes We need to somehow figure out how to initialise this repository on first use. annnnd we need to deal with merge conflicts. we have a new concern: not only just unstaged changes in the recipe but now in the `.env` file! a simple first stab at it could bail on merge conflicts with a diff and ask the user to resolve it.
moritz commented 2024-07-28 07:54:20 +00:00
Member
Copy Link

Should the abra.yml be global or per repository? I think the sync should be enabled or disabled per repo and not globally.
Why not simply check if the env dir is a git repo and has a remote and only sync it if it has?

Should the `abra.yml` be global or per repository? I think the sync should be enabled or disabled per repo and not globally. Why not simply check if the env dir is a git repo and has a remote and only sync it if it has?
moritz commented 2024-07-28 08:24:36 +00:00
Member
Copy Link

I'm not sure if these commands really need an automated push sync:

  • abra app new foo
  • abra app config

This would flood the commit history with every little .env change and creation. But an automated pull for these commands would avoid merge conflicts. I think most important is that deploy/upgrade/rollback/undeploy commands automatically create commits containing the version and push them because this changes the running state of an app and must be synced as fast as possible to your teammates to avoid any conflicts.

So my proposal is:

  1. a env/server dir should enable autosync if it is a git repo with a remote
    • auto sync could be disabled using a config file, or something like a .nosync file in the repo
  2. Autosync can always be disabled with a flag like --nosync
  3. app new/config/deploy/undeploy/upgrade/rollback / sync
    • before running the command: perform a git pull
    • If the chosen app is affected by some pulled commits show the diff of the affected .env
    • If there is a merge conflict abort and ask to solve the conflict and give some advice how to do this. (maybe there could be some extra abra commands for solving merge conflicts)
  4. app deploy/undeploy/upgrade/rollback / sync
    • after running the command: perform a git commit with a commit message containing the app name and the version number of the deployed/undeployed app: <command> <appname> <version>
    • the upgrade commit message should additionally contain the old version of the app
    • push the commit
I'm not sure if these commands really need an automated push sync: - `abra app new foo` - `abra app config` This would flood the commit history with every little .env change and creation. But an automated pull for these commands would avoid merge conflicts. I think most important is that `deploy`/`upgrade`/`rollback`/`undeploy` commands automatically create commits containing the version and push them because this changes the running state of an app and must be synced as fast as possible to your teammates to avoid any conflicts. So my proposal is: 1. a env/server dir should enable autosync if it is a git repo with a remote - auto sync could be disabled using a config file, or something like a `.nosync` file in the repo 2. Autosync can always be disabled with a flag like `--nosync` 3. `app` `new`/`config`/`deploy`/`undeploy`/`upgrade`/`rollback` / `sync` - before running the command: perform a `git pull` - If the chosen app is affected by some pulled commits show the diff of the affected .env - If there is a merge conflict abort and ask to solve the conflict and give some advice how to do this. (maybe there could be some extra abra commands for solving merge conflicts) 4. `app` `deploy`/`undeploy`/`upgrade`/`rollback` / `sync` - after running the command: perform a `git commit` with a commit message containing the app name and the version number of the deployed/undeployed app: `<command> <appname> <version>` - the `upgrade` commit message should additionally contain the old version of the app - push the commit
👍 2
p4u1 commented 2024-07-28 09:41:49 +00:00
Member
Copy Link

@moritz

Should the abra.yml be global or per repository? I think the sync should be enabled or disabled per repo and not globally.
Why not simply check if the env dir is a git repo and has a remote and only sync it if it has?

For now the abra.yml is detected by going up the directory tree until the home or root directory, taking the first one that is found

@moritz >Should the `abra.yml` be global or per repository? I think the sync should be enabled or disabled per repo and not globally. >Why not simply check if the env dir is a git repo and has a remote and only sync it if it has? For now the abra.yml is detected by going up the directory tree until the home or root directory, taking the first one that is found
p4u1 commented 2024-07-28 09:48:16 +00:00
Member
Copy Link

@moritz I like your proposal about! Some remarks on the points:

  1. I also think auto detecting the git repo is good. It should always take the file location of the app env file and look for a repo there
    1.1. Disabling autosync should be configurable via the abra.yml not via an extra file
  2. We should also respect the --nosync here. Because git pull does not work if you have uncomitted changes
@moritz I like your proposal about! Some remarks on the points: 1. I also think auto detecting the git repo is good. It should always take the file location of the app env file and look for a repo there 1.1. Disabling autosync should be configurable via the abra.yml not via an extra file 3. We should also respect the --nosync here. Because git pull does not work if you have uncomitted changes
decentral1se commented 2024-12-28 22:50:46 +00:00
Owner
Copy Link

OK, everything is in place now to tackle this 🥹

OK, everything is in place now to tackle this 🥹
decentral1se commented 2024-12-30 00:20:56 +00:00
Owner
Copy Link

Send your design brains to coop-cloud/abra#457 🙏

Send your design brains to https://git.coopcloud.tech/coop-cloud/abra/pulls/457 🙏
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
abra
Everything to do with abra
abra-gandi
Abra Gandi plugin
awaiting-feedback
Ping/pong on comms
backups
bug
Something is not working
build
Go build related issues
ci/cd
Getting the robots into the mix
community organising
Opening this thing up
contributing
Contributors stuff
coopcloud.tech
Our main website
democracy
Democratic decision making
design
Design thinking required
documentation
Let's write things together
duplicate
This issue or pull request already exists
enhancement
New feature
finance
Money things
funding
Anything related to grant funding
good first issue
Easy start with development
help wanted
Need some help
installer
Installation related issues
kadabra
performance
Performance related
proposal
Large change which requires feedback & decisin making
question
More information is needed
recipes.coopcloud.tech
Our recipes catalogue
security
Securing our shit
test
Unit or integration test suite
wontfix
This won't be fixed
No Label
abra
enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
3wordchant abra-bot ammaratef45 Apfelwurm carla cas decentral1se fauno kawaiipunk knoflook lambdabundesverband moritz p4u1 simon trav val yksflip
No Assignees
5 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/organising#467
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?