Operator sync resurrected #809

New Issue

iexos · 2026-03-26T16:50:23Z

iexos commented

2026-03-26 16:50:23 +00:00

Last night I wrote down some ideas I have for operator sync. After reading through existing conversations (toolshed/organising#467, #457 and more) I am happy to see it aligns and overlaps well with existing ideas as well as addressing some of the concerns.

Differences to previous proposals

The initial implementation would be automatic but also fully opt-in. To enable operator sync, you will need your env files within a git repo (its root can be anywhere) and have a remote called operator-sync. Opting in/out is as simple as renaming the git remote and can be done individually.

The other main difference to existing ideas would be to commit&push only and all changes to server side state (even if no files have changed, creating empty commits) which would make the git log work as an operator log.

Initial implementation

Operator sync is activated if the server dir is part of a git repo and it has a remote called operator-sync.

Operator sync would function automatically on abra app commands in two different ways:

Commands modifying server state

Commands which modify the server state should

pull the operator-sync remote with git pull --rebase behaviour
- fatal out if not successful, suggest using --no-operator-sync
let the cmd do its thing
- in some cases of failure, where server state has already changed, we should still continue with the steps below
create a commit
- the message containing the command and its arguments and possibly output/failure state
- only add the .env file of the app in question
- if there are other new/modified files in the repository, list them in a warning
- this can also be an empty commit with no files changed
push to operator-sync remote

Commands with this behaviour would be the following abra app commands:

deploy
move
- should operate on both server dirs with different messages
new --secrets
remove
restore
rollback
secret generate/insert/remove
- inserted secret content should not end up in log
undeploy
upgrade
volume remove

This might lead to seemingly weird side effects like that the commit with secret insert will add the .env of a new app. I think that is correct though as this is exactly the point where the something about this app is created on the server.

To only commit the <domain>.env in question will prevent unintended side effects. A warning about other changed files should help you keep the repo clean. If you added other files related to this app you might want to commit them manually. I am not quite sure how to deal with staged changes yet.

git pull --rebase behaviour should reduce the amount of manual intervention needed to a minimum. In these cases intervention is probably needed anyway.

Some others are also (potentially) modifying server state (command, restart, ...), though I consider not important enough to keep a log of. I might have missed smth though!

Other `abra app` commands

Every other abra app command should execute the pull-part only - they all depend on the list of services. In this case a pull failure could be a non-fatal error. Though you might miss that in the wall of text that the command probably produces.

I do not envision creating new abra commands initially and would like it to work as automatic as possible after setup. I think we can also rely on operators' ability to use git if anything breaks.

Other considerations

Since every app command is now pulling the remote, this might slow down operations. To mitigate abra could somehow take note of the last successful pull and only pull again after a few minutes or so.

To override behaviour, I envision these flags:

--no-operator-sync do not do pulls and pushes
- also, --offline should be respected
--no-operator-log do not create git commit (implies --no-operator-sync)
--force-sync always pull on commands without cooldown

I tend to think a bit far sometimes and I recognize that this might be a bit much for an initial implementation. Implementing only part of this could be helpful already.

Future possibilities

This may be a bit far off right now, I just want to get it out of my head. If initial implementation turns out to be useful, further steps could include:

creating dedicated abra commands related to operator sync, depending on needs (e.g. syncing, handling remote)
creating abra.yml options to change sync/log behaviour
enabling commits without remote for creating the log
ask to create a repo on abra server add
additionally ask to create a remote on the server itself, e.g. in <server>:~/.abra/operator-sync.git
- clone if exists, otherwise creating the bare repo and then clone
- potential issues: operators might have different users on server, dir might not be writable, git might not be installed...

Please leave comments on what you think of this!

Last night I wrote down some ideas I have for operator sync. After reading through existing conversations (https://git.coopcloud.tech/toolshed/organising/issues/467, https://git.coopcloud.tech/toolshed/abra/pulls/457 and more) I am happy to see it aligns and overlaps well with existing ideas as well as addressing some of the concerns. ## Differences to previous proposals The initial implementation would be automatic but also fully opt-in. To enable operator sync, you will need your env files within a git repo (its root can be anywhere) and have a remote called `operator-sync`. Opting in/out is as simple as renaming the git remote and can be done individually. The other main difference to existing ideas would be to commit&push only and all changes to server side state (even if no files have changed, creating empty commits) which would make the git log work as an operator log. ## Initial implementation Operator sync is activated if the server dir is part of a git repo and it has a remote called `operator-sync`. Operator sync would function automatically on `abra app` commands in two different ways: ### Commands modifying server state Commands which modify the server state should - pull the `operator-sync` remote with `git pull --rebase` behaviour - fatal out if not successful, suggest using `--no-operator-sync` - let the cmd do its thing - in some cases of failure, where server state has already changed, we should still continue with the steps below - create a commit - the message containing the command and its arguments and possibly output/failure state - only add the `.env` file of the app in question - if there are other new/modified files in the repository, list them in a warning - this can also be an empty commit with no files changed - push to `operator-sync` remote Commands with this behaviour would be the following `abra app` commands: - `deploy` - `move` - should operate on both server dirs with different messages - `new --secrets` - `remove` - `restore` - `rollback` - `secret generate/insert/remove` - inserted secret content should not end up in log - `undeploy` - `upgrade` - `volume remove` This might lead to seemingly weird side effects like that the commit with `secret insert` will add the `.env` of a new app. I think that is correct though as this is exactly the point where the something about this app is created on the server. To only commit the `<domain>.env` in question will prevent unintended side effects. A warning about other changed files should help you keep the repo clean. If you added other files related to this app you might want to commit them manually. I am not quite sure how to deal with staged changes yet. `git pull --rebase` behaviour should reduce the amount of manual intervention needed to a minimum. In these cases intervention is probably needed anyway. Some others are also (potentially) modifying server state (`command`, `restart`, ...), though I consider not important enough to keep a log of. I might have missed smth though! ### Other `abra app` commands Every other `abra app` command should execute the pull-part only - they all depend on the list of services. In this case a pull failure could be a non-fatal error. Though you might miss that in the wall of text that the command probably produces. I do not envision creating new abra commands initially and would like it to work as automatic as possible after setup. I think we can also rely on operators' ability to use `git` if anything breaks. ### Other considerations Since every app command is now pulling the remote, this might slow down operations. To mitigate `abra` could somehow take note of the last successful pull and only pull again after a few minutes or so. To override behaviour, I envision these flags: - `--no-operator-sync` do not do pulls and pushes - also, `--offline` should be respected - `--no-operator-log` do not create git commit (implies `--no-operator-sync`) - `--force-sync` always pull on commands without cooldown I tend to think a bit far sometimes and I recognize that this might be a bit much for an initial implementation. Implementing only part of this could be helpful already. ## Future possibilities This may be a bit far off right now, I just want to get it out of my head. If initial implementation turns out to be useful, further steps could include: - creating dedicated abra commands related to operator sync, depending on needs (e.g. syncing, handling remote) - creating `abra.yml` options to change sync/log behaviour - enabling commits without remote for creating the log - ask to create a repo on `abra server add` - additionally ask to create a remote on the server itself, e.g. in `<server>:~/.abra/operator-sync.git` - clone if exists, otherwise creating the bare repo and then clone - potential issues: operators might have different users on server, dir might not be writable, git might not be installed... ------ Please leave comments on what you think of this!

❤️ 2

iexos added the

enhancement

design

labels 2026-03-26 16:50:24 +00:00

decentral1se commented

2026-03-27 13:26:21 +00:00

I'm all for this approach. Thanks for drawing it up! If it's something people can slowly opt-in to and we can fix as we go without some big breaking migration then I think it's doable. It's more a question for me of how to break this down into some logical order of bit-by-bit implementation that we can roll out and have people test.

I think we will have to invest in making sure we can somehow cover all the failure scenarios of Swarm deployments and that they are reflected in the env version. I feel like we are still building on shaky foundations. #808 shows the current limitations and I'm not sure what to do about it.

I'm all for this approach. Thanks for drawing it up! If it's something people can slowly opt-in to and we can fix as we go without some big breaking migration then I think it's doable. It's more a question for me of how to break this down into some logical order of bit-by-bit implementation that we can roll out and have people test. I think we will have to invest in making sure we can somehow cover all the failure scenarios of Swarm deployments and that they are reflected in the env version. I feel like we are still building on shaky foundations. https://git.coopcloud.tech/toolshed/abra/issues/808 shows the current limitations and I'm not sure what to do about it.

👍 1

p4u1 commented

2026-03-27 14:09:38 +00:00

@iexos How would this work if we already work in a git repo? Could you describe that use case? Like how would this interact with normal git usage?

iexos commented

2026-03-27 14:34:48 +00:00

@p4u1
Assuming you have repo with a single remote origin, you will have to rename that remote to operator-sync. You can continue to use the repo as before.

Now when using any abra app cmd this repo will be pulled.
Additionally, any time you execute one of the server-modifying cmds above like abra app deploy <domain> a commit will be created in which only changes to <domain>.env are added. Then the repo is pushed to remote.

So what you potentially need to watch out for is that all commits made before will also be pushed and that the .env file will be committed.
Also, your git log will likely grow much faster.

@p4u1 Assuming you have repo with a single remote `origin`, you will have to rename that remote to `operator-sync`. You can continue to use the repo as before. Now when using any `abra app` cmd this repo will be pulled. Additionally, any time you execute one of the server-modifying cmds above like `abra app deploy <domain>` a commit will be created in which only changes to `<domain>.env` are added. Then the repo is pushed to remote. So what you potentially need to watch out for is that all commits made before will also be pushed and that the `.env` file will be committed. Also, your git log will likely grow much faster.

decentral1se commented

2026-03-27 19:49:46 +00:00

Hmmm good question. Being required to git remote rename origin operator-sync is kind of breaking an existing workflow? I had actually imagined it being a 2nd remote that you add alongside origin (which also seems weird now that I type it out). Another possibility is triggering via a config value so that you can keep the same git remote name?

# abra.yml
sync: true

Hmmm good question. Being required to `git remote rename origin operator-sync` is kind of breaking an existing workflow? I had actually imagined it being a *2nd* remote that you add alongside `origin` (which also seems weird now that I type it out). Another possibility is triggering via a config value so that you can keep the same git remote name? ```yml # abra.yml sync: true ```

iexos commented

2026-03-27 20:44:09 +00:00

Both can be possible, either renaming or adding the remote. I thought if only one remote is present, cmds like pull, push will continue to work as normal but maybe that is wrong?

Where I come from is that the double remotes of recipes (origin & origin-ssh) confuse my shell prompt as it thinks origin is not up-to-date because abra pushes the other one, showing me unpushed commits. But if that is no issue for you then just use both.

Both can be possible, either renaming or adding the remote. I thought if only one remote is present, cmds like pull, push will continue to work as normal but maybe that is wrong? Where I come from is that the double remotes of recipes (`origin` & `origin-ssh`) confuse my shell prompt as it thinks origin is not up-to-date because abra pushes the other one, showing me unpushed commits. But if that is no issue for you then just use both.

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/abra#809

Operator sync resurrected #809

Differences to previous proposals

Initial implementation

Commands modifying server state

Other abra app commands

Other considerations

Future possibilities

Other `abra app` commands