Method for running ad-hoc containers #729

New Issue

benjaminlyng · 2025-11-13T20:49:19Z

benjaminlyng commented

2025-11-13 20:49:19 +00:00

In some scenarios a command needs to be run in a docker-container on a server. One use-case is to run database migrations when upgrading Mastodon (See "Update steps" on release notes)

I suggest adding a method that can deploy a new container specifically to run the given command. Something equivalent to running docker-compose run --rm some_app <some command> which starts up a new container with the same configuration as some other container, runs the command in that container and then shuts down and removes the container again.

In some scenarios a command needs to be run in a docker-container on a server. One use-case is to run database migrations when upgrading Mastodon (See "Update steps" on [release notes](https://github.com/mastodon/mastodon/releases/tag/v4.5.0)) I suggest adding a method that can deploy a new container specifically to run the given command. Something equivalent to running `docker-compose run --rm some_app <some command> ` which starts up a new container with the same configuration as some other container, runs the command in that container and then shuts down and removes the container again.

decentral1se added the

enhancement

label 2025-11-14 07:55:08 +00:00

decentral1se commented

2025-11-14 07:56:19 +00:00

Nice @benjaminlyng, do you have a proposal UI/UX for abra? abra app run ... has some different considerations than docker-compose in terms of CLI arguments. I'd appreciate if you could take a look and see if you can figure out a design. The implementation is probably not that bad, the design is always the harded 🙃 How would you like it to work?

Nice @benjaminlyng, do you have a proposal UI/UX for `abra`? `abra app run ...` has some different considerations than `docker-compose` in terms of CLI arguments. I'd appreciate if you could take a look and see if you can figure out a design. The implementation is probably not that bad, the design is always the harded 🙃 How would you like it to work?

benjaminlyng commented

2025-11-14 13:31:07 +00:00

Right, it's already a little confusing that abra app command is different from abra app run so adding another similar option might not be preferable. But if I understand correctly abra app run does the same as docker compose exec. Personally I would keep the CLI as close to docker composes as possible, but thats just because that's what i know. If you renamed abra app run to abra app exec you could match docker composes cli by adding this functionality with abra app run. Changing the behaviour of a CLI-method is obviously quite dangerous, I know, but I think it will be easier for people who are used to docker compose.

Right, it's already a little confusing that `abra app command` is different from `abra app run` so adding another similar option might not be preferable. But if I understand correctly `abra app run` does the same as `docker compose exec`. Personally I would keep the CLI as close to docker composes as possible, but thats just because that's what i know. If you renamed `abra app run` to `abra app exec` you could match docker composes cli by adding this functionality with `abra app run`. Changing the behaviour of a CLI-method is obviously quite dangerous, I know, but I think it will be easier for people who are used to docker compose.

decentral1se commented

2025-11-15 09:02:10 +00:00

We can easily do sub-command aliases which won't break other peoples scripts and be backwards compatible. I'm not that concerned with matching docker-compose UI/UX myself. got a draft of how the CLI arguments / flags could work?

We can easily do sub-command aliases which won't break other peoples scripts and be backwards compatible. I'm not that concerned with matching `docker-compose` UI/UX myself. got a draft of how the CLI arguments / flags could work?

3wordchant commented

2025-11-21 15:43:44 +00:00

Personally I'm -0 on switching to run → exec and introducing run for this; the Docker commands have seemed confusing to many people I've worked with (arguably they're the wrong way around, for one thing), and I expect folks will rarely need the command being proposed here.

That said, I also don't have a great alternative suggestion 😖 Does swarm even support running one-shot containers?

One ratchet workaround in the meantime is that we can define cmd commands to be used with --local which could run containers on the specified (usually remote) swarm using DOCKER_HOST 😈

Personally I'm -0 on switching to `run` → `exec` and introducing `run` for this; the Docker commands have seemed confusing to many people I've worked with (arguably they're the wrong way around, for one thing), and I expect folks will rarely need the command being proposed here. That said, I also don't have a great alternative suggestion 😖 Does swarm even support running one-shot containers? One ratchet workaround in the meantime is that we can define `cmd` commands to be used with `--local` which could run containers on the specified (usually remote) swarm using `DOCKER_HOST` 😈

benjaminlyng commented

2025-11-22 07:43:03 +00:00

I can't find any info on how to do it in docker swarm. And I'm not sure whether its on purpose. Perhaps it is an antipattern, it's just how mastodon suggests doing migrations during upgrades. But that's for docker-compose so it might be more suitable for development. I'm not sure.
Lets not rename commands for this.

As far as I can tell using the --local flag specifies to run a command in an ad-hoc container locally - right? I think that's why I assumed running it without --local would spawn a new container in the swarm.

Another thing I'm confused by is the difference between using abra app run <domain> app tootctl (which doesn't work, it lacks some environment variables) and running tootctl from abra app cmd <domain> app shell.

I can't find any info on how to do it in docker swarm. And I'm not sure whether its on purpose. Perhaps it is an antipattern, it's just how mastodon suggests doing migrations during upgrades. But that's for docker-compose so it might be more suitable for development. I'm not sure. Lets not rename commands for this. As far as I can tell using the `--local` flag specifies to run a command in an ad-hoc container locally - right? I think that's why I assumed running it without `--local` would spawn a new container in the swarm. Another thing I'm confused by is the difference between using `abra app run <domain> app tootctl` (which doesn't work, it lacks some environment variables) and running `tootctl` from `abra app cmd <domain> app shell`.

decentral1se commented

2025-11-22 23:20:34 +00:00

@benjaminlyng abra app cmd uses bash functions in a abra.sh under the hood and exposes the correct environment variables. See the recipe for more:

		abra.sh
		Lines 37 to 48 in 53ea5afb5b
	
				environment() {

				    # for sidekiq service bundle exec env var threading

				    file_env "DB_PASS"

				    file_env "OTP_SECRET"

				    file_env "SECRET_KEY_BASE"

				    file_env "VAPID_PRIVATE_KEY"

				    file_env "ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY"

				    file_env "ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT"

				    file_env "ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY"

				    declare -x RAILS_ENV=production

				}

@benjaminlyng `abra app cmd` uses bash functions in a `abra.sh` under the hood and exposes the correct environment variables. See the recipe for more: https://git.coopcloud.tech/coop-cloud/mastodon/src/commit/53ea5afb5b324278ebf25d6a159159df65250963/abra.sh#L37-L48

benjaminlyng commented

2025-11-25 21:31:09 +00:00

Right, I just assumed the environment variables would be present when I entered the container with the run command. But that requires extra setup which is handled in the shell function.

Perhaps there is no need for ad-hoc containers anyway. For mastodon the use-case with database migrations is written into the entrypoint, so there's no need to start new containers. And I thought it would be best to run tootctl from new containers to avoid stopping any processes internally, but it doesn't to cause any issues. I've added a new command to make it easier to use the tootctl CLI. Perhaps that's actually the only interface I was missing.
coop-cloud/mastodon#31 (comment)

Right, I just assumed the environment variables would be present when I entered the container with the `run` command. But that requires extra setup which is handled in the `shell` function. Perhaps there is no need for ad-hoc containers anyway. For mastodon the use-case with database migrations is written into the entrypoint, so there's no need to start new containers. And I thought it would be best to run tootctl from new containers to avoid stopping any processes internally, but it doesn't to cause any issues. I've added a new command to make it easier to use the tootctl CLI. Perhaps that's actually the only interface I was missing. https://git.coopcloud.tech/coop-cloud/mastodon/pulls/31#issuecomment-28830

3wordchant commented

2025-11-26 03:41:29 +00:00

As far as I can tell using the --local flag specifies to run a command in an ad-hoc container locally - right? I think that's why I assumed running it without --local would spawn a new container in the swarm.

I think cmd --local doesn't launch a container, it just directly runs the shell script using bash/sh`, for what it's worth.

Right, I just assumed the environment variables would be present when I entered the container with the run command. But that requires extra setup which is handled in the shell function.

Yes, this is because Mastodon doesn't know how to load secrets from files (required for Docker secrets); maybe we could open an issue upstream (so that Gagron can tell us no lol). I looked into this recently for another Ruby on Rails project, and I haven't been able to find any kind of standard way of doing this so unfortunately I don't yet have a good suggestion how to implement it natively.

Perhaps there is no need for ad-hoc containers anyway. For mastodon the use-case with database migrations is written into the entrypoint, so there's no need to start new containers. And I thought it would be best to run tootctl from new containers to avoid stopping any processes internally, but it doesn't to cause any issues.

I don't have enough of an understanding of Rails migrations, or how Mastodon uses them (I think the concept of "pre deploy" / "post deploy" migrations is something they invented? Maybe?) to say if this could cause problems. I can just say it hasn't for me so far! 🙃

> As far as I can tell using the --local flag specifies to run a command in an ad-hoc container locally - right? I think that's why I assumed running it without --local would spawn a new container in the swarm. I think `cmd --local` doesn't launch a container, [it just directly runs the shell script using `bash/`sh`](https://git.coopcloud.tech/toolshed/abra/src/branch/main/cli/app/cmd.go#L141-L146), for what it's worth. > Right, I just assumed the environment variables would be present when I entered the container with the run command. But that requires extra setup which is handled in the shell function. Yes, this is because Mastodon doesn't know how to load secrets from files (required for Docker secrets); maybe we could open an issue upstream (so that Gagron can tell us no lol). I looked into this recently for another Ruby on Rails project, and I haven't been able to find any kind of standard way of doing this so unfortunately I don't yet have a good suggestion how to implement it natively. > Perhaps there is no need for ad-hoc containers anyway. For mastodon the use-case with database migrations is written into the entrypoint, so there's no need to start new containers. And I thought it would be best to run tootctl from new containers to avoid stopping any processes internally, but it doesn't to cause any issues. I don't have enough of an understanding of Rails migrations, or how Mastodon uses them (I think the concept of "pre deploy" / "post deploy" migrations is something they invented? Maybe?) to say if this could cause problems. I can just say it hasn't for me so far! 🙃

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/abra#729