Proposal: abra-entrypoint.sh #381

New Issue

cas · 2023-01-21T20:49:45Z

cas commented

2023-01-21 20:49:45 +00:00

Overview

There are many hacks contained in recipes in the entrypoint scripts. These are cut and pasted as needed, and there's no centralized way of providing the functionality that there's definitely a call for. In particular there are two hacks which are regularly used surrounding the entry point; one being converting secrets into environment variables (contained in boilerplate called file_env) and overriding the entrypoint for debugging purposes (as discussed in #373, but this is the normal way of debugging) on a temporary basis. There are undoubtedly numerous similar things contained in custom entrypoints in recipes.

The solution that I propose is for abra to always override the entrypoint of the containers with a script that contains mechanisms to do things like file_env and overriding the entrypoint temporarily, but also is transparent to the recipe (thus working as it does now).

Technical Details

The basic idea is that abra would override the entrypoint with its script, ship its script to the app, and set an environment variable which corrosponds to the entrypoint which was overriden. The script shipped to the app would ultimately call the script in this environment variable. This allows a trivial override of the entrypoint by merely overriding the environment variable, for things such as debugging, mediated by a new abra command (something like abra app entrypoint <whateverapp> <whatevercontainer> commandline..., or abra app entrypoint <whateverapp> <whatevercontainer> reset to return to the default).

Additionally, abra could set other environment variables which would cause other functionality to be used. This would be useful for recipe authors needing to load, for example, secrets into the environment. They would be passed by setting variables in the .env file.

I propose all of the environment variables be set with an ABRA_ prefix; that we set other variables for use of the recipe authors, such as:

ABRA_APP_DEFAULT_ENTRYPOINT - the non-overriden entrypoint
ABRA_APP_ENTRYPOINT - the current entrypoint abra-entrypoint.sh will call
ABRA_FILEENV_<KEY> - every one of these would find the file pointed to by the value and load it as environment variable <KEY>

I would also include things such as the fact that the container is running on coop (maybe like ABRA_VERSION or something) and the name of the app and other things of bookkeeping interest.

We could also include mechanisms to run initialization scripts from abra-entrypoint.sh similar to docker-entrypoint.d, since this is a very common use of custom entrypoints.

This mechanism would not override entrypoint or command keys in compose files, but would insert a layer that would call them.

Special thanks to @mayel:matrix.org for some inspiration, and discovering that you can find the original entrypoint, ultimately, with:

docker inspect -f '{{.Config.Entrypoint}}' postgres:14.3 for an image and docker container inspect -f '{{.Config.Entrypoint}}' bonfire-app-db-1 for a container

# Overview There are many hacks contained in recipes in the entrypoint scripts. These are cut and pasted as needed, and there's no centralized way of providing the functionality that there's definitely a call for. In particular there are two hacks which are regularly used surrounding the entry point; one being converting secrets into environment variables (contained in boilerplate called `file_env`) and overriding the entrypoint for debugging purposes (as discussed in #373, but this is the normal way of debugging) on a temporary basis. There are undoubtedly numerous similar things contained in custom entrypoints in recipes. The solution that I propose is for abra to always override the entrypoint of the containers with a script that contains mechanisms to do things like `file_env` and overriding the entrypoint temporarily, but also is transparent to the recipe (thus working as it does now). # Technical Details The basic idea is that abra would override the entrypoint with its script, ship its script to the app, and set an environment variable which corrosponds to the entrypoint which was overriden. The script shipped to the app would ultimately call the script in this environment variable. This allows a trivial override of the entrypoint by merely overriding the environment variable, for things such as debugging, mediated by a new abra command (something like `abra app entrypoint <whateverapp> <whatevercontainer> commandline...`, or `abra app entrypoint <whateverapp> <whatevercontainer> reset` to return to the default). Additionally, abra could set other environment variables which would cause other functionality to be used. This would be useful for recipe authors needing to load, for example, secrets into the environment. They would be passed by setting variables in the .env file. I propose all of the environment variables be set with an `ABRA_` prefix; that we set other variables for use of the recipe authors, such as: - `ABRA_APP_DEFAULT_ENTRYPOINT` - the non-overriden entrypoint - `ABRA_APP_ENTRYPOINT` - the current entrypoint abra-entrypoint.sh will call - `ABRA_FILEENV_<KEY>` - every one of these would find the file pointed to by the value and load it as environment variable `<KEY>` I would also include things such as the fact that the container is running on coop (maybe like `ABRA_VERSION` or something) and the name of the app and other things of bookkeeping interest. We could also include mechanisms to run initialization scripts from `abra-entrypoint.sh` similar to docker-entrypoint.d, since this is a very common use of custom entrypoints. This mechanism would not override `entrypoint` or `command` keys in compose files, but would insert a layer that would call them. Special thanks to @mayel:matrix.org for some inspiration, and discovering that you can find the original entrypoint, ultimately, with: > docker inspect -f '{{.Config.Entrypoint}}' postgres:14.3 for an image and docker container inspect -f '{{.Config.Entrypoint}}' bonfire-app-db-1 for a container

❤️ 2

cas added the

enhancement

abra

labels 2023-01-21 20:49:45 +00:00

cas commented

2023-01-21 20:54:30 +00:00

Thinking about it, the namespace COOP or similar might be better than ABRA since that's the namespace used in labels that have similar functionality.

Thinking about it, the namespace `COOP` or similar might be better than `ABRA` since that's the namespace used in labels that have similar functionality.

decentral1se commented

2023-01-21 22:06:53 +00:00

Nice, thanks for writing it out! A lot there... some thoughts!

I have often just avoided debugging a thing or just dropped packaging a recipe because I had to override the entrypoint. It is a bit of work to do. And, it is very "wtf magic" for new packagers.

If this can be achieved in the backwards comptaible way it is being proposed and documented clearly on the recipe maintainers & operators document, it'd be great. I think this has a lot potential to make things smoother. It's feels like quite a substantial change, so it'd be great to really break it down.

For a concrete example, in https://git.coopcloud.tech/coop-cloud/peertube/src/branch/main/entrypoint.sh.tmpl, I could migrate the file_env into ...FILEENV... right? But I'd have to keep this override due to needing mkdir/apt commands? Or is this could be packed into the initialisation scripts?

Those scripts would need to support Golang templating, I guess 🤔 It also seems a bit unclear how abra would internally manage the attached entrypoint config version... it would maybe have to remove and re-attach to avoid having to version but that might break swarm assumptions of being able to rollback... some devil in the details there...

Do we imagine that existing recipes would migrate to this? Or just new maintainers would use it? Would this complicate stuff if people are looking for examples and see two approaches? Or is this just a handy thing for debugging?

My main concern would be that this creates a reliance between the recipe configurations and abra that we've tried to avoid so far? If abra just dissapeared tomorrow, the recipe configs would live on in glory. If we have custom entrypoints that can only be loaded in via abra to do core init commands in order to run the container? Unsure if we want that.

Nice, thanks for writing it out! A lot there... some thoughts! I have often just avoided debugging a thing or just dropped packaging a recipe because I had to override the entrypoint. It is a bit of work to do. And, it is very "wtf magic" for new packagers. If this can be achieved in the backwards comptaible way it is being proposed and documented clearly on the recipe maintainers & operators document, it'd be great. I think this has a lot potential to make things smoother. It's feels like quite a substantial change, so it'd be great to really break it down. For a concrete example, in https://git.coopcloud.tech/coop-cloud/peertube/src/branch/main/entrypoint.sh.tmpl, I could migrate the `file_env` into `...FILEENV...` right? But I'd have to keep this override due to needing `mkdir`/`apt` commands? Or is this could be packed into the initialisation scripts? Those scripts would need to support Golang templating, I guess 🤔 It also seems a bit unclear how `abra` would internally manage the attached entrypoint config version... it would maybe have to remove and re-attach to avoid having to version but that might break swarm assumptions of being able to rollback... some devil in the details there... Do we imagine that existing recipes would migrate to this? Or just new maintainers would use it? Would this complicate stuff if people are looking for examples and see two approaches? Or is this just a handy thing for debugging? My main concern would be that this creates a reliance between the recipe configurations and `abra` that we've tried to avoid so far? If `abra` just dissapeared tomorrow, the recipe configs would live on in glory. If we have custom entrypoints that can only be loaded in via `abra` to do core init commands in order to run the container? Unsure if we want that.

decentral1se added

proposal

and removed

enhancement

abra

labels 2023-01-22 08:55:14 +00:00

decentral1se referenced this issue

2023-01-23 18:07:56 +00:00

Command to help debug failing/restarting containers #373

moritz referenced this issue

2023-01-24 11:15:06 +00:00

Automatic patch/security updates for deployed apps #236

cas commented

2023-02-05 02:55:49 +00:00

Just some thoughts and replies after writing a prototype of the script itself for my own use:

This should be backwards compatible and invisible to recipes (except insofar as recipes use the functionality going forward) since it should grab the original entrypoint from the recipe and/or docker image. I'd consider it a bug in the implementation if it caused issues with existing recipes. It should effectively pass through to the entrypoint/command that's set. I see some complexity overriding command and it may be that it only does entrypoint and if there's a command set it lets that take over. I haven't played with enough docker i guess to know if this is a reasonable way of doing it, and if we need to ovverride command too, i think that's reasonable.
Ideally it should provide functionality for including some number of scripts as 'startup' scripts that are expected to exit without having to build a new docker layer. I'm not completely certain how to do that, I think maybe as a volume or a set of config files that are automatically managed by abra for the app. Like perhaps we allow you to include a list of additional scripts in the recipe, and they are versioned and shipped to the container automatically. This would make it magical for including extra startup and setup scripts. The part that specifies the script list would also be passed to the entrypoint via an environment variable and it'd call each one.
I believe we'd just make the entrypoint a configuration file and version it automatically. Abra would perhaps contain the script internally with its version and deploy it to the container as a config file and set the entrypoint to it.
I imagine existing recipes would migrate to this system, but I think also that it shouldn't be at all required. It should make the existing custom entrypoint scripts become a bit vestigial. Like a lot of them doing just initialization tasks should become startup scripts instead of being managed manually. Not explicitly doing things with the startup system still gets you the nice debugging of course.
I feel like the ecosystem is already a bit diverged from pure 'docker swarm' things, and it only makes it more appealing and useful (to me) if more magic is included in the box. As far as coupling, I mean it's not like you wouldn't be able to move the recipe out of the coop ecosystem and take the entrypoint script with you and set the variables manually. The documentation should include ways to do that if you want imo.

Just some thoughts and replies after writing a prototype of the script itself for my own use: - This should be backwards compatible and invisible to recipes (except insofar as recipes use the functionality going forward) since it should grab the original entrypoint from the recipe and/or docker image. I'd consider it a bug in the implementation if it caused issues with existing recipes. It should effectively pass through to the entrypoint/command that's set. I see some complexity overriding command and it may be that it only does entrypoint and if there's a command set it lets that take over. I haven't played with enough docker i guess to know if this is a reasonable way of doing it, and if we need to ovverride command too, i think that's reasonable. - Ideally it should provide functionality for including some number of scripts as 'startup' scripts that are expected to exit without having to build a new docker layer. I'm not completely certain how to do that, I think maybe as a volume or a set of config files that are automatically managed by abra for the app. Like perhaps we allow you to include a list of additional scripts in the recipe, and they are versioned and shipped to the container automatically. This would make it magical for including extra startup and setup scripts. The part that specifies the script list would also be passed to the entrypoint via an environment variable and it'd call each one. - I believe we'd just make the entrypoint a configuration file and version it automatically. Abra would perhaps contain the script internally with its version and deploy it to the container as a config file and set the entrypoint to it. - I imagine existing recipes would migrate to this system, but I think also that it shouldn't be at all required. It should make the existing custom entrypoint scripts become a bit vestigial. Like a lot of them doing just initialization tasks should become startup scripts instead of being managed manually. Not explicitly doing things with the startup system still gets you the nice debugging of course. - I feel like the ecosystem is already a bit diverged from pure 'docker swarm' things, and it only makes it more appealing and useful (to me) if more magic is included in the box. As far as coupling, I mean it's not like you wouldn't be able to move the recipe out of the coop ecosystem and take the entrypoint script with you and set the variables manually. The documentation should include ways to do that if you want imo.

decentral1se commented

2023-02-07 21:27:18 +00:00

Thanks @cas! Sounds great 😄 It may be useful to try spec out each piece with some docker cli experiments or try to break it up into small pieces for code review or what not. Whatever you feel like! Looking forward to see where this goes.

yksflip commented

2023-02-08 09:16:38 +00:00

this sounds really cool! I think this can be a big win for maintaining recipes and I really like to have the startup-scripts too. But also +1 for "how to use recipes without abra"-docs.

🎉 1

decentral1se referenced this issue

2023-02-08 09:34:20 +00:00

Document how to interact with the recipes without abra #400

decentral1se commented

2023-02-14 07:25:35 +00:00

Documented current kludge approach: