coop-cloud/lasuite-docs
2
0
Fork 1
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Fix Recipe And Properly Use Secrets #8

Merged
notplants merged 19 commits from secrets into main 2025-11-07 15:38:10 +00:00
Conversation 14 Commits 19 Files Changed 6 +155 -45
notplants commented 2025-10-31 19:44:53 +00:00
Owner
Copy Link
No description provided.
👀 1
notplants added 4 commits 2025-10-31 19:44:54 +00:00
Add alias of lasuite-app instead of app to prevent docker DNS name conflict 9a4d5058f9
working on secrets 51c939dd2c
almost working with docker secrets 184255b249
almost working secrets 664d0df1f6
notplants added 1 commit 2025-11-01 20:34:20 +00:00
Attempt at universal abra wrapper 293dd80e53
notplants added 3 commits 2025-11-02 19:38:06 +00:00
no unique db_password secret 3a33573fba
still working 3e4cf66594
db password stops working with secret d863bdbe81
decentral1se commented 2025-11-04 13:58:12 +00:00
Owner
Copy Link

@notplants 👋 I have an issue on the abra side of things, but I am wondering if that is just an issue with this recipe configuration. Here's the issue: toolshed/abra#692 It looks like there are just issues with the secret handling here? I'm not sure, I'll keep looking at it 🤔

@notplants 👋 I have an issue on the `abra` side of things, but I am wondering if that is just an issue with this recipe configuration. Here's the issue: https://git.coopcloud.tech/toolshed/abra/issues/692 It looks like there are just issues with the secret handling here? I'm not sure, I'll keep looking at it 🤔
notplants added 3 commits 2025-11-04 19:20:39 +00:00
Database working with secrets 6a0ebbd641
working with secrets 8bda1893f3
working with secrets 761320788b
notplants commented 2025-11-04 19:32:17 +00:00
Author
Owner
Copy Link

hi @decentral1se I've just now gotten a version of this that is fully working with secrets for everything other than email, and was still hoping to get a code review to see if I've done things properly

figuring out all the configurations for the different services and how they interact with secrets had some little unexpected gotchas on the way... but appears to be working now

I actually haven't tested out the email related configs or secrets yet so I would still need to look into that

hi @decentral1se I've just now gotten a version of this that is fully working with secrets for everything other than email, and was still hoping to get a code review to see if I've done things properly figuring out all the configurations for the different services and how they interact with secrets had some little unexpected gotchas on the way... but appears to be working now I actually haven't tested out the email related configs or secrets yet so I would still need to look into that
❤️ 1
notplants commented 2025-11-04 19:35:22 +00:00
Author
Owner
Copy Link

remaining things that could be done:

  • didn't do anything to connect email

  • currently I have included all secrets in the secrets configuration for each of the services. I could by trial of elimiation figure out the minimum secrets that each service needs in order for things to still work (but this sounds a bit tedious)

  • add documentation for the configurations needed within keycloak to get this to work
    (in short: add realm, add client, add user. take client secret key from keycloak for that client and insert it into abra via abra app secret insert d5.swarm.docs.coop oidc_rp_client_secret v2 yoursecrethere --chaos and increment secret number appropriatley in config)

  • there are a couple things remaining that I should put into secrets also (Y_PROVIDER_API_KEY, anything else?)

additional questions:

  • i needed to also add secret exporting to abra.sh in addition to abra-entrypoint? is there a way to reduce this error-prone boilerplate? or generally how are the patterns I've used here for secret mangement?

  • it occurs to me that oidc_rp_client_secret is the only "real" secret, that comes from keycloak, and is not just some internal wiring. I guess its good practice to keep DB passwords etc. in secrets, but in theory they should already be protected by docker networking configuration? even if the password was just password... but I guess this provides multiple layers of defense?

  • anything else that looks weird? (I wrote these questions down before seeing the comment from @decentralize. so don't want to put you on the spot to answer all of them ... but just writing them down here for now)

**remaining things that could be done:** - didn't do anything to connect email - currently I have included all secrets in the secrets configuration for each of the services. I could by trial of elimiation figure out the minimum secrets that each service needs in order for things to still work (but this sounds a bit tedious) - add documentation for the configurations needed within keycloak to get this to work (in short: add realm, add client, add user. take client secret key from keycloak for that client and insert it into abra via `abra app secret insert d5.swarm.docs.coop oidc_rp_client_secret v2 yoursecrethere --chaos` and increment secret number appropriatley in config) - there are a couple things remaining that I should put into secrets also (Y_PROVIDER_API_KEY, anything else?) **additional questions:** - i needed to also add secret exporting to abra.sh in addition to abra-entrypoint? is there a way to reduce this error-prone boilerplate? or generally how are the patterns I've used here for secret mangement? - it occurs to me that oidc_rp_client_secret is the only "real" secret, that comes from keycloak, and is not just some internal wiring. I guess its good practice to keep DB passwords etc. in secrets, but in theory they should already be protected by docker networking configuration? even if the password was just password... but I guess this provides multiple layers of defense? - anything else that looks weird? (I wrote these questions down before seeing the comment from @decentralize. so don't want to put you on the spot to answer all of them ... but just writing them down here for now)
3wordchant commented 2025-11-04 20:25:52 +00:00
Owner
Copy Link

i needed to also add secret exporting to abra.sh in addition to abra-entrypoint?

The longterm answer is to patch upstream to accept _FILE variants for all secrets.

The "standard" interim measure is to make the custom entrypoint handle an -e ("env") option which bails from the actual entrypoint commands – then abra.sh can call the entrypoint with the -e option to load the secrets. Example from Outline: https://git.coopcloud.tech/coop-cloud/outline/src/branch/main/entrypoint.sh.tmpl#L24-L27

I guess its good practice to keep DB passwords etc. in secrets, but in theory they should already be protected by docker networking configuration? even if the password was just password... but I guess this provides multiple layers of defense?

Yes. Given that both Mysql / Mariadb and Postgres support _FILE vars, it's seemed worth the minimal extra friction to add another layer of defence – the complexity of Docker networking makes it not seem 100% certain that we can always rely that remote access won't be possible (although it definitely shouldn't be).

anything else that looks weird?

Personally I always (try to?) use file_env instead of a bare SOME_SECRET=$(cat ..., because it does additional nice things like unsetting SOME_SECRET_FILE, bailing if both SOME_SECRET and SOME_SECRET_FILE are set. It looks like this was removed in outline, maybe @decentral1se has some context about why which could be relevant here? (The commit message says it's about bash → sh, but I'm pretty sure there's a file_env variant which works in POSIX sh, but I can't find it now).

I'm adding a couple of inline comments also.

> i needed to also add secret exporting to abra.sh in addition to abra-entrypoint? The longterm answer is to patch upstream to accept `_FILE` variants for all secrets. The "standard" interim measure is to make the custom entrypoint handle an `-e` ("env") option which bails from the actual entrypoint commands – then `abra.sh` can call the entrypoint with the `-e` option to load the secrets. Example from Outline: https://git.coopcloud.tech/coop-cloud/outline/src/branch/main/entrypoint.sh.tmpl#L24-L27 > I guess its good practice to keep DB passwords etc. in secrets, but in theory they should already be protected by docker networking configuration? even if the password was just password... but I guess this provides multiple layers of defense? Yes. Given that both Mysql / Mariadb and Postgres support `_FILE` vars, it's seemed worth the minimal extra friction to add another layer of defence – the complexity of Docker networking makes it not seem 100% certain that we can always rely that remote access won't be possible (although it definitely shouldn't be). > anything else that looks weird? Personally I always (try to?) use `file_env` instead of a bare `SOME_SECRET=$(cat ...`, because it does additional nice things like unsetting `SOME_SECRET_FILE`, bailing if both `SOME_SECRET` and `SOME_SECRET_FILE` are set. It looks like [this was removed in outline](https://git.coopcloud.tech/coop-cloud/outline/commit/6bfb2ac153fbb7600c039850007f5b6374d4a186), maybe @decentral1se has some context about why which could be relevant here? (The commit message says it's about `bash` → `sh`, but I'm pretty sure there's a `file_env` variant which works in POSIX `sh`, but I can't find it now). I'm adding a couple of inline comments also.
❤️ 1
3wordchant approved these changes 2025-11-04 20:41:09 +00:00
3wordchant left a comment
Owner
Copy Link

👍 ing this because optimistic merging; I don't personally host an instance of this any more, but if I did I would love to see these comments addressed soonishly.

:+1: ing this because [optimistic merging](https://dmerej.info/blog/post/optimistic-merging/); I don't personally host an instance of this any more, but if I did I would love to see these comments addressed soonishly.
abra-entrypoint.sh Outdated
@ -0,0 +4,4 @@
ORIGINAL_ENTRYPOINT="$1"
shift
[ -f /run/secrets/postgres_password ] && export DB_PASSWORD="$(cat /run/secrets/postgres_password)"
3wordchant commented 2025-11-04 20:38:18 +00:00
Owner
Copy Link

Ah another thing would be handy, which I think file_env does, is bail if neither DB_PASSWORD nor DB_PASSWORD_FILE is set.

Ah another thing would be handy, which I think `file_env` does, is bail if neither `DB_PASSWORD` nor `DB_PASSWORD_FILE` is set.
abra-entrypoint.sh Outdated
@ -0,0 +5,4 @@
shift
[ -f /run/secrets/postgres_password ] && export DB_PASSWORD="$(cat /run/secrets/postgres_password)"
[ -f /run/secrets/postgres_password ] && export POSTGRES_PASSWORD="$(cat /run/secrets/postgres_password)"
3wordchant commented 2025-11-04 20:39:39 +00:00
Owner
Copy Link

And, related to above, I think loading the secret path from e.g. POSTGRES_PASSWORD_FILE is preferable to hard-coding the path here, for 2 reasons:

  1. It prefigures the utopian future where all apps add _FILE variants of their environment variables
  2. It's more consistent with the other services in the stack
And, related to above, I think loading the secret path from e.g. `POSTGRES_PASSWORD_FILE` is preferable to hard-coding the path here, for 2 reasons: 1. It prefigures the utopian future where all apps add `_FILE` variants of their environment variables 2. It's more consistent with the other services in the stack
compose.yml Outdated
@ -89,1 +85,3 @@
- backend
backend:
aliases:
- lasuite-app
3wordchant commented 2025-11-04 20:28:21 +00:00
Owner
Copy Link

If the problem is that the web container isn't resolving app consistently – which has definitely happened to me before, because it's on the proxy network so it has access to several containers named app – then I think using $(STACK_NAME}_app in the Nginx config would be a better solution; less complexity in the compose file, and less chance of problems if there are >1 lasuite-docs instances in the same swarm.

If the problem is that the `web` container isn't resolving `app` consistently – which has definitely happened to me before, because it's on the `proxy` network so it has access to several containers named `app` – then I think using `$(STACK_NAME}_app` in the Nginx config would be a better solution; less complexity in the compose file, and less chance of problems if there are >1 lasuite-docs instances in the same swarm.
compose.yml Outdated
@ -142,2 +179,4 @@
volumes:
- postgres:/var/lib/postgresql/data/pgdata
command: ["postgres"]
entrypoint: ["/abra-entrypoint.sh", "docker-entrypoint.sh"]
3wordchant commented 2025-11-04 20:31:30 +00:00
Owner
Copy Link

It seems a bit unusual to have the same entrypoint used for services with different image; these situations are usually handled with a separate entrypoint file. But are you sure this is necessary? The postgresql image handles _FILE variables itself: https://hub.docker.com/_/postgres#docker-secrets

It seems a bit unusual to have the same entrypoint used for services with different `image`; these situations are usually handled with a separate entrypoint file. But are you sure this is necessary? The `postgresql` image handles `_FILE` variables itself: https://hub.docker.com/_/postgres#docker-secrets
compose.yml Outdated
@ -187,3 +243,4 @@
networks:
- backend
command: minio server /data
entrypoint: ["/abra-entrypoint.sh", "/usr/bin/docker-entrypoint.sh"]
3wordchant commented 2025-11-04 20:34:49 +00:00
Owner
Copy Link

Same comment about entrypoint as from Postgres; Minio also natively supports Docker Secrets (the dream): https://github.com/minio/minio/blob/master/docs/docker/README.md#minio-custom-access-and-secret-keys-using-docker-secrets

Same comment about entrypoint as from Postgres; Minio also natively supports Docker Secrets (the dream): https://github.com/minio/minio/blob/master/docs/docker/README.md#minio-custom-access-and-secret-keys-using-docker-secrets
compose.yml Outdated
@ -230,0 +315,4 @@
name: ${STACK_NAME}_postgres_password_${SECRET_POSTGRES_PASSWORD_VERSION}
collaboration_server_secret:
external: true
name: ${STACK_NAME}_collaboration_server_secret_${SECRET_COLLABORATION_SERVER_SECRET_VERSION}
3wordchant commented 2025-11-04 20:36:12 +00:00
Owner
Copy Link

Does abra recipe lint warn about this? There's a maximum length for Docker secret names which can be a bit of a landmine because the final secret name depends on STACK_NAME. This maybe seems like a good candidate for abbreviating.

Does `abra recipe lint` warn about this? There's a maximum length for Docker secret names which can be a bit of a landmine because the final secret name depends on `STACK_NAME`. This maybe seems like a good candidate for abbreviating.
nginx.conf Outdated
@ -4,3 +4,3 @@
upstream docs_frontend {
server app:8080 fail_timeout=0;
server lasuite-app:8080 fail_timeout=0;
3wordchant commented 2025-11-04 20:37:32 +00:00
Owner
Copy Link

See above comment about using STACK_NAME (might require making this into a Golang template; I think Nginx can't access environment variables inside .conf files by default). Example here: https://git.coopcloud.tech/coop-cloud/nextcloud/src/branch/main/nginx.conf.tmpl#L34

See above comment about using `STACK_NAME` (might require making this into a Golang template; I think Nginx can't access environment variables inside `.conf` files by default). Example here: https://git.coopcloud.tech/coop-cloud/nextcloud/src/branch/main/nginx.conf.tmpl#L34
👍 1
3wordchant commented 2025-11-04 20:42:32 +00:00
Owner
Copy Link

Oh and one more on:

it occurs to me that oidc_rp_client_secret is the only "real" secret, that comes from keycloak, and is not just some internal wiring.

The Minio instance is web-accessible, so I think MINIO_ROOT_PASSWORD is also fairly "real".

Anyway, great work overall, it's a fairly gnarly stack and these all seem like sensible improvements in the right direction 👌

Oh and one more on: > it occurs to me that oidc_rp_client_secret is the only "real" secret, that comes from keycloak, and is not just some internal wiring. The Minio instance is web-accessible, so I think `MINIO_ROOT_PASSWORD` is also fairly "real". --- Anyway, great work overall, it's a fairly gnarly stack and these all seem like sensible improvements in the right direction 👌
❤️ 1
notplants added 1 commit 2025-11-05 16:30:38 +00:00
working on nginx.conf.tmpl 82ca119cdb
3wordchant approved these changes 2025-11-05 17:07:56 +00:00
.env.sample Outdated
@ -34,3 +34,1 @@
OIDC_RP_CLIENT_ID=impress
# FIXME: Move to docker secret
OIDC_RP_CLIENT_SECRET=example
OIDC_OP_JWKS_ENDPOINT=https://auth.${DOMAIN}/realms/${DOMAIN}/protocol/openid-connect/certs
3wordchant commented 2025-11-05 17:07:50 +00:00
Owner
Copy Link

Changing from hardcoding the default as impress certainly seems like an improvement, but I haven't seen many (any?) cases where the realm is a FQDN. Maybe a separate OIDC_REALM var, and then using ${OIDC_REALM} in the other URLs, would be the best? This is a very low-priority suggestion though, because everyone will likely need to edit these anyway.

(And I think the longterm solution is for docs to just please be normal and use an OIDC configuration URL instead of this sheaf of settings; I opened a ticket on their issue tracker for this: https://github.com/suitenumerique/docs/issues/1562 )

Changing from hardcoding the default as `impress` certainly seems like an improvement, but I haven't seen many (any?) cases where the realm is a FQDN. Maybe a separate `OIDC_REALM` var, and then using `${OIDC_REALM}` in the other URLs, would be the best? This is a very low-priority suggestion though, because everyone will likely need to edit these anyway. (And I think the longterm solution is for docs to just please be normal and use an OIDC configuration URL instead of this sheaf of settings; I opened a ticket on their issue tracker for this: https://github.com/suitenumerique/docs/issues/1562 )
notplants commented 2025-11-06 16:42:30 +00:00
Author
Owner
Copy Link

agreed OIDC_REALM var makes more sense. just changed it.

I just wanted it to not be hardcoded as impress, because when I was first working on this recipe I got thrown off not realizing impress was arbitrary

agreed OIDC_REALM var makes more sense. just changed it. I just wanted it to not be hardcoded as impress, because when I was first working on this recipe I got thrown off not realizing impress was arbitrary
🎉 1
notplants marked this conversation as resolved
notplants commented 2025-11-05 20:11:44 +00:00
Author
Owner
Copy Link

@3wordchant thanks for the code review, I appreciate it ! all very helpful

will incorporate these suggestions

@3wordchant thanks for the code review, I appreciate it ! all very helpful will incorporate these suggestions
❤️ 2
notplants commented 2025-11-06 17:20:36 +00:00
Author
Owner
Copy Link

Personally I always (try to?) use file_env instead of a bare SOME_SECRET=$(cat ..., because it does additional nice things like unsetting SOME_SECRET_FILE, bailing if both SOME_SECRET and SOME_SECRET_FILE are set. It looks like this was removed in outline, maybe @decentral1se has some context about why which could be relevant here? (The commit message says it's about bash → sh, but I'm pretty sure there's a file_env variant which works in POSIX sh, but I can't find it now).

Indeed I was using SOME_SECRET=$(cat ...) here because not all the containers have bash. The coop cloud docs here say to do it this way, so maybe there isn't a file_env variant for POSIX sh? (cc @decentral1se)

It seems a bit unusual to have the same entrypoint used for services with different image; these situations are usually handled with a separate entrypoint file. But are you sure this is necessary?

I see how having a separate entrypoint for each container would be more logical in a way, but it feels like a lot of extra files to make since there are so many containers. Sharing an entrypoint feels like maybe less boilerplate? devops rant: tbh, I found myself wishing that docker compose provided some type of optional flags to the secret property itself so that it could just directly be exported as an env var ... given how docker works I see how if all upstreams had SECRET_FILE option that would be preferred and is more secure ... but given thats not universally the case seems like why not add the feature, and in a way I kind of like env vars as a standard that works across many different environments even beyond docker (but in any event "lord grant me the serenity to accept the things I cannot change etc") ... but I could imagine some version of abra that optionally used a superset of the docker compose spec to add this feature, and could even transform and output standard docker compose files + secret-setting-entrypoint-wrappers for backwards-compatability (or just silently make it work)

I think loading the secret path from e.g. POSTGRES_PASSWORD_FILE is preferable to hard-coding

tbh, I tried first using SECRET_FILE for all the secrets I could. but I was running into things inexplicably not working, and so sanity-check back-tracked to using the plain env vars directly, which I had a known working version of before introducing secrets. later on I realized that things were not working not because of SECRET vs SECRET_FILE, but because there were multiple places that the secrets needed to be injected that I was not aware of at first (in the main entrypoints, but also for healthchecks, manually when you connect a shell, and in abra.sh). just sharing this as info for the small stumbling blocks I ran into as someone getting started with coop cloud recipes ... short of automating this away, maybe a "Common Footguns and Issues" section could also be helpful in the coop cloud packaging handbook? or again maybe just having a forum to make common issues more visible ... and also I figured it out eventually with some help

> Personally I always (try to?) use file_env instead of a bare SOME_SECRET=$(cat ..., because it does additional nice things like unsetting SOME_SECRET_FILE, bailing if both SOME_SECRET and SOME_SECRET_FILE are set. It looks like this was removed in outline, maybe @decentral1se has some context about why which could be relevant here? (The commit message says it's about bash → sh, but I'm pretty sure there's a file_env variant which works in POSIX sh, but I can't find it now). Indeed I was using `SOME_SECRET=$(cat ...)` here because not all the containers have bash. The coop cloud docs [here](https://docs.coopcloud.tech/maintainers/handbook/#binbash-is-missing) say to do it this way, so maybe there isn't a file_env variant for POSIX sh? (cc @decentral1se) > It seems a bit unusual to have the same entrypoint used for services with different image; these situations are usually handled with a separate entrypoint file. But are you sure this is necessary? I see how having a separate entrypoint for each container would be more logical in a way, but it feels like a lot of extra files to make since there are so many containers. Sharing an entrypoint feels like maybe less boilerplate? devops rant: tbh, I found myself wishing that docker compose provided some type of optional flags to the secret property itself so that it could just directly be exported as an env var ... given how docker works I see how if all upstreams had SECRET_FILE option that would be preferred and is more secure ... but given thats not universally the case seems like why not add the feature, and in a way I kind of like env vars as a standard that works across many different environments even beyond docker (but in any event "lord grant me the serenity to accept the things I cannot change etc") ... but I could imagine some version of abra that optionally used a superset of the docker compose spec to add this feature, and could even transform and output standard docker compose files + secret-setting-entrypoint-wrappers for backwards-compatability (or just silently make it work) > I think loading the secret path from e.g. POSTGRES_PASSWORD_FILE is preferable to hard-coding tbh, I tried first using SECRET_FILE for all the secrets I could. but I was running into things inexplicably not working, and so sanity-check back-tracked to using the plain env vars directly, which I had a known working version of before introducing secrets. later on I realized that things were not working not because of SECRET vs SECRET_FILE, but because there were multiple places that the secrets needed to be injected that I was not aware of at first (in the main entrypoints, but also for healthchecks, manually when you connect a shell, and in abra.sh). just sharing this as info for the small stumbling blocks I ran into as someone getting started with coop cloud recipes ... short of automating this away, maybe a "Common Footguns and Issues" section could also be helpful in the coop cloud packaging handbook? or again maybe just having a forum to make common issues more visible ... and also I figured it out eventually with some help
3wordchant commented 2025-11-06 17:39:49 +00:00
Owner
Copy Link

The coop cloud docs here say to do it this way, so maybe there isn't a file_env variant for POSIX sh?

Ah great find. Yeah on that basis let's just stick with the $(🐈... for now.

I see how having a separate entrypoint for each container would be more logical in a way, but it feels like a lot of extra files to make since there are so many containers

To be clear I was suggesting custom entrypoint per image, not per container (so e.g. backend and celery could share one). And, pending learning otherwise, I don't think the db, redis, web, minio, minio-bootstrap containers need custom entrypoints (because they all either don't need secrets, or natively support loading them from files) – so we'd be talking 2 different entrypoints (for backend / celery, and frontend), not 8.

But the higher-priority suggestion would be to remove the custom entrypoint from the Completely Different containers which AFAIK don't require it, namely db, redis, web, minio, minio-bootstrap.

I found myself wishing that docker compose provided some type of optional flags to the secret property itself so that it could just directly be exported as an env var

Agreed. Dare we open an issue on docker/cli about this?

in a way I kind of like env vars as a standard that works across many different environments even beyond docker

I think loading secrets from files is increasingly-standard as well, some enlightened apps support that as part of the env-var-loading code for any var. Although it's interesting that koobernetus seemingly allows loading its secrets as env vars as you're describing: https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#define-container-environment-variables-using-secret-data

I could imagine some version of abra that optionally used a superset of the docker compose spec to add this feature, and could even transform and output standard docker compose files + secret-setting-entrypoint-wrappers for backwards-compatability (or just silently make it work)

"Maybe". I'm still curious to see how far we can ride "co-op cloud recipes are usable immediately without abra", which I think this would end. That might happen at some point, but I think it would be good to make an intentional project-wide decision if and when it does.

tbh, I tried first using SECRET_FILE for all the secrets I could. but I was running into things inexplicably not working,

Could also be that containers which handle _FILE natively were confused if both were present? Can't remember off-top what happens in those situations.

"Common Footguns and Issues" section could also be helpful in the coop cloud packaging handbook?

Agreed, great suggestion.

> The coop cloud docs [here](https://docs.coopcloud.tech/maintainers/handbook/#binbash-is-missing) say to do it this way, so maybe there isn't a `file_env` variant for POSIX sh? Ah great find. Yeah on that basis let's just stick with the `$(🐈...` for now. > I see how having a separate entrypoint for each container would be more logical in a way, but it feels like a lot of extra files to make since there are so many containers To be clear I was suggesting custom entrypoint per `image`, not per container (so e.g. `backend` and `celery` could share one). And, pending learning otherwise, I don't think the `db`, `redis`, `web`, `minio`, `minio-bootstrap` containers need custom entrypoints (because they all either don't need secrets, or natively support loading them from files) – so we'd be talking 2 different entrypoints (for `backend` / `celery`, and `frontend`), not 8. But the higher-priority suggestion would be to remove the custom entrypoint from the Completely Different containers which AFAIK don't require it, namely `db`, `redis`, `web`, `minio`, `minio-bootstrap`. > I found myself wishing that docker compose provided some type of optional flags to the secret property itself so that it could just directly be exported as an env var Agreed. Dare we [open an issue on `docker/cli`](https://github.com/docker/cli/issues) about this? > in a way I kind of like env vars as a standard that works across many different environments even beyond docker I think loading secrets from files is increasingly-standard as well, some enlightened apps support that as part of the env-var-loading code for _any_ var. Although it's interesting that koobernetus seemingly allows loading its secrets as env vars as you're describing: https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#define-container-environment-variables-using-secret-data > I could imagine some version of abra that optionally used a superset of the docker compose spec to add this feature, and could even transform and output standard docker compose files + secret-setting-entrypoint-wrappers for backwards-compatability (or just silently make it work) "Maybe". I'm still curious to see how far we can ride "co-op cloud recipes are usable immediately without `abra`", which I think this would end. That might happen at some point, but I think it would be good to make an intentional project-wide decision if and when it does. > tbh, I tried first using SECRET_FILE for all the secrets I could. but I was running into things inexplicably not working, Could also be that containers which handle `_FILE` natively were confused if both were present? Can't remember off-top what happens in those situations. > "Common Footguns and Issues" section could also be helpful in the coop cloud packaging handbook? Agreed, great suggestion.
👍 1
notplants added 3 commits 2025-11-06 20:02:17 +00:00
working before secret shortening 54acd65edd
working with abra recipe lint 9c6bb70b7a
working other than minio_root_password_file 7846746050
notplants commented 2025-11-06 20:18:34 +00:00
Author
Owner
Copy Link

"Maybe". I'm still curious to see how far we can ride "co-op cloud recipes are usable immediately without abra", which I think this would end. That might happen at some point, but I think it would be good to make an intentional project-wide decision if and when it does.

makes sense

Agreed. Dare we open an issue on docker/cli about this?

I would guess they have heard this request before... but couldn't hurt? if they did add it ... would have saved hours working on this recipe (and probably others) ...

To be clear I was suggesting custom entrypoint per image, not per container (so e.g. backend and celery could share one). And, pending learning otherwise, I don't think the db, redis, web, minio, minio-bootstrap containers need custom entrypoints (because they all either don't need secrets, or natively support loading them from files) – so we'd be talking 2 different entrypoints (for backend / celery, and frontend), not 8. But the higher-priority suggestion would be to remove the custom entrypoint from the Completely Different containers which AFAIK don't require it, namely db, redis, web, minio, minio-bootstrap.

  • I was able to get SECRET_FILE working with the db service so that it no longer required the abra-entrpoint

  • with minio I tried but it is still not working. branch with attempt is here. from my reading, it seems to require adding yet another secret (for minio_root_user_file), as well as getting minio-bootstrap to also work with this. I'm sure I'm just missing some little detail, but the devil is in the details. Any additional security by using _FILE here is also lost because the backend service needs to rexport these secrets as env vars anyway for AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY (afaik unless they also have some undocumented _FILE support)... so I'm tempted to just leave it as using MINIO_ROOT_PASSWORD with abra-entrypoint.sh instead of MINIO_ROOT_PASSWORD_FILE because it is working ... even though I would like to follow best practice

  • it looks like the y-provider service also requires a secret as an env var.
    ... so in ideal version: backend/celery could share an entrypoint, and y-provider could have a second entrypoint, and minio and minio-boostrap would use _FILE
    ... in current version: backend/celery/yprovider/minio all share one entrypoint that exports the secrets as env vars

wondering if I should hunker down to fix this, or just continue for now and put this in an issue...

> "Maybe". I'm still curious to see how far we can ride "co-op cloud recipes are usable immediately without abra", which I think this would end. That might happen at some point, but I think it would be good to make an intentional project-wide decision if and when it does. makes sense > Agreed. Dare we open an issue on docker/cli about this? I would guess they have heard this request before... but couldn't hurt? if they did add it ... would have saved hours working on this recipe (and probably others) ... > To be clear I was suggesting custom entrypoint per image, not per container (so e.g. backend and celery could share one). And, pending learning otherwise, I don't think the db, redis, web, minio, minio-bootstrap containers need custom entrypoints (because they all either don't need secrets, or natively support loading them from files) – so we'd be talking 2 different entrypoints (for backend / celery, and frontend), not 8. But the higher-priority suggestion would be to remove the custom entrypoint from the Completely Different containers which AFAIK don't require it, namely db, redis, web, minio, minio-bootstrap. - I was able to get SECRET_FILE working with the db service so that it no longer required the abra-entrpoint - with minio I tried but it is still not working. branch with attempt is [here](https://git.coopcloud.tech/coop-cloud/lasuite-docs/src/branch/minio_root_password_file). from my reading, it seems to require adding yet another secret (for minio_root_user_file), as well as getting minio-bootstrap to also work with this. I'm sure I'm just missing some little detail, but the devil is in the details. Any additional security by using _FILE here is also lost because the backend service needs to rexport these secrets as env vars anyway for AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY (afaik unless they also have some undocumented _FILE support)... so I'm tempted to just leave it as using MINIO_ROOT_PASSWORD with abra-entrypoint.sh instead of MINIO_ROOT_PASSWORD_FILE because it is working ... even though I would like to follow best practice - it looks like the y-provider service also requires a secret as an env var. ... so in ideal version: backend/celery could share an entrypoint, and y-provider could have a second entrypoint, and minio and minio-boostrap would use _FILE ... in current version: backend/celery/yprovider/minio all share one entrypoint that exports the secrets as env vars wondering if I should hunker down to fix this, or just continue for now and put this in an issue...
notplants commented 2025-11-06 20:24:33 +00:00
Author
Owner
Copy Link

I also shortened all the secrets to be less than 12 char now so that the acronyms are kind of weird now but abra lint is happy

still have a polishing todo list something like this:

  • use best practices (minio_root_password_file, then split out abra-entrypoint into two separate entrypoints, one for y-provider and one for backend/celery)
  • fix two more abra lint errors (enable healthchecks for all services, semver for images)
  • test backupbot
  • email
  • keycloak integration documentation
I also shortened all the secrets to be less than 12 char now so that the acronyms are kind of weird now but abra lint is happy still have a polishing todo list something like this: - use best practices (minio_root_password_file, then split out abra-entrypoint into two separate entrypoints, one for y-provider and one for backend/celery) - fix two more abra lint errors (enable healthchecks for all services, semver for images) - test backupbot - email - keycloak integration documentation
notplants commented 2025-11-07 14:52:46 +00:00
Author
Owner
Copy Link

Apologies of the excessive dev log about mino ... that was partly just comments for myself of where I was at

tried again this morning after sleep and got it working... was just missing additionally adding the minio_root_user secret to the backend container

Apologies of the excessive dev log about mino ... that was partly just comments for myself of where I was at tried again this morning after sleep and got it working... was just missing additionally adding the minio_root_user secret to the backend container
😆 1
notplants added 3 commits 2025-11-07 14:55:18 +00:00
attempt at minio_pass_file 720d305d66
trying to use minio_root_pass_file 41ad58f47d
minio using files for secrets instead of env vars b527d802ca
notplants added 1 commit 2025-11-07 15:33:06 +00:00
inline secret export for y-provider d82844ba59
notplants commented 2025-11-07 15:36:40 +00:00
Author
Owner
Copy Link

just inlined the secret export entrypoint for y-provider, so that now only celery and backend (with the same image) share abra-entrypoint.sh

appears to be working, merging this now 🔧

just inlined the secret export entrypoint for y-provider, so that now only celery and backend (with the same image) share abra-entrypoint.sh appears to be working, merging this now 🔧
🚀 1
notplants changed title from WIP: Fix Recipe And Properly Use Secrets to Fix Recipe And Properly Use Secrets 2025-11-07 15:36:59 +00:00
notplants merged commit 00b0c2e64e into main 2025-11-07 15:38:10 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
3wordchant
Labels
Clear labels
bug
Something is not working
duplicate
This issue or pull request already exists
enhancement
New feature
help wanted
Need some help
invalid
Something is wrong
question
More information is needed
wontfix
This won't be fixed
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 Apfelwurm appletalk arjan basebuilder Brooke carla cas codegod100 coopcloud cyrnel decentral1se fauno flancian Frando iexos javielico jmakdah2 joe-irving kawaiipunk knoflook kolaente lambdabundesverband marlon mayel mirsal moritz nicksellen notplants p4u1 pau pharaohgraphy renovate-bot ripclap rix rscmbbng sef simon sixsmith stevensting tobias trav val virtualboys wolcen wykwit xynosis yksflip
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: coop-cloud/lasuite-docs#8
No description provided.
Delete Branch "secrets"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?