Allow operator to specify arbitrary variables in their .env #532

New Issue

2025-04-14T23:33:43Z

FunPecan commented

2025-04-14 23:33:43 +00:00

This is closely related to #530 but I want to offer a broader solution.

PROBLEM - As a design choice, it seems like it will create an really high degree of friction to always need to define variables in the compose.yaml and .env.template before the operator can use them.

(NOT GREAT) SOLUTION #1 - If the operator always has to fork the repo to add new variables, that's a big increase in work for them.

(NOT GREAT) SOLUTION #2 - If the recipe maintainer always has to make sure to include every possible variable, that's a little more achievable. But it seems like it will always fall behind the upstream changes.

PROPOSED SOLUTION - I'm curious about a happy middle ground: Allow the operator to specify anything in .env and have abra (or a specific recipe) understand what to do with those arbtrary "unmapped" variables. At least for matrix.org, this would be a simple conversion from "UPPER_CASE_VARIABLE=true" (.env) to "lower_case_variable: true" (in the homeserver.yaml). Maybe other recipes would need different configuration to do the translation properly.

Curious what y'all think.

This is closely related to #530 but I want to offer a broader solution. PROBLEM - As a design choice, it seems like it will create an really high degree of friction to always need to define variables in the compose.yaml and .env.template before the operator can use them. (NOT GREAT) SOLUTION #1 - If the operator always has to fork the repo to add new variables, that's a big increase in work for them. (NOT GREAT) SOLUTION #2 - If the recipe maintainer always has to make sure to include every possible variable, that's a little more achievable. But it seems like it will always fall behind the upstream changes. PROPOSED SOLUTION - I'm curious about a happy middle ground: Allow the operator to specify anything in .env and have abra (or a specific recipe) understand what to do with those arbtrary "unmapped" variables. At least for matrix.org, this would be a simple conversion from "UPPER_CASE_VARIABLE=true" (.env) to "lower_case_variable: true" (in the homeserver.yaml). Maybe other recipes would need different configuration to do the translation properly. Curious what y'all think.

FunPecan changed title from ~~Allow arbitrary variables through from .env~~ to Allow operator to specify arbitrary variables in their .env

2025-04-14 23:34:39 +00:00

decentral1se commented

2025-04-15 07:05:11 +00:00

it seems like it will create an really high degree of friction

I know it might seem like a total pain in the ass as a newcomer but we've been doing it like this roughly 4-5 years now. When I say "we" I mean the initiators of the project but also all the new people that joined over the years. I would say this doesn't really strike me as a major issue and mostly just an annoying hoop to jump through which most people pick up after a few hacking sessions and just move on. You'll see the wealth of recipes that have been packaged and this would be the first time someone said "damn we need to smash this specific hoop that we're jumping through".

Allow the operator to specify anything in .env and have abra (or a specific recipe) understand what to do with those arbtrary "unmapped" variables.

Experience has shown that it's better that abra does less and less and specifically as less "magic" as possible. We've reduced so many things that we thought "would be good" that turned out to be maintenance issues or not applicable or didn't work completely for everyone. This seems like one of those things that will be v hard to get right and I'm not sure it's worth the effort given my comment above. We also need to remember that specific env vars need to go in specific compose files (e.g. compose.smtp.yml) and abra would have to handle all that as well. I'm v doubtful of that working out.

I'm open to hear what the rest think but I'm generally in favour of just sticking with the current imperfect solution of jumping through the hoop. I'm all for #530.

> it seems like it will create an really high degree of friction I know it might seem like a total pain in the ass as a newcomer but we've been doing it like this roughly 4-5 years now. When I say "we" I mean the initiators of the project but also all the new people that joined over the years. I would say this doesn't really strike me as a major issue and mostly just an annoying hoop to jump through which most people pick up after a few hacking sessions and just move on. You'll see the wealth of recipes that have been packaged and this would be the first time someone said "damn we need to smash this specific hoop that we're jumping through". > Allow the operator to specify anything in .env and have abra (or a specific recipe) understand what to do with those arbtrary "unmapped" variables. Experience has shown that it's better that `abra` does less and less and specifically as less "magic" as possible. We've reduced so many things that we thought "would be good" that turned out to be maintenance issues or not applicable or didn't work completely for everyone. This seems like one of those things that will be v hard to get right and I'm not sure it's worth the effort given my comment above. We also need to remember that specific env vars need to go in specific compose files (e.g. `compose.smtp.yml`) and `abra` would have to handle all that as well. I'm v doubtful of that working out. I'm open to hear what the rest think but I'm generally in favour of just sticking with the current imperfect solution of jumping through the hoop. I'm all for https://git.coopcloud.tech/toolshed/abra/issues/530.

decentral1se added the question label 2025-04-15 07:06:14 +00:00

FunPecan commented

2025-04-15 15:57:51 +00:00

Gotcha. Yeah I agree that it seems hard to implement reliably.

#530 is definitely better than nothing. And definitely simpler!

I guess the solution (in addition to #530) is just: attempt to maintain a robust enough compose.yml that most operators won't encounter any config variables they can't access. Is that right?

Gotcha. Yeah I agree that it seems hard to implement reliably. #530 is definitely better than nothing. And definitely simpler! I guess the solution (in addition to #530) is just: attempt to maintain a robust enough `compose.yml` that most operators won't encounter any config variables they can't access. Is that right?

decentral1se commented

2025-04-16 05:15:16 +00:00

@FunPecan yeh, that's it. In practice, we see people add these things over time and it's generally "OK" and not so much of a hindrance. There's like a handful of things that the Swarm runtime make awkward and probably if we could start over, we'd do things differently 🙃 Also the docs do generally help people: here. If anyone want's to get a PR in for #530, I'll review it! I'm gonna close this for now.

@FunPecan yeh, that's it. In practice, we see people add these things over time and it's generally "OK" and not so much of a hindrance. There's like a handful of things that the Swarm runtime make awkward and probably if we could start over, we'd do things differently 🙃 Also the docs do generally help people: [here](https://docs.coopcloud.tech/maintainers/handbook/#manage-environment-variables). If anyone want's to get a PR in for #530, I'll review it! I'm gonna close this for now.

decentral1se closed this issue

2025-04-16 05:15:16 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/abra#532