Dealing with secret names that are too long #463

Closed
opened 2023-07-19 11:53:30 +00:00 by moritz · 8 comments
Member

In docker a secret name can have a maximum of 64 characters.
The matrix recipe contains a secret name called "registration_shared_secret_v1" that is already quite long with 30 characters. So only 34 characters are left for the domain part of the secret.
With this secret we got some problems as we have a domain that is already 30 characters long. Together with the matrix. subdomain we have 37 characters.
Now we are not able to setup the matrix recipe for this domain without either changing the recipe or changing the domain.

For the future I think there should be a length restriction on secret naming conventions and there should be a warning for long domains, so that you never choose a too long domain together with abra.

In docker a secret name can have a maximum of 64 characters. The matrix recipe contains a secret name called "registration_shared_secret_v1" that is already quite long with 30 characters. So only 34 characters are left for the domain part of the secret. With this secret we got some problems as we have a domain that is already 30 characters long. Together with the `matrix.` subdomain we have 37 characters. Now we are not able to setup the matrix recipe for this domain without either changing the recipe or changing the domain. For the future I think there should be a length restriction on secret naming conventions and there should be a warning for long domains, so that you never choose a too long domain together with abra.
Owner

Ah yes, that's true 😬 Sucks that you ran into this...

I vaguely remember dealing with this on the STACK_NAME which we internally trim via

pkg/config/app.go Lines 60 to 63 in a7ce71d6cf
if len(stackName) > 45 {
logrus.Debugf("trimming %s to %s to avoid runtime limits", stackName, stackName[:45])
stackName = stackName[:45]
}
. 45 was an arbitrary choice which leaves enough space on the end for other things...

Do you want to put a PR into the synapse recipe to rename registration_shared_secret to reg_shared_secret? It might save others future pain for the current code. We can write release notes and help people migrate their secrets.

Do you have ideas where we can put the length restriction for the secret naming that will be picked up early? We have recipe linting but unsure how many people use it.

The long domain name warning could help alright. Any ideas on what length we should start to warn at? Sometimes you can change a sub-domain but if you have a long domain, it could get annoying?

Thanks for opening!

Ah yes, that's true 😬 Sucks that you ran into this... I vaguely remember dealing with this on the `STACK_NAME` which we internally trim via https://git.coopcloud.tech/coop-cloud/abra/src/commit/a7ce71d6cfe0e47900cfff51db32885187f10647/pkg/config/app.go#L60-L63. 45 was an arbitrary choice which leaves enough space on the end for other things... Do you want to put a PR into the synapse recipe to rename `registration_shared_secret` to `reg_shared_secret`? It might save others future pain for the current code. We can write release notes and help people migrate their secrets. Do you have ideas where we can put the length restriction for the secret naming that will be picked up early? We have recipe linting but unsure how many people use it. The long domain name warning could help alright. Any ideas on what length we should start to warn at? Sometimes you can change a sub-domain but if you have a long domain, it could get annoying? Thanks for opening!
decentral1se added the
bug
label 2023-07-19 22:12:57 +00:00
Owner

Ah, I see 7b1b5c37ed00241b95c14b361247af278dea99cd (synapse recipe) now 👍

Ah, I see `7b1b5c37ed00241b95c14b361247af278dea99cd` (`synapse` recipe) now 👍
decentral1se added this to the Critical fixes project 2023-07-19 22:27:16 +00:00
Author
Member

Yes I already changed the synapse recipe with a release note.

Not sure what is a good length.

Here I listed the current secret name lengths.
7 ./backup-bot-two/.env.sample:#SECRET_SSH_KEY_VERSION=v1
7 ./firefly-iii/.env.sample:SECRET_APP_KEY_VERSION=v1           # length=32
7 ./invoiceninja/.env.sample:SECRET_APP_KEY_VERSION=v1
8 ./authentik/.env.sample:# SECRET_WEKAN_ID_VERSION=v1
8 ./bonfire/.env.sample:SECRET_SEEDS_PW_VERSION=v1
8 ./matrix-synapse/.env.sample:SECRET_MACAROON_VERSION=v1
8 ./rallly/.env.sample:SECRET_SMTP_PWD_VERSION=v1
8 ./redmine/.env.sample:SECRET_KEY_BASE_VERSION=v1
9 ./authentik/.env.sample:# SECRET_MATRIX_ID_VERSION=v1
9 ./hedgedoc/.env.sample:#SECRET_OAUTH_KEY_VERSION=v1
9 ./invoiceninja/.env.sample:SECRET_DB_PASSWD_VERSION=v1
9 ./nocodb/.env.sample:SECRET_NC_DB_URL_VERSION=v1
9 ./rocketchat/.env.sample:#SECRET_OAUTH_KEY_VERSION=v1
9 ./traefik/.env.sample:#SECRET_USERSFILE_VERSION=v1
10 ./authentik/.env.sample:SECRET_ADMIN_PASS_VERSION=v1
10 ./authentik/.env.sample:SECRET_EMAIL_PASS_VERSION=v1
10 ./authentik/.env.sample:SECRET_SECRET_KEY_VERSION=v1
10 ./authentik/.env.sample:# SECRET_VIKUNJA_ID_VERSION=v1
10 ./dashboard-recipe/.env.sample:SECRET_SECRET_KEY_VERSION=v1
10 ./gitea/.env.sample:SECRET_JWT_SECRET_VERSION=v1 # length=43
10 ./gitea/.env.sample:SECRET_SECRET_KEY_VERSION=v1 # length=64
10 ./hometown/.env.sample:SECRET_OTP_SECRET_VERSION=v1
10 ./invoiceninja/.env.sample:SECRET_API_SECRET_VERSION=v1
10 ./minio/.env.sample:SECRET_ACCESS_KEY_VERSION=v1 # length=40
10 ./minio/.env.sample:SECRET_SECRET_KEY_VERSION=v1 # length=20
10 ./nextcloud/.env.sample:# SECRET_BBB_SECRET_VERSION=v1
10 ./nextcloud/README.md:SECRET_BBB_SECRET_VERSION=v1
10 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_BBB_SECRET_VERSION=v1
10 ./ohmyform/.env.sample:SECRET_SECRET_KEY_VERSION=v1
10 ./onlyoffice/.env.sample:# SECRET_JWT_SECRET_VERSION=v1
10 ./outline/.env.sample:SECRET_SECRET_KEY_VERSION=v1  # length=64
10 ./rallly/.env.sample:SECRET_SECRET_KEY_VERSION=v1
10 ./vikunja/.env.sample:SECRET_JWT_SECRET_VERSION=v1
10 ./wallabag/.env.sample:SECRET_APP_SECRET_VERSION=v1
11 ./adapt_authoring/.env.sample:SECRET_SESSION_KEY_VERSION=v1
11 ./authentik/.env.sample:SECRET_ADMIN_TOKEN_VERSION=v1
11 ./authentik/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./discourse/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./etherpad/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./firefly-iii/.env.sample:SECRET_DB_PASSWORD_VERSION=v1       # length=32
11 ./gitea/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./gotosocial/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./hedgedoc/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./hometown/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./invidious/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./kanboard/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./keycloak/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./kimai/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./limesurvey/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./matrix-synapse/abra.sh:export SHARED_SECRET_AUTH_VERSION=v1
11 ./matrix-synapse/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./matrix-synapse/.env.sample:SECRET_FORM_SECRET_VERSION=v1
11 ./nextcloud/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./nocodb/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./onlyoffice/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./onlyoffice/release/2.0.0+7.1.1:SECRET_DB_PASSWORD_VERSION=v1
11 ./outline/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./phpservermon/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./pixelfed/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./rallly/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./redmine/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./vikunja/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./wallabag/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./wordpress/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
11 ./zammad/.env.sample:SECRET_DB_PASSWORD_VERSION=v1
12 ./authentik/.env.sample:# SECRET_NEXTCLOUD_ID_VERSION=v1
12 ./authentik/.env.sample:# SECRET_WEKAN_SECRET_VERSION=v1
12 ./authentik/.env.sample:# SECRET_WORDPRESS_ID_VERSION=v1
12 ./authentik/README.md:SECRET_NEXTCLOUD_ID_VERSION=v1
12 ./authentik/release/3.0.0+2023.2.3:    SECRET_NEXTCLOUD_ID_VERSION=v1
12 ./bonfire/.env.sample:SECRET_SIGNING_SALT_VERSION=v1 # length=128
12 ./matrix-synapse/.env.sample:SECRET_REGISTRATION_VERSION=v1
12 ./nextcloud/.env.sample:# SECRET_AUTHENTIK_ID_VERSION=v1
12 ./outline/.env.sample:SECRET_UTILS_SECRET_VERSION=v1  # length=64
12 ./vikunja/compose.oauth.yml:      - SECRET_OAUTH_SECRET_VERSION=V1
12 ./vikunja/.env.sample:# SECRET_OAUTH_SECRET_VERSION=v1
12 ./woodpecker/.env.sample:SECRET_AGENT_SECRET_VERSION=v1
12 ./wordpress/.env.sample:# SECRET_AUTHENTIK_ID_VERSION=v1
13 ./authentik/.env.sample:# SECRET_MATRIX_SECRET_VERSION=v1
13 ./authentik/.env.sample:# SECRET_MONITORING_ID_VERSION=v1
13 ./discourse/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1
13 ./gitea/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1
13 ./hometown/.env.sample:SECRET_SMTP_PASSWORD_VERSION=v1
13 ./matrix-synapse/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1
13 ./nextcloud/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1
13 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_SMTP_PASSWORD_VERSION=v1
13 ./ohmyform/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1
13 ./snikket/.env.sample:SECRET_COTURN_SECRET_VERSION=v1
13 ./vikunja/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1
13 ./wallabag/.env.sample:SECRET_SMTP_PASSWORD_VERSION=v1
13 ./wekan/.env.sample:SECRET_OAUTH2_SECRET_VERSION=v1
13 ./wordpress/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1
14 ./adapt_authoring/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./authentik/.env.sample:# SECRET_VIKUNJA_SECRET_VERSION=v1
14 ./collabora/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./filestash/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./gitea/.env.sample:SECRET_INTERNAL_TOKEN_VERSION=v1 # length=105
14 ./invoiceninja/.env.sample:SECRET_DB_ROOT_PASSWD_VERSION=v1
14 ./keycloak/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./kimai/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./lemmy/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./nextcloud/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./nextcloud/.env.sample:# SECRET_ONLYOFFICE_JWT_VERSION=v1
14 ./nextcloud/README.md:SECRET_ONLYOFFICE_JWT_VERSION=v1
14 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_ONLYOFFICE_JWT_VERSION=v1
14 ./ohmyform/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./outline/.env.sample:SECRET_AWS_SECRET_KEY_VERSION=v1
14 ./rocketchat/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1
14 ./traefik/.env.sample:#SECRET_OVH_APP_SECRET_VERSION=v1
15 ./backup-bot-two/.env.sample:SECRET_RESTIC_PASSWORD_VERSION=v1
15 ./bonfire/.env.sample:SECRET_ENCRYPTION_SALT_VERSION=v1 # length=128
15 ./bonfire/.env.sample:SECRET_SECRET_KEY_BASE_VERSION=v1 # length=128
15 ./hometown/.env.sample:SECRET_SECRET_KEY_BASE_VERSION=v1
15 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_AS_TOKEN_VERSION=v1
15 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_HS_TOKEN_VERSION=v1
15 ./traefik/.env.sample:#SECRET_GANDIV5_API_KEY_VERSION=v1
16 ./authentik/.env.sample:# SECRET_NEXTCLOUD_SECRET_VERSION=v1
16 ./authentik/.env.sample:# SECRET_WORDPRESS_SECRET_VERSION=v1
16 ./authentik/README.md:SECRET_NEXTCLOUD_SECRET_VERSION=v1
16 ./authentik/release/3.0.0+2023.2.3:    SECRET_NEXTCLOUD_SECRET_VERSION=v1
16 ./bonfire/.env.sample:SECRET_MEILI_MASTER_KEY_VERSION=v1
16 ./gitea/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./keycloak/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./kimai/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./limesurvey/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./nextcloud/.env.sample:# SECRET_AUTHENTIK_SECRET_VERSION=v1
16 ./nextcloud/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./phpservermon/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./pixelfed/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./seafile/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./strapi/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./wallabag/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
16 ./wordpress/.env.sample:# SECRET_AUTHENTIK_SECRET_VERSION=v1
16 ./wordpress/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1
17 ./authentik/.env.sample:# SECRET_MONITORING_SECRET_VERSION=v1
17 ./bonfire/.env.sample:SECRET_LIVEBOOK_PASSWORD_VERSION=v1
17 ./bonfire/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1
17 ./firefly-iii/.env.sample:SECRET_STATIC_CRON_TOKEN_VERSION=v1 # length=32
17 ./hometown/.env.sample:SECRET_VAPID_PRIVATE_KEY_VERSION=v1
17 ./lemmy/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1
17 ./matrix-synapse/.env.sample:#SECRET_DISCORD_BOT_TOKEN_VERSION=v1
17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_API_HASH_VERSION=v1
17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_AS_TOKEN_VERSION=v1
17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_HS_TOKEN_VERSION=v1
17 ./mattermost/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1
18 ./hometown/.env.sample:# SECRET_OIDC_CLIENT_SECRET_VERSION=v1
18 ./matrix-synapse/.env.sample:#SECRET_SHARED_SECRET_AUTH_VERSION=v1 # length=128
18 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_DB_PASSWORD_VERSION=v1
18 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_BOT_TOKEN_VERSION=v1
18 ./matrix-synapse/.env.sample:#SECRET_TURN_SHARED_SECRET_VERSION=v1
18 ./outline/.env.sample:#SECRET_OIDC_CLIENT_SECRET_VERSION=v1
19 ./dashboard-recipe/.env.sample:SECRET_HYDRA_CLIENT_SECRET_VERSION=v1
19 ./matrix-synapse/.env.sample:#SECRET_DISCORD_DB_PASSWORD_VERSION=v1
20 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_DB_PASSWORD_VERSION=v1
20 ./outline/.env.sample:#SECRET_GOOGLE_CLIENT_SECRET_VERSION=v1
21 ./backup-bot-two/.env.sample:#SECRET_AWS_SECRET_ACCESS_KEY_VERSION=v1
22 ./matrix-synapse/.env.sample:#SECRET_KEYCLOAK_CLIENT_SECRET_VERSION=v1
22 ./monitoring-lite/.env.sample:SECRET_GRAFANA_ADMIN_PASSWORD_VERSION=v1
22 ./seafile/.env.sample:SECRET_SEAFILE_ADMIN_PASSWORD_VERSION=v1
23 ./matrix-synapse/.env.sample:#SECRET_KEYCLOAK3_CLIENT_SECRET_VERSION=v1
25 ./limesurvey/.env.sample:SECRET_LIMESURVEY_ADMIN_PASSWORD_VERSION=v1
25 ./monitoring-lite/.env.sample:SECRET_PROMETHEUS_ADMIN_PASSWORD_VERSION=v1
26 ./monitoring-lite/.env.sample:SECRET_ALERTMANAGER_SMTP_PASSWORD_VERSION=v1
26 ./monitoring-lite/.env.sample:SECRET_LOKI_ADMIN_PASSWORD_HASHED_VERSION=v1
26 ./monitoring-lite/.env.sample:SECRET_LOKI_AWS_SECRET_ACCESS_KEY_VERSION=v1
27 ./monitoring-lite/.env.sample:SECRET_GRAFANA_OAUTH_CLIENT_SECRET_VERSION=v1
32 ./monitoring-lite/.env.sample:SECRET_PROMETHEUS_ADMIN_PASSWORD_HASHED_VERSION=v1

About the half of the secret names are less equal 12 chars. With SECRET_<secret_name>_VERSION they expand to 27 chars, so 37 chars are left for the complete subdomain. I would say this could be a good number.
A warning should be shown for domains about 30 chars, and an error for subdomains longer than 37 chars.

The downside is that we need to change the secrets of about 33 recipes.
adapt_authoring
authentik
backup-bot-two
bonfire
collabora
dashboard-recipe
discourse
filestash
firefly-iii
gitea
hometown
invoiceninja
keycloak
kimai
lemmy
limesurvey
matrix-synapse
mattermost
monitoring-lite
nextcloud
ohmyform
outline
phpservermon
pixelfed
rocketchat
seafile
snikket
strapi
traefik
vikunja
wallabag
wekan
wordpress

I think the ideal place for the length restriction should be the linting. Isn't the linting automatically executed before parsing the recipe?

Yes I already changed the synapse recipe with a release note. Not sure what is a good length. <p> <details> <summary>Here I listed the current secret name lengths.</summary> ``` 7 ./backup-bot-two/.env.sample:#SECRET_SSH_KEY_VERSION=v1 7 ./firefly-iii/.env.sample:SECRET_APP_KEY_VERSION=v1 # length=32 7 ./invoiceninja/.env.sample:SECRET_APP_KEY_VERSION=v1 8 ./authentik/.env.sample:# SECRET_WEKAN_ID_VERSION=v1 8 ./bonfire/.env.sample:SECRET_SEEDS_PW_VERSION=v1 8 ./matrix-synapse/.env.sample:SECRET_MACAROON_VERSION=v1 8 ./rallly/.env.sample:SECRET_SMTP_PWD_VERSION=v1 8 ./redmine/.env.sample:SECRET_KEY_BASE_VERSION=v1 9 ./authentik/.env.sample:# SECRET_MATRIX_ID_VERSION=v1 9 ./hedgedoc/.env.sample:#SECRET_OAUTH_KEY_VERSION=v1 9 ./invoiceninja/.env.sample:SECRET_DB_PASSWD_VERSION=v1 9 ./nocodb/.env.sample:SECRET_NC_DB_URL_VERSION=v1 9 ./rocketchat/.env.sample:#SECRET_OAUTH_KEY_VERSION=v1 9 ./traefik/.env.sample:#SECRET_USERSFILE_VERSION=v1 10 ./authentik/.env.sample:SECRET_ADMIN_PASS_VERSION=v1 10 ./authentik/.env.sample:SECRET_EMAIL_PASS_VERSION=v1 10 ./authentik/.env.sample:SECRET_SECRET_KEY_VERSION=v1 10 ./authentik/.env.sample:# SECRET_VIKUNJA_ID_VERSION=v1 10 ./dashboard-recipe/.env.sample:SECRET_SECRET_KEY_VERSION=v1 10 ./gitea/.env.sample:SECRET_JWT_SECRET_VERSION=v1 # length=43 10 ./gitea/.env.sample:SECRET_SECRET_KEY_VERSION=v1 # length=64 10 ./hometown/.env.sample:SECRET_OTP_SECRET_VERSION=v1 10 ./invoiceninja/.env.sample:SECRET_API_SECRET_VERSION=v1 10 ./minio/.env.sample:SECRET_ACCESS_KEY_VERSION=v1 # length=40 10 ./minio/.env.sample:SECRET_SECRET_KEY_VERSION=v1 # length=20 10 ./nextcloud/.env.sample:# SECRET_BBB_SECRET_VERSION=v1 10 ./nextcloud/README.md:SECRET_BBB_SECRET_VERSION=v1 10 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_BBB_SECRET_VERSION=v1 10 ./ohmyform/.env.sample:SECRET_SECRET_KEY_VERSION=v1 10 ./onlyoffice/.env.sample:# SECRET_JWT_SECRET_VERSION=v1 10 ./outline/.env.sample:SECRET_SECRET_KEY_VERSION=v1 # length=64 10 ./rallly/.env.sample:SECRET_SECRET_KEY_VERSION=v1 10 ./vikunja/.env.sample:SECRET_JWT_SECRET_VERSION=v1 10 ./wallabag/.env.sample:SECRET_APP_SECRET_VERSION=v1 11 ./adapt_authoring/.env.sample:SECRET_SESSION_KEY_VERSION=v1 11 ./authentik/.env.sample:SECRET_ADMIN_TOKEN_VERSION=v1 11 ./authentik/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./discourse/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./etherpad/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./firefly-iii/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 # length=32 11 ./gitea/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./gotosocial/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./hedgedoc/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./hometown/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./invidious/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./kanboard/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./keycloak/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./kimai/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./limesurvey/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./matrix-synapse/abra.sh:export SHARED_SECRET_AUTH_VERSION=v1 11 ./matrix-synapse/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./matrix-synapse/.env.sample:SECRET_FORM_SECRET_VERSION=v1 11 ./nextcloud/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./nocodb/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./onlyoffice/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./onlyoffice/release/2.0.0+7.1.1:SECRET_DB_PASSWORD_VERSION=v1 11 ./outline/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./phpservermon/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./pixelfed/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./rallly/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./redmine/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./vikunja/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./wallabag/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./wordpress/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 11 ./zammad/.env.sample:SECRET_DB_PASSWORD_VERSION=v1 12 ./authentik/.env.sample:# SECRET_NEXTCLOUD_ID_VERSION=v1 12 ./authentik/.env.sample:# SECRET_WEKAN_SECRET_VERSION=v1 12 ./authentik/.env.sample:# SECRET_WORDPRESS_ID_VERSION=v1 12 ./authentik/README.md:SECRET_NEXTCLOUD_ID_VERSION=v1 12 ./authentik/release/3.0.0+2023.2.3: SECRET_NEXTCLOUD_ID_VERSION=v1 12 ./bonfire/.env.sample:SECRET_SIGNING_SALT_VERSION=v1 # length=128 12 ./matrix-synapse/.env.sample:SECRET_REGISTRATION_VERSION=v1 12 ./nextcloud/.env.sample:# SECRET_AUTHENTIK_ID_VERSION=v1 12 ./outline/.env.sample:SECRET_UTILS_SECRET_VERSION=v1 # length=64 12 ./vikunja/compose.oauth.yml: - SECRET_OAUTH_SECRET_VERSION=V1 12 ./vikunja/.env.sample:# SECRET_OAUTH_SECRET_VERSION=v1 12 ./woodpecker/.env.sample:SECRET_AGENT_SECRET_VERSION=v1 12 ./wordpress/.env.sample:# SECRET_AUTHENTIK_ID_VERSION=v1 13 ./authentik/.env.sample:# SECRET_MATRIX_SECRET_VERSION=v1 13 ./authentik/.env.sample:# SECRET_MONITORING_ID_VERSION=v1 13 ./discourse/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1 13 ./gitea/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1 13 ./hometown/.env.sample:SECRET_SMTP_PASSWORD_VERSION=v1 13 ./matrix-synapse/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1 13 ./nextcloud/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1 13 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_SMTP_PASSWORD_VERSION=v1 13 ./ohmyform/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1 13 ./snikket/.env.sample:SECRET_COTURN_SECRET_VERSION=v1 13 ./vikunja/.env.sample:# SECRET_SMTP_PASSWORD_VERSION=v1 13 ./wallabag/.env.sample:SECRET_SMTP_PASSWORD_VERSION=v1 13 ./wekan/.env.sample:SECRET_OAUTH2_SECRET_VERSION=v1 13 ./wordpress/.env.sample:#SECRET_SMTP_PASSWORD_VERSION=v1 14 ./adapt_authoring/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./authentik/.env.sample:# SECRET_VIKUNJA_SECRET_VERSION=v1 14 ./collabora/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./filestash/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./gitea/.env.sample:SECRET_INTERNAL_TOKEN_VERSION=v1 # length=105 14 ./invoiceninja/.env.sample:SECRET_DB_ROOT_PASSWD_VERSION=v1 14 ./keycloak/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./kimai/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./lemmy/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./nextcloud/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./nextcloud/.env.sample:# SECRET_ONLYOFFICE_JWT_VERSION=v1 14 ./nextcloud/README.md:SECRET_ONLYOFFICE_JWT_VERSION=v1 14 ./nextcloud/release/3.1.0+25.0.1-fpm:# SECRET_ONLYOFFICE_JWT_VERSION=v1 14 ./ohmyform/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./outline/.env.sample:SECRET_AWS_SECRET_KEY_VERSION=v1 14 ./rocketchat/.env.sample:SECRET_ADMIN_PASSWORD_VERSION=v1 14 ./traefik/.env.sample:#SECRET_OVH_APP_SECRET_VERSION=v1 15 ./backup-bot-two/.env.sample:SECRET_RESTIC_PASSWORD_VERSION=v1 15 ./bonfire/.env.sample:SECRET_ENCRYPTION_SALT_VERSION=v1 # length=128 15 ./bonfire/.env.sample:SECRET_SECRET_KEY_BASE_VERSION=v1 # length=128 15 ./hometown/.env.sample:SECRET_SECRET_KEY_BASE_VERSION=v1 15 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_AS_TOKEN_VERSION=v1 15 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_HS_TOKEN_VERSION=v1 15 ./traefik/.env.sample:#SECRET_GANDIV5_API_KEY_VERSION=v1 16 ./authentik/.env.sample:# SECRET_NEXTCLOUD_SECRET_VERSION=v1 16 ./authentik/.env.sample:# SECRET_WORDPRESS_SECRET_VERSION=v1 16 ./authentik/README.md:SECRET_NEXTCLOUD_SECRET_VERSION=v1 16 ./authentik/release/3.0.0+2023.2.3: SECRET_NEXTCLOUD_SECRET_VERSION=v1 16 ./bonfire/.env.sample:SECRET_MEILI_MASTER_KEY_VERSION=v1 16 ./gitea/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./keycloak/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./kimai/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./limesurvey/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./nextcloud/.env.sample:# SECRET_AUTHENTIK_SECRET_VERSION=v1 16 ./nextcloud/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./phpservermon/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./pixelfed/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./seafile/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./strapi/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./wallabag/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 16 ./wordpress/.env.sample:# SECRET_AUTHENTIK_SECRET_VERSION=v1 16 ./wordpress/.env.sample:SECRET_DB_ROOT_PASSWORD_VERSION=v1 17 ./authentik/.env.sample:# SECRET_MONITORING_SECRET_VERSION=v1 17 ./bonfire/.env.sample:SECRET_LIVEBOOK_PASSWORD_VERSION=v1 17 ./bonfire/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1 17 ./firefly-iii/.env.sample:SECRET_STATIC_CRON_TOKEN_VERSION=v1 # length=32 17 ./hometown/.env.sample:SECRET_VAPID_PRIVATE_KEY_VERSION=v1 17 ./lemmy/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1 17 ./matrix-synapse/.env.sample:#SECRET_DISCORD_BOT_TOKEN_VERSION=v1 17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_API_HASH_VERSION=v1 17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_AS_TOKEN_VERSION=v1 17 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_HS_TOKEN_VERSION=v1 17 ./mattermost/.env.sample:SECRET_POSTGRES_PASSWORD_VERSION=v1 18 ./hometown/.env.sample:# SECRET_OIDC_CLIENT_SECRET_VERSION=v1 18 ./matrix-synapse/.env.sample:#SECRET_SHARED_SECRET_AUTH_VERSION=v1 # length=128 18 ./matrix-synapse/.env.sample:#SECRET_SIGNAL_DB_PASSWORD_VERSION=v1 18 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_BOT_TOKEN_VERSION=v1 18 ./matrix-synapse/.env.sample:#SECRET_TURN_SHARED_SECRET_VERSION=v1 18 ./outline/.env.sample:#SECRET_OIDC_CLIENT_SECRET_VERSION=v1 19 ./dashboard-recipe/.env.sample:SECRET_HYDRA_CLIENT_SECRET_VERSION=v1 19 ./matrix-synapse/.env.sample:#SECRET_DISCORD_DB_PASSWORD_VERSION=v1 20 ./matrix-synapse/.env.sample:#SECRET_TELEGRAM_DB_PASSWORD_VERSION=v1 20 ./outline/.env.sample:#SECRET_GOOGLE_CLIENT_SECRET_VERSION=v1 21 ./backup-bot-two/.env.sample:#SECRET_AWS_SECRET_ACCESS_KEY_VERSION=v1 22 ./matrix-synapse/.env.sample:#SECRET_KEYCLOAK_CLIENT_SECRET_VERSION=v1 22 ./monitoring-lite/.env.sample:SECRET_GRAFANA_ADMIN_PASSWORD_VERSION=v1 22 ./seafile/.env.sample:SECRET_SEAFILE_ADMIN_PASSWORD_VERSION=v1 23 ./matrix-synapse/.env.sample:#SECRET_KEYCLOAK3_CLIENT_SECRET_VERSION=v1 25 ./limesurvey/.env.sample:SECRET_LIMESURVEY_ADMIN_PASSWORD_VERSION=v1 25 ./monitoring-lite/.env.sample:SECRET_PROMETHEUS_ADMIN_PASSWORD_VERSION=v1 26 ./monitoring-lite/.env.sample:SECRET_ALERTMANAGER_SMTP_PASSWORD_VERSION=v1 26 ./monitoring-lite/.env.sample:SECRET_LOKI_ADMIN_PASSWORD_HASHED_VERSION=v1 26 ./monitoring-lite/.env.sample:SECRET_LOKI_AWS_SECRET_ACCESS_KEY_VERSION=v1 27 ./monitoring-lite/.env.sample:SECRET_GRAFANA_OAUTH_CLIENT_SECRET_VERSION=v1 32 ./monitoring-lite/.env.sample:SECRET_PROMETHEUS_ADMIN_PASSWORD_HASHED_VERSION=v1 ``` </details> </p> About the half of the secret names are less equal 12 chars. With `SECRET_<secret_name>_VERSION` they expand to 27 chars, so 37 chars are left for the complete subdomain. I would say this could be a good number. A warning should be shown for domains about 30 chars, and an error for subdomains longer than 37 chars. <p> <details> <summary>The downside is that we need to change the secrets of about 33 recipes.</summary> ``` adapt_authoring authentik backup-bot-two bonfire collabora dashboard-recipe discourse filestash firefly-iii gitea hometown invoiceninja keycloak kimai lemmy limesurvey matrix-synapse mattermost monitoring-lite nextcloud ohmyform outline phpservermon pixelfed rocketchat seafile snikket strapi traefik vikunja wallabag wekan wordpress ``` </details> </p> I think the ideal place for the length restriction should be the linting. Isn't the linting automatically executed before parsing the recipe?
Owner

Including via linting sounds good! We only lint for errors before upgrade/deploy/rollback, so we might want to tweak that, potentially controversial 🙃 Numbers look good. At least people will be informed and can take action from there instead of running into a docker runtime error.

Including via linting sounds good! We only lint for errors before `upgrade`/`deploy`/`rollback`, so we might want to tweak that, potentially controversial 🙃 Numbers look good. At least people will be informed and can take action from there instead of running into a docker runtime error.
decentral1se added the
abra
label 2023-07-26 06:30:20 +00:00
decentral1se changed title from Too long secret names to Dealing with secret names that are too long 2023-07-26 07:33:11 +00:00

Hey,

I was looking at potentially picking this up but I want to make sure I understand the problem and the proposed solution before I start. From my limited understanding we have 2 problems:

  1. Recipes that have long secret names means we're reducing the number of potential domains we could deploy to (see the original example given by moritz in the first post).
  2. There's no explicit handling for the case where you attempt to deploy a recipe with a domain that's too long FATA[0014] Error response from daemon: rpc error: code = InvalidArgument desc = invalid name, only 64 [a-zA-Z0-9-_.] characters allowed, and the start and end character must be [a-zA-Z0-9]

From my limited understanding of the code it seems like linting can solve problem 1 but not problem 2. Linting seems to take place on the recipe rather than the app where it's the combination of the two things that causes the problem. Obviously linting secrets bigger than a certain size could help but someone can always come up with a very long domain name and encounter the same problem.

Hopefully I've managed to grasp the issue though I'm still not 100% on how the final secret names are generated : ) So if linting doesn't check the app name, do we want a variant that does? Do we want a separate check before deployment for this specific scenario? Do we want to just lint the secret names in the recipe to just reduce the probability of long domain names causing a problem? Should I give up and try and tackle something else that boils my brain a little less 😅

Let me know what you think : )

Hey, I was looking at potentially picking this up but I want to make sure I understand the problem and the proposed solution before I start. From my limited understanding we have 2 problems: 1. Recipes that have long secret names means we're reducing the number of potential domains we could deploy to (see the original example given by moritz in the first post). 2. There's no explicit handling for the case where you attempt to deploy a recipe with a domain that's too long FATA[0014] Error response from daemon: rpc error: code = InvalidArgument desc = invalid name, only 64 [a-zA-Z0-9-_.] characters allowed, and the start and end character must be [a-zA-Z0-9] From my limited understanding of the code it seems like linting can solve problem 1 but not problem 2. Linting seems to take place on the recipe rather than the app where it's the combination of the two things that causes the problem. Obviously linting secrets bigger than a certain size could help but someone can always come up with a very long domain name and encounter the same problem. Hopefully I've managed to grasp the issue though I'm still not 100% on how the final secret names are generated : ) So if linting doesn't check the app name, do we want a variant that does? Do we want a separate check before deployment for this specific scenario? Do we want to just lint the secret names in the recipe to just reduce the probability of long domain names causing a problem? Should I give up and try and tackle something else that boils my brain a little less 😅 Let me know what you think : )
Owner

Hopefully I've managed to grasp the issue

heyyy @rix, that's great! this is a difficult one alright. you've summed it up perfectly!

So if linting doesn't check the app name, do we want a variant that does?

As @moritz mentioned:

I think the ideal place for the length restriction should be the linting.

I think we should try implement a linting rule (https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/pkg/lint/recipe.go).

If Level: "error" then

pkg/lint/recipe.go Lines 166 to 195 in 1e6a6e6174
// LintForErrors lints specifically for errors and not other levels. This is
// used in code paths such as "app deploy" to avoid nasty surprises but not for
// the typical linting commands, which do handle other levels.
func LintForErrors(recipe recipe.Recipe) error {
logrus.Debugf("linting for critical errors in %s configs", recipe.Name)
for level := range LintRules {
if level != "error" {
continue
}
for _, rule := range LintRules[level] {
if rule.Skip(recipe) {
continue
}
ok, err := rule.Function(recipe)
if err != nil {
return err
}
if !ok {
return fmt.Errorf("lint error in %s configs: \"%s\" failed lint checks (%s)", recipe.Name, rule.Description, rule.Ref)
}
}
}
logrus.Debugf("linting successful, %s is well configured", recipe.Name)
return nil
}
will fail deployments. So, let's keep it at "warn" as mentioned by @moritz:

About the half of the secret names are less equal 12 chars. With SECRET_<secret_name>_VERSION they expand to 27 chars, so 37 chars are left for the complete subdomain. I would say this could be a good number. A warning should be shown for domains about 30 chars, and an error for subdomains longer than 37 chars.

AFAIU then people won't be plagued by this warning on deploy/upgrade/etc. but when they lint the recipe, they'll see they need to take action. As mentioned in coop-cloud/organising#463 (comment) I don't think we should go on a renaming rampage as part of this ticket, people can update their own recipes.

We should also document this in the recipe maintainers handbook?

Do we want to just lint the secret names in the recipe to just reduce the probability of long domain names causing a problem?

If you want to implement this additional rule, feel free! might be nice but if we have already something for the domain name, not sure how useful it will be.

Thanks!

> Hopefully I've managed to grasp the issue heyyy @rix, that's great! this is a difficult one alright. you've summed it up perfectly! > So if linting doesn't check the app name, do we want a variant that does? As @moritz mentioned: > I think the ideal place for the length restriction should be the linting. I think we should try implement a linting rule (https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/pkg/lint/recipe.go). If `Level: "error"` then https://git.coopcloud.tech/coop-cloud/abra/src/commit/1e6a6e6174b69380b7f8901c5e40d05adb69f6ca/pkg/lint/recipe.go#L166-L195 will fail deployments. So, let's keep it at `"warn"` as mentioned by @moritz: > About the half of the secret names are less equal 12 chars. With SECRET_<secret_name>_VERSION they expand to 27 chars, so 37 chars are left for the complete subdomain. I would say this could be a good number. A warning should be shown for domains about 30 chars, and an error for subdomains longer than 37 chars. AFAIU then people won't be plagued by this warning on deploy/upgrade/etc. but when they lint the recipe, they'll see they need to take action. As mentioned in https://git.coopcloud.tech/coop-cloud/organising/issues/463#issuecomment-17253 I don't think we should go on a renaming rampage as part of this ticket, people can update their own recipes. We should also document this in the recipe maintainers handbook? > Do we want to just lint the secret names in the recipe to just reduce the probability of long domain names causing a problem? If you want to implement this additional rule, feel free! might be nice but if we have already something for the domain name, not sure how useful it will be. Thanks!
Owner

Work underway coop-cloud/abra#359 👏

Work underway https://git.coopcloud.tech/coop-cloud/abra/pulls/359 👏
decentral1se modified the project from Critical fixes to (deleted) 2023-11-27 10:46:18 +00:00
decentral1se modified the project from (deleted) to Critical fixes 2023-11-27 10:46:23 +00:00
decentral1se self-assigned this 2023-11-27 10:46:28 +00:00
decentral1se removed their assignment 2023-11-27 10:47:51 +00:00
Owner

coop-cloud/abra#359 is merged! I pushed a few fixups. I also pushed 064a26e182 for docs. There might be more we're missing, so please do comment if you have more to add. Closing for now, thanks all.

https://git.coopcloud.tech/coop-cloud/abra/pulls/359 is merged! I pushed a few fixups. I also pushed https://git.coopcloud.tech/coop-cloud/docs.coopcloud.tech/commit/064a26e182fd8e15098aa428f027f7159301f348 for docs. There might be more we're missing, so please do comment if you have more to add. Closing for now, thanks all.
Sign in to join this conversation.
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/organising#463
No description provided.