coop-cloud/keycloak
4
0
Fork 3
Code Issues 7 Pull Requests 1 Releases Wiki Activity

add_kcadm #28

Open
oxaliq wants to merge 6 commits from oxaliq/keycloak:add_kcadm into master
pull from: oxaliq/keycloak:add_kcadm
merge into: coop-cloud:master
Conversation 11 Commits 6 Files Changed 5 +79 -12
oxaliq commented 2026-04-01 15:44:43 +00:00
Owner
Copy Link

Closes #27

There is a subtle breaking change introduced, as the semantics of the admin_password secret are changed. kcadm.sh requires authentication against the Keycloak REST API for a user of sufficient permissions to perform the actions attempted. To solve this, this PR uses the admin_password secret as the secret for the permanent admin user that is created after authenticating with the temporary bootstrapped user.

The result is that running login_kcadm and run_kcadm will not work for existing Keycloak deployments without upgrading this secret. I can write a short guide on how to do this if this PR is accepted.

Closes #27 There is a subtle breaking change introduced, as the semantics of the admin_password secret are changed. kcadm.sh requires authentication against the Keycloak REST API for a user of sufficient permissions to perform the actions attempted. To solve this, this PR uses the admin_password secret as the secret for the permanent admin user that is created after authenticating with the temporary bootstrapped user. The result is that running `login_kcadm` and `run_kcadm` will not work for existing Keycloak deployments without upgrading this secret. I can write a short guide on how to do this if this PR is accepted.
❤️ 2
oxaliq added 3 commits 2026-04-01 15:44:46 +00:00
implement kcadm and changes to allow kcadm.sh to authenticate c6eb27e10c 
in order to run kcadm.sh commands the script must authenticate to the REST API.
this commit includes an init_kc command that replaces the bootstrap user with
a permanent admin user (whose password is a docker swarm secret) that can be
used to authenticate before running kcadm commands. this reuses the secret
'admin_password' that was previously used as the password for the bootstrap
admin user.
remove password option from kcadm authentication where possible 1fe630211a
add documentation for admin cli
Some checks failed
continuous-integration/drone/pr Build is failing
Details
a85d4f3b8f 
also updates initial setup documentation to reflect admin cli usage
in creating permanent admin user.
oxaliq requested review from 3wordchant 2026-04-01 15:48:16 +00:00
oxaliq requested review from ammaratef45 2026-04-01 15:48:17 +00:00
oxaliq requested review from cyrnel 2026-04-01 15:48:17 +00:00
oxaliq requested review from notplants 2026-04-01 15:48:17 +00:00
oxaliq commented 2026-04-01 15:48:33 +00:00
Author
Owner
Copy Link

adding reviewers from recent recipe activity. ty!

adding reviewers from recent recipe activity. ty!
decentral1se commented 2026-04-03 14:20:35 +00:00
Owner
Copy Link

The result is that running login_kcadm and run_kcadm will not work for existing Keycloak deployments without upgrading this secret. I can write a short guide on how to do this if this PR is accepted.

OK, I tried to grok this breaking issue but my brain won't let it in 😆 Can you write up the short migration guide in a release note and that might help?

This looks like a solid change in general! Just want to get the breaking stuff agreed upon by a few people. I think the release note will help.

> The result is that running login_kcadm and run_kcadm will not work for existing Keycloak deployments without upgrading this secret. I can write a short guide on how to do this if this PR is accepted. OK, I tried to grok this breaking issue but my brain won't let it in 😆 Can you write up the short migration guide in a [release note](https://docs.coopcloud.tech/maintainers/handbook/#how-do-i-write-version-release-notes) and that might help? This looks like a solid change in general! Just want to get the breaking stuff agreed upon by a few people. I think the release note will help.
👍 1
oxaliq added 1 commit 2026-04-03 16:10:19 +00:00
prepare kcadm release
Some checks failed
continuous-integration/drone/pr Build is failing
Details
a6b84ce9b6
oxaliq commented 2026-04-03 16:12:14 +00:00
Author
Owner
Copy Link

OK, I tried to grok this breaking issue but my brain won't let it in 😆 Can you write up the short migration guide in a release note and that might help?

Added a release note! thanks @decentral1se

> OK, I tried to grok this breaking issue but my brain won't let it in 😆 Can you write up the short migration guide in a [release note](https://docs.coopcloud.tech/maintainers/handbook/#how-do-i-write-version-release-notes) and that might help? Added a release note! thanks @decentral1se
decentral1se reviewed 2026-04-03 16:28:11 +00:00
decentral1se left a comment
Owner
Copy Link

Keycloak is hands down the most bizarre software 😆 Great work! I think I've reached the limits of evaluating this PR. I don't maintain Keycloak anymore. I hope the others can weigh in. You might need to chase them up on the Matrix chats.

Keycloak is hands down the most bizarre software 😆 Great work! I think I've reached the limits of evaluating this PR. I don't maintain Keycloak anymore. I hope the others can weigh in. You might need to chase them up on the Matrix chats.
👍 1 ❤️ 1
.env.sample Outdated
@ -6,3 +6,3 @@
LETS_ENCRYPT_ENV=production
ADMIN_USERNAME=admin
BOOTSTRAP_PASSWORD= # temporary admin password
decentral1se commented 2026-04-03 16:21:24 +00:00
Owner
Copy Link

Do I understand that you want operators to set this secret value themselves manually? That seems fine since it is temporary but it is a bit of an unexpected context switch. I don't think any other single recipe asks operators to put plain secrets in their .env files. How annoying is it to generate this temporary secret and remove it after you run init_kc? That would be more "standard" 😬

Do I understand that you want operators to set this secret value themselves manually? That seems fine since it is temporary but it is a bit of an unexpected context switch. I don't think any other single recipe asks operators to put plain secrets in their `.env` files. How annoying is it to generate this temporary secret and remove it after you run `init_kc`? That would be more "standard" 😬
oxaliq commented 2026-04-03 19:10:23 +00:00
Author
Owner
Copy Link

That makes sense! I was hoping to really highlight how this password should not be relied upon, but I can make it a temporary secret again to avoid a strange pattern.

That makes sense! I was hoping to really highlight how this password should not be relied upon, but I can make it a temporary secret again to avoid a strange pattern.
❤️ 1
oxaliq marked this conversation as resolved
release/next
@ -0,0 +12,4 @@
To insert your permanent admin password:
"abra app secret insert $APP SECRET_ADMIN_PASSWORD_VERSION \
$NEW_VERSION $ADMIN_PASSWORD"
decentral1se commented 2026-04-03 16:27:00 +00:00
Owner
Copy Link

Do we want to qualify here that $ADMIN_PASSWORD is the 2nd password they entered in manually in the web UI when creating their "permanent admin" in the previous workflow? Idk how others see this but I'm really confused but this whole change because Keycloak is just so weird 😅

Do we want to qualify here that `$ADMIN_PASSWORD` is the 2nd password they entered in manually in the web UI when creating their "permanent admin" in the previous workflow? Idk how others see this but I'm really confused but this whole change because Keycloak is just so weird 😅
oxaliq commented 2026-04-03 19:11:19 +00:00
Author
Owner
Copy Link

I'll add more language to that effect. I was really struggling with how to delineate the two passwords. The pattern is super strange!

I'll add more language to that effect. I was really struggling with how to delineate the two passwords. The pattern is super strange!
❤️ 1
oxaliq marked this conversation as resolved
oxaliq added 2 commits 2026-04-10 18:52:31 +00:00
add bootstrap password to secrets 44529dc36a 
also changes environment variables to reflect KC_BOOTSTRAP values for
temporary bootstrap admin user
add bootstrap password generation to release note
Some checks failed
continuous-integration/drone/pr Build is failing
Details
9555bc7980
oxaliq commented 2026-04-10 18:56:01 +00:00
Author
Owner
Copy Link

I've removed the temporary secret from the environment file and confirmed the upgrade reflected in release notes works.

I've removed the temporary secret from the environment file and confirmed the upgrade reflected in release notes works.
❤️ 1
notplants commented 2026-04-29 14:53:50 +00:00
Owner
Copy Link

@oxaliq I am taking a closer look at this now and running it locally.

It worked for me on a fresh deploy, but running into issues with the migration path for an already existing deployment, with init_kc perhaps expecting a fresh deploy where an admin user doesn't already exist. Debugging more now.

@oxaliq I am taking a closer look at this now and running it locally. It worked for me on a fresh deploy, but running into issues with the migration path for an already existing deployment, with `init_kc` perhaps expecting a fresh deploy where an admin user doesn't already exist. Debugging more now.
❤️ 1
notplants commented 2026-04-29 15:30:14 +00:00
Owner
Copy Link

Ok I see now that init_kc is not required to be rerun with this PR. So an existing deployment will continue running as long as the new secret is inserted like in the release notes:

# You will also need to generate a bootstrap password (this will not be used)
abra app secret generate $APP bootstrap_password v1

A part of me still would prefer to err the sides of not making operators do anything manual unless it is required.

But it is nice this PR encourages a more secure happy path, and moving this PR into compose.bootstrap.yml would be a bit awkward.

The only other idea I have thought of: what if BOOTSTRAP_PW is not stored as an abra secret... but instead stored in some other more automated and temporary way, like a temporary plain text file with an autogenerated secret. The bootstrap user is deleted anyway on running of init_kc. That way it could be temporarily used, but not require generating an additional secret for existing deployments (but maybe this idea is confusing?). does this make sense or any other ideas to make the upgrade path not a breaking change if possible?

I also see one other possible issue in this PR. in the new version, this was removed in favor of the boostrap version:

      bash -c "KEYCLOAK_ADMIN_PASSWORD=\"$$(cat /run/secrets/admin_password)\" KC_DB_PASSWORD=\"$$(cat /run/secrets/db_password)\" /opt/keycloak/bin/kc.sh start"

Does this mean that now if the operator changes the keycloak admin secret via abra, it will not get automatically reflected in the keycloak deployment until they manually change it? could you clarify this @oxaliq ?

This PR would also close #19 . imo seems like we should do it, even if it requires some operator confusion -- but would be nice to avoid if possible

But still wondering about that last question

Ok I see now that `init_kc` is not required to be rerun with this PR. So an existing deployment will continue running as long as the new secret is inserted like in the release notes: ``` # You will also need to generate a bootstrap password (this will not be used) abra app secret generate $APP bootstrap_password v1 ``` A part of me still would prefer to err the sides of not making operators do anything manual unless it is required. But it is nice this PR encourages a more secure happy path, and moving this PR into `compose.bootstrap.yml` would be a bit awkward. The only other idea I have thought of: what if BOOTSTRAP_PW is not stored as an abra secret... but instead stored in some other more automated and temporary way, like a temporary plain text file with an autogenerated secret. The bootstrap user is deleted anyway on running of `init_kc`. That way it could be temporarily used, but not require generating an additional secret for existing deployments (but maybe this idea is confusing?). does this make sense or any other ideas to make the upgrade path not a breaking change if possible? I also see one other possible issue in this PR. in the new version, this was removed in favor of the boostrap version: ``` bash -c "KEYCLOAK_ADMIN_PASSWORD=\"$$(cat /run/secrets/admin_password)\" KC_DB_PASSWORD=\"$$(cat /run/secrets/db_password)\" /opt/keycloak/bin/kc.sh start" ``` Does this mean that now if the operator changes the keycloak admin secret via abra, it will not get automatically reflected in the keycloak deployment until they manually change it? could you clarify this @oxaliq ? ----- This PR would also close https://git.coopcloud.tech/coop-cloud/keycloak/issues/19 . imo seems like we should do it, even if it requires some operator confusion -- but would be nice to avoid if possible But still wondering about that last question
❤️ 2
oxaliq commented 2026-04-29 22:59:26 +00:00
Author
Owner
Copy Link

thanks for review @notplants !

The only other idea I have thought of: what if BOOTSTRAP_PW is not stored as an abra secret... but instead stored in some other more automated and temporary way, like a temporary plain text file with an autogenerated secret. The bootstrap user is deleted anyway on running of init_kc. That way it could be temporarily used, but not require generating an additional secret for existing deployments (but maybe this idea is confusing?). does this make sense or any other ideas to make the upgrade path not a breaking change if possible?

i do like this alternative to the BOOTSTRAP_PASSWORD being either a secret or environment variable since it's only ever used on initial deployment. i think this would require a custom entrypoint.sh to handle file creation and deletion. unfortunately, this change would still require some operator intervention for existing deployments.

existing deployments have two admin passwords SECRET_ADMIN_PASSWORD which is the initial bootstrap password generated when they first deployed, and the real admin password generated outside of abra. from readme:

  1. If this is an admin account for you, then log out and back in as the new admin user and complete the password change and OTP steps. If this is an admin account for someone else, securely send the initial username and password to the user. They must complete the password change and OTP setup when they first log in.

in order to run kcadm.sh commands, the script must authenticate to keycloak's rest API. if we keep SECRET_ADMIN_PASSWORD as the only secret, the result of running abra app command $DOMAIN app login_kcadm is an attempt to authenticate with the old bootstrap password, since that's the only password stored as a secret in existing deployments.

Does this mean that now if the operator changes the keycloak admin secret via abra, it will not get automatically reflected in the keycloak deployment until they manually change it? could you clarify this @oxaliq ?

to clarify the change to the entrypoint: the enviroment variables KEYCLOAK_ADMIN_PASSWORD or KC_BOOTSTRAP_ADMIN_PASSWORD are synonymous and are only ever used on initial deployment. keycloak will ignore this value on subsequent deployments regardless of whether the operator of updates the ADMIN_PASSWORD_SECRET. as long as that $$(cat /run/secrets/...) evaluation doesn't error there will be no change in behavior. a consequence of making the bootstrap password a temporary file would mean adding some

This PR would also close #19 . imo seems like we should do it, even if it requires some operator confusion -- but would be nice to avoid if possible

yes, sorry! i changed the name to quiet a warning i kept getting while working, but i did not think to check against existing issues. the name change is non-essential to this PR and i can revert it if that makes things less confusing!

thanks for review @notplants ! > The only other idea I have thought of: what if BOOTSTRAP_PW is not stored as an abra secret... but instead stored in some other more automated and temporary way, like a temporary plain text file with an autogenerated secret. The bootstrap user is deleted anyway on running of init_kc. That way it could be temporarily used, but not require generating an additional secret for existing deployments (but maybe this idea is confusing?). does this make sense or any other ideas to make the upgrade path not a breaking change if possible? i do like this alternative to the BOOTSTRAP_PASSWORD being either a secret or environment variable since it's only ever used on initial deployment. i think this would require a custom `entrypoint.sh` to handle file creation and deletion. unfortunately, this change would still require some operator intervention for existing deployments. existing deployments have two admin passwords SECRET_ADMIN_PASSWORD which is the initial bootstrap password generated when they first deployed, and the real admin password generated outside of abra. from [readme](https://git.coopcloud.tech/coop-cloud/keycloak/src/branch/master/README.md?display=source#L38): > 8. If this is an admin account for you, then log out and back in as the new admin user and complete the password change and OTP steps. If this is an admin account for someone else, securely send the initial username and password to the user. They must complete the password change and OTP setup when they first log in. in order to run `kcadm.sh` commands, the script must authenticate to keycloak's rest API. if we keep SECRET_ADMIN_PASSWORD as the only secret, the result of running `abra app command $DOMAIN app login_kcadm` is an attempt to authenticate with the old bootstrap password, since that's the only password stored as a secret in existing deployments. > Does this mean that now if the operator changes the keycloak admin secret via abra, it will not get automatically reflected in the keycloak deployment until they manually change it? could you clarify this @oxaliq ? to clarify the change to the entrypoint: the enviroment variables KEYCLOAK_ADMIN_PASSWORD or KC_BOOTSTRAP_ADMIN_PASSWORD are synonymous and are only ever used on initial deployment. keycloak will ignore this value on subsequent deployments regardless of whether the operator of updates the ADMIN_PASSWORD_SECRET. as long as that `$$(cat /run/secrets/...)` evaluation doesn't error there will be no change in behavior. a consequence of making the bootstrap password a temporary file would mean adding some > This PR would also close #19 . imo seems like we should do it, even if it requires some operator confusion -- but would be nice to avoid if possible yes, sorry! i changed the name to quiet a warning i kept getting while working, but i did not think to check against existing issues. the name change is non-essential to this PR and i can revert it if that makes things less confusing!
❤️ 1
notplants commented 2026-04-30 16:42:24 +00:00
Owner
Copy Link

thank you for your patience and the clarification @oxaliq

still one bit here I am trying to wrap my head around:

in order to run kcadm.sh commands, the script must authenticate to keycloak's rest API. if we keep SECRET_ADMIN_PASSWORD as the only secret, the result of running abra app command $DOMAIN app login_kcadm is an attempt to authenticate with the old bootstrap password, since that's the only password stored as a secret in existing deployments.

does this mean that if we moved BOOTSTRAP_PASSWORD to be a temporary file and not a secret, that the only thing that would break for the current operators is abra app command $DOMAIN app login_kcadm ?

unfortunately, this change would still require some operator intervention for existing deployment

so the manual intervention required would be to make login_kcadm work, instead of manual intervention required to just keep keycloak deployed and running at all (which is the case of adding a new bootstrap secret)?

if the temporary-file-entrypoint method fulfills the need, and only requires intervention by existing operators when they want to use the new abra commands, that seems preferrable

or maybe I have misunderstood something and I am not seeing what manual intervention would be needed by operators for the case of a custom entrypoint which handles the boostrap logic (I am imagining for example some type of if statement that checks if a non-bootstrap admin already exists, and in that event, just starts normally). is there some other manual intervention that would be required?

thank you for your patience and the clarification @oxaliq still one bit here I am trying to wrap my head around: > in order to run kcadm.sh commands, the script must authenticate to keycloak's rest API. if we keep SECRET_ADMIN_PASSWORD as the only secret, the result of running abra app command $DOMAIN app login_kcadm is an attempt to authenticate with the old bootstrap password, since that's the only password stored as a secret in existing deployments. does this mean that if we moved BOOTSTRAP_PASSWORD to be a temporary file and not a secret, that the only thing that would break for the current operators is `abra app command $DOMAIN app login_kcadm` ? > unfortunately, this change would still require some operator intervention for existing deployment so the manual intervention required would be to make `login_kcadm` work, instead of manual intervention required to just keep keycloak deployed and running at all (which is the case of adding a new bootstrap secret)? if the temporary-file-entrypoint method fulfills the need, and only requires intervention by existing operators when they want to use the new abra commands, that seems preferrable or maybe I have misunderstood something and I am not seeing what manual intervention would be needed by operators for the case of a custom entrypoint which handles the boostrap logic (I am imagining for example some type of if statement that checks if a non-bootstrap admin already exists, and in that event, just starts normally). is there some other manual intervention that would be required?
❤️ 1
oxaliq commented 2026-04-30 19:15:06 +00:00
Author
Owner
Copy Link

or maybe I have misunderstood something and I am not seeing what manual intervention would be needed by operators for the case of a custom entrypoint which handles the boostrap logic (I am imagining for example some type of if statement that checks if a non-bootstrap admin already exists, and in that event, just starts normally). is there some other manual intervention that would be required?

yes, an operator of an existing deployment would not need manual intervention with this pattern, unless they wished to use kc_adm commands. my assumption was to require updating the SECRET_ADMIN_PASSWORD for existing deployments, so that any future recipe work that builds on kc_adm availability would not cause those deployments to drift further.

i will have to do some reading and thinking on how a custom entrypoint would check for a non-bootstrap admin prior to launching keycloak

> or maybe I have misunderstood something and I am not seeing what manual intervention would be needed by operators for the case of a custom entrypoint which handles the boostrap logic (I am imagining for example some type of if statement that checks if a non-bootstrap admin already exists, and in that event, just starts normally). is there some other manual intervention that would be required? yes, an operator of an existing deployment would not need manual intervention with this pattern, unless they wished to use kc_adm commands. my assumption was to require updating the SECRET_ADMIN_PASSWORD for existing deployments, so that any future recipe work that builds on kc_adm availability would not cause those deployments to drift further. i will have to do some reading and thinking on how a custom entrypoint would check for a non-bootstrap admin prior to launching keycloak
❤️ 1
notplants commented 2026-05-01 14:33:44 +00:00
Owner
Copy Link

@oxaliq I was just thinking about this a bit and had a couple ideas.

  1. we add an env value called BOOTSTRAP_ENABLED which gets a default value of true in .env.sample. in the custom entrypoint, if BOOTSTRAP_ENABLED is true , then we do the boostrap thing. if its false, then we start normally

this would make the default case for new operators the same as this PR, and for existing operators the change is opt-in

  1. there could also maybe be more complex ways to do checks in a custom entrypoint. for example starting kc.sh in the background, then querying it to see if there is an admin user, and if not then rebooting it and doing the bootstrap thing. I did something kind of like this here with minio initialization in lasuite-docs. but this probably sounds like overkill for this case.

do you like the env var approach (1) ? any other ideas?

I also hear you on wanting to avoid drift, although in this case I could definitely imagine many keycloak deployment just using the UI and not using kc_adm. I am also probably biased against avoiding breaking changes if at all possible, because at autonomic we are taking care of so many cc deployments, than when an upgrade requires anything manual, it is often a big headache

@oxaliq I was just thinking about this a bit and had a couple ideas. 1. we add an env value called BOOTSTRAP_ENABLED which gets a default value of true in .env.sample. in the custom entrypoint, if BOOTSTRAP_ENABLED is true , then we do the boostrap thing. if its false, then we start normally this would make the default case for new operators the same as this PR, and for existing operators the change is opt-in 2. there could also maybe be more complex ways to do checks in a custom entrypoint. for example starting kc.sh in the background, then querying it to see if there is an admin user, and if not then rebooting it and doing the bootstrap thing. I did something kind of like this [here](https://git.coopcloud.tech/coop-cloud/lasuite-docs/src/commit/4714d13b47142222d8caad6fc607ca0b0e368719/compose.yml#L236) with minio initialization in lasuite-docs. but this probably sounds like overkill for this case. do you like the env var approach (1) ? any other ideas? I also hear you on wanting to avoid drift, although in this case I could definitely imagine many keycloak deployment just using the UI and not using kc_adm. I am also probably biased against avoiding breaking changes if at all possible, because at autonomic we are taking care of so many cc deployments, than when an upgrade requires anything manual, it is often a big headache
👍 3
Some checks failed
continuous-integration/drone/pr Build is failing
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u add_kcadm:oxaliq-add_kcadm
git checkout oxaliq-add_kcadm
Sign in to join this conversation.
Reviewers
No Reviewers
3wordchant
ammaratef45
cyrnel
notplants
Labels
No items
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 amras Apfelwurm appletalk arjan basebuilder BornDeleuze Brooke carla cas codegod100 coopcloud cyrnel decentral1se dede devydave fauno flancian Frando iexos jade javielico jjsfunhouse jmakdah2 joe-irving kawaiipunk knoflook kolaente lambdabundesverband linnealovespie marlon mayel mirsal moosemower moritz nicksellen notplants oxaliq p4u1 pau pharaohgraphy PhiNatalie renovate-bot ripclap rix rscmbbng sef simon sixsmith stevensting tobias trav val vaznasty virtualboys wolcen wykwit xynosis yksflip
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: coop-cloud/keycloak#28
No description provided.
Delete Branch "oxaliq/keycloak:add_kcadm"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?