abra app secret get <domain> <secret-name> #480

New Issue

2023-08-25T14:30:31Z

moritz commented

2023-08-25 14:30:31 +00:00

For the autoconfigurator https://git.coopcloud.tech/moritz/alakazam I need to exchange secrets between several apps.
abra app secret get <domain> <secret-name> would be a very useful command.
The current workaround is to run abra app run <domain> <container> cat /var/run/secrets/<secret_name>.
The problem with this command is that the app already needs to be deployed and I need to know which container holds the secret.

Update
These are the possible approaches how to implement it:

run cat /var/run/secrets/<secret_name> inside the container
Use ssh to read /var/lib/docker/containers/<container-id>/mounts/secrets/<secret_id> of the host server
Create a lightweight container, mount all the secrets and read them using cat /var/run/secrets/<secret_name>

For 1. and 2. abra should automatically select the correct container, so the user don't need to know the container. The disadvantage of these approaches is that there need to be a running container. It won't work for new apps that are not yet deployed or for crashed container. Approach 3. would solve this issue, but it would create a lot of overhead by launching a new container just to read the secrets.

For the autoconfigurator https://git.coopcloud.tech/moritz/alakazam I need to exchange secrets between several apps. `abra app secret get <domain> <secret-name>` would be a very useful command. The current workaround is to run `abra app run <domain> <container> cat /var/run/secrets/<secret_name>`. The problem with this command is that the app already needs to be deployed and I need to know which container holds the secret. **Update** These are the possible approaches how to implement it: 1. run `cat /var/run/secrets/<secret_name>` inside the container 2. Use ssh to read `/var/lib/docker/containers/<container-id>/mounts/secrets/<secret_id>` of the host server 3. Create a lightweight container, mount all the secrets and read them using `cat /var/run/secrets/<secret_name>` For `1.` and `2.` abra should automatically select the correct container, so the user don't need to know the container. The disadvantage of these approaches is that there need to be a running container. It won't work for new apps that are not yet deployed or for crashed container. Approach `3.` would solve this issue, but it would create a lot of overhead by launching a new container just to read the secrets.

moritz added the enhancement label 2023-08-25 14:30:31 +00:00

moritz changed title from ~~abra app secret get <domain> <secret-name>~~ to `abra app secret get <domain> <secret-name>`

2023-08-25 14:39:34 +00:00

decentral1se commented

2023-08-26 07:59:42 +00:00

abra app secret get <domain> <secret-name> would be a very useful command.

👍

The problem with this command is that the app already needs to be deployed and I need to know which container holds the secret.

Maybe abra app secret list --machine to get a list of secrets? Altho the current implement of secret ls doesn't list which container holds the secret. This could be added perhaps?

> `abra app secret get <domain> <secret-name>` would be a very useful command. 👍 > The problem with this command is that the app already needs to be deployed and I need to know which container holds the secret. Maybe `abra app secret list --machine` to get a list of secrets? Altho the current implement of `secret ls` doesn't list which container holds the secret. This could be added perhaps?

moritz referenced this issue from coop-cloud/backup-bot-two

2023-09-21 22:46:18 +00:00

Backup Secrets #28

moritz commented

2023-09-21 22:49:56 +00:00

coop-cloud/backup-bot-two#28

Unfortunately I couldn't find any other way to access the secrets, than attaching them to a container and reading /var/run/secrets.
Therefore my idea is to attach all secrets of all apps that have the 'backupbot.backup label to the backupbot. Than the secrets can easily be handled by restic.

Maybe the backupbot container could be used for this command. Another idea would be to start a very lightweight container and attach all secrets to it. But running this command would take a lot of time if always a whole container needs to be started.

https://git.coopcloud.tech/coop-cloud/backup-bot-two/issues/28 > Unfortunately I couldn't find any other way to access the secrets, than attaching them to a container and reading `/var/run/secrets`. > Therefore my idea is to attach all secrets of all apps that have the `'backupbot.backup` label to the backupbot. Than the secrets can easily be handled by restic. Maybe the backupbot container could be used for this command. Another idea would be to start a very lightweight container and attach all secrets to it. But running this command would take a lot of time if always a whole container needs to be started.

👀 1

moritz commented

2023-09-22 12:07:42 +00:00

It could be solved another way by accessing /var/lib/docker/containers/<container-id>/mounts/secrets via ssh.
A way for exporting and importing all secrets would be very nice for migrating an app to another server.
abra app secret list <old_domain> --machine | abra app secret import <new_domain> -

It could be solved another way by accessing `/var/lib/docker/containers/<container-id>/mounts/secrets` via ssh. A way for exporting and importing all secrets would be very nice for migrating an app to another server. `abra app secret list <old_domain> --machine | abra app secret import <new_domain> - `

moritz commented

2023-09-22 14:21:54 +00:00

It could be solved another way by accessing /var/lib/docker/containers/<container-id>/mounts/secrets via ssh.

Unfortunately via /var/lib/docker only secrets that are mounted into a running container are accessible. So the only way to access a secret that is not mounted inside a container is to mount it into a container...

> It could be solved another way by accessing `/var/lib/docker/containers/<container-id>/mounts/secrets` via ssh. Unfortunately via `/var/lib/docker` only secrets that are mounted into a running container are accessible. So the only way to access a secret that is not mounted inside a container is to mount it into a container...

👀 1

moritz commented

2023-09-22 16:08:12 +00:00

Create a lightweight container, mount all the secrets and read them using cat /var/run/secrets/<secret_name>

For estimating the the runtime overhead:

docker service create -q --name secret-service --secret smtp_password_v1  busybox tail -f /dev/null
container_id=`docker ps --filter name=secret-service -q`
docker container exec $container_id grep "" -r /run/secrets/
docker service rm secret-service > /dev/null

This took about 12s-20s independent of the number of secrets.

> 3. Create a lightweight container, mount all the secrets and read them using `cat /var/run/secrets/<secret_name>` For estimating the the runtime overhead: ``` docker service create -q --name secret-service --secret smtp_password_v1 busybox tail -f /dev/null container_id=`docker ps --filter name=secret-service -q` docker container exec $container_id grep "" -r /run/secrets/ docker service rm secret-service > /dev/null ``` This took about 12s-20s independent of the number of secrets.

moritz commented

2023-10-03 13:02:24 +00:00

There is another way to get secrets that are not mounted to any container without creating a container.
https://medium.com/lucjuggery/raft-logs-on-swarm-mode-1351eff1e690

Using the tool swarm-rafttool I can read the content of a secret referenced by its id or name from the raft log.

$ cp -r /var/lib/docker/swarm /tmp/swarm_unlocked
$ ./swarmkit/bin/swarm-rafttool dump-object secret --name test_secret_007 --state-dir /tmp/swarm_unlocked/

id: "jbbd13y83xruo4l8ez0bb1ii9"
meta: <
  version: <
    index: 41407080
  >
  created_at: <
    seconds: 1696260004
    nanos: 793386990
  >
  updated_at: <
    seconds: 1696260004
    nanos: 793386990
  >
>
spec: <
  annotations: <
    name: "test_secret_007"
    labels: <
      key: "test_secret_187"
      value: ""
    >
  >
  data: "VERY SECRET VALUE 1337\n"
>

Execution time is about 4s.
On the test server the Raft log is about 130MB, this is way too large for downloading it for dumping the secrets.
The question is how to get the go binary on the server?

There is another way to get secrets that are not mounted to any container without creating a container. https://medium.com/lucjuggery/raft-logs-on-swarm-mode-1351eff1e690 Using the tool [`swarm-rafttool`](https://github.com/moby/swarmkit) I can read the content of a secret referenced by its id or name from the raft log. ``` $ cp -r /var/lib/docker/swarm /tmp/swarm_unlocked $ ./swarmkit/bin/swarm-rafttool dump-object secret --name test_secret_007 --state-dir /tmp/swarm_unlocked/ id: "jbbd13y83xruo4l8ez0bb1ii9" meta: < version: < index: 41407080 > created_at: < seconds: 1696260004 nanos: 793386990 > updated_at: < seconds: 1696260004 nanos: 793386990 > > spec: < annotations: < name: "test_secret_007" labels: < key: "test_secret_187" value: "" > > data: "VERY SECRET VALUE 1337\n" > ``` Execution time is about 4s. On the test server the Raft log is about 130MB, this is way too large for downloading it for dumping the secrets. The question is how to get the go binary on the server?

👀 1

decentral1se commented

2023-10-11 16:44:22 +00:00

@moritz Since we already have a lot of these dependencies, we could just re-work the code from the command itself? Some copy/pasta and delete because we know we only need the secret handling code. See https://github.com/moby/swarmkit/blob/f082dd7a0ceeabf146a4705597b48b1c21aa3b8a/cmd/swarm-rafttool/dump.go#L250-L463 Sounds good?

decentral1se added the abra label 2023-10-11 16:45:15 +00:00

moritz commented

2023-10-12 08:24:24 +00:00

I think we can use it as it is instead of extra copy/pasta code maintenance. I doubt we can integrate the code directly in abra, because it should be executed on the server side, as it needs to read the raft log. The only way to completely integrate it into abra would be to download the whole raft log, but this can be huge.

Do you have a better idea than building a binary that comes together with abra and abra copies it to the server to parse the raft log? Or is there some magic way for remote go code execution?

I think we can use it as it is instead of extra copy/pasta code maintenance. I doubt we can integrate the code directly in abra, because it should be executed on the server side, as it needs to read the raft log. The only way to completely integrate it into abra would be to download the whole raft log, but this can be huge. Do you have a better idea than building a binary that comes together with abra and abra copies it to the server to parse the raft log? Or is there some magic way for remote go code execution?

👀 1

moritz commented

2025-10-14 15:40:33 +00:00

from @cyrnel :

docker service create --name temp-reader --secret <secret-name> --mode replicated-job alpine:latest sh -c "cat /run/secrets/<secret-name>" && docker service logs --raw temp-reader && echo && docker service rm temp-reader

from @cyrnel : ``` docker service create --name temp-reader --secret <secret-name> --mode replicated-job alpine:latest sh -c "cat /run/secrets/<secret-name>" && docker service logs --raw temp-reader && echo && docker service rm temp-reader ```

decentral1se referenced this issue from toolshed/abra

2025-10-15 08:01:26 +00:00

`abra app secret get <domain> <secret-name>` #694

decentral1se commented

2025-10-15 08:01:43 +00:00

Let's continue on toolshed/abra#694.

Thanks for all the input so far folks.

Let's continue on https://git.coopcloud.tech/toolshed/abra/issues/694. Thanks for all the input so far folks.

decentral1se closed this issue

2025-10-15 08:01:43 +00:00

Sign in to join this conversation.