add abra app secret backup command? #539

New Issue

mayel · 2025-04-21T14:48:33Z

mayel commented

2025-04-21 14:48:33 +00:00

here's a proof of concept:

#!/bin/bash

export DOMAIN="$1"

# Get the list of secret names using a more robust approach
secrets=$(abra app secret list ${DOMAIN} --chaos)

# Check if we got any secrets
if [ -z "$secrets" ]; then
  echo "No secrets found or error retrieving secrets"
  exit 1
fi

echo "$secrets"

# Extract just the secret names from the table output
parsed_secrets=$(echo "$secrets" | awk -F '┃' '/^┃/ && NR>3 {gsub(/ /, "", $2); if($2 != "") print $2}')

# Check if we got any secrets
if [ -z "$parsed_secrets" ]; then
  echo "Error parsing secrets"
  exit 1
fi

echo "Found the following secrets:"
echo "$parsed_secrets"
echo "------------------------"
echo "Retrieving secret values:"
echo "------------------------"

# Create a backup file
backup_file="secrets_backup_$(date +%Y%m%d_%H%M%S).sh"
echo -e "# Secret backup from $DOMAIN on $(date)\n\nexport DOMAIN="\$1"
\n" > "$backup_file"

# Loop through each secret name and fetch its value
while IFS= read -r secret_name; do
  if [ -n "$secret_name" ]; then
    echo "Secret: $secret_name"
    echo "--------------"
    
    # Fetch the secret value
    secret_value=$(abra app run $DOMAIN --no-tty app cat "/run/secrets/$secret_name" </dev/null 2>/dev/null)
    
    # Display the secret
    # echo "$secret_value"
    
    # Save to backup file
    echo -e "\nabra app secret insert \$DOMAIN $secret_name v1 $secret_value" >> "$backup_file"
    
    echo "--------------"
  fi
done < <(printf "%s\n" "$parsed_secrets")

chmod +x "$backup_file"

echo "All secrets have been retrieved and saved to $backup_file"

here's a proof of concept: ```bash #!/bin/bash export DOMAIN="$1" # Get the list of secret names using a more robust approach secrets=$(abra app secret list ${DOMAIN} --chaos) # Check if we got any secrets if [ -z "$secrets" ]; then echo "No secrets found or error retrieving secrets" exit 1 fi echo "$secrets" # Extract just the secret names from the table output parsed_secrets=$(echo "$secrets" | awk -F '┃' '/^┃/ && NR>3 {gsub(/ /, "", $2); if($2 != "") print $2}') # Check if we got any secrets if [ -z "$parsed_secrets" ]; then echo "Error parsing secrets" exit 1 fi echo "Found the following secrets:" echo "$parsed_secrets" echo "------------------------" echo "Retrieving secret values:" echo "------------------------" # Create a backup file backup_file="secrets_backup_$(date +%Y%m%d_%H%M%S).sh" echo -e "# Secret backup from $DOMAIN on $(date)\n\nexport DOMAIN="\$1" \n" > "$backup_file" # Loop through each secret name and fetch its value while IFS= read -r secret_name; do if [ -n "$secret_name" ]; then echo "Secret: $secret_name" echo "--------------" # Fetch the secret value secret_value=$(abra app run $DOMAIN --no-tty app cat "/run/secrets/$secret_name" </dev/null 2>/dev/null) # Display the secret # echo "$secret_value" # Save to backup file echo -e "\nabra app secret insert \$DOMAIN $secret_name v1 $secret_value" >> "$backup_file" echo "--------------" fi done < <(printf "%s\n" "$parsed_secrets") chmod +x "$backup_file" echo "All secrets have been retrieved and saved to $backup_file"

🚀 1

decentral1se commented

2025-04-21 15:44:05 +00:00

@moritz does backup-bot-two take care of this?

@moritz does `backup-bot-two` take care of this?

decentral1se commented

2025-04-21 17:09:20 +00:00

It is covered AFAIU.

		backupbot.py
		Lines 357 to 382 in f7f46d7b7b
	
				def copy_secrets(apps):

				    # TODO: check if it is deployed

				    rmtree(SECRET_PATH, ignore_errors=True)

				    os.mkdir(SECRET_PATH)

				    client = docker.from_env()

				    container_by_service = {

				        c.labels.get('com.docker.swarm.service.name'): c for c in client.containers.list()}

				    services = client.services.list()

				    for s in services:

				        app_name = s.attrs['Spec']['Labels']['com.docker.stack.namespace']

				        if (app_name in apps and

				                (app_secs := s.attrs['Spec']['TaskTemplate']['ContainerSpec'].get('Secrets'))):

				            if not container_by_service.get(s.name):

				                logger.warning(

				                    f"Container {s.name} is not running, secrets can not be copied.")

				                continue

				            container_id = container_by_service[s.name].id

				            for sec in app_secs:

				                src = f'/var/lib/docker/containers/{container_id}/mounts/secrets/{sec["SecretID"]}'

				                if not Path(src).exists():

				                    logger.error(

				                        f"For the secret {sec['SecretName']} the file {src} does not exist for {s.name}")

				                    continue

				                dst = SECRET_PATH + sec['SecretName']

				                logger.debug(f"Copy Secret {sec['SecretName']}")

				                copyfile(src, dst)

		backupbot.py
		Lines 501 to 563 in f7f46d7b7b
	
				@cli.command()

				@click.option('snapshot', '--snapshot', '-s', envvar='SNAPSHOT', default='latest')

				@click.option('path', '--path', '-p', envvar='INCLUDE_PATH')

				@click.option('volumes', '--volumes', '-v', envvar='VOLUMES')

				@click.option('secrets', '--secrets', '-c', is_flag=True, envvar='SECRETS')

				def download(snapshot, path, volumes, secrets):

				    file_dumps = []

				    if snapshot == 'latest':

				        latest_snapshot = get_snapshots('latest')

				        if not latest_snapshot:

				            logger.error(f"There is no latest snapshot for {SERVICE}")

				            exit(1)

				        snapshot = latest_snapshot[0]['short_id']

				    if not any([path, volumes, secrets]):

				        volumes = secrets = True

				    if path:

				        path = path.removesuffix('/')

				        binary_output = dump(snapshot, path)

				        files = list_files(snapshot, path)

				        filetype = [f.get('type') for f in files if f.get('path') == path][0]

				        filename = Path(path).name

				        if filetype == 'dir':

				            filename = filename + ".tar"

				        tarinfo = tarfile.TarInfo(name=filename)

				        tarinfo.size = len(binary_output)

				        file_dumps.append((binary_output, tarinfo))

				    if volumes:

				        if SERVICE == 'ALL':

				            logger.error("Please specify '--host' when using '--volumes'")

				            exit(1)

				        files = list_files(snapshot, VOLUME_PATH)

				        for f in files[1:]:

				            path = f['path']

				            if Path(path).name.startswith(SERVICE) and f['type'] == 'dir':

				                binary_output = dump(snapshot, path)

				                filename = f"{Path(path).name}.tar"

				                tarinfo = tarfile.TarInfo(name=filename)

				                tarinfo.size = len(binary_output)

				                file_dumps.append((binary_output, tarinfo))

				    if secrets:

				        if SERVICE == 'ALL':

				            logger.error("Please specify '--host' when using '--secrets'")

				            exit(1)

				        filename = f"{SERVICE}.json"

				        files = list_files(snapshot, SECRET_PATH)

				        secrets = {}

				        for f in files[1:]:

				            path = f['path']

				            if Path(path).name.startswith(SERVICE) and f['type'] == 'file':

				                secret = dump(snapshot, path).decode()

				                secret_name = path.removeprefix(f'{SECRET_PATH}{SERVICE}_')

				                secrets[secret_name] = secret

				        binary_output = json.dumps(secrets).encode()

				        tarinfo = tarfile.TarInfo(name=filename)

				        tarinfo.size = len(binary_output)

				        file_dumps.append((binary_output, tarinfo))

				    with tarfile.open('/tmp/backup.tar.gz', "w:gz") as tar:

				        print(f"Writing files to /tmp/backup.tar.gz...")

				        for binary_output, tarinfo in file_dumps:

				            tar.addfile(tarinfo, fileobj=io.BytesIO(binary_output))

				    size = get_formatted_size('/tmp/backup.tar.gz')

				    print(

				        f"Backup has been written to /tmp/backup.tar.gz with a size of {size}")

It is covered AFAIU. https://git.coopcloud.tech/coop-cloud/backup-bot-two/src/commit/f7f46d7b7bc350aa8f46261be070fd2e14bc6ea1/backupbot.py#L357-L382 https://git.coopcloud.tech/coop-cloud/backup-bot-two/src/commit/f7f46d7b7bc350aa8f46261be070fd2e14bc6ea1/backupbot.py#L501-L563

decentral1se closed this issue

2025-04-21 17:09:20 +00:00

decentral1se added the

enhancement

label 2025-04-21 17:09:41 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: toolshed/abra#539

add abra app secret backup command? #539

add `abra app secret backup` command? #539