support multi-hdd config #5

New Issue

2026-04-02T16:36:57Z

fauno commented

2026-04-02 16:36:57 +00:00

how could something like this could be achieved via .env and garage.toml.templ? or include/concatenate an external file with custom configurations?

https://garagehq.deuxfleurs.fr/documentation/operations/multi-hdd/

data_dir = [
    { path = "/path/to/old_data", read_only = true },
    { path = "/path/to/new_hdd1", capacity = "2T" },
    { path = "/path/to/new_hdd2", capacity = "4T" },
]

how could something like this could be achieved via `.env` and `garage.toml.templ`? or include/concatenate an external file with custom configurations? https://garagehq.deuxfleurs.fr/documentation/operations/multi-hdd/ ``` data_dir = [ { path = "/path/to/old_data", read_only = true }, { path = "/path/to/new_hdd1", capacity = "2T" }, { path = "/path/to/new_hdd2", capacity = "4T" }, ] ```

decentral1se commented

2026-04-02 18:39:08 +00:00

If you put data_dir = [{{ env "UGLY_DATA_DIR_DUMP" }}] and then your .tmpl file and then just dump a super ugly '{ path = "/path/to/old_data", read_only = true }, { path = "/path/to/new_hdd1", capacity = "2T" }, { path = "/path/to/new_hdd2", capacity = "4T" },' then you're done? This might be guaranteed to cause further headaches 😬 I can't quite think of a recipe that deals with a complex config setup like this 🤔

If you put `data_dir = [{{ env "UGLY_DATA_DIR_DUMP" }}]` and then your `.tmpl` file and then just dump a super ugly `'{ path = "/path/to/old_data", read_only = true }, { path = "/path/to/new_hdd1", capacity = "2T" }, { path = "/path/to/new_hdd2", capacity = "4T" },'` then you're done? This might be guaranteed to cause further headaches 😬 I can't quite think of a recipe that deals with a complex config setup like this 🤔

fauno commented

2026-04-02 18:45:47 +00:00

thanks! but yeah we'll be dealing with quote-escaping issues then, no?

👍 1

decentral1se commented

2026-04-02 18:47:00 +00:00

Yes, you'd need to check the usage of ' and ". You could potentially escape \" on the .env side. There might be some obscure functionality in Golang templating to help you out here but I wouldn't know. Good luck!

Yes, you'd need to check the usage of `'` and `"`. You could potentially escape `\"` on the `.env` side. There might be some obscure functionality in Golang templating to help you out here but I wouldn't know. Good luck!

fauno commented

2026-04-02 19:02:34 +00:00

@decentral1se what's the templating called? i seem to find a lot of these for golang and never the official reference

decentral1se commented

2026-04-02 20:22:34 +00:00

It's a disaster in terms of docs: https://pkg.go.dev/text/template#hdr-Actions I just search online and read what other people did. Looking at other recipes... there are even features which are only implemented in swarms version of the templating which aren't even supported in the compose spec. Dog bless us all.

fauno commented

2026-04-03 15:00:00 +00:00

besides templating issues, i'm thinking disks could be mounted on $LOCAL_FOLDER_DATA/diskX so they example config would be like this:

data_dir = [
    { path = "/var/lib/garage/data/disk1", read_only = true },
    { path = "/var/lib/garage/data/disk2", capacity = "2T" },
    { path = "/var/lib/garage/data/disk3", capacity = "4T" },
]

but i'm not sure if docker will show the nested mount points to the container.

reference:

https://git.coopcloud.tech/coop-cloud/garage/src/branch/main/compose.yml#L33

besides templating issues, i'm thinking disks could be mounted on `$LOCAL_FOLDER_DATA/diskX` so they example config would be like this: ``` data_dir = [ { path = "/var/lib/garage/data/disk1", read_only = true }, { path = "/var/lib/garage/data/disk2", capacity = "2T" }, { path = "/var/lib/garage/data/disk3", capacity = "4T" }, ] ``` but i'm not sure if docker will show the nested mount points to the container. reference: https://git.coopcloud.tech/coop-cloud/garage/src/branch/main/compose.yml#L33

fauno commented

2026-04-03 15:40:26 +00:00

looks like it would work, so no changes needed to compose.yml:

mkdir local_folder_data

for i in 1 2; do
  dd if=/dev/zero of=disk$i.img bs=1M count=100
  losetup /dev/loop$i disk$i.img
  mkfs.ext4 /dev/loop$i
  mkdir -p local_folder_data/disk$i
  mount /dev/loop$i local_folder_data/disk$i
done

docker run -v ./local_folder_data:/srv -it --rm alpine:latest sh

# now inside the container
touch /srv/disk1/hi
touch /srv/disk2/hi
exit

# back to host
ls -hal local_folder_data/disk*

for i in 1 2; do
  umount local_folder_data/disk$i
  losetup -d /dev/loop$i
done

looks like it would work, so no changes needed to `compose.yml`: ```sh mkdir local_folder_data for i in 1 2; do dd if=/dev/zero of=disk$i.img bs=1M count=100 losetup /dev/loop$i disk$i.img mkfs.ext4 /dev/loop$i mkdir -p local_folder_data/disk$i mount /dev/loop$i local_folder_data/disk$i done docker run -v ./local_folder_data:/srv -it --rm alpine:latest sh # now inside the container touch /srv/disk1/hi touch /srv/disk2/hi exit # back to host ls -hal local_folder_data/disk* for i in 1 2; do umount local_folder_data/disk$i losetup -d /dev/loop$i done ```

Brooke added this to the Issue tracking project 2026-05-08 19:29:39 +00:00

Brooke added the Kind/Enhancement

Priority

Low

4

labels 2026-05-15 20:19:05 +00:00

Brooke commented

2026-06-17 22:24:53 +00:00

looks like it would work, so no changes needed to compose.yml:

mkdir local_folder_data

for i in 1 2; do
  dd if=/dev/zero of=disk$i.img bs=1M count=100
  losetup /dev/loop$i disk$i.img
  mkfs.ext4 /dev/loop$i
  mkdir -p local_folder_data/disk$i
  mount /dev/loop$i local_folder_data/disk$i
done

docker run -v ./local_folder_data:/srv -it --rm alpine:latest sh

# now inside the container
touch /srv/disk1/hi
touch /srv/disk2/hi
exit

# back to host
ls -hal local_folder_data/disk*

for i in 1 2; do
  umount local_folder_data/disk$i
  losetup -d /dev/loop$i
done

I'm confused about the purpose of this script, what is the suggested change to the recipe to support multi-hdd?

> looks like it would work, so no changes needed to `compose.yml`: > > ```sh > mkdir local_folder_data > > for i in 1 2; do > dd if=/dev/zero of=disk$i.img bs=1M count=100 > losetup /dev/loop$i disk$i.img > mkfs.ext4 /dev/loop$i > mkdir -p local_folder_data/disk$i > mount /dev/loop$i local_folder_data/disk$i > done > > docker run -v ./local_folder_data:/srv -it --rm alpine:latest sh > > # now inside the container > touch /srv/disk1/hi > touch /srv/disk2/hi > exit > > # back to host > ls -hal local_folder_data/disk* > > for i in 1 2; do > umount local_folder_data/disk$i > losetup -d /dev/loop$i > done > ``` I'm confused about the purpose of this script, what is the suggested change to the recipe to support multi-hdd?

Brooke commented

2026-06-17 23:16:34 +00:00

I was thinking through this, what do people think about a syntax like:

MULTI_DISKS_ROOT=/mnt
MULTI_DISKS="disk1:2T:false,disk2:4T:false,disk3:1T:true,disk4:8T:false"

Where disks are set with a defined format and assumed to be relative to the set root, this way there's not a ton of complex changes to compose.yml

I'm still working on it but I believe that I have a working tmpl file that can ingest this syntax.

I was thinking through this, what do people think about a syntax like: ``` MULTI_DISKS_ROOT=/mnt MULTI_DISKS="disk1:2T:false,disk2:4T:false,disk3:1T:true,disk4:8T:false" ``` Where disks are set with a defined format and assumed to be relative to the set root, this way there's not a ton of complex changes to compose.yml I'm still working on it but I believe that I have a working tmpl file that can ingest this syntax.

fauno commented

2026-06-18 15:50:25 +00:00

looks great @Brooke! the script was an experiment to see if bind mounting partitions into a volume would make them available to the container, which seemed to be the case and you can render a template like this #5 (comment) without changes to the compose file

looks great @Brooke! the script was an experiment to see if bind mounting partitions into a volume would make them available to the container, which seemed to be the case and you can render a template like this https://git.coopcloud.tech/coop-cloud/garage/issues/5#issuecomment-32693 without changes to the compose file

👍 1

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: coop-cloud/garage#5