Support metadata or full data backup + documentation

.env.sample

@@ -5,7 +5,7 @@ DOMAIN=garage.example.com
 LETS_ENCRYPT_ENV=production
 COMPOSE_FILE="compose.yml"
 
-SECRET_RPC_SECRET_VERSION=v1
+SECRET_RPC_SECRET_VERSION=v1 # length=32 charset=hex
 
  # Changing the replication factor after initial deployment is not 
  # supported and requires deleting the existing cluster layout metadata.
@@ -19,4 +19,7 @@ BLOCK_SIZE=1MiB # only increase if there is a fast network connection between no
 
 # Use a directory on the host instead of a docker volume for storage
 #LOCAL_FOLDER_META=/path/on/docker/host
-#LOCAL_FOLDER_DATA=/path/on/docker/host
+#LOCAL_FOLDER_DATA=/path/on/docker/host
+
+## Enable Full Data Backups (not just metadata)
+# COMPOSE_FILE="$COMPOSE_FILE:compose.fullbackup.yml"

README.md

@@ -18,39 +18,44 @@
 ## Quick start
 
 * `abra app new garage`
-* Garage is particular about the rpc secret, generate it locally with `openssl rand -hex 32` then insert the result
-* `abra app secret i <app-domain> rpc_secret v1 <rpc-secret>`
+* If you don't already have an RPC secret for your Garage cluster, generate one: `abra app secret generate --all`
+    * Note: In older versions of abra you must generate the secret locally with `openssl rand -hex 32` then insert the result as described below
+* If this Garage node is joining a cluster with an existing RPC secret, insert it: `abra app secret insert <app-domain> rpc_secret v1 <rpc-secret>`
 > Note: all nodes must share the same rpc secret, do not lose this value if you plan to cluster garage!
 * `abra app config <app-domain>`
 * `abra app deploy <app-domain>`
 
-## Peering
+## Configuration
 
-#### Garage CLI
+### Allow RPC Connections
+
+* Your ingress controller must be set up to allow connections on port 3901. We assume you're using Traefik
+* `abra app configure <traefik-app-name>`
+* Uncomment the block that starts with `## Garage`
+* Re-deploy Traefik: `abra app undeploy -n <traefik-app-name> && sleep 5 && abra app deploy -n <traefik-app-name>`
+
+### Prepare the Garage Client
 Start by creating an alias for the abra run command
 ```
-alias garage="abra app run <app-domain> app /garage"
+alias garage="abra app run <app domain> -- app /garage"
 ```
 Run `garage status` to verify everything is working
 
-#### Assign Roles
-
-Terms:
-* `node id` (reqired) - Node identifier supplied by the garage CLI, can be found by running `garage node id`.
-* `zone` (reqired) - Identifier for how nodes will be grouped, a zone usually refers to a geographical location (us-east, paris-1, etc.) no specific syntax is required, zones can be called anything.
-* `capacity` (reqired) - Disk space the node will be allocating to the cluster, use T and G for units (Terabytes and Gigabytes respectively).
-* `tag` (optional) - Additional notes appended to garage status, usually a title for the node.
-
-> Role assignment command conflicts with `abra app run`'s -t option\
-> Connecting not currently implemented
+### Garage Quick Start Guide
+Once `garage status` works, you can follow the guide here: https://garagehq.deuxfleurs.fr/documentation/quick-start/#checking-that-garage-runs-correctly
 
 ## Backups
 
-> Not currently implemented
+> In development, not currently reliable
 
-Backups will only capture a snapshot of the metadata directory, which includes bucket names, hashed secrets, and other related information. However, they do not include the actual data!
+By default, backups will only capture a snapshot of the metadata directory, which includes bucket names, hashed secrets, and other related information. 
+By default, the actual data will not be backed up!
+If you're running Garage in a cluster, when you restore the metadata, other nodes will send the new node any missing data.
+
+### To enable full data backups
+* `abra app config <app domain>`
+* Uncomment the block that starts with `## Enable Full Data Backups`
+* Re-deploy Garaga: `abra app undeploy -n <app domain> && sleep 5 && abra app deploy -n <app domain>`
 
-If you're running Garage in a cluster, when you restore the metadata, other nodes will send the new node any missing data.\
-Finally, please note that Abra backups are not a substitute for a proper data replication strategy, and it's recommended to run Garage in a cluster if you need data redundancy.
 
 For more, see [`garagehq.deuxfleurs.fr`](https://garagehq.deuxfleurs.fr/documentation/cookbook/real-world/).

compose-fullbackup.yml

@@ -0,0 +1,9 @@
+---
+version: "3.8"
+
+services:
+  app:
+    deploy:
+      labels:
+        - "backupbot.backup=true"
+        - "backupbot.backup.path=/var/lib/garage/meta,/var/lib/garage/data"

compose.yml

@@ -3,7 +3,7 @@ version: "3.8"
 
 services:
   app:
-    image: dxflrs/garage:v1.0.0
+    image: dxflrs/garage:v2.1.0
     configs:
       - source: garage_conf
         target: /etc/garage.toml
@@ -11,6 +11,7 @@ services:
       - rpc_secret
     networks:
       - proxy
+      - internal
     deploy:
       restart_policy:
         condition: on-failure
@@ -20,7 +21,12 @@ services:
         - "traefik.http.routers.${STACK_NAME}.rule=Host(`${DOMAIN}`)"
         - "traefik.http.routers.${STACK_NAME}.entrypoints=web-secure"
         - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
+        - "traefik.tcp.routers.${STACK_NAME}-rpc.rule=HostSNI(`*`)"
+        - "traefik.tcp.routers.${STACK_NAME}-rpc.entrypoints=garage-rpc"
+        - "traefik.tcp.services.${STACK_NAME}-rpc.loadbalancer.server.port=3901"
         - "coop-cloud.${STACK_NAME}.version=0.0.1+1.0.0"
+        - "backupbot.backup=true"
+        - "backupbot.backup.path=/var/lib/garage/meta,/var/lib/garage/data"
     volumes:
       - "${LOCAL_FOLDER_META:-meta}:/var/lib/garage/meta"
       - "${LOCAL_FOLDER_DATA:-data}:/var/lib/garage/data"
@@ -28,6 +34,7 @@ services:
 networks:
   proxy:
     external: true
+  internal:
 
 configs:
   garage_conf:

garage.toml.tmpl

@@ -13,6 +13,7 @@ block_size = '{{ env "BLOCK_SIZE" }}'
 compression_level = 2
 
 rpc_bind_addr = "[::]:3901"
+rpc_public_addr = "{{ env "DOMAIN" }}:3901"
 rpc_addr = "[::]:3901"
 rpc_secret = "{{ secret "rpc_secret" }}"