131 lines
5.0 KiB
Markdown
131 lines
5.0 KiB
Markdown
# Garage
|
|
|
|
> An open-source distributed object storage service tailored for selfhosting at a small-to-medium scale.
|
|
|
|
<!-- metadata -->
|
|
|
|
* **Category**: Apps
|
|
* **Maintainers**: [Brooke](https://git.coopcloud.tech/Brooke), [fauno](https://git.coopcloud.tech/fauno)
|
|
* **Status**: 3
|
|
* **Image**: [`garage`](https://hub.docker.com/r/dxflrs/garage), 4, upstream
|
|
* **Healthcheck**: Yes
|
|
* **Backups**: No
|
|
* **Email**: N/A
|
|
* **Tests**: 3
|
|
* **SSO**: N/A
|
|
|
|
<!-- endmetadata -->
|
|
|
|
## Quick start
|
|
|
|
- `abra app new garage`
|
|
- If you are **creating a new cluster**:
|
|
- Generate a new rpc_secret: `abra app secret generate --all`
|
|
- Note: all nodes must share the same rpc secret, do not lose this value if you plan to cluster garage!
|
|
- Note: In older versions of abra you must generate the secret locally with `openssl rand -hex 32` then insert the result as described below
|
|
- `abra app config <app-domain>`
|
|
- If this Garage node is **joining an existing cluster**:
|
|
- Insert the existing rpc_secret: `abra app secret insert <app-domain> rpc_secret v1 <rpc-secret>`
|
|
- `abra app config <app-domain>`
|
|
- Uncomment the block that starts with `## Bootstrap this node`
|
|
- Set `BOOTSTRAP_HOST` and `BOOTSTRAP_ID`
|
|
- `abra app deploy <app-domain>`
|
|
|
|
## Configuration
|
|
|
|
### Allow RPC Connections
|
|
|
|
* Your ingress controller must be set up to allow connections on port 3901. We assume you're using Traefik
|
|
* `abra app config <traefik-app-name>`
|
|
* Uncomment the lines:
|
|
* `#COMPOSE_FILE="$COMPOSE_FILE:compose.garage.yml"`
|
|
* `#GARAGE_RPC_ENABLED=1`
|
|
* Re-deploy Traefik: `abra app deploy -f <traefik-app-name>`
|
|
|
|
### Publish admin endpoint
|
|
|
|
* Your ingress controller must be set up to allow connections on port 3903. We assume you're using Traefik
|
|
* `abra app config <traefik-app-name>`
|
|
* Uncomment the lines:
|
|
* `#COMPOSE_FILE="$COMPOSE_FILE:compose.garage-admin.yml"`
|
|
* `#GARAGE_ADMIN_ENABLED=1`
|
|
* Re-deploy Traefik: `abra app deploy -f <traefik-app-name>`
|
|
|
|
### Prepare the Garage Client
|
|
To interact with garage inside docker, it's best to create an alias for the following abra run command.
|
|
```
|
|
alias garage="abra app run <app domain> -- app /garage"
|
|
```
|
|
Run `garage status` to verify everything is working.
|
|
|
|
You can optionally add this alias to your `.bashrc` (or similar) file to avoid having to define it repeatedly.
|
|
|
|
|
|
### Garage Quick Start Guide
|
|
Once `garage status` works, you can follow the guide here: https://garagehq.deuxfleurs.fr/documentation/quick-start/#checking-that-garage-runs-correctly
|
|
|
|
## Monitoring
|
|
|
|
### Enabling
|
|
|
|
By default monitoring is disabled and must be enabled in your config.
|
|
|
|
To enable, set `MONITORING` to `true` and uncomment the line `#COMPOSE_FILE="$COMPOSE_FILE:compose.monitoring.yml"`.
|
|
|
|
> If you've deployed garage before ver `0.0.2+v2.3.0` then you will need to add the following lines to your config:
|
|
> ```
|
|
> MONITORING_DOMAIN=monitoring.garage.example.com
|
|
> SECRET_ADMIN_TOKEN_SECRET_VERSION=v1 # length=64 charset=hex
|
|
> SECRET_METRICS_TOKEN_SECRET_VERSION=v1 # length=64 charset=hex
|
|
>
|
|
> #COMPOSE_FILE="$COMPOSE_FILE:compose.monitoring.yml"
|
|
> MONITORING="true"
|
|
> ```
|
|
|
|
If you're using the
|
|
[monitoring-ng](https://recipes.coopcloud.tech/monitoring-ng) recipe,
|
|
insert the `metrics_token` with the value of `basic_auth`:
|
|
|
|
```sh
|
|
abra app secret insert garage.example.coop v1 metrics_token BASIC_AUTH
|
|
```
|
|
|
|
### Deploying
|
|
|
|
Now, undeploy the service, generate the new secrets, and finally re-deploy:
|
|
```
|
|
abra app undeploy <app-domain>
|
|
abra app secret generate --all <app-domain>
|
|
abra app deploy <app-domain>
|
|
```
|
|
### Utilizing metrics
|
|
|
|
Within your chosen monitoring software (ie. Telegraf, Prometheus, etc.), you'll need to make sure it interprets the correct scheme (https), and point it at <app-domain>/metrics as the monitoring endpoint. The secret you copied earlier called metrics_token will be used to authenticate the request.
|
|
|
|
## Upload limits
|
|
|
|
To prevent "499 Client Closed Request" errors you should make sure your
|
|
`READ_TIMEOUT` and `WRITE_TIMEOUT` variables on Traefik are high enough.
|
|
Same on the proxy if you're reverse-proxying Traefik.
|
|
|
|
Change `MAX_REQUEST_BODY_BYTES` to limit the maximum uploaded chunk
|
|
size, which could range from 5MiB to 5GiB. Take into account S3 clients
|
|
will perform multipart uploads, so this variable doesn't limit the
|
|
maximum file size, just what amount of bytes Traefik considers before
|
|
closing the connection.
|
|
|
|
## Backups
|
|
|
|
> In development, not currently reliable
|
|
|
|
By default, backups will only capture a snapshot of the metadata directory, which includes bucket names, hashed secrets, and other related information.
|
|
By default, the actual data will not be backed up!
|
|
If you're running Garage in a cluster, when you restore the metadata, other nodes will provide any missing data (assuming a replication factor >1).
|
|
|
|
### To enable full data backups
|
|
* `abra app config <app domain>`
|
|
* Uncomment the block that starts with `## Enable Full Data Backups`
|
|
* Re-deploy Garage: `abra app undeploy -n <app domain> && sleep 5 && abra app deploy -n <app domain>`
|
|
|
|
For more, see [`garagehq.deuxfleurs.fr`](https://garagehq.deuxfleurs.fr/documentation/cookbook/real-world/).
|