Improve maintenance import-data script

MAINTENANCE.md

@@ -30,3 +30,68 @@ Everyone can apply to be a recipe maintainer:
 1. Watch the repository to always get updates
 2. Simply add your self to the list in the [README.md](./README.md) and open a new pull request with the change.
 3. Once the pull request gets merged you will be added to the [loomio maintainers team](https://git.coopcloud.tech/org/coop-cloud/teams/loomio-maintainers).
+
+# Testing Loomio Changes
+
+As a maintainer, it is helpful to have a dev environment to test changes. One can easily be created in the usual way with `abra app new loomio`, but to test changes and upgrades it's good to have a database with a bunch of test data.
+
+## Populating with test data
+
+You can either make some artificial test data, or import test data from an existing Loomio instance. https://help.loomio.com/en/user_manual/groups/data_export/index.html is super helpful here. Here's a summary:
+
+1. Export existing data from a running instance by having an admin go to a group and click [the "Export group data" settings option](https://help.loomio.com/en/user_manual/groups/data_export/index.html#export-data)
+
+2. Fully deploy a clean test instance of Loomio. Note: importing might not work well on a database that has data in it, so you might want to wipe your test instance when doing an import, if it already has data in it
+
+3. Copy the json into the Loomio `app` container by doing
+
+```shell
+[local] $ scp loomio-testdata.json username@your-domain.org:loomio-deploy/import/
+[local] $ ssh username@your-domain.org
+[your-domain.org] $ docker ps | grep loomio | grep app  # note down the loomio app container id
+[your-domain.org] $ sudo docker cp ~/loomio-deploy/import/loomio-testdata.json <loomio-app-container-id>:/import/loomio-testdata.json
+```
+
+4. Set up your rails tools
+
+```shell
+[your-domain.org] $ docker exec -it <loomio-app-container-id> bash
+[loomio] $ DB_PASS="$(cat /run/secrets/db_password)"
+[loomio] $ echo "production:
+  adapter: postgresql
+  host: db
+  database: loomio_production
+  username: postgres
+  password: ${DB_PASS}
+  port: 5432" > config/database.yml
+
+[loomio] $ cd /loomio && RAILS_ENV=production EDITOR=vi bundle exec rails credentials:edit
+```
+
+5. Process the Loomio test data export, to get it into your dev instance's database:
+
+```shell
+[loomio] $ rails console
+
+[loomio][rails] > GroupExportService.import('/import/loomio-testdata.json')
+```
+
+## Manipulating the loomio db
+
+If you want to drop your loomio db(!!), you can do so by logging in to the `db` container:
+
+```shell
+[your-domain.org] $ docker ps | grep loomio | grep _db  # find your loomio _db_ container id
+[your-domain.org] $ docker exec -it <loomio-db-container-id> bash
+[loomio-db] # su postgres
+[loomio-db] $ dropdb loomio_production
+[loomio-db] $ createdb loomio_production
+```
+
+From here, you can do something like recreate it, from the app container:
+
+```shell
+[your-domain.org] $ docker ps | grep loomio | grep app  # find your loomio app container id
+[your-domain.org] $ docker exec -it <loomio-app-container-id> bash
+[loomio-app] $ rake db:setup  # you may have to run through step 4 from earlier
+```

README.md

@@ -3,7 +3,7 @@
 "Loomio is a collaborative decision-making tool that makes it easy for anyone to participate in decisions which affect them. To find out more, visit Loomio.org."
 
 <!-- metadata -->
-* **Maintainer**: [@jmakdah2](https://git.coopcloud.tech/jmakdah2)[@moosemower](https://git.coopcloud.tech/moosemower)
+* **Maintainer**: [@jmakdah2](https://git.coopcloud.tech/jmakdah2) and [@moosemower](https://git.coopcloud.tech/moosemower)
 * **Category**: Apps
 * **Status**: 3, work-in-progress
 * **Image**: [`loomio/*`](https://hub.docker.com/r/loomio), 4, upstream

abra.sh

@@ -1,5 +1,4 @@
 export LOOMIO_ENTRYPOINT_VERSION=v8
-export PG_BACKUP_VERSION=v1
 
 # cannot be integrated into entrypoint.sh as it requires the operator to create a user first
 function make_last_user_admin()

compose.yml

@@ -115,14 +115,12 @@ services:
       PGDATA: /pgdata
     deploy:
       labels:
-        backupbot.backup: ${ENABLE_BACKUPS:-true}
-        backupbot.backup.volumes.pgdata.path: "backup.sql"
-        backupbot.backup.pre-hook: "/pg_backup.sh backup"
-        backupbot.restore.post-hook: '/pg_backup.sh restore'
-    configs:
-        - source: pg_backup
-          target: /pg_backup.sh
-          mode: 0555
+        backupbot.backup: "${ENABLE_BACKUPS:-true}"
+        backupbot.backup.pre-hook: sh -c 'pg_dump -U "$$POSTGRES_USER" -Fc "$$POSTGRES_DB" | gzip > "/postgres.dump.gz"'
+        backupbot.backup.path: "/postgres.dump.gz"
+        backupbot.backup.post-hook: "rm -f /postgres.dump.gz"
+        backupbot.restore: "true"
+        backupbot.restore.post-hook: sh -c 'gzip -d /postgres.dump.gz && pg_restore --clean -U "$$POSTGRES_USER" --dbname="$$POSTGRES_DB" < /postgres.dump && rm -f /postgres.dump'
   redis:
     image: redis:8.0
     networks:
@@ -205,9 +203,6 @@ configs:
   entrypoint:
     name: ${STACK_NAME}_entrypoint_${LOOMIO_ENTRYPOINT_VERSION}
     file: entrypoint.sh
-  pg_backup:
-    name: ${STACK_NAME}_pg_backup_${PG_BACKUP_VERSION}
-    file: pg_backup.sh
 
 secrets:
   devise_secret:

pg_backup.sh

@@ -1,70 +0,0 @@
-#!/bin/bash
-
-set -e
-
-BACKUP_PATH="/pgdata"
-LATEST_BACKUP_FILE="${BACKUP_PATH}/backup.sql"
-
-function backup {
-  FILE_WITH_DATE="${BACKUP_PATH}/backup_$(date +%F).sql"
-  
-  if [ -f "$POSTGRES_PASSWORD_FILE" ]; then
-    export PGPASSWORD=$(cat "$POSTGRES_PASSWORD_FILE")
-  fi
-  
-  echo "Creating backup at ${FILE_WITH_DATE}..."
-  pg_dump -U "${POSTGRES_USER:-postgres}" "${POSTGRES_DB:-postgres}" > "${FILE_WITH_DATE}"
-  
-  echo "Copying to ${LATEST_BACKUP_FILE}..."
-  cp -f "${FILE_WITH_DATE}" "${LATEST_BACKUP_FILE}"
-  
-  echo "Backup done. You will find it at ${LATEST_BACKUP_FILE}"
-}
-
-function restore {
-    echo "Restoring database from ${LATEST_BACKUP_FILE}..."
-
-    cd ${BACKUP_PATH}
-    
-    function restore_config {
-        echo "Restoring original pg_hba.conf configuration..."
-        cat pg_hba.conf.bak > pg_hba.conf
-        su postgres -c 'pg_ctl reload'
-    }
-    
-    # Don't allow any other connections than local
-    echo "Setting up temporary pg_hba.conf to only allow local connections..."
-    cp pg_hba.conf pg_hba.conf.bak
-    echo "local all all trust" > pg_hba.conf
-    su postgres -c 'pg_ctl reload'
-    trap restore_config EXIT INT TERM
-
-    # Recreate Database
-    echo "Dropping existing database ${POSTGRES_DB:-postgres}..."
-    psql -U "${POSTGRES_USER:-postgres}" -d postgres -c "DROP DATABASE \"${POSTGRES_DB:-postgres}\" WITH (FORCE);" 
-    
-    echo "Creating fresh database ${POSTGRES_DB:-postgres}..."
-    createdb -U "${POSTGRES_USER:-postgres}" "${POSTGRES_DB:-postgres}"
-    
-    echo "Restoring data from ${LATEST_BACKUP_FILE}..."
-    psql -U "${POSTGRES_USER:-postgres}" -d "${POSTGRES_DB:-postgres}" -1 -f "${LATEST_BACKUP_FILE}"
-
-    trap - EXIT INT TERM
-    restore_config
-    
-    echo "Database restore completed successfully."
-}
-
-# Execute the function passed as argument
-case "$1" in
-  backup)
-    backup
-    ;;
-  restore)
-    restore
-    ;;
-  *)
-    echo "Usage: $0 {backup|restore}"
-    exit 1
-    ;;
-esac