Merge pull request 'make channels container externally available' (#12) from fix-channels-container into main

Reviewed-on: #12
Reviewed-by: 3wordchant <3wordchant@noreply.git.coopcloud.tech>

README.md

@@ -21,8 +21,23 @@
 * `abra app deploy <app-name>`
 * Open the configured domain in your browser to create your user account (only works in case mail is configured correctly)
 * Give yourself admin rights by running `abra app cmd <app-name> app make_last_user_admin`
+* Deploy [swarm-cronjob](https://recipes.coopcloud.tech/swarm-cronjob) on your server if it is not running yet. This is needed for loomios cron container to be started to do hourly chores.
 
-## Manuel migration steps when upgrading from 0.6.0+v2.25.2 and earlier to 1.0.0+v2.25.2 and later
+## Migration guide
+
+### Upgrading to 4.0.0+v2.25.3 and later
+
+* Before upgrading to `4.0.0+v2.25.3` and later, one successful deployment of `3.0.0+v2.25.3` shall be performed. Doing both at the same time will lead to complications as 3.0.0 runs rails DB migrations and 4.0.0 upgrades the postgres DB version.
+
+### Broken releases
+
+Please do not use these releases, as they are having issues regarding database migrations:
+
+* 1.0.0+v2.25.2
+* 2.0.0+v2.25.3
+* 2.1.0+v2.25.3
+
+### Manual migration steps when upgrading from 0.6.0+v2.25.2 and earlier to 1.0.0+v2.25.2 and later
 
 * adapt your env file with the new vars, especially SECRET_DB_PASSWORD_VERSION=v1 and SECRET_SMTP_PASSWORD_VERSION=v1
 * insert your smtp password with `abra app secret insert <app-name> smtp_password v1 "<your-password>"`

abra.sh

@@ -1,4 +1,4 @@
-export LOOMIO_ENTRYPOINT_VERSION=v5
+export LOOMIO_ENTRYPOINT_VERSION=v6
 
 # cannot be integrated into entrypoint.sh as it requires the operator to create a user first
 function make_last_user_admin()

compose.yml

@@ -10,8 +10,7 @@ x-redis-env: &redis-env
   REDIS_URL: redis://redis:6379
 
 x-environment: &default-env
-  <<: *db-env
+  <<: [*db-env, *redis-env]
-  <<: *redis-env
   CANONICAL_HOST: ${DOMAIN}
   VIRTUAL_HOST: ${DOMAIN}
   CHANNELS_URI: wss://channels.${DOMAIN}
@@ -33,7 +32,7 @@ x-environment: &default-env
 
 services:
   app:
-    image: loomio/loomio:v2.25.3
+    image: loomio/loomio:v2.25.4
     configs:
       - source: entrypoint
         target: /entrypoint.sh
@@ -71,10 +70,16 @@ services:
         - "traefik.http.routers.${STACK_NAME}.rule=Host(`${DOMAIN}`${EXTRA_DOMAINS})"
         - "traefik.http.routers.${STACK_NAME}.entrypoints=web-secure"
         - "traefik.http.routers.${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
-        - "coop-cloud.${STACK_NAME}.version=2.0.0+v2.25.3"
+        - "coop-cloud.${STACK_NAME}.version=4.0.2+v2.25.4"
         - "backupbot.backup:=${ENABLE_BACKUPS:-true}"
+        - "traefik.http.routers.${STACK_NAME}.middlewares=${STACK_NAME}"
+        - "traefik.http.middlewares.${STACK_NAME}.headers.accesscontrolallowmethods=GET,OPTIONS,PUT"
+        - "traefik.http.middlewares.${STACK_NAME}.headers.accesscontrolallowheaders=*"
+        - "traefik.http.middlewares.${STACK_NAME}.headers.accessControlAllowOriginList=https://*.${DOMAIN}"
+        - "traefik.http.middlewares.${STACK_NAME}.headers.accesscontrolmaxage=100"
+        - "traefik.http.middlewares.${STACK_NAME}.headers.addvaryheader=true"
   worker:
-    image: loomio/loomio:v2.25.3
+    image: loomio/loomio:v2.25.4
     configs:
       - source: entrypoint
         target: /entrypoint.sh
@@ -97,6 +102,7 @@ services:
       - loomio_files:/loomio/public/files
       - loomio_plugins:/loomio/plugins/docker
   db:
+    # loomio version upgrades and postgres version upgrade should not be performed at the same time.
     image: pgautoupgrade/pgautoupgrade:17-debian
     networks:
       - backend
@@ -133,12 +139,24 @@ services:
     image: loomio/loomio_channel_server
     networks:
       - backend
+      - proxy
     depends_on:
       - redis
     environment:
       <<: *redis-env
+      VIRTUAL_HOST: channels.${DOMAIN}
+    deploy:
+      labels:
+        - "traefik.enable=true"
+        - "traefik.docker.network=proxy"
+        - "traefik.http.routers.channels${STACK_NAME}.rule=Host(`channels.${DOMAIN}`)"
+        - "traefik.http.routers.channels${STACK_NAME}.tls=true"
+        - "traefik.http.routers.channels${STACK_NAME}.tls.certresolver=${LETS_ENCRYPT_ENV}"
+        - "traefik.http.services.channels${STACK_NAME}.loadbalancer.server.port=5000"
+        - "traefik.http.routers.channels${STACK_NAME}.entrypoints=web-secure"
+
   cron:
-    image: loomio/loomio:v2.25.3
+    image: loomio/loomio:v2.25.4
     configs:
       - source: entrypoint
         target: /entrypoint.sh

entrypoint.sh

@@ -25,22 +25,42 @@ file_env "DEVISE_SECRET"
 file_env "SECRET_COOKIE_TOKEN"
 file_env "POSTGRES_PASSWORD"
 file_env "SMTP_PASSWORD"
+export DB_HOST="db"
 export DATABASE_URL="postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db/${POSTGRES_DB}"
 
 if [ -n "$1" ]; then
 	echo "Running '$1'"
 	$1
 else
-	if [ ! -f /loomio/storage/migrations_ran ] && [ "${TASK:-}" = "worker" ]; then
-		echo "first deploy, running DB setup..."
-		rake db:setup
-		touch /loomio/storage/migrations_ran
-	fi
-
-	echo "running DB migrations..."
-	rake db:migrate
-	echo "DB migrations finished"
 
 	echo "starting loomio!"
-	/loomio/docker_start.sh
+	if [ "$TASK" = "worker" ]; then
+  		bundle exec sidekiq
+	else
+		sudo apt update -y && sudo apt install -y postgresql-client
+		bundle install
+
+		# running this code instaed of db:prepare in docker_start.sh in loomio container
+		# as postgres container creates empty db, somehow db:prepare cannot cope.
+		# therefore we run db:setup or db:migrate individually
+		if PGPASSWORD=$(cat /run/secrets/db_password) psql -U "$POSTGRES_USER" -h "$DB_HOST" -lqt | cut -d \| -f 1 | grep -wq "$POSTGRES_DB"; then
+			echo "database '$POSTGRES_DB' exists."
+
+			# check if the database contains tables
+			TABLE_COUNT=$(PGPASSWORD=$(cat /run/secrets/db_password) psql -U "$POSTGRES_USER" -h "$DB_HOST" -d "$POSTGRES_DB" -t -c "SELECT count(*) FROM information_schema.tables WHERE table_schema = 'public';")
+
+			if [ "$TABLE_COUNT" -eq 0 ]; then
+				echo "Database '$POSTGRES_DB' is empty, running db:setup."
+				bundle exec rake db:setup
+			else
+				echo "database '$POSTGRES_DB' not empty, running migrations."
+				bundle exec rake db:migrate
+			fi
+		else
+			echo "database '$POSTGRES_DB' does not exist, running db:setup."
+			bundle exec rake db:setup
+		fi
+
+		bundle exec puma -C config/puma.rb
+	fi
 fi

release/1.0.0+v2.25.2

@@ -1,9 +1,14 @@
 In this release the passwords for smtp and postgres DB were moved into docker secrets. Therefore a few manual steps need to be performed (also available in recipe documentation)
 
-* adapt your env file with the new vars, especially SECRET_DB_PASSWORD_VERSION=v1 and SECRET_SMTP_PASSWORD_VERSION=v1
+* adapt your env file with the new vars, especially `SECRET_DB_PASSWORD_VERSION=v1` and `SECRET_SMTP_PASSWORD_VERSION=v1` (remember, you can use `abra app check <app-name>` to check for any missing variables)
-* insert your smtp password with abra app secret insert <app-name> smtp_password v1 "<your-password>"
+* insert your existing smtp password with `abra app secret insert <app-name> smtp_password v1 "<your-password>"`
-* abra app secret generate <app-name> db_password v1
+
-* abra app deploy <app-name>
+Then, choose whether to keep the existing insecure database password (easy):
-* set the new password in DB: abra app cmd <app-name> db set_new_db_password
+* `abra app secret insert <app-name> db_password v1 password`
+
+Or, switch to a new secure password (harder, better):
+* `abra app secret generate <app-name> db_password v1`
+* `abra app deploy <app-name>`
+* set the new password in DB: `abra app cmd <app-name> db set_new_db_password` (NOTE: if you get "FATA loomio doesn't have a set_new_db_password function" here, run `cd ~/.abra/recipes/loomio && git checkout main`, then re-run the `abra app cmd` command with `-C` at the end)
 * redeploy
 

release/2.1.0+v2.25.3

@@ -0,0 +1 @@
+DB initialization issues with wrong migration should be fixed now. (But not really sure why)

release/3.0.0+v2.25.3

@@ -0,0 +1 @@
+ATTENTION: do not skip this relase, otherwise it will lead to database migration complications. This release fixes migration issues of 1.x and 2.x releases.

release/4.0.0+v2.25.3

@@ -0,0 +1 @@
+ATTENTION: Perform a deployment of version 3.0.0+v2.25.3 before upgrading to this version. Otherwise there will be complications with database migrations. This version upgrades the used postgres container to version 17 with auto-upgrade functionality.