diff --git a/README.md b/README.md index 41af243..0ae0333 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ Fully automated luxury Nextcloud via docker-swarm. -* **Maintainer**: [@dannygroenewegen](https://git.coopcloud.tech/dannygroenewegen), [@ineiti](https://git.coopcloud.tech/ineiti) +* **Maintainer**: [@dannygroenewegen](https://git.coopcloud.tech/dannygroenewegen), [@ineiti](https://git.coopcloud.tech/ineiti), [@javielico](https://git.coopcloud.tech/javielico) * **Category**: Apps * **Status**: 5 * **Image**: [`nextcloud`](https://hub.docker.com/_/nextcloud), 4, upstream @@ -144,9 +144,49 @@ To disable dashboard app (since it is so corporate): - Configure a `defaultapp` in your `config.php` or use [apporder](https://apps.nextcloud.com/apps/apporder) ## Upgrading Nextcloud -Upgrading Nextcloud can be a hair raising experiance. They [don't support downgrading](https://docs.nextcloud.com/server/latest/admin_manual/maintenance/upgrade.html) even for minor versions. -Many of us have found that jumping major versions when upgrading is also a bad idea. We have however found that it's ok to skip minor version upgrades and go to the last minor version before a major version (e.g. 24.0.0 to 24.9.9 before going to 25.0.0). To extra cautious just upgrade one release at a time. Read the release notes and check your logs. +Upgrading Nextcloud can be a hair raising experiance. They +[don't support downgrading](https://docs.nextcloud.com/server/latest/admin_manual/maintenance/upgrade.html) +even for minor versions. + +### Upgrade path + +Many of us have found that jumping major versions when upgrading is a bad idea. +We have however found that it's ok to skip minor version upgrades and go to the +last minor version before a major version (e.g. 24.0.0 to 24.9.9 before going to +25.0.0). To be extra cautious just upgrade one release at a time. Read the +release notes and check your logs. + +### Checking upgrade readiness + +Before upgrading to a new major, check whether the instance (and its enabled +apps) look ready: + +`abra app cmd app check_major_upgrade` + +This checks that you're not skipping a major version, that there's no pending DB +upgrade left over from a previous update, and that every enabled non-shipped app +declares support for the target major (falling back to an apps.nextcloud.com +lookup to see if an app update would fix it). Shipped apps (`files`, `settings`, +etc.) are skipped since they come bundled and are upgraded within the Docker +image. Pass an explicit target major as the first argument (e.g. +`check_major_upgrade 33`) to check readiness for a specific major, or it +defaults to current major + 1. This is a sanity check, not a guarantee. Still +read Nextcloud's release notes for any +[critical changes between major versions](https://docs.nextcloud.com/server/stable/admin_manual/release_notes/index.html#critical-changes). + +### Staying on an old major version + +If you're not able to move to a new major version yet (e.g. because of app +incompatibility), note that (starting from v32) before we release a recipe +container a new major Nextcloud version, we also publish one more release of the +previous major that points its image at the floating `nextcloud:XX-fpm` tag +(e.g. `nextcloud:32-fpm`) instead of a pinned patch version. Deploying that +release is less predictable: every redeploy pulls whatever the latest `32-fpm` +build happens to be at that moment, rather than a fixed, reproducible version. +But it means you keep getting security patches for the old major if you can't +move to the next major. See [MAINTENANCE.md](./MAINTENANCE.md#release-cadence) +for how this fits into our release process. ## Upgrading Nextcloud apps (plug-ins) diff --git a/abra.sh b/abra.sh index af39f71..f201ab7 100644 --- a/abra.sh +++ b/abra.sh @@ -10,7 +10,8 @@ export CRONTAB_VERSION=v1 export PG_BACKUP_VERSION=v2 run_occ() { - su -p www-data -s /bin/sh -c "/var/www/html/occ $@" + # NOTE: uses $* (not $@) so this still works when called with multiple args as seperate words. + su -p www-data -s /bin/sh -c "/var/www/html/occ $*" } install_apps() { @@ -193,3 +194,159 @@ set_windowsfriendly_filenames() { upgrade_mariadb() { mariadb-upgrade -p`cat /run/secrets/db_root_password` } + +# Checks whether this instance looks ready to update to the next Nextcloud +# major version. +# +# Usage: +# abra app cmd app check_major_upgrade +# abra app cmd app check_major_upgrade 33 # check readiness for a specific target +# +# What it checks: +# - current version is exactly one major behind the target +# - no pending DB upgrade from a previous, unfinished update +# - whether a newer release is available on the current major +# (recommended before upgradeing to the next major) +# - every enabled, non-shipped app's compatibility with the target major +# - for apps that don't, whether apps.nextcloud.com already has a newer +# release that does +# +# It does NOT check every precondition, always read the release notes +# from Nextcloud too. +check_major_upgrade() { + target_major=$1 + + echo "=== Nextcloud major upgrade readiness check ===" + + status_json=$(run_occ status --output=json 2>/dev/null) + if [ -z "$status_json" ]; then + echo "[FAIL] Could not read 'occ status' - is Nextcloud installed and reachable?" + return 1 + fi + + current_version=$(echo "$status_json" | php -r '$d=json_decode(stream_get_contents(STDIN),true); echo $d["versionstring"] ?? "";') + current_major=${current_version%%.*} + needs_db_upgrade=$(echo "$status_json" | php -r '$d=json_decode(stream_get_contents(STDIN),true); echo ($d["needsDbUpgrade"] ?? false) ? "true" : "false";') + + if [ -z "$current_major" ]; then + echo "[FAIL] Could not determine the current Nextcloud version from 'occ status'." + return 1 + fi + + if [ -z "$target_major" ]; then + target_major=$((current_major + 1)) + fi + + echo "Current version: $current_version" + echo "Target major version: $target_major" + + ok=true + + if [ "$target_major" -le "$current_major" ]; then + echo "[FAIL] Target major ($target_major) is not newer than the current major ($current_major)." + ok=false + elif [ "$target_major" -gt "$((current_major + 1))" ]; then + echo "[FAIL] Cannot skip major versions. Upgrade to $((current_major + 1)) first." + ok=false + fi + + if [ "$needs_db_upgrade" = "true" ]; then + echo "[FAIL] A pending database upgrade was detected. Run 'occ upgrade' for the current version first." + ok=false + fi + + echo + echo "--- occ update:check ---" + update_check_output=$(run_occ "update:check" 2>&1) + if [ -z "$update_check_output" ]; then + echo "[WARN] 'occ update:check' produced no output, could not verify." + elif echo "$update_check_output" | grep -q "Everything up to date"; then + echo "[OK] Everything up to date." + else + available_version=$(echo "$update_check_output" | grep -oE 'Nextcloud [0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+)?' | head -n1 | awk '{print $2}') + available_major=${available_version%%.*} + if [ -z "$available_major" ]; then + echo "[WARN] Could not parse 'occ update:check' output to determine the available version." + elif [ "$available_major" = "$current_major" ]; then + echo "[WARN] $available_version is available on the current major. Recommended to update to that before upgrading to $target_major." + else + echo "[OK] Already on the latest release of major $current_major (next available update is $available_version)." + fi + fi + echo + + echo "--- Non-shipped app compatibility with Nextcloud $target_major ---" + echo "(shipped apps are skipped, they come bundled with the docker image)" + apps_json=$(run_occ "app:list --shipped=false --enabled --output=json" 2>/dev/null) + + if [ -z "$apps_json" ]; then + echo "[WARN] 'occ app:list' returned no output, could not check non-shipped app compatibility." + enabled_apps="" + else + apps_json_valid=$(echo "$apps_json" | php -r '$d=json_decode(stream_get_contents(STDIN),true); echo (json_last_error() === JSON_ERROR_NONE && is_array($d)) ? "1" : "0";') + if [ "$apps_json_valid" != "1" ]; then + echo "[WARN] Could not parse 'occ app:list' output, could not check non-shipped app compatibility." + enabled_apps="" + else + enabled_apps=$(echo "$apps_json" | php -r '$d=json_decode(stream_get_contents(STDIN),true); foreach(array_keys($d["enabled"] ?? []) as $a) echo $a."\n";') + if [ -z "$enabled_apps" ]; then + echo "No non-shipped apps are enabled - nothing to check here." + fi + fi + fi + + compatible_apps="" + compatible_apps_fetched=0 + + for app in $enabled_apps; do + info_file=$(find /var/www/html/apps /var/www/html/custom_apps -maxdepth 3 -type f -ipath "*/$app/appinfo/info.xml" 2>/dev/null | head -n1) + + if [ -z "$info_file" ]; then + echo "[WARN] $app: could not locate appinfo/info.xml, skipping" + continue + fi + + max_version=$(php -r ' + $x = @simplexml_load_file($argv[1]); + $dep = $x ? ($x->dependencies->nextcloud ?? null) : null; + echo $dep !== null ? (string)$dep["max-version"] : ""; + ' "$info_file") + + if [ -z "$max_version" ]; then + echo "[WARN] $app: no max-version declared in info.xml, assume compatible but verify manually" + continue + fi + + if [ "${max_version%%.*}" -ge "$target_major" ] 2>/dev/null; then + echo "[OK] $app: installed version supports up to Nextcloud $max_version" + continue + fi + + echo "[INFO] $app: installed version only supports up to Nextcloud $max_version" + + if [ "$compatible_apps_fetched" != "1" ]; then + compatible_apps_fetched=1 + compatible_apps=$(curl -fsSL --max-time 30 "https://apps.nextcloud.com/api/v1/platform/${target_major}.0.0/apps.json" 2>/dev/null \ + | php -r '$d=json_decode(stream_get_contents(STDIN),true); if(is_array($d)) foreach($d as $a) echo $a["id"]."\n";') + fi + + if [ -z "$compatible_apps" ]; then + echo "[FAIL] $app: could not reach apps.nextcloud.com to check for a newer compatible release, verify manually" + ok=false + elif echo "$compatible_apps" | grep -qxF "$app"; then + echo "[WARN] $app: apps.nextcloud.com has a release that supports $target_major. It may not update until Nextcloud is upgraded, occ upgrade will try to update it automatically" + else + echo "[FAIL] $app: no apps.nextcloud.com release supports $target_major yet, it will be disabled during the upgrade" + ok=false + fi + done + + echo + if [ "$ok" = true ]; then + echo "=== READY: no blocking issues found for upgrade to major $target_major ===" + return 0 + else + echo "=== NOT READY: resolve the [FAIL] items above before running the upgrade ===" + return 1 + fi +} diff --git a/compose.fulltextsearch.yml b/compose.fulltextsearch.yml index 3ecc656..1e19d42 100644 --- a/compose.fulltextsearch.yml +++ b/compose.fulltextsearch.yml @@ -2,7 +2,7 @@ version: "3.8" services: elasticsearch: - image: "docker.elastic.co/elasticsearch/elasticsearch:8.19.16" + image: "docker.elastic.co/elasticsearch/elasticsearch:8.19.20" environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true @@ -29,7 +29,7 @@ services: mode: 0600 searchindexer: - image: nextcloud:32.0.12-fpm + image: nextcloud:33.0.8-fpm volumes: - nextcloud:/var/www/html/ - nextapps:/var/www/html/custom_apps:cached diff --git a/compose.mariadb.yml b/compose.mariadb.yml index 5cb08bc..94557af 100644 --- a/compose.mariadb.yml +++ b/compose.mariadb.yml @@ -9,7 +9,7 @@ services: - MYSQL_PASSWORD_FILE=/run/secrets/db_password db: - image: "mariadb:11.4" + image: "mariadb:12.3" environment: - MYSQL_DATABASE=nextcloud - MYSQL_USER=nextcloud diff --git a/compose.postgres.yml b/compose.postgres.yml index cee110c..b9c581a 100644 --- a/compose.postgres.yml +++ b/compose.postgres.yml @@ -10,8 +10,9 @@ services: - NEXTCLOUD_UPDATE=1 db: - image: "pgautoupgrade/pgautoupgrade:14-debian" - command: -c "max_connections=${MAX_DB_CONNECTIONS:-100}" + image: "pgautoupgrade/pgautoupgrade:17-debian" + #setting max_connections with -c breaks pgautoupgrade + #command: -c "max_connections=${MAX_DB_CONNECTIONS:-100}" volumes: - "postgres:/var/lib/postgresql/data" networks: @@ -22,11 +23,11 @@ services: POSTGRES_DB: nextcloud secrets: - db_password - healthcheck: - test: ["CMD-SHELL", "pg_isready", "-U", "nextcloud"] - interval: 10s - timeout: 5s - retries: 5 + # The pgautoupgrade image already ships its own HEALTHCHECK. + # This runs pg_isready but also takes into accounts if pg_upgrade is being run. + # No need to override it here. + # healthcheck: + # test: ["CMD", "/usr/local/bin/pgautoupgrade-healthcheck.sh"] deploy: labels: backupbot.backup.pre-hook: "/pg_backup.sh backup" diff --git a/compose.yml b/compose.yml index d416a7d..5a25c76 100644 --- a/compose.yml +++ b/compose.yml @@ -1,7 +1,7 @@ version: "3.8" services: web: - image: nginx:1.31.1 + image: nginx:1.31.3 depends_on: - app configs: @@ -48,7 +48,7 @@ services: start_period: 5m app: - image: nextcloud:32.0.12-fpm + image: nextcloud:33.0.8-fpm depends_on: - db configs: @@ -95,7 +95,7 @@ services: failure_action: rollback order: start-first labels: - - "coop-cloud.${STACK_NAME}.version=13.1.0+32.0.11-fpm" + - "coop-cloud.${STACK_NAME}.version=14.1.1+33.0.8-fpm" - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT}" - "backupbot.backup=${ENABLE_BACKUPS:-true}" - "backupbot.backup.volumes.redis=false" @@ -109,7 +109,7 @@ services: start_period: 15m cron: - image: nextcloud:32.0.12-fpm + image: nextcloud:33.0.8-fpm volumes: - nextcloud:/var/www/html/ - nextapps:/var/www/html/custom_apps:cached @@ -125,7 +125,7 @@ services: cache: - image: redis:8.8.0-alpine + image: redis:8.8.1-alpine networks: - internal volumes: @@ -154,7 +154,7 @@ volumes: configs: nginx_conf: - name: ${STACK_NAME}_nginx_${NGINX_CONF_VERSION} + name: ${STACK_NAME}_nginx_conf_${NGINX_CONF_VERSION} file: nginx.conf.tmpl template_driver: golang fpm_tune: diff --git a/release/13.1.4+32.0.13-fpm b/release/13.1.4+32.0.13-fpm new file mode 100644 index 0000000..5081da3 --- /dev/null +++ b/release/13.1.4+32.0.13-fpm @@ -0,0 +1,4 @@ + chore(deps): update mariadb docker tag to v12 + chore(deps): update mariadb docker tag to v11.8 + chore(deps): update nginx docker tag to v1.31.3 + chore(deps): update docker.elastic.co/elasticsearch/elasticsearch docker tag to v8.19.19 diff --git a/release/13.1.5+32.0.13-fpm b/release/13.1.5+32.0.13-fpm new file mode 100644 index 0000000..d71cfdc --- /dev/null +++ b/release/13.1.5+32.0.13-fpm @@ -0,0 +1,2 @@ +Important: +The pgautoupgrade switch from 13.1.0+32.0.11-fpm was released untested and had a bug: the db service's `-c max_connections=...` command breaks pgautoupgrade, failing with `initdb: invalid option -- 'c'`. If your db got stuck mid-upgrade on 13.1.0-13.1.4 with that error: restore the old data dir and remove the upgrade lock file. Then redeploy this version. diff --git a/release/13.2.0+32-fpm b/release/13.2.0+32-fpm new file mode 100644 index 0000000..6fcc63a --- /dev/null +++ b/release/13.2.0+32-fpm @@ -0,0 +1 @@ +Last release for nextcloud-32, pointing to the latest version of nextcloud-32. diff --git a/release/14.0.0+33.0.7-fpm b/release/14.0.0+33.0.7-fpm new file mode 100644 index 0000000..e104a8f --- /dev/null +++ b/release/14.0.0+33.0.7-fpm @@ -0,0 +1,13 @@ +Upgrades Nextcloud from 32.0.13 to 33 (major version upgrade). + +IMPORTANT: +- Do not skip major versions: your instance must be on the latest 32.x before + upgrading to 33. If you are on an older 32.x, deploy 32.0.13 first. +- Added `check_major_upgrade` (`abra app cmd app check_major_upgrade`) + to check whether an instance is ready to upgrade to the next Nextcloud + major version. +- Nextcloud does NOT support downgrades. Take a backup before deploying. +- After deploying, check the logs and run any pending repair/upgrade steps: + `abra app cmd app run_occ '"app:update --all"'` +- Review app (plug-in) compatibility with Nextcloud 33 before upgrading; some + apps may need to be updated or temporarily disabled. diff --git a/release/14.1.0+33.0.7-fpm b/release/14.1.0+33.0.7-fpm new file mode 100644 index 0000000..10a29c1 --- /dev/null +++ b/release/14.1.0+33.0.7-fpm @@ -0,0 +1 @@ +Update pgautoupgrade to postgresql v17 \ No newline at end of file