docs: Adds troubleshooting section for fulltextsearch

Reviewed-on: coop-cloud/nextcloud#45

.drone.yml

@@ -3,7 +3,7 @@ kind: pipeline
 name: deploy to swarm-test.autonomic.zone
 steps:
   - name: deployment
-    image: decentral1se/stack-ssh-deploy:latest
+    image: git.coopcloud.tech/coop-cloud/stack-ssh-deploy:latest
     settings:
       host: swarm-test.autonomic.zone
       stack: nextcloud
@@ -11,15 +11,41 @@ steps:
       purge: true
       deploy_key:
         from_secret: drone_ssh_swarm_test
+      networks:
+        - proxy
     environment:
       DOMAIN: nextcloud.swarm-test.autonomic.zone
       STACK_NAME: nextcloud
       LETS_ENCRYPT_ENV: production
       ADMIN_USER: foobar
+      FPM_TUNE_VERSION: v1
+      NGINX_CONF_VERSION: v1
+      MY_CNF_VERSION: v1
+      ENTRYPOINT_VERSION: v1
+      CRONTAB_VERSION: v1
+      PG_BACKUP_VERSION: v2
       SECRET_DB_PASSWORD_VERSION: v1
       SECRET_DB_ROOT_PASSWORD_VERSION: v1
       SECRET_ADMIN_PASSWORD_VERSION: v1
+      SECRET_ONLYOFFICE_JWT_VERSION: v1
+      SECRET_BBB_SECRET_VERSION: v1
       EXTRA_VOLUME: "/dev/null:/tmp/.dummy"
 trigger:
   branch:
     - main
+---
+kind: pipeline
+name: generate recipe catalogue
+steps:
+  - name: release a new version
+    image: plugins/downstream
+    settings:
+      server: https://build.coopcloud.tech
+      token:
+        from_secret: drone_abra-bot_token
+      fork: true
+      repositories:
+        - toolshed/auto-recipes-catalogue-json
+
+trigger:
+  event: tag

.env.sample

@@ -1,6 +1,9 @@
 TYPE=nextcloud
+TIMEOUT=900
+ENABLE_AUTO_UPDATE=true
+ENABLE_BACKUPS=true
 
-DOMAIN={{ .Domain }}
+DOMAIN=nextcloud.example.com
 ## Domain aliases
 #EXTRA_DOMAINS=', `www.nextcloud.example.com`'
 LETS_ENCRYPT_ENV=production
@@ -9,6 +12,8 @@ COMPOSE_FILE="compose.yml"
 COMPOSE_FILE="$COMPOSE_FILE:compose.mariadb.yml"
 #COMPOSE_FILE="$COMPOSE_FILE:compose.postgres.yml"
 
+#MAX_DB_CONNECTIONS=500
+
 ADMIN_USER=admin
 
 SECRET_DB_ROOT_PASSWORD_VERSION=v1
@@ -17,7 +22,62 @@ SECRET_ADMIN_PASSWORD_VERSION=v1
 
 EXTRA_VOLUME=/dev/null:/tmp/.dummy
 
+PHP_MEMORY_LIMIT=1G
+# fpm-tune, see: https://spot13.com/pmcalculator/
+FPM_MAX_CHILDREN=16
+FPM_START_SERVERS=4
+FPM_MIN_SPARE_SERVERS=4
+FPM_MAX_SPARE_SERVERS=12
+
+DEFAULT_QUOTA="10 GB"
+
 # X_FRAME_OPTIONS_ENABLED=1
 # X_FRAME_OPTIONS_ALLOW_FROM=embedding-site.example.org
-# APPS="calendar sociallogin onlyoffice"
 
+# COMPOSE_FILE="$COMPOSE_FILE:compose.smtp.yml"
+# See https://github.com/nextcloud/docker#auto-configuration-via-environment-variables for default values
+# SMTP_AUTHTYPE=
+# SMTP_HOST=
+# SMTP_SECURE=
+# SMTP_NAME=
+# SMTP_PORT=
+# MAIL_FROM_ADDRESS=
+# MAIL_DOMAIN=
+# SECRET_SMTP_PASSWORD_VERSION=v1
+
+## Customization
+# THEMING_COLOR=
+# THEMING_SLOGAN=
+# COPY_ASSETS="flow_background.jpg|app:/var/www/html/themes/"
+# COPY_ASSETS="$COPY_ASSETS icon_left_brand.svg|app:/var/www/html/themes/"
+# COPY_ASSETS="$COPY_ASSETS icon.png|app:/var/www/html/themes/"
+
+# APPS="calendar"
+
+# COLLABORA_URL=https://collabora.example.com
+
+# COMPOSE_FILE="$COMPOSE_FILE:compose.onlyoffice.yml"
+# ONLYOFFICE_URL=https://onlyoffice.example.com
+# APPS="$APPS onlyoffice"
+# SECRET_ONLYOFFICE_JWT_VERSION=v1
+
+# COMPOSE_FILE="$COMPOSE_FILE:compose.bbb.yml"
+# BBB_URL=https://talk.example.org/bigbluebutton/ # trailing slash!
+# SECRET_BBB_SECRET_VERSION=v1
+
+# COMPOSE_FILE="$COMPOSE_FILE:compose.whiteboard.yml"
+# APPS="$APPS whiteboard"
+# SECRET_WHITEBOARD_JWT_VERSION=v1
+
+# COMPOSE_FILE="$COMPOSE_FILE:compose.authentik.yml"
+# APPS="$APPS sociallogin"
+# AUTHENTIK_USER_PREFIX=authentik
+# AUTHENTIK_DOMAIN=authentik.example.com
+# SECRET_AUTHENTIK_SECRET_VERSION=v1
+# SECRET_AUTHENTIK_ID_VERSION=v1
+# OCC_CMDS="app:disable dashboard"
+# OCC_CMDS="$OCC_CMDS|config:app:set sociallogin auto_create_groups --value 1"
+# OCC_CMDS="$OCC_CMDS|config:app:set sociallogin hide_default_login --value 1"
+
+#COMPOSE_FILE="$COMPOSE_FILE:compose.fulltextsearch.yml"
+#SECRET_ELASTICSEARCH_PASSWORD_VERSION=v1

README.md

@@ -6,38 +6,96 @@ Fully automated luxury Nextcloud via docker-swarm.
 
 <!-- metadata -->
 * **Category**: Apps
-* **Status**: 2, beta
+* **Status**: 5
 * **Image**: [`nextcloud`](https://hub.docker.com/_/nextcloud), 4, upstream
 * **Healthcheck**: Yes
-* **Backups**: No
+* **Backups**: Yes
 * **Email**: 3
 * **Tests**: 2
 * **SSO**: 1 (OAuth)
 <!-- endmetadata -->
 
-## Basic usage
+## Quick start
 
-1. Set up Docker Swarm and [`abra`]
+* `abra app new nextcloud`
-2. Deploy [`coop-cloud/traefik`]
+* `abra app config <app-name>`
-3. `abra app new nextcloud --secrets` (optionally with `--pass` if you'd like
+* `abra app secret insert <app-name> smtp_password v1 <SMTP_PASSWORD>`
-   to save secrets in `pass`)
+* `abra app secret generate -a <app-name>`
-4. `abra app YOURAPPDOMAIN config` - be sure to change `$DOMAIN` to something that resolves to
+* `abra app deploy <app-name>`
-   your Docker swarm box
-5. `abra app YOURAPPDOMAIN deploy`
 
-## How do I customise the default home page when logging in?
+### Onlyoffice Integration
 
-- Delete the dashboard app since it is so corporate
+`abra app config <app-name>` 
-- Follow [these docs](https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/default_files_configuration.html) to set the default files list for each user in the Files app
+Configure the following envs:
-- Configure a `defaultapp` in your `config.php` or use [apporder](https://apps.nextcloud.com/apps/apporder)
+```
+COMPOSE_FILE="$COMPOSE_FILE:compose.apps.yml"
+ONLYOFFICE_URL=https://onlyoffice.example.com
+SECRET_ONLYOFFICE_JWT_VERSION=v1
+```
+
+`abra app secret insert <app-name> onlyoffice_jwt v1 <jwt_secret>`
+`abra app cmd <app-name> app install_onlyoffice`
+
+### BBB Integration
+
+`abra app config <app-name>` 
+Configure the following envs:
+```
+COMPOSE_FILE="$COMPOSE_FILE:compose.apps.yml"
+BBB_URL=https://talk.example.org/bigbluebutton/ # trailing slash!
+SECRET_BBB_SECRET_VERSION=v1
+```
+
+`abra app secret insert <app-name> bbb_secret v1 <bbb_secret>`
+`abra app cmd <app-name> app install_bbb`
+
+### Authentik Integration
+
+
+`abra app config <app-name>` 
+Configure the following envs:
+```
+COMPOSE_FILE="$COMPOSE_FILE:compose.authentik.yml"
+AUTHENTIK_USER_PREFIX=authentik
+AUTHENTIK_DOMAIN=authentik.example.com
+AUTHENTIK_SECRET_NAME=authentik_example_com_nextcloud_secret_v1  # the same as in authentik
+AUTHENTIK_ID_NAME=authentik_example_com_nextcloud_id_v1  # the same as in authentik
+```
+
+`abra app cmd <app-name> app set_authentik`
+
+### Disable Dashboard
+
+Disable dashboard app since it is so corporate:
+
+`abra app config <app-name>` 
+Configure the following envs:
+```
+OCC_CMDS="app:disable dashboard"
+```
+`abra app cmd <app-name> app post_install_occ`
 
 ## Running `occ`
 
-`abra app run --user www-data YOURAPPDOMAIN app occ user:list --help`
+`abra app cmd <app-name> app run_occ '"user:list --help"'`
 
-## Upgrading Nextcloud apps
+## Default user files
+
+- Follow [these docs](https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/default_files_configuration.html) to set the default files list for each user in the Files app
+
+## Default App
+
+- Configure a `defaultapp` in your `config.php` or use [apporder](https://apps.nextcloud.com/apps/apporder)
+
+## Upgrading Nextcloud
+Upgrading Nextcloud can be a hair raising experiance. They [don't support downgrading](https://docs.nextcloud.com/server/latest/admin_manual/maintenance/upgrade.html) even for minor versions.
+
+Many of us  have found that jumping major versions when upgrading is also a bad idea. We have however found that it's ok to skip minor version upgrades and go to the last minor version before a major version (e.g. 24.0.0 to 24.9.9 before going to 25.0.0). To extra cautious just upgrade one release at a time. Read the release notes and check your logs.
+
+## Upgrading Nextcloud apps (plug-ins)
+
+`abra app cmd <app-name> app run_occ '"app:update --all"'`
 
-`abra app run --user www-data YOURAPPDOMAIN app occ app:update --all`
 
 ## How do I fix a Nextcloud version snafu?
 
@@ -66,7 +124,7 @@ Use [this plugin](https://github.com/pulsejet/nextcloud-oidc-login). Unlike the
 ```
   'oidc_login_client_id' => 'nextcloud',
   'oidc_login_client_secret' => 'mysecret',
-  'oidc_login_provider_url' => 'https://example.com/auth/realms/myrealm',
+  'oidc_login_provider_url' => 'https://example.com/realms/myrealm',
   'oidc_login_disable_registration' => false,
   'oidc_login_hide_password_form' => true,
   'oidc_login_button_text' => 'Log in with your myssodomain',
@@ -190,3 +248,49 @@ docker exec -u www-data $(docker ps -f name=foo_com_app -q) ./occ preview:pre-ge
 ```
 
 This app will improve performance of image browsing at the cost of storage space.
+
+## Fulltextsearch using elasticsearch
+
+1. Uncomment the following lines in your env file:
+```
+#COMPOSE_FILE="$COMPOSE_FILE:compose.fulltextsearch.yml"
+#SECRET_ELASTICSEARCH_PASSWORD_VERSION=v1
+```
+
+2. Generate the secret for elasticsearch:
+```bash
+abra app secret generate <domain> elasticsearch_password v1
+```
+
+3. Deploy your app:
+```bash
+abra app deploy <domain>
+```
+
+4. Install the apps and configure them:
+```
+abra app cmd <domain> app install_fulltextsearch
+```
+
+5. You might need to configure the files_fulltextsearch app. run this command to check its settings:
+```
+abra app cmd <domain> app run_occ '"config:list files_fulltextsearch"
+```
+
+6. You can check if the nextcloud can connect to elasticsearch:
+```
+abra app cmd <domain> app run_occ '"fulltextsearch:test"'
+```
+
+And you can populate the index manually and check if any errors occur:
+```
+abra app cmd <domain> app run_occ '"fulltextsearch:index"'
+```
+
+### Troubleshooting fulltextsearch
+
+The fulltextsearch plugin might be stuck with this error: "Index is already running". In that case the following command can get things runing again:
+
+```
+abra app run <domain> db /bin/sh -- -c 'echo "delete from oc_fulltextsearch_ticks;" | mariadb -u root -p$(cat /run/secrets/db_root_password) nextcloud'
+```

abra.sh

@@ -1,122 +1,172 @@
-export FPM_TUNE_VERSION=v4
+#!/bin/bash
-export NGINX_CONF_VERSION=v4
-export MY_CNF_VERSION=v4
-export ENTRYPOINT_VERSION=v2
 
-NC_APP_DIR="app:/var/www/html"
+export FPM_TUNE_VERSION=v5
+export NGINX_CONF_VERSION=v7
+export MY_CNF_VERSION=v5
+export ENTRYPOINT_VERSION=v3
+export ENTRYPOINT_WHITEBOARD_VERSION=v1
+export CRONTAB_VERSION=v1
+export PG_BACKUP_VERSION=v2
 
-sub_occ(){
+run_occ() {
-  # shellcheck disable=SC2034
-  abra__service_="app"
-  # shellcheck disable=SC2034
-  abra___user="www-data"
-  sub_app_run php /var/www/html/occ "$@"
-}
-
-run_occ(){
     su -p www-data -s /bin/sh -c "/var/www/html/occ $@"
 }
 
-install_apps(){
+post_install_occ() {
+    IFS='|' read -ra CMD <<<"$OCC_CMDS"
+    for cmd in "${CMD[@]}"; do
+        run_occ "$cmd"
+    done
+}
+
+install_apps() {
     install_apps="$@"
-    if [ -z "$install_apps" ]
+    if [ -z "$install_apps" ]; then
-    then
         install_apps=$APPS
     fi
-    for app in $install_apps
+    for app in $install_apps; do
-    do
         run_occ "app:install $app"
     done
 }
 
-_backup_app() {
+set_app_config() {
-  # Copied _abra_backup_dir to make UX better on restore and backup
+    APP=$1
-  {
+    KEY=$2
-    abra__src_="$1"
+    VALUE=$3
-    abra__dst_="-"
+    run_occ "config:app:set $APP $KEY --value '$VALUE'"
-  }
-
-  # shellcheck disable=SC2154
-  FILENAME="$(basename "$1").tar"
-
-  debug "Copying '$1' to '$FILENAME'"
-
-  silence
-  mkdir -p /tmp/abra
-  sub_app_cp > /tmp/abra/$FILENAME
-  unsilence
 }
 
-next_maintenance_on() {
+set_system_config() {
-  silence
+    KEY=$1
-  sub_occ maintenance:mode --on > /dev/null
+    VALUE=$2
-  unsilence
+    run_occ "config:system:set $KEY --value '$VALUE'"
-  debug "Nextcloud maintenance mode enabled"
 }
 
-next_maintenance_off() {
+set_trusted_proxies() {
-  silence
+    trusted_proxies="$@"
-  sub_occ maintenance:mode --off > /dev/null
+    if [ -z "$1" ]; then
-  unsilence
+        trusted_proxies="$TRUSTED_PROXIES"
-  debug "Nextcloud maintenance mode disabled"
+    fi
+    set_system_config trusted_proxies "$trusted_proxies"
 }
 
-abra_backup_app() {
+set_logfile_stdout() {
-  # shellcheck disable=SC2154
+    set_system_config logfile '/dev/stdout'
-  ARK_FILENAME="$ABRA_BACKUP_DIR/${abra__app_}_app_$(date +%F).tar.gz"
-  # Cant be FILENAME as that gets changed by something
-  next_maintenance_on
-  _backup_app $NC_APP_DIR/config
-  _backup_app $NC_APP_DIR/data
-  _backup_app $NC_APP_DIR/themes
-  # Combine archives
-  tar -Af /tmp/abra/config.tar /tmp/abra/data.tar
-  tar -Af /tmp/abra/config.tar /tmp/abra/themes.tar
-  gzip /tmp/abra/config.tar -c > "$ARK_FILENAME"
-  rm /tmp/abra/*.tar
-  success "Backed up 'app' to $ARK_FILENAME"
-  next_maintenance_off
 }
 
-abra_backup_db() {
+customize() {
-  next_maintenance_on
+    if [ -z "$1" ]
-  _abra_backup_mysql "db" "nextcloud"
+    then
-  next_maintenance_off
+            echo "Usage: ... customize <assets_path>"
+            exit 1
+    fi
+    asset_dir=$1
+    for asset in $COPY_ASSETS; do
+        source=$(echo $asset | cut -d "|" -f1)
+        target=$(echo $asset | cut -d "|" -f2)
+        echo copy $source to $target
+        abra app cp $APP_NAME $asset_dir/$source $target
+    done
+
+    abra app cmd -T $APP_NAME app set_app_config theming color \"$THEMING_COLOR\"
+    abra app cmd -T $APP_NAME app set_app_config theming slogan \"$THEMING_SLOGAN\"
+    abra app cmd -T $APP_NAME app run_occ '"theming:config background \"/var/www/html/themes/flow_background.jpg\""'
+    abra app cmd -T $APP_NAME app run_occ '"theming:config logo \"/var/www/html/themes/icon_left_brand.svg\""'
+    abra app cmd -T $APP_NAME app run_occ '"theming:config logoheader \"/var/www/html/themes/icon.png\""'
 }
 
-abra_backup() {
+install_bbb() {
-  abra_backup_app && abra_backup_db
+    install_apps bbb
+    set_app_config bbb app.navigation true
+    set_app_config bbb api.url "$BBB_URL"
+    set_app_config bbb api.secret "$(cat /run/secrets/bbb_secret)"
+}
+
+install_onlyoffice() {
+    install_apps onlyoffice
+    set_app_config onlyoffice DocumentServerUrl "$ONLYOFFICE_URL"
+    set_app_config onlyoffice jwt_secret "$(cat /run/secrets/onlyoffice_jwt)"
+    set_app_config onlyoffice customizationForcesave true
+}
+
+install_collabora() {
+    install_apps richdocuments
+    set_app_config richdocuments wopi_url "$COLLABORA_URL"
+}
+
+install_whiteboard() {
+    install_apps whiteboard
+    set_app_config whiteboard collabBackendUrl "https://${DOMAIN}/whiteboard"
+    set_app_config whiteboard jwt_secret_key "$(cat /run/secrets/whiteboard_jwt)"
 }
 
 
-abra_restore_app() {
+install_fulltextsearch() {
-  next_maintenance_on
+    install_apps fulltextsearch
-  # shellcheck disable=SC2034
+    install_apps fulltextsearch_elasticsearch
-  {
+    install_apps files_fulltextsearch
-  abra__src_="-"
+    set_app_config fulltextsearch search_platform "OCA\\FullTextSearch_Elasticsearch\\Platform\\ElasticSearchPlatform"
-  abra__dst_=$NC_APP_DIR
+    set_app_config fulltextsearch_elasticsearch elastic_host "http://elastic:$(cat /run/secrets/elasticsearch_password)@elasticsearch:9200/"
-  }
+    set_app_config fulltextsearch_elasticsearch elastic_index "nextcloud"
-
+    set_app_config files_fulltextsearch files_local "1"
-  zcat "$@" | sub_app_cp
-
-  next_maintenance_off
-  sub_occ files:scan --all > /dev/null # Needs to be run in normal mode
-  success "Restored 'app'"
 }
 
-# abra_restore_db() {
+set_default_quota() {
-#   warning "Restoring the database is on a existing app and not a new one has not been tested. Use with caution."
+    set_app_config files default_quota "$DEFAULT_QUOTA"
-#   next_maintenance_on
+}
-#   # 3wc: unlike abra_backup_db, we can assume abra__service_ will be 'db' if we
-#   # got this far..
 
-#   # shellcheck disable=SC2034
+set_authentik() {
-#   abra___no_tty="true"
+    install_apps sociallogin
+    AUTHENTIK_SECRET=$(cat /run/secrets/authentik_secret)
+    AUTHENTIK_ID=$(cat /run/secrets/authentik_id)
+    set_system_config logo_url https://$AUTHENTIK_DOMAIN
+    set_app_config sociallogin custom_providers "
+{
+    \"custom_oidc\":[
+    {
+        \"name\":\"$AUTHENTIK_USER_PREFIX\",
+        \"title\":\"authentik\",
+        \"authorizeUrl\": \"https://$AUTHENTIK_DOMAIN/application/o/authorize/\",
+        \"tokenUrl\": \"https://$AUTHENTIK_DOMAIN/application/o/token/\",
+        \"displayNameClaim\":\"preferred_username\",
+        \"userInfoUrl\": \"https://$AUTHENTIK_DOMAIN/application/o/userinfo/\",
+        \"logoutUrl\": \"https://$AUTHENTIK_DOMAIN/application/o/nextcloud/end-session/\",
+        \"clientId\":\"$AUTHENTIK_ID\",
+        \"clientSecret\":\"$AUTHENTIK_SECRET\",
+        \"scope\":\"openid profile email nextcloud\",
+        \"groupsClaim\":\"nextcloud_groups\",
+        \"style\":\"openid\",
+        \"defaultGroup\":\"\",
+        \"groupMapping\": {
+          \"admin\": \"admin\",
+          \"authentik Admins\": \"admin\"
+        }
+    }
+]
+}"
 
-#   DB_PASSWORD=$(sub_app_run cat /run/secrets/db_password)
+    set_app_config sociallogin update_profile_on_login 1
+    set_app_config sociallogin auto_create_groups 1
+    set_app_config sociallogin hide_default_login 1
+    run_occ 'config:system:set social_login_auto_redirect --value true'
+    run_occ 'config:system:set allow_user_to_change_display_name --value=false'
+    run_occ 'config:system:set lost_password_link --value=disabled'
+}
 
-#   zcat "$@" | sub_app_run mysql -u root -p"$DB_PASSWORD" wordpress
+disable_skeletondirectory() {
+    run_occ "config:system:set skeletondirectory --value ''"
+}
 
-#   success "Restored 'db'"
+set_windowsfriendly_filenames() {
-#   next_maintenance_off
+    run_occ 'config:system:set forbidden_filename_characters 0 --value=?'
-# }
+    run_occ 'config:system:set forbidden_filename_characters 1 --value=\<'
+    run_occ 'config:system:set forbidden_filename_characters 2 --value=\>'
+    run_occ 'config:system:set forbidden_filename_characters 3 --value=:'
+    run_occ 'config:system:set forbidden_filename_characters 4 --value=*'
+    run_occ 'config:system:set forbidden_filename_characters 5 --value=\|'
+    run_occ 'config:system:set forbidden_filename_characters 6 --value=\"'
+}
+
+upgrade_mariadb() {
+    mariadb-upgrade -p`cat /run/secrets/db_root_password`
+}

alaconnect.yml

@@ -0,0 +1,24 @@
+authentik:
+    uncomment:
+        - compose.authentik.yml
+        - AUTHENTIK_USER_PREFIX
+        - AUTHENTIK_DOMAIN
+        - SECRET_AUTHENTIK_SECRET_VERSION
+        - SECRET_AUTHENTIK_ID_VERSION
+    initial-hooks:
+        - app set_authentik
+    shared_secrets:
+        nextcloud_secret: authentik_secret
+        nextcloud_id: authentik_id
+onlyoffice:
+    uncomment:
+        - compose.onlyoffice.yml
+        - ONLYOFFICE_URL
+        - SECRET_ONLYOFFICE_JWT_VERSION
+    initial-hooks:
+        - app install_onlyoffice
+collabora:
+    uncomment:
+        - COLLABORA_URL
+    initial-hooks:
+        - app install_collabora

compose.authentik.yml

@@ -0,0 +1,14 @@
+version: "3.8"
+services:
+  app:
+    secrets:
+      - authentik_secret
+      - authentik_id
+
+secrets:
+  authentik_secret:
+    external: true
+    name: ${STACK_NAME}_authentik_secret_${SECRET_AUTHENTIK_SECRET_VERSION}
+  authentik_id:
+    external: true
+    name: ${STACK_NAME}_authentik_id_${SECRET_AUTHENTIK_ID_VERSION}

compose.bbb.yml

@@ -0,0 +1,12 @@
+version: "3.8"
+services:
+  app:
+    secrets:
+      - bbb_secret
+    environment:
+      - BBB_URL
+
+secrets:
+  bbb_secret:
+    external: true
+    name: ${STACK_NAME}_bbb_secret_${SECRET_BBB_SECRET_VERSION}

compose.fulltextsearch.yml

@@ -0,0 +1,55 @@
+version: "3.8"
+
+services:
+  elasticsearch:
+    image: "docker.elastic.co/elasticsearch/elasticsearch:8.17.2"
+    environment:
+      - cluster.name=docker-cluster
+      - bootstrap.memory_lock=true
+      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+      - discovery.type=single-node
+      # Disable authentication and ssl completely
+      # - xpack.security.enabled=false
+      # Use this to enable Basic Authentication:
+      - xpack.security.enabled=true
+      - xpack.security.http.ssl.enabled=false
+      - ELASTIC_PASSWORD_FILE=/var/run/secrets/elasticsearch_password
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+    volumes:
+      - elasticsearch:/usr/share/elasticsearch/data
+    networks:
+      - internal
+    secrets:
+      - source: elasticsearch_password
+        uid: "1000"
+        gid: "1000"
+        mode: 0600
+
+  searchindexer:
+    image: nextcloud:30.0.6-fpm
+    volumes:
+      - nextcloud:/var/www/html/
+      - nextapps:/var/www/html/custom_apps:cached
+      - nextdata:/var/www/html/data:cached
+      - nextconfig:/var/www/html/config:cached
+      - ${EXTRA_VOLUME}
+    networks:
+      - internal
+    entrypoint: su -p www-data -s /bin/sh -c '/var/www/html/occ fulltextsearch:live'
+
+  # Add the secret to the app service so it is avaiable in the
+  # install_fulltextsearch command
+  app:
+    secrets:
+      - elasticsearch_password
+
+secrets:
+  elasticsearch_password:
+    external: true
+    name: ${STACK_NAME}_elasticsearch_password_${SECRET_ELASTICSEARCH_PASSWORD_VERSION}
+
+volumes:
+  elasticsearch:

compose.mariadb.yml

@@ -9,12 +9,13 @@ services:
       - MYSQL_PASSWORD_FILE=/run/secrets/db_password
 
   db:
-    image: "mariadb:10.5"
+    image: "mariadb:11.4"
     environment:
       - MYSQL_DATABASE=nextcloud
       - MYSQL_USER=nextcloud
       - MYSQL_PASSWORD_FILE=/run/secrets/db_password
       - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db_root_password
+      - MAX_DB_CONNECTIONS=${MAX_DB_CONNECTIONS:-100}
     configs:
       - source: my_tune
         target: /etc/mysql/conf.d/my-tune.cnf
@@ -27,14 +28,25 @@ services:
       - internal
     deploy:
       labels:
-          backupbot.backup: "true"
+        backupbot.backup.pre-hook: 'mariadb-dump --single-transaction -u root -p"$$(cat /run/secrets/db_root_password)" nextcloud > /var/lib/mysql/backup.sql'
-          backupbot.backup.pre-hook: 'mkdir -p /tmp/backup/ && mysqldump --single-transaction -u root -p"$$(cat /run/secrets/db_root_password)" nextcloud > /tmp/backup/backup.sql'
+        backupbot.backup.volumes.mariadb.path: "backup.sql"
-          backupbot.backup.post-hook: "rm -rf /tmp/backup"
+        backupbot.restore.post-hook: 'mariadb -u root -p"$$(cat /run/secrets/db_root_password)" nextcloud < /var/lib/mysql/backup.sql'
-          backupbot.backup.path: "/tmp/backup/"
+    healthcheck:
+      test: ["CMD-SHELL", 'mariadb-admin -p"$$(cat /run/secrets/db_root_password)"  ping']
+      interval: 30s
+      timeout: 10s
+      retries: 10
+      start_period: 1m
 configs:
   my_tune:
     name: ${STACK_NAME}_my_cnf_${MY_CNF_VERSION}
     file: my-tune.cnf
+    template_driver: golang
+
+secrets:
+  db_root_password:
+    external: true
+    name: ${STACK_NAME}_db_root_password_${SECRET_DB_ROOT_PASSWORD_VERSION}
 
 volumes:
   mariadb:

compose.onlyoffice.yml

@@ -0,0 +1,12 @@
+version: "3.8"
+services:
+  app:
+    secrets:
+      - onlyoffice_jwt
+    environment:
+      - ONLYOFFICE_URL
+
+secrets:
+  onlyoffice_jwt:
+    external: true
+    name: ${STACK_NAME}_onlyoffice_jwt_${SECRET_ONLYOFFICE_JWT_VERSION}

compose.postgres.yml

@@ -10,7 +10,8 @@ services:
       - NEXTCLOUD_UPDATE=1
 
   db:
-    image: "postgres:12"
+    image: "postgres:13"
+    command: -c "max_connections=${MAX_DB_CONNECTIONS:-100}"
     volumes:
       - "postgres:/var/lib/postgresql/data"
     networks:
@@ -22,16 +23,24 @@ services:
     secrets:
       - db_password
     healthcheck:
-      test: ["CMD-SHELL", "pg_isready"]
+      test: ["CMD-SHELL", "pg_isready", "-U", "nextcloud"]
       interval: 10s
       timeout: 5s
       retries: 5
     deploy:
       labels:
-            backupbot.backup: "true"
+        backupbot.backup.pre-hook: "/pg_backup.sh backup"
-            backupbot.backup.pre-hook: "mkdir -p /tmp/backup/ && PGPASSWORD=$$(cat $${POSTGRES_PASSWORD_FILE}) pg_dump -U $${POSTGRES_USER} $${POSTGRES_DB} > /tmp/backup/backup.sql"
+        backupbot.backup.volumes.postgres.path: "backup.sql"
-            backupbot.backup.post-hook: "rm -rf /tmp/backup"
+        backupbot.restore.post-hook: '/pg_backup.sh restore'
-            backupbot.backup.path: "/tmp/backup/"
+    configs:
+        - source: pg_backup
+          target: /pg_backup.sh
+          mode: 0555
 
 volumes:
   postgres:
+
+configs:
+  pg_backup:
+    name: ${STACK_NAME}_pg_backup_${PG_BACKUP_VERSION}
+    file: pg_backup.sh

compose.smtp.yml

@@ -0,0 +1,19 @@
+version: "3.8"
+services:
+  app:
+    secrets:
+      - smtp_password
+    environment:
+      - SMTP_AUTHTYPE
+      - SMTP_HOST
+      - SMTP_SECURE
+      - SMTP_NAME
+      - SMTP_PORT
+      - SMTP_PASSWORD_FILE=/run/secrets/smtp_password
+      - MAIL_FROM_ADDRESS
+      - MAIL_DOMAIN
+
+secrets:
+  smtp_password:
+    external: true
+    name: ${STACK_NAME}_smtp_password_${SECRET_SMTP_PASSWORD_VERSION}

compose.whiteboard.yml

@@ -0,0 +1,44 @@
+version: "3.8"
+
+services:
+  app:
+    secrets:
+      - whiteboard_jwt
+
+  whiteboard:
+    image: ghcr.io/nextcloud-releases/whiteboard:v1.0.5
+    deploy:
+      labels:
+        - traefik.enable=true
+        - traefik.docker.network=proxy
+        - traefik.http.services.${STACK_NAME}_whiteboard.loadbalancer.server.port=3002
+        - traefik.http.routers.${STACK_NAME}_whiteboard.rule=Host(`${DOMAIN}`${EXTRA_DOMAINS}) && PathPrefix(`/whiteboard`)
+        - traefik.http.routers.${STACK_NAME}_whiteboard.entrypoints=web-secure
+        - traefik.http.routers.${STACK_NAME}_whiteboard.tls.certresolver=${LETS_ENCRYPT_ENV}
+        - traefik.http.middlewares.${STACK_NAME}_whiteboard-stripprefix.stripprefix.prefixes=/whiteboard
+        - traefik.http.routers.${STACK_NAME}_whiteboard.middlewares=${STACK_NAME}_whiteboard-stripprefix
+    configs:
+      - source: entrypoint_whiteboard
+        target: /custom-entrypoint.sh
+    entrypoint: ["sh", "/custom-entrypoint.sh"]
+    user: root
+    networks:
+     - proxy
+    ports:
+      - 3002:3002
+    secrets:
+      - whiteboard_jwt
+    environment:
+      - NEXTCLOUD_URL=https://$DOMAIN
+      - JWT_SECRET_KEY_FILE=/run/secrets/whiteboard_jwt
+
+secrets:
+  whiteboard_jwt:
+    external: true
+    name: ${STACK_NAME}_whiteboard_jwt_${SECRET_WHITEBOARD_JWT_VERSION}
+
+configs:
+  entrypoint_whiteboard:
+    name: ${STACK_NAME}_entrypoint_whiteboard_${ENTRYPOINT_WHITEBOARD_VERSION}
+    file: entrypoint.whiteboard.sh.tmpl
+    template_driver: golang

compose.yml

@@ -1,7 +1,9 @@
 version: "3.8"
 services:
   web:
-    image: nginx:1.23.1
+    image: nginx:1.27.4
+    depends_on:
+      - app
     configs:
       - source: nginx_conf
         target: /etc/nginx/nginx.conf
@@ -33,14 +35,23 @@ services:
         - "traefik.http.routers.${STACK_NAME}.middlewares=${STACK_NAME}-redirect"
         - "traefik.http.middlewares.${STACK_NAME}-redirect.headers.SSLForceHost=true"
         - "traefik.http.middlewares.${STACK_NAME}-redirect.headers.SSLHost=${DOMAIN}"
+        - "caddy=${DOMAIN}"
+        - "caddy.reverse_proxy={{upstreams 80}}"
+        - "caddy.tls.on_demand="
+    healthcheck:
+      test: ["CMD-SHELL", 'curl -s -N curl -Ns localhost/status.php |  grep "installed\":true"']
+      interval: 30s
+      timeout: 10s
+      retries: 10
+      start_period: 5m
 
   app:
-    image: nextcloud:24.0.6-fpm
+    image: nextcloud:30.0.6-fpm
     depends_on:
       - db
     configs:
       - source: fpm_tune
-        target: /usr/local/etc/php-fpm.d/fpm-tune.conf
+        target: /usr/local/etc/php-fpm.d/zzz-fpm-tune.conf
       - source: entrypoint
         target: /custom-entrypoint.sh
         mode: 555
@@ -50,6 +61,7 @@ services:
       - admin_password
     environment:
       - APPS
+      - OCC_CMDS
       - X_FRAME_OPTIONS_ALLOW_FROM
       - X_FRAME_OPTIONS_ENABLED
       - DOMAIN
@@ -57,14 +69,15 @@ services:
       - NEXTCLOUD_ADMIN_USER=${ADMIN_USER}
       - NEXTCLOUD_ADMIN_PASSWORD_FILE=/run/secrets/admin_password
       - NEXTCLOUD_TRUSTED_DOMAINS=${DOMAIN}
-      - TRUSTED_PROXIES=traefik
+      - TRUSTED_PROXIES=10.0.0.0/8
       - REDIS_HOST=cache
-      - SMTP_HOST
-      - MAIL_FROM_ADDRESS
-      - MAIL_DOMAIN
-      - SMTP_AUTHTYPE=PLAIN
       - OVERWRITEPROTOCOL=https
-      - PHP_MEMORY_LIMIT=1G
+      - PHP_MEMORY_LIMIT=${PHP_MEMORY_LIMIT:-1G}
+      - FPM_MAX_CHILDREN=${FPM_MAX_CHILDREN:-131}
+      - FPM_START_SERVERS=${FPM_START_SERVERS:-32}
+      - FPM_MIN_SPARE_SERVERS=${FPM_MIN_SPARE_SERVERS:-32}
+      - FPM_MAX_SPARE_SERVERS=${FPM_MAX_SPARE_SERVERS:-98}
+      - DEFAULT_QUOTA
     volumes:
       - nextcloud:/var/www/html/
       - nextapps:/var/www/html/custom_apps:cached
@@ -78,12 +91,21 @@ services:
         failure_action: rollback
         order: start-first
       labels:
-        - "coop-cloud.${STACK_NAME}.version=2.1.4+24.0.6-fpm"
+        - "coop-cloud.${STACK_NAME}.version=11.3.0+30.0.6-fpm"
-        - "backupbot.backup=true"
+        - "coop-cloud.${STACK_NAME}.timeout=${TIMEOUT:-120}"
-        - "backupbot.backup.path=/var/www/html/config/,/var/www/html/data/,/var/www/html/custom_apps/"
+        - "backupbot.backup=${ENABLE_BACKUPS:-true}"
+        - "backupbot.backup.volumes.redis=false"
+       #- "backupbot.backup.volumes.nextcloud=false"
+
+    healthcheck:
+      test: ["CMD-SHELL", 'SCRIPT_NAME=status SCRIPT_FILENAME=/var/www/html/status.php REQUEST_METHOD=GET cgi-fcgi -bind -connect 127.0.0.1:9000 | grep "installed\":true"']
+      interval: 30s
+      timeout: 10s
+      retries: 10
+      start_period: 15m
 
   cron:
-    image: nextcloud:24.0.6-fpm
+    image: nextcloud:30.0.6-fpm
     volumes:
       - nextcloud:/var/www/html/
       - nextapps:/var/www/html/custom_apps:cached
@@ -93,21 +115,27 @@ services:
     networks:
       - internal
     entrypoint: /cron.sh
+    configs:
+      - source: crontab
+        target: /var/spool/cron/crontabs/www-data
+
 
   cache:
-    image: redis:7.0.5-alpine
+    image: redis:7.4.2-alpine
     networks:
       - internal
     volumes:
       - "redis:/data"
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 3s
+      timeout: 5s
+      retries: 20
 
 secrets:
-  db_root_password:
-    external: true
-    name: ${STACK_NAME}_db_root_password_${SECRET_DB_ROOT_PASSWORD_VERSION}
   db_password:
     external: true
-    name: ${STACK_NAME}_db_password_${SECRET_DB_ROOT_PASSWORD_VERSION}
+    name: ${STACK_NAME}_db_password_${SECRET_DB_PASSWORD_VERSION}
   admin_password:
     external: true
     name: ${STACK_NAME}_admin_password_${SECRET_ADMIN_PASSWORD_VERSION}
@@ -128,10 +156,14 @@ configs:
   fpm_tune:
     name: ${STACK_NAME}_fpm_tune_${FPM_TUNE_VERSION}
     file: fpm-tune.ini
+    template_driver: golang
   entrypoint:
     name: ${STACK_NAME}_entrypoint_${ENTRYPOINT_VERSION}
     file: entrypoint.sh.tmpl
     template_driver: golang
+  crontab:
+    name: ${STACK_NAME}_crontab_${CRONTAB_VERSION}
+    file: crontab
 
 networks:
   proxy:

crontab

@@ -0,0 +1 @@
+*/5 * * * * php -d memory_limit=1G -f /var/www/html/cron.php

entrypoint.sh.tmpl

@@ -1,5 +1,30 @@
 #!/bin/bash
 
+set -eu
+
+file_env() {
+  local var="$1"
+  local fileVar="${var}_FILE"
+  local def="${2:-}"
+
+  if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
+    echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
+    exit 1
+  fi
+
+  local val="$def"
+  if [ "${!var:-}" ]; then
+    val="${!var}"
+  elif [ "${!fileVar:-}" ]; then
+    val="$(< "${!fileVar}")"
+  fi
+
+  export "$var"="$val"
+  unset "$fileVar"
+}
+
+file_env "SMTP_PASSWORD"
+
 echo "Giving the db container some time to come up"; sleep 20
 # see this issue with postgres db https://github.com/nextcloud/docker/issues/1204
 
@@ -9,4 +34,8 @@ if ! [[ $(grep {{ env "X_FRAME_OPTIONS_ALLOW_FROM" }} lib/public/AppFramework/Ht
 fi
 {{ end }}
 
+# Required for healthcheck
+which cgi-fcgi > /dev/null || (apt-get update && apt-get install -y libfcgi-bin)
+
+
 /entrypoint.sh php-fpm

entrypoint.whiteboard.sh.tmpl

@@ -0,0 +1,6 @@
+#!/bin/sh
+set -e
+
+export JWT_SECRET_KEY=$(cat /run/secrets/whiteboard_jwt)
+
+exec npm run server:start

fpm-tune.ini

@@ -1,5 +1,5 @@
 pm = dynamic
-pm.max_children = 131
+pm.max_children = {{ env "FPM_MAX_CHILDREN" }}
-pm.start_servers = 32
+pm.start_servers = {{ env "FPM_START_SERVERS" }}
-pm.min_spare_servers = 32
+pm.min_spare_servers = {{ env "FPM_MIN_SPARE_SERVERS" }}
-pm.max_spare_servers = 98
+pm.max_spare_servers = {{ env "FPM_MAX_SPARE_SERVERS" }}

my-tune.cnf

@@ -13,7 +13,7 @@ key_buffer_size                = 16M
 innodb_log_file_size           = 256M
 long_query_time                = 1
 max_allowed_packet             = 256M
-max_connections                = 100
+max_connections                = {{ env "MAX_DB_CONNECTIONS" }}
 max_heap_table_size            = 64M
 max_user_connections           = 0
 myisam_recover_options         = BACKUP

nginx.conf.tmpl

@@ -11,6 +11,10 @@ events {
 
 http {
     include       /etc/nginx/mime.types;
+    # See https://github.com/nextcloud/forms/issues/1838#issuecomment-1860497200
+    types {
+        application/javascript js mjs;
+    }
     default_type  application/octet-stream;
 
     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
@@ -59,12 +63,12 @@ http {
         #pagespeed off;
 
         # HTTP response headers borrowed from Nextcloud `.htaccess`
-        add_header Referrer-Policy                      "no-referrer"   always;
+        add_header Referrer-Policy                      "no-referrer"       always;
-        add_header X-Content-Type-Options               "nosniff"       always;
+        add_header X-Content-Type-Options               "nosniff"           always;
-        add_header X-Download-Options                   "noopen"        always;
+        add_header X-Download-Options                   "noopen"            always;
-        add_header X-Permitted-Cross-Domain-Policies    "none"          always;
+        add_header X-Permitted-Cross-Domain-Policies    "none"              always;
-        add_header X-Robots-Tag                         "none"          always;
+        add_header X-Robots-Tag                         "noindex, nofollow" always;
-        add_header X-XSS-Protection                     "1; mode=block" always;
+        add_header X-XSS-Protection                     "1; mode=block"     always;
 
         {{ if eq (env "X_FRAME_OPTIONS_ENABLED") "1" }}
         add_header Content-Security-Policy              "frame-ancestors {{ env "X_FRAME_OPTIONS_ALLOW_FROM" }} {{ env "DOMAIN" }}";
@@ -132,6 +136,9 @@ http {
         # then Nginx will encounter an infinite rewriting loop when it prepends `/index.php`
         # to the URI, resulting in a HTTP 500 error response.
         location ~ \.php(?:$|/) {
+            # Required for legacy support
+            rewrite ^/(?!index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|ocs-provider\/.+|.+\/richdocumentscode(_arm64)?\/proxy) /index.php$request_uri;
+
             fastcgi_split_path_info ^(.+?\.php)(/.*)$;
             set $path_info $fastcgi_path_info;
 

pg_backup.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+set -e
+
+BACKUP_FILE='/var/lib/postgresql/data/backup.sql'
+
+function backup {
+  export PGPASSWORD=$(cat /run/secrets/db_password)
+  pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} > $BACKUP_FILE
+}
+
+function restore {
+    cd /var/lib/postgresql/data/
+    restore_config(){
+        # Restore allowed connections
+        cat pg_hba.conf.bak > pg_hba.conf
+        su postgres -c 'pg_ctl reload'
+    }
+    # Don't allow any other connections than local
+    cp pg_hba.conf pg_hba.conf.bak
+    echo "local all all trust" > pg_hba.conf
+    su postgres -c 'pg_ctl reload'
+    trap restore_config EXIT INT TERM
+
+    # Recreate Database
+    psql -U ${POSTGRES_USER} -d postgres -c "DROP DATABASE ${POSTGRES_DB} WITH (FORCE);" 
+    createdb -U ${POSTGRES_USER} ${POSTGRES_DB}
+    psql -U ${POSTGRES_USER} -d ${POSTGRES_DB} -1 -f $BACKUP_FILE
+
+    trap - EXIT INT TERM
+    restore_config
+}
+
+$@

release/10.0.0+30.0.4-fpm

@@ -0,0 +1 @@
+https://docs.nextcloud.com/server/latest/admin_manual/release_notes/upgrade_to_30.html

release/11.0.0+30.0.4-fpm

@@ -0,0 +1,4 @@
+Upgrades mariadb from 10.5 to 11.4
+NOTE: If your Nextcloud instance is using mariadb, after running this update you MUST run the database upgrade command:
+`abra app command nextcloud.yourserver.org db upgrade_mariadb`
+More info: https://mariadb.com/kb/en/upgrading-from-mariadb-10-11-to-mariadb-11-4/

release/3.1.0+25.0.1-fpm

@@ -0,0 +1,57 @@
+
+## FPM Tune
+
+The fpm-tune.ini settings are now configurable by `.env`. Please add this to your servers configs:
+
+```
+# fpm-tune, see: https://spot13.com/pmcalculator/
+FPM_MAX_CHILDREN=131
+FPM_START_SERVERS=32
+FPM_MIN_SPARE_SERVERS=32
+FPM_MAX_SPARE_SERVERS=98
+```
+
+## SMTP
+
+Add SMTP Config to your .env file:
+
+```
+# COMPOSE_FILE="$COMPOSE_FILE:compose.smtp.yml"
+# See https://github.com/nextcloud/docker#auto-configuration-via-environment-variables for default values
+# SMTP_AUTHTYPE=
+# SMTP_HOST=
+# SMTP_SECURE=
+# SMTP_NAME=
+# SMTP_PORT=
+# MAIL_FROM_ADDRESS=
+# MAIL_DOMAIN=
+# SECRET_SMTP_PASSWORD_VERSION=v1
+abra app secret insert example.com smtp_password v1 example_password
+```
+
+
+## Post Deploy Commands
+
+Some Apps can also be managed with abra app cmd!
+
+```
+# COMPOSE_FILE="$COMPOSE_FILE:compose.apps.yml"
+# APPS="calendar sociallogin onlyoffice"
+abra app cmd example.com app install_apps
+# ONLYOFFICE_URL=https://onlyoffice.example.com
+# SECRET_ONLYOFFICE_JWT_VERSION=v1
+abra app secret insert example.com onlyoffice_jwt v1 example_password
+abra app cmd example.com app install_onlyoffice
+# BBB_URL=https://talk.example.org/bigbluebutton/ # trailing slash!
+# SECRET_BBB_SECRET_VERSION=v1
+abra app secret insert example.com bbb_secret v1 example_password
+abra app cmd example.com app install_bbb
+```
+
+## Set Quota
+
+```
+# DEFAULT_QUOTA="10 GB"
+abra app cmd example.com app set_default_quota
+```
+

release/3.2.0+25.0.4-fpm

@@ -0,0 +1,11 @@
+If the authentik configuration should be handled by abra add the following to the env:
+
+    COMPOSE_FILE="$COMPOSE_FILE:compose.authentik.yml"
+    AUTHENTIK_USER_PREFIX=authentik
+    AUTHENTIK_DOMAIN=authentik.example.com
+    AUTHENTIK_SECRET_NAME=authentik_example_com_nextcloud_secret_v1  # the same as in authentik
+    AUTHENTIK_ID_NAME=authentik_example_com_nextcloud_id_v1  # the same as in authentik
+
+And run:
+
+    abra app cmd <app-name> app set_authentik

release/5.0.1+27.0.1-fpm

@@ -0,0 +1 @@
+The authentik secrets need to be inserted again, as nextcloud is not sharing the secret with authentik any more.

release/8.0.0+29.0.1-fpm

@@ -0,0 +1 @@
+BREAKING CHANGE: compose.apps.yml is now split for bbb and onlyoffice, configs must be updated

release/9.1.0+29.0.5-fpm

@@ -0,0 +1 @@
+Added automated customization options. Config needs to be updated to be able to use it.