coop-cloud/organising
coop-cloud
/
organising
7
0
Fork 0
Code Issues 157 Pull Requests Projects 2 Releases Activity

How do we manage versions of apps? #34

New Issue
Closed
opened 2020-10-26 17:59:31 +00:00 by decentral1se · 12 comments
decentral1se commented 2020-10-26 17:59:31 +00:00
Owner
Copy Link

Now that https://git.autonomic.zone/coop-cloud/organising/issues/33 is out, I'd like to see that we have a version field in the package.yml or whatever it will be. How do we deal with that?

Do we maintain our own version number? Or do we rely on the version of the docker image? We'll probably be hacking away furiously on the app configs, so I think maintaining our own semver makes sense. Then somehow tagging the stack when deployed? Abra can then do a compare later to check if an upgrade is needed?

Thoughts?

Now that https://git.autonomic.zone/coop-cloud/organising/issues/33 is out, I'd like to see that we have a version field in the package.yml or whatever it will be. How do we deal with that? Do we maintain our own version number? Or do we rely on the version of the docker image? We'll probably be hacking away furiously on the app configs, so I think maintaining our own semver makes sense. Then somehow tagging the stack when deployed? Abra can then do a compare later to check if an upgrade is needed? Thoughts?
3wordchant commented 2020-10-27 00:24:40 +00:00
Owner
Copy Link

Do we maintain our own version number? Or do we rely on the version of the docker image?

Pro-posal: ${UPSTREAM_VERSION}_${OUR_VERSION}, e.g. Wordpress 5.5.1_0.1? Fine with just using our own version tho.

Then somehow tagging the stack when deployed? Abra can then do a compare later to check if an upgrade is needed?

Yeah good plan, unsure the best way to do store stack-level metadata, will research unless you get there first.

> Do we maintain our own version number? Or do we rely on the version of the docker image? Pro-posal: `${UPSTREAM_VERSION}_${OUR_VERSION}`, e.g. Wordpress `5.5.1_0.1`? Fine with just using our own version tho. > Then somehow tagging the stack when deployed? Abra can then do a compare later to check if an upgrade is needed? Yeah good plan, unsure the best way to do store stack-level metadata, will research unless you get there first.
decentral1se added this to the Public mini-launch milestone 2020-10-27 08:04:48 +00:00
decentral1se added the
design
 label 2020-10-27 08:09:14 +00:00
decentral1se added this to the (deleted) project 2020-12-27 11:47:57 +00:00
3wordchant commented 2021-02-03 16:20:57 +00:00
Owner
Copy Link

We ran into this with TSA: coop-cloud/nextcloud had updated so we ended up with an unexpected upgrade when we ran abra app nextcloud.third... deploy. This also highlighted that we didn't get notified of the available Nextcloud update before.

We ran into this with TSA: coop-cloud/nextcloud had updated so we ended up with an unexpected upgrade when we ran `abra app nextcloud.third... deploy`. This also highlighted that we didn't get notified of the available Nextcloud update before.
👍 1
decentral1se commented 2021-02-17 13:34:09 +00:00
Author
Owner
Copy Link

I think we should manually add the image tag to the abra.sh for the app version. So, for mediawiki, we've add export APP_VERSION=1.35.1. Then abra can look that up and compare with the deployed container image tag and show a difference. We can also just use the latest commit of the app repo too to keep things simple. So, when you ask for a version of an app, you get the image tag and the commit.

I think we should manually add the image tag to the `abra.sh` for the app version. So, for mediawiki, we've add `export APP_VERSION=1.35.1`. Then abra can look that up and compare with the deployed container image tag and show a difference. We can also just use the latest commit of the app repo too to keep things simple. So, when you ask for a version of an app, you get the image tag and the commit.
decentral1se self-assigned this 2021-02-17 13:34:57 +00:00
decentral1se commented 2021-02-17 16:38:43 +00:00
Author
Owner
Copy Link

Also, I have a suspicion that the images are getting overwritten by some upstreams. So, it would be good know which hash as well as the image tag we are deploying. Sometimes you think you are deploying the same tag but the upstream has overwritten it and things can break. We need to warn in those cases.

Also, I have a suspicion that the images are getting overwritten by some upstreams. So, it would be good know which hash as well as the image tag we are deploying. Sometimes you think you are deploying the same tag but the upstream has overwritten it and things can break. We need to warn in those cases.
decentral1se commented 2021-02-17 22:43:55 +00:00
Author
Owner
Copy Link

Some notes from the chat:

  • I've been thinking about it. wondering if whacking ABRA_TYPE_VERSION=<upstream_tag>_<short upstream hash>_<our version> might be a good start?

  • we could use https://docs.docker.com/engine/reference/commandline/tag/ to create that scheme when deploying? Then pull it apart when we want to show it on the CLI via abra maybe enough to have deploy make a version check with an existing tagged container and current state to-be-deployed and then make a warning. The warning could be chill if we see some commit bump in the app or pretty critical when we see a major upgrade on the image tag

I could get the two values out fairly easily:

  • App repo latest commit: git rev-parse --short HEAD
  • Image SHA: docker image inspect $(docker image ls -q -f reference=mediawiki) --format "{{ .Config.Image }}"

But as for the upstream tag, it seems that when I run docker image ls, none of my local images have tags! Some of them do, but mostly not. I guess that isn't something we can rely on then. It seems to be sometimes available from the running container but we won't always have a running container. We might have to manually specify this then ourselves.

(I haven't really looked into this before so if anyone sees a way, please shout)

Some notes from the chat: - I've been thinking about it. wondering if whacking `ABRA_TYPE_VERSION=<upstream_tag>_<short upstream hash>_<our version>` might be a good start? - we could use https://docs.docker.com/engine/reference/commandline/tag/ to create that scheme when deploying? Then pull it apart when we want to show it on the CLI via abra maybe enough to have deploy make a version check with an existing tagged container and current state to-be-deployed and then make a warning. The warning could be chill if we see some commit bump in the app or pretty critical when we see a major upgrade on the image tag I could get the two values out fairly easily: - App repo latest commit: `git rev-parse --short HEAD` - Image SHA: `docker image inspect $(docker image ls -q -f reference=mediawiki) --format "{{ .Config.Image }}"` But as for the upstream tag, it seems that when I run `docker image ls`, none of my local images have tags! Some of them do, but mostly not. I guess that isn't something we can rely on then. It seems to be sometimes available from the running container but we won't always have a running container. We might have to manually specify this then ourselves. (I haven't really looked into this before so if anyone sees a way, please shout)
decentral1se commented 2021-02-17 22:55:44 +00:00
Author
Owner
Copy Link

Thinking to maybe just manually specify everything on the image for now! Probably the most reliable. So, we might have a diff like this for the Gitea app then.

diff --git a/abra.sh b/abra.sh
index f29b389..12500db 100644
--- a/abra.sh
+++ b/abra.sh
@@ -1 +1,5 @@
+export ABRA_TYPE_TAG=1.35.1
+export ABRA_TYPE_DIGEST=2f0fdcf
+export ABRA_TYPE_COMMIT=$(git rev-parse --short HEAD)
+
 export APP_INI_VERSION=v2

Which I ripped from https://hub.docker.com/layers/gitea/gitea/1.13.2/images/sha256-2f0fdcfc5131f390cf88dad3ea8e1c98432e4514963383dae6964b98196ffada?context=explore.

Thinking to maybe just manually specify everything on the image for now! Probably the most reliable. So, we might have a diff like this for the Gitea app then. ```diff diff --git a/abra.sh b/abra.sh index f29b389..12500db 100644 --- a/abra.sh +++ b/abra.sh @@ -1 +1,5 @@ +export ABRA_TYPE_TAG=1.35.1 +export ABRA_TYPE_DIGEST=2f0fdcf +export ABRA_TYPE_COMMIT=$(git rev-parse --short HEAD) + export APP_INI_VERSION=v2 ``` Which I ripped from https://hub.docker.com/layers/gitea/gitea/1.13.2/images/sha256-2f0fdcfc5131f390cf88dad3ea8e1c98432e4514963383dae6964b98196ffada?context=explore.
❤️ 1
decentral1se commented 2021-02-18 11:23:58 +00:00
Author
Owner
Copy Link

My only doubt at this point is that with the COMMIT value, the end-user can't tell if we're doing a breaking update from the actual app repo config files. For example, if we change an env var name or something. So, I am thinking of doing semver for that also. Which will be manually controlled also.

I think each app will need a changelog as well, which is fine. So the end-user will see two versions, the upstream tag and repo semver. We'll tuck the digest of the image into the implementation details and only do warnings when we suspect something.

I'll try to implement this for the Gitea repo is complicated enough to test it.

My only doubt at this point is that with the `COMMIT` value, the end-user can't tell if we're doing a breaking update from the actual app repo config files. For example, if we change an env var name or something. So, I am thinking of doing semver for that also. Which will be manually controlled also. I think each app will need a changelog as well, which is fine. So the end-user will see two versions, the upstream tag and repo semver. We'll tuck the digest of the image into the implementation details and only do warnings when we suspect something. I'll try to implement this for the Gitea repo is complicated enough to test it.
decentral1se commented 2021-02-24 11:11:24 +00:00
Author
Owner
Copy Link

Looking at YunoHost again, I see they just use the actual shipped app version as their own version so maybe we should just do that also because they have thousands of users and that seems to work for them. See ee18542f4b/manifest.json (L9).

I'll just go with the tag and the digest then. For breaking changes in the app repo we can just rely on the change log and document this heavily.

Looking at YunoHost again, I see they just use the actual shipped app version as their own version so maybe we should just do that also because they have thousands of users and that seems to work for them. See https://github.com/YunoHost-Apps/nextcloud_ynh/blob/ee18542f4bc7dd888e7f8bfbae92d9e076c27ccb/manifest.json#L9. I'll just go with the tag and the digest then. For breaking changes in the app repo we can just rely on the change log and document this heavily.
decentral1se commented 2021-02-24 11:25:49 +00:00
Author
Owner
Copy Link

I seem to have misunderstood tags, we need to instead use labels to add metadata on the containers https://docs.docker.com/config/labels-custom-metadata/. We can deploy the stack and then run a docker service update --label-add on the _app service to mark it. We apparently need to use some reverse DNS convention like coop-cloud.gitea.1.35.1.2f0fdcf where it goes like <project>.<app-type>.<app-version>.<app-digest>

I seem to have misunderstood tags, we need to instead use labels to add metadata on the containers https://docs.docker.com/config/labels-custom-metadata/. We can deploy the stack and then run a `docker service update --label-add` on the ` _app` service to mark it. We apparently need to use some reverse DNS convention like `coop-cloud.gitea.1.35.1.2f0fdcf` where it goes like `<project>.<app-type>.<app-version>.<app-digest>`
decentral1se removed this from the Public mini-launch milestone 2021-03-02 09:15:35 +00:00
decentral1se added this to the (deleted) milestone 2021-03-02 09:20:38 +00:00
decentral1se commented 2021-03-02 10:46:05 +00:00
Author
Owner
Copy Link

Progress! So, we can just use the existing label mechanism in the compose file.

c145440355/compose.yml (L59)

Then, when you deploy that app, you can run:

$ docker service inspect gitea_app

And this will show a whole bunch of stuff, but mainly:

        "Spec": {
            "Name": "gitea_app",
            "Labels": {
                "com.docker.stack.image": "gitea/gitea:1.13.2",
                "com.docker.stack.namespace": "gitea",
                "coop-cloud.gitea.version": "1.35.1.2f0fdcf",
                "traefik.enable": "true",
                "traefik.http.routers.gitea.entrypoints": "web-secure",
                "traefik.http.routers.gitea.rule": "Host(`git.autonomic.zone`)",
                "traefik.http.routers.gitea.tls.certresolver": "production",
                "traefik.http.services.gitea.loadbalancer.server.port": "3000",
                "traefik.tcp.routers.gitea-ssh.entrypoints": "gitea-ssh",
                "traefik.tcp.routers.gitea-ssh.rule": "HostSNI(`*`)",
                "traefik.tcp.services.gitea-ssh.loadbalancer.server.port": "2222"
            }

Where "coop-cloud.gitea.version": "1.35.1.2f0fdcf" is what we need.

You can get that out directly with the following:

$ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.version" }}' gitea_app

🚀

Progress! So, we can just use the existing label mechanism in the compose file. > https://git.autonomic.zone/coop-cloud/gitea/src/commit/c145440355f5cfe27bec4dd2d6065d5908dce05a/compose.yml#L59 Then, when you deploy that app, you can run: ```bash $ docker service inspect gitea_app ``` And this will show a whole bunch of stuff, but mainly: ```json "Spec": { "Name": "gitea_app", "Labels": { "com.docker.stack.image": "gitea/gitea:1.13.2", "com.docker.stack.namespace": "gitea", "coop-cloud.gitea.version": "1.35.1.2f0fdcf", "traefik.enable": "true", "traefik.http.routers.gitea.entrypoints": "web-secure", "traefik.http.routers.gitea.rule": "Host(`git.autonomic.zone`)", "traefik.http.routers.gitea.tls.certresolver": "production", "traefik.http.services.gitea.loadbalancer.server.port": "3000", "traefik.tcp.routers.gitea-ssh.entrypoints": "gitea-ssh", "traefik.tcp.routers.gitea-ssh.rule": "HostSNI(`*`)", "traefik.tcp.services.gitea-ssh.loadbalancer.server.port": "2222" } ``` Where `"coop-cloud.gitea.version": "1.35.1.2f0fdcf"` is what we need. You can get that out directly with the following: ```bash $ docker service inspect -f '{{index .Spec.Labels "coop-cloud.gitea.version" }}' gitea_app ``` 🚀
❤️ 1
decentral1se commented 2021-03-02 12:07:51 +00:00
Author
Owner
Copy Link

OK, so, I think the answer to this is basically do c145440355/compose.yml (L59) for all the apps and that is how we will do versioning! I'm gonna open up tickets for further support for this abra and close this off.

OK, so, I think the answer to this is basically do https://git.autonomic.zone/coop-cloud/gitea/src/commit/c145440355f5cfe27bec4dd2d6065d5908dce05a/compose.yml#L59 for all the apps and that is how we will do versioning! I'm gonna open up tickets for further support for this `abra` and close this off.
❤️ 1
3wordchant commented 2021-03-03 13:57:12 +00:00
Owner
Copy Link

If any future travellers get stuck at the same place I did -- to add ABRA_TYPE_VERSION and ABRA_TYPE_DIGEST:

  1. Use the upstream tag for ABRA_TYPE_VERSION
  2. On a server where an app is deployed (or via DOCKER_CONTEXT):

docker image ls
docker image inspect <imageid> -f "{{ .Id }}"

Then use the first 8 characters of that hash.

If any future travellers get stuck at the same place I did -- to add `ABRA_TYPE_VERSION` and `ABRA_TYPE_DIGEST`: 1. Use the upstream tag for `ABRA_TYPE_VERSION` 2. On a server where an app is deployed (or via `DOCKER_CONTEXT`): `docker image ls` `docker image inspect <imageid> -f "{{ .Id }}"` Then use the first 8 characters of that hash.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
master
Labels
Clear labels   
abra

Everything to do with abra

  
abra-gandi

Abra Gandi plugin

  
awaiting-feedback

Ping/pong on comms

  
backups

   
bug

Something is not working

  
build

Go build related issues

  
ci/cd

Getting the robots into the mix

  
community organising

Opening this thing up

  
contributing

Contributors stuff

  
coopcloud.tech

Our main website

  
democracy

Democratic decision making

  
design

Design thinking required

  
documentation

Let's write things together

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
finance

Money things

  
funding

Anything related to grant funding

  
good first issue

Easy start with development

  
help wanted

Need some help

  
installer

Installation related issues

  
kadabra

   
performance

Performance related

  
proposal

Large change which requires feedback & decisin making

  
question

More information is needed

  
recipes.coopcloud.tech

Our recipes catalogue

  
security

Securing our shit

  
test

Unit or integration test suite

  
wontfix

This won't be fixed

No Label
abra
abra-gandi
awaiting-feedback
backups
bug
build
ci/cd
community organising
contributing
coopcloud.tech
democracy
design
documentation
duplicate
enhancement
finance
funding
good first issue
help wanted
installer
kadabra
performance
proposal
question
recipes.coopcloud.tech
security
test
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
decentral1se
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: coop-cloud/organising#34
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?