toolshed/recipes-catalogue-json
1
0
Fork 0
Code Issues 1 Pull Requests Projects Releases Activity

Reinstate automatic catalogue generation #4

New Issue
Closed
opened 2022-11-13 06:20:43 +00:00 by 3wordchant · 11 comments
3wordchant commented 2022-11-13 06:20:43 +00:00
Owner
Copy Link

Ideally the catalogue should be regenerated automatically.

Wired: cronjob like we used to have
Inspired: webhook whenever there's a new released version of a recipe

Ideally the catalogue should be regenerated automatically. Wired: cronjob like we used to have Inspired: webhook whenever there's a new released version of a recipe
👍 1
3wordchant commented 2023-01-17 21:41:28 +00:00
Author
Owner
Copy Link

Current status:

TODO:

  • Why are commits to recipes-catalogue-json showing up as from me? (Example). Drone is authenticating using a key added to the abra-bot repo, shouldn't it show up as abra-bot? Or maybe Drone is picking up git "author" settings from the source repo (in this case, auto-recipes-catalogue-json)
  • Why is Drone failing to look up digests for a bunch of the recipes? In the same example, collabora (and many other recipes) have switched from a digest to "unknown". Or maybe this isn't too critical, as the image digests seem to be changing upstream anyway 🙄
  • How do we trigger the auto-recipes-catalogue-json build whenever there's a new release of a recipe? Ideally it'd be nice to avoid needing to add to .drone.yml in all the recipe repos; maybe a global webhook linked up to something? (n8n? ducks). This might become easier if all the recipes are in a separate organisation, see coop-cloud/organising#377.
Current status: - We have a docker image for abra again, [`git.coopcloud.tech/coop-cloud/abra`](https://git.coopcloud.tech/coop-cloud/-/packages/container/abra/dev), automatically published to `dev` tag on push (and should make tagged releases whenever we push a tag) - [The `drone-abra` Drone plugin](https://git.coopcloud.tech/coop-cloud/drone-abra/) is up-to-date as well (still needs moving to git.coopcloud.tech from Docker Hub tho) - `auto-apps-json` is [renamed to `auto-recipes-catalogue-json`](https://git.coopcloud.tech/coop-cloud/auto-recipes-catalogue-json/), and configured to do a Drone build to generate and publish the catalogue ([example](https://build.coopcloud.tech/coop-cloud/auto-recipes-catalogue-json/80/1/2)) -- it's currently pushing to [a `testing` branch in `recipes-catalogue-json`](https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/src/branch/testing). TODO: - Why are commits to `recipes-catalogue-json` showing up as from me? ([Example](https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/commit/6be5e7ecd556ac3c0ca6b38041dfd1f4ed5be09a)). Drone is authenticating using a key added to the `abra-bot` repo, shouldn't it show up as `abra-bot`? Or maybe Drone is picking up git "author" settings from the source repo (in this case, `auto-recipes-catalogue-json`) - Why is Drone failing to look up digests for a bunch of the recipes? In [the same example](https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/commit/6be5e7ecd556ac3c0ca6b38041dfd1f4ed5be09a), `collabora` (and many other recipes) have switched from a digest to `"unknown"`. Or maybe this isn't too critical, as the image digests seem to be changing upstream anyway 🙄 - How do we trigger the `auto-recipes-catalogue-json` build whenever there's a new release of a recipe? Ideally it'd be nice to avoid needing to add to `.drone.yml` in all the recipe repos; maybe a global webhook linked up to something? (`n8n`? _ducks_). This might become easier if all the recipes are in a separate organisation, see coop-cloud/organising#377.
mayel commented 2023-01-17 21:50:21 +00:00
Copy Link

Why are commits to recipes-catalogue-json showing up as from me? (Example). Drone is authenticating using a key added to the abra-bot repo, shouldn't it show up as abra-bot? Or maybe Drone is picking up git "author" settings from the source repo (in this case, auto-recipes-catalogue-json)

Did you create the key while authenticated as yourself? Typically you have to create a seperate user and create a key once logged in with it

> Why are commits to recipes-catalogue-json showing up as from me? (Example). Drone is authenticating using a key added to the abra-bot repo, shouldn't it show up as abra-bot? Or maybe Drone is picking up git "author" settings from the source repo (in this case, auto-recipes-catalogue-json) Did you create the key while authenticated as yourself? Typically you have to create a seperate user and create a key once logged in with it
3wordchant commented 2023-01-17 21:55:26 +00:00
Author
Owner
Copy Link

thanks for the reply @mayel!

Did you create the key while authenticated as yourself? Typically you have to create a seperate user and create a key once logged in with it

I generated a new SSH key using ssh-keygen and added it to the abra-bot user on Gitea.

thanks for the reply @mayel! > Did you create the key while authenticated as yourself? Typically you have to create a seperate user and create a key once logged in with it I generated a new SSH key using `ssh-keygen` and added it to the `abra-bot` user on Gitea.
moritz commented 2023-01-17 23:08:38 +00:00
Member
Copy Link
  • Why is Drone failing to look up digests for a bunch of the recipes? In the same example, collabora (and many other recipes) have switched from a digest to "unknown". Or maybe this isn't too critical, as the image digests seem to be changing upstream anyway 🙄

I think this is related to coop-cloud/organising#368 and it is caused by a rate limit.

> - Why is Drone failing to look up digests for a bunch of the recipes? In [the same example](https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/commit/6be5e7ecd556ac3c0ca6b38041dfd1f4ed5be09a), `collabora` (and many other recipes) have switched from a digest to `"unknown"`. Or maybe this isn't too critical, as the image digests seem to be changing upstream anyway 🙄 I think this is related to https://git.coopcloud.tech/coop-cloud/organising/issues/368 and it is caused by a rate limit.
3wordchant commented 2023-01-17 23:24:10 +00:00
Author
Owner
Copy Link

Thanks @moritz! I thought the docker login step would fix that, but perhaps it only gets us a higher rate limit, which still isn't high enough to look up all the images in the catalogue?

I agree with the comment by @decentral1se on Matrix, though:

great stuff 👏 one passing by comment is that i'm thinking we should maybe abandon the digest generation on the catalogue side of the things and instead let people pin the build digests in their recipe configus (this approach was shown somewhere in a ticket)

Maybe we just drop digest generation entirely, for now?

Thanks @moritz! I thought [the `docker login` step](https://git.coopcloud.tech/coop-cloud/auto-recipes-catalogue-json/src/branch/main/.drone.yml#L22) would fix that, but perhaps it only gets us a higher rate limit, which still isn't high enough to look up all the images in the catalogue? I agree with the comment by @decentral1se on Matrix, though: > great stuff 👏 one passing by comment is that i'm thinking we should maybe abandon the digest generation on the catalogue side of the things and instead let people pin the build digests in their recipe configus (this approach was shown somewhere in a ticket) Maybe we just drop digest generation entirely, for now?
decentral1se commented 2023-01-18 10:51:06 +00:00
Owner
Copy Link

great stuff 👏 one passing by comment is that i'm thinking we should maybe abandon the digest generation on the catalogue side of the things and instead let people pin the build digests in their recipe configus (this approach was shown somewhere in a ticket)

Maybe we just drop digest generation entirely, for now?

coop-cloud/organising#379

> > great stuff 👏 one passing by comment is that i'm thinking we should maybe abandon the digest generation on the catalogue side of the things and instead let people pin the build digests in their recipe configus (this approach was shown somewhere in a ticket) > > Maybe we just drop digest generation entirely, for now? https://git.coopcloud.tech/coop-cloud/organising/issues/379
decentral1se commented 2023-01-21 19:52:08 +00:00
Owner
Copy Link

How do we trigger the auto-recipes-catalogue-json build whenever there's a new release of a recipe? Ideally it'd be nice to avoid needing to add to .drone.yml in all the recipe repos; maybe a global webhook linked up to something? (n8n? ducks). This might become easier if all the recipes are in a separate organisation, see coop-cloud/organising#377.

I feel like if we had a Not Gnarly way to template the .drone.yml files for all recipes from a central repository, then it wouldn't be too bad? We already have this config for the recipe CI, so adding a separate stanza for per-tag catalogue generation doesn't seem too bad? I think we can get inter-repository webhooks going, so when a tag gets pushed to a recipe repo, it pings the catalogue generation repo. It's drone plugins somewhere afaicr.

> How do we trigger the auto-recipes-catalogue-json build whenever there's a new release of a recipe? Ideally it'd be nice to avoid needing to add to .drone.yml in all the recipe repos; maybe a global webhook linked up to something? (n8n? ducks). This might become easier if all the recipes are in a separate organisation, see coop-cloud/organising#377. I feel like if we had a Not Gnarly way to template the `.drone.yml` files for all recipes from a central repository, then it wouldn't be too bad? We already have this config for the recipe CI, so adding a separate stanza for per-tag catalogue generation doesn't seem too bad? I think we can get inter-repository webhooks going, so when a tag gets pushed to a recipe repo, it pings the catalogue generation repo. It's drone plugins somewhere afaicr.
3wordchant commented 2023-01-21 19:53:47 +00:00
Author
Owner
Copy Link

Oh yeah forgot to update here, another mass-borking later, and we're already doing this, using the Drone downstream plugin, e.g. for vaultwarden

Oh yeah forgot to update here, another mass-borking later, and we're already doing this, using the Drone `downstream` plugin, e.g. [for vaultwarden](https://git.coopcloud.tech/coop-cloud/vaultwarden/src/branch/main/.drone.yml#L26-L41)
3wordchant commented 2023-01-21 19:54:03 +00:00
Author
Owner
Copy Link

Could maybe close this off now? It's pretty automagic again.

Could maybe close this off now? It's pretty automagic again.
decentral1se commented 2023-01-21 21:31:39 +00:00
Owner
Copy Link

🤟

🤟
decentral1se commented 2023-01-24 13:33:53 +00:00
Owner
Copy Link

We'll soon (again) have automatic recipe generation, see coop-cloud/recipes-catalogue-json#4. The usual process from there is to run abra app upgrade; unsure how to translate this to server-side where there isn't (usually) a copy of ~/.abra.

This issue is not available anymore.

@moritz open again 🙈

>> We'll soon (again) have automatic recipe generation, see coop-cloud/recipes-catalogue-json#4. The usual process from there is to run abra app upgrade; unsure how to translate this to server-side where there isn't (usually) a copy of ~/.abra. > > This issue is not available anymore. @moritz open again 🙈
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 Apfelwurm carla cas coopcloud cyrnel decentral1se fauno iexos jade jmakdah2 joe-irving kawaiipunk knoflook kolaente lambdabundesverband linnealovespie moritz notplants p4u1 pharaohgraphy renovate-bot ripclap simon sixsmith trav val yksflip
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/recipes-catalogue-json#4
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?