Migrate non-recipe repos from co-op cloud org -> toolshed #666

Closed
opened 2025-01-03 07:40:38 +00:00 by decentral1se · 15 comments
Owner

The new version of #569. The number of the beast!

Remaining repos left on the co-op cloud org below.

They should all be migrated to toolshed.

(UPDATE: see below for moar migration tickets)

abra shit

https://git.coopcloud.tech/coop-cloud/abra
https://git.coopcloud.tech/coop-cloud/example CANNOT BE MIGRATED YET
https://git.coopcloud.tech/coop-cloud/godotenv
https://git.coopcloud.tech/coop-cloud/abra-test-recipe

web/docs shit

https://git.coopcloud.tech/coop-cloud/docs.coopcloud.tech
https://git.coopcloud.tech/coop-cloud/coopcloud.tech

catalogue shit

https://git.coopcloud.tech/coop-cloud/recipes.coopcloud.tech
https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json
https://git.coopcloud.tech/coop-cloud/auto-recipes-catalogue-json

test shit

https://git.coopcloud.tech/coop-cloud/drone-abra
https://git.coopcloud.tech/coop-cloud/docker-cp-deploy
https://git.coopcloud.tech/coop-cloud/stack-ssh-deploy

other

https://git.coopcloud.tech/coop-cloud/coop-cloud-apps

abra source code refs

Here's all the refs to git.coopcloud.tech on latest abra release:

grep -iR "git\.coopcloud\.tech" .
./.gitea/ISSUE_TEMPLATE.md:Please report your issue on [`coop-cloud/organising`](https://git.coopcloud.tech/coop-cloud/organising)
./scripts/installer/installer:ABRA_RELEASE_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$ABRA_VERSION"
./scripts/installer/installer:RC_VERSION_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$RC_VERSION"
./scripts/installer/installer:    echo "https://git.coopcloud.tech/coop-cloud/abra/releases"
./README.md:[![Go Report Card](https://goreportcard.com/badge/git.coopcloud.tech/coop-cloud/abra)](https://goreportcard.com/report/git.coopcloud.tech/coop-cloud/abra)
./cli/app/cmd_test.go:		// see https://git.coopcloud.tech/coop-cloud/organising/issues/336 for more
./cli/app/errors.go:    https://git.coopcloud.tech/coop-cloud/organising/issues/new/choose
./cli/catalogue/catalogue.go:It polls the entire git.coopcloud.tech/coop-cloud/... recipe repository
./cli/catalogue/catalogue.go:Push your new release to git.coopcloud.tech with "-p/--publish". This requires
./cli/cli.go:			url := fmt.Sprintf("https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/autocomplete/%s", shellType)
./cli/cli.go:			releaseCandidateURL := "https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/installer/installer"
./cli/recipe/new.go:    https://git.coopcloud.tech/coop-cloud/example
./cli/recipe/new.go:    https://git.coopcloud.tech/coop-cloud/%s
./cli/recipe/release.go:Publish your new release to git.coopcloud.tech with "-p/--publish". This
./cli/internal/cli.go:	Usage:       "Publish changes to git.coopcloud.tech",
./tests/integration/app_version.bats:  skip "known issue, see https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/issues/6"
./tests/integration/recipe_version.bats:  skip "known issue, see https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/issues/6"
./tests/integration/helpers/recipe.bash:    run git clone "https://git.coopcloud.tech/coop-cloud/$TEST_RECIPE" "$ABRA_DIR/recipes/$TEST_RECIPE"
./go.sum:git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355 h1:tCv2B4qoN6RMheKDnCzIafOkWS5BB1h7hwhmo+9bVeE=
./go.sum:git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355/go.mod h1:Q8V1zbtPAlzYSr/Dvky3wS6x58IQAl3rot2me1oSO2Q=
./pkg/lint/recipe.go:			Description:  "git.coopcloud.tech repo exists",
./pkg/lint/recipe.go:			HowToResolve: "upload your recipe to git.coopcloud.tech/coop-cloud/...",
./pkg/config/env.go:	"git.coopcloud.tech/coop-cloud/godotenv"
./pkg/config/env.go:var REPOS_BASE_URL = "https://git.coopcloud.tech/coop-cloud"
./pkg/config/env.go:var SSH_URL_TEMPLATE = "ssh://git@git.coopcloud.tech:2222/coop-cloud/%s.git"
./pkg/recipe/recipe_test.go:		// https://git.coopcloud.tech/coop-cloud/traefik/src/commit/ac3a47fe8ca3ef92db84f64cfedfbb348000faee/.env.sample#L2
./pkg/recipe/recipe.go:const ReposMetadataURL = "https://git.coopcloud.tech/api/v1/orgs/coop-cloud/repos"
./pkg/recipe/recipe.go:// local SSH configuration for git.coopcloud.tech working for this to work
./pkg/recipe/recipe.go:	bar := formatter.CreateProgressbar(-1, "retrieving recipe repos list from git.coopcloud.tech...")
./pkg/recipe/recipe.go:	return fmt.Sprintf("https://git.coopcloud.tech/api/v1/repos/coop-cloud/%s/topics", repoName)
grep: ./kadabra: binary file matches
./tags:REPOS_BASE_URL	pkg/config/abra.go	/^	REPOS_BASE_URL           = "https:\/\/git.coopcloud.tech\/coop-cloud"$/;"	v	language:Go
./tags:ReposMetadataURL	pkg/recipe/recipe.go	/^const ReposMetadataURL = "https:\/\/git.coopcloud.tech\/api\/v1\/orgs\/coop-cloud\/repos"$/;"	c	language:Go
./tags:SSH_URL_TEMPLATE	pkg/config/abra.go	/^	SSH_URL_TEMPLATE         = "ssh:\/\/git@git.coopcloud.tech:2222\/coop-cloud\/%s.git"$/;"	v	language:Go
./go.mod:	git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355
./.goreleaser.yml:  api: https://git.coopcloud.tech/api/v1
./.goreleaser.yml:  download: https://git.coopcloud.tech/
./.drone.yml:      repo: git.coopcloud.tech/coop-cloud/abra
./.drone.yml:      registry: git.coopcloud.tech
The new version of https://git.coopcloud.tech/toolshed/organising/issues/569. The number of the beast! Remaining repos left on [the co-op cloud org](https://git.coopcloud.tech/coop-cloud) below. They should all be migrated to [toolshed](https://git.coopcloud.tech/toolshed). (**UPDATE**: see below for moar migration tickets) ### `abra` shit ~~https://git.coopcloud.tech/coop-cloud/abra~~ ~~https://git.coopcloud.tech/coop-cloud/example~~ **CANNOT BE MIGRATED YET** ~~https://git.coopcloud.tech/coop-cloud/godotenv~~ ~~https://git.coopcloud.tech/coop-cloud/abra-test-recipe~~ ### web/docs shit ~~https://git.coopcloud.tech/coop-cloud/docs.coopcloud.tech~~ ~~https://git.coopcloud.tech/coop-cloud/coopcloud.tech~~ ### catalogue shit ~~https://git.coopcloud.tech/coop-cloud/recipes.coopcloud.tech~~ ~~https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json~~ ~~https://git.coopcloud.tech/coop-cloud/auto-recipes-catalogue-json~~ ### test shit ~~https://git.coopcloud.tech/coop-cloud/drone-abra~~ ~~https://git.coopcloud.tech/coop-cloud/docker-cp-deploy~~ ~~https://git.coopcloud.tech/coop-cloud/stack-ssh-deploy~~ ### ~~other~~ ~~https://git.coopcloud.tech/coop-cloud/coop-cloud-apps~~ ### `abra` source code refs Here's all the refs to `git.coopcloud.tech` on latest `abra` release: <details> ``` grep -iR "git\.coopcloud\.tech" . ./.gitea/ISSUE_TEMPLATE.md:Please report your issue on [`coop-cloud/organising`](https://git.coopcloud.tech/coop-cloud/organising) ./scripts/installer/installer:ABRA_RELEASE_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$ABRA_VERSION" ./scripts/installer/installer:RC_VERSION_URL="https://git.coopcloud.tech/api/v1/repos/coop-cloud/abra/releases/tags/$RC_VERSION" ./scripts/installer/installer: echo "https://git.coopcloud.tech/coop-cloud/abra/releases" ./README.md:[![Go Report Card](https://goreportcard.com/badge/git.coopcloud.tech/coop-cloud/abra)](https://goreportcard.com/report/git.coopcloud.tech/coop-cloud/abra) ./cli/app/cmd_test.go: // see https://git.coopcloud.tech/coop-cloud/organising/issues/336 for more ./cli/app/errors.go: https://git.coopcloud.tech/coop-cloud/organising/issues/new/choose ./cli/catalogue/catalogue.go:It polls the entire git.coopcloud.tech/coop-cloud/... recipe repository ./cli/catalogue/catalogue.go:Push your new release to git.coopcloud.tech with "-p/--publish". This requires ./cli/cli.go: url := fmt.Sprintf("https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/autocomplete/%s", shellType) ./cli/cli.go: releaseCandidateURL := "https://git.coopcloud.tech/coop-cloud/abra/raw/branch/main/scripts/installer/installer" ./cli/recipe/new.go: https://git.coopcloud.tech/coop-cloud/example ./cli/recipe/new.go: https://git.coopcloud.tech/coop-cloud/%s ./cli/recipe/release.go:Publish your new release to git.coopcloud.tech with "-p/--publish". This ./cli/internal/cli.go: Usage: "Publish changes to git.coopcloud.tech", ./tests/integration/app_version.bats: skip "known issue, see https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/issues/6" ./tests/integration/recipe_version.bats: skip "known issue, see https://git.coopcloud.tech/coop-cloud/recipes-catalogue-json/issues/6" ./tests/integration/helpers/recipe.bash: run git clone "https://git.coopcloud.tech/coop-cloud/$TEST_RECIPE" "$ABRA_DIR/recipes/$TEST_RECIPE" ./go.sum:git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355 h1:tCv2B4qoN6RMheKDnCzIafOkWS5BB1h7hwhmo+9bVeE= ./go.sum:git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355/go.mod h1:Q8V1zbtPAlzYSr/Dvky3wS6x58IQAl3rot2me1oSO2Q= ./pkg/lint/recipe.go: Description: "git.coopcloud.tech repo exists", ./pkg/lint/recipe.go: HowToResolve: "upload your recipe to git.coopcloud.tech/coop-cloud/...", ./pkg/config/env.go: "git.coopcloud.tech/coop-cloud/godotenv" ./pkg/config/env.go:var REPOS_BASE_URL = "https://git.coopcloud.tech/coop-cloud" ./pkg/config/env.go:var SSH_URL_TEMPLATE = "ssh://git@git.coopcloud.tech:2222/coop-cloud/%s.git" ./pkg/recipe/recipe_test.go: // https://git.coopcloud.tech/coop-cloud/traefik/src/commit/ac3a47fe8ca3ef92db84f64cfedfbb348000faee/.env.sample#L2 ./pkg/recipe/recipe.go:const ReposMetadataURL = "https://git.coopcloud.tech/api/v1/orgs/coop-cloud/repos" ./pkg/recipe/recipe.go:// local SSH configuration for git.coopcloud.tech working for this to work ./pkg/recipe/recipe.go: bar := formatter.CreateProgressbar(-1, "retrieving recipe repos list from git.coopcloud.tech...") ./pkg/recipe/recipe.go: return fmt.Sprintf("https://git.coopcloud.tech/api/v1/repos/coop-cloud/%s/topics", repoName) grep: ./kadabra: binary file matches ./tags:REPOS_BASE_URL pkg/config/abra.go /^ REPOS_BASE_URL = "https:\/\/git.coopcloud.tech\/coop-cloud"$/;" v language:Go ./tags:ReposMetadataURL pkg/recipe/recipe.go /^const ReposMetadataURL = "https:\/\/git.coopcloud.tech\/api\/v1\/orgs\/coop-cloud\/repos"$/;" c language:Go ./tags:SSH_URL_TEMPLATE pkg/config/abra.go /^ SSH_URL_TEMPLATE = "ssh:\/\/git@git.coopcloud.tech:2222\/coop-cloud\/%s.git"$/;" v language:Go ./go.mod: git.coopcloud.tech/coop-cloud/godotenv v1.5.2-0.20231130100509-01bff8284355 ./.goreleaser.yml: api: https://git.coopcloud.tech/api/v1 ./.goreleaser.yml: download: https://git.coopcloud.tech/ ./.drone.yml: repo: git.coopcloud.tech/coop-cloud/abra ./.drone.yml: registry: git.coopcloud.tech ``` </details>
decentral1se added the
enhancement
label 2025-01-03 07:40:38 +00:00
Author
Owner

Should we migrate in reverse order? other -> test -> catalogue etc. (fixing along the way?) I think we can't migrate example without breaking previous abra releases (it's what is cloned for a new recipe), so it will have to stay for a while longer? technically it is a "recipe" 🙃

Should we migrate in reverse order? other -> test -> catalogue etc. (fixing along the way?) I think we can't migrate [`example`](https://git.coopcloud.tech/coop-cloud/example) without breaking previous `abra` releases (it's what is cloned for a new recipe), so it will have to stay for a while longer? technically it is a "recipe" 🙃
Owner
https://git.coopcloud.tech/toolshed/coop-cloud-apps
Author
Owner
https://git.coopcloud.tech/toolshed/docker-cp-deploy https://git.coopcloud.tech/toolshed/docker-cp-deploy/issues/7
Author
Owner
https://git.coopcloud.tech/toolshed/drone-abra https://git.coopcloud.tech/toolshed/drone-abra/issues/6
Author
Owner
https://git.coopcloud.tech/toolshed/stack-ssh-deploy https://git.coopcloud.tech/toolshed/stack-ssh-deploy/issues/10
Author
Owner
https://git.coopcloud.tech/toolshed/auto-recipes-catalogue-json https://git.coopcloud.tech/toolshed/auto-recipes-catalogue-json/issues/4
Author
Owner
https://git.coopcloud.tech/toolshed/recipes-catalogue-json https://git.coopcloud.tech/toolshed/recipes-catalogue-json/issues/7
Author
Owner
https://git.coopcloud.tech/toolshed/recipes.coopcloud.tech https://git.coopcloud.tech/toolshed/recipes.coopcloud.tech/issues/37
Author
Owner
https://git.coopcloud.tech/toolshed/coopcloud.tech https://git.coopcloud.tech/toolshed/coopcloud.tech/issues/49 https://git.coopcloud.tech/toolshed/docs.coopcloud.tech https://git.coopcloud.tech/toolshed/docs.coopcloud.tech/issues/264
Author
Owner
https://git.coopcloud.tech/toolshed/abra-test-recipe https://git.coopcloud.tech/toolshed/godotenv https://git.coopcloud.tech/toolshed/abra `abra` still working apparently 🤷🤷🤷 More manual testing...
Author
Owner

I've migrated the abra container package. Here's what I did:

  • docker login
  • docker build -t git.coopcloud.tech/toolshed/abra .
  • docker push git.coopcloud.tech/toolshed/abra
  • Visited the toolshed packages tab
  • Linked the abra container with the abra repo packages
  • I then logged in as admin and reset the abra-bot user account (forgot password 🙈 DM me for a copy). This is the account used for the matrix notifications "back in the day". It can be used as a shared account for general automation? I generated a token for package repo automation (DM me for a copy)
  • I then added abra-bot to the toolshed robot org team
  • I then logged into https://build.coopcloud.tech as abra-bot and added git_coopcloud_tech_token_abra_bot as a secret using the CLI: drone secret add --repository toolshed/abra --name git_coopcloud_tech_token_abra_bot --data "..." --allow-push-on-pull-request --allow-pull-request (see #666 (comment) for a better way of doing this)
  • Annnnnd it works: https://build.coopcloud.tech/toolshed/abra/2467/1/4

.drone.yml Lines 46 to 61 in 3869d6bce9
- name: publish image
image: plugins/docker
settings:
auto_tag: true
username: abra-bot
password:
from_secret: git_coopcloud_tech_token_abra_bot
repo: git.coopcloud.tech/toolshed/abra
tags: dev
registry: git.coopcloud.tech
when:
branch:
- main
depends_on:
- make check
- make test

I've migrated the `abra` container package. Here's what I did: * `docker login` * `docker build -t git.coopcloud.tech/toolshed/abra .` * `docker push git.coopcloud.tech/toolshed/abra` * Visited the [`toolshed` packages tab](https://git.coopcloud.tech/toolshed/-/packages/container/abra/latest) * Linked the `abra` container with the [`abra` repo packages](https://git.coopcloud.tech/toolshed/-/packages) * I then logged in as admin and reset the `abra-bot` user account (forgot password 🙈 DM me for a copy). This is the account used for the matrix notifications "back in the day". It can be used as a shared account for general automation? I generated a token for package repo automation (DM me for a copy) * I then added `abra-bot` to the [toolshed robot org team](https://git.coopcloud.tech/org/toolshed/teams/robots) * I then logged into https://build.coopcloud.tech as `abra-bot` and added `git_coopcloud_tech_token_abra_bot` as a secret using the CLI: `drone secret add --repository toolshed/abra --name git_coopcloud_tech_token_abra_bot --data "..." --allow-push-on-pull-request --allow-pull-request` (see https://git.coopcloud.tech/toolshed/organising/issues/666#issuecomment-22316 for a better way of doing this) * Annnnnd it works: https://build.coopcloud.tech/toolshed/abra/2467/1/4 https://git.coopcloud.tech/toolshed/abra/src/commit/3869d6bce91bd55e6ff8bb7a990efd5fdf4ac2d7/.drone.yml#L46-L61
Author
Owner

@3wordchant lemme know what you think but if you could add git_coopcloud_tech_token_abra_bot as an "orgsecret" for the toolshed org on https://build.coopcloud.tech then it would be available for the CI builds.

Something like: drone orgsecret add toolshed git_coopcloud_tech_token_abra_bot "<token>" --allow-push-on-pull-request --allow-pull-request

I'll DM you the token now.

I can't seem to do this as it requires "system administration permissions" according to the docs. Maybe you need to run that in the container of the Drone deployment itself?

@3wordchant lemme know what you think but if you could add `git_coopcloud_tech_token_abra_bot` as an "orgsecret" for the `toolshed` org on https://build.coopcloud.tech then it would be available for the CI builds. Something like: `drone orgsecret add toolshed git_coopcloud_tech_token_abra_bot "<token>" --allow-push-on-pull-request --allow-pull-request` I'll DM you the token now. I can't seem to do this as it requires "system administration permissions" according to the docs. Maybe you need to run that in the container of the Drone deployment itself?
Owner

Something like: drone orgsecret add toolshed git_coopcloud_tech_token_abra_bot "<token>" --allow-push-on-pull-request --allow-pull-request

Done. A bit unsure about --allow-push-on-pull-request --allow-pull-request because my understanding is this means anyone with a git.coopcloud.tech account (including drive-by github / gitlab logins) can now read out the token. But I went with it for now.

I can't seem to do this as it requires "system administration permissions" according to the docs. Maybe you need to run that in the container of the Drone deployment itself?

Only the coopcloud account has permissions to edit orgsecrets, I think. LMK if you need the password for that again.

> Something like: `drone orgsecret add toolshed git_coopcloud_tech_token_abra_bot "<token>" --allow-push-on-pull-request --allow-pull-request` Done. A bit unsure about `--allow-push-on-pull-request --allow-pull-request` because my understanding is this means anyone with a git.coopcloud.tech account (including drive-by github / gitlab logins) can now read out the token. But I went with it for now. > I can't seem to do this as it requires "system administration permissions" according to the docs. Maybe you need to run that in the container of the Drone deployment itself? Only the `coopcloud` account has permissions to edit orgsecrets, I think. LMK if you need the password for that again.
Owner

Also while we're smashing CI to pieces maybe a good moment to migrate away from my personal token, #668

Also while we're smashing CI to pieces maybe a good moment to migrate away from my personal token, #668
Author
Owner

There's still bits & bobs to be done but this mostly "done".

There's still bits & bobs to be done but this mostly "done".
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/organising#666
No description provided.