toolshed/abra
4
4
Fork 23
Code Issues 82 Pull Requests 12 Packages Releases 45 Activity

test: automated integration test suite #424

Merged
decentral1se merged 3 commits from auto-int-suite into main 2024-07-02 15:19:15 +00:00
Conversation 3 Commits 3 Files Changed 50
+383 -470
decentral1se commented 2024-06-29 16:27:05 +00:00
Owner
Copy Link
Copy Source

coop-cloud/organising#584 coop-cloud/organising#595 coop-cloud/organising#619

https://git.coopcloud.tech/coop-cloud/organising/issues/584 https://git.coopcloud.tech/coop-cloud/organising/issues/595 https://git.coopcloud.tech/coop-cloud/organising/issues/619
❤️ 1
decentral1se changed title from WIP: automated integration test suite to WIP: test: automated integration test suite 2024-06-29 16:27:42 +00:00
decentral1se force-pushed auto-int-suite from c099344b60 to c5e5395dd8 2024-06-29 22:27:46 +00:00 Compare
decentral1se force-pushed auto-int-suite from c5e5395dd8 to 745e073396 2024-06-30 13:22:02 +00:00 Compare
decentral1se force-pushed auto-int-suite from b604d7170a to 8a7a4759a3 2024-06-30 13:33:30 +00:00 Compare
decentral1se force-pushed auto-int-suite from 8a7a4759a3 to 06ab3797fe 2024-06-30 13:34:07 +00:00 Compare
decentral1se force-pushed auto-int-suite from 06ab3797fe to 6c12f9e2bf 2024-06-30 13:35:12 +00:00 Compare
decentral1se force-pushed auto-int-suite from 6c12f9e2bf to b4dd28728e 2024-06-30 13:37:00 +00:00 Compare
decentral1se force-pushed auto-int-suite from b4dd28728e to b18c8445a5 2024-06-30 13:37:45 +00:00 Compare
decentral1se force-pushed auto-int-suite from b18c8445a5 to 4e8f77e772 2024-06-30 13:43:04 +00:00 Compare
decentral1se force-pushed auto-int-suite from 4e8f77e772 to fbba72cd66 2024-06-30 13:43:49 +00:00 Compare
decentral1se force-pushed auto-int-suite from fbba72cd66 to 0f8e97863f 2024-06-30 13:52:38 +00:00 Compare
decentral1se force-pushed auto-int-suite from 0f8e97863f to 3cf2b9ea2d 2024-06-30 14:04:32 +00:00 Compare
decentral1se force-pushed auto-int-suite from 3cf2b9ea2d to 4b876c715c 2024-06-30 14:09:57 +00:00 Compare
decentral1se force-pushed auto-int-suite from 4b876c715c to 849d82d2d7 2024-06-30 14:11:28 +00:00 Compare
decentral1se force-pushed auto-int-suite from 849d82d2d7 to 1d548e6257 2024-06-30 14:12:20 +00:00 Compare
decentral1se force-pushed auto-int-suite from 1d548e6257 to 8f1e5dff3f 2024-06-30 14:14:27 +00:00 Compare
decentral1se force-pushed auto-int-suite from 8f1e5dff3f to ced88ce719 2024-06-30 14:18:38 +00:00 Compare
decentral1se force-pushed auto-int-suite from ced88ce719 to 630cbaac10 2024-06-30 14:20:35 +00:00 Compare
decentral1se force-pushed auto-int-suite from 630cbaac10 to cbfd47a857 2024-06-30 14:25:34 +00:00 Compare
decentral1se force-pushed auto-int-suite from cbfd47a857 to 2271471d69 2024-06-30 14:26:44 +00:00 Compare
decentral1se force-pushed auto-int-suite from 2271471d69 to ffcfaa674e 2024-06-30 14:29:13 +00:00 Compare
decentral1se force-pushed auto-int-suite from ffcfaa674e to 60e73cac97 2024-06-30 14:36:14 +00:00 Compare
decentral1se force-pushed auto-int-suite from 60e73cac97 to 34ae5455fb 2024-06-30 14:42:35 +00:00 Compare
decentral1se force-pushed auto-int-suite from 34ae5455fb to 7d9a7aed59 2024-06-30 14:44:23 +00:00 Compare
decentral1se commented 2024-07-02 08:21:06 +00:00
Author
Owner
Copy Link
Copy Source

OK, it seems like doing these monster builds blew up the storage and led to the infra meltdown so I think maybe let's switch gears on approach. I'll just set up the integration test suite machine to run the suite and the build will hopefully somehow be able to ssh over and trigger the build. This was more or less already the approach but there was so much deps to install. And I have a feeling that it's a bit too unpredictable to select the right context / remote (the danger being, nuking the machine that the builds are running on which have access to the docker socket). Will try get something working today.

OK, it seems like doing these monster builds blew up the storage and led to the [infra meltdown](https://social.coop/@coopcloud/112712426835259414) so I think maybe let's switch gears on approach. I'll just set up the integration test suite machine to run the suite and the build will hopefully somehow be able to ssh over and trigger the build. This was more or less already the approach but there was so much deps to install. And I have a feeling that it's a bit too unpredictable to select the right context / remote (the danger being, nuking the machine that the builds are running on which have access to the docker socket). Will try get something working today.
decentral1se force-pushed auto-int-suite from 7d9a7aed59 to 930b291317 2024-07-02 09:12:48 +00:00 Compare
decentral1se force-pushed auto-int-suite from 930b291317 to d632a0b159 2024-07-02 09:46:13 +00:00 Compare
decentral1se force-pushed auto-int-suite from d632a0b159 to e88968a3b7 2024-07-02 10:55:34 +00:00 Compare
decentral1se force-pushed auto-int-suite from e88968a3b7 to 6a86695023 2024-07-02 12:57:07 +00:00 Compare
decentral1se force-pushed auto-int-suite from 6a86695023 to 9554ad40c8 2024-07-02 13:14:18 +00:00 Compare
decentral1se commented 2024-07-02 13:37:34 +00:00
Author
Owner
Copy Link
Copy Source

Timing for test run (will vary depending on machine but a rough guide. running these on the donated machine for now):

  • Fast tests (no deploys): 176 tests, 0 failures, 17 skipped in 360 seconds
  • Slow tests (with deploys): 83 tests, 0 failure, 9 skipped in 978 seconds
  • DNS tests (public DNS required): 2 tests, 0 failures in 38 seconds

So, the full test suite is 360 + 978 + 38 = 1376 / 60 = 22 minutes.

I'm going to try to use https://plugins.drone.io/plugins/ssh and https://docs.drone.io/cron/ to wire up a nightly cron to run the entire suite.

Backup/restore tests currently being skipped due to incomplete new implementation. More hopefully coming soon on that.

The change for coop-cloud/organising#607 really straightened out so many failures! Very happy with that.

I got rate limited on the server (faaaaack) by Docker hub so had to log in as thecoopcloud account 🙄 Dunno what is the state of paying for that but it is still very much needed. The bastards.

Timing for test run (will vary depending on machine but a rough guide. running these on the donated machine for now): * Fast tests (no deploys): `176 tests, 0 failures, 17 skipped in 360 seconds` * Slow tests (with deploys): `83 tests, 0 failure, 9 skipped in 978 seconds` * DNS tests (public DNS required): `2 tests, 0 failures in 38 seconds` So, the full test suite is 360 + 978 + 38 = 1376 / 60 = 22 minutes. I'm going to try to use https://plugins.drone.io/plugins/ssh and https://docs.drone.io/cron/ to wire up a nightly cron to run the entire suite. Backup/restore tests currently being skipped due to incomplete new implementation. More hopefully coming soon on that. The change for https://git.coopcloud.tech/coop-cloud/organising/issues/607 really straightened out so many failures! Very happy with that. I got rate limited on the server (faaaaack) by Docker hub so had to log in as `thecoopcloud` account 🙄 Dunno what is the state of paying for that but it is still very much needed. The bastards.
decentral1se force-pushed auto-int-suite from 89e1b22ed7 to 548e11bb65 2024-07-02 13:57:03 +00:00 Compare
decentral1se force-pushed auto-int-suite from 548e11bb65 to 7498f089cb 2024-07-02 13:58:25 +00:00 Compare
decentral1se force-pushed auto-int-suite from 7498f089cb to 79073a9c4a 2024-07-02 14:02:02 +00:00 Compare
decentral1se force-pushed auto-int-suite from 79073a9c4a to 4cd8f3e77b 2024-07-02 14:07:57 +00:00 Compare
decentral1se force-pushed auto-int-suite from 4cd8f3e77b to 945bd32d76 2024-07-02 14:12:12 +00:00 Compare
decentral1se force-pushed auto-int-suite from 945bd32d76 to 7f3de376ec 2024-07-02 14:14:54 +00:00 Compare
decentral1se force-pushed auto-int-suite from 7f3de376ec to de8e3fc2b2 2024-07-02 14:18:43 +00:00 Compare
decentral1se force-pushed auto-int-suite from de8e3fc2b2 to 0dac05c5ed 2024-07-02 14:40:52 +00:00 Compare
decentral1se force-pushed auto-int-suite from 0dac05c5ed to a51725911e 2024-07-02 14:43:54 +00:00 Compare
decentral1se force-pushed auto-int-suite from a51725911e to b0c09691f4 2024-07-02 14:47:19 +00:00 Compare
decentral1se commented 2024-07-02 15:03:30 +00:00
Author
Owner
Copy Link
Copy Source

BUILDING: https://build.coopcloud.tech/coop-cloud/abra/1947/1/2

Nearly there... just polishing now & docs. Hopefully finished up this week 🙏

BUILDING: https://build.coopcloud.tech/coop-cloud/abra/1947/1/2 Nearly there... just polishing now & docs. Hopefully finished up this week 🙏
🎉 1
decentral1se force-pushed auto-int-suite from b0c09691f4 to 63c60420b2 2024-07-02 15:13:06 +00:00 Compare
decentral1se force-pushed auto-int-suite from 63c60420b2 to 2319214c47 2024-07-02 15:16:54 +00:00 Compare
decentral1se force-pushed auto-int-suite from 2319214c47 to 6a2db1abaa 2024-07-02 15:18:12 +00:00 Compare
decentral1se marked the pull request as ready for review 2024-07-02 15:18:51 +00:00
decentral1se merged commit 6a2db1abaa into main 2024-07-02 15:19:15 +00:00
decentral1se deleted branch auto-int-suite 2024-07-02 15:19:16 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
bug
Something is not working
 build
go build related issues
 ci/cd
Building things with CI/CD
 critical fix
https://docs.coopcloud.tech/federation/resolutions/passed/010/
 design
UI/UX
 documentation
Documenting all the things
 duplicate
This issue or pull request already exists
 easy-first-issue
Something for new people to get stuck into. We hope it's easy!
 enhancement
New feature
 help wanted
Need some help
 i10n
Everything to do with localisation
 i18n
Everything to do with internationalisation
 installer
Everything to do with the install script.
 invalid
Something is wrong
 question
More information is needed
 release
Release management
 release-candidate
Related to the new release candidate
 security
Security related
 tech-debt
test
Unit/integration testing
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
3wordchant aadil (Aadil Ayub) abra-bot (Abra Bot) ammaratef45 amras (Sarma) Apfelwurm BornDeleuze Brooke carla cas (Cassowary) coopcloud cyrnel decentral1se (d1) dede devydave fauno (fauno) iexos jade (Jade Ambrose) jjsfunhouse jmakdah2 (Jackie Makdah) joe-irving (Joe Irving) kawaiipunk (KawaiiPunk) knoflook kolaente lambdabundesverband linnealovespie (April) moosemower moritz notplants oxaliq (sorrel) p4u1 pharaohgraphy (Andrew 🐦‍🔥❤️‍🔥✴️) renovate-bot (Comrade Renovate Bot) ripclap simon sixsmith (Sixsmith) stevensting trav (Trav Fryer) val (val (he/him)) yksflip
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/abra#424