toolshed/abra
4
2
Fork 18
Code Issues 21 Pull Requests 4 Packages Releases 40 Activity

autocomplete on abra recipe upgrade is slow #567

New Issue
Closed
opened 2025-06-04 17:15:29 +00:00 by 3wordchant · 9 comments
3wordchant commented 2025-06-04 17:15:29 +00:00
Owner
Copy Link

Not quite a "bug" but it does cause a little friction.

Autocomplete on recipe names is much slower than on app names. Probably because app autocomplete is based on listing files but recipe autocomplete is based on parsing JSON. Maybe there's nothing we can do but if it's possible to optimise it'd be awesome

CC @trav

Not quite a "bug" but it does cause a little friction. Autocomplete on recipe names is much slower than on app names. Probably because app autocomplete is based on listing files but recipe autocomplete is based on parsing JSON. Maybe there's nothing we can do but if it's possible to optimise it'd be awesome CC @trav
👍 1
3wordchant added the
bug
 label 2025-06-04 17:15:29 +00:00
decentral1se commented 2025-06-05 07:59:03 +00:00
Owner
Copy Link

Yeh I would love to speed this up also, I routinely get frozen waiting for this...

I think it might be doing some network activity in there somewhere....

Definitely something to improve!

Yeh I would love to speed this up also, I routinely get frozen waiting for this... I think it might be doing some network activity in there somewhere.... Definitely something to improve!
decentral1se self-assigned this 2025-07-08 09:59:25 +00:00
decentral1se added the
red abya yala
 label 2025-07-08 10:02:24 +00:00
decentral1se added this to the Abra "next" project 2025-07-08 10:06:51 +00:00
p4u1 commented 2025-08-11 09:38:06 +00:00
Member
Copy Link

When autocompleting recipe names, it first ensures the recipe catalogue is up to date. This makes it slower. We have the following options:

  1. Don't ensure at all
  2. Only ensure exists and not ensure up to date
  3. Only ensure on the first autocomplete invocation. When doing abra recipe upgrade <tab><tab> It only ensures once
When autocompleting recipe names, it first ensures the recipe catalogue is up to date. This makes it slower. We have the following options: 1. Don't ensure at all 2. Only ensure exists and not ensure up to date 3. Only ensure on the first autocomplete invocation. When doing `abra recipe upgrade <tab><tab>` It only ensures once
👀 1
3wordchant commented 2025-08-11 10:14:56 +00:00
Author
Owner
Copy Link

Thanks @p4u1 ! 🙏

  1. Don't ensure at all
  2. Only ensure exists and not ensure up to date
  3. Only ensure on the first autocomplete invocation. When doing abra recipe upgrade It only ensures once

#3 seems good, but maybe the most complex to implement – and it might be hard to avoid it being "first autocomplete invocation per shell".

#1 seems easiest but most fragile.

I am leaning towards #2 – but am I correct that we don't have a specific command to sync the recipe catalogue?

Thanks @p4u1 ! 🙏 > 1. Don't ensure at all > 2. Only ensure exists and not ensure up to date > 3. Only ensure on the first autocomplete invocation. When doing abra recipe upgrade <tab><tab> It only ensures once #3 seems good, but maybe the most complex to implement – and it might be hard to avoid it being "first autocomplete invocation _per shell_". #1 seems easiest but most fragile. I am leaning towards #2 – but am I correct that we don't have a specific command to sync the recipe catalogue?
p4u1 commented 2025-08-11 10:24:29 +00:00
Member
Copy Link

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade

also you are correct that there is no command to sync the recipe catalogue. With option two would then need that i guess..

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first <tab> each time you run abra recipe upgrade also you are correct that there is no command to sync the recipe catalogue. With option two would then need that i guess..
👍 1
3wordchant commented 2025-08-11 12:35:48 +00:00
Author
Owner
Copy Link

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade

OK! Out of interest, how are you thinking abra would know whether to refresh the catalogue?

> option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade OK! Out of interest, how are you thinking abra would know whether to refresh the catalogue?
decentral1se commented 2025-08-11 13:11:14 +00:00
Owner
Copy Link

Thanks for thinking this one through! It might make sense to do a abra catalogue sync and then make all catalogue operations local only? Then if you're missing something you expect to be there, you run a sync?

This is all the places that recipe.ReadRecipeCatalogue() is called, so it's quite significant and contributes to slowing things down. Also, we tried to get around this before by adding a true/false argument if it should only do an offline operation.

pkg/recipe/recipe.go|449 col 15| GetRecipeMeta
cli/recipe/list.go|22 col 23| init
cli/recipe/version.go|31 col 26| init
pkg/autocomplete/autocomplete.go|57 col 22| RecipeVersionComplete
cli/catalogue/catalogue.go|157 col 26| init
cli/internal/validate.go|23 col 23| ValidateRecipe
cli/recipe/fetch.go|88 col 28| init
cli/updater/updater.go|260 col 22| getAvailableUpgrades
pkg/autocomplete/autocomplete.go|41 col 22| RecipeNameComplete
pkg/lint/recipe.go|396 col 25| LintHasPublishedVersion
pkg/recipe/recipe.go|414 col 20| VersionsOfService

One optimisation could be to check if go-git has any way to remotely query the last modified value of the catalogue repository? Then you would know potentially faster if you need to actually do a full pull or not. I'm not sure this is provided and if it would offer a better experience in the "need to pull" case.

Not sure on this one!

Thanks for thinking this one through! It might make sense to do a `abra catalogue sync` and then make all catalogue operations local only? Then if you're missing something you expect to be there, you run a sync? This is all the places that `recipe.ReadRecipeCatalogue()` is called, so it's quite significant and contributes to slowing things down. Also, we tried to get around this before by adding a `true`/`false` argument if it should only do an offline operation. ``` pkg/recipe/recipe.go|449 col 15| GetRecipeMeta cli/recipe/list.go|22 col 23| init cli/recipe/version.go|31 col 26| init pkg/autocomplete/autocomplete.go|57 col 22| RecipeVersionComplete cli/catalogue/catalogue.go|157 col 26| init cli/internal/validate.go|23 col 23| ValidateRecipe cli/recipe/fetch.go|88 col 28| init cli/updater/updater.go|260 col 22| getAvailableUpgrades pkg/autocomplete/autocomplete.go|41 col 22| RecipeNameComplete pkg/lint/recipe.go|396 col 25| LintHasPublishedVersion pkg/recipe/recipe.go|414 col 20| VersionsOfService ``` One optimisation could be to check if `go-git` has any way to remotely query the last modified value of the catalogue repository? Then you would know potentially faster if you need to actually do a full pull or not. I'm not sure this is provided and if it would offer a better experience in the "need to pull" case. Not sure on this one!
decentral1se moved this to In Progress in Abra "next" on 2025-08-12 05:23:00 +00:00
decentral1se commented 2025-08-18 07:56:09 +00:00
Owner
Copy Link

My proposal is option 2. I don't understand how option 3 would know how to only do it once and I am not much of a fan of doing some sort of bookkeeping to remember this. Please do let us know though if there is some way to manage it. Then, I would go for:

  • Add abra catalogue sync to update catalogue
  • Ensure catalogue exists but don't pull latest

This way, it's less automatic but should be faster for the day-to-day use. If you can't find it in the auto-complete, you know you should abra catalogue sync. And the implementation should also make sure it pulls in the local recipe listing too (which I think it does).

My proposal is option 2. I don't understand how option 3 would know how to only do it once and I am not much of a fan of doing some sort of bookkeeping to remember this. Please do let us know though if there is some way to manage it. Then, I would go for: * Add `abra catalogue sync` to update catalogue * Ensure catalogue exists but don't pull latest This way, it's less automatic but should be faster for the day-to-day use. If you can't find it in the auto-complete, you know you should `abra catalogue sync`. And the implementation should also make sure it pulls in the local recipe listing too (which I think it does).
decentral1se commented 2025-08-18 08:15:16 +00:00
Owner
Copy Link

RFC: #604

RFC: https://git.coopcloud.tech/toolshed/abra/pulls/604
decentral1se commented 2025-08-24 07:26:14 +00:00
Owner
Copy Link

I think #604 made a bit of an improvement for now!

Please re-open if there are still ways to improve it.

I think https://git.coopcloud.tech/toolshed/abra/pulls/604 made a bit of an improvement for now! Please re-open if there are still ways to improve it.
decentral1se moved this to Done in Abra "next" on 2025-08-24 07:28:26 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is not working
build
go build related issues
ci/cd
Building things with CI/CD
contributing
Dealing with contributing
critical fix
https://docs.coopcloud.tech/federation/resolutions/passed/010/
design
UI/UX
documentation
Documenting all the things
duplicate
This issue or pull request already exists
enhancement
New feature
help wanted
Need some help
invalid
Something is wrong
meta
relating to the project or repo itself
question
More information is needed
red abya yala
Work funded by Calyx institute for Red Abya Yala
release
Release management
release-candidate
Related to the new release candidate
security
Security related
test
Unit/integration testing
wontfix
This won't be fixed
No Label
bug
red abya yala
Milestone
No items
No Milestone
Projects
Clear projects
No project Abra "next"
Assignees
Clear assignees
3wordchant abra-bot ammaratef45 Apfelwurm carla cas coopcloud decentral1se fauno jmakdah2 kawaiipunk knoflook lambdabundesverband moritz p4u1 ripclap simon sixsmith trav val yksflip
No Assignees decentral1se
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/abra#567
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?