autocomplete on abra recipe upgrade is slow #567

New Issue

3wordchant · 2025-06-04T17:15:29Z

3wordchant commented

2025-06-04 17:15:29 +00:00

Not quite a "bug" but it does cause a little friction.

Autocomplete on recipe names is much slower than on app names. Probably because app autocomplete is based on listing files but recipe autocomplete is based on parsing JSON. Maybe there's nothing we can do but if it's possible to optimise it'd be awesome

CC @trav

Not quite a "bug" but it does cause a little friction. Autocomplete on recipe names is much slower than on app names. Probably because app autocomplete is based on listing files but recipe autocomplete is based on parsing JSON. Maybe there's nothing we can do but if it's possible to optimise it'd be awesome CC @trav

👍 1

3wordchant added the

bug

label 2025-06-04 17:15:29 +00:00

decentral1se commented

2025-06-05 07:59:03 +00:00

Yeh I would love to speed this up also, I routinely get frozen waiting for this...

I think it might be doing some network activity in there somewhere....

Definitely something to improve!

Yeh I would love to speed this up also, I routinely get frozen waiting for this... I think it might be doing some network activity in there somewhere.... Definitely something to improve!

decentral1se self-assigned this 2025-07-08 09:59:25 +00:00

decentral1se added the

red abya yala

label 2025-07-08 10:02:24 +00:00

decentral1se added this to the Abra "next" project 2025-07-08 10:06:51 +00:00

p4u1 commented

2025-08-11 09:38:06 +00:00

When autocompleting recipe names, it first ensures the recipe catalogue is up to date. This makes it slower. We have the following options:

Don't ensure at all
Only ensure exists and not ensure up to date
Only ensure on the first autocomplete invocation. When doing abra recipe upgrade <tab><tab> It only ensures once

When autocompleting recipe names, it first ensures the recipe catalogue is up to date. This makes it slower. We have the following options: 1. Don't ensure at all 2. Only ensure exists and not ensure up to date 3. Only ensure on the first autocomplete invocation. When doing `abra recipe upgrade <tab><tab>` It only ensures once

👀 1

3wordchant commented

2025-08-11 10:14:56 +00:00

Thanks @p4u1 ! 🙏

Don't ensure at all

Only ensure exists and not ensure up to date

Only ensure on the first autocomplete invocation. When doing abra recipe upgrade It only ensures once

#3 seems good, but maybe the most complex to implement – and it might be hard to avoid it being "first autocomplete invocation per shell".

#1 seems easiest but most fragile.

I am leaning towards #2 – but am I correct that we don't have a specific command to sync the recipe catalogue?

Thanks @p4u1 ! 🙏 > 1. Don't ensure at all > 2. Only ensure exists and not ensure up to date > 3. Only ensure on the first autocomplete invocation. When doing abra recipe upgrade <tab><tab> It only ensures once #3 seems good, but maybe the most complex to implement – and it might be hard to avoid it being "first autocomplete invocation _per shell_". #1 seems easiest but most fragile. I am leaning towards #2 – but am I correct that we don't have a specific command to sync the recipe catalogue?

p4u1 commented

2025-08-11 10:24:29 +00:00

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade

also you are correct that there is no command to sync the recipe catalogue. With option two would then need that i guess..

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first <tab> each time you run abra recipe upgrade also you are correct that there is no command to sync the recipe catalogue. With option two would then need that i guess..

👍 1

3wordchant commented

2025-08-11 12:35:48 +00:00

option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade

OK! Out of interest, how are you thinking abra would know whether to refresh the catalogue?

> option 3 is actually quite easy to implement. This would also be the safest option, but it would still be slow on the first each time you run abra recipe upgrade OK! Out of interest, how are you thinking abra would know whether to refresh the catalogue?

decentral1se commented

2025-08-11 13:11:14 +00:00

Thanks for thinking this one through! It might make sense to do a abra catalogue sync and then make all catalogue operations local only? Then if you're missing something you expect to be there, you run a sync?

This is all the places that recipe.ReadRecipeCatalogue() is called, so it's quite significant and contributes to slowing things down. Also, we tried to get around this before by adding a true/false argument if it should only do an offline operation.

pkg/recipe/recipe.go|449 col 15| GetRecipeMeta
cli/recipe/list.go|22 col 23| init
cli/recipe/version.go|31 col 26| init
pkg/autocomplete/autocomplete.go|57 col 22| RecipeVersionComplete
cli/catalogue/catalogue.go|157 col 26| init
cli/internal/validate.go|23 col 23| ValidateRecipe
cli/recipe/fetch.go|88 col 28| init
cli/updater/updater.go|260 col 22| getAvailableUpgrades
pkg/autocomplete/autocomplete.go|41 col 22| RecipeNameComplete
pkg/lint/recipe.go|396 col 25| LintHasPublishedVersion
pkg/recipe/recipe.go|414 col 20| VersionsOfService

One optimisation could be to check if go-git has any way to remotely query the last modified value of the catalogue repository? Then you would know potentially faster if you need to actually do a full pull or not. I'm not sure this is provided and if it would offer a better experience in the "need to pull" case.

Not sure on this one!

Thanks for thinking this one through! It might make sense to do a `abra catalogue sync` and then make all catalogue operations local only? Then if you're missing something you expect to be there, you run a sync? This is all the places that `recipe.ReadRecipeCatalogue()` is called, so it's quite significant and contributes to slowing things down. Also, we tried to get around this before by adding a `true`/`false` argument if it should only do an offline operation. ``` pkg/recipe/recipe.go|449 col 15| GetRecipeMeta cli/recipe/list.go|22 col 23| init cli/recipe/version.go|31 col 26| init pkg/autocomplete/autocomplete.go|57 col 22| RecipeVersionComplete cli/catalogue/catalogue.go|157 col 26| init cli/internal/validate.go|23 col 23| ValidateRecipe cli/recipe/fetch.go|88 col 28| init cli/updater/updater.go|260 col 22| getAvailableUpgrades pkg/autocomplete/autocomplete.go|41 col 22| RecipeNameComplete pkg/lint/recipe.go|396 col 25| LintHasPublishedVersion pkg/recipe/recipe.go|414 col 20| VersionsOfService ``` One optimisation could be to check if `go-git` has any way to remotely query the last modified value of the catalogue repository? Then you would know potentially faster if you need to actually do a full pull or not. I'm not sure this is provided and if it would offer a better experience in the "need to pull" case. Not sure on this one!

decentral1se moved this to In Progress in Abra "next" on 2025-08-12 05:23:00 +00:00

decentral1se commented

2025-08-18 07:56:09 +00:00

My proposal is option 2. I don't understand how option 3 would know how to only do it once and I am not much of a fan of doing some sort of bookkeeping to remember this. Please do let us know though if there is some way to manage it. Then, I would go for:

Add abra catalogue sync to update catalogue
Ensure catalogue exists but don't pull latest

This way, it's less automatic but should be faster for the day-to-day use. If you can't find it in the auto-complete, you know you should abra catalogue sync. And the implementation should also make sure it pulls in the local recipe listing too (which I think it does).

My proposal is option 2. I don't understand how option 3 would know how to only do it once and I am not much of a fan of doing some sort of bookkeeping to remember this. Please do let us know though if there is some way to manage it. Then, I would go for: * Add `abra catalogue sync` to update catalogue * Ensure catalogue exists but don't pull latest This way, it's less automatic but should be faster for the day-to-day use. If you can't find it in the auto-complete, you know you should `abra catalogue sync`. And the implementation should also make sure it pulls in the local recipe listing too (which I think it does).

decentral1se referenced this issue from a commit

2025-08-18 08:13:32 +00:00

fix: don't ensure latest on auto-complete

decentral1se referenced this issue

2025-08-18 08:15:06 +00:00

fix: make recipe name autocomplete less slow #604

decentral1se commented

2025-08-18 08:15:16 +00:00

RFC: #604

RFC: https://git.coopcloud.tech/toolshed/abra/pulls/604

Sign in to join this conversation.