toolshed/abra
4
2
Fork 18
Code Issues 24 Pull Requests Packages Releases 41 Activity

Disambiguate identical alias strings (e.g. "a" for "app" and "add") #647

New Issue
Closed
opened 2025-09-04 16:19:28 +00:00 by 3wordchant · 13 comments
3wordchant commented 2025-09-04 16:19:28 +00:00
Owner
Copy Link

For example, this string is showing up with 2 translation notes:

  • translators: abra app aliases. use a comma separated list of aliases with no spaces in between
  • translators: abra server add aliases. use a comma separated list of aliases with no spaces in between

So, it's really just a coincidence that both strings are "a" in the English version; both should be translated separately.

It seems like the most gettext-y way to handle this is with "contexts" – in our case, using GetC instead of Get.

For example, [this string](https://translate.coopcloud.tech/translate/co-op-cloud/abra/es/?checksum=70f263be86bc0985&q=has%3Anote&sort_by=-priority%2Cposition) is showing up with 2 translation notes: * translators: `abra app` aliases. use a comma separated list of aliases with no spaces in between * translators: `abra server add` aliases. use a comma separated list of aliases with no spaces in between So, it's really just a coincidence that both strings are "a" in the English version; both should be translated separately. It seems like the most gettext-y way to handle this is with "contexts" – in our case, using [`GetC`](https://pkg.go.dev/github.com/leonelquinteros/gotext#GetC) instead of `Get`.
❤️ 1
3wordchant self-assigned this 2025-09-04 16:19:42 +00:00
3wordchant added this to the Language translation project 2025-09-04 16:19:45 +00:00
3wordchant commented 2025-09-04 16:38:57 +00:00
Author
Owner
Copy Link

Ouch, could be wrong but I think snapd's xgettext-go doesn't support context at all? 🫠

Not seeing it here: https://github.com/canonical/snapd/blob/master/i18n/xgettext-go/main.go

Ouch, could be wrong but I think `snapd`'s `xgettext-go` doesn't support context at all? 🫠 Not seeing it here: https://github.com/canonical/snapd/blob/master/i18n/xgettext-go/main.go
👀 1
decentral1se commented 2025-09-04 17:16:55 +00:00
Owner
Copy Link

Hmmmmm, maybe we need to do like incus in 092990416a/internal/i18n/i18n_linux.go (L9-L12) and wrap our call:

pkg/i18n/i18n.go Line 62 in bddf8039af
var G = LoadLocale().Get

In a function istead of a direct var and do some finagling there? I think you could do string... variable arguments in a function which takes 0 or more strings? There's no optional arguments in Go but you can do it by using what is called the "functional options" approach. I would hope this would be easy to sort... then we could do one of these two?

i18n.G("foo")  # usual suspect
i18n.G("foo", "mycontext")  # string varargs might work but maybe less clear?
i18n.G("foo", i18n.WithContext("mycontext"))  # function options maybe more future-proof also

https://golang.cafe/blog/golang-functional-options-pattern.html

Hmmmmm, maybe we need to do like incus in https://github.com/lxc/incus/blob/092990416a5c2ffc056822e2d2ad62d7ad657204/internal/i18n/i18n_linux.go#L9-L12 and wrap our call: https://git.coopcloud.tech/toolshed/abra/src/commit/bddf8039af31c97eff035b64a582eb26344bc178/pkg/i18n/i18n.go#L62 In a function istead of a direct `var` and do some finagling there? I think you could do `string...` variable arguments in a function which takes 0 or more strings? There's no optional arguments in Go but you can do it by using what is called the "functional options" approach. I would hope this would be easy to sort... then we could do one of these two? ```go i18n.G("foo") # usual suspect i18n.G("foo", "mycontext") # string varargs might work but maybe less clear? i18n.G("foo", i18n.WithContext("mycontext")) # function options maybe more future-proof also ``` https://golang.cafe/blog/golang-functional-options-pattern.html
3wordchant commented 2025-09-04 17:43:18 +00:00
Author
Owner
Copy Link

Yep the function side makes sense, I had tried out this but indeed functional options is maybe cleaner:

func G(s string, options ...string) string {
	if len(options) == 1 {
		return LoadLocale().GetC(s, options[0])
	} else if len(options) > 1 {
		log.Fatalf("Unexpected additional arguments: %v", options[1:])
	}
	return LoadLocale().Get(s)
}

The problem is that, although xgettext-go will pick up all the i18n.G invocations, it won't add the context; we should be seeing an extra line in the POT file.

i.e. with the above, making this change:

diff --git a/cli/upgrade.go b/cli/upgrade.go
index ae6883e5..4f766e80 100644
--- a/cli/upgrade.go
+++ b/cli/upgrade.go
@@ -14,7 +14,7 @@ import (
 
 // translators: `abra upgrade` aliases. use a comma separated list of aliases with
 // no spaces in between
-var upgradeAliases = i18n.G("u")
+var upgradeAliases = i18n.G("u", "upgrade")

Should produce this POT content:

msgid   "u"
msgctxt "upgrade"
msgstr  ""

But, msgctxt doesn't get added.

Yep the function side makes sense, I had tried out this but indeed functional options is maybe cleaner: ```golang func G(s string, options ...string) string { if len(options) == 1 { return LoadLocale().GetC(s, options[0]) } else if len(options) > 1 { log.Fatalf("Unexpected additional arguments: %v", options[1:]) } return LoadLocale().Get(s) } ``` The problem is that, although `xgettext-go` will pick up all the `i18n.G` invocations, it won't add the context; [we should be seeing an extra line in the POT file](https://help.smartling.com/hc/en-us/articles/360007894594-Gettext-PO-POT#Keys-Variants). i.e. with the above, making this change: ```diff diff --git a/cli/upgrade.go b/cli/upgrade.go index ae6883e5..4f766e80 100644 --- a/cli/upgrade.go +++ b/cli/upgrade.go @@ -14,7 +14,7 @@ import ( // translators: `abra upgrade` aliases. use a comma separated list of aliases with // no spaces in between -var upgradeAliases = i18n.G("u") +var upgradeAliases = i18n.G("u", "upgrade") ``` Should produce this POT content: ``` msgid "u" msgctxt "upgrade" msgstr "" ``` But, `msgctxt` doesn't get added.
👀 1
decentral1se commented 2025-09-04 22:13:22 +00:00
Owner
Copy Link

@3wordchant thanks! That makes sense 🧗

Fuck me, we really have been backed into a wall on this one. It's not our fault ofc and is due to the Go ecosystem being so shaky on translation support. I have a work-around which is not actually the worst thing ever but it's pretty damn bad. It's a migration back to xgotext (😱)

We would need to chaos fork it (😱) and override 270f280ba5/cli/xgotext/parser/pkg-tree/golang.go (L88-L90) (i'm guessing) to match coopcloud.tech/abra/pkg/gotext and then define a Get / GetC function there (therefore hacking the gotext.Get/GetC matching but being our version of Get / GetC and not the upstream (😱). We need to do this because we need to initialise the locale once and read the translations from the embedded filesystem and have a central import from there. Then we just sed -i all uses if i18n in the import URLs and function calls...

You could theoretically hack the matching on the function name but I tried to dive into this and ran into an inscrutable list of integers for which there is no explanation for yet https://github.com/leonelquinteros/gotext/issues/124#issuecomment-3211589971 (😱)

I noticed that the -pkg-tree option (instead of -in) on xgotext is much faster so it might not be such a massive slow down for our automation. It is however a noticeably slow down however compared to the zippy xgettext-go...

I could take a run at the migration but this seems like a pretty drastic / nuclear option... especially the need to re-work our CI automation also once more...

@3wordchant thanks! That makes sense 🧗 Fuck me, we really have been backed into a wall on this one. It's not our fault ofc and is due to the Go ecosystem being so shaky on translation support. I have a work-around which is not actually the worst thing ever but it's pretty damn bad. It's a migration back to `xgotext` (😱) We would need to chaos fork it (😱) and override https://github.com/leonelquinteros/gotext/blob/270f280ba5443979a5ddfc6010d874f19b558d1b/cli/xgotext/parser/pkg-tree/golang.go#L88-L90 (i'm guessing) to match `coopcloud.tech/abra/pkg/gotext` and then define a `Get` / `GetC` function there (therefore hacking the `gotext.Get/GetC` matching but being our version of `Get` / `GetC` and not the upstream (😱). We need to do this because we need to initialise the locale once and read the translations from the embedded filesystem and have a central import from there. Then we just `sed -i` all uses if `i18n` in the import URLs and function calls... You could theoretically hack the matching on the function name but I tried to dive into this and ran into an inscrutable list of integers for which there is no explanation for yet https://github.com/leonelquinteros/gotext/issues/124#issuecomment-3211589971 (😱) I noticed that the `-pkg-tree` option (instead of `-in`) on `xgotext` is *much faster* so it might not be such a massive slow down for our automation. It is however a noticeably slow down however compared to the zippy `xgettext-go`... I could take a run at the migration but this seems like a pretty drastic / nuclear option... especially the need to re-work our CI automation also once more...
3wordchant commented 2025-09-05 18:52:29 +00:00
Author
Owner
Copy Link

I tried to dive into this and ran into an inscrutable list of integers for which there is no explanation for yet

Wow full-on magical software numerology incantation 😅

I noticed that the -pkg-tree option (instead of -in) on xgotext is much faster so it might not be such a massive slow down for our automation. It is however a noticeably slow down however compared to the zippy xgettext-go...

Oh very nice 👏

especially the need to re-work our CI automation also once more...

I don't mind doing that part.

Obviously forking does seem pretty last-resort, but I'm not sure what else we do? The only alternative I can think of would be to establish our own (cursed) convention of e.g. u#upgrade in the string, then we strip # and everything after it? Basically rebuilding gettext contexts ourselves.

> I tried to dive into this and ran into an inscrutable list of integers for which there is no explanation for yet Wow full-on magical software numerology incantation 😅 > I noticed that the `-pkg-tree` option (instead of `-in`) on xgotext is much faster so it might not be such a massive slow down for our automation. It is however a noticeably slow down however compared to the zippy xgettext-go... Oh very nice 👏 > especially the need to re-work our CI automation also once more... I don't mind doing that part. Obviously forking does seem pretty last-resort, but I'm not sure what else we do? The only alternative I can think of would be to establish our own (cursed) convention of e.g. `u#upgrade` in the string, then we strip `#` and everything after it? Basically rebuilding gettext contexts ourselves.
decentral1se commented 2025-09-06 06:15:53 +00:00
Owner
Copy Link

@3wordchant hahaha u#uppgrade that would be spectacularly cursed 😂 love it. OK and I guess option 3 is really diving into xgettext-go and trying to patch it also to make it work. Bearing in mind we're running an ancient verison of the software also. I feel like getting back on xgotext is maybe more future-proof as it is the "new kid on the block" but at the same time, the maintainer hasn't replied to my issue and i don't see much activity... so pros/cons everywhere... any preferences? will hopefully have time to dive into this next week unless you take a run at it!

@3wordchant hahaha `u#uppgrade` that would be spectacularly cursed 😂 love it. OK and I guess option 3 is really diving into `xgettext-go` and trying to patch it also to make it work. Bearing in mind we're running an ancient verison of the software also. I feel like getting back on `xgotext` is maybe more future-proof as it is the "new kid on the block" but at the same time, the maintainer hasn't replied to my issue and i don't see much activity... so pros/cons everywhere... any preferences? will hopefully have time to dive into this next week unless you take a run at it!
decentral1se moved this to In Progress in Language translation on 2025-09-06 06:19:26 +00:00
decentral1se commented 2025-09-06 07:42:19 +00:00
Owner
Copy Link

OK, I couldn't resist and have managed to get a working prototype with xgotext and managed to test / patch abra and see the msgctxt turn up! I've submitted a PR. If they don't accept it, we can fork but hopefully we can get this merged upstream. If we want to go in this direction, I can clean it all up and temporarily fork so we can make progress...

OK, I couldn't resist and have managed to get a working prototype with `xgotext` and managed to test / patch `abra` and see the `msgctxt` turn up! I've submitted a [PR](https://github.com/leonelquinteros/gotext/pull/125). If they don't accept it, we can fork but hopefully we can get this merged upstream. If we want to go in this direction, I can clean it all up and temporarily fork so we can make progress...
3wordchant commented 2025-09-06 15:34:46 +00:00
Author
Owner
Copy Link

Incredible 🤩

Incredible 🤩
decentral1se added the
i18n
 label 2025-09-09 19:06:06 +00:00
3wordchant commented 2025-09-09 20:05:44 +00:00
Author
Owner
Copy Link

Decision from Monday: we play the waiting game, hope that leonelquinteros merges the PR by the time we need to launch, decide between various unsavoury forking options if not.

Decision from Monday: we play the waiting game, hope that `leonelquinteros` merges the PR by the time we need to launch, decide between various unsavoury forking options if not.
👍 1
3wordchant commented 2025-09-09 21:44:38 +00:00
Author
Owner
Copy Link

Decision from Monday: we play the waiting game, hope that leonelquinteros merges the PR by the time we need to launch, decide between various unsavoury forking options if not.

Wait I'm confused, this was our plan for #629 - does it apply here too?

> Decision from Monday: we play the waiting game, hope that `leonelquinteros` merges the PR by the time we need to launch, decide between various unsavoury forking options if not. Wait I'm confused, this was our plan for #629 - does it apply here too?
decentral1se commented 2025-09-09 22:11:19 +00:00
Owner
Copy Link

I think for this we do need a temporary fork to make progress and have those translations put in place for testing? Sorry, I think I wasn't that clear... I can arrange this and it's not much work. No reason to hold up the new release of abra now, I'd say. This can be punted into "abra next next" as we improve the translation situation...

I think for this we do need a temporary fork to make progress and have those translations put in place for testing? Sorry, I think I wasn't that clear... I can arrange this and it's not much work. No reason to hold up the new release of `abra` now, I'd say. This can be punted into "abra next next" as we improve the translation situation...
👍 1
decentral1se commented 2025-09-20 21:09:25 +00:00
Owner
Copy Link

Fuck me, xgotext doesn't support comments 😱 I'm going back to a final wildcard attempt to just fork xgettext-go and add context support somehow. I think it might be the best way forward and maybe we'll need to add more stuff also in the future... will be poking at this in the coming days...

Fuck me, `xgotext` doesn't support comments 😱 I'm going back to a final wildcard attempt to just fork `xgettext-go` and add context support somehow. I think it might be the best way forward and maybe we'll need to add more stuff also in the future... will be poking at [this](https://git.coopcloud.tech/toolshed/xgettext-go) in the coming days...
decentral1se referenced this issue from a commit 2025-09-28 08:45:03 +00:00
feat: keyword-ctx
decentral1se referenced this issue from a commit 2025-09-28 10:34:30 +00:00
feat: keyword-ctx
decentral1se referenced this issue from a commit 2025-09-28 10:35:33 +00:00
feat: keyword-ctx
decentral1se referenced this issue from a commit 2025-09-28 10:37:12 +00:00
feat: msgctx support
decentral1se referenced this issue from a commit 2025-09-28 10:38:27 +00:00
feat: msgctx support
decentral1se commented 2025-09-28 10:42:27 +00:00
Owner
Copy Link

OK, I think I've got a lead on a fix: #672

I've just realised that the latest gettext actually supports Go natively 😱 It's not really conveniently available in most distros, so maybe we can hold off on that for now and migrate to it in a few months. Then things should be much simpler. I did compile it from source but it was pretty gnarly, so I'm not sure if worth the effort to docker-ize it right now. Feel free to take a run at it if you want! I assume it supports everything we need as it is the OG upstream.

Serious yak shaving going down here!

OK, I think I've got a lead on a fix: https://git.coopcloud.tech/toolshed/abra/pulls/672 I've just realised that the [latest `gettext`](https://savannah.gnu.org/projects/gettext/) actually supports Go natively 😱 It's not really conveniently available in most distros, so maybe we can hold off on that for now and migrate to it in a few months. Then things should be *much* simpler. I did compile it from source but it was pretty gnarly, so I'm not sure if worth the effort to docker-ize it right now. Feel free to take a run at it if you want! I assume it supports everything we need as it is the OG upstream. Serious yak shaving going down here!
decentral1se referenced this issue from a commit 2025-09-28 10:53:16 +00:00
feat: msgctx support
decentral1se referenced this issue from a commit 2025-09-30 17:09:01 +00:00
feat: msgctxt support
decentral1se moved this to Done in Language translation on 2025-09-30 17:10:45 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is not working
build
go build related issues
ci/cd
Building things with CI/CD
critical fix
https://docs.coopcloud.tech/federation/resolutions/passed/010/
design
UI/UX
documentation
Documenting all the things
duplicate
This issue or pull request already exists
easy-first-issue
Something for new people to get stuck into. We hope it's easy!
enhancement
New feature
help wanted
Need some help
i10n
Everything to do with localisation
i18n
Everything to do with internationalisation
invalid
Something is wrong
question
More information is needed
release
Release management
release-candidate
Related to the new release candidate
security
Security related
test
Unit/integration testing
wontfix
This won't be fixed
No Label
i18n
Milestone
No items
No Milestone
Projects
Clear projects
No project Language translation
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 Apfelwurm carla cas coopcloud cyrnel decentral1se fauno iexos jmakdah2 kawaiipunk knoflook kolaente lambdabundesverband moritz p4u1 ripclap simon sixsmith trav val yksflip
No Assignees 3wordchant
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/abra#647
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?