Support translations in abra #483

Closed
opened 2025-01-09 21:45:36 +00:00 by decentral1se · 14 comments
Owner
https://git.coopcloud.tech/toolshed/organising/issues/640
decentral1se added the
enhancement
label 2025-01-09 21:45:36 +00:00
decentral1se self-assigned this 2025-07-08 10:00:13 +00:00
3wordchant was assigned by decentral1se 2025-07-08 10:00:13 +00:00
decentral1se added this to the Abra v0.11.x project 2025-07-08 10:06:51 +00:00
3wordchant modified the project from Abra v0.11.x to Language translation 2025-07-28 13:47:26 +00:00
3wordchant moved this to To Do in Language translation on 2025-07-28 13:47:37 +00:00
decentral1se moved this to In Progress in Language translation on 2025-08-12 10:46:11 +00:00
decentral1se modified the project from Language translation to Abra v0.11.x 2025-08-12 21:14:59 +00:00
decentral1se moved this to In Progress in Abra v0.11.x on 2025-08-17 13:28:18 +00:00
decentral1se modified the project from Abra v0.11.x to Language translation 2025-08-17 13:28:41 +00:00
decentral1se modified the project from Language translation to Abra v0.11.x 2025-08-17 13:28:47 +00:00
decentral1se moved this to In Progress in Abra v0.11.x on 2025-08-17 13:29:12 +00:00
Author
Owner
Current progress: https://git.coopcloud.tech/toolshed/abra/commits/branch/weblate-first-steps Potential libraries: * https://github.com/nicksnyder/go-i18n * https://github.com/leonelquinteros/gotext * https://pkg.go.dev/golang.org/x/text@v0.27.0/message
Author
Owner

@3wordchant not exactly sure but I think that I need to be added as project admin on weblate to see more of the settings? i guess we can just merge whatever we have now and focus on the main branch. i'm just gonna go with gotext and make a start.

@3wordchant not exactly sure but I think that I need to be added as project admin on weblate to see more of the settings? i guess we can just merge whatever we have now and focus on the main branch. i'm just gonna go with `gotext` and make a start.
decentral1se referenced this issue from a commit 2025-08-19 09:23:21 +00:00
Author
Owner

This is going fine. I struggled a bit to understand how to deal with the format handling when using our logging library / fmt but that is resolved now, I think. Luckily, our unit test suite explodes if I fuck up the gotext.Get call, so that's keeping me in check. I think it will be another couple of hours of work and I'll have it covered.

I'm not sure how the multi-line strings of the Example: "..." / Long: "..." in the cobra CLI command definitions will turn up in the weblate UI but I hope this will be fine!

One thought: we're threading output from the runtime into the deploy realtime status reporting. I think we should probably be catching those outputs and translating stuff we replace the output with. I don't think it will be too much work. In general, I think that output can always be improved, so will take a look at it again.

This is going fine. I struggled a bit to understand how to deal with the format handling when using our logging library / `fmt` but that is resolved now, I think. Luckily, our unit test suite explodes if I fuck up the `gotext.Get` call, so that's keeping me in check. I think it will be another couple of hours of work and I'll have it covered. I'm not sure how the multi-line strings of the `Example: "..."` / `Long: "..."` in the cobra CLI command definitions will turn up in the weblate UI but I hope this will be fine! One thought: we're threading output from the runtime into the deploy realtime status reporting. I think we should probably be catching those outputs and translating stuff we replace the output with. I don't think it will be too much work. In general, I think that output can always be improved, so will take a look at it again.
Author
Owner

@3wordchant cool way to invoke xgotext: 87ebf66f0b/main.go (L3) these "magic comments" then get read by go generate and invoked. kind of nice for helping dev folk run the generation without knowing all the xgotext magic. let's try add this at some stage to cmd/.../main.go 👀

@3wordchant cool way to invoke `xgotext`: https://codeberg.org/tslocum/boxcars/src/commit/87ebf66f0b24489a9d2ce61ae210506bf80e2054/main.go#L3 these "magic comments" then get read by `go generate` and invoked. kind of nice for helping dev folk run the generation without knowing all the `xgotext` magic. let's try add this at some stage to `cmd/.../main.go` 👀
Author
Owner

@3wordchant holy sheyat, witness this additional chaos: 8a2598a524/Makefile (L270-L291) they manage to specify the keyword of the function you use to mark the strings 🤯 I will try to wire up some Makefile magic so there is a "canonical" way of dealing with all the translation tools in-repository and hopefully this will then be possible to coordinate with weblate / drone CI / etc. Working on it...

@3wordchant holy sheyat, witness this additional chaos: https://github.com/lxc/incus/blob/8a2598a524bbe5581e0ce62d2ae99262199a4717/Makefile#L270-L291 they manage to specify the keyword of the function you use to mark the strings 🤯 I will try to wire up some Makefile magic so there is a "canonical" way of dealing with all the translation tools in-repository and hopefully this will then be possible to coordinate with weblate / drone CI / etc. Working on it...
Member

can you add @ChasquiLabo as a translator on weblate?

can you add @ChasquiLabo as a translator on weblate?
Member

does this support translated abra commands?

something like:

abra coso despachar hedgedoc.sutty.abyaya.la
does this support translated abra commands? something like: ```sh abra coso despachar hedgedoc.sutty.abyaya.la ```
Author
Owner

@fauno still too early to do translation work just yet. will add weblate accounts ASAP once the dust settles

it seems that this might be a bit trickier than expected as I just realised that cobra itself (our CLI lib) is still not supporting translation: https://github.com/spf13/cobra/pull/2090 I hope I can override and/or help push this forward

i'm hoping we can translate the sub-commands, yes. working towards this now... do you also want to translate the flags used e.g. --no-domain-checks?

@fauno still too early to do translation work just yet. will add weblate accounts ASAP once the dust settles it seems that this might be a bit trickier than expected as I just realised that cobra itself (our CLI lib) is still not supporting translation: https://github.com/spf13/cobra/pull/2090 I hope I can override and/or help push this forward i'm hoping we can translate the sub-commands, yes. working towards this now... do you also want to translate the flags used e.g. `--no-domain-checks`?
Author
Owner

FYI I've already been running into nearly all the problems described in https://github.com/spf13/cobra/pull/2090#discussion_r1435606770 when making small changes and trying stuff out. I think it makes sense to refactor to use "keys" in the source code and then also have the en translators do the same workflow as the rest. I've seen xgotext / msgmerge already complain about formatting and quotes and such, so I think doing all translation work in a translation editor (poedit / weblate) will be more consistent in the long-term. This required more up-front work now but I'm fine with it.

Back to trying to figure out how to translate sub-commands / flags / etc.

Anyone with experience doing this already, please let me know!

FYI I've already been running into nearly all the problems described in https://github.com/spf13/cobra/pull/2090#discussion_r1435606770 when making small changes and trying stuff out. I think it makes sense to refactor to use "keys" in the source code and then also have the `en` translators do the same workflow as the rest. I've seen `xgotext` / `msgmerge` already complain about formatting and quotes and such, so I think doing all translation work in a translation editor (`poedit` / `weblate`) will be more consistent in the long-term. This required more up-front work now but I'm fine with it. Back to trying to figure out how to translate sub-commands / flags / etc. Anyone with experience doing this already, please let me know!
Author
Owner

OK after some research, I don't see people doing this at all... do we have any references for tools that translate their sub-commands / flags? It seems like a very logical thing to want to do... I've raised https://github.com/spf13/cobra/issues/2301 against the CLI lib we use to see what they think!

OK after some research, I don't see people doing this at all... do we have any references for tools that translate their sub-commands / flags? It seems like a very logical thing to want to do... I've raised https://github.com/spf13/cobra/issues/2301 against the CLI lib we use to see what they think!
Author
Owner

abra coso despachar hedgedoc.sutty.abyaya.la

OK, it's possible! I have a working example with fully translated sub-commands and help output.

I updated the ticket because it is not pretty https://github.com/spf13/cobra/issues/2301#issuecomment-3209828124

Now it's just a question of whether to do a large refactor in our codebase to accomodate this new way to initialise everything so that translation is present OR trying to get cobra to support initialisation in a more convenient way. I will try to judge which is a faster way through when I come back to this tmoro.

> `abra coso despachar hedgedoc.sutty.abyaya.la` OK, it's possible! I have a working example with fully translated sub-commands and help output. I updated the ticket because it is not pretty https://github.com/spf13/cobra/issues/2301#issuecomment-3209828124 Now it's just a question of whether to do a large refactor in our codebase to accomodate this new way to initialise everything so that translation is present OR trying to get cobra to support initialisation in a more convenient way. I will try to judge which is a faster way through when I come back to this tmoro.
Owner

Wow incredible work @decentral1se 🤩

Wow incredible work @decentral1se 🤩
Author
Owner

Thanks!

I think step one is to check if I can initialise the localisation function in a central place in the code base. We need --keyword / --keyword-plural arguments for xgotext first for that because it involves an non-standard import which xgotext needs to recognise. Starting to look into that in https://github.com/leonelquinteros/gotext/issues/124

Then we don't do a tonne of gotext.Get calls but a lang.G (or whatever) call. That is less verbose, xgotext can pick it up and then the localisation (hopefully!) is always initialised when called. If this works, it would make it much less work to translate all the (sub-)commands...

Thanks! I think step one is to check if I can initialise the localisation function in a central place in the code base. We need `--keyword` / `--keyword-plural` arguments for `xgotext` first for that because it involves an non-standard import which `xgotext` needs to recognise. Starting to look into that in https://github.com/leonelquinteros/gotext/issues/124 Then we don't do a tonne of `gotext.Get` calls but a `lang.G` (or whatever) call. That is less verbose, `xgotext` can pick it up and then the localisation (hopefully!) is always initialised when called. If this works, it would make it much less work to translate all the (sub-)commands...
Author
Owner

@3wordchant aight, #608 is merged 🎉 closing this one off.... things we still need to deal with:

Hit me up for a co-work if you like, or lemme know how you get on async whenever time allows.

PS. re: #483 (comment) I ultimately didn't go for the translation keys based on this thread which highlighted a lot of issues you can also run into. also it would be crazy work to replace all strings with a key. it seems like convenience of the developer / default locale is also something to keep in mind... xgettext-go solved all the issues I was facing...

@3wordchant aight, https://git.coopcloud.tech/toolshed/abra/pulls/608 is merged 🎉 closing this one off.... things we still need to deal with: * https://git.coopcloud.tech/toolshed/drone-xgotext/issues/1 * get weblate to deal with the new file setup: https://git.coopcloud.tech/toolshed/abra/src/branch/main/pkg/i18n/locales * `abra.pot` is the "default domain" * all `.po` files sit alongside it in this repository Hit me up for a co-work if you like, or lemme know how you get on async whenever time allows. PS. re: https://git.coopcloud.tech/toolshed/abra/issues/483#issuecomment-26039 I ultimately didn't go for the translation keys based on [this thread](https://elixirforum.com/t/gettext-msgids-keys-or-strings/4528) which highlighted a lot of issues you can also run into. also it would be crazy work to replace all strings with a key. it seems like convenience of the developer / default locale is also something to keep in mind... `xgettext-go` solved all the issues I was facing...
decentral1se moved this to Done in Abra v0.11.x on 2025-08-23 16:05:00 +00:00
Sign in to join this conversation.
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: toolshed/abra#483
No description provided.