Allowing user-defined settings for go-sbot #16

New Issue

cblgh · 2021-03-19T09:15:25Z

mycognosist commented

2021-03-19 09:15:25 +00:00

(Migrated from github.com)

This might not be the best place for this topic but I think it's OK for now, since it might alter the build process for peach-go-sbot.

While playing with go-sbot and looking at all the ways to modify its behaviour using flags, I found myself wondering how we might allow for peach caretakers to alter the behaviour of their go-sbot instances in PeachCloud.

As a simple example: the hops count currently defaults to 1 but can be set to 2 by passing the -hops 2 flag (go-sbot -hops 2). Other examples include setting a non-standard network key (to create a cypherverse separate from the Scuttleverse), setting a custom mountpoint for the .ssb-go directory, or enabling local network discovery by listening for UDP packets and connecting to the source server.

Defaults for all of the above examples, and more, are defined in ssb/main.go.

Let's imagine an 'SSB Server Configuration' page on the peach-web interface which allows these various settings to be modified; how might we go about storing the settings and applying them to our go-sbot instance?

Currently we use a systemd service file which simply calls /usr/bin/go-sbot. One possible solution would be to create an argument config file defining all of the go-sbot flags and import it into the service file using the EnvironmentFile parameter (as described in this superuser post).

So we would have a config file (for example, /etc/.go-sbot.conf) with a list of flags:

DISCOV=-localdiscov true
EBT=-enable-ebt true

And we'd pass them to go-sbot as arguments in the service file:

EnvironmentFile=/etc/.go-sbot.conf
ExecStart = /usr/bin/go-sbot $DISCOV $EBT

With this approach, I imagine any changes to the settings on the 'SSB Server Configuration' page would result in the following events: 1) stop the peach-go-sbot service, 2) write the changes to the /etc/.go-sbot.conf file, 3) start the peach-go-sbot service. We could keep a backup of the default settings at /etc/.go-sbot.conf.bak, which would allow us to have a Reset to default option in the web interface (overwrite /etc/.go-sbot.conf with /etc/.go-sbot.conf.bak).

@mhfowler I'm interested to hear your thoughts on this. Maybe you've tackled a similar problem in the past? I'm going to do some more research to see what other approaches there are. I imagine it also wouldn't be difficult to fork go-sbot and customise main.go to read the flags directly from a config file (instead of parsing them from arguments). Then we could continue to run /usr/bin/go-sbot in the service file and alter the conf file as needed.

_This might not be the best place for this topic but I think it's OK for now, since it might alter the build process for `peach-go-sbot`_. While playing with go-sbot and looking at all the ways to modify its behaviour using flags, I found myself wondering how we might allow for peach caretakers to alter the behaviour of their go-sbot instances in PeachCloud. As a simple example: the hops count currently defaults to 1 but can be set to 2 by passing the `-hops 2` flag (`go-sbot -hops 2`). Other examples include setting a non-standard network key (to create a cypherverse separate from the Scuttleverse), setting a custom mountpoint for the `.ssb-go` directory, or enabling local network discovery by listening for UDP packets and connecting to the source server. Defaults for all of the above examples, and more, are defined in [ssb/main.go](https://github.com/cryptoscope/ssb/blob/master/cmd/go-sbot/main.go). Let's imagine an 'SSB Server Configuration' page on the `peach-web` interface which allows these various settings to be modified; how might we go about storing the settings and applying them to our `go-sbot` instance? Currently we use a `systemd` service file which simply calls `/usr/bin/go-sbot`. One possible solution would be to create an argument config file defining all of the `go-sbot` flags and import it into the service file using the `EnvironmentFile` parameter (as described in [this superuser post](https://superuser.com/a/728962)). So we would have a config file (for example, `/etc/.go-sbot.conf`) with a list of flags: ```bash DISCOV=-localdiscov true EBT=-enable-ebt true ``` And we'd pass them to `go-sbot` as arguments in the service file: ```bash EnvironmentFile=/etc/.go-sbot.conf ExecStart = /usr/bin/go-sbot $DISCOV $EBT ``` With this approach, I imagine any changes to the settings on the 'SSB Server Configuration' page would result in the following events: 1) stop the `peach-go-sbot` service, 2) write the changes to the `/etc/.go-sbot.conf` file, 3) start the `peach-go-sbot` service. We could keep a backup of the default settings at `/etc/.go-sbot.conf.bak`, which would allow us to have a `Reset to default` option in the web interface (overwrite `/etc/.go-sbot.conf` with `/etc/.go-sbot.conf.bak`). @mhfowler I'm interested to hear your thoughts on this. Maybe you've tackled a similar problem in the past? I'm going to do some more research to see what other approaches there are. I imagine it also wouldn't be difficult to fork `go-sbot` and customise `main.go` to read the flags directly from a config file (instead of parsing them from arguments). Then we could continue to run `/usr/bin/go-sbot` in the service file and alter the conf file as needed.

mhfowler commented

2021-03-19 10:09:46 +00:00

(Migrated from github.com)

@mycognosist this makes sense to me and sounds good.

one idea: for resetting to defaults, instead of keeping the default settings in /etc/.go-sbot.conf.bak, I would keep the default settings as a file that is managed and versioned by peach-web. Like when you click "default" in peach-web perhaps it should be peach-web's interpretation what that means.

I think it would be nice to have clarity over which files are managed by peachcloud and which are not. I see why based on debian guidelines .go-sbot.conf would live in /etc, but I'm also suspicious of having files in many different locations which are read and updated by peach microservices.

It still seems to me like it would be a nice design if the whole peachcloud state lived within a single folder. So that when you go into that folder you can see the entirety of State. For clarity, simplicity and making backups. One thing I appreciate about the scuttlebutt design is all the data and configuration lives in ~.ssb, and I can just go in there and check it out. And the ease with which I can just move in a different ssb folder to the ~/.ssb location and everything works.

I wonder if we could make the design constraint that /var/lib/peachcloud is this folder. I can see this goes against the debian recommendation to have config files in /etc , but maybe since the go-sbot config files are managed by an application its also a slightly different situation? I'm not entirely sure, but I'm not really sure what is gained by spreading the files around as opposed to having all state in one folder, other than meeting the debian convention

@mycognosist this makes sense to me and sounds good. one idea: for resetting to defaults, instead of keeping the default settings in /etc/.go-sbot.conf.bak, I would keep the default settings as a file that is managed and versioned by peach-web. Like when you click "default" in peach-web perhaps it should be peach-web's interpretation what that means. I think it would be nice to have clarity over which files are managed by peachcloud and which are not. I see why based on debian guidelines .go-sbot.conf would live in /etc, but I'm also suspicious of having files in many different locations which are read and updated by peach microservices. It still seems to me like it would be a nice design if the whole peachcloud state lived within a single folder. So that when you go into that folder you can see the entirety of State. For clarity, simplicity and making backups. One thing I appreciate about the scuttlebutt design is all the data and configuration lives in ~.ssb, and I can just go in there and check it out. And the ease with which I can just move in a different ssb folder to the ~/.ssb location and everything works. I wonder if we could make the design constraint that /var/lib/peachcloud is this folder. I can see this goes against the debian recommendation to have config files in /etc , but maybe since the go-sbot config files are managed by an application its also a slightly different situation? I'm not entirely sure, but I'm not really sure what is gained by spreading the files around as opposed to having all state in one folder, other than meeting the debian convention

mhfowler commented

2021-03-19 10:14:13 +00:00

(Migrated from github.com)

also fwiw, as you suggested, it does sound nicer to make go-sbot itself have the capability to take in a conf file, than to convert a conf file into CLI arguments via env variables, but the conversion sounds like a practical solution that could be used in the meantime if we don't want to go into go land.

or instead of making a fork of go-sbot, we could make a PR? it sounds like something that would be useful to many people I imagine

also imagine it would be relatively trivial to implement if either of us had the most basic go knowledge

also fwiw, as you suggested, it does sound nicer to make go-sbot itself have the capability to take in a conf file, than to convert a conf file into CLI arguments via env variables, but the conversion sounds like a practical solution that could be used in the meantime if we don't want to go into go land. or instead of making a fork of go-sbot, we could make a PR? it sounds like something that would be useful to many people I imagine also imagine it would be relatively trivial to implement if either of us had the most basic go knowledge

mycognosist commented

2021-03-19 10:35:00 +00:00

(Migrated from github.com)

@mhfowler

It still seems to me like it would be a nice design if the whole peachcloud state lived within a single folder.

Totally agree with you on this one! I'm onboard with your idea of having /var/lib/peachcloud be the path of that folder. Having to dig through multiple directories to find all the config sounds awful.

it does sound nicer to make go-sbot itself have the capability to take in a conf file ... we could make a PR?

Cool, agreed that this sounds really nice. I'm already running into unexplained behaviour while trying to implement the EnvironmentFile solution outlined above, so adding the conf file feature to go-sbot seems like a good move. There is already precedent for this approach in SSB (with .ssb/config containing a JSON representation of settings).

I'm keen to dip my toes into Go land so I'll work on the PR.

I would keep the default settings as a file that is managed and versioned by peach-web. Like when you click "default" in peach-web perhaps it should be peach-web's interpretation what that means.

Hmm I'm not too sure about this approach. My impulse is to maintain a separation of concerns and have the settings file bundled with peach-go-sbot. This ensures that peach-go-sbot has everything it needs to be deployed as a standalone program (want to run an SSB pub on arm64 without any peachiness? No problem, just apt install peach-go-sbot and you're good to go).

That approach would still allow for peach-web to define the behaviour of the 'reset to default' function (ie. write the following values to the file...).

I'm going to mull over this some more today. Might come to a different conclusion during my afternoon walk :)

@mhfowler > It still seems to me like it would be a nice design if the whole peachcloud state lived within a single folder. Totally agree with you on this one! I'm onboard with your idea of having `/var/lib/peachcloud` be the path of that folder. Having to dig through multiple directories to find all the config sounds awful. > it does sound nicer to make go-sbot itself have the capability to take in a conf file ... we could make a PR? Cool, agreed that this sounds really nice. I'm already running into unexplained behaviour while trying to implement the `EnvironmentFile` solution outlined above, so adding the conf file feature to `go-sbot` seems like a good move. There is already precedent for this approach in SSB (with `.ssb/config` containing a JSON representation of settings). I'm keen to dip my toes into Go land so I'll work on the PR. > I would keep the default settings as a file that is managed and versioned by peach-web. Like when you click "default" in peach-web perhaps it should be peach-web's interpretation what that means. Hmm I'm not too sure about this approach. My impulse is to maintain a separation of concerns and have the settings file bundled with `peach-go-sbot`. This ensures that `peach-go-sbot` has everything it needs to be deployed as a standalone program (want to run an SSB pub on arm64 without any peachiness? No problem, just `apt install peach-go-sbot` and you're good to go). That approach would still allow for `peach-web` to define the behaviour of the 'reset to default' function (ie. write the following values to the file...). I'm going to mull over this some more today. Might come to a different conclusion during my afternoon walk :)

mhfowler commented

2021-03-19 14:03:52 +00:00

(Migrated from github.com)

@mycognosist

maintain a separation of concerns and have the settings file bundled with peach-go-sbot

I could see arguments in either direction to ponder while you walk :) I imagine if the defaults are bundled as part of peach-go-sbot or peach-web, if its part of something peachcloud configures and is not external to peachland (my main concern) then its not a big deal either way

and cool about working on the go implementation and single-folder-state-plan

@mycognosist > maintain a separation of concerns and have the settings file bundled with peach-go-sbot I could see arguments in either direction to ponder while you walk :) I imagine if the defaults are bundled as part of peach-go-sbot or peach-web, if its part of something peachcloud configures and is not external to peachland (my main concern) then its not a big deal either way and cool about working on the go implementation and single-folder-state-plan

mycognosist commented

2021-03-19 15:51:19 +00:00

(Migrated from github.com)

@mhfowler

I hear what you're saying about wanting to keep the defaults bundled. My walk ended up being a bicycle ride and I think I have a middle-ground:

The default config file is bundled with peach-go-sbot. This allows it to be run as a standalone pub with defaults befitting life on a Debian arm64 system. peach-web then provides a friendly interface for changing the runtime settings of peach-go-sbot. Those changes are made programmatically by writing to the config file (could be a JSON file). The restore default option in peach-web works by writing the default settings (as defined in the peach-web code).

I don't foresee the defaults needing to be different for peach-go-sbot and peach-web, but this at least gives us a way to achieve that.

Thinking further ahead: peach-web could include a start-up wizard which allows the caretaker to design the garden (as it were). Part of the start-up wizard could include the settings for peach-go-sbot. The caretaker could go with the default design (SSB main-network, EBT-enabled etc. - all with easy-to-follow language) or they could customise their design (alt-network key, USB flashdrive mountpoint, no local network peer discovery etc.). The peach-go-sbot service would then only be started once the design parameters had been entered. This feels in tune with the PeachCloud design approach of providing "an opinionated foundation to build a flexible ecosystem".

I'm getting quite excited thinking about this!

@mhfowler I hear what you're saying about wanting to keep the defaults bundled. My walk ended up being a bicycle ride and I think I have a middle-ground: The default config file is bundled with `peach-go-sbot`. This allows it to be run as a standalone pub with defaults befitting life on a Debian `arm64` system. `peach-web` then provides a friendly interface for changing the runtime settings of `peach-go-sbot`. Those changes are made programmatically by writing to the config file (could be a `JSON` file). The `restore default` option in `peach-web` works by writing the `default` settings (as defined in the `peach-web` code). I don't foresee the defaults needing to be different for `peach-go-sbot` and `peach-web`, but this at least gives us a way to achieve that. Thinking further ahead: `peach-web` could include a start-up wizard which allows the caretaker to design the garden (as it were). Part of the start-up wizard could include the settings for `peach-go-sbot`. The caretaker could go with the default design (SSB main-network, EBT-enabled etc. - all with easy-to-follow language) or they could customise their design (alt-network key, USB flashdrive mountpoint, no local network peer discovery etc.). The `peach-go-sbot` service would then only be started once the design parameters had been entered. This feels in tune with the PeachCloud design approach of providing "an opinionated foundation to build a flexible ecosystem". I'm getting quite excited thinking about this!

👍 1

cblgh referenced this issue from PeachCloud/go-sbotcli-rs

2021-12-02 10:10:17 +00:00

go-ssb tasks for the peachpub grant #9

Sign in to join this conversation.