Add script to build peach-go-sbot debian package #10

Merged
mhfowler merged 3 commits from peach-go-sbot into main 2021-03-08 09:36:59 +00:00
mhfowler commented 2021-03-03 17:59:15 +00:00 (Migrated from github.com)

There are a few ways to build a debian package, from more low-level to more scripted.

It seems to me the most polished way to build a debian package for a go binary would be to use one of the go-specific tools, as described here https://go-team.pages.debian.net/packaging.html. But it looks to me like this goes into go land a bit. Ways to make a package which include the "source" and the "package".

What I did here was make a simple version, which is just a debian package for a binary executable. I cross-compile go to create the binary executable separately, and then I create a debian package around the binary executables.

The script here,

  • cross-compiles the go-binaries for go-sbot and sbotcli for linux-arm64
  • creates the debian package for these binaries along with the relevant systemd unit for go-sbot
  • adds the debian package to the freight repo

The debian configuration I ended up using was based off of this SO post:
https://unix.stackexchange.com/questions/627689/how-to-create-a-debian-package-from-a-bash-script-and-a-systemd-service

I ended up the using maintainer scripts that were generated by cargo-deb (as for our other services) but then modified for this service.

More to discuss, but this is tested on the pi and working.

There are a few ways to build a debian package, from more low-level to more scripted. It seems to me the most polished way to build a debian package for a go binary would be to use one of the go-specific tools, as described here https://go-team.pages.debian.net/packaging.html. But it looks to me like this goes into go land a bit. Ways to make a package which include the "source" and the "package". What I did here was make a simple version, which is just a debian package for a binary executable. I cross-compile go to create the binary executable separately, and then I create a debian package around the binary executables. The script here, - cross-compiles the go-binaries for go-sbot and sbotcli for linux-arm64 - creates the debian package for these binaries along with the relevant systemd unit for go-sbot - adds the debian package to the freight repo The debian configuration I ended up using was based off of this SO post: https://unix.stackexchange.com/questions/627689/how-to-create-a-debian-package-from-a-bash-script-and-a-systemd-service I ended up the using maintainer scripts that were generated by cargo-deb (as for our other services) but then modified for this service. More to discuss, but this is tested on the pi and working.
mhfowler commented 2021-03-03 18:02:45 +00:00 (Migrated from github.com)

There are also some interesting questions that came up about users, systemd and debian packages.
See this article: https://wiki.debian.org/AccountHandlingInMaintainerScripts

We could consider to modify our other microservices to create their own users that they need,
instead of depending on setup_dev_env to have previously created the users.

Something like this could have advantages in terms of update-ability. Each package doing the necessary installation it needs to run instead of relying on something else. I also like the idea of packages doing their own installation stuff, so that you don't need to make a PR to peach-config at the same time as you make a PR to a microservice, in order for it to work, ideally.

We could also consider to wrap peach-config in a debian package,
which could be useful for thinking about how upgrading works in the future
... as upgrades may very well involve changes to stuff that currently happens in peach-config and not just the microservices
... but just some things that came up and outside the scope of this PR

for this PR, I am using useradd to create a peach-go-sbot user in the postint script,
which then is the user that runs go-sbot via the systemd unit

There are also some interesting questions that came up about users, systemd and debian packages. See this article: https://wiki.debian.org/AccountHandlingInMaintainerScripts We could consider to modify our other microservices to create their own users that they need, instead of depending on setup_dev_env to have previously created the users. Something like this could have advantages in terms of update-ability. Each package doing the necessary installation it needs to run instead of relying on something else. I also like the idea of packages doing their own installation stuff, so that you don't need to make a PR to peach-config at the same time as you make a PR to a microservice, in order for it to work, ideally. We could also consider to wrap peach-config in a debian package, which could be useful for thinking about how upgrading works in the future ... as upgrades may very well involve changes to stuff that currently happens in peach-config and not just the microservices ... but just some things that came up and outside the scope of this PR for this PR, I am using useradd to create a peach-go-sbot user in the postint script, which then is the user that runs go-sbot via the systemd unit
mhfowler commented 2021-03-03 18:04:20 +00:00 (Migrated from github.com)

Lastly, one more issue, it seems that go-sbot and sbotcli expect to run as the same user to work together (at least without some other config which I haven't seen yet, when I run them as different users it throws an error). So we need to think about how other peach services running as other users will interact with go-sbot and sbotcli in terms of user permissions, or maybe talk to cryptix about this

Lastly, one more issue, it seems that go-sbot and sbotcli expect to run as the same user to work together (at least without some other config which I haven't seen yet, when I run them as different users it throws an error). So we need to think about how other peach services running as other users will interact with go-sbot and sbotcli in terms of user permissions, or maybe talk to cryptix about this
mycognosist (Migrated from github.com) reviewed 2021-03-08 08:34:12 +00:00
mycognosist (Migrated from github.com) commented 2021-03-08 08:34:12 +00:00

Small typo here: "This build ..." -> "This builds...".

Please capitalize Debian and Freight so we can keep uniformity in the docs 🙏🏻 It's such a silly thing but is somehow highly satisfying to my brain.

Small typo here: "This build ..." -> "This builds...". Please capitalize Debian and Freight so we can keep uniformity in the docs 🙏🏻 It's such a silly thing but is somehow highly satisfying to my brain.
mycognosist (Migrated from github.com) requested changes 2021-03-08 08:45:57 +00:00
mycognosist (Migrated from github.com) left a comment

Looks great overall! I'm excited to try this. I've left a few comments. The "requested changes" is mostly about the doc consistency (comment above). No other changes are needed if the URL stuff I pointed out isn't an issue.

Looks great overall! I'm excited to try this. I've left a few comments. The "requested changes" is mostly about the doc consistency (comment above). No other changes are needed if the URL stuff I pointed out isn't an issue.
@ -0,0 +2,4 @@
set -e
# create user which go-sbot runs as
adduser --quiet --system peach-go-sbot
mycognosist (Migrated from github.com) commented 2021-03-08 08:36:26 +00:00

This is so nice <3 Being able to create the user and set binary permissions right here is awesome. Great find! This is going to make our whole installation and system maintenance process so much simpler.

This is so nice <3 Being able to create the user and set binary permissions right here is awesome. Great find! This is going to make our whole installation and system maintenance process so much simpler.
@ -0,0 +22,4 @@
def crosscompile_peach_go_sbot():
subprocess.check_call(["git", "pull"], cwd=GO_SSB_DIR)
# TODO: confirm that version number in the repo matches the version number we set for the package we are building
mycognosist (Migrated from github.com) commented 2021-03-08 08:39:11 +00:00

Sounds like a job for Regex :)

Sounds like a job for Regex :)
@ -0,0 +47,4 @@
render_template(src=src, dest=dest, template_vars={"version": version})
# copy systemd service file
SERVICE_DIR = os.path.join(DEB_BUILD_DIR, 'lib/systemd/system')
mycognosist (Migrated from github.com) commented 2021-03-08 08:43:22 +00:00

Is SERVICE_DIR missing a forward slash?

DEB_BUILD_DIR = "/tmp/peach_go_sbot" and we're appending "lib/systemd/system", which would make SERVICE_DIR =

/tmp/peach_go_sbotlib/systemd/system

Maybe that's intended and I'm missing something?

Is `SERVICE_DIR` missing a forward slash? `DEB_BUILD_DIR` = "/tmp/peach_go_sbot" and we're appending "lib/systemd/system", which would make `SERVICE_DIR` = `/tmp/peach_go_sbotlib/systemd/system` Maybe that's intended and I'm missing something?
mycognosist (Migrated from github.com) commented 2021-03-08 08:43:58 +00:00

Same thing appears on line 59 for BIN_DIR.

Same thing appears on line 59 for `BIN_DIR`.
mhfowler (Migrated from github.com) reviewed 2021-03-08 09:28:31 +00:00
@ -0,0 +47,4 @@
render_template(src=src, dest=dest, template_vars={"version": version})
# copy systemd service file
SERVICE_DIR = os.path.join(DEB_BUILD_DIR, 'lib/systemd/system')
mhfowler (Migrated from github.com) commented 2021-03-08 09:28:31 +00:00

I just confirmed in a python shell, os.path.join inserts the "/" in between parts as needed.
the way I think about it, the second part is like a relative path joined to the first part (and so the second part doesn't start with a slash)

I just confirmed in a python shell, os.path.join inserts the "/" in between parts as needed. the way I think about it, the second part is like a relative path joined to the first part (and so the second part doesn't start with a slash)
mhfowler (Migrated from github.com) reviewed 2021-03-08 09:29:57 +00:00
@ -0,0 +22,4 @@
def crosscompile_peach_go_sbot():
subprocess.check_call(["git", "pull"], cwd=GO_SSB_DIR)
# TODO: confirm that version number in the repo matches the version number we set for the package we are building
mhfowler (Migrated from github.com) commented 2021-03-08 09:29:57 +00:00

Just added a github issue for this so we don't forget it https://github.com/peachcloud/peach-vps/issues/11

Just added a github issue for this so we don't forget it https://github.com/peachcloud/peach-vps/issues/11
mhfowler commented 2021-03-08 09:34:49 +00:00 (Migrated from github.com)

@mycognosist just pushed a commit fixing the capitalization and typo and will merge this now.

Will try to be more mindful and feel free to keep reminding me in future PRs, I'm happy to make docs more consistent.

@mycognosist just pushed a commit fixing the capitalization and typo and will merge this now. Will try to be more mindful and feel free to keep reminding me in future PRs, I'm happy to make docs more consistent.
mycognosist (Migrated from github.com) reviewed 2021-03-08 09:35:15 +00:00
@ -0,0 +47,4 @@
render_template(src=src, dest=dest, template_vars={"version": version})
# copy systemd service file
SERVICE_DIR = os.path.join(DEB_BUILD_DIR, 'lib/systemd/system')
mycognosist (Migrated from github.com) commented 2021-03-08 09:35:15 +00:00

Ok awesome, thanks for checking! I figured it must work somehow, since you've already been using the script successfully. Nice to have learned a little extra about os.path.join.

Ok awesome, thanks for checking! I figured it must work somehow, since you've already been using the script successfully. Nice to have learned a little extra about `os.path.join`.
Sign in to join this conversation.
No description provided.