Optimise compile times and binary sizes #9

New Issue

glyph · 2021-10-29T16:15:33Z

glyph commented

2021-10-29 16:15:33 +00:00

Sharing a few resources here to inform further exploration and discussion.

This is definitely not a major priority, but I would like to refactor everything at some point to reduce compile times and binary sizes (and generally improve the handling and reporting capabilities of PeachCloud).

Rust: Structuring and handling errors in 2020

Response to above article by burntsushi

thiserror crate

anyhow crate

Sharing a few resources here to inform further exploration and discussion. This is definitely not a major priority, but I would like to refactor everything at some point to reduce compile times and binary sizes (and generally improve the handling and reporting capabilities of PeachCloud). [Rust: Structuring and handling errors in 2020](https://nick.groenen.me/posts/rust-error-handling/) [Response to above article by burntsushi](https://www.reddit.com/r/rust/comments/gj8inf/rust_structuring_and_handling_errors_in_2020/fqlmknt/) [thiserror crate](https://crates.io/crates/thiserror) [anyhow crate](https://crates.io/crates/anyhow)

glyph added the

refactor

label 2021-10-29 16:15:44 +00:00

glyph self-assigned this 2021-10-29 16:36:15 +00:00

glyph changed title from ~~Re-evaluate error handling and report~~ to Re-evaluate error handling and reporting

2021-10-30 11:37:35 +00:00

glyph commented

2021-10-30 11:44:53 +00:00

I experimented with refactoring peach-stats today. By replacing serde with miniserde, removing snafu and replacing it with custom error implementations, I was able to achieve the following:

43% reduction in dependencies
20% reduction in compile time
16% reduction in binary size

Total number of dependencies dropped from 224 to 128.

Release build compilation time dropped from 27.31s to 21.97s.

Release build binary size dropped from 8402184 to 7047600 bytes.

After running strip on the post-refactor binary, size dropped to 2716368. That represents a 67% reduction in binary size; from 8 MB to 2.6 MB. If we really want to squeeze the size, we can also tell the compiler to abort on panic (rather than unwinding the stack). This gets us down to 2 MB.

I also upgraded direct dependencies to their latest versions.

The actual code refactoring did not take very long. I think this set of changes could be applied across our codebase with great effect.

I experimented with refactoring peach-stats today. By replacing `serde` with `miniserde`, removing `snafu` and replacing it with custom error implementations, I was able to achieve the following: 43% reduction in dependencies 20% reduction in compile time 16% reduction in binary size Total number of dependencies dropped from 224 to 128. Release build compilation time dropped from 27.31s to 21.97s. Release build binary size dropped from 8402184 to 7047600 bytes. After running `strip` on the post-refactor binary, size dropped to 2716368. That represents a 67% reduction in binary size; from 8 MB to 2.6 MB. If we really want to squeeze the size, we can also tell the compiler to abort on panic (rather than unwinding the stack). This gets us down to 2 MB. I also upgraded direct dependencies to their latest versions. The actual code refactoring did not take very long. I think this set of changes could be applied across our codebase with great effect.

glyph changed title from ~~Re-evaluate error handling and reporting~~ to Optimise compile times and binary sizes

2021-10-30 12:00:20 +00:00

glyph referenced this issue

2021-11-01 12:21:39 +00:00

Miniserde refactor with custom error impls #11

notplants commented

2021-11-02 09:00:45 +00:00

@glyph cool that you got inspired by this rabbit hole, and thanks for sharing the results in this very scientific way!

these seem like really solid gains, and I just looked through the PR, and the interface for working with it as a developer feels just as easy to work with as before

so this seems like a good switch to me too

I'd be curious to understand more what are the features that snafu tries to give you over the standard error implementation, but just out of curiosity

@glyph cool that you got inspired by this rabbit hole, and thanks for sharing the results in this very scientific way! these seem like really solid gains, and I just looked through the PR, and the interface for working with it as a developer feels just as easy to work with as before so this seems like a good switch to me too I'd be curious to understand more what are the features that snafu tries to give you over the standard error implementation, but just out of curiosity

notplants referenced this issue

2021-11-02 09:02:20 +00:00

Miniserde refactor with custom error impls #11

notplants commented

2021-11-02 09:05:18 +00:00

in general I think we should also value "developer efficiency" (ease of working with, understanding and making changes to the code) for the patterns we use, in addition to "compile time and binary size efficiency", but this feels like a win in both categories

glyph commented

2021-11-02 09:32:45 +00:00

@notplants

Thanks for taking a look. I agree on aiming for a balance of developer efficiency and code optimisation. One of the intentions of the PeachCloud codebase is to be an idiomatic, best-practice resource for learning Rust. I'm starting to see that reaching for the easiest-to-use crate is not always best practice; sometimes it's better to take the handwritten route (and you generally learn more in the process).

I'd be curious to understand more what are the features that snafu tries to give you over the standard error implementation, but just out of curiosity

I must admit, I'm still learning all the facets of what makes a "good" error implementation. I think my code in #11 is probably missing a few nice-to-have features.

Snafu have some good docs to learn more about this:

SNAFU's design philosophy
Details about the output of the Snafu macro

@notplants Thanks for taking a look. I agree on aiming for a balance of developer efficiency and code optimisation. One of the intentions of the PeachCloud codebase is to be an idiomatic, best-practice resource for learning Rust. I'm starting to see that reaching for the easiest-to-use crate is not always best practice; sometimes it's better to take the handwritten route (and you generally learn more in the process). > I'd be curious to understand more what are the features that snafu tries to give you over the standard error implementation, but just out of curiosity I must admit, I'm still learning all the facets of what makes a "good" error implementation. I think my code in https://git.coopcloud.tech/PeachCloud/peach-workspace/pulls/11 is probably missing a few nice-to-have features. Snafu have some good docs to learn more about this: [SNAFU's design philosophy](https://docs.rs/snafu/0.6.10/snafu/guide/philosophy/index.html) [Details about the output of the Snafu macro](https://docs.rs/snafu/0.6.10/snafu/guide/the_macro/index.html)

👍 1

glyph commented

2021-11-22 08:57:38 +00:00

This weekend I discovered upx: "a free, portable, extendable, high-performance executable packer for several executable formats".

upx adds decompression logic to the binary itself, meaning that the compressed binary can be executed as normal - without having to first decompress it.

I'm seeing 30-40% reduction in binary size. This could be a huge win for us!

The path, as I currently see it:

Reduce unnecessary dependencies
strip the release build binary
Compress the stripped binary with upx

This weekend I discovered [upx](https://upx.github.io/): "a free, portable, extendable, high-performance executable packer for several executable formats". upx adds decompression logic to the binary itself, meaning that the compressed binary can be executed as normal - without having to first decompress it. I'm seeing 30-40% reduction in binary size. This could be a huge win for us! The path, as I currently see it: 1) Reduce unnecessary dependencies 2) `strip` the release build binary 3) Compress the stripped binary with `upx`

glyph commented

2022-03-08 08:34:23 +00:00

Another great resource:

Tips for Faster Rust Compile Times by Matthias Endler

I intend on refactoring peach-web to use maud and rouille, now that I know for certain that we'll still be able to invoke async golgi functions.

Another great resource: [Tips for Faster Rust Compile Times by Matthias Endler](https://endler.dev/2020/rust-compile-times) I intend on refactoring peach-web to use maud and rouille, now that I know for certain that we'll still be able to invoke async golgi functions.

glyph closed this issue

2022-03-25 12:51:50 +00:00

Sign in to join this conversation.