Replace peach-monitor with vnStat / vnstatd #24

New Issue

2021-11-14T10:03:51Z

glyph commented

2021-11-14 10:03:51 +00:00

Original issue text

peach-monitor currently uses nest to store data usage limits and totals as JSON in the filesystem. Since we're currently aiming to have a single yaml configuration file for PeachCloud, it makes sense for the data associated with peach-monitor to live in that file.

Edited text

I have a better idea: forget about peach-monitor and use vnStat / vnstatd instead.

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources regardless of network traffic rate.

This program is open source / GPL'ed and can be installed either as root or as a single user.

The purpose of vnstatd is to provide a flexible and robust way for updating the database that vnstat(1) uses.

https://humdi.net/vnstat/
https://humdi.net/vnstat/man/vnstatd.html

Advantages of this approach:

no programming required
no binary compilation or Debian packaging required
widely used "old tech" (since 2002)
light, minimal resource usage
same low cpu usage regardless of traffic
can be used without root permissions
can be bundled as part of the PeachCloud Debian image (I think)
can output database as json or xml
includes neat database queries

**Original issue text** `peach-monitor` currently uses [nest](https://crates.io/crates/nest) to store data usage limits and totals as JSON in the filesystem. Since we're currently aiming to have a single `yaml` configuration file for PeachCloud, it makes sense for the data associated with `peach-monitor` to live in that file. **Edited text** I have a better idea: forget about `peach-monitor` and use `vnStat` / `vnstatd` instead. > vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources regardless of network traffic rate. > This program is open source / GPL'ed and can be installed either as root or as a single user. > The purpose of vnstatd is to provide a flexible and robust way for updating the database that vnstat(1) uses. https://humdi.net/vnstat/ https://humdi.net/vnstat/man/vnstatd.html Advantages of this approach: - no programming required - no binary compilation or Debian packaging required - widely used "old tech" (since 2002) - light, minimal resource usage - same low cpu usage regardless of traffic - can be used without root permissions - can be bundled as part of the PeachCloud Debian image (I think) - can output database as json or xml - includes neat database queries

glyph added the refactor label 2021-11-14 10:03:56 +00:00

glyph changed title from ~~Refactor peach-monitor to use yaml instead of nest~~ to Replace peach-monitor with vnStat

2021-11-29 09:12:56 +00:00

glyph changed title from ~~Replace peach-monitor with vnStat~~ to Replace peach-monitor with vnstatd

2021-11-29 09:16:05 +00:00

glyph changed title from ~~Replace peach-monitor with vnstatd~~ to Replace peach-monitor with vnStat / vnstatd

2021-11-29 09:18:11 +00:00

glyph commented

2021-11-29 10:01:36 +00:00

The vnstat database can be outputted as JSON. Here's an example from my local instance:

{"vnstatversion":"2.8","jsonversion":"2","interfaces":[{"name":"eno1","alias":"","created":{"date":{"year":2021,"month":11,"day":29}},"updated":{"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":45}},"traffic":{"total":{"rx":3646266,"tx":5563636},"fiveminute":[{"id":2,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":25},"rx":127763,"tx":251037},{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":30},"rx":246883,"tx":633816},{"id":3,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":35},"rx":471953,"tx":1424034},{"id":4,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":40},"rx":2799667,"tx":3254749}],"hour":[{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":0},"rx":3646266,"tx":5563636}],"day":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}],"month":[{"id":1,"date":{"year":2021,"month":11},"rx":3646266,"tx":5563636}],"year":[{"id":1,"date":{"year":2021},"rx":3646266,"tx":5563636}],"top":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}]}}]}

We can write some Rust code to invoke the command and deserialize the output to a struct (or Vec<> of struct to represent an array of interfaces). This can then be used to create a Tera context object for displaying usage data in peach-web.

I think it makes sense to bundle this functionality into the peach-network library.

The `vnstat` database can be outputted as JSON. Here's an example from my local instance: ```json {"vnstatversion":"2.8","jsonversion":"2","interfaces":[{"name":"eno1","alias":"","created":{"date":{"year":2021,"month":11,"day":29}},"updated":{"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":45}},"traffic":{"total":{"rx":3646266,"tx":5563636},"fiveminute":[{"id":2,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":25},"rx":127763,"tx":251037},{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":30},"rx":246883,"tx":633816},{"id":3,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":35},"rx":471953,"tx":1424034},{"id":4,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":40},"rx":2799667,"tx":3254749}],"hour":[{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":0},"rx":3646266,"tx":5563636}],"day":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}],"month":[{"id":1,"date":{"year":2021,"month":11},"rx":3646266,"tx":5563636}],"year":[{"id":1,"date":{"year":2021},"rx":3646266,"tx":5563636}],"top":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}]}}]} ``` We can write some Rust code to invoke the command and deserialize the output to a `struct` (or `Vec<>` of `struct` to represent an array of interfaces). This can then be used to create a Tera context object for displaying usage data in `peach-web`. I think it makes sense to bundle this functionality into the `peach-network` library.

glyph commented

2021-12-06 12:48:49 +00:00

The parser is complete: vnstat-parse.

Now we just need to refactor peach-web to use the parser for building the network traffic context object.

As a bonus, we'll be able to offer higher-granularity statistics compared with the previous implementation :) (ie. day, month, all-time traffic totals).

The parser is complete: [vnstat-parse](https://crates.io/crates/vnstat_parse/0.1.0). Now we just need to refactor `peach-web` to use the parser for building the network traffic context object. As a bonus, we'll be able to offer higher-granularity statistics compared with the previous implementation :) (ie. day, month, all-time traffic totals).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: PeachCloud/peach-workspace#24