Replace peach-monitor with vnStat / vnstatd #24

Open
opened 2021-11-14 10:03:51 +00:00 by glyph · 2 comments
Owner

Original issue text

peach-monitor currently uses nest to store data usage limits and totals as JSON in the filesystem. Since we're currently aiming to have a single yaml configuration file for PeachCloud, it makes sense for the data associated with peach-monitor to live in that file.

Edited text

I have a better idea: forget about peach-monitor and use vnStat / vnstatd instead.

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources regardless of network traffic rate.

This program is open source / GPL'ed and can be installed either as root or as a single user.

The purpose of vnstatd is to provide a flexible and robust way for updating the database that vnstat(1) uses.

https://humdi.net/vnstat/
https://humdi.net/vnstat/man/vnstatd.html

Advantages of this approach:

  • no programming required
  • no binary compilation or Debian packaging required
  • widely used "old tech" (since 2002)
  • light, minimal resource usage
  • same low cpu usage regardless of traffic
  • can be used without root permissions
  • can be bundled as part of the PeachCloud Debian image (I think)
  • can output database as json or xml
  • includes neat database queries
**Original issue text** `peach-monitor` currently uses [nest](https://crates.io/crates/nest) to store data usage limits and totals as JSON in the filesystem. Since we're currently aiming to have a single `yaml` configuration file for PeachCloud, it makes sense for the data associated with `peach-monitor` to live in that file. **Edited text** I have a better idea: forget about `peach-monitor` and use `vnStat` / `vnstatd` instead. > vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources regardless of network traffic rate. > This program is open source / GPL'ed and can be installed either as root or as a single user. > The purpose of vnstatd is to provide a flexible and robust way for updating the database that vnstat(1) uses. https://humdi.net/vnstat/ https://humdi.net/vnstat/man/vnstatd.html Advantages of this approach: - no programming required - no binary compilation or Debian packaging required - widely used "old tech" (since 2002) - light, minimal resource usage - same low cpu usage regardless of traffic - can be used without root permissions - can be bundled as part of the PeachCloud Debian image (I think) - can output database as json or xml - includes neat database queries
glyph added the
refactor
label 2021-11-14 10:03:56 +00:00
glyph changed title from Refactor peach-monitor to use yaml instead of nest to Replace peach-monitor with vnStat 2021-11-29 09:12:56 +00:00
glyph changed title from Replace peach-monitor with vnStat to Replace peach-monitor with vnstatd 2021-11-29 09:16:05 +00:00
glyph changed title from Replace peach-monitor with vnstatd to Replace peach-monitor with vnStat / vnstatd 2021-11-29 09:18:11 +00:00
Author
Owner

The vnstat database can be outputted as JSON. Here's an example from my local instance:

{"vnstatversion":"2.8","jsonversion":"2","interfaces":[{"name":"eno1","alias":"","created":{"date":{"year":2021,"month":11,"day":29}},"updated":{"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":45}},"traffic":{"total":{"rx":3646266,"tx":5563636},"fiveminute":[{"id":2,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":25},"rx":127763,"tx":251037},{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":30},"rx":246883,"tx":633816},{"id":3,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":35},"rx":471953,"tx":1424034},{"id":4,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":40},"rx":2799667,"tx":3254749}],"hour":[{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":0},"rx":3646266,"tx":5563636}],"day":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}],"month":[{"id":1,"date":{"year":2021,"month":11},"rx":3646266,"tx":5563636}],"year":[{"id":1,"date":{"year":2021},"rx":3646266,"tx":5563636}],"top":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}]}}]}

We can write some Rust code to invoke the command and deserialize the output to a struct (or Vec<> of struct to represent an array of interfaces). This can then be used to create a Tera context object for displaying usage data in peach-web.

I think it makes sense to bundle this functionality into the peach-network library.

The `vnstat` database can be outputted as JSON. Here's an example from my local instance: ```json {"vnstatversion":"2.8","jsonversion":"2","interfaces":[{"name":"eno1","alias":"","created":{"date":{"year":2021,"month":11,"day":29}},"updated":{"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":45}},"traffic":{"total":{"rx":3646266,"tx":5563636},"fiveminute":[{"id":2,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":25},"rx":127763,"tx":251037},{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":30},"rx":246883,"tx":633816},{"id":3,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":35},"rx":471953,"tx":1424034},{"id":4,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":40},"rx":2799667,"tx":3254749}],"hour":[{"id":1,"date":{"year":2021,"month":11,"day":29},"time":{"hour":11,"minute":0},"rx":3646266,"tx":5563636}],"day":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}],"month":[{"id":1,"date":{"year":2021,"month":11},"rx":3646266,"tx":5563636}],"year":[{"id":1,"date":{"year":2021},"rx":3646266,"tx":5563636}],"top":[{"id":1,"date":{"year":2021,"month":11,"day":29},"rx":3646266,"tx":5563636}]}}]} ``` We can write some Rust code to invoke the command and deserialize the output to a `struct` (or `Vec<>` of `struct` to represent an array of interfaces). This can then be used to create a Tera context object for displaying usage data in `peach-web`. I think it makes sense to bundle this functionality into the `peach-network` library.
Author
Owner

The parser is complete: vnstat-parse.

Now we just need to refactor peach-web to use the parser for building the network traffic context object.

As a bonus, we'll be able to offer higher-granularity statistics compared with the previous implementation :) (ie. day, month, all-time traffic totals).

The parser is complete: [vnstat-parse](https://crates.io/crates/vnstat_parse/0.1.0). Now we just need to refactor `peach-web` to use the parser for building the network traffic context object. As a bonus, we'll be able to offer higher-granularity statistics compared with the previous implementation :) (ie. day, month, all-time traffic totals).
Sign in to join this conversation.
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: PeachCloud/peach-workspace#24
No description provided.