Differences

This shows you the differences between two versions of the page.

--- packet:bpq_logging_reduction [2024/05/16 08:58] – created g7taj
+++ packet:bpq_logging_reduction [2025/04/14 20:05] (current) – [Newer versions of Raspberry Pi OS (i.e. Debian 12)] m0mzf
@@ Line 1: / Line 1: @@
-LibBPQ logs most bits to /var/log/syslog
+=====Reducing logging writes to a Pi SD card=====
-This also includes some very chatty 'Heartbeats'
+Generally speaking, software running under GNU/Linux writes logs using the [[https://www.man7.org/linux/man-pages/man3/syslog.3.html|syslog]] function which is provided by the standard C library, libc. A logger then attaches to this, and does "things" with the stream of information. Classic loggers include the almost ubiquitous ''rsyslogd'' and others, notably ''syslog-ng'' - but there are many. Typically these programs will write logs to /var/log/blah.log, and these files will then be compressed and "rotated" by another bit of software, 'logrotate'.
-<code>May 13 22:38:39 linbpq_box1 LINBPQ[5905]: BPQ32 Heartbeat Buffers 998 APRS Queues 0 0#015</code>
+Newer Linux distributions use journald, a syslogger built into the love-it-or-hate-it-but-it's-everywhere SystemD. This can also ouput to rsyslogd, for example, if you want the best (or worst!) of both worlds.
+Like most programs LibBPQ logs most bits to syslog; this also includes some very chatty 'Heartbeats'
+<code>May 13 22:38:39 linbpq_box1 LINBPQ[5905]: BPQ32 Heartbeat Buffers 998 APRS Queues 0 0#015</code>
 To save SD card writes, you can re-direct this out to RAM (/run - or somewhere else that is RAM on your system)
+====Older versions of Raspberry Pi OS====
 To do this, you need to create a new file
@@ Line 29: / Line 34: @@
 To use logrotate, to compress the files and only keep 7 days:
-Create a file <code>cat /etc/logrotate.d/linbpq-buffers</code>
+Create a file <code>touch /etc/logrotate.d/linbpq-buffers</code>
 Add this to the file
 <code>/run/linbpq-buffers.log {
@@ Line 38: / Line 43: @@
 }</code>
+====Newer versions of Raspberry Pi OS (i.e. Debian 12)====
+SystemD's journald is a complex beast which works very differently to classic *nix sysloggers. All logs are stored together in binary files and queried with ''journalctl'', instead of being stored as separate files in ''/var/log''. You'll find your journal files in ''/var/log/journal'', see how big they are by running
+<code>
+du -hd1 /var/log/journal
+M	/var/log/journal/94137e38fc9f42bdb4a34b67ee9ce397
+M	/var/log/journal/
+</code>
+They can get extremely large over time, many gigabytes! journald uses some sort of "auto" function to determine exactly how much space to use before "vacuuming" older logs - and it comes up with very large sizes.
+So let's get some control of this monster; edit the following file:
+<code>sudo nano /etc/systemd/journald.conf</code>
+and uncomment / edit the following lines underneath ''[Journal]'':
+<code>
+[Journal]
+Storage=volatile        # This tells journald to store in RAM under /run/log
+SystemMaxUse=200MB      # make sure things don't get out of hand and use all your RAM!
+MaxRetentionSec=1week   # Retain logs for one week before vacuuming
+MaxFileSec=1day         # Rotate log files every day
+ForwardToSyslog=no      # Turn off forwarding to syslog (if installed)
+</code>
+Now that's done restart journald
+<code>sudo systemctl restart systemd-journald.service</code>
+and optionally delete the old journal files
+<code>sudo rm -rf /var/log/journal</code>
+There will still be some stuff ending up in /var/log - on a Debian system this will be logs from apt, dpkg etc, and other packages (ginx, mosquitto etc) by default are configured to write directly to it, but all syslogging will now be in RAM and sizes kept under control. As the syslog is in RAM rebooting the Pi / if the Pi crashes all the logs will be lost. If you want persistent log storage (for example on a Pi 5 with an NVMe SSD) then you can get some control by vacuuming / configuring journald - that's beyond the scope of "save your SD card" so there's some good reading [[https://linuxhandbook.com/clear-systemd-journal-logs/|here]]
+To see what linbpq has been saying run
+<code>journalctl -xeu linbpq.service</code>
+and to watch the live log run
+<code>journalctl -u linbpq.service -n200 -f</code>
+====Writing all linbpq packets to syslog====
+If you really want to you can save out all the packets your station has heard to syslog. With some parsing this could be a useful diagnostic tool; here's one way to do it using [[https://raymii.org/s/software/Fast_MQTT_Logger.html|an MQTT syslogger]]
+To do this you'll need to have LinBPQ publishing it's packets to MQTT. To get that set up [[packet:bpqmonweb|follow this guide]] - might as well go the whole hog and set up the web monitor too.
+Install Paho, download, install and compile the software:
+<code>
+sudo apt install libpaho-mqtt-dev
+mkdir ~/src; cd ~/src
+git clone https://github.com/RaymiiOrg/remys_fast_mqtt_logger
+cd remys_fast_mqtt_logger; mkdir build; cd build
+cmake -DCMAKE_BUILD_TYPE=Release  ..
+</code>
+Now that's done create a script to run it:
+<code>
+nano packet-syslog.sh && chmod +x packet-syslog.sh
+</code>
+and put this in the script, setting your hostname and mosquitto credentials accordingly
+<code>
+#!/bin/bash
+/home/pi/src/remys_fast_mqtt_logger/build/remys_fast_mqtt_logger \
+-b "mosquitto:1883" \
+-t "PACKETNODE/ax25/trace/bpqformat/#" \
+-u "bpqmonweb" \
+-p "bpqmonweb" \
+> /dev/null 2>&1
+</code>
+then create a systemd unit file to run it
+<code>
+sudo nano /etc/systemd/system/packet-syslog.service
+</code>
+and fill it with the following
+<code>
+[Unit]
+Description=Packet Log
+[Service]
+Environment=TERM=linux
+User=root
+ExecStart=-/home/pi/bpqscripts/packet-syslog.sh
+[Install]
+WantedBy=default.target
+</code>
+Then finally start it up
+<code>
+sudo systemctl daemon-reload
+sudo systemctl enable packet-syslog.service
+sudo systemctl start packet-syslog.service
+</code>
+To watch the packets fly by run
+<code>journalctl -u packet-syslog.service -n 1 -f</code>
+Expect this to use up maybe a few MB/day, depending on how busy your node is.