Online Amateur Radio Community Wiki

Generally speaking, software running under GNU/Linux writes logs using the syslog function which is provided by the standard C library, libc. A logger then attaches to this, and does “things” with the stream of information. Classic loggers include the almost ubiquitous rsyslogd and others, notably syslog-ng - but there are many. Typically these programs will write logs to /var/log/blah.log, and these files will then be compressed and “rotated” by another bit of software, 'logrotate'.

Newer Linux distributions use journald, a syslogger built into the love-it-or-hate-it-but-it's-everywhere SystemD. This can also ouput to rsyslogd, for example, if you want the best (or worst!) of both worlds.

Like most programs LibBPQ logs most bits to syslog; this also includes some very chatty 'Heartbeats'

May 13 22:38:39 linbpq_box1 LINBPQ[5905]: BPQ32 Heartbeat Buffers 998 APRS Queues 0 0#015

To save SD card writes, you can re-direct this out to RAM (/run - or somewhere else that is RAM on your system)

To do this, you need to create a new file

/etc/rsyslog.d/00-linbpq-buffers.conf

and add to it:

if $msg contains 'BPQ32 Heartbeat' then /run/linbpq-buffers.log
& STOP

( note, if you are on an older version on rsyslog, you need & ~ instead of & STOP )

Then restart rsyslogd

sudo systemctl restart rsyslogd

This may fill up your RAM, so you can do several things

• Create a CRON job to delete this file (or archive it etc)
• use logrotate

To use logrotate, to compress the files and only keep 7 days:

Create a file

touch /etc/logrotate.d/linbpq-buffers

Add this to the file

/run/linbpq-buffers.log {
     rotate 7
     daily
     missingok
     compress
}

SystemD's journald is a complex beast which works very differently to classic *nix sysloggers. All logs are stored together in binary files and queried with journalctl, instead of being stored as separate files in /var/log. You'll find your journal files in /var/log/journal, see how big they are by running

du -hd1 /var/log/journal
153M	/var/log/journal/94137e38fc9f42bdb4a34b67ee9ce397
153M	/var/log/journal/

They can get extremely large over time, many gigabytes! journald uses some sort of “auto” function to determine exactly how much space to use before “vacuuming” older logs - and it comes up with very large sizes.

So let's get some control of this monster; edit the following file:

sudo nano /etc/systemd/journald.conf

and uncomment / edit the following lines underneath [Journal]:

[Journal]
Storage=volatile        # This tells journald to store in RAM under /run/log
SystemMaxUse=200MB      # make sure things don't get out of hand and use all your RAM!
MaxRetentionSec=1week   # Retain logs for one week before vacuuming
MaxFileSec=1day         # Rotate log files every day
ForwardToSyslog=no      # Turn off forwarding to syslog (if installed)

Now that's done restart journald

sudo systemctl restart systemd-journald.service

and optionally delete the old journal files

sudo rm -rf /var/log/journal

There will still be some stuff ending up in /var/log - on a Debian system this will be logs from apt, dpkg etc, but all syslogging will now be in RAM and sizes kept under control. As the syslog is in RAM rebooting the Pi / if the Pi crashes all the logs will be lost. If you want persistent log storage (for example on a Pi 5 with an NVMe SSD) then you can get some control by vacuuming / configuring journald - that's beyond the scope of “save your SD card” so there's some good reading here

To see what linbpq has been saying run

journalctl -xeu linbpq.service

and to watch the live log run

journalctl -u linbpq.service -n200 -f

If you really want to you can save out all the packets your station has heard to syslog. With some parsing this could be a useful diagnostic tool; here's one way to do it using an MQTT syslogger

To do this you'll need to have LinBPQ publishing it's packets to MQTT. To get that set up follow this guide - might as well go the whole hog and set up the web monitor too.

Install Paho, download, install and compile the software:

sudo apt install libpaho-mqtt-dev
mkdir ~/src; cd ~/src
git clone https://github.com/RaymiiOrg/remys_fast_mqtt_logger
cd remys_fast_mqtt_logger
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release  ..

Now that's done create a script to run it:

nano packet-syslog.sh && chmod +x packet-syslog.sh

and put this in the script, setting your hostname and mosquitto credentials accordingly

#!/bin/bash
/home/pi/src/remys_fast_mqtt_logger/build/remys_fast_mqtt_logger \
-b "mosquitto:1883" \
-t "PACKETNODE/ax25/trace/bpqformat/#" \
-u "bpqmonweb" \
-p "bpqmonweb" \
> /dev/null 2>&1

then create a systemd unit file to run it

sudo nano /etc/systemd/system/packet-syslog.service

and fill it with the following

[Unit]
Description=Packet Log

[Service]
Environment=TERM=linux
User=root
ExecStart=-/home/pi/bpqscripts/packet-syslog.sh

[Install]
WantedBy=default.target

Then finally start it up

sudo systemctl daemon-reload
sudo systemctl enable packet-syslog.service
sudo systemctl start packet-syslog.service

To watch the packets fly by run

journalctl -u packet-syslog.service -n 1 -f

Expect this to use up maybe a dozen MB/day, depending on how busy your node is.