Pronix Information Technologies
Docs

Operations

Log rotation

The daemon handles both standard log rotation patterns:

  • Rotate-and-rename (the access log is renamed, a new file takes its place). Detected via inode/device change on the path the daemon is watching.
  • Copytruncate (the file content is copied elsewhere then the original is truncated to zero). Detected via tell() > stat().

On either event the daemon reopens the file from the beginning and logs an info reopen after rotate dev=… ino=… (rotate-and-rename) or info truncate file shrank (copytruncate) message to stderr.

If you observe missing entries around rotation time, check your rotation tool config:

  • On OpenBSD newsyslog, the default rotation is rotate-and-rename — no special configuration needed.
  • On Linux logrotate, prefer the default (rotate then SIGHUP the web server) over copytruncate mode. Both work, but rotate-and-rename is cheaper and loses no lines.

rc.d script (OpenBSD)

A minimal /etc/rc.d/abusive_http_watch:

#!/bin/ksh
daemon="/usr/local/sbin/abusive_http_watch"
daemon_flags="/var/www/logs/access.log"
daemon_user="_www"
. /etc/rc.d/rc.subr
rc_bg=YES
rc_reload=NO
rc_cmd $1

Then:

doas rcctl enable abusive_http_watch
doas rcctl start abusive_http_watch
doas rcctl check abusive_http_watch

On Linux with systemd, see your distribution’s documentation; a simple Type=simple unit pointing at /usr/local/sbin/abusive_http_watch with the same arguments works.

Syslog events

The daemon emits one event class — added — when a new IP crosses the hit threshold. Format:

<syslog-tag>[<pid>]: added ip=<ip> path=<blocklist-path>

Example:

abusive_http_watch[12345]: added ip=198.51.100.42 path=/var/www/run/abusive_http_hosts

Route these via syslog.conf to a dedicated file for review:

local0.notice                                          /var/log/abusive_http.log

Combined with ABUSIVE_SYSLOG_FACILITY=local0, this gives a clean audit log of every IP the daemon has blocked.

Monitoring signals

For automated monitoring, two indicators are useful:

  • Blocklist growth rate — wc -l /var/www/run/abusive_http_hosts over time. On a public web server this should grow steadily (several IPs per day to many per hour, depending on exposure). A flatline for 24h on a public site suggests the daemon is not running or is reading the wrong log.
  • Process health — pgrep -f abusive_http_watch should always return one PID. Alert if zero.

For systems with Prometheus, a tiny exporter that just wc -ls the blocklist file and exposes the count as a gauge is enough. No metrics endpoint exists in the daemon itself — by design, to keep it dependency-free.

Troubleshooting

The blocklist file stays empty

Possible causes, in order of likelihood:

  1. Daemon is reading the wrong file. Run with ABUSIVE_DEBUG=1 in the foreground. Confirm it logs start following dev=… ino=… for the path you expect. If not, fix the argument.
  2. No actual attack traffic. Unlikely on a public web server, but possible on an internal one. Run with ABUSIVE_DEBUG=1 and look for [dbg] ip=… lines without matching [dbg] token match.
  3. Whitelist swallows everything. Check /etc/abusive_http_whitelist — a stray /0 CIDR entry whitelists the entire internet. Remove and restart.
  4. Hit threshold too high. Lower ABUSIVE_MIN_HITS to 1 temporarily and watch behaviour.

The blocklist grows but the firewall does not block

The daemon writes the file. The firewall has to be told to read it. Check:

  • Cron is reloading the pf table / nftables set every few minutes (the daemon does not call pfctl itself).
  • The firewall rule referencing the table is in the right chain and is quick enough to take effect before any later pass rule.

Legitimate users are being blocked

Add their public IP or CIDR range to /etc/abusive_http_whitelist. The daemon re-reads the whitelist on every line, so changes take effect within seconds.

To clear a single IP from the existing blocklist, edit the file directly and reload the firewall table. The daemon does not need a restart — it will simply not re-add the IP unless the user matches tokens again.

Daemon crashes after log rotation

Should not happen given the rotation handling described above. If it does, run the daemon in the foreground with ABUSIVE_DEBUG=1, trigger a rotation, and capture the stderr. Send the captured output via the Contact page — that is a real bug worth fixing.

Graceful shutdown

The daemon traps SIGTERM and exits cleanly after the current iteration:

doas rcctl stop abusive_http_watch

Logs [info] TERM received, exiting, closes the log file handle, flushes any pending writes, and exits with status 0. No state corruption, even on a busy log.

Prev
Operations