# OperationsRunning the daemon in production: log rotation, syslog, daemon lifecycle, and known failure modes.

## Log rotation

The daemon handles both standard log rotation patterns:

- **Rotate-and-rename** (the access log is renamed, a new file
  takes its place). Detected via inode/device change on the path
  the daemon is watching.
- **Copytruncate** (the file content is copied elsewhere then the
  original is truncated to zero). Detected via `tell() > stat()`.

On either event the daemon reopens the file from the beginning and
logs an `info reopen after rotate dev=… ino=…` (rotate-and-rename)
or `info truncate file shrank` (copytruncate) message to stderr.

If you observe missing entries around rotation time, check
your rotation tool config:

- On OpenBSD `newsyslog`, the default rotation is rotate-and-rename
  — no special configuration needed.
- On Linux `logrotate`, prefer the default (rotate then SIGHUP the
  web server) over `copytruncate` mode. Both work, but
  rotate-and-rename is cheaper and loses no lines.

## `rc.d` script (OpenBSD)

A minimal `/etc/rc.d/abusive_http_watch`:

```sh
#!/bin/ksh
daemon="/usr/local/sbin/abusive_http_watch"
daemon_flags="/var/www/logs/access.log"
daemon_user="_www"
. /etc/rc.d/rc.subr
rc_bg=YES
rc_reload=NO
rc_cmd $1
```

Then:

```sh
doas rcctl enable abusive_http_watch
doas rcctl start abusive_http_watch
doas rcctl check abusive_http_watch
```

On Linux with systemd, see your distribution's documentation; a
simple `Type=simple` unit pointing at `/usr/local/sbin/abusive_http_watch`
with the same arguments works.

## Syslog events

The daemon emits one event class — `added` — when a new IP crosses
the hit threshold. Format:

```
<syslog-tag>[<pid>]: added ip=<ip> path=<blocklist-path>
```

Example:

```
abusive_http_watch[12345]: added ip=198.51.100.42 path=/var/www/run/abusive_http_hosts
```

Route these via `syslog.conf` to a dedicated file for review:

```
local0.notice                                          /var/log/abusive_http.log
```

Combined with `ABUSIVE_SYSLOG_FACILITY=local0`, this gives a clean
audit log of every IP the daemon has blocked.

## Monitoring signals

For automated monitoring, two indicators are useful:

- **Blocklist growth rate** — `wc -l /var/www/run/abusive_http_hosts`
  over time. On a public web server this should grow steadily
  (several IPs per day to many per hour, depending on exposure).
  A flatline for 24h on a public site suggests the daemon is not
  running or is reading the wrong log.
- **Process health** — `pgrep -f abusive_http_watch` should always
  return one PID. Alert if zero.

For systems with Prometheus, a tiny exporter that just `wc -l`s the
blocklist file and exposes the count as a gauge is enough. No
metrics endpoint exists in the daemon itself — by design, to keep
it dependency-free.

## Troubleshooting

### The blocklist file stays empty

Possible causes, in order of likelihood:

1. **Daemon is reading the wrong file.** Run with `ABUSIVE_DEBUG=1`
   in the foreground. Confirm it logs `start following dev=… ino=…`
   for the path you expect. If not, fix the argument.
2. **No actual attack traffic.** Unlikely on a public web server,
   but possible on an internal one. Run with `ABUSIVE_DEBUG=1` and
   look for `[dbg] ip=… ` lines without matching `[dbg] token match`.
3. **Whitelist swallows everything.** Check
   `/etc/abusive_http_whitelist` — a stray `/0` CIDR entry
   whitelists the entire internet. Remove and restart.
4. **Hit threshold too high.** Lower `ABUSIVE_MIN_HITS` to `1`
   temporarily and watch behaviour.

### The blocklist grows but the firewall does not block

The daemon writes the file. The firewall has to be told to read it.
Check:

- Cron is reloading the `pf` table / `nftables` set every few
  minutes (the daemon does not call `pfctl` itself).
- The firewall rule referencing the table is in the right chain
  and is `quick` enough to take effect before any later `pass` rule.

### Legitimate users are being blocked

Add their public IP or CIDR range to
`/etc/abusive_http_whitelist`. The daemon re-reads the whitelist
on every line, so changes take effect within seconds.

To clear a single IP from the existing blocklist, edit the file
directly and reload the firewall table. The daemon does not need a
restart — it will simply not re-add the IP unless the user matches
tokens again.

### Daemon crashes after log rotation

Should not happen given the rotation handling described above. If
it does, run the daemon in the foreground with `ABUSIVE_DEBUG=1`,
trigger a rotation, and capture the stderr. Send the captured
output via the [Contact](/contact/) page — that is a real bug worth
fixing.

## Graceful shutdown

The daemon traps `SIGTERM` and exits cleanly after the current
iteration:

```sh
doas rcctl stop abusive_http_watch
```

Logs `[info] TERM received, exiting`, closes the log file handle,
flushes any pending writes, and exits with status 0. No state
corruption, even on a busy log.