Abusive HTTP Watch

Abusive HTTP Watch is a lightweight OpenBSD daemon that tails the web server access log in real time, detects malicious HTTP probes against a curated token set, and writes the offending source IPs to a file that can be consumed directly by the firewall (e.g. pf). In active production protecting web shops.

  flowchart LR
    A[Webserver logs] --> B[Pattern match]
    B --> C{"Threshold hit?"}
    C -->|Yes| D[PF block rule]
    C -->|No| A
    D --> E[(Block table)]
    E -->|TTL expired| F[Auto release]

What it does

Web shops and production websites are constantly probed for vulnerabilities — PHP exploits, WordPress login probes, /etc/passwd lookups, path traversal, SQLi, null-byte injection. These probes leave a clear token pattern in the access log, regardless of the HTTP status code returned.

Abusive HTTP Watch tails the log, evaluates each line against a precise token set, and writes IPs with repeated hits into an append-only blocklist that drops straight into OpenBSD pf tables, Apache Require directives, or nginx deny lists.

Detection categories

The token set targets attack classes that demonstrably show up in production logs — deliberately conservative to avoid false positives.

• PHP exploitation — allow_url_include, auto_prepend_file, php://input, eval(
• WordPress probes — wp-admin, wp-login.php and variants
• Path traversal — ../.. plus common encodings
• Sensitive paths — .env, .git, .svn, id_rsa, /etc/passwd
• RCE / CGI attempts — cgi-bin, ?cmd=, =shell, eval(
• SQL injection — or 1=1 and conservative variants
• Login loop probes — multi-cascaded back= parameters, including URL-encoded
• Null-byte injection — %00 in URI

Operational features

• 🔄 Log-rotation safe — Detects both rotate (inode change) and copytruncate (pos > size) and recovers without missing lines.
• 🌍 Dual stack — Native IPv4 and IPv6 support including CIDR whitelisting (e.g. 203.0.113.0/24, 2001:db8::/32).
• 🛡️ Public IPs only — Private RFC 1918, link-local, loopback, multicast, and shared-address-space addresses are skipped automatically.
• 🎯 Scoped matching — Per request, matching is restricted to request, user-agent, and optionally referrer. Referrer scope is off by default to prevent drive-by blocks via third-party pages.
• 🔒 Atomic blocklist updates — flock(LOCK_EX) on writes, in-memory dedup prevents duplicate entries across restarts.
• 📋 Whitelist with priority — /etc/abusive_http_whitelist accepts single IPs or CIDRs that are always let through.
• ⏳ Hit threshold — Only after MIN_HITS hits (default 3) is an IP added to the blocklist — protects against accidental singletons.
• 📡 Syslog integration — Structured events (tag, facility, prio configurable via env) for added actions and diagnostics.
• 🛑 Clean shutdown — SIGTERM is caught, the daemon finishes its current loop iteration and closes handles in order.

Configuration

Fully controllable via environment variables — no code change required.

Technology foundation

Deliberately minimal — only OpenBSD base tools and Perl standard modules.

Why this matters at the CEO level

• 🏷️ Tiny compliance footprint — Only reads existing server logs, writes a single file, no cloud, no telemetry. GDPR-friendly out of the box.
• 💰 Operating cost ~0 € — Single-file script, no containers, no dependencies. Maintenance limited to the token set, which rarely changes.
• 🔐 Attack-surface reduction — Repeatedly suspicious IPs disappear from the pool within seconds — most attackers try only 3–10 paths before moving to the next IP.
• 📊 Auditable — 381 lines of Perl, no external dependencies, every token rule is a commented regex. Code review fits in a single sitting.
• 🛠️ No one gets locked out by mistake — Whitelist with IPv4/IPv6 CIDR plus the 3-hit threshold prevent search-engine bots or stray probes from causing damage.

Abusive HTTP Watch — Version 2.1.2 · OpenBSD rc.d daemon · in production at web shops · ~380 SLOC pure Perl