AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser

2026-03-12T00:00:00+00:00

AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser🔗</a></h1>

AWFFull 4.0.0</a> is a web server log analyser. It reads Apache/nginx/Squid access logs and produces HTML reports with graphs showing traffic, top URLs, referrers, user agents, country statistics, and more. It was originally forked from The Webalizer</a> by Bradford L. Barrett (1997–2001), extended as AWFFull by Steve McInerney from 2004, and actively developed until 2008, when it was last released as version 3.10.2.</p>

It still works well in 2026 — the core log-parsing logic is solid and the output is clean. But the codebase had accumulated 18 years of code: EOL libraries, dead links, obsolete command-line flags, a broken GeoIP implementation, and a very old config.guess</code>. Time for a proper cleanup.</p>

This post covers what changed in AWFFull 4.0.0 and why.</p>

Is a Static Web Log Analyser Still Useful?🔗</a></h2>
Good question. The short answer is: yes — and arguably more so than before.</p>
Modern web analytics platforms — Google Analytics, Matomo, Plausible, Grafana
with Loki, ELK stacks — are excellent for live dashboards and drill-down
exploration. But they share a common weakness: they are slow when you need to
process large volumes of historical log data and render the results as a
complete set of graphs and tables</strong>.</p>
Feed a year of Apache logs (say, 500 million lines across 50 GB of gzipped
files) into an ELK pipeline and you’ll be waiting a long time for the data to
be ingested, indexed, and ready to query. Then wait again each time you open a
dashboard. Grafana rendering a year’s worth of nginx traffic across 30 panels?
Expect seconds of load time per page, per query.</p>
AWFFull’s approach is different: it processes the raw log files directly, keeps
all state in a compact binary history file, and writes static HTML + PNG
output</strong> that loads instantly in any browser — even on a machine from 2010,
over a slow connection, or offline entirely. There is no query layer, no index,
no database, no running server. The report is just files.</p>
For the typical use case — a sysadmin or developer who wants a monthly
overview of who accessed their server, from where, with what agents, hitting
which URLs — this is exactly the right trade-off:</p>

Fast to generate</strong>: processing a month of logs takes seconds, not minutes</li>
Instant to view</strong>: static HTML, no round-trips, no query engine</li>
Self-contained</strong>: the report directory can be archived, moved, or served
from any static file server</li>
Privacy-friendly</strong>: no third-party JavaScript, no tracking pixels, no data
leaving your infrastructure — the raw IP addresses stay in your own logs</li>
Works on air-gapped systems</strong>: nothing requires internet access at runtime</li>
</ul>
It also complements rather than replaces live monitoring. You would typically
use Prometheus/Grafana (or similar) to watch your infrastructure in real time,
and AWFFull to produce the monthly historical report that goes into a ticket or
a management summary.</p>
Shipping Access Logs to Object Storage and Running in Containers🔗</a></h3>
The operational details — how to ship access logs to S3 or Azure Blob with
Fluent Bit, Vector, or Filebeat/Logstash, how to handle concurrent writes and
log loss at high traffic, and how to run AWFFull as a stateless container
against object storage — are covered in the companion post:</p>
AWFFull in the Cloud: Shipping Logs to Object Storage and Running in Containers</a></strong></p>
The Big Ticket Items🔗</a></h2>
PCRE → PCRE2🔗</a></h3>
AWFFull used libpcre (PCRE version 1) for log-line regex parsing. PCRE 1 has
been officially end-of-life since 2012 and is being dropped by distributions.
We migrated to PCRE2</strong>, the current maintained successor with a cleaner API
and better Unicode support. The change touched parser.c</code>, common.h</code>, and
configure.ac</code>.</p>
GeoIP: From Legacy .dat</code> to libmaxminddb🔗</a></h3>
The old code used libGeoIP</strong> with a proprietary binary .dat</code> format that
MaxMind deprecated in 2019. Country lookups were broken for anyone who hadn’t
kept a seven-year-old database around.</p>
AWFFull 4.0.0 replaces this with libmaxminddb</strong>, the open standard MMDB
format used by both MaxMind (GeoLite2) and DB-IP. The migration is
transparent — IPv4 and IPv6 are both handled by a single MMDB_lookup_string()</code>
call, with no separate code paths needed.</p>
Free databases updated monthly:</p>

Country</strong>: DB-IP Country Lite</a></li>
ASN</strong>: DB-IP ASN Lite</a></li>
</ul>
ASN Statistics: New Table and Pie Chart🔗</a></h3>
Once you have a MMDB-capable lookup layer, adding ASN (Autonomous System
Number) statistics is almost free. A second database (GeoASNDatabase</code>) maps
IPs to their network operator — think “Cloudflare”, “Amazon AWS”,
“Deutsche Telekom”. The report now shows a Top ASNs</strong> table and pie chart,
answering the question “which networks send the most traffic to my site?”</em></p>
GeoASN          yes</span></span>
GeoASNDatabase  /usr/share/GeoIP/dbip-asn-lite.mmdb</span></span></code></pre>
Performance note: both GeoIP and GeoASN lookups happen in the output phase</em>,
once per unique IP address — not once per log line. On a typical site the
overhead is negligible.</p>
Hyperscan: SIMD Pattern Matching🔗</a></h3>
AWFFull supports extensive pattern lists for ignoring, grouping, and hiding
hosts, URLs, referrers, and agents. The original code used a
Boyer-Moore-Horspool</strong> linear scan: each log record was matched against every
pattern in turn.</p>
When Intel Hyperscan</strong> (libhyperscan</code>) is installed, AWFFull now compiles all
patterns from a given list into a single Hyperscan database and matches them in
one SIMD pass per record. For large pattern lists — say, 50+ bot/spider agent
patterns — this is significantly faster. Hyperscan is auto-detected at
configure time and falls back to the existing BMH path if unavailable.</p>
sudo</span> apt</span> install</span> libhyperscan-dev</span></span>
./configure</span> &&</span> make</span></span></code></pre>mimalloc: Better Heap Performance🔗</a></h3>
For large log files with millions of unique hosts, URLs, and referrers, heap
allocation overhead becomes measurable. Adding mimalloc</strong> (Microsoft’s
high-performance allocator) as an optional drop-in was straightforward via the
explicit mi_calloc</code> API. Auto-detected at configure time.</p>
XXH3 Hash Function🔗</a></h3>
The internal hash table used djb2, a good general-purpose hash from 1991.
AWFFull 4.0.0 replaces it with XXH3</strong> (via the bundled xxhash.h</code>), which
offers better distribution and significantly higher throughput on modern CPUs.
The table load-factor was also tuned to reduce collisions.</p>

User-Visible Improvements🔗</a></h2>
Date Stamp on Graphs🔗</a></h3>
Every generated PNG graph now shows the current date</strong> (YYYY-MM-DD</code>) in the
top-right corner, aligned with the title baseline. Small but useful when you’re
looking at archived reports six months later.</p>
Next / Previous Month Navigation🔗</a></h3>
Monthly report pages now show navigation links at the top:</p>
← February 2026  |  [Index]  |  April 2026 →</span></span></code></pre>
Links are derived from the history file — only months that actually exist in
the database are linked.</p>
Updated Browser/Bot Detection🔗</a></h3>
The sample.conf</code> browser/agent detection list dated from 2005 and contained
entries like “Netscape 4.5”, “Internet Explorer 3.0 (Win95)”, and
“MSNBot” — none of which have been seen in the wild for a decade or more. The
list has been completely rewritten:</p>
Removed:</strong> Netscape 1–4.x, Internet Explorer 3–7, Firebird, Galeon, Camino,
msnbot (all entries)</p>
Added:</strong> Chrome/Chromium, Edge (Chromium), Brave, Samsung Browser, Firefox,
Safari, Opera (OPR/), modern search bots (Bingbot, DuckDuckBot, YandexBot,
Baidu, Applebot, AhrefsBot, SemrushBot), AI crawlers (GPTBot, ClaudeBot,
PerplexityBot, Bytespider, CCBot, Amazonbot), and CLI tools (curl, Wget,
python-requests).</p>

Build System and Tooling🔗</a></h2>
CLI Cleanup: 14 Flags Removed🔗</a></h3>
AWFFull inherited many single-letter flags from the original Webalizer. Flags
like -G</code> (suppress hourly graph), -L</code> (suppress legend), -M</code> (mangle
agents), -x</code> (HTML extension), and -I</code> (index alias) had exact equivalents
in the config file. At version 4.0.0 — a major version bump — these were
removed from the CLI, along with three fully deprecated no-op flags (-d</code>,
-q</code>, -Q</code>).</p>
The --help</code> output now documents the new verbosity levels (-v</code> through
-vvvvv</code>) and includes a note pointing config-file-only settings to
awffull.conf(5)</code>.</p>
Man Pages: DocBook XML → Pandoc Markdown🔗</a></h3>
The existing man pages were written in DocBook XML 4.5 and built with
xsltproc</code> — a tool that requires xsltproc</code>, docbook-xsl</code>, and several XML
schema packages, none of which are installed by default on most systems in
2026.</p>
The man pages are now maintained as pandoc Markdown</strong> (doc/awffull.1.md</code>,
doc/awffull.conf.5.md</code>) and built with:</p>
pandoc</span> -</span>s</span> -</span>t</span> man</span> awffull.1.md</span> -</span>o</span> awffull.1</span></span></code></pre>
configure.ac</code> auto-detects pandoc. Pre-built man pages remain in the
repository as a fallback for systems without pandoc. The DocBook XML sources
are retained in doc/</code> for historical reference.</p>
Translations: Quality Over Quantity🔗</a></h3>
AWFFull shipped with 33 language files, most of which were approximately 29% complete and
had not been updated since 2008. Rather than shipping incomplete translations
that silently fall back to English mid-report, we reduced to 6 fully
complete languages</strong> (German, Finnish, Italian, Swedish, Brazilian Portuguese,
Indonesian) and brought all of them to 100% (524/524 strings)</strong>.</p>
The .pot</code> template was regenerated with xgettext</code> to include all new strings
added in 4.0.0 (GeoASN messages, MaxMind DB errors, pattern matching warnings).</p>

Repository Modernisation🔗</a></h2>
The project moved from a collection of legacy files to a cleaner structure:</p>
Before</th> After</th></tr></thead>

README</code> + README.FIRST</code> + README.webalizer</code></td> README.md</code> (single entry point)</td></tr>
ChangeLog</code> (GNU format, frozen)</td> CHANGELOG.md</code> (Keep a Changelog)</td></tr>
COPYING</code></td> LICENSE</code></td></tr>
TODO</code> (plain text, 2008)</td> TODO.md</code> (Markdown with [x]</code> checkboxes)</td></tr>
country-codes.txt</code> (2005, informational)</td> removed</td></tr>
DNS.README</code> (dead link)</td> removed</td></tr>
contrib/</code> (Webalizer migration scripts)</td> removed</td></tr>
config.guess</code> from 2008</td> updated to automake 1.16 (2022)</td></tr>
</tbody></table>




  
    
      tip
    </div>
    tip</strong></span>
  </div>
  The full list of technical changes is available in CHANGES.md</a>.</p>
</div>
</div>

CI/CD Pipeline🔗</a></h2>
A full GitLab CI pipeline was added covering:</p>

Build verification</strong> on Debian 12/13, Ubuntu 22.04/24.04, AlmaLinux 8/9/10</li>
Source and binary tarballs</strong> on every push to main</code></li>
.deb</code> package</strong> (Ubuntu 24.04 amd64) on release tags</li>
.rpm</code> package</strong> (AlmaLinux 9 amd64) on release tags</li>
Container image</strong> pushed to GitLab Container Registry (Ubuntu 24.04 base, all optional libs included)</li>
GitLab Release</strong> creation with all artifacts uploaded to the Generic Package Registry</li>
</ul>
Releases are triggered by pushing a semver tag:</p>
git</span> tag</span> v4.0.0</span></span>
git</span> push</span> -</span>-tags</span></span></code></pre>
The container image is available as:</p>
docker</span> pull</span> registry.gitlab.com/aleks001/awffull:v4.0.0</span></span>
docker</span> pull</span> registry.gitlab.com/aleks001/awffull:latest</span></span></code></pre>
The other artifacts are available on the Release Page</a></p>

What’s Next🔗</a></h2>
The TODO.md</code> tracks longer-term ideas: SVG graphs (replacing the aging
libgd dependency), weekly summaries, a templating system for the HTML output,
and better browser/OS reporting. Contributions welcome.</p>
The project is at https://gitlab.com/aleks001/awfull</a>.</p>

AWFFull is free software licensed under GPL v3 or later.</em>
Based on The Webalizer by Bradford L. Barrett (1997–2001).</em>
Extended by Steve McInerney (2004–2008) and Aleksandar Lazic (2026–present).</em></p>

Before</th>	After</th></tr></thead>
`README</code> + README.FIRST</code> + README.webalizer</code></td>`	`README.md</code> (single entry point)</td></tr>`
`ChangeLog</code> (GNU format, frozen)</td>`	`CHANGELOG.md</code> (Keep a Changelog)</td></tr>`
`COPYING</code></td>`	`LICENSE</code></td></tr>`
`TODO</code> (plain text, 2008)</td>`	`TODO.md</code> (Markdown with [x]</code> checkboxes)</td></tr>`
`country-codes.txt</code> (2005, informational)</td>`	removed</td></tr>
`DNS.README</code> (dead link)</td>`	removed</td></tr>
`contrib/</code> (Webalizer migration scripts)</td>`	removed</td></tr>
`config.guess</code> from 2008</td>`	updated to automake 1.16 (2022)</td></tr> </tbody></table> tip </div> tip</strong></span> </div> The full list of technical changes is available in CHANGES.md</a>.</p> </div> </div> CI/CD Pipeline 🔗</a></h2> A full GitLab CI pipeline was added covering:</p> Build verification</strong> on Debian 12/13, Ubuntu 22.04/24.04, AlmaLinux 8/9/10</li> Source and binary tarballs</strong> on every push to `main</code></li>` `.deb</code> package</strong> (Ubuntu 24.04 amd64) on release tags</li>` `.rpm</code> package</strong> (AlmaLinux 9 amd64) on release tags</li>` `Container image</strong> pushed to GitLab Container Registry (Ubuntu 24.04 base, all optional libs included)</li>` `GitLab Release</strong> creation with all artifacts uploaded to the Generic Package Registry</li> </ul> Releases are triggered by pushing a semver tag:</p>`git</span> tag</span> v4.0.0</span></span> git</span> push</span> -</span>-tags</span></span></code></pre> The container image is available as:</p> docker</span> pull</span> registry.gitlab.com/aleks001/awffull:v4.0.0</span></span> docker</span> pull</span> registry.gitlab.com/aleks001/awffull:latest</span></span></code></pre> The other artifacts are available on the Release Page</a></p> What’s Next🔗</a></h2> The TODO.md</code> tracks longer-term ideas: SVG graphs (replacing the aging libgd dependency), weekly summaries, a templating system for the HTML output, and better browser/OS reporting. Contributions welcome.</p> The project is at https://gitlab.com/aleks001/awfull</a>.</p> AWFFull is free software licensed under GPL v3 or later.</em> Based on The Webalizer by Bradford L. Barrett (1997–2001).</em> Extended by Steve McInerney (2004–2008) and Aleksandar Lazic (2026–present).</em></p>

Alek's Blog - gitlab-ci

AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser