mergelog-rs: Rewriting a Year-2000 C Tool in Rust — and Making It 2.26× Faster

aleks — Sun, 15 Mar 2026 00:00:00 +0000

mergelog</a> is a small but handy tool: it takes multiple Apache log files from servers behind a round-robin DNS and merges them into a single chronologically sorted stream. The original version 4.5 was written in C around 2000–2001, weighs in at ~450 lines, and does exactly what it promises — reliably and fast.

This post documents the journey from mergelog-4.5 to mergelog-rs</a>: what changed, what was surprising, and how six targeted optimizations made the Rust binary 2.26× faster than the C original.

Full respect and honour to Bertrand Demiddelaer, who created mergelog back in 2000. Writing a tool this fast, this correct, and this compact in C — with no external dependencies beyond zlib — is genuinely impressive. mergelog-rs stands on his shoulders.

The Original: mergelog 4.5 🔗</a></h2>
The C program reads any number of log files in Common Log Format (CLF) / NCSA Combined Format:
%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"</code></pre> Example line:93.184.216.34 - - [15/Mar/2026:12:00:00 +0100] "GET /index.html HTTP/1.1" 200 1234 "-" "curl/7.88"</code></pre> The algorithm in 4.5 is straightforward: find the earliest timestamp across all files, then iterate second-by-second from there to the end, writing all matching lines at each step. That is O(timespan × K) — slow when logs span multiple years or many files are involved. Gzip support exists, but as a separate binary (zmergelog</code>) compiled with -DUSE_ZLIB</code>. The Rust Rewrite: What Changed🔗</a></h2> 1. Algorithm: K-Way Merge with a Binary Heap🔗</a></h3> Instead of time-based iteration, mergelog-rs uses a k-way merge with a min-heap (std::collections::BinaryHeap</code> with reversed ordering): Seed the heap with the first line from each file</li> Main loop: pop the smallest timestamp, write the line, push the next line from the same file</li> Complexity: O(N log K) — independent of the time span of the logs</li> </ul> while let Some(Entry { mut line, mut reader, file_idx, .. }) = heap.pop() { out.write_all(line.as_bytes())?; line.clear(); if fill_line(&mut *reader, &mut line)? { let ts = parse_clf_timestamp(&line).unwrap_or_else(sentinel); heap.push(Entry { ts, file_idx, line, reader }); } }</code></pre>2. Magic-Byte Detection Instead of File Extension🔗</a></h3> The original detects gzip by binary type (zmergelog</code> vs. mergelog</code>). mergelog-rs reads the first 6 bytes of each file and detects the format automatically: Format</th> Magic Bytes</th></tr></thead> gzip</td> 1F 8B</code></td></tr> bzip2</td> 42 5A 68</code> (BZh</code>)</td></tr> xz</td> FD 37 7A 58 5A 00</code></td></tr> zstd</td> 28 B5 2F FD</code></td></tr> </tbody></table> pub fn detect_from_bytes(magic: &[u8]) -> Compression { if magic.len() >= 2 && magic[..2] == [0x1f, 0x8b] { Compression::Gz } else if magic.len() >= 3 && magic[..3] == [0x42, 0x5a, 0x68] { Compression::Bz2 } else if magic.len() >= 6 && magic[..6] == [0xfd, 0x37, 0x7a, 0x58, 0x5a, 0x00] { Compression::Xz } else if magic.len() >= 4 && magic[..4] == [0x28, 0xb5, 0x2f, 0xfd] { Compression::Zstd } else { Compression::None } }</code></pre> One binary, all formats, no file extension required. The same peek-based detection also works for stdin — BufReader::fill_buf()</code> inspects the first bytes without consuming them, so no seek is needed. 3. Timezone-Aware Timestamp Comparison🔗</a></h3> The C original ignores the timezone offset in timestamps — it converts date components to local time via mktime()</code>. For logs from servers in different timezones this can produce incorrectly ordered output. mergelog-rs converts every timestamp to UTC before comparison. A +0900</code> timestamp and a -0500</code> timestamp are ordered correctly, regardless of the system’s local timezone. Profiling: Where Does the Time Go?🔗</a></h2> Since perf</code> was locked down on the system (perf_event_paranoid = 4</code>), a dedicated timing binary was built that measures each phase manually with std::time::Instant</code>: ──── profile_timing (41.65M lines) ────────────────── I/O (read_line) 3.91 s 32.4% Parse (jiff strptime) 5.52 s 45.7% ← bottleneck Heap (push/pop) 1.97 s 16.3% Write (write_all) 0.69 s 5.7% ───────────────────────────────────────────────── Σ 12.08 s</code></pre> Nearly half the time was in the parser. The reason: jiff::Zoned::strptime</code> is a full parsing pipeline — format-string interpretation, timezone database lookup, error handling, heap allocation. Across ~42 million lines that adds up fast. This full parsing pipeline is super handy, but in our use case it is not necessary — therefore we built a hand-rolled CLF timestamp parser. Optimization 1: Hand-Rolled CLF Parser🔗</a></h2> The CLF timestamp format is completely fixed: [DD/Mon/YYYY:HH:MM:SS ±HHMM] 0123456789012345678901234 5</code></pre> A hand-written parser only needs to: Find [</code> in the string</li> Read 6 integers from fixed byte positions</li> Compute the Unix timestamp via integer arithmetic</li> </ol> For step 3 we use Howard Hinnant’s proleptic Gregorian algorithm — no branches, pure multiplication: fn days_since_epoch(y: i64, m: i64, d: i64) -> i64 { let y = if m <= 2 { y - 1 } else { y }; let era = if y >= 0 { y } else { y - 399 } / 400; let yoe = y - era * 400; let doy = (153 * (m + if m > 2 { -3 } else { 9 }) + 2) / 5 + d - 1; let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; era * 146_097 + doe - 719_468 }</code></pre> Result: parse time dropped from 5.52 s to 1.50 s — 3.7× faster. Optimization 2: Reuse the String Buffer🔗</a></h2> The original implementation allocated a new String</code> for every line read: // before: fresh allocation per line fn read_line(reader: &mut dyn BufRead) -> Result<Option<String>> { let mut line = String::new(); // ← new every time reader.read_line(&mut line)?; ... }</code></pre> The fix is simple: when an Entry</code> is popped from the heap, you receive line: String</code> with full ownership. Instead of dropping it, clear it and read the next line directly into the same buffer: // after: capacity is retained while let Some(Entry { mut line, mut reader, file_idx, .. }) = heap.pop() { out.write_all(line.as_bytes())?; line.clear(); // ← keeps the heap allocation if fill_line(&mut *reader, &mut line)? { let ts = parse_clf_timestamp(&line).unwrap_or_else(sentinel); heap.push(Entry { ts, file_idx, line, reader }); } }</code></pre> Since consecutive log lines are similar in length, the buffer stops growing after the first few lines — no malloc</code> calls in the hot path. Optimization 3: mimalloc as the Global Allocator🔗</a></h2> With most per-line allocations already eliminated, there are still smaller allocations happening — buffer growth during seeding, internal BinaryHeap</code> resizing, and the occasional String</code> capacity bump. Swapping out the default glibc allocator for mimalloc</a> is a one-liner in Rust: # Cargo.toml mimalloc = { version = "0.1", default-features = false }</code></pre>// main.rs use mimalloc::MiMalloc; #[global_allocator] static GLOBAL: MiMalloc = MiMalloc;</code></pre> That’s it — no other code changes. mimalloc is designed for throughput-heavy workloads with many small, short-lived allocations, which matches our remaining allocation pattern well. Result: 5.55 s → 5.39 s — a further ~3% improvement. Not a dramatic win on its own, but it’s essentially free: two lines of code, no logic changes, and it stacks on top of everything else. Optimization 4: 4 MiB Read Buffers + SIMD Newline Search🔗</a></h2> After the previous three optimizations, profiling showed I/O was still the largest remaining cost at 47% of total time. Two changes address this together. 4 MiB read buffers — The previous BufReader</code> used a 256 KiB internal buffer, meaning a syscall every ~256 KiB of data. Increasing to 4 MiB reduces syscall frequency by 16×, which shows up directly in system time. SIMD newline search via memchr</code> — Instead of BufReader::read_line</code>, the line reader now uses fill_buf()</code> + memchr::memchr()</code> directly. The memchr</code> crate uses AVX2/SSE2 to scan 16 or 32 bytes per cycle when searching for \n</code>: fn fill_line(reader: &mut dyn BufRead, buf: &mut String) -> Result<bool> { loop { let available = reader.fill_buf()?; if available.is_empty() { return Ok(false); } match memchr(b'\n', available) { Some(pos) => { buf.push_str(std::str::from_utf8(&available[..pos + 1])?); reader.consume(pos + 1); return Ok(true); } None => { buf.push_str(std::str::from_utf8(available)?); let len = available.len(); reader.consume(len); } } } }</code></pre> Result: 5.39 s → 4.43 s — another 18% improvement, bringing the total to 2.26× faster than the C original. A Road Not Taken: Parallel Reader Threads🔗</a></h2> An obvious next step seemed to be spawning one thread per input file — each thread handles its own I/O and decompression, sending (timestamp, line)</code> pairs to the merge thread via a bounded mpsc</code> channel: Thread 1: read + decompress + parse → channel ─┐ Thread 2: read + decompress + parse → channel ─┤→ Merge thread → stdout ... ─┘</code></pre> For compressed files (bz2, xz) this is conceptually attractive: decompression is CPU-bound and can genuinely run in parallel. For plain-text files on a single disk, it does not help — the bottleneck is disk read bandwidth, and parallel reads from the same physical device cause seek contention or simply saturate the same I/O bus. Benchmarking confirmed this: the threaded version achieved identical wall-clock time on plain-text files, while adding per-line heap allocation overhead (the channel requires sending owned String</code> values, making buffer reuse impossible across the thread boundary). The result was slightly slower than the sequential version with large buffers. Takeaway: threading is the right tool when the bottleneck is CPU-bound work that can be parallelised per-file (compressed files on a multi-core system). It is the wrong tool when the bottleneck is a shared I/O resource. Measure first. Final Results🔗</a></h2> Measured on 7 log files × 1 GiB = 7 GiB total, 41.65 million lines: Version</th> Wall time</th> User</th> System</th></tr></thead> mergelog-4.5 (C)</td> 10.03 s</td> 5.93 s</td> 4.12 s</td></tr> mergelog-rs v1 (jiff)</td> 10.39 s</td> 9.13 s</td> 1.25 s</td></tr> + hand-rolled parser</td> 5.99 s</td> 4.75 s</td> 1.23 s</td></tr> + buffer reuse</td> 5.55 s</td> 4.29 s</td> 1.26 s</td></tr> + mimalloc</td> 5.39 s</td> 4.17 s</td> 1.22 s</td></tr> + 4 MiB buffers + memchr</td> 4.43 s</td> 3.61 s</td> 0.81 s</td></tr> </tbody></table> mergelog-rs ran 2.26 ± 0.02 times faster than mergelog-4.5</code></pre> Memory and CPU Usage🔗</a></h2> Measured with /usr/bin/time -v</code> on the same 7 × 1 GiB workload: </th> mergelog-4.5</th> mergelog-rs</th></tr></thead> Wall time</td> 9.63 s</td> 4.28 s</td></tr> User time</td> 5.61 s</td> 3.42 s</td></tr> System time</td> 4.01 s</td> 0.84 s</td></tr> CPU utilisation</td> 99%</td> 99%</td></tr> Peak RSS</td> 1.6 MiB</td> 30.7 MiB</td></tr> Involuntary ctx switches</td> 105</td> 122</td></tr> </tbody></table> Memory — The picture changed significantly with the 4 MiB read buffers. mergelog-4.5 uses 1.6 MiB; mergelog-rs now uses 30.7 MiB. The breakdown is straightforward: 7 files × 4 MiB read buffer = 28 MiB, plus the write BufWriter</code>, the binary itself, and mimalloc metadata. This is a deliberate trade-off — RAM is cheap, syscalls are not. At 30 MiB to process 7 GiB of data the ratio is still extremely lean, and the buffer size is a single constant in reader.rs</code> that can be tuned if memory is constrained. System time — The most striking difference is kernel time: mergelog-4.5 spends 4.01 s in the kernel versus 0.84 s for mergelog-rs. Two factors: the C version calls write(1, ...)</code> directly for every line (~42 million syscalls), and its read buffers are only 32 KiB. mergelog-rs batches writes via BufWriter</code> and reads via 4 MiB BufReader</code>, cutting kernel round-trips dramatically. CPU — Both tools are single-threaded and max out one core at 99%. Takeaways🔗</a></h2> Five lessons from this project: 1. Measure before optimizing. Without the timing binary the focus might have landed on the heap. But jiff::strptime</code> was costing nearly 3× more than every other phase combined. 2. Libraries are great until they aren’t. jiff</code> is an excellent crate for correct timezone-aware date handling. But when you parse the same fixed-format string 42 million times, you pay a steep price for generality. A 30-line hand-rolled parser that exploits the known fixed layout beats it by 3.7×. 3. Sometimes the allocator is the last few percent. After eliminating almost all allocations from the hot path, swapping in mimalloc took two lines of code and gave another 3% for free. It won’t rescue a poorly written program, but on an already optimized one it’s a cheap final step worth trying. 4. Buffer size matters more than you think. Going from 256 KiB to 4 MiB read buffers — a one-line change — cut system time by 34% and wall time by 18%. Every syscall is a round-trip to the kernel; fewer is better. 5. Parallelism requires the right bottleneck. Spawning reader threads per file is conceptually appealing for compressed logs where decompression is CPU-bound. But for plain-text files on a single disk, the bottleneck is I/O bandwidth — adding threads just adds channel overhead and per-line heap allocation, making things slightly slower. Threading is a tool, not a universal solution. 6. Ownership is not a burden. Buffer reuse in C requires careful explicit lifetime management and is easy to get wrong. In Rust, the ownership system makes it the obvious and safe solution — line.clear()</code> instead of free(line); line = malloc(...)</code>, with the compiler guaranteeing no use-after-free. The code is available under GPL-3.0-or-later. The original mergelog 4.5 was released as GPL-2.0-or-later, which explicitly permits upgrading to any later version of the GPL. GPL-3.0 adds patent protection clauses and anti-tivoization provisions that GPL-2.0 lacks — a straightforward upgrade with no downsides for a command-line tool like this. AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser aleks — Thu, 12 Mar 2026 00:00:00 +0000 AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser🔗</a></h1> Background🔗</a></h2> AWFFull 4.0.0</a> is a web server log analyser. It reads Apache/nginx/Squid access logs and produces HTML reports with graphs showing traffic, top URLs, referrers, user agents, country statistics, and more. It was originally forked from The Webalizer</a> by Bradford L. Barrett (1997–2001), extended as AWFFull by Steve McInerney from 2004, and actively developed until 2008, when it was last released as version 3.10.2. It still works well in 2026 — the core log-parsing logic is solid and the output is clean. But the codebase had accumulated 18 years of code: EOL libraries, dead links, obsolete command-line flags, a broken GeoIP implementation, and a very old config.guess</code>. Time for a proper cleanup. This post covers what changed in AWFFull 4.0.0 and why. Is a Static Web Log Analyser Still Useful?🔗</a></h2> Good question. The short answer is: yes — and arguably more so than before. Modern web analytics platforms — Google Analytics, Matomo, Plausible, Grafana with Loki, ELK stacks — are excellent for live dashboards and drill-down exploration. But they share a common weakness: they are slow when you need to process large volumes of historical log data and render the results as a complete set of graphs and tables. Feed a year of Apache logs (say, 500 million lines across 50 GB of gzipped files) into an ELK pipeline and you’ll be waiting a long time for the data to be ingested, indexed, and ready to query. Then wait again each time you open a dashboard. Grafana rendering a year’s worth of nginx traffic across 30 panels? Expect seconds of load time per page, per query. AWFFull’s approach is different: it processes the raw log files directly, keeps all state in a compact binary history file, and writes static HTML + PNG output that loads instantly in any browser — even on a machine from 2010, over a slow connection, or offline entirely. There is no query layer, no index, no database, no running server. The report is just files. For the typical use case — a sysadmin or developer who wants a monthly overview of who accessed their server, from where, with what agents, hitting which URLs — this is exactly the right trade-off: Fast to generate: processing a month of logs takes seconds, not minutes</li> Instant to view: static HTML, no round-trips, no query engine</li> Self-contained: the report directory can be archived, moved, or served from any static file server</li> Privacy-friendly: no third-party JavaScript, no tracking pixels, no data leaving your infrastructure — the raw IP addresses stay in your own logs</li> Works on air-gapped systems: nothing requires internet access at runtime</li> </ul> It also complements rather than replaces live monitoring. You would typically use Prometheus/Grafana (or similar) to watch your infrastructure in real time, and AWFFull to produce the monthly historical report that goes into a ticket or a management summary. Shipping Access Logs to Object Storage and Running in Containers🔗</a></h3> The operational details — how to ship access logs to S3 or Azure Blob with Fluent Bit, Vector, or Filebeat/Logstash, how to handle concurrent writes and log loss at high traffic, and how to run AWFFull as a stateless container against object storage — are covered in the companion post: AWFFull in the Cloud: Shipping Logs to Object Storage and Running in Containers</a> The Big Ticket Items🔗</a></h2> PCRE → PCRE2🔗</a></h3> AWFFull used libpcre (PCRE version 1) for log-line regex parsing. PCRE 1 has been officially end-of-life since 2012 and is being dropped by distributions. We migrated to PCRE2, the current maintained successor with a cleaner API and better Unicode support. The change touched parser.c</code>, common.h</code>, and configure.ac</code>. GeoIP: From Legacy .dat</code> to libmaxminddb🔗</a></h3> The old code used libGeoIP with a proprietary binary .dat</code> format that MaxMind deprecated in 2019. Country lookups were broken for anyone who hadn’t kept a seven-year-old database around. AWFFull 4.0.0 replaces this with libmaxminddb, the open standard MMDB format used by both MaxMind (GeoLite2) and DB-IP. The migration is transparent — IPv4 and IPv6 are both handled by a single MMDB_lookup_string()</code> call, with no separate code paths needed. Free databases updated monthly: Country: DB-IP Country Lite</a></li> ASN: DB-IP ASN Lite</a></li> </ul> ASN Statistics: New Table and Pie Chart🔗</a></h3> Once you have a MMDB-capable lookup layer, adding ASN (Autonomous System Number) statistics is almost free. A second database (GeoASNDatabase</code>) maps IPs to their network operator — think “Cloudflare”, “Amazon AWS”, “Deutsche Telekom”. The report now shows a Top ASNs table and pie chart, answering the question “which networks send the most traffic to my site?” GeoASN yes GeoASNDatabase /usr/share/GeoIP/dbip-asn-lite.mmdb</code></pre> Performance note: both GeoIP and GeoASN lookups happen in the output phase, once per unique IP address — not once per log line. On a typical site the overhead is negligible. Hyperscan: SIMD Pattern Matching🔗</a></h3> AWFFull supports extensive pattern lists for ignoring, grouping, and hiding hosts, URLs, referrers, and agents. The original code used a Boyer-Moore-Horspool linear scan: each log record was matched against every pattern in turn. When Intel Hyperscan (libhyperscan</code>) is installed, AWFFull now compiles all patterns from a given list into a single Hyperscan database and matches them in one SIMD pass per record. For large pattern lists — say, 50+ bot/spider agent patterns — this is significantly faster. Hyperscan is auto-detected at configure time and falls back to the existing BMH path if unavailable. sudo apt install libhyperscan-dev ./configure && make</code></pre>mimalloc: Better Heap Performance🔗</a></h3> For large log files with millions of unique hosts, URLs, and referrers, heap allocation overhead becomes measurable. Adding mimalloc (Microsoft’s high-performance allocator) as an optional drop-in was straightforward via the explicit mi_calloc</code> API. Auto-detected at configure time. XXH3 Hash Function🔗</a></h3> The internal hash table used djb2, a good general-purpose hash from 1991. AWFFull 4.0.0 replaces it with XXH3 (via the bundled xxhash.h</code>), which offers better distribution and significantly higher throughput on modern CPUs. The table load-factor was also tuned to reduce collisions. User-Visible Improvements🔗</a></h2> Date Stamp on Graphs🔗</a></h3> Every generated PNG graph now shows the current date (YYYY-MM-DD</code>) in the top-right corner, aligned with the title baseline. Small but useful when you’re looking at archived reports six months later. Next / Previous Month Navigation🔗</a></h3> Monthly report pages now show navigation links at the top: ← February 2026 | [Index] | April 2026 →</code></pre> Links are derived from the history file — only months that actually exist in the database are linked. Updated Browser/Bot Detection🔗</a></h3> The sample.conf</code> browser/agent detection list dated from 2005 and contained entries like “Netscape 4.5”, “Internet Explorer 3.0 (Win95)”, and “MSNBot” — none of which have been seen in the wild for a decade or more. The list has been completely rewritten: Removed: Netscape 1–4.x, Internet Explorer 3–7, Firebird, Galeon, Camino, msnbot (all entries) Added: Chrome/Chromium, Edge (Chromium), Brave, Samsung Browser, Firefox, Safari, Opera (OPR/), modern search bots (Bingbot, DuckDuckBot, YandexBot, Baidu, Applebot, AhrefsBot, SemrushBot), AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Bytespider, CCBot, Amazonbot), and CLI tools (curl, Wget, python-requests). Build System and Tooling🔗</a></h2> CLI Cleanup: 14 Flags Removed🔗</a></h3> AWFFull inherited many single-letter flags from the original Webalizer. Flags like -G</code> (suppress hourly graph), -L</code> (suppress legend), -M</code> (mangle agents), -x</code> (HTML extension), and -I</code> (index alias) had exact equivalents in the config file. At version 4.0.0 — a major version bump — these were removed from the CLI, along with three fully deprecated no-op flags (-d</code>, -q</code>, -Q</code>). The --help</code> output now documents the new verbosity levels (-v</code> through -vvvvv</code>) and includes a note pointing config-file-only settings to awffull.conf(5)</code>. Man Pages: DocBook XML → Pandoc Markdown🔗</a></h3> The existing man pages were written in DocBook XML 4.5 and built with xsltproc</code> — a tool that requires xsltproc</code>, docbook-xsl</code>, and several XML schema packages, none of which are installed by default on most systems in 2026. The man pages are now maintained as pandoc Markdown (doc/awffull.1.md</code>, doc/awffull.conf.5.md</code>) and built with: pandoc -s -t man awffull.1.md -o awffull.1</code></pre> configure.ac</code> auto-detects pandoc. Pre-built man pages remain in the repository as a fallback for systems without pandoc. The DocBook XML sources are retained in doc/</code> for historical reference. Translations: Quality Over Quantity🔗</a></h3> AWFFull shipped with 33 language files, most of which were approximately 29% complete and had not been updated since 2008. Rather than shipping incomplete translations that silently fall back to English mid-report, we reduced to 6 fully complete languages (German, Finnish, Italian, Swedish, Brazilian Portuguese, Indonesian) and brought all of them to 100% (524/524 strings). The .pot</code> template was regenerated with xgettext</code> to include all new strings added in 4.0.0 (GeoASN messages, MaxMind DB errors, pattern matching warnings). Repository Modernisation🔗</a></h2> The project moved from a collection of legacy files to a cleaner structure: Before</th> After</th></tr></thead> README</code> + README.FIRST</code> + README.webalizer</code></td> README.md</code> (single entry point)</td></tr> ChangeLog</code> (GNU format, frozen)</td> CHANGELOG.md</code> (Keep a Changelog)</td></tr> COPYING</code></td> LICENSE</code></td></tr> TODO</code> (plain text, 2008)</td> TODO.md</code> (Markdown with [x]</code> checkboxes)</td></tr> country-codes.txt</code> (2005, informational)</td> removed</td></tr> DNS.README</code> (dead link)</td> removed</td></tr> contrib/</code> (Webalizer migration scripts)</td> removed</td></tr> config.guess</code> from 2008</td> updated to automake 1.16 (2022)</td></tr> </tbody></table> tip </div> tip </div> The full list of technical changes is available in CHANGES.md</a>. </div> </div> CI/CD Pipeline🔗</a></h2> A full GitLab CI pipeline was added covering: Build verification on Debian 12/13, Ubuntu 22.04/24.04, AlmaLinux 8/9/10</li> Source and binary tarballs on every push to main</code></li> .deb</code> package (Ubuntu 24.04 amd64) on release tags</li> .rpm</code> package (AlmaLinux 9 amd64) on release tags</li> Container image pushed to GitLab Container Registry (Ubuntu 24.04 base, all optional libs included)</li> GitLab Release creation with all artifacts uploaded to the Generic Package Registry</li> </ul> Releases are triggered by pushing a semver tag: git tag v4.0.0 git push --tags</code></pre> The container image is available as: docker pull registry.gitlab.com/aleks001/awffull:v4.0.0 docker pull registry.gitlab.com/aleks001/awffull:latest</code></pre> The other artifacts are available on the Release Page</a> What’s Next🔗</a></h2> The TODO.md</code> tracks longer-term ideas: SVG graphs (replacing the aging libgd dependency), weekly summaries, a templating system for the HTML output, and better browser/OS reporting. Contributions welcome. The project is at https://gitlab.com/aleks001/awfull</a>. AWFFull is free software licensed under GPL v3 or later. Based on The Webalizer by Bradford L. Barrett (1997–2001). Extended by Steve McInerney (2004–2008) and Aleksandar Lazic (2026–present).

Flag</th>	ENV</th>	Subcommands</th></tr></thead>
`--deployment</code></td>`	`K8S_SCALE_DEPLOYMENT</code></td>`	both</td></tr>
`--namespace</code></td>`	`K8S_SCALE_NAMESPACE</code></td>`	both</td></tr>
`--replicas</code></td>`	`K8S_SCALE_REPLICAS</code></td>`	scale</td></tr>
`--dry-run</code></td>`	`K8S_SCALE_DRY_RUN</code></td>`	both</td></tr>
`--extra-ca-bundle</code></td>`	`K8S_SCALE_EXTRA_CA_BUNDLE</code></td>`	both</td></tr> </tbody></table> The `--namespace</code> flag exists, but the CronJob does not hardcode it. Instead, the pod reads its own namespace from the Downward API and passes it through as an environment variable. That keeps the ServiceAccount’s Role safely namespace-scoped: the tool can only ever act on Deployments in the namespace where the CronJob itself runs. Not a general-purpose “scale anything anywhere” tool.</p>` Design highlights🔗</a></h2> Minimal RBAC via the scale</code> subresource🔗</a></h3> Setting replicas does not need patch</code> on the full Deployment. Kubernetes exposes deployments/scale</code> as a dedicated subresource:</p> r</span>ules</span>:</span></span> -</span> a</span>piGroups</span>:</span> [</span>"</span>apps</span>"</span>]</span></span> r</span>esources</span>:</span> [</span>"</span>deployments/scale</span>"</span>]</span></span> v</span>erbs</span>:</span> [</span>"</span>get</span>"</span>,</span> "</span>patch</span>"</span>,</span> "</span>update</span>"</span>]</span></span></code></pre> That is all the scale mode needs. Restart mode does need patch</code> on deployments</code> itself (to update the pod template annotation), so the shipped Role covers both:</p> -</span> a</span>piGroups</span>:</span> [</span>"</span>apps</span>"</span>]</span></span> r</span>esources</span>:</span> [</span>"</span>deployments</span>"</span>]</span></span> v</span>erbs</span>:</span> [</span>"</span>get</span>"</span>,</span> "</span>patch</span>"</span>]</span></span></code></pre> patch</code> on the full Deployment is broader than patch</code> on deployments/scale</code> — RBAC review at deploy time should factor that in when the CronJob is configured for restart mode.</p> Extra-CA bundle merged into the trust store🔗</a></h3> The kube client uses the in-cluster CA from the ServiceAccount token secret automatically. For clusters whose API server certificate is signed by a corporate CA that is not in the SA-mounted ca.crt</code>, --extra-ca-bundle <path></code> reads a PEM file (or a full chain of them) and appends every CERTIFICATE</code> block to kube::Config.root_cert</code>. Non-certificate blocks in the file are filtered out.</p> Rendered in Helm as an optional ConfigMap mount; in Kustomize as an opt-in Component. Neither path forces the mount when there is no corporate CA to add.</p> The rustls 0.23 CryptoProvider gotcha🔗</a></h3> If you build against kube-rs</code> with the rustls-tls</code> feature on the current release, the binary compiles cleanly — and then panics at the first TLS handshake with:</p> Could not automatically determine the process-level CryptoProvider from Rustls crate features</span></span></code></pre> rustls</code> 0.23 no longer picks a crypto backend based on downstream feature flags. The application itself must call rustls::crypto::ring::default_provider().install_default()</code> before any TLS work happens. In k8s-scale-app-rs this runs as the first step in main()</code>.</p> Two notes for anyone hitting the same wall: the panic never fires at compile time, and no test caught it locally because integration tests without a cluster do not open a TLS connection. It only surfaces when the CronJob actually tries to reach the API server.</p> Two deploy paths, independent🔗</a></h3> Helm and Kustomize both ship in the repo, and either can be used on its own:</p> Helm chart</strong> with values-{dev,preprod,prod}.yaml</code>, a mode: scale \| restart</code> toggle that swaps the subcommand argument and drops the unused replicas env, and serviceAccount.create</code> / rbac.create</code> / extraCA.enabled</code> toggles for the pieces that a given cluster may or may not need.</li> Kustomize</strong> with a base that has only the CronJob, plus three opt-in Components (serviceaccount/</code>, rbac/</code>, extra-ca/</code>) that overlays include as needed. Overlays under overlays/{dev,preprod,prod}/</code> set namespace and image tag and a patch that touches only the fields that vary per stage.</li> </ul> No Helm-inside-Kustomize glue, no Kustomize-on-Helm-render. Whichever tool matches an existing pipeline is the one to use.</p> Two container images🔗</a></h2> The default image and a -debug</code> variant, both built in the same GitHub Actions matrix — the production/-debug</code> split itself is a general pattern, covered in more depth in Part 2 of the K8s/OpenShift Best Practices series</a>:</p> File</th> Base</th> Contents</th> Approx. size</th></tr></thead> Containerfile</code></td> registry.access.redhat.com/ubi10/ubi-micro</code></td> Binary + CA trust store + /licenses/</code>, no package manager</td> ≈35 MB</td></tr> Containerfile.debug</code></td> registry.access.redhat.com/ubi10/ubi-minimal</code></td> Everything above + bash</code>, curl</code>, bind-utils</code> (dig</code>)</td> ≈115 MB</td></tr> </tbody></table> ubi10-micro</code> is not fully distroless in the Google sense — bash and coreutils are still there — but it has no package manager and no network tools, which fits the pod’s actual operational surface for a scheduled scale or restart call. The -debug</code> variant exists for the day a pod loops in ImagePullBackOff</code> or a network policy silently drops the API-server traffic and someone needs to kubectl exec</code> in with tools to check.</p> Why UBI10🔗</a></h3> The base itself, not just the two variants built from it, was a deliberate choice. I went with Red Hat’s UBI over an Alpine or a generic Docker Hub base image for one reason: it gives the image a level of maintenance stability I don’t have to police myself — CVE scanning and rebuilds are Red Hat’s job, not something I track on my own. That comes with a tradeoff worth stating plainly: UBI is not “digitally sovereign” in the sense the Digital Sovereignty in Practice</a> series covers — it is a US vendor’s supply chain, the same as pulling from Docker Hub or almost any other public registry would be. I did not find a base that solved that concern without giving up the maintenance guarantees I wanted, so UBI10 here is a compromise I made deliberately, not a claim that the underlying problem is solved.</p> Both images ship an aggregated /licenses/LICENSES.txt</code> bundled with cargo-about</code> during the container build, generated from a whitelist of accepted SPDX identifiers. A new dependency that pulls in an unfamiliar license fails the container build on purpose.</p> Supply chain🔗</a></h2> The build-publish workflow</a> runs three jobs on every push, plus a release job on tag pushes:</p> test</code></strong> — cargo fmt --check</code>, cargo clippy -D warnings</code>, cargo test --release</code>. Cluster tests auto-skip in CI (no KUBECONFIG</code>, no in-cluster SA token).</li> build-image</code></strong> — matrix over Containerfile</code> and Containerfile.debug</code>. Each variant is built with buildx, pushed to ghcr.io/git001/k8s-scale-app-rs</code>, smoke-tested (--version</code>, scale --help</code>, restart --help</code>), then: signed with cosign keyless</strong> via the GitHub OIDC token (no key management — Fulcio issues a short-lived certificate, the signature is logged to Rekor);</li> has an SPDX-JSON SBOM</strong> generated with syft and attached via cosign attest --type spdxjson</code>;</li> has SLSA build provenance</strong> pushed with actions/attest-build-provenance</code>.</li> </ul> </li> publish-chart</code></strong> — helm lint</code> and helm package</code> on every push; OCI push to oci://ghcr.io/git001/charts/k8s-scale-app-rs</code>, plus cosign sign and SLSA provenance only on tag events (avoids overwrite conflicts on GHCR-immutable tags).</li> release</code></strong> (tag events only) — creates the GitHub Release with auto-generated notes from commits since the previous tag, attaches the packaged chart .tgz</code> as an asset, and prepends the release body with pull and verify snippets.</li> </ol> All of that runs with permissions: contents:read, packages:write, id-token:write, attestations:write</code> — the minimum needed for OIDC-based signing plus attestation upload.</p> Verifying a published image🔗</a></h2> A consumer that wants to check what came out of that pipeline needs cosign v2:</p> IMG</span>=</span>g</span>h</span>c</span>r</span>.</span>i</span>o</span>/</span>g</span>i</span>t</span>0</span>0</span>1</span>/</span>k</span>8</span>s</span>-</span>s</span>c</span>a</span>l</span>e</span>-</span>a</span>p</span>p</span>-</span>r</span>s</span>:</span>v</span>1</span>.</span>0</span>.</span>0</span></span> IDENTITY</span>=</span>'</span>^https://github.com/git001/k8s-scale-app-rs/</span>'</span></span> ISSUER</span>=</span>h</span>t</span>t</span>p</span>s</span>:</span>/</span>/</span>t</span>o</span>k</span>e</span>n</span>.</span>a</span>c</span>t</span>i</span>o</span>n</span>s</span>.</span>g</span>i</span>t</span>h</span>u</span>b</span>u</span>s</span>e</span>r</span>c</span>o</span>n</span>t</span>e</span>n</span>t</span>.</span>c</span>o</span>m</span></span> </span> cosign</span> verify</span> \</span></span> -</span>-certificate-identity-regexp</span> "</span>$</span>IDENTITY</span>"</span> \</span></span> -</span>-certificate-oidc-issuer</span> "</span>$</span>ISSUER</span>"</span> \</span></span> "</span>$</span>IMG</span>"</span></span> </span> cosign</span> verify-attestation</span> -</span>-type</span> spdxjson</span> \</span></span> -</span>-certificate-identity-regexp</span> "</span>$</span>IDENTITY</span>"</span> \</span></span> -</span>-certificate-oidc-issuer</span> "</span>$</span>ISSUER</span>"</span> \</span></span> "</span>$</span>IMG</span>"</span></span> </span> cosign</span> verify-attestation</span> -</span>-type</span> slsaprovenance</span> \</span></span> -</span>-certificate-identity-regexp</span> "</span>$</span>IDENTITY</span>"</span> \</span></span> -</span>-certificate-oidc-issuer</span> "</span>$</span>ISSUER</span>"</span> \</span></span> "</span>$</span>IMG</span>"</span></span></code></pre> For enforcement inside the cluster, a cosign-aware admission controller — Kyverno</a>, Sigstore policy-controller</a>, or Connaisseur</a> — can turn the signature into a hard gate that rejects any pod trying to pull an image that was not produced by exactly this workflow. That is a separate deployment concern; the tool repo does not ship the policy itself.</p> Mirroring the image into another registry🔗</a></h2> Verifying a signature only works if the signature actually travels with the image. Not every docker pull && docker push</code>-style mirroring tool preserves it — some tools do, some quietly drop it, and one (oc-mirror v2</code>) solves a related but entirely different signature problem. I wrote up a full, sourced comparison — Which Tool Mirrors a Cosign-Signed Image into a Private Registry?</a> — with regctl</code> coming out as the most complete option for this specific image.</p> Code and repository🔗</a></h2> Repository: k8s-scale-app-rs on GitHub</a></li> Container images: ghcr.io/git001/k8s-scale-app-rs:v1.0.0</code> and ghcr.io/git001/k8s-scale-app-rs:v1.0.0-debug</code></li> Helm chart: oci://ghcr.io/git001/charts/k8s-scale-app-rs:1.0.0</code></li> </ul> Feedback and PRs welcome.</p> mergelog-rs: Rewriting a Year-2000 C Tool in Rust — and Making It 2.26× Faster aleks — Sun, 15 Mar 2026 00:00:00 +0000 mergelog</a> is a small but handy tool: it takes multiple Apache log files from servers behind a round-robin DNS and merges them into a single chronologically sorted stream. The original version 4.5 was written in C around 2000–2001, weighs in at ~450 lines, and does exactly what it promises — reliably and fast.</p> This post documents the journey from mergelog-4.5 to mergelog-rs</a>: what changed, what was surprising, and how six targeted optimizations made the Rust binary 2.26× faster</strong> than the C original.</p> Full respect and honour to Bertrand Demiddelaer</strong>, who created mergelog back in 2000. Writing a tool this fast, this correct, and this compact in C — with no external dependencies beyond zlib — is genuinely impressive. mergelog-rs stands on his shoulders.</p> The Original: mergelog 4.5🔗</a></h2> The C program reads any number of log files in Common Log Format</em> (CLF) / NCSA Combined Format</em>:</p> %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"</span></span></code></pre> Example line:</p> 93.184.216.34 - - [15/Mar/2026:12:00:00 +0100] "GET /index.html HTTP/1.1" 200 1234 "-" "curl/7.88"</span></span></code></pre> The algorithm in 4.5 is straightforward: find the earliest timestamp across all files, then iterate second-by-second from there to the end, writing all matching lines at each step. That is O(timespan × K)</strong> — slow when logs span multiple years or many files are involved.</p> Gzip support exists, but as a separate binary (zmergelog</code>) compiled with -DUSE_ZLIB</code>.</p> The Rust Rewrite: What Changed🔗</a></h2> 1. Algorithm: K-Way Merge with a Binary Heap🔗</a></h3> Instead of time-based iteration, mergelog-rs uses a k-way merge with a min-heap</strong> (std::collections::BinaryHeap</code> with reversed ordering):</p> Seed the heap with the first line from each file</li> Main loop: pop the smallest timestamp, write the line, push the next line from the same file</li> Complexity: O(N log K)</strong> — independent of the time span of the logs</li> </ul> while</span> let</span> Some</span>(</span>Entry</span> {</span> mut</span> line</span>,</span> mut</span> reader</span>,</span> file_idx</span>,</span> ..</span> }</span>)</span> =</span> heap</span>.</span>pop</span>(</span>)</span> {</span></span> out</span>.</span>write_all</span>(</span>line</span>.</span>as_bytes</span>(</span>)</span>)</span>?</span>;</span></span> line</span>.</span>clear</span>(</span>)</span>;</span></span> if</span> fill_line</span>(</span>&</span>mut</span> </span>reader</span>,</span> &</span>mut</span> line</span>)</span>?</span> {</span></span> let</span> ts</span> =</span> parse_clf_timestamp</span>(</span>&</span>line</span>)</span>.</span>unwrap_or_else</span>(</span>sentinel</span>)</span>;</span></span> heap</span>.</span>push</span>(</span>Entry</span> {</span> ts</span>,</span> file_idx</span>,</span> line</span>,</span> reader</span> }</span>)</span>;</span></span> }</span></span> }</span></span></code></pre>2. Magic-Byte Detection Instead of File Extension🔗</a></h3> The original detects gzip by binary type (zmergelog</code> vs. mergelog</code>). mergelog-rs reads the first 6 bytes of each file and detects the format automatically:</p> Format</th> Magic Bytes</th></tr></thead> gzip</td> 1F 8B</code></td></tr> bzip2</td> 42 5A 68</code> (BZh</code>)</td></tr> xz</td> FD 37 7A 58 5A 00</code></td></tr> zstd</td> 28 B5 2F FD</code></td></tr> </tbody></table> pub</span> fn</span> detect_from_bytes</span>(</span>magic</span>:</span> &</span>[</span>u8</span>]</span>)</span> -></span> Compression</span> {</span></span> if</span> magic</span>.</span>len</span>(</span>)</span> >=</span> 2</span> &&</span> magic</span>[</span>..</span>2</span>]</span> ==</span> [</span>0x1f</span>,</span> 0x8b</span>]</span> {</span> Compression</span>::</span>Gz</span> }</span></span> else</span> if</span> magic</span>.</span>len</span>(</span>)</span> >=</span> 3</span> &&</span> magic</span>[</span>..</span>3</span>]</span> ==</span> [</span>0x42</span>,</span> 0x5a</span>,</span> 0x68</span>]</span> {</span> Compression</span>::</span>Bz2</span> }</span></span> else</span> if</span> magic</span>.</span>len</span>(</span>)</span> >=</span> 6</span> &&</span> magic</span>[</span>..</span>6</span>]</span> ==</span> [</span>0xfd</span>,</span> 0x37</span>,</span> 0x7a</span>,</span> 0x58</span>,</span> 0x5a</span>,</span> 0x00</span>]</span> {</span> Compression</span>::</span>Xz</span> }</span></span> else</span> if</span> magic</span>.</span>len</span>(</span>)</span> >=</span> 4</span> &&</span> magic</span>[</span>..</span>4</span>]</span> ==</span> [</span>0x28</span>,</span> 0xb5</span>,</span> 0x2f</span>,</span> 0xfd</span>]</span> {</span> Compression</span>::</span>Zstd</span> }</span></span> else</span> {</span> Compression</span>::</span>None</span> }</span></span> }</span></span></code></pre> One binary, all formats, no file extension required. The same peek-based detection also works for stdin — BufReader::fill_buf()</code> inspects the first bytes without consuming them, so no seek is needed.</p> 3. Timezone-Aware Timestamp Comparison🔗</a></h3> The C original ignores the timezone offset in timestamps — it converts date components to local time via mktime()</code>. For logs from servers in different timezones this can produce incorrectly ordered output.</p> mergelog-rs converts every timestamp to UTC before comparison. A +0900</code> timestamp and a -0500</code> timestamp are ordered correctly, regardless of the system’s local timezone.</p> Profiling: Where Does the Time Go?🔗</a></h2> Since perf</code> was locked down on the system (perf_event_paranoid = 4</code>), a dedicated timing binary was built that measures each phase manually with std::time::Instant</code>:</p> ──── profile_timing (41.65M lines) ──────────────────</span></span> I/O (read_line) 3.91 s 32.4%</span></span> Parse (jiff strptime) 5.52 s 45.7% ← bottleneck</span></span> Heap (push/pop) 1.97 s 16.3%</span></span> Write (write_all) 0.69 s 5.7%</span></span> ─────────────────────────────────────────────────</span></span> Σ 12.08 s</span></span></code></pre> Nearly half the time was in the parser.</strong> The reason: jiff::Zoned::strptime</code> is a full parsing pipeline — format-string interpretation, timezone database lookup, error handling, heap allocation. Across ~42 million lines that adds up fast.</p> This full parsing pipeline is super handy, but in our use case it is not necessary — therefore we built a hand-rolled CLF timestamp parser.</p> Optimization 1: Hand-Rolled CLF Parser🔗</a></h2> The CLF timestamp format is completely fixed:</p> [DD/Mon/YYYY:HH:MM:SS ±HHMM]</span></span> 0123456789012345678901234 5</span></span></code></pre> A hand-written parser only needs to:</p> Find [</code> in the string</li> Read 6 integers from fixed byte positions</li> Compute the Unix timestamp via integer arithmetic</li> </ol> For step 3 we use Howard Hinnant’s proleptic Gregorian algorithm — no branches, pure multiplication:</p> fn</span> days_since_epoch</span>(</span>y</span>:</span> i64</span>,</span> m</span>:</span> i64</span>,</span> d</span>:</span> i64</span>)</span> -></span> i64</span> {</span></span> let</span> y</span> =</span> if</span> m</span> <=</span> 2</span> {</span> y</span> -</span> 1</span> }</span> else</span> {</span> y</span> }</span>;</span></span> let</span> era</span> =</span> if</span> y</span> >=</span> 0</span> {</span> y</span> }</span> else</span> {</span> y</span> -</span> 399</span> }</span> /</span> 400</span>;</span></span> let</span> yoe</span> =</span> y</span> -</span> era</span> </span> 400</span>;</span></span> let</span> doy</span> =</span> (</span>153</span> </span> (</span>m</span> +</span> if</span> m</span> ></span> 2</span> {</span> -</span>3</span> }</span> else</span> {</span> 9</span> }</span>)</span> +</span> 2</span>)</span> /</span> 5</span> +</span> d</span> -</span> 1</span>;</span></span> let</span> doe</span> =</span> yoe</span> </span> 365</span> +</span> yoe</span> /</span> 4</span> -</span> yoe</span> /</span> 100</span> +</span> doy</span>;</span></span> era</span> </span> 146_097</span> +</span> doe</span> -</span> 719_468</span></span> }</span></span></code></pre> Result:</strong> parse time dropped from 5.52 s to 1.50 s — 3.7× faster</strong>.</p> Optimization 2: Reuse the String Buffer🔗</a></h2> The original implementation allocated a new String</code> for every line read:</p> //</span> before: fresh allocation per line</span></span> fn</span> read_line</span>(</span>reader</span>:</span> &</span>mut</span> dyn</span> BufRead</span>)</span> -></span> Result</span><</span>Option</span><</span>String</span>></span>></span> {</span></span> let</span> mut</span> line</span> =</span> String</span>::</span>new</span>(</span>)</span>;</span> //</span> ← new every time</span></span> reader</span>.</span>read_line</span>(</span>&</span>mut</span> line</span>)</span>?</span>;</span></span> ...</span></span> }</span></span></code></pre> The fix is simple: when an Entry</code> is popped from the heap, you receive line: String</code> with full ownership. Instead of dropping it, clear it and read the next line directly into the same buffer:</p> //</span> after: capacity is retained</span></span> while</span> let</span> Some</span>(</span>Entry</span> {</span> mut</span> line</span>,</span> mut</span> reader</span>,</span> file_idx</span>,</span> ..</span> }</span>)</span> =</span> heap</span>.</span>pop</span>(</span>)</span> {</span></span> out</span>.</span>write_all</span>(</span>line</span>.</span>as_bytes</span>(</span>)</span>)</span>?</span>;</span></span> line</span>.</span>clear</span>(</span>)</span>;</span> //</span> ← keeps the heap allocation</span></span> if</span> fill_line</span>(</span>&</span>mut</span> </span>reader</span>,</span> &</span>mut</span> line</span>)</span>?</span> {</span></span> let</span> ts</span> =</span> parse_clf_timestamp</span>(</span>&</span>line</span>)</span>.</span>unwrap_or_else</span>(</span>sentinel</span>)</span>;</span></span> heap</span>.</span>push</span>(</span>Entry</span> {</span> ts</span>,</span> file_idx</span>,</span> line</span>,</span> reader</span> }</span>)</span>;</span></span> }</span></span> }</span></span></code></pre> Since consecutive log lines are similar in length, the buffer stops growing after the first few lines — no malloc</code> calls in the hot path.</p> Optimization 3: mimalloc as the Global Allocator🔗</a></h2> With most per-line allocations already eliminated, there are still smaller allocations happening — buffer growth during seeding, internal BinaryHeap</code> resizing, and the occasional String</code> capacity bump. Swapping out the default glibc allocator for mimalloc</a> is a one-liner in Rust:</p> #</span> Cargo.toml</span></span> mimalloc</span> =</span> {</span> version</span> =</span> "</span>0.1</span>"</span>,</span> default-features</span> =</span> false</span> }</span></span></code></pre>//</span> main.rs</span></span> use</span> mimalloc</span>::</span>MiMalloc</span>;</span></span> #</span>[</span>global_allocator</span>]</span></span> static</span> GLOBAL</span>:</span> MiMalloc</span> =</span> MiMalloc</span>;</span></span></code></pre> That’s it — no other code changes. mimalloc is designed for throughput-heavy workloads with many small, short-lived allocations, which matches our remaining allocation pattern well.</p> Result:</strong> 5.55 s → 5.39 s — a further ~3% improvement.</p> Not a dramatic win on its own, but it’s essentially free: two lines of code, no logic changes, and it stacks on top of everything else.</p> Optimization 4: 4 MiB Read Buffers + SIMD Newline Search🔗</a></h2> After the previous three optimizations, profiling showed I/O was still the largest remaining cost at 47% of total time. Two changes address this together.</p> 4 MiB read buffers</strong> — The previous BufReader</code> used a 256 KiB internal buffer, meaning a syscall every ~256 KiB of data. Increasing to 4 MiB reduces syscall frequency by 16×, which shows up directly in system time.</p> SIMD newline search via memchr</code></strong> — Instead of BufReader::read_line</code>, the line reader now uses fill_buf()</code> + memchr::memchr()</code> directly. The memchr</code> crate uses AVX2/SSE2 to scan 16 or 32 bytes per cycle when searching for \n</code>:</p> fn</span> fill_line</span>(</span>reader</span>:</span> &</span>mut</span> dyn</span> BufRead</span>,</span> buf</span>:</span> &</span>mut</span> String</span>)</span> -></span> Result</span><</span>bool</span>></span> {</span></span> loop</span> {</span></span> let</span> available</span> =</span> reader</span>.</span>fill_buf</span>(</span>)</span>?</span>;</span></span> if</span> available</span>.</span>is_empty</span>(</span>)</span> {</span> return</span> Ok</span>(</span>false</span>)</span>;</span> }</span></span> match</span> memchr</span>(</span>b</span>'</span>\</span>n</span>'</span>,</span> available</span>)</span> {</span></span> Some</span>(</span>pos</span>)</span> =></span> {</span></span> buf</span>.</span>push_str</span>(</span>std</span>::</span>str</span>::</span>from_utf8</span>(</span>&</span>available</span>[</span>..</span>pos</span> +</span> 1</span>]</span>)</span>?</span>)</span>;</span></span> reader</span>.</span>consume</span>(</span>pos</span> +</span> 1</span>)</span>;</span></span> return</span> Ok</span>(</span>true</span>)</span>;</span></span> }</span></span> None</span> =></span> {</span></span> buf</span>.</span>push_str</span>(</span>std</span>::</span>str</span>::</span>from_utf8</span>(</span>available</span>)</span>?</span>)</span>;</span></span> let</span> len</span> =</span> available</span>.</span>len</span>(</span>)</span>;</span></span> reader</span>.</span>consume</span>(</span>len</span>)</span>;</span></span> }</span></span> }</span></span> }</span></span> }</span></span></code></pre> Result:</strong> 5.39 s → 4.43 s — another 18% improvement</strong>, bringing the total to 2.26× faster</strong> than the C original.</p> A Road Not Taken: Parallel Reader Threads🔗</a></h2> An obvious next step seemed to be spawning one thread per input file — each thread handles its own I/O and decompression, sending (timestamp, line)</code> pairs to the merge thread via a bounded mpsc</code> channel:</p> Thread 1: read + decompress + parse → channel ─┐</span></span> Thread 2: read + decompress + parse → channel ─┤→ Merge thread → stdout</span></span> ... ─┘</span></span></code></pre> For compressed files</strong> (bz2, xz) this is conceptually attractive: decompression is CPU-bound and can genuinely run in parallel.</p> For plain-text files</strong> on a single disk, it does not help — the bottleneck is disk read bandwidth, and parallel reads from the same physical device cause seek contention or simply saturate the same I/O bus. Benchmarking confirmed this: the threaded version achieved identical wall-clock time on plain-text files, while adding per-line heap allocation overhead (the channel requires sending owned String</code> values, making buffer reuse impossible across the thread boundary). The result was slightly slower</em> than the sequential version with large buffers.</p> Takeaway:</strong> threading is the right tool when the bottleneck is CPU-bound work that can be parallelised per-file (compressed files on a multi-core system). It is the wrong tool when the bottleneck is a shared I/O resource. Measure first.</p> Final Results🔗</a></h2> Measured on 7 log files × 1 GiB = 7 GiB total, 41.65 million lines:</p> Version</th> Wall time</th> User</th> System</th></tr></thead> mergelog-4.5 (C)</td> 10.03 s</td> 5.93 s</td> 4.12 s</td></tr> mergelog-rs v1 (jiff)</td> 10.39 s</td> 9.13 s</td> 1.25 s</td></tr> + hand-rolled parser</td> 5.99 s</td> 4.75 s</td> 1.23 s</td></tr> + buffer reuse</td> 5.55 s</td> 4.29 s</td> 1.26 s</td></tr> + mimalloc</td> 5.39 s</td> 4.17 s</td> 1.22 s</td></tr> + 4 MiB buffers + memchr</strong></td> 4.43 s</strong></td> 3.61 s</strong></td> 0.81 s</strong></td></tr> </tbody></table> mergelog-rs ran 2.26 ± 0.02 times faster than mergelog-4.5</span></span></code></pre> Memory and CPU Usage🔗</a></h2> Measured with /usr/bin/time -v</code> on the same 7 × 1 GiB workload:</p> </th> mergelog-4.5</th> mergelog-rs</th></tr></thead> Wall time</td> 9.63 s</td> 4.28 s</td></tr> User time</td> 5.61 s</td> 3.42 s</td></tr> System time</td> 4.01 s</td> 0.84 s</td></tr> CPU utilisation</td> 99%</td> 99%</td></tr> Peak RSS</strong></td> 1.6 MiB</strong></td> 30.7 MiB</strong></td></tr> Involuntary ctx switches</td> 105</td> 122</td></tr> </tbody></table> Memory</strong> — The picture changed significantly with the 4 MiB read buffers. mergelog-4.5 uses 1.6 MiB; mergelog-rs now uses 30.7 MiB. The breakdown is straightforward: 7 files × 4 MiB read buffer = 28 MiB, plus the write BufWriter</code>, the binary itself, and mimalloc metadata. This is a deliberate trade-off — RAM is cheap, syscalls are not. At 30 MiB to process 7 GiB of data the ratio is still extremely lean, and the buffer size is a single constant in reader.rs</code> that can be tuned if memory is constrained.</p> System time</strong> — The most striking difference is kernel time: mergelog-4.5 spends 4.01 s in the kernel versus 0.84 s for mergelog-rs. Two factors: the C version calls write(1, ...)</code> directly for every line (~42 million syscalls), and its read buffers are only 32 KiB. mergelog-rs batches writes via BufWriter</code> and reads via 4 MiB BufReader</code>, cutting kernel round-trips dramatically.</p> CPU</strong> — Both tools are single-threaded and max out one core at 99%.</p> Takeaways🔗</a></h2> Five lessons from this project:</p> 1. Measure before optimizing.</strong> Without the timing binary the focus might have landed on the heap. But jiff::strptime</code> was costing nearly 3× more than every other phase combined.</p> 2. Libraries are great until they aren’t.</strong> jiff</code> is an excellent crate for correct timezone-aware date handling. But when you parse the same fixed-format string 42 million times, you pay a steep price for generality. A 30-line hand-rolled parser that exploits the known fixed layout beats it by 3.7×.</p> 3. Sometimes the allocator is the last few percent.</strong> After eliminating almost all allocations from the hot path, swapping in mimalloc took two lines of code and gave another 3% for free. It won’t rescue a poorly written program, but on an already optimized one it’s a cheap final step worth trying.</p> 4. Buffer size matters more than you think.</strong> Going from 256 KiB to 4 MiB read buffers — a one-line change — cut system time by 34% and wall time by 18%. Every syscall is a round-trip to the kernel; fewer is better.</p> 5. Parallelism requires the right bottleneck.</strong> Spawning reader threads per file is conceptually appealing for compressed logs where decompression is CPU-bound. But for plain-text files on a single disk, the bottleneck is I/O bandwidth — adding threads just adds channel overhead and per-line heap allocation, making things slightly slower</em>. Threading is a tool, not a universal solution.</p> 6. Ownership is not a burden.</strong> Buffer reuse in C requires careful explicit lifetime management and is easy to get wrong. In Rust, the ownership system makes it the obvious and safe solution — line.clear()</code> instead of free(line); line = malloc(...)</code>, with the compiler guaranteeing no use-after-free.</p> The code is available under GPL-3.0-or-later. The original mergelog 4.5 was released as GPL-2.0-or-later, which explicitly permits upgrading to any later version of the GPL. GPL-3.0 adds patent protection clauses and anti-tivoization provisions that GPL-2.0 lacks — a straightforward upgrade with no downsides for a command-line tool like this.</p> AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser aleks — Thu, 12 Mar 2026 00:00:00 +0000 AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser🔗</a></h1> Background🔗</a></h2> AWFFull 4.0.0</a> is a web server log analyser. It reads Apache/nginx/Squid access logs and produces HTML reports with graphs showing traffic, top URLs, referrers, user agents, country statistics, and more. It was originally forked from The Webalizer</a> by Bradford L. Barrett (1997–2001), extended as AWFFull by Steve McInerney from 2004, and actively developed until 2008, when it was last released as version 3.10.2.</p> It still works well in 2026 — the core log-parsing logic is solid and the output is clean. But the codebase had accumulated 18 years of code: EOL libraries, dead links, obsolete command-line flags, a broken GeoIP implementation, and a very old config.guess</code>. Time for a proper cleanup.</p> This post covers what changed in AWFFull 4.0.0 and why.</p> Is a Static Web Log Analyser Still Useful?🔗</a></h2> Good question. The short answer is: yes — and arguably more so than before.</p> Modern web analytics platforms — Google Analytics, Matomo, Plausible, Grafana with Loki, ELK stacks — are excellent for live dashboards and drill-down exploration. But they share a common weakness: they are slow when you need to process large volumes of historical log data and render the results as a complete set of graphs and tables</strong>.</p> Feed a year of Apache logs (say, 500 million lines across 50 GB of gzipped files) into an ELK pipeline and you’ll be waiting a long time for the data to be ingested, indexed, and ready to query. Then wait again each time you open a dashboard. Grafana rendering a year’s worth of nginx traffic across 30 panels? Expect seconds of load time per page, per query.</p> AWFFull’s approach is different: it processes the raw log files directly, keeps all state in a compact binary history file, and writes static HTML + PNG output</strong> that loads instantly in any browser — even on a machine from 2010, over a slow connection, or offline entirely. There is no query layer, no index, no database, no running server. The report is just files.</p> For the typical use case — a sysadmin or developer who wants a monthly overview of who accessed their server, from where, with what agents, hitting which URLs — this is exactly the right trade-off:</p> Fast to generate</strong>: processing a month of logs takes seconds, not minutes</li> Instant to view</strong>: static HTML, no round-trips, no query engine</li> Self-contained</strong>: the report directory can be archived, moved, or served from any static file server</li> Privacy-friendly</strong>: no third-party JavaScript, no tracking pixels, no data leaving your infrastructure — the raw IP addresses stay in your own logs</li> Works on air-gapped systems</strong>: nothing requires internet access at runtime</li> </ul> It also complements rather than replaces live monitoring. You would typically use Prometheus/Grafana (or similar) to watch your infrastructure in real time, and AWFFull to produce the monthly historical report that goes into a ticket or a management summary.</p> Shipping Access Logs to Object Storage and Running in Containers🔗</a></h3> The operational details — how to ship access logs to S3 or Azure Blob with Fluent Bit, Vector, or Filebeat/Logstash, how to handle concurrent writes and log loss at high traffic, and how to run AWFFull as a stateless container against object storage — are covered in the companion post:</p> AWFFull in the Cloud: Shipping Logs to Object Storage and Running in Containers</a></strong></p> The Big Ticket Items🔗</a></h2> PCRE → PCRE2🔗</a></h3> AWFFull used libpcre (PCRE version 1) for log-line regex parsing. PCRE 1 has been officially end-of-life since 2012 and is being dropped by distributions. We migrated to PCRE2</strong>, the current maintained successor with a cleaner API and better Unicode support. The change touched parser.c</code>, common.h</code>, and configure.ac</code>.</p> GeoIP: From Legacy .dat</code> to libmaxminddb🔗</a></h3> The old code used libGeoIP</strong> with a proprietary binary .dat</code> format that MaxMind deprecated in 2019. Country lookups were broken for anyone who hadn’t kept a seven-year-old database around.</p> AWFFull 4.0.0 replaces this with libmaxminddb</strong>, the open standard MMDB format used by both MaxMind (GeoLite2) and DB-IP. The migration is transparent — IPv4 and IPv6 are both handled by a single MMDB_lookup_string()</code> call, with no separate code paths needed.</p> Free databases updated monthly:</p> Country</strong>: DB-IP Country Lite</a></li> ASN</strong>: DB-IP ASN Lite</a></li> </ul> ASN Statistics: New Table and Pie Chart🔗</a></h3> Once you have a MMDB-capable lookup layer, adding ASN (Autonomous System Number) statistics is almost free. A second database (GeoASNDatabase</code>) maps IPs to their network operator — think “Cloudflare”, “Amazon AWS”, “Deutsche Telekom”. The report now shows a Top ASNs</strong> table and pie chart, answering the question “which networks send the most traffic to my site?”</em></p> GeoASN yes</span></span> GeoASNDatabase /usr/share/GeoIP/dbip-asn-lite.mmdb</span></span></code></pre> Performance note: both GeoIP and GeoASN lookups happen in the output phase</em>, once per unique IP address — not once per log line. On a typical site the overhead is negligible.</p> Hyperscan: SIMD Pattern Matching🔗</a></h3> AWFFull supports extensive pattern lists for ignoring, grouping, and hiding hosts, URLs, referrers, and agents. The original code used a Boyer-Moore-Horspool</strong> linear scan: each log record was matched against every pattern in turn.</p> When Intel Hyperscan</strong> (libhyperscan</code>) is installed, AWFFull now compiles all patterns from a given list into a single Hyperscan database and matches them in one SIMD pass per record. For large pattern lists — say, 50+ bot/spider agent patterns — this is significantly faster. Hyperscan is auto-detected at configure time and falls back to the existing BMH path if unavailable.</p> sudo</span> apt</span> install</span> libhyperscan-dev</span></span> ./configure</span> &&</span> make</span></span></code></pre>mimalloc: Better Heap Performance🔗</a></h3> For large log files with millions of unique hosts, URLs, and referrers, heap allocation overhead becomes measurable. Adding mimalloc</strong> (Microsoft’s high-performance allocator) as an optional drop-in was straightforward via the explicit mi_calloc</code> API. Auto-detected at configure time.</p> XXH3 Hash Function🔗</a></h3> The internal hash table used djb2, a good general-purpose hash from 1991. AWFFull 4.0.0 replaces it with XXH3</strong> (via the bundled xxhash.h</code>), which offers better distribution and significantly higher throughput on modern CPUs. The table load-factor was also tuned to reduce collisions.</p> User-Visible Improvements🔗</a></h2> Date Stamp on Graphs🔗</a></h3> Every generated PNG graph now shows the current date</strong> (YYYY-MM-DD</code>) in the top-right corner, aligned with the title baseline. Small but useful when you’re looking at archived reports six months later.</p> Next / Previous Month Navigation🔗</a></h3> Monthly report pages now show navigation links at the top:</p> ← February 2026 \| [Index] \| April 2026 →</span></span></code></pre> Links are derived from the history file — only months that actually exist in the database are linked.</p> Updated Browser/Bot Detection🔗</a></h3> The sample.conf</code> browser/agent detection list dated from 2005 and contained entries like “Netscape 4.5”, “Internet Explorer 3.0 (Win95)”, and “MSNBot” — none of which have been seen in the wild for a decade or more. The list has been completely rewritten:</p> Removed:</strong> Netscape 1–4.x, Internet Explorer 3–7, Firebird, Galeon, Camino, msnbot (all entries)</p> Added:</strong> Chrome/Chromium, Edge (Chromium), Brave, Samsung Browser, Firefox, Safari, Opera (OPR/), modern search bots (Bingbot, DuckDuckBot, YandexBot, Baidu, Applebot, AhrefsBot, SemrushBot), AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Bytespider, CCBot, Amazonbot), and CLI tools (curl, Wget, python-requests).</p> Build System and Tooling🔗</a></h2> CLI Cleanup: 14 Flags Removed🔗</a></h3> AWFFull inherited many single-letter flags from the original Webalizer. Flags like -G</code> (suppress hourly graph), -L</code> (suppress legend), -M</code> (mangle agents), -x</code> (HTML extension), and -I</code> (index alias) had exact equivalents in the config file. At version 4.0.0 — a major version bump — these were removed from the CLI, along with three fully deprecated no-op flags (-d</code>, -q</code>, -Q</code>).</p> The --help</code> output now documents the new verbosity levels (-v</code> through -vvvvv</code>) and includes a note pointing config-file-only settings to awffull.conf(5)</code>.</p> Man Pages: DocBook XML → Pandoc Markdown🔗</a></h3> The existing man pages were written in DocBook XML 4.5 and built with xsltproc</code> — a tool that requires xsltproc</code>, docbook-xsl</code>, and several XML schema packages, none of which are installed by default on most systems in 2026.</p> The man pages are now maintained as pandoc Markdown</strong> (doc/awffull.1.md</code>, doc/awffull.conf.5.md</code>) and built with:</p> pandoc</span> -</span>s</span> -</span>t</span> man</span> awffull.1.md</span> -</span>o</span> awffull.1</span></span></code></pre> configure.ac</code> auto-detects pandoc. Pre-built man pages remain in the repository as a fallback for systems without pandoc. The DocBook XML sources are retained in doc/</code> for historical reference.</p> Translations: Quality Over Quantity🔗</a></h3> AWFFull shipped with 33 language files, most of which were approximately 29% complete and had not been updated since 2008. Rather than shipping incomplete translations that silently fall back to English mid-report, we reduced to 6 fully complete languages</strong> (German, Finnish, Italian, Swedish, Brazilian Portuguese, Indonesian) and brought all of them to 100% (524/524 strings)</strong>.</p> The .pot</code> template was regenerated with xgettext</code> to include all new strings added in 4.0.0 (GeoASN messages, MaxMind DB errors, pattern matching warnings).</p> Repository Modernisation🔗</a></h2> The project moved from a collection of legacy files to a cleaner structure:</p> Before</th> After</th></tr></thead> README</code> + README.FIRST</code> + README.webalizer</code></td> README.md</code> (single entry point)</td></tr> ChangeLog</code> (GNU format, frozen)</td> CHANGELOG.md</code> (Keep a Changelog)</td></tr> COPYING</code></td> LICENSE</code></td></tr> TODO</code> (plain text, 2008)</td> TODO.md</code> (Markdown with [x]</code> checkboxes)</td></tr> country-codes.txt</code> (2005, informational)</td> removed</td></tr> DNS.README</code> (dead link)</td> removed</td></tr> contrib/</code> (Webalizer migration scripts)</td> removed</td></tr> config.guess</code> from 2008</td> updated to automake 1.16 (2022)</td></tr> </tbody></table> tip </div> tip</strong></span> </div> The full list of technical changes is available in CHANGES.md</a>.</p> </div> </div> CI/CD Pipeline🔗</a></h2> A full GitLab CI pipeline was added covering:</p> Build verification</strong> on Debian 12/13, Ubuntu 22.04/24.04, AlmaLinux 8/9/10</li> Source and binary tarballs</strong> on every push to main</code></li> .deb</code> package</strong> (Ubuntu 24.04 amd64) on release tags</li> .rpm</code> package</strong> (AlmaLinux 9 amd64) on release tags</li> Container image</strong> pushed to GitLab Container Registry (Ubuntu 24.04 base, all optional libs included)</li> GitLab Release</strong> creation with all artifacts uploaded to the Generic Package Registry</li> </ul> Releases are triggered by pushing a semver tag:</p> git</span> tag</span> v4.0.0</span></span> git</span> push</span> -</span>-tags</span></span></code></pre> The container image is available as:</p> docker</span> pull</span> registry.gitlab.com/aleks001/awffull:v4.0.0</span></span> docker</span> pull</span> registry.gitlab.com/aleks001/awffull:latest</span></span></code></pre> The other artifacts are available on the Release Page</a></p> What’s Next🔗</a></h2> The TODO.md</code> tracks longer-term ideas: SVG graphs (replacing the aging libgd dependency), weekly summaries, a templating system for the HTML output, and better browser/OS reporting. Contributions welcome.</p> The project is at https://gitlab.com/aleks001/awfull</a>.</p> AWFFull is free software licensed under GPL v3 or later.</em> Based on The Webalizer by Bradford L. Barrett (1997–2001).</em> Extended by Steve McInerney (2004–2008) and Aleksandar Lazic (2026–present).</em></p>

mergelog-rs: Rewriting a Year-2000 C Tool in Rust — and Making It 2.26× Faster

AWFFull 4.0.0 — Modernising a 2008 Web Log Analyser

Alek's Blog - mimalloc

k8s-scale-app-rs: Scale or Restart a Kubernetes Deployment from a CronJob