regex-bench — ezi_gex vs Rust regex vs Go regexp

A real, rigorous, three-way regex benchmark environment for the three engines, modelled on rebar (the de-facto cross-engine regex barometer). It runs real-world patterns over real haystacks — the same files rebar uses — with one identical measurement protocol implemented natively in each language, and renders a single human-readable report.

./run.sh                  # build + run all three engines, then write results/REPORT.md
./run.sh --quick          # fast, lower-confidence pass (for iterating)
./run.sh --engines ezi,rust   # subset (names: ezi, rust, go)
./run.sh --report-only        # re-render the report from existing results/*.tsv

Requirements: zig (0.17.0-dev), cargo + rustc, go (1.24+), python3. First run fetches the real corpora (≈1.8 MB, once), the Rust regex crate, and the ezi_gex engine itself from GitHub — pinned to a commit in zig/build.zig.zon, so the benchmark is fully self-contained and anyone can reproduce it without a local checkout. To measure a different build, bump that #<commit> (e.g. zig fetch --save a newer one).

What it measures

• Operation: count every non-overlapping leftmost match over the whole haystack. Each engine uses its natural API for that:
  • ezi_gex — re.count(&scratch, haystack) (allocation-free)
  • Rust — re.find_iter(haystack).count() (allocation-free)
  • Go — len(re.FindAllIndex(haystack, -1)). Go’s stdlib has no allocation-free counting iterator, so its number includes materializing the match positions — a real, unavoidable cost of Go’s API, reported honestly.
• Compile time — time to build the matcher object from the pattern string (engine construction). For ezi_gex this includes its eager-DFA determinization, so it is the most interesting column to watch.
• Correctness gate — all three engines must report the same match count for every (pattern × corpus) cell, or that cell is flagged. This is what makes the comparison genuinely apples-to-apples (and it doubles as a differential test of all three engines against each other).

Measurement protocol (identical in every runner)

1. Compile the regex once (excluded from search timing).
2. Warm up (RB_WARMUP iterations) to warm lazy caches / branch predictors.
3. Collect one timing sample per iteration until both a minimum sample count and a minimum wall-clock have elapsed, capped by a maximum.
4. Report the median (headline), plus min (best) and p90 (spread).
5. Compile time is measured the same way, separately.

Tuning knobs (env vars; run.sh sets rigorous defaults, or quick ones with --quick, and forwards them to all three engines):

(The Zig runner takes these as CLI args — this toolchain’s env API moved under std.Io; run.sh forwards them positionally so all three stay in lockstep.)

Corpora (corpus/)

The first three are downloaded byte-identical to rebar, so results sit on the same haystacks as the reference benchmark. logs.txt is generated deterministically because the prose corpora contain no emails / dates / IPs / URLs to exercise the data-extraction patterns.

Patterns (cases.tsv)

Tab-separated: name, corpora (comma list), pattern (the default, fed verbatim to every engine), then optional engine:pattern overrides.

The pattern set covers the regex jobs that actually show up in the wild: literal & multi-literal search, case-insensitive search, word/character-class tokenization, Unicode letters/numbers/scripts, proximity (“X near Y”), and structured extraction (IPv4, ISO date, email, URL, full log-line parse).

Cross-engine parity rules

These are baked into the corpus mapping so that every measured cell is a true apples-to-apples comparison (and the correctness gate stays green):

• Perl classes \w \d \s and \b are Unicode in Rust/ezi_gex but ASCII in Go (RE2). They therefore run only on pure-ASCII corpora (logs; plus sherlock for \d and for \b around an ASCII literal, both verified to agree). Unicode word scans use \p{L} / \p{N}, which are Unicode in all three.
• Bare script names \p{Han} / \p{Cyrillic} are accepted by Go and Rust; ezi_gex spells them \p{Script=…} — handled by an ezi: override. The match-count cross-check proves the variants are equivalent.

Output

run.sh writes:

• results/zig.tsv, results/rust.tsv, results/go.tsv — raw per-cell data (median/min/p90 search ns, compile ns, match counts, corpus bytes).
• results/meta.txt — machine + engine versions.
• results/REPORT.md — the human-readable report: environment, correctness cross-check, per-corpus search-throughput tables (auto-scaled MiB/s / GiB/s, fastest engine bolded), a compile-time table, and an aggregate summary (geometric-mean speed vs ezi_gex, cells-won).

Absolute throughput drifts run-to-run with thermal / background load on a laptop; read the numbers as directional. The match-count cross-check is exact.

Layout

regex-bench/
├── run.sh            # one-command driver (build + run + report)
├── corpus.py         # provision corpora (download real + generate logs)
├── report.py         # merge results/*.tsv → REPORT.md (+ console summary)
├── cases.tsv         # benchmark definitions (name, corpora, pattern, overrides)
├── corpus/           # haystacks (provisioned by corpus.py; git-ignored)
├── results/          # *.tsv + REPORT.md (sample run committed; regenerated by ./run.sh)
├── zig/              # ezi_gex runner (fetches the engine from GitHub, pinned in build.zig.zon)
├── rust/             # Rust `regex` runner (release + LTO, edition 2024)
└── go/               # Go `regexp` runner (stdlib RE2)