Performance

Headline numbers from the offline benchmark suite at benchmark/. Run locally with:

julia --project=benchmark benchmark/runbench.jl

The full report (benchmark/PERF_REPORT.md) covers methodology, hardware, record-by-record allocation profiles, and a flat profile per hot path.

Read throughput

For million-record-plus queries, prefer foreach_record over get_range — it streams record-by-record with effectively no GC pressure.

Live throughput

In-process loopback measurements against a mock TCP gateway:

These are upper bounds. Real network feeds are typically far slower (the gateway throttles to wire rate) and the bottleneck shifts to the gateway, not the client.

Capture: compression level

stream_to_file and stream_multi_to_files default to compress_level = 1. Empirically L1 is ~35% faster than L3 on the write boundary in exchange for ~15% larger files. For archival captures where storage cost dominates, bump to L3-L9; for low-latency / high-throughput captures keep the default.

Typed channels in Live

Live(...; typed = true) returns concrete-typed channels (Channel{T} per schema) from subscribe!. For the OPRA CMBP-1 replay benchmark this delivers ~25% higher per-cycle throughput over untyped mode. Allocation is roughly equal — Julia's stdlib Channel.put!/take! allocates Condition bookkeeping regardless of element type — so the win is in dispatch, not GC.

When to use what

• Historical, small query (≤ 1 GB): get_range → DBNStore → conversion helpers. Easy and ergonomic.
• Historical, large query: foreach_record with a typed callback, or submit_job + batch_download and decode the resulting files with DBN.foreach_record directly.
• Live, single consumer: Live do-block + untyped iterator. Easy to write, no setup.
• Live, multi-schema high throughput: typed mode (typed = true), one @async per schema reading the typed channel.
• Live capture to disk: stream_to_file or stream_multi_to_files. For custom loops, open_dbn_writer
  • write_record!.