Micro-benchmarks

Criterion measurements of Spindle alone, per commit — the algorithmic layer, measured inside the library. For the four-way comparison against Synapse, Continuwuity and Tuwunel, including every loss and its investigation, see Spindle vs the field.

37measurements

3comparisons

Commit 63c8c30001bf on main · 2026-09-02T21:47:40Z · runner github-actions/ubuntu-24.04

Generated from Criterion output on every push to main. Ratios are the result; wall times are context. Absolute times do not survive a change of runner, and a shared CI runner is both slower and noisier than a workstation by an amount that varies run to run. Everything here is algorithmic, measured inside the library — none of it is a server throughput figure and none of it should be quoted as one.

Fork resolution vs ruma-state-res

Size	Spindle	Comparison	Ratio
1	9.93 µs	18 µs	1.81x
4	23.3 µs	59.8 µs	2.57x
16	104 µs	245 µs	2.36x
64	539 µs	961 µs	1.78x
256	2.45 ms	4.15 ms	1.69x

State lookup: our HAMT vs HashMap

Size	Spindle	Comparison	Ratio
1000	168 ns	24.4 ns	6.87x slower
50000	185 ns	26.3 ns	7.03x slower

Every measurement

All benchmarks in this run

Benchmark	Mean	Median	95% CI
`delayed_firing_tick/100`	1.73 ms	1.73 ms	1.69 ms – 1.77 ms
`delayed_firing_tick/1000`	3.59 ms	3.61 ms	3.53 ms – 3.65 ms
`delayed_firing_tick/10000`	26 ms	25.1 ms	24.7 ms – 27.4 ms
`delayed_idle_tick/100`	587 ns	586 ns	586 ns – 589 ns
`delayed_idle_tick/1000`	607 ns	606 ns	606 ns – 609 ns
`delayed_idle_tick/10000`	669 ns	667 ns	667 ns – 671 ns
`delayed_restart/10`	1.44 µs	1.43 µs	1.43 µs – 1.44 µs
`delayed_restart/100`	1.59 µs	1.58 µs	1.58 µs – 1.59 µs
`delayed_restart/1000`	1.89 µs	1.89 µs	1.89 µs – 1.9 µs
`fork resolution/ruma-state-res/1`	18 µs	17.9 µs	17.9 µs – 18.1 µs
`fork resolution/ruma-state-res/16`	245 µs	245 µs	245 µs – 246 µs
`fork resolution/ruma-state-res/256`	4.15 ms	4.14 ms	4.14 ms – 4.15 ms
`fork resolution/ruma-state-res/4`	59.8 µs	59.6 µs	59.6 µs – 60 µs
`fork resolution/ruma-state-res/64`	961 µs	960 µs	960 µs – 962 µs
`fork resolution/spindle window merge/1`	9.93 µs	9.69 µs	9.84 µs – 10 µs
`fork resolution/spindle window merge/16`	104 µs	102 µs	103 µs – 105 µs
`fork resolution/spindle window merge/256`	2.45 ms	2.44 ms	2.44 ms – 2.46 ms
`fork resolution/spindle window merge/4`	23.3 µs	23 µs	23.1 µs – 23.5 µs
`fork resolution/spindle window merge/64`	539 µs	541 µs	534 µs – 544 µs
`fork_window_two_event_fork/bounded/100`	404 ns	403 ns	403 ns – 405 ns
`fork_window_two_event_fork/bounded/1000`	453 ns	451 ns	451 ns – 455 ns
`fork_window_two_event_fork/bounded/10000`	455 ns	453 ns	453 ns – 457 ns
`fork_window_two_event_fork/exhaustive/100`	22.7 µs	22.6 µs	22.6 µs – 22.8 µs
`fork_window_two_event_fork/exhaustive/1000`	292 µs	290 µs	291 µs – 294 µs
`fork_window_two_event_fork/exhaustive/10000`	3.6 ms	3.6 ms	3.6 ms – 3.61 ms
`state_lookup/hamt/1000`	168 ns	168 ns	168 ns – 168 ns
`state_lookup/hamt/50000`	185 ns	185 ns	184 ns – 185 ns
`state_lookup/hashmap/1000`	24.4 ns	24.4 ns	24.4 ns – 24.5 ns
`state_lookup/hashmap/50000`	26.3 ns	26.2 ns	26.2 ns – 26.3 ns
`state_retained_updates/clone_per_event/100`	453 µs	452 µs	452 µs – 454 µs
`state_retained_updates/clone_per_event/1000`	54.3 ms	53.5 ms	52.8 ms – 56 ms
`state_retained_updates/hamt/100`	246 µs	246 µs	246 µs – 247 µs
`state_retained_updates/hamt/1000`	3.88 ms	3.87 ms	3.88 ms – 3.89 ms
`state_retained_updates/hamt/10000`	62.3 ms	62.2 ms	62.2 ms – 62.5 ms
`state_retained_updates/imbl/100`	148 µs	148 µs	148 µs – 149 µs
`state_retained_updates/imbl/1000`	1.76 ms	1.76 ms	1.76 ms – 1.77 ms
`state_retained_updates/imbl/10000`	37.9 ms	38 ms	37.7 ms – 38.1 ms

Raw data: latest.json. Method, caveats and what each comparison does and does not establish: docs/benchmarks.md.