Micro-benchmarks

Criterion measurements of Spindle alone, per commit — the algorithmic layer, measured inside the library. For the four-way comparison against Synapse, Continuwuity and Tuwunel, including every loss and its investigation, see Spindle vs the field.

62measurements
7comparisons

Commit b6fba412f686 on main · 2026-10-17T19:42:59Z · runner github-actions/ubuntu-24.04

Generated from Criterion output on every push to main. Ratios are the result; wall times are context. Absolute times do not survive a change of runner, and a shared CI runner is both slower and noisier than a workstation by an amount that varies run to run. Everything here is algorithmic, measured inside the library — none of it is a server throughput figure and none of it should be quoted as one.

State updates: our HAMT vs imbl

SizeSpindleComparisonRatio
100239 µs153 µs1.57x slower
10003.73 ms2.15 ms1.74x slower
1000053.1 ms47.4 ms1.12x slower
50000382 ms341 ms1.12x slower

Replacing existing state slots: our HAMT vs imbl

SizeSpindleComparisonRatio
100308 µs163 µs1.89x slower
10004.49 ms2.57 ms1.75x slower
1000063 ms37.7 ms1.67x slower
50000446 ms295 ms1.51x slower

State lookup: our HAMT vs HashMap

SizeSpindleComparisonRatio
1000199 ns24 ns8.32x slower
50000210 ns25.5 ns8.21x slower

Persisting one state change: content-addressed delta vs serialising an im map

SizeSpindleComparisonRatio
1003.97 µs3.24 µs1.23x slower
10005.52 µs19.1 µs3.47x
100006.78 µs183 µs27.06x

State at a past point, inside the resident window: snapshot vs a 100-hop delta replay

SizeSpindleComparisonRatio
10016.5 ns12.8 µs778.24x
100023 ns75.3 µs3272.71x
1000024.3 ns123 µs5056.64x

State at a past point, outside the window: rehydrate from the node store vs a 100-hop delta replay

SizeSpindleComparisonRatio
10026.4 µs12.8 µs2.06x slower
1000280 µs75.3 µs3.71x slower
100003 ms123 µs24.39x slower

Every measurement

All benchmarks in this run
BenchmarkMeanMedian95% CI
delayed_firing_tick/1001.99 ms1.96 ms1.95 ms – 2.06 ms
delayed_firing_tick/10003.73 ms3.74 ms3.68 ms – 3.76 ms
delayed_firing_tick/1000021 ms20.9 ms20.7 ms – 21.3 ms
delayed_idle_tick/100613 ns610 ns610 ns – 617 ns
delayed_idle_tick/1000617 ns614 ns614 ns – 619 ns
delayed_idle_tick/10000672 ns668 ns669 ns – 677 ns
delayed_restart/101.47 µs1.46 µs1.47 µs – 1.48 µs
delayed_restart/1001.64 µs1.63 µs1.63 µs – 1.64 µs
delayed_restart/10001.97 µs1.93 µs1.95 µs – 1.99 µs
fork resolution/ruma-state-res/117.7 µs17.6 µs17.6 µs – 17.7 µs
fork resolution/ruma-state-res/16244 µs244 µs244 µs – 245 µs
fork resolution/ruma-state-res/2564.16 ms4.16 ms4.16 ms – 4.17 ms
fork resolution/ruma-state-res/459.8 µs59.6 µs59.6 µs – 60.1 µs
fork resolution/ruma-state-res/64959 µs959 µs958 µs – 961 µs
fork resolution/spindle conflict split/12.45 µs2.74 µs2.35 µs – 2.55 µs
fork resolution/spindle conflict split/1618.5 µs18.4 µs18.4 µs – 18.6 µs
fork resolution/spindle conflict split/256397 µs398 µs396 µs – 398 µs
fork resolution/spindle conflict split/44.68 µs3.74 µs4.44 µs – 4.94 µs
fork resolution/spindle conflict split/6481.1 µs82.4 µs80 µs – 81.9 µs
fork_window_two_event_fork/bounded/100396 ns392 ns393 ns – 401 ns
fork_window_two_event_fork/bounded/1000417 ns416 ns416 ns – 419 ns
fork_window_two_event_fork/bounded/10000435 ns434 ns434 ns – 437 ns
fork_window_two_event_fork/exhaustive/10022.9 µs22.9 µs22.9 µs – 22.9 µs
fork_window_two_event_fork/exhaustive/1000293 µs291 µs291 µs – 295 µs
fork_window_two_event_fork/exhaustive/100003.62 ms3.62 ms3.62 ms – 3.63 ms
state_history_at/delta_replay_100/10012.8 µs12.8 µs12.8 µs – 12.8 µs
state_history_at/delta_replay_100/100075.3 µs75.2 µs75.2 µs – 75.5 µs
state_history_at/delta_replay_100/10000123 µs122 µs123 µs – 123 µs
state_history_at/rehydrate/10026.4 µs26.3 µs26.3 µs – 26.5 µs
state_history_at/rehydrate/1000280 µs279 µs279 µs – 280 µs
state_history_at/rehydrate/100003 ms3 ms3 ms – 3 ms
state_history_at/resident/10016.5 ns16.4 ns16.4 ns – 16.5 ns
state_history_at/resident/100023 ns23 ns22.9 ns – 23.2 ns
state_history_at/resident/1000024.3 ns24.3 ns24.3 ns – 24.4 ns
state_lookup/hamt/1000199 ns198 ns199 ns – 200 ns
state_lookup/hamt/50000210 ns210 ns209 ns – 210 ns
state_lookup/hashmap/100024 ns23.9 ns23.9 ns – 24 ns
state_lookup/hashmap/5000025.5 ns25.2 ns25.4 ns – 25.7 ns
state_persist/hamt_delta/1003.97 µs3.97 µs3.97 µs – 3.99 µs
state_persist/hamt_delta/10005.52 µs5.51 µs5.51 µs – 5.52 µs
state_persist/hamt_delta/100006.78 µs6.77 µs6.77 µs – 6.79 µs
state_persist/im_serialise/1003.24 µs3.24 µs3.24 µs – 3.25 µs
state_persist/im_serialise/100019.1 µs19.1 µs19.1 µs – 19.2 µs
state_persist/im_serialise/10000183 µs183 µs183 µs – 184 µs
state_retained_replacements/hamt/100308 µs303 µs306 µs – 310 µs
state_retained_replacements/hamt/10004.49 ms4.39 ms4.45 ms – 4.53 ms
state_retained_replacements/hamt/1000063 ms62.4 ms62.5 ms – 63.5 ms
state_retained_replacements/hamt/50000446 ms444 ms444 ms – 448 ms
state_retained_replacements/imbl/100163 µs163 µs163 µs – 164 µs
state_retained_replacements/imbl/10002.57 ms2.56 ms2.56 ms – 2.58 ms
state_retained_replacements/imbl/1000037.7 ms37.7 ms37.5 ms – 37.9 ms
state_retained_replacements/imbl/50000295 ms294 ms294 ms – 296 ms
state_retained_updates/clone_per_event/100448 µs446 µs447 µs – 449 µs
state_retained_updates/clone_per_event/100051.7 ms51 ms50.3 ms – 53 ms
state_retained_updates/hamt/100239 µs239 µs239 µs – 240 µs
state_retained_updates/hamt/10003.73 ms3.74 ms3.73 ms – 3.74 ms
state_retained_updates/hamt/1000053.1 ms52.8 ms52.9 ms – 53.3 ms
state_retained_updates/hamt/50000382 ms382 ms380 ms – 384 ms
state_retained_updates/imbl/100153 µs152 µs152 µs – 153 µs
state_retained_updates/imbl/10002.15 ms2.15 ms2.15 ms – 2.16 ms
state_retained_updates/imbl/1000047.4 ms47.7 ms46.8 ms – 47.9 ms
state_retained_updates/imbl/50000341 ms341 ms340 ms – 342 ms

Raw data: latest.json. Method, caveats and what each comparison does and does not establish: docs/benchmarks.md.