One of the things I enjoy about running validators is getting to help the people building the next generation of clients. This week, we put our spare server to work testing Mithril, the Solana client from the Overclock team, including 7Layer and Dubbelosix.
The goal was practical: capture real network data, replay it reproducibly, and give the team measurements they could use.
We started with Mithril PR #352. There is a lot to appreciate about a team making this work available for operators to test. Efficient client software is an exciting direction for Solana. We wanted to explore it on hardware we already had and contribute something concrete in return.
What we actually tested
Our spare has an AMD EPYC 9375F with 32 physical cores, 64 hardware threads, roughly 499 GiB of RAM visible to Linux, and four 2 TB Sabrent Rocket 5 NVMe drives. It is a capable server, and these results do not establish a minimum hardware requirement.
For the controlled comparison, we restricted Mithril to eight physical cores across two CPU cache groups and set both GOMAXPROCS and transaction parallelism to eight. That limited its CPU allocation, but we did not impose a tight memory limit.
We ran Mithril as an unstaked, non-voting node, separate from production identities and data. The build used Go 1.27.3 on Linux/amd64 and pinned PR #352 to commit 9e6a4d9aa3a8c28e86ce8b6e50d7fdc7c834ca2b.
The first live session caught up and then spent two hours near the network tip. That was useful operational evidence, but our first reproducible replay corpus was smaller than intended: 605 blocks. We wanted a broader sample, so we went back and improved the capture.
The capture work was part of the experiment
A live repair spool and a benchmark archive have different jobs. We found that the spool had an 8 GiB limit and discarded newer slots first when full. During historical catch-up, it also removed consumed shreds behind the replay frontier. Simply leaving the client running longer would not produce the dataset we needed.
For our local benchmark build, we increased the bounded spool to 64 GiB, retained 8,192 slots of consumed history, and kept a consistent rewind window. We also used loopback-only diagnostic listeners. These are test-specific modifications, not recommendations for production deployment.
That gave us a historical corpus of 5,320 blocks containing 1,947,923 transactions. We exported it, imported it again, compared the capture metadata, and ran an initial correctness replay before starting the controlled comparisons.
What the longer comparison showed
We compared the Go and assembly execution-loop modes in the same binary, on the same corpus, with the same CPU placement. After two excluded warmup runs, we ran two Go/ASM/ASM/Go sequences. Each run replayed the full 5,320 blocks; the timing comparison used a fixed 5,192-block range after excluding the first 128 blocks.
All ten runs completed successfully and passed the replay bank-hash checks.
| Measurement | Go | Assembly |
|---|---|---|
| Median of each run’s mean SlotReplay time | 49.180 ms | 47.905 ms |
| Median full replay time | 297.254 s | 290.491 s |
On this workload, assembly reduced the SlotReplay measure by about 2.6% and full replay time by about 2.3%. The benefit was modest and consistent across the controlled comparisons.
We collected resource telemetry throughout the batch. We also paused the spare’s other soak workload during those runs so that the two execution modes were not competing with a changing workload. The soak was restored afterward and verified healthy and caught up.
How to read these numbers
This is a same-build execution-mode comparison. We did not compare PR #352 against its parent or run a head-to-head test against Firedancer or Agave. The replay used a historical slice of mainnet activity and warm filesystem caches. A different workload or machine may behave differently.
The bank hashes were recorded by our capture and checked during replay. They provide reproducibility evidence, not independent proof of chain authenticity. The offline backtest’s default did not check shred signatures.
Our spare’s ECC counters stayed at zero, but its separate hardware soak was intentionally interrupted for clean measurements. This was not an uninterrupted hardware endurance test.
What comes next
The expanded fresh-snapshot, near-tip experiment is still in progress. We will report that separately rather than use the historical replay as a substitute. Our comment on PR #352 shares the current findings and their limitations with the team.
There are other questions worth exploring, including CPU placement and how much parallelism actually helps. Our small pilot suggested that fewer cores could sometimes be faster, but topology and parallelism changed together. That needs a better-controlled experiment before drawing a conclusion.
For now, I want to give the Overclock team credit for building a client that operators can run, measure, and help improve. Getting real network data into a reproducible test gives us a useful way to contribute beyond simply watching development from the sidelines.
Thanks to 7Layer, Dubbelosix, and everyone working on Mithril. We are glad to lend a spare machine and some operator time to the effort.
