TestU01 SmallCrush Benchmark

Pierre L'Ecuyer & Richard Simard's TestU01 academic PRNG benchmark suite — SmallCrush battery. Evaluates pseudo-random generators across 10 stringent statistical tests including birthday spacings, collision, gap, poker, coupon collector, weight distribution, matrix rank, Hamming independence, and run tests.

Academic Gold Standard: TestU01 by Pierre L’Ecuyer & Richard Simard (Universite de Montreal) is the reference benchmark suite for PRNG research in academic literature. SmallCrush is the fastest battery — 10 tests requiring approximately 50 million numbers — with BigCrush (the full suite) applying 106 tests.

TestU01 SmallCrush Battery (TU01–TU10)

SmallCrush applies a significance level of $p \in [0.001, 0.999]$ — a two-tailed rejection region stricter than NIST’s 0.01 — across 10 tests drawn from Knuth, Marsaglia, and L’Ecuyer’s statistical frameworks:

IDTest NameBatteryPass CriteriaDescription
TU01smarsa_BirthdaySpacingsSmallCrush$p \in [0.001, 0.999]$Collision rate in sparse birthday-problem distribution (planned).
TU02sknuth_CollisionSmallCrush$p \in [0.001, 0.999]$Frequency of hash collisions across uniformly mapped outputs.
TU03sknuth_GapSmallCrush$p \in [0.001, 0.999]$Lengths of gaps between successive values falling in a subinterval (planned).
TU04sknuth_SimpPokerSmallCrush$p \in [0.001, 0.999]$Poker-hand frequency distribution in groups of output values.
TU05sknuth_CouponCollectorSmallCrush$p \in [0.001, 0.999]$Draws required to collect all symbols in a set (planned).
TU06sknuth_MaxOftSmallCrush$p \in [0.001, 0.999]$Distribution of maximum over groups of values (planned).
TU07svar_WeightDistribSmallCrush$p \in [0.001, 0.999]$Weight (popcount) distribution across output blocks.
TU08smarsa_MatrixRankSmallCrush$p \in [0.001, 0.999]$Rank of binary matrices constructed from output bits (planned).
TU09sstring_HammingIndepSmallCrush$p \in [0.001, 0.999]$Independence of Hamming weights across pairs of output values (planned).
TU10sstring_RunSmallCrush$p \in [0.001, 0.999]$Run length distribution in bit-level output streams.

[!NOTE]
TU02, TU04, TU07, TU10 are fully implemented. TU01, TU03, TU05, TU06, TU08, TU09 are specification stubs shown as NOT IMPLEMENTED.

SmallCrush vs BigCrush

  • SmallCrush (this tool): 10 tests, ~50 million numbers, ~seconds to minutes. Catches obvious failures quickly.
  • BigCrush: 106 tests, ~100 billion numbers, ~hours. The definitive academic benchmark — nearly every known PRNG weakness is detected here.

When to Use TestU01

  1. Academic PRNG Research: The mandatory benchmark for any PRNG paper submitted to ACM TOMS, IEEE TPDS, or IACR ePrint. Reviewers expect SmallCrush and BigCrush results as part of standard evaluation evidence.
  2. LCG / MLCG Parameter Selection: Testing candidate multiplier/modulus pairs for Linear Congruential Generators and Multiplicative LCGs — TestU01’s Knuth-derived tests directly expose the lattice structure defects that afflict poorly-chosen LCG parameters.
  3. Simulation Framework Validation: Verifying generators used in discrete-event simulation (SimPy, SIMUL8), queuing theory, and network traffic modeling, where long-range correlations in sequential outputs cause systematic bias in performance estimates.
  4. Cryptography Education: Demonstrating PRNG weaknesses to security students — TestU01’s named tests map directly to textbook attacks on weak generators, making it ideal for hands-on lectures on randomness and unpredictability.

L’Ecuyer’s Hierarchy: Pierre L’Ecuyer maintains a ranked list of generator quality based on TestU01 results. As of 2024: WELL generators and xoshiro256** pass BigCrush; Mersenne Twister (MT19937) fails at ~1 TB in PractRand but passes BigCrush; classic rand() and LCG(2^32) fail SmallCrush immediately.

Number Volume: SmallCrush requires approximately 50 million 32-bit integers (~200 MB of raw data). Drop a binary file of at least 200 MB for complete SmallCrush evaluation. Smaller files will return INSUFFICIENT DATA for several tests.