GPU Monte Carlo Simulation Validation: Accuracy and Speed

You cannot validate a GPU Monte Carlo engine by checking that it reproduces the CPU engine bit for bit. GPU fused multiply-add, reordered parallel operations and higher CPU precision all change results between the two engines. GPU Monte Carlo simulation validation works differently: run both engines over many independent seeds, compare the distributions of their estimates with a formal statistical test, and accept the GPU engine when the difference falls inside a tolerance agreed beforehand with the people who own the risk. Accuracy, reproducibility and throughput each need their own evidence, and passing one test tells you nothing about the other two.

TechnoLynx ports, profiles and validates CUDA Monte Carlo engines through its GPU Performance Engineering work. TechnoLynx starts by pinning down which outputs, tolerances and deadlines the ported engine must meet.

Why cuRAND paths do not match QuantLib paths

On the CPU side, QuantLib’s best-known uniform generators are the Mersenne Twister (pseudo-random) and Sobol (low-discrepancy). The cuRAND library in the CUDA toolkit offers nine generator types, including XORWOW, MRG32k3a, MTGP32, Philox and MT19937. XORWOW, a member of the xorshift family, is the default. NVIDIA’s cuRAND documentation says its MT19937 uses the same parameters as the CPU version, but the ordering of the output is different. For that generator, the faster CURAND_ORDERING_PSEUDO_BEST ordering may also give different output on different GPU models. The dynamic ordering mode is deterministic within a run, but NVIDIA does not guarantee that it stays the same across GPUs or cuRAND releases.

LynxBenchAI publishes results showing how each device runs the public PyTorch stack, which helps you shortlist GPUs before porting, though it does not measure cuRAND or a hand-written Monte Carlo kernel. The first step is to check its results for your candidate GPUs, then run the multi-seed comparison below on the ones you pick.

One GPU generator design splits one long sequence into independent subsequences, one per thread block, each with its own state in shared memory. A serial CPU implementation runs path after path in a single thread. Comparing individual paths between cuRAND and QuantLib measures the difference between two generators, not the correctness of the port.

Should we seed-match paths or compare percentile distributions over many seeds?

For engine-level equivalence, compare distributions over many seeds. Wallerberger and Gull (2018) point out that tests built on a fixed seed break as soon as the sequence of updates changes, so they produce false negatives: tests that fail even though the results are correct. NIST SP 800-22 sets the same principle for generators themselves, stating that testing a PRNG on the output of a single seed is inadequate. That rule was written for randomness testing, but it carries straight over to engine comparison. In practice, a GPU study of quasi-Monte Carlo Greeks (arXiv 2209.11337, 2022) ran 500 independent runs for every method it compared.

How many seeds is enough depends on the tolerance you want to defend. Monte Carlo error shrinks as O(1/√N) in the number of samples. If the interval is as wide as the tolerance, a pass means very little.

The comparison itself should be a hypothesis test, not someone eyeballing a chart. Where a closed-form answer exists, such as Black-Scholes for European options, use it as the reference for both engines. Where none exists, a two-sample test compares the two stochastic results directly. Wallerberger and Gull find a significance level of about 0.01 a good trade-off for a single test. For a suite of K tests, they suggest dividing it by K to hold the false-alarm rate constant.

There is one exception. If you need path-level identity, for example to replay a specific trade’s paths when debugging, give both engines the same random inputs. The QMC Greeks study fed its CPU implementation the very normals it had generated for the GPU. cuRAND’s host API produces the same sequence on the CPU as on the device. Generating once and injecting the numbers into the CPU path construction takes the generator out of the comparison.

What you need to show	Method	What a pass proves
The GPU engine prices and measures risk like the CPU engine	Distributional comparison over many independent seeds, with a hypothesis test at a fixed significance level	Differences are not statistically significant at the chosen significance level
A specific path or trade reproduces exactly	Shared random inputs (generate once, feed both engines) or the same low-discrepancy sequence on both	Remaining differences come from arithmetic, not from the RNG
The engine is close to the true price	Both engines against a closed-form benchmark such as Black-Scholes	Each engine’s error against the analytic value, measured separately

Identical random inputs still give different arithmetic

Sharing inputs does not buy bitwise equality either. Fused multiply-add computes x×y+z with one rounding instead of two. NVIDIA’s floating-point whitepaper notes that GPUs used FMA by default when x86 CPUs had no hardware FMA, and that both results are correct under IEEE 754. A parallel reduction that changes threads per block also changes the order in which numbers are added. The same paper contrasts a GPU running 128 threads per block with a CPU using 4 threads in total. A CPU using x87 can carry extended precision you did not ask for. Common maths functions are not required to be correctly rounded, so they should not be expected to match between platforms. NVIDIA’s advice is to compare results carefully, because even the placement of parentheses or the thread count changes the final answer.

The GPU result is sometimes the more accurate one. Pairwise summation through a reduction tree reduces round-off compared with sequential summation. In NVIDIA’s Monte Carlo option-pricing whitepaper, growing the shared-memory reduction array from 128 to 1,024 elements cut the relative difference between single-precision GPU and double-precision CPU results from 7.6e-7 to 6e-8. To prove equivalence for each kernel after a port, see our guide on validating ported CUDA kernels against a tolerance.

Seeding and variance reduction that keep the comparison fair

On the GPU side, follow cuRAND’s own seeding recommendation. Give each experiment a unique seed and each thread a unique sequence number. When an experiment spans several kernel launches, keep the seed and increase the sequence numbers monotonically. Seeding every thread separately with sequence number 0 sets up faster but gives weaker guarantees, since a bad interaction between the seed hash and the generator’s period could correlate some threads. When the same model runs at several parameter values, give each value its own stream, seeded at the start, so that every estimate is a reproducible function of its parameters and seeds.

Variance reduction lowers the variance of each estimate, which tightens the comparison. Common random numbers reduce the variance of a difference whenever the two outputs are positively correlated. Antithetic variates are cheap to add on a GPU: the complement of each normal drives a second path. Apply the same technique, configured the same way, on both engines or on neither. Low-discrepancy sequences need extra care: cuRAND’s Sobol output depends on how dimensions are assigned to timesteps, and for quasirandom generators many short calls lay out memory differently from one large call. The trade-offs between speed and validity get a fuller treatment in variance reduction for faster, still-valid GPU Monte Carlo pricing.

Does mixed precision move our risk and collateral numbers beyond tolerance?

To find out, run the legacy double-precision CPU engine and the mixed-precision GPU engine on the same scenario set, measure the deviation on every regulatory output, and judge it against a tolerance that risk and audit agreed before the first run. If the deviation is inside tolerance on every metric, cut over. If it is not, keep higher precision where it matters until it is.

TechnoLynx handles this within GPU Performance Engineering. TechnoLynx identifies which RNG and accumulation steps move to lower precision and which risk outputs depend on them, runs the mixed-precision kernels beside the double-precision baseline on the target hardware, and documents a pass or fail for each metric.

In the published study below, precision was not the largest error, but you have to check, not assume. A 2019 arXiv study, “Tensor Processing Units for Financial Monte Carlo”, found that temporal discretisation bias dominates and that bias from numerical precision was negligible by comparison. On VaR and CVaR, the same study measured a bias under 1% relative to a double-precision simulation. That hardware was a TPU, not a GPU, so treat the result as a sign of scale and not as your number. For a large book, a deviation under 1% can still be material.

NVIDIA’s option-pricing whitepaper shows when precision starts to show. Against Black-Scholes, single-precision GPU and double-precision CPU both had a relative error of 1.1e-5 at 65k samples. At 16M samples the GPU error stopped at 1.9e-7, while the CPU’s fell to 2.9e-8. In that table the two errors match at low sample counts and separate only at the highest counts. A mixed-precision test at a modest path count can pass and then fail at production path counts. Test at the path counts you will run.

The tolerance is not a universal constant. The PRA’s SS1/23 expects a bank’s board to set a model risk appetite that includes thresholds for acceptable model performance and tolerance for errors. Setting that metric for a GPU risk engine is covered in defining parity in GPU-parallelised Monte Carlo risk simulations. If deviations exceed it, one lever is to widen the reduction tree before giving up on the GPU. The TPU study names multilevel Monte Carlo as a way to correct precision bias while keeping most of the speed-up.

Should you keep a double-precision CPU shadow run during the transition? We think it is a reasonable control for a defined period, not a permanent requirement. A shadow run catches drift as the portfolio changes, at the price of paying for both engines while it runs.

Is this pricing deviation sampling noise or a kernel bug?

Before sign-off, separate two error budgets. Characterise the sampling-noise band from repeated runs on each GPU, then treat any cross-hardware or cross-precision deviation outside that band as a candidate bug to investigate. The settlement-price replay tolerance then becomes the noise envelope plus a margin that risk and compliance sign off on.

TechnoLynx characterises noise per GPU and per precision mode, keeps a separate deterministic test suite for logic errors, and produces a reproducibility report with the noise bounds and the pass or fail tolerance written down.

Statistical tests catch what visual inspection misses. In Wallerberger and Gull’s example, a result that looked consistent with the exact answer was rejected by Hotelling’s T² test at p ≈ 0.0026. The reverse also holds: a deviation inside the band does not prove there is no bug, only that the bug cannot be told apart from noise at that sample size. Low-precision formats add a further risk of systematic bias. Check for systematic bias separately from the noise-band test.

Can anyone guarantee bitwise reproducibility across GPU generations and precision modes? No. cuRAND’s faster orderings can differ between GPU models, and reduction order follows the launch configuration. Reproducible summation does exist. Ahrens, Demmel and Nguyen (2016) give an algorithm whose sum is independent of summation order, at a cost of 7n additions with a 6-word accumulator. It solves summation, not generator ordering or maths-library rounding, so the honest promise across hardware is statistical equivalence within documented bounds.

Profile the CUDA kernels before adding GPUs

Whether a full Monte Carlo coverage sweep fits overnight depends on two numbers: the measured throughput of one GPU on your kernels, and where that kernel is bottlenecked. If the kernel is inefficient, optimise it first. If it already runs near hardware limits, add GPUs.

TechnoLynx does this through GPU Performance Engineering. TechnoLynx sizes the sweep against the time budget, profiles the CUDA kernels for occupancy and memory bandwidth, then optimises, configures multi-GPU scaling, or both. TechnoLynx then reruns the correctness protocol so the speed-up does not cost accuracy.

We profile before we optimise, because advice such as “avoid branches” only makes sense in the context profiling supplies. Roofline analysis in Nsight Compute plots a kernel’s arithmetic intensity against FLOP/s to show whether it is memory-bound or compute-bound. NVIDIA’s own worked example shows why guessing fails: a change meant to improve the kernel slowed it by 10%, from 1.74 s to 1.92 s. Memory traffic can bottleneck GPU Monte Carlo. The QMC Greeks study notes that global memory reads are roughly 100 times slower than shared memory. In that study, storing the Brownian-bridge path in global memory cut speed-ups of over 500x and 900x down to around 200x. Problem shape matters too. NVIDIA’s pricing sample lost throughput at small path counts until it switched implementation based on the ratio of paths to options, after which it mostly stayed above 10 billion paths per second. Our guide to diagnosing GPU bottlenecks with kernel-level profiling covers the profiling workflow in detail.

Worked sizing logic

The arithmetic is simple once throughput is measured, not assumed. GPU-hours needed = total trials in the sweep ÷ measured trials per GPU-hour. GPUs needed = GPU-hours ÷ hours in the overnight window, rounded up, plus headroom for aggregation. The assumption to watch is linear scaling. Measure scaling on the target cluster rather than assuming it is linear.

For shortlisting hardware, LynxBenchAI’s published results show how each device runs the public PyTorch stack. They are broadly indicative for custom PyTorch models, but they do not run or measure a hand-written CUDA Monte Carlo kernel, so your own profile still decides the count. For risk workloads specifically, STAC-A2 is an industry benchmark built on Monte Carlo estimation of Heston-based Greeks, and its audits cover performance, scaling, quality and resource efficiency.

What evidence will the risk committee need to approve replacing the CPU engine?

The committee needs a reproducible package. It should contain kernel-level correctness tests, the distributional comparison against the CPU engine, measured throughput and multi-GPU scaling on the target hardware, and the methodology, data, seeds and acceptance thresholds an auditor needs to rerun every claim. If only internal engineering acceptance is needed, kernel tests and performance benchmarks may be enough.

TechnoLynx assembles the engineering evidence as part of GPU Performance Engineering: kernel tests, reproducible validation runs and a documented package aligned with the firm’s existing model-validation standards. TechnoLynx does not give regulatory sign-off. That decision stays with the firm’s validation function and its regulator.

PRA SS1/23 expects performance tests to include comparisons with challenger models, meaning alternative implementations of the same theory. A legacy CPU engine fits that role naturally. SS1/23 also asks for ongoing, independent challenge to model development, and for clear guidelines on the maximum tolerance for performance exceptions. In the US, the OCC’s updated guidance replaces the 2011 model risk guidance with a risk-based approach tailored to each organisation’s model risk profile.

Sign-off package checklist

Kernel unit and regression tests for each operation, with expected values.
Distributional comparison: number of seeds, statistics compared, the test used, significance level and correction for multiple tests.
The tolerance per metric, set within the board’s model risk appetite.
Seeds, generator type and cuRAND ordering mode, since some orderings differ across GPU models.
CUDA, driver and library versions, plus the GPU model and topology.
Throughput, latency and multi-GPU scaling measured on production-like hardware.
Names of the validators and the validation date, recorded in the model inventory.
Who implemented the kernels and who validated them, with any overlap disclosed.

Can a GPU repricing run be guaranteed to finish inside a live-requote SLA?

Only worst-case timing, measured under realistic contention, supports an SLA claim. An average-case benchmark on an idle GPU does not. Meeting a hard SLA means designing the scheme (path count, variance reduction) together with how the GPU is shared and scheduled.

TechnoLynx would frame the SLA against worst-case timing for your scheme, bound completion time through scheduling and variance-reduction choices, and validate latency under negotiation-load contention. That work sits in the Inference Cost-Cut Pack.

In published timings, the kernel itself is fast. The QMC Greeks study timed a kernel at about 0.2 ms for 2^15 paths on a Tesla T4, against roughly 150 ms for its basic CPU version. Measure tail latency beyond the kernel, including data movement and queuing behind other jobs. Cutting paths or leaning harder on variance reduction shortens the run, but it can bias the price, so the shortcut has to pass the same accuracy protocol. Any guarantee is only as strong as the load scenarios it was tested under.

Who can validate and speed up our GPU Monte Carlo engine?

TechnoLynx ports CUDA Monte Carlo engines, validates them against the legacy CPU engine and makes them fast enough to meet the deadline. For validation, TechnoLynx designs the multi-seed comparison, the precision tests and the noise-versus-bug protocol, and documents seeds, versions and tolerances so the evidence can be rerun. For performance, TechnoLynx profiles the kernels and chooses between optimisation and scale-out. Both are part of GPU Performance Engineering, which starts with a scoping step that fixes the outputs, tolerances and time budget.

The engagement does not suit everyone. It does not replace your independent model validation function or regulatory approval, and the tolerance remains a risk decision your firm owns. A team that already has CUDA profiling and statistical-testing skills in house can run this protocol itself, using the checklist above. Firms whose engine comes from a vendor library should first ask that vendor for validation evidence.

Validation does not end at cutover

Noise bounds, precision effects and throughput profiles all depend on the workload and on the version. A different GPU generation, a new cuRAND release or a changed thread count can each move results that passed last quarter. A protocol you can rerun on every change is worth more than one excellent report, because a regulator will eventually ask whether the engine still matches, not whether it once did.