Before a trading model goes live, conformance testing has to show that it interacts with the venue’s matching logic and processes its data flows correctly. After a client-caused production issue, Euronext asks for proof that the production version passed, and RTS 6 requires stress tests at twice recent peak volumes. That is the minimum. If the model feeds a go/no-go threshold, such as an execution-cost cutoff, we would want the evidence to show three more things: how often the threshold wrongly passes in calm and volatile regimes, how many threshold variants were tried, and where the stress scenarios stop being reliable.

Exchange conformance testing evidence: what venues require and when

Exchange-based conformance testing usually follows a script of tests written by the exchange. The tests run in an environment the exchange provides, which imitates production trading. Under RTS 6 Article 7, as published in the FCA Handbook, conformance testing has to show that the algorithm interacts with the venue’s matching logic as intended and processes the venue’s data flows properly.

Euronext requires a validated conformance test before deploying a new client trading application, algorithm or strategy, and before any substantial update to one. It also requires one for a new Euronext trading application or a substantial Euronext update that needs client development. The FIA’s July 2024 best-practice paper recommends that exchanges require conformance testing for every new exchange-facing interface and again after material changes. The CFTC Technology Advisory Committee’s 2012 reference document says the trading firm carries the burden of deciding when a change in its own system logic means it must recertify. For an ML-driven model, that means the firm needs its own written definition of “material change” (does a retrain on new data count? does a threshold move?) before the question comes up.

Euronext issues a certificate that records the conformance test results. Eurex asks for a certificate for each Algo ID, naming the participant that wants the algorithm registered. CME Group requires every client system that routes orders through iLink or processes its market data to be certified through AutoCert+. If a production problem traces back to a client application, Euronext will ask for proof that the current software version passed conformance, and it can block that application’s access until it does.

Stress testing is a separate RTS 6 obligation. RTS 6 Article 10 requires an investment firm, as part of its annual self-assessment, to test that its algorithmic trading systems and controls can withstand increased order flows or market stress. Those tests must run in an environment separate from production. They must also include message volumes and trade volumes at twice the highest levels the firm reached in the previous six months. Venues such as Cboe provide test environments that simulate disorderly conditions. Cboe’s MiFID II test symbols, for example, run load profiles from one order per second up to bursts of one thousand, and include symbols that stay halted all day or switch between halted and unhalted. FINRA’s 2026 market access report describes, as effective practice, testing whether control parameters would block erroneous orders under different scenarios and checking that hard blocks actually trigger.

Stress-testing go/no-go thresholds beyond a single backtest window

We would trust a go/no-go threshold on an execution-cost prediction model only when two things are true. It would need stress testing across a designed range of volatility regimes and exchange edge cases, with false-pass and false-fail rates measured at each candidate threshold rather than read off one historical window.

LynxBenchAI runs the model at each candidate threshold across a designed scenario suite and measures false-pass and false-fail rates for each regime, and the first step is to define what “pass” means for conformance and for each stress scenario.

TechnoLynx builds a repeatable test harness that runs the model at each candidate threshold across the scenario suite and measures false-pass and false-fail rates for each regime.

The first step is to define what “pass” means for conformance and for each stress scenario. After that, the work covers four things: extending the scenario suite across volatility regimes and venue edge cases, running the model at several candidate thresholds in a test harness, comparing the measured error rates with the backtest-only figures, and delivering a go/no-go recommendation together with the harness for later re-validation. TechnoLynx treats this as release-readiness work, separate from model accuracy. A model can score well and still fail under load or on rare conditions that unit tests never reach.

Why a strong backtest window is weak evidence on its own

The Federal Reserve’s SR 11-7 guidance on model risk asks for testing under a variety of market conditions, including scenarios outside the range of ordinary expectations. It says stress testing helps establish the boundaries of acceptable inputs and the conditions under which a model becomes unstable. A single backtest window contains only the conditions that happened to occur in it.

Selection bias is a further problem. Harvey and Liu (2015) show that a single-test p-value greatly overstates significance when the strategy reported is the best of many tried. With ten tests, a result with a single-test p-value of 0.05 has an adjusted p-value of about 0.40. Bailey and López de Prado make the related point that a fixed cutoff for accepting candidates lets the false positive rate keep rising as more candidates are considered. They also argue that a backtest which does not control for the number of trials behind it is worthless, however good it looks.

We treat thresholds the same way. Every cutoff tried against the same window is a trial, so the evidence pack should record how many were tried and how they were chosen. A threshold tuned to maximise performance on one sample is overfitted to that sample. It fits the noise in the window rather than the behaviour you will see in production.

How many volatility conditions do we need before trusting the thresholds?

RTS 6 leaves the design of stress tests to the firm, based on the nature of its trading activity and systems. The Federal Reserve’s SR 12-7 stress-testing guidance asks for “enough scenarios to explore the range of potential outcomes”. So the answer is coverage, argued on its merits, not a count.

The supervisory evidence shows what thin coverage looks like. The FCA’s multi-firm review of algorithmic trading controls found that firms stressed algorithms in different ways: some used theoretical scenarios, some used historical stress periods, and some combined the two. It also found that some firms’ simulation testing lacked sophistication or did not consider a wide range of market scenarios. The FCA noted as good practice that some firms deliberately chose historical periods with higher stress and quickly added new stress events to their test data when they happened. SR 12-7 also warns that relationships seen in benign periods may not hold under severe stress, and that linear relationships can become nonlinear. It asks for stressful periods of extended duration as well as instantaneous shocks.

We would define regimes relative to a volatility trend rather than by absolute level. That finding comes from crisis prediction, not trade execution, but it supports measuring “calm” and “stressed” relative to the recent baseline. We would build the regime set for an execution-cost threshold from:

• prolonged below-trend volatility, where spreads are tight and the model may grow complacent
• above-trend volatility sustained over days, not a single spike
• intraday bursts at or beyond the regulatory volume floor of twice the six-month peak
• halts and halt toggling, plus post-trade amendments and cancellations of the kind venues simulate
• non-historical scenarios and a reverse stress test that starts from an unacceptable outcome and works back to the conditions that would cause it

Our rule for stopping is a judgement, not a published standard. Stop adding scenarios when a new one no longer moves the false-pass estimate outside its interval, and when the reverse stress test has found where the threshold breaks.

Measuring the false-pass rate at each candidate threshold

The false-pass rate is the share of cases the threshold approved where the realised execution cost turned out to be unacceptable. The false-fail rate is the share it rejected where the outcome would have been acceptable. These are the type I and type II errors of a binary decision, and they trade off against each other.

The 2016 Federal Reserve paper is a useful template for reporting. It evaluates type I and type II errors explicitly, as the trade-off between missed distress and false alarms. It uses the area under the ROC curve, which compares true-positive and false-positive rates across every possible threshold. It reports an AUROC of 77% with a 95% confidence interval of 73% to 80%, stated alongside the point estimate. A per-threshold false-pass table should carry intervals in the same way. Reporting a range rather than a point estimate avoids spurious precision.

What happens to false-pass if we tighten the threshold

The same paper describes the mechanism: as the threshold rises, fewer signals fire, so fewer real events are caught and fewer false alarms are raised. For an approval gate, a tighter cutoff approves fewer cases, which lowers the false-pass rate and raises the false-fail rate. Keep the current threshold if stress testing across the target regimes shows a false-pass rate within risk tolerance at a false-fail cost the desk can bear. Tighten it if false-pass under stress goes beyond tolerance, and accept the extra false fails. Loosen it only if the tighter setting causes heavy false fails without a meaningful cut in false-pass risk. Measure the rates for each regime separately. A pooled figure can look acceptable while the stressed regime alone is not.

A per-threshold validation run, step by step

1. Write down the pass definition: what counts as an acceptable execution-cost outcome, and what counts as passing for each conformance and stress scenario.
2. Fix the regime set and the candidate thresholds before you look at results, and log every variant you add later.
3. Run the model in a test environment separate from production, at volumes that include twice the six-month peak.
4. For each regime and threshold, compute false-pass and false-fail rates with confidence intervals.
5. Compare against the backtest-only figures and record the gap, along with the number of trials behind the chosen threshold.
6. Pick the threshold using the decision rule above and keep the harness so the run can be repeated after any material change.

These rates describe the scenarios and data you used, and nothing beyond them. They can shift when market microstructure or exchange rules change, and a designed set of scenarios will never cover every future combination. Conformance testing verifies that the basic elements of the system operate in accordance with the venue’s requirements. It says nothing about whether the threshold is economically well calibrated.

What the evidence pack must contain alongside the pass/fail thresholds

Euronext can ask for proof that the production version passed conformance, and for a documented algorithm-testing methodology and change-management policy on request. The firm’s own pack should then link each threshold to its risk appetite, so that the same file can largely serve internal audit as well.

TechnoLynx’s AI Governance and Trust covers assurance of this kind of evidence.

Euronext’s policy, citing Article 10 of RTS 7, requires members to certify that their algorithms were tested to avoid disorderly trading and to explain how that testing was done. The methodology document has to describe the tests, their scope, the occurrences and the tools used. The change-management policy has to cover how upgrades are made, what checks are run and by whom, the mitigation plan and how rollback works. Both are supplied to Euronext on request. Eurex sends participants an annual due diligence questionnaire that includes technical and functional conformance testing. FINRA’s 2026 report lists two documentation failures to avoid: not keeping records that show control thresholds are reasonable, and not documenting the annual effectiveness review, including an inventory of the systems, controls and thresholds that were reviewed.

Producing evidence that a compliance owner or exchange reviewer can read and sign against is engineering output, not policy writing. There is more on that discipline in approval-grade evidence for audit and procurement review.

Pack item	Who asks for it	Basis
Conformance certificate per version or Algo ID	Exchange	Euronext and Eurex require certificates
Testing methodology: tests, scope, occurrences, tools	Exchange	Euronext policy
Change-management and rollback policy	Exchange	Euronext policy
Stress results at twice six-month peak message and trade volumes	Regulator (RTS 6)	FCA Handbook text of RTS 6
Per-regime false-pass and false-fail table with intervals and trial count	Internal risk, model validation	Our recommendation, building on validation error thresholds
Threshold reasonableness rationale tied to business model	Regulator, internal audit	FINRA 2026 findings
Link from thresholds to risk appetite	Internal risk committee	Stress results inform limits consistent with risk appetite
Validation report with limitations and key assumptions	Internal audit, model risk	SR 11-7

Documenting stress scenario assumptions so reviewers don’t overtrust them

SR 12-7 is direct here: whatever kind of stress test is used, the firm should understand and clearly document all assumptions, uncertainties and limitations, and give that information to the people using the results. Results should come with qualitative context, such as key assumptions, so readers can interpret them, and the process should be documented well enough to be replicated. OSFI’s stress-testing guideline also asks for the reasoning behind the chosen scenarios and the sensitivity of results to their range and severity. Report ranges rather than point estimates to avoid spurious precision. Back any claim that an output is conservative with a stated definition and measurement of that conservatism. Where validation could not be carried out before use, document that fact and report it.

These texts are written for banking organisations, not for exchanges. Their documentation discipline still carries over directly to a threshold on a trading model. In practice, each scenario entry should state where it came from (a named historical window or a theoretical construction), how severe it is relative to trend, and what it does not model, such as queue position or contagion across venues.

Caveats and residual risks the pack must disclose

OSFI warns that a “failure of imagination” in scenario design can underestimate extreme events and create a false sense of security. SR 12-7 states that stress testing cannot provide absolute certainty. The pack should therefore list what it leaves open:

• regimes or venue behaviours that were not tested
• the trial count behind the chosen threshold
• any validation step limited by lack of data, which SR 11-7 says needs extra scrutiny and full disclosure to senior management
• the conditions that trigger re-validation, since SR 11-7 asks for monitoring wherever a model’s working range is approached or exceeded
• procured or outsourced components, because RTS 6 keeps the firm fully responsible for them

Can we reuse the same evidence package for internal audit and the exchange submission?

Mostly yes, as long as the pack is built to the stricter combined set of both audiences’ requirements rather than to the exchange checklist alone. RTS 6 requires compliance staff to understand in general terms how the firm’s algorithms work, and gives an outsourced compliance function the same access to information. OSFI expects internal audit to play a key role in assessing a stress-testing programme.

Map the exchange requirements and the internal risk-appetite and self-assessment criteria into one checklist. If internal audit’s needs are the exchange’s needs plus the risk-appetite rationale, reuse the pack unchanged. Otherwise, append the items only internal audit needs, such as governance sign-off records. Reuse breaks quietly when exchange rules change and drift away from internal risk policy, so review the mapping every time either side changes. Turning pipeline artefacts into such a pack is covered in building a defensible evidence pack for audit and procurement review.

Who can build the stress harness and evidence pack for our execution-cost thresholds?

Through AI engineering for finance, TechnoLynx builds the engineering behind a defensible go/no-go threshold: the scenario suite and harness that measure false-pass and false-fail rates for each regime, and the evidence pack that maps those results to venue requirements and your risk appetite. When choosing a partner, ask whether they will report error rates with intervals and trial counts, and whether they will hand over a harness you can re-run after a material change.

This work does not include legal advice, regulatory sign-off or certification. It produces the artefacts an audit needs, not a promise of the outcome. A firm with an established quant validation function and an existing scenario harness may only need an outside review. For the conformance script itself, Euronext recommends testing first on your own or with the help of one of its CTSG analysts.

FAQ

What is the stress test for banks?

SR 12-7 defines stress testing as a forward-looking assessment of how adverse events and circumstances could affect a banking organisation. OSFI describes it as evaluating the effect of exceptional but plausible changes in risk factors on an institution’s financial condition. Algorithm stress tests under RTS 6 are narrower: they check that trading systems and controls withstand higher order flow and market stress.

How do you tell if a model is overfitting?

Compare its performance on data it was not developed on with its performance in sample. SR 11-7 describes backtesting as comparing forecasts with actual outcomes over a period that was not used in development. For trading strategies, the number of variants tried has to be part of that judgement, because a single-test significance level overstates the result when the best of many is reported.