Skip to model research
Systematic Trading ModelsSpecification / Evaluation / Release

Repeatability: freeze the inputs

Repeatability requires source identity, settings, environment and data versions. Repeating a calculation on the same inspected sample checks reproducibility, not independent validation. Preserve the execution contract too.

Original editorial revision: 9 October 2026. Expanded edition: 14 October 2026.

Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.

Specification

Repeatability requires source identity, settings, environment and data versions. Repeating a calculation on the same inspected sample checks reproducibility, not independent validation. Preserve the execution contract too.

Audit request

Request immutable model identity, training and label periods, feature availability, transformation fit scope, action and exit policies, complete trial registry and independent evaluation. A purchased signal record additionally needs its original instructions, amendments and execution assumptions. Ask what the documented tool does before comparing it with a proprietary model.

Boundary check

Declare the source, causal information clock, transformation, target, fitting rule, action, exit, sizing and costs. Choose the independent evaluation boundary before looking at its outcome. Preserve unsuccessful experiments and later amendments.

Source documents

Documentation establishes the stated mechanism, not tested trading performance. Checked 9 October 2026; a later version can differ.

Detailed research

Reconstruct the same object, not a similarly named run

Freeze code, settings, data identity, transformation parameters and label ownership. Reproduce the count calculation from its full row population and retain the exact output contract. A deterministic repeat on the same input checks implementation stability; it does not create a second independent financial observation. If a source is revised or a feature is added, the prior result does not automatically belong to the new version. Keep parent and successor IDs visible even when the public model nickname stays familiar.

Inputs alone may not capture availability

The latest downloaded source can reconstruct a revised history while failing to reconstruct what was known at the earlier decision. Store effective and availability boundaries, source vintages and allowed correction policy. Fitted transforms need their actual parameters, not just the training date. Randomized procedures need declared seed and environment without claiming that a matching seed certifies strategy validity. A cloud artifact should be read back to identify the served version. The repeated website build checks publication bytes, not a proprietary model's research reproduction.

Preserve the permitted observation window

A real research fill reference must reuse its established observation-window and directional policy. Switching silently to first-available prices or extending the window would reproduce a different financial object, not the original experiment. A count-only fixture cannot test that policy because it contains no prices or native fills. Keep reproduction artifacts tied to the actual convention and identify any authorized successor separately.