Skip to model research
Systematic Trading ModelsSpecification / Evaluation / Release

Measurability: define the population

A metric needs a population, label definition, period and accounting convention. Report abstentions, missing labels, exclusions and unresolved outcomes. A convenient win rate without those objects is not an inspectable result.

Original editorial revision: 9 October 2026. Expanded edition: 14 October 2026.

Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.

Specification

A metric needs a population, label definition, period and accounting convention. Report abstentions, missing labels, exclusions and unresolved outcomes. A convenient win rate without those objects is not an inspectable result.

Audit request

Request immutable model identity, training and label periods, feature availability, transformation fit scope, action and exit policies, complete trial registry and independent evaluation. A purchased signal record additionally needs its original instructions, amendments and execution assumptions. Ask what the documented tool does before comparing it with a proprietary model.

Boundary check

Declare the source, causal information clock, transformation, target, fitting rule, action, exit, sizing and costs. Choose the independent evaluation boundary before looking at its outcome. Preserve unsuccessful experiments and later amendments.

Source documents

Documentation establishes the stated mechanism, not tested trading performance. Checked 9 October 2026; a later version can differ.

Detailed research

Name the denominator before the percentage

For a classifier, identify labelled opportunities, attempted predictions, abstentions, unresolved labels and exclusions. State whether the table covers all eligible rows or only selected cases. The default compact probability example covers its entire authored population and makes every row downloadable; that completeness is fictional, not a genuine market claim. A real score on selected successful messages can conceal missed decisions. An empty group gets no invented success fraction. If labels concern cash profit, their own exit and cost definitions must be identified before calling positive labels wins.

Metrics with different objects stay different

Brier loss measures supplied probabilities against binary labels. Accuracy measures thresholded decisions, while same-table prevalence supplies a conditional reference. Each needs its own definition and scale. Adding financial claims requires quantities, weights and cash-flow accounting absent from this lens. A numerical summary is meaningful only inside its declared object. Record the metric's period, source version and unresolved outcomes next to the value; do not let a product's marketing name quietly substitute for them.

A measurable teaching population is not a market sample

The complete default CSV makes the loss denominator inspectable through one hundred authored rows. Every row is marked fictional, and its binary label is not a profitable received trade. A real measurement needs the actual source and eligibility process as well as arithmetic. Preserve that difference in downloads, captions and summaries so numerical completeness cannot silently become a stronger performance assertion.