Classification metrics versus trading value
Read accuracy, precision, recall and class imbalance without confusing a correct label with a profitable executable decision.
A metric needs a population, label definition, period and accounting convention. Report abstentions, missing labels, exclusions and unresolved outcomes. A convenient win rate without those objects is not an inspectable result.
Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.
A metric needs a population, label definition, period and accounting convention. Report abstentions, missing labels, exclusions and unresolved outcomes. A convenient win rate without those objects is not an inspectable result.
Request immutable model identity, training and label periods, feature availability, transformation fit scope, action and exit policies, complete trial registry and independent evaluation. A purchased signal record additionally needs its original instructions, amendments and execution assumptions. Ask what the documented tool does before comparing it with a proprietary model.
Declare the source, causal information clock, transformation, target, fitting rule, action, exit, sizing and costs. Choose the independent evaluation boundary before looking at its outcome. Preserve unsuccessful experiments and later amendments.
Documentation establishes the stated mechanism, not tested trading performance. Checked 9 October 2026; a later version can differ.
Read accuracy, precision, recall and class imbalance without confusing a correct label with a profitable executable decision.
Audit availability times, train-only transformations and forward targets before trusting a trading-model split.
Keep candidate selection, independent strategy validation, software checks and cloud production state as separate durable records.
For a classifier, identify labelled opportunities, attempted predictions, abstentions, unresolved labels and exclusions. State whether the table covers all eligible rows or only selected cases. The default compact probability example covers its entire authored population and makes every row downloadable; that completeness is fictional, not a genuine market claim. A real score on selected successful messages can conceal missed decisions. An empty group gets no invented success fraction. If labels concern cash profit, their own exit and cost definitions must be identified before calling positive labels wins.
Brier loss measures supplied probabilities against binary labels. Accuracy measures thresholded decisions, while same-table prevalence supplies a conditional reference. Each needs its own definition and scale. Adding financial claims requires quantities, weights and cash-flow accounting absent from this lens. A numerical summary is meaningful only inside its declared object. Record the metric's period, source version and unresolved outcomes next to the value; do not let a product's marketing name quietly substitute for them.
The complete default CSV makes the loss denominator inspectable through one hundred authored rows. Every row is marked fictional, and its binary label is not a profitable received trade. A real measurement needs the actual source and eligibility process as well as arithmetic. Preserve that difference in downloads, captions and summaries so numerical completeness cannot silently become a stronger performance assertion.