Skip to model research
Systematic Trading ModelsSpecification / Evaluation / Release

Classification metrics versus trading value

Read accuracy, precision, recall and class imbalance without confusing a correct label with a profitable executable decision.

Original editorial revision: 9 October 2026. Expanded edition: 14 October 2026.

Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.

A count has meaning only after the label

Begin with the exact observation and positive target. A positive label might mean a later close above an earlier close; it need not mean a profitable long trade after costs. Define the decision clock, target horizon, handling of equal values and missing labels. Record whether the classifier can abstain and whether abstentions are inside the displayed denominator. A confusion matrix cannot repair an ambiguous label. It also does not identify how many times a model was revised before the selected result appeared. Preserve those decisions before opening an evaluation rather than explaining them after reading its scores.

Work through the full invented matrix

Use true positives 18, false positives 12, false negatives 8 and true negatives 62. The population contains 100 observations and the actual negative class contains 74 of them. Accuracy is 80%, but an always-negative prediction already achieves 74%. Positive precision is 18 divided by 30, or 60%. Positive recall is 18 divided by 26, approximately 69.2308%; negative specificity is 62 divided by 74, approximately 83.7838%. The example is invented arithmetic. It does not establish a useful gain over an operational baseline or a statistical significance claim.

Undefined is an informative result

If no positive predictions exist, positive precision has a zero denominator and should remain undefined. If the evaluation contains no actual positive observations, recall is undefined. In this worksheet, balanced accuracy is undefined when either actual class is absent. Filling those cells with zero or one changes the interpretation and can make an incomplete population look decisive. Record which class is missing and why. An empty or selected population is not rescued by a colourful chart. Weighted or multiclass metrics need their own declared convention and are outside this binary worksheet's narrow contract.

Threshold selection belongs to development

A probabilistic classifier becomes a binary decision only after choosing a threshold or action policy. Selecting that threshold on the evaluation counts consumes the evaluation. Save the threshold, training period and permitted calibration procedure before opening the reserved sample. If costs change the desired action, specify that transformation within the development design rather than trying many thresholds until a pleasing test score appears. Probability calibration is also a separate question: a model can rank outcomes usefully while assigning poor probabilities. None of the four count totals reveals calibration or a confidence interval by itself.

A correct direction can still lose money

An assumed price move of one unit does not pay the same as an incorrect move of ten units. Timing, spread, fees, order rejection, exposure and stop behaviour can further separate label accuracy from monetary value. Define the executable action and native exit separately, then measure its result under the frozen contract. Do not multiply the accuracy by an imagined reward to manufacture expectancy. Classification diagnostics can help reject or understand a candidate, but a trading result needs complete outcome accounting, including unfilled actions and the costs actually required by the proposed implementation.

Report the modest thing you measured

Publish the label definition, source period, four counts, denominator rules, threshold version and baseline. State whether observations overlap and whether the sample was used in model selection. Use the worksheet to check arithmetic and keep its JSON export attached to those assumptions. Software fixtures for count formulas are not fresh trading observations. An independent strategy evaluation must come from its own declared evidence. The honest headline is that a specified classifier produced specified labelled counts on a specified population, not that an impressive percentage proves the best trading model or a future subscriber return.

Exercise the absent-prediction case

Enter zero true positives, zero false positives, eight false negatives and ninety-two true negatives. Accuracy remains calculable, but positive precision is undefined because no positive prediction was made. Keep that distinction in an export rather than filling the missing metric with a convenient number. Then explain what the model never attempted to identify. The exercise exposes a reporting boundary; it does not estimate a confidence interval, recommend a threshold or demonstrate that always predicting one class is economically sensible.

Source documents

Documentation establishes the stated mechanism, not tested trading performance. Checked 9 October 2026; a later version can differ.

Continue the investigation

Binary prediction count audit

Probability error remains separate from payout

Compare the default group packet with its overconfident case. Their binary labels are the same while forecast loss changes. Changing the action threshold then changes decision counts without changing forecast loss. Neither exercise adds payouts, orders or capital. In a financial study, the positive class must have an explicit causal target and economic policy; it cannot be assumed synonymous with a profitable received trade. Keep label-score calculations and account accounting in separate artifacts before investigating their relationship. The site supplies no actual trained forecast or independent performance sample through these comparisons.