Skip to model research
Systematic Trading ModelsSpecification / Evaluation / Release

Binary prediction count audit

Inspect a fixed binary confusion matrix, its class imbalance and undefined denominators. This is not a backtest.

Original editorial revision: 9 October 2026. Expanded edition: 14 October 2026.

Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.

Declared population and labels

The input counts must come from the same frozen population and the same positive-label definition. With TP 18, FP 12, FN 8 and TN 62 there are 100 observations: 26 actual positives and 74 actual negatives. Accuracy is 80%; precision is 60%; recall is approximately 69.2308%; specificity is approximately 83.7838%. The majority-class baseline is 74%.

Formula and undefined cases

Precision = TP / (TP + FP). Recall = TP / (TP + FN). Specificity = TN / (TN + FP). Accuracy = (TP + TN) / total. Balanced accuracy is the mean of recall and specificity. A zero denominator returns Undefined, not a fabricated zero or perfect score. Balanced accuracy is undefined when either actual class is absent. All counts must be non-negative integers and the total must be positive.

Not an economic result

The worksheet does not model payoffs, orders, abstentions outside the entered population, fees, exposure, class calibration or independent sampling. A minority class can be economically important without being well predicted. Record threshold selection and the source period before treating these counts as an evaluation. Export keeps the assumptions and limitations with the calculation.

Invented labelled population: keep all four counts
Invented TP 18, FP 12, FN 8 and TN 62. The negative actual class totals 74. A count display is not a return distribution or validated model result. Underlying assumptions

Source documents

Documentation establishes the stated mechanism, not tested trading performance. Checked 9 October 2026; a later version can differ.

Threshold counts alongside supplied probabilities

The probability-group lens reuses the original confusion-count arithmetic rather than creating another definition of precision or balanced accuracy. Its default three groups produce TP36, FP24, FN4 and TN36 at an inclusive 0.5 threshold. These are different authored counts from this worksheet's original 18/12/8/62. Keep both populations identifiable. Brier loss uses supplied probabilities before thresholding and therefore remains unchanged when a threshold moves. The count worksheet cannot recover probabilities from four aggregate counts. A shared arithmetic helper does not imply the two fictional populations came from one genuine trained model.

A complete population needs more than four cells

Four cells omit abstentions outside the labelled sample, label timing, repeated decisions, dependency and economic outcomes. In real research, preserve eligible rows and reasons for every exclusion. A threshold chosen after observing these cells has selection history. Do not relabel that same table an untouched confirmation sample. Precision and recall are undefined when their required denominator is zero; balancing needs both actual classes. These honest missing values matter more than making every exported metric look complete.

Model-card and probability references