Skip to model research
Systematic Trading ModelsSpecification / Evaluation / Release

Decision thresholds and equality policy

Keep forecast probabilities fixed while tracing threshold ownership, changed confusion counts and consumed selection evidence.

The comparison operator is part of the policy

A positive prediction occurs when the declared group probability is greater than or equal to the entered threshold. Equality is deliberately included. A different operator could produce different predictions at an exact boundary even with the same forecasts and labels. Save that operator with the version and threshold; a threshold number alone is incomplete. The lens supports values from zero through one and does not infer an abstention state. Real trading policies may need such a state, but it must be described rather than silently added to the binary calculation.

Probability scoring precedes thresholding

Brier loss uses each supplied probability and label before conversion into a class decision. Moving the threshold does not change those inputs and therefore does not change that loss. Confusion counts do change when a group crosses the decision boundary. True positives are selected positive labels, false positives selected negatives, false negatives unselected positives and true negatives unselected negatives. The lens reuses the original count worksheet's metric helper so denominator conventions stay consistent. Neither the count helper nor the threshold policy supplies payouts, costs or an economically optimal action.

Selection history does not disappear

Searching thresholds on an inspected evaluation table uses that table for selection. Reporting the best threshold on the same labels as untouched confirmation confuses two roles. A real experiment needs its predeclared selection process and reserved evaluation. Freeze the underlying model identity before validating the financial system. The teaching picker offers seven known fixtures to inspect behavior, not an answer-free threshold tournament. Trying each fixture many times does not create new independent observations. A later material threshold change should identify what evidence still applies and which new validation is required.

Undefined metrics remain honest

If no positive prediction is made, positive precision has no denominator. If no actual positive exists, recall has no denominator; absent actual negatives likewise makes specificity undefined. Balanced accuracy requires both actual classes under this worksheet's contract. A threshold yielding high accuracy can still omit a rare class or select no useful decisions. Preserve those missing metrics in JSON as null rather than a fabricated zero or one hundred percent. Their existence is a reporting boundary, not proof that a model was unsafe, profitable or unsuitable for every possible economic policy.

Fictional model evidence case

At threshold 0.5, the authored B and C groups are positive predictions: TP36, FP24, FN4 and TN36. Move the threshold to 0.8 and equality retains group C: TP16, FP4, FN24 and TN56. Precision rises from 60% to 80% while recall drops from 90% to 40%. Brier loss stays 0.168. This is a transparent count comparison on fixed fictional labels, not a recommendation to use 0.8 in live trading or a finding that the higher precision produces more money.

ObjectRecordBounded finding
Threshold 0.5B and C selected72% accuracy / 90% recall
Threshold 0.8C selected including equality72% accuracy / 40% recall
Forecasts unchangedSame probabilities and labelsBrier remains 0.168
Authored group probabilities, positive label fractions and an overconfident counterexample
Declared fictional probabilities and labels, not independently calibrated trading forecasts.

Primary references and scope

Source review: 14 October 2026. Official mechanism descriptions are not this desk's paid-account test, independently captured fills or vendor-return audit.

Related model questions