Classification metrics versus trading value
Read accuracy, precision, recall and class imbalance without confusing a correct label with a profitable executable decision.
Document decision inputs, timing and intervention authority so systematic and discretionary records can be compared without hindsight.
Research design and arithmetic, not model recommendations. Classification scores are not monetary returns; a registered artifact and a passing software test are not independent strategy validation.
Rules-based means a declared process maps available inputs to an action; it does not imply that the process is profitable or fully automated. Discretionary means authorised judgment can influence the action; it does not imply that the judgment must be undocumented. A mixed workflow should identify where each kind of decision occurs instead of describing every favourable intervention as part of an unchanged system.
Name who owns entry, quantity, exit, pause and override decisions. A chart indicator can be rules-based while the trade selection remains discretionary. A model's output can be deterministic while a human decides which account receives it. Describe each boundary separately. The reader should be able to reconstruct the decision using its actual information set, or identify the intervention that cannot be mechanically reproduced. Neither label grants an automatic claim of discipline, objectivity or superior results.
A rule requires input versions, receipt clocks and a missing-data policy. A discretionary decision also needs a dated rationale and the observations that informed it. An explanation written after the outcome may be useful reflection, but it cannot establish what was known earlier. Keep original notes and later commentary as different fields, with amendments visible.
Suppose an invented decision occurs at 09:30 using an earlier completed-bar signal, while a news item arrives at 09:32. A note recorded at 09:35 cannot make the 09:32 item part of the original 09:30 decision. If it changes an open instruction at 09:35, record that as a later intervention. The timestamps are a fictional clock fixture, not a claimed real trade. Event time, publication time and receipt time still need evidence in an actual study.
Store the original model instruction and the final authorised action independently. A discretionary override may decline a trade, reduce quantity, change an exit or suspend the system. Preserve the reason and authority for that override, together with its effect on outstanding orders. A cancellation request does not establish that a fill could no longer occur. Reconcile later broker reports against both the proposal and the accepted action.
Keep abstentions and rejected proposals in the population. Comparing only accepted winners conceals the selection introduced by a human gate. A shadow proposal can support a separately labelled counterfactual research calculation when its fill and accounting assumptions are explicit; it is not an actual account outcome. Do not assign the shadow strategy the prices obtained by a different real action without a justified convention. Proposal, hypothetical path and execution path need their own identifiers.
A new rule, altered threshold or changed intervention mandate creates a new policy version. Save effective time, previous version and authorisation. A system may intentionally include a manual emergency gate, but its scope must be written before it is exercised. A profitable exception does not erase a rule breach; an adverse exception does not automatically prove the underlying model failed.
For entry research, preserve the authorised raw objective and frozen identity before evaluating a predeclared exit. Do not retrofit a discretionary rescue into an earlier systematic backtest or reopen exit selection after observing reserved outcomes. A post-event improvement can be researched as a new candidate with an appropriate information boundary. An experiment tracker can retain this lineage, but logging the final flattering version without its predecessors does not provide the missing history.
A fair comparison needs the same opportunity clock, input coverage and economic accounting. Show the proposal population, intervention population and actual execution population rather than collapsing them into one win fraction. Manual selection may change trade frequency, concentration and unresolved exposure. Automated rules may expose stale-input or operating-failure cases. Keep these differences visible without assuming one category is inherently safer.
Freeze the comparison question before choosing examples. A careful narrative can describe why a decision was made without proving it improved expected value. Conversely, reproducible code can repeatedly implement a poor rule. Use clearly labelled correctness fixtures to check timing and order-state behaviour, and reserved observations for any empirical performance claim. This guide has no trader comparison sample, interviews or paid-account results; it does not invent a superior discretionary or systematic return.
The assessment should identify model identity, intervention authority, information provenance, order lineage and unresolved states. Where a production system is discussed, distinguish research status from deployed revision, active route and durable cloud execution receipts. A local notebook labelled live is not evidence that a subscriber-facing service continues with the workstation off. Local tests remain useful without becoming production dependencies.
Use the blank decision record to capture the fictional clock example, then document actual interventions if lawful records become available. A bounded conclusion may be that rules are reproducible while discretionary lineage remains incomplete, or the reverse. Do not claim independent verification merely because software stored the notes. The useful distinction is what can be reconstructed and who had authority at each moment, not a slogan that automatically awards trust to either decision style.
| Object | Preserve | Do not infer |
|---|---|---|
| Information | Receipt cutoff and original rationale | Later commentary establishes earlier knowledge |
| Action | Proposal, override and execution lineage | A rejected proposal is an executed trade |
| Assessment | Versioned policy and compatible populations | A decision style proves superior performance |
Download blank review record (CSV)
Headers only. No trades, subscriber outcomes or operating receipts are supplied.
Documentation establishes the stated mechanism, not tested trading performance. Checked 10 October 2026; a later version can differ.
Read accuracy, precision, recall and class imbalance without confusing a correct label with a profitable executable decision.
Audit availability times, train-only transformations and forward targets before trusting a trading-model split.
Keep candidate selection, independent strategy validation, software checks and cloud production state as separate durable records.
A discretionary override should identify who changed the action, what was observable and how that intervention is evaluated. A systematic rule should state how unavailable inputs and ambiguous labels are handled. Both can be documented; neither receives validation solely from its style label. A rule selected through repeated hindsight has a selection history despite having no fitted weights. A classifier with frozen parameters can still serve a revised feature join. Use a model card to preserve these identities rather than allowing the same informal strategy name to cover incompatible decision processes.