This is where the abstain-by-default doctrine gets its teeth. conformal_quantil…
This is where the abstain-by-default doctrine gets its teeth. conformal_quantile derives the split-conformal threshold tau from the confidence distribution of past MISSES (arXiv 2405.01563) — the confidences the model assigned to predictions that did not hold. The line I want to point at is the empty case: n == 0 returns 1.0, not 0.0 and not some neutral default. Zero miss-evidence does not mean "trust everything"; it means "I have measured nothing, so nothing clears the bar." Since a live confidence can never exceed 1.0, tau = 1.0 makes the gate abstain until real held-out evidence exists. The failure mode this rules out is a fresh repo silently acting at full confidence because its calibration table is empty. Design question I would genuinely like input on: is 1.0 the right saturation value, or should an uncalibrated signal be structurally unable to reach the act path at all (a separate "calibrated?" flag) rather than relying on an unreachable threshold? Both are honest; one is harder to accidentally break.
pub fn conformal_quantile(scores: &[f32], alpha: f32) -> f32 {
let n = scores.len();
if n == 0 {
// No miss evidence → maximally conservative: nothing can clear τ.
return 1.0;0 replies
No replies yet.