SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection

The Problem: Overconfident Hallucinations

Real-time object detectors routinely produce high-confidence predictions for objects they've never seen during training. A model trained on cars and pedestrians might confidently "detect" a bicycle as a car with 95% confidence—a failure mode called an out-of-distribution (OoD) hallucination. Existing defenses either bolt scoring functions onto detector outputs or retrain the detector itself, but they miss something crucial: object detectors already contain latent knowledge about what distinguishes in-distribution objects from unknown ones. Nobody has explicitly extracted and decoded that hidden knowledge.

The Approach: Eliciting Structured Priors

SPK treats the pretrained detector as a source of interpretable structure rather than a black box. The key insight is to use in-distribution training data and hallucination-inducing samples as diagnostic signals to reverse-engineer what the detector has implicitly learned.

The method elicits part-level semantic concepts—think of these as learned proto-features corresponding to object parts (wheel, window, headlight). Using these diagnostic labels, SPK identifies which semantic parts the detector activates when making predictions. Rather than just accepting or rejecting outputs, the framework composes these discovered semantic priors with two other types of knowledge: geometric priors (spatial relationships, size constraints) and contextual priors (object co-occurrence patterns).

These three prior types are integrated into a compact five-dimensional SPK representation that encodes OoD-relevant information. This compact space then becomes the basis for OoD detection—predictions with unusual prior combinations get flagged as suspicious.

Why It Matters

The results are striking: SPK achieves state-of-the-art OoD detection across multiple object detector architectures (YOLO, Faster R-CNN, etc.) and diverse OoD benchmarks. More importantly, the framework is transparent. Because the five-dimensional representation is built from explicit semantic, geometric, and contextual priors, practitioners can inspect why a prediction is flagged—which parts triggered activation, which spatial patterns looked wrong, which context was violated.

The deeper finding challenges conventional wisdom: pretrained detectors encode far richer latent knowledge about object categories than current OoD methods exploit. Rather than adapting or retraining detectors (expensive and risky), the authors show you can unlock existing knowledge by asking the right diagnostic questions of in-distribution and hallucination data.

This opens a practical route for improving detector reliability in safety-critical applications (autonomous vehicles, robotics) without retraining, and with interpretability baked in. The code and data are released, making the approach reproducible and adoptable.