New Method Improves Medical VLM Coverage for Rare Diseases

Medical Vision-Language Models (VLMs) can maintain high overall prediction coverage even when clinical data shifts occur, but they often severely under-cover specific disease classes, particularly rare ones. This undercoverage is problematic because the affected classes can vary unpredictably with different data acquisition protocols or model architectures. Standard conformal methods, which adapt to local test neighborhoods or source-frequency tails, fail to adequately address these class-wise coverage failures. To tackle this, researchers developed Class-Tail Adaptive Localized Conformal Deferral (CALCoDe), a post-hoc reliability layer designed for frozen medical VLMs. CALCoDe first identifies classes at risk of undercoverage using cross-fitted validation predictions. It then estimates class-conditional tail thresholds from a separate calibration split. These protected thresholds are combined with localized conformal thresholds, ensuring that all labels admitted by the localized rule are included, with additional protection for identified high-risk classes. An independently calibrated support audit further defers cases lacking sufficient inlier support. CALCoDe guarantees finite-sample coverage at a prespecified guard level for accepted examples within protected classes. Empirical evaluations across two dermatology dataset shifts and four VLM backbones showed that CALCoDe was the only approach to achieve 0.95 marginal and worst-class accepted coverage in all eight settings, significantly outperforming other conformal baselines.