De-FuSE―Disentangled Frequency and Semantic Embeddings for Unified Vision Anomaly Detection

초록

Vision anomaly detection (VAD) is an open research problem due to underlying challenges pertaining to cross-functional deployment and anomaly diversity. Among the key solutions in literature, unified anomaly detection (UAD) aims to learn a single generalized model for heterogeneous anomalies. However, achieving this generalizability is constrained by limited data availability, particularly due to the stochastic nature of anomalies. Frozen foundation models like DINOv2 overcome this data constraint by providing a strong semantic backbone for self-supervised training. But their integration into reconstruction-based UAD is often hindered by identity shortcuts where powerful semantic priors can trivially reconstruct the input regardless of anomalies, thus failing to produce the high reconstruction error necessary for detection. We argue that decoupling the invariant semantic signal from variant high-frequency residuals prevents the decoder from trivially mapping anomalous regions to the normal manifold. Therefore, the proposed De-FuSE architecture disentangles semantic representations from high-frequency residuals as a functional separation. Our method introduces a semantic-guided cross-frequency attention (CFA) mechanism in the hybrid decoder, which acts as a conditional gate to inject frequency priors with semantically aligned queries. We further enforce spectral fidelity through a composite loss function, combining hard-mining strategies with focal frequency loss. Extensive experimentation on the MPDD, VisA, and MVTec-AD benchmarks demonstrates the superiority of our approach for VAD. In the challenging unified setting, De-FuSE achieves competitive accuracy with an efficient design. With just 59.1M trainable parameters, De-FuSE delivers a massive 31.75% reduction in model complexity compared to baseline unified frameworks. The code is available here: https://github.com/mariam-162/De-FuSE.

키워드