ShadowFormer++: multi-scale shadow priors and diffusion-guided refinement for high-fidelity shadow removal

Abstract

Shadows in images cause significant challenges for computer vision applications as they degrade image quality and hinder tasks such as object detection, visual tracking, and semantic segmentation. Existing shadow removal methods often struggle as they fail to localize shadow regions accurately without suppressing non-shadow areas; they tend to lose fine-grained textures and illumination consistency across boundaries, and they also demand heavy computation that limits their use in real-time or resource constrained systems. To address these limitations, we propose a novel framework called ShadowFormer++, which uses a combination of a transformer and diffusion model for efficient shadow removal. Our architecture integrates Multi-Scale Local Shadow Perception Module (MS-LSPM) for robust local shadow feature extraction, Shadow-Aware Transformer Encoder (SATE) for global context modelling and structure preservation, and a Diffusion-Inspired Refinement Module (DIRM) for progressive, fine-grained shadow-free image reconstruction. Extensive experiments on ISTD, ISTD+, and SRD datasets show that ShadowFormer++ achieves a Mean Absolute Error (MAE) of 3.79, a Peak Signal-to-Noise Ratio (PSNR) of 33.58 dB, and a Structural Similarity Index (SSIM) of 0.972, which is better than state-of-the-art methods such as ShadowFormer, SpA-Former, and Diff-Shadow. Our approach balances computational efficiency with superior shadow removal quality, making it suitable for real-time applications in robotics, autonomous systems, and augmented reality.