MP-HSIR:
A Multi-Prompt Framework for Universal Hyperspectral Image Restoration

Zhehui Wu1, Yong Chen2, Naoto Yokoya3,4, Wei He1
LIESMARS, Wuhan University, Wuhan, China1
School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China2
Graduate School of Frontier Sciences, The University of Tokyo, Tokyo, Japan3
RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan4
arXiv Code Dataset Model
Teaser Image

Overview of HSI characteristics and prompt-based all-in-one restoration methods. (a) Differences between HSIs and RGB images. (b) Impacts of different degradations on spectral features. (c) Visual prompt-based models. (d) Textual prompt-based models. (e) Dual-modality prompt-based models. (f) The proposed model integrating spectral, textual, and visual prompts.

Abstract

Hyperspectral images (HSIs) often suffer from diverse and unknown degradations during imaging, leading to severe spectral and spatial distortions. Existing HSI restoration methods typically rely on specific degradation assumptions, limiting their effectiveness in complex scenarios. In this paper, we propose MP-HSIR, a novel multi-prompt framework that effectively integrates spectral, textual, and visual prompts to achieve universal HSI restoration across diverse degradation types and intensities. Specifically, we develop a prompt-guided spatial-spectral transformer, which incorporates spatial self-attention and a prompt-guided dual-branch spectral self-attention. Since degradations affect spectral features differently, we introduce spectral prompts in the local spectral branch to provide universal low-rank spectral patterns as prior knowledge for enhancing spectral reconstruction. Furthermore, the text-visual synergistic prompt fuses high-level semantic representations with fine-grained visual features to encode degradation information, thereby guiding the restoration process. Extensive experiments on 9 HSI restoration tasks, including all-in-one scenarios, generalization tests, and real-world cases, demonstrate that MP-HSIR not only consistently outperforms existing all-in-one methods but also surpasses state-of-the-art task-specific approaches across multiple tasks.

Teaser Image

(a) The architecture of the proposed MP-HSIR. (b) Prompt-Guided Spatial-Spectral Transformer Block (PGSSTB). (c) Design of the Prompt-Guided Spectral Self-Attention (PGSSA) Module. (d) Text-Visual Synergistic Prompt (TVSP) Module.

Teaser Image

Quantitative comparison of all-in-one and state-of-the-art task-specific methods on 7 HSI restoration tasks. The best and second-best performances are highlighted in red and blue, respectively.

BibTeX

@misc{wu2025mphsirmultipromptframeworkuniversal,
      title={MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration}, 
      author={Zhehui Wu and Yong Chen and Naoto Yokoya and Wei He},
      year={2025},
      eprint={2503.09131},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.09131}, 
}