Citation: Wang, L.; Sole, A.;
Hardeberg, J.Y. Densely Residual
Network with Dual Attention for
Hyperspectral Reconstruction from
RGB Images. Remote Sens. 2022, 14,
3128. https://doi.org/
10.3390/rs14133128
Academic Editors: Antonio J. Plaza
and Pedro Latorre-Carmona
Received: 5 March 2022
Accepted: 10 June 2022
Published: 29 June 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
remote sensing
Article
Densely Residual Network with Dual Attention
for Hyperspectral Reconstruction from RGB Images
Lixia Wang
1,2
, Aditya Sole
2
and Jon Yngve Hardeberg
2,
*
1
Xiaomi, Nanjing 210019, China; lixiawang@whu.edu.cn
2
Department of Computer Science, Faculty of Information Technology and Electrical Engineering,
Norwegian University of Science and Technology, N-2815 Gjøvik, Norway; aditya.sole@ntnu.no
* Correspondence: jon.hardeberg@ntnu.no
Abstract: In the last several years, deep learning has been introduced to recover a hyperspectral
image (HSI) from a single RGB image and demonstrated good performance. In particular, attention
mechanisms have further strengthened discriminative features, but most of them are learned by
convolutions with limited receptive fields or require much computational cost, which hinders the
function of attention modules. Furthermore, the performance of these deep learning methods is
hampered by tackling multi-level features equally. To this end, in this paper, based on multiple
lightweight densely residual modules, we propose a densely residual network with dual attention
(DRN-DA), which utilizes advanced attention and adaptive fusion strategy for more efficient feature
correlation learning and more powerful feature extraction. Specifically, an SE layer is applied to
learn channel-wise dependencies, and dual downsampling spatial attention (DDSA) is developed
to capture long-range spatial contextual information. All the intermediate-layer feature maps are
adaptively fused. Experimental results on four data sets from the NTIRE 2018 and NTIRE 2020
Spectral Reconstruction Challenges demonstrate the superiority of the proposed DRN-DA over
state-of-the-art methods (at least −6.19% and −1.43% on NTIRE 2018 “Clean” and “Real World”
track, −6.85% and −5.30% on NTIRE 2020 “Clean” and “Real World” track) in terms of mean relative
absolute error.
Keywords: attention mechanism; densely residual network with dual attention (DRN-DA); receptive
field; channel attention; spatial attention
1. Introduction
With the increasing applications of computer vision technology in various engineering
fields [1–11], hyperspectral images (HSIs) have proved to obtain more helpful information
than RGB images. Hyperspectral images contain the reflectance of objects or scenes in
different spectral bands, usually ranging from several dozens to hundreds, even outside the
visible spectrum (e.g., in the ultraviolet or infrared spectrum). Compared with traditional
RGB images with increased spectral range and resolution, HSIs provide much richer infor-
mation, which has been widely used in cultural heritage [1,2], medical diagnosis [3], remote
sensing [4], food quality inspection [5], color quality control [6], and various computer vision
tasks, such as face recognition, object tracking, and image classification [7–9].
Due to the growing need for HSIs, various hyperspectral imaging systems (HISs)
have been developed in the last several decades. The first HISs, such as NASA’s airborne
visible/infrared imaging spectrometer (AVIRIS) [12], employed a prism to disperse the
reflected light and a linear array detector to record the reflected light. This kind of HISs
can acquire images with high spatial/spectral resolution in “whisk broom” imaging mode,
but image acquisition is time-consuming since they adopt the point-scanning method.
Afterwards, “push broom” HISs, such as NASA’s advanced land imager (ALI) [13], and
“staring” HISs, often used in microscopy or other lab applications, have been developed.
Remote Sens. 2022, 14, 3128. https://doi.org/10.3390/rs14133128 https://www.mdpi.com/journal/remotesensing