Deep High Dynamic Range Imaging with Large Foreground Motions
Authors: Shangzhe Wu (Wu Shangzhe), Jiarui Xu (Xu Jiarui), Yu-Wing Tai (Dai Yurong), Chi-Keung Tang (Deng Zhiqiang)
Abstract:
High dynamic range (HDR) imaging has been a subject of interest in recent years, particularly in the context of capturing dynamic scenes with large foreground motions. Traditional HDR imaging methods rely on optical flow alignment to generate high-quality images, but these methods often suffer from defects such as ghosting and distortion. In this paper, we propose a novel non-optical flow HDR imaging method based on deep learning, which can overcome a wide range of prospects for movement in a dynamic scene.
Introduction:
Currently, the best HDR image forming methods, such as Kalantari’s approach in 2017, typically involve the first optical flow of the input image alignment, followed by resynthesis HDR image. However, due to the presence of large occlusion and moving the input image, these methods often generate images with many defects. In contrast, we avoid the light flow directly to the HDR image as a problem of image conversion, which provides an important inspiration for HDR imaging.
Contribution:
Existing digital cameras generally are not sufficient to record the entire dynamic range of the scene, and some for taking HDR images are often too expensive special equipment or heavy. Therefore, merge multiple shots at different exposure low dynamic range (LDR) is another image more realistic approach. However, in the presence of large-scale displacement and prospects mild camera displacement, dislocation prospects and background it is inevitable. Our proposed method can learn how to merge multiple low dynamic map into a high dynamic map no ghosting, even if there is a case where the prospect of a large range of displacement.
Network Architecture:
We use the Internet to learn LDR translation to map multiple HDR image. Our network architecture is based on a symmetrical variation ResNet encoder decoder architecture, which is a commonly used tool for mapping study. The network structure can be divided into three parts: an encoder, combined, and a decoder. The encoder is responsible for extracting features from the input LDR images, the combiner learns how to merge these features, and the decoder generates HDR images.
Qualitative and Quantitative Comparison:
We compared our proposed method with existing methods, such as Kalantari’s approach, and found that our method can generate high-quality HDR images with reduced distortion and color defects. Our method also outperforms existing methods in terms of robustness, as it can handle images with large foreground motions and missing details. We also tested our model on mobile phones and found that it can generate high-quality HDR images in real-time.
Results:
Our proposed method can generate high-quality HDR images with reduced distortion and color defects, even in the presence of large foreground motions and missing details. We also found that our method can handle images with different exposure levels and can generate HDR images in real-time on mobile phones.
Conclusion:
In this paper, we proposed a novel non-optical flow HDR imaging method based on deep learning, which can overcome a wide range of prospects for movement in a dynamic scene. Our method can generate high-quality HDR images with reduced distortion and color defects, even in the presence of large foreground motions and missing details. We also tested our model on mobile phones and found that it can generate high-quality HDR images in real-time.
Network Architecture:
The network architecture is based on a symmetrical variation ResNet encoder decoder architecture, which is a commonly used tool for mapping study.
- Encoder: The encoder is responsible for extracting features from the input LDR images.
- Combiner: The combiner learns how to merge the features extracted by the encoder.
- Decoder: The decoder generates HDR images from the merged features.
Qualitative Comparison:
No displacement of objects in the region, all methods are generated good results, but when the displacement of the larger object overexposed areas occur, other methods will be obvious geometric distortion defects or color. In contrast, we propose two networks have generated very good results, ResNet structural performance is usually better than Unet structure.
Quantitative Comparison:
We compared our proposed method with existing methods, such as Kalantari’s approach, and found that our method can generate high-quality HDR images with reduced distortion and color defects. Our method also outperforms existing methods in terms of robustness, as it can handle images with large foreground motions and missing details. We also tested our model on mobile phones and found that it can generate high-quality HDR images in real-time.
Results:
Our proposed method can generate high-quality HDR images with reduced distortion and color defects, even in the presence of large foreground motions and missing details. We also found that our method can handle images with different exposure levels and can generate HDR images in real-time on mobile phones.
Conclusion:
In this paper, we proposed a novel non-optical flow HDR imaging method based on deep learning, which can overcome a wide range of prospects for movement in a dynamic scene. Our method can generate high-quality HDR images with reduced distortion and color defects, even in the presence of large foreground motions and missing details. We also tested our model on mobile phones and found that it can generate high-quality HDR images in real-time.