Deep learning-based single image super-resolution: an Investigation for dense scene reconstruction with UAS photogrammetry

Simple item page
dc.contributor.authorPashaei, Mohammad
dc.contributor.authorStarek, Michael J.
dc.contributor.authorKamangir, Hamid
dc.contributor.authorBerryhill, Jacob
dc.creator.orcidhttps://orcid.org/0000-0002-1427-6265en_US
dc.creator.orcidhttps://orcid.org/0000-0002-7996-0594en_US
dc.creator.orcidhttps://orcid.org/0000-0001-9718-7518en_US
dc.creator.orcidhttps://orcid.org/0000-0002-1427-6265
dc.creator.orcidhttps://orcid.org/0000-0002-7996-0594
dc.creator.orcidhttps://orcid.org/0000-0001-9718-7518
dc.creator.orcidhttps://orcid.org/0000-0002-1427-6265
dc.creator.orcidhttps://orcid.org/0000-0002-7996-0594
dc.creator.orcidhttps://orcid.org/0000-0001-9718-7518https://orcid.org/0000-0002-1427-6265
dc.creator.orcidhttps://orcid.org/0000-0002-7996-0594
dc.creator.orcidhttps://orcid.org/0000-0001-9718-7518
dc.date.accessioned2021-10-26T20:08:51Z
dc.date.available2021-10-26T20:08:51Z
dc.date.issued2020-05-29
dc.description.abstractThe deep convolutional neural network (DCNN) has recently been applied to the highly challenging and ill-posed problem of single image super-resolution (SISR), which aims to predict high-resolution (HR) images from their corresponding low-resolution (LR) images. In many remote sensing (RS) applications, spatial resolution of the aerial or satellite imagery has a great impact on the accuracy and reliability of information extracted from the images. In this study, the potential of a DCNN-based SISR model, called enhanced super-resolution generative adversarial network (ESRGAN), to predict the spatial information degraded or lost in a hyper-spatial resolution unmanned aircraft system (UAS) RGB image set is investigated. ESRGAN model is trained over a limited number of original HR (50 out of 450 total images) and virtually-generated LR UAS images by downsampling the original HR images using a bicubic kernel with a factor ×4 . Quantitative and qualitative assessments of super-resolved images using standard image quality measures (IQMs) confirm that the DCNN-based SISR approach can be successfully applied on LR UAS imagery for spatial resolution enhancement. The performance of DCNN-based SISR approach for the UAS image set closely approximates performances reported on standard SISR image sets with mean peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index values of around 28 dB and 0.85 dB, respectively. Furthermore, by exploiting the rigorous Structure-from-Motion (SfM) photogrammetry procedure, an accurate task-based IQM for evaluating the quality of the super-resolved images is carried out. Results verify that the interior and exterior imaging geometry, which are extremely important for extracting highly accurate spatial information from UAS imagery in photogrammetric applications, can be accurately retrieved from a super-resolved image set. The number of corresponding keypoints and dense points generated from the SfM photogrammetry process are about 6 and 17 times more than those extracted from the corresponding LR image set, respectively.en_US
dc.description.abstractThe deep convolutional neural network (DCNN) has recently been applied to the highly challenging and ill-posed problem of single image super-resolution (SISR), which aims to predict high-resolution (HR) images from their corresponding low-resolution (LR) images. In many remote sensing (RS) applications, spatial resolution of the aerial or satellite imagery has a great impact on the accuracy and reliability of information extracted from the images. In this study, the potential of a DCNN-based SISR model, called enhanced super-resolution generative adversarial network (ESRGAN), to predict the spatial information degraded or lost in a hyper-spatial resolution unmanned aircraft system (UAS) RGB image set is investigated. ESRGAN model is trained over a limited number of original HR (50 out of 450 total images) and virtually-generated LR UAS images by downsampling the original HR images using a bicubic kernel with a factor ×4 . Quantitative and qualitative assessments of super-resolved images using standard image quality measures (IQMs) confirm that the DCNN-based SISR approach can be successfully applied on LR UAS imagery for spatial resolution enhancement. The performance of DCNN-based SISR approach for the UAS image set closely approximates performances reported on standard SISR image sets with mean peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index values of around 28 dB and 0.85 dB, respectively. Furthermore, by exploiting the rigorous Structure-from-Motion (SfM) photogrammetry procedure, an accurate task-based IQM for evaluating the quality of the super-resolved images is carried out. Results verify that the interior and exterior imaging geometry, which are extremely important for extracting highly accurate spatial information from UAS imagery in photogrammetric applications, can be accurately retrieved from a super-resolved image set. The number of corresponding keypoints and dense points generated from the SfM photogrammetry process are about 6 and 17 times more than those extracted from the corresponding LR image set, respectively.
dc.identifier.citationPashaei, M., Starek, M.J., Kamangir, H. and Berryhill, J., 2020. Deep learning-based single image super-resolution: An investigation for dense scene reconstruction with UAS photogrammetry. Remote Sensing, 12(11), p.1757.en_US
dc.identifier.citationPashaei, M., Starek, M.J., Kamangir, H. and Berryhill, J., 2020. Deep learning-based single image super-resolution: An investigation for dense scene reconstruction with UAS photogrammetry. Remote Sensing, 12(11), p.1757.
dc.identifier.doihttps://doi.org/10.3390/rs12111757
dc.identifier.urihttps://hdl.handle.net/1969.6/89862
dc.language.isoen_USen_US
dc.language.isoen_US
dc.publisherMDPIen_US
dc.publisherMDPI
dc.rightsAttribution 4.0 International*
dc.rightsAttribution 4.0 International
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/*
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.subjectunmanned aircraft system (uas)en_US
dc.subjectdeep-learningen_US
dc.subjectsuper-resolution (sr)en_US
dc.subjectconvolutional neural network (cnn)en_US
dc.subjectgenerative adversarial network (gan)en_US
dc.subjectstructure from motionen_US
dc.subjectphotogrammetryen_US
dc.subjectremote sensingen_US
dc.subjectunmanned aircraft system (uas)
dc.subjectdeep-learning
dc.subjectsuper-resolution (sr)
dc.subjectconvolutional neural network (cnn)
dc.subjectgenerative adversarial network (gan)
dc.subjectstructure from motion
dc.subjectphotogrammetry
dc.subjectremote sensing
dc.titleDeep learning-based single image super-resolution: an Investigation for dense scene reconstruction with UAS photogrammetryen_US
dc.titleDeep learning-based single image super-resolution: an Investigation for dense scene reconstruction with UAS photogrammetry
dc.typeArticleen_US
dc.typeArticle

Files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
Pashaei_Mohammad_RemoteSensing.pdf
Size:
25.34 MB
Format:
Adobe Portable Document Format
Description:
Article
Download

License bundle

Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.72 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

COE Faculty Works