Abstract

Many times, meaningful information is derived from the image fusion. In applications like change detection, cancer growth detection, etc., there is a need of alignment of two or more images first. In the image registration process, two images are geometrically aligned, which is an important pre-processing step in the fields such as remote sensing, medical, etc. This paper focuses specifically on satellite images, which are commonly used in applications like change detection, weather forecasting, and growth monitoring. In these applications, image registration is a crucial step, and the accuracy of the registration process is essential. However, there are various challenges to image registration, one of them is illumination change in multi-sensor, multi-spectral satellite images. To address this challenge, paper proposes a feature based approach, where feature detection using speeded up robust feature (SURF) and descriptor from modified visual geometry group (VGG16) convolutional neural network (CNN) structure are used. The descriptors are generated from the initial convolutional layers of the modified VGG16 structure for each key point detected by SURF. The main goal of this approach is to reduce incorrect matches, which in turn improves image registration. Results of this experiment demonstrate 20% to 40% of significant improvement in correct match rate (CMR), which in turn improves image registration by the proposed approach, as compared to the method where only the original SURF is used for feature detection and descriptor generation. Therefore, it is found that, the use of CNN features as descriptor with SURF as feature detector provides improved results in terms of CMR and thus improves image registration compared to the taken method in comparison. This shows that the use of learned feature as descriptor has potential to improve the image registration.

Keyword

Image registration, Speeded up robust feature (SURF), Convolutional neural network (CNN), VGG16, Correct match rate (CMR).

Cite this article

Patel LK, Patel MI

Refference

[1][1]Brown LG. A survey of image registration techniques. ACM Computing Surveys. 1992; 24(4):325-76.

[2][2]Zitova B, Flusser J. Image registration methods: a survey. Image and vision computing. 2003; 21(11):977-1000.

[3][3]Patel MI, Thakar VK, Shah SK. Image registration of satellite images with varying illumination level using HOG descriptor based SURF. Procedia Computer Science. 2016; 93:382-8.

[4][4]Lowe DG. Distinctive image features from scale-invariant key points. International Journal of Computer Vision. 2004; 60:91-110.

[5][5]Bay H, Tuytelaars T, Van GL. Surf: speeded up robust features. Lecture Notes in Computer Science. 2006; 3951:404-17.

[6][6]Rublee E, Rabaud V, Konolige K, Bradski G. ORB: an efficient alternative to SIFT or SURF. In 2011 international conference on computer vision 2011 (pp. 2564-71). IEEE.

[7][7]El-gayar MM, Soliman H. A comparative study of image low level feature extraction algorithms. Egyptian Informatics Journal. 2013; 14(2):175-81.

[8][8]Joshi K, Patel MI. Recent advances in local feature detector and descriptor: a literature survey. International Journal of Multimedia Information Retrieval. 2020; 9(4):231-47.

[9][9]Sreeja G, Saraniya O. A comparative study on image registration techniques for SAR images. In 5th international conference on advanced computing & communication systems 2019 (pp. 947-53). IEEE.

[10][10]Mistry D, Banerjee A. Comparison of feature detection and matching approaches: SIFT and SURF. GRD Journals-Global Research and Development Journal for Engineering. 2017; 2(4):7-13.

[11][11]Kai W, Bo C, Lu M, Song X. Multi-source remote sensing image registration based on normalized SURF algorithm. In international conference on computer science and electronics engineering 2012 (pp. 373-7). IEEE.

[12][12]Huiqing Z, Lin G. Image registration research based on SUSAN-SURF algorithm. In the 26th Chinese control and decision conference 2014 (pp. 5292-6). IEEE.

[13][13]Zhao D, Yang Y, Ji Z, Hu X. Rapid multimodality registration based on MM-SURF. Neurocomputing. 2014; 131:87-97.

[14][14]Zhu XX, Tuia D, Mou L, Xia GS, Zhang L, Xu F. Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine. 2017; 5(4):8-36.

[15][15]Ma L, Liu Y, Zhang X, Ye Y, Yin G, Johnson BA. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing. 2019; 152:166-77.

[16][16]Kuppala K, Banda S, Barige TR. An overview of deep learning methods for image registration with focus on feature-based approaches. International Journal of Image and Data Fusion. 2020; 11(2):113-35.

[17][17]Liu X, Han F, Ghazali KH, Mohamed II, Zhao Y. A review of convolutional neural networks in remote sensing image. In proceedings of 8th international conference on software and computer applications 2019 (pp. 263-7).

[18][18]Ye F, Su Y, Xiao H, Zhao X, Min W. Remote sensing image registration using convolutional neural network features. IEEE Geoscience and Remote Sensing Letters. 2018; 15(2):232-6.

[19][19]Marmanis D, Datcu M, Esch T, Stilla U. Deep learning earth observation classification using imagenet pretrained networks. IEEE Geoscience and Remote Sensing Letters. 2015; 13(1):105-9.

[20][20]Zhou W, Newsam S, Li C, Shao Z. Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval. Remote Sensing. 2017; 9(5):1-20.

[21][21]Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Communications of the ACM. 2017; 60(6):84-90.

[22][22]Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ICLR 2014 (pp.1-14).

[23][23]Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In proceedings of the IEEE conference on computer vision and pattern recognition 2015 (pp. 1-9).

[24][24]Shah U, Mistry D, Banerjee A. Image registration of multiview satellite images using best features points detection and matching methods from SURF, SIFT and PCA-SIFT. Journal of Emerging Technologies and Innovative Research. 2014; 1(1):1-13.

[25][25]Bouchiha R, Besbes K. Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering. 2013; 5(1):406-10.

[26][26]Al-najjar H. UAV and lidar image registration: a SURF-based approach for ground control points selection. ISPRS Geospatial Week. 2019:413-18.

[27][27]Aarathi MR, Raju J. Influence of different descriptors to enhance image registration techniques using FREAK: case study. In 1st international conference on innovations in information and communication technology 2019 (pp. 1-6). IEEE.

[28][28]Sheng Z, Peihua L, Yuli L, Mingsi Q, Changgang J, Meng Z. Image registration method based on optimized SURF algorithm. American Journal of Optics and Photonics. 2019; 7(4):63-9.

[29][29]Liao F, Chen Y, Chen Y, Lu Y. SAR image registration based on optimized Ransac algorithm with mixed feature extraction. In IGARSS international geoscience and remote sensing symposium 2020 (pp. 1153-6). IEEE.

[30][30]Li X, He H, Huang C, Shi Y. PCB Image registration based on improved SURF algorithm. In international conference on image processing, computer vision and machine learning 2022 (pp. 76-9). IEEE.

[31][31]Wu Y, Wang Z. Remote sensing image registration algorithm based on improved surf in wavelet domain. Journal of Tianjin University Science and Technology. 2017; 50(10): 1084-92.

[32][32]Zhongjun W, Yanfeng C. Image registration algorithm using SURF feature and local cross-correlation information. Infrared and Laser Engineering. 2022; 51(6):1-6.

[33][33]Zhao X, Li H, Wang P, Jing L. An image registration method using deep residual network features for multisource high-resolution remote sensing images. Remote Sensing. 2021; 13(17):1-25.

[34][34]Yang Z, Dan T, Yang Y. Multi-temporal remote sensing image registration using deep convolutional features. IEEE Access. 2018; 6:38544-55.

[35][35]Ma W, Zhang J, Wu Y, Jiao L, Zhu H, Zhao W. A novel two-step registration method for remote sensing images based on deep and local features. IEEE Transactions on Geoscience and Remote Sensing. 2019; 57(7):4834-43.

[36][36]Li C, You Y, Cao J, Zhou W. MoatNet: registration for multi-temporal optical remote sensing images using deep convolutional features. In international geoscience and remote sensing symposium IGARSS 2021 (pp. 2154-7). IEEE.

[37][37]Ying C, Guoqing L, Hengshi C. Multi-temporal remote sensing image registration based on multi-layer feature fusion of deep residual network. In international conference on intelligent informatics and biomedical sciences 2019 (pp. 363-7). IEEE.

[38][38]Zeng L, Du Y, Lin H, Wang J, Yin J, Yang J. A novel region-based image registration method for multisource remote sensing images via CNN. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2020; 14:1821-31.

[39][39]Zou B, Li H, Zhang L. Self-supervised SAR image registration with SAR-superpoint and transformation aggregation. IEEE Transactions on Geoscience and Remote Sensing. 2022; 61:1-15.

[40][40]Ibañez D, Fernandez-beltran R, Pla F. FloU-Net: an optical flow network for multimodal self-supervised image registration. IEEE Geoscience and Remote Sensing Letters. 2023; 20:1-5.

[41][41]Li L, Han L, Ding M, Liu Z, Cao H. Remote sensing image registration based on deep learning regression model. IEEE Geoscience and Remote Sensing Letters. 2020; 19:1-5.

[42][42]https://docs.opencv.org/4.x/dc/dc3/tutorial_py_matcher.html. Accessed 24 August 2022.

[43][43]Zuliani M. Computational methods for automatic image registration. University of California, Santa Barbara; 2007:1-24.

[44][44] http://bhuvan.nrsc.gov.in. Accessed 20 August 2022.

[45][45]Gong M, Zhao S, Jiao L, Tian D, Wang S. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Transactions on Geoscience and Remote Sensing. 2013; 52(7):4328-38.

[46][46]http://www.landinfo.com/GalSatResComp.htm, Accessed 20 August 2022.