Abstract

This paper discusses anomaly detection, which is one of the most well-known applications of human activity recognition. Due to the ever-increasing activities posing risks ranging from planned aggression to harm caused by an accident, providing security to an individual is a major issue in any community today. Traditional closed-circuit television does not suffice since it necessitates a human being to be awake and always watch the cameras, which is costly. This necessitates the creation of an automated security system that detects anomalous activity in real-time and provides rapid assistance to victims. However, identifying activity from long surveillance footage takes time. Hence, in this research, we study the effect of the down-sampling concept of the challenging database, namely the university of central Florida (UCF Crime) using high efficiency video coding (HEVC)-H265 before feeding them into the anomaly detection system. This step reduced the size of the data, making it easier to store and transfer, and highlights the unique properties of each video clip. In the proposed work, first, we are down-sampling each video’s frame into half by using H265 on the fast forward moving picture experts group (FFMPEG) platform, and then spatiotemporal features are extracted from a series of frames (frame level) using a pre-trained convolutional neural network (CNN) called Resnet50, then to boost the feature we are combining the features of every 15 video frames to generate a new feature vector that will be fed into the classifier model. The values of the new feature vectors represent the summation of the values of the original feature vectors obtained from Resnet50. Finally, the features obtained from a series of frames are fed to the bidirectional long short-term memory (BiLSTM) model, to classify the video as normal or abnormal. We conducted comprehensive tests on a different benchmark dataset for anomaly detection to verify the proposed framework's functionality in complex surveillance scenarios. The numerical results were carried out on the UCF crime dataset, with the proposed approach achieving an area under curve (AUC) score of 90.16% on the database's test set.

Keyword

Anomaly detection, Video surveillance system, BiLSTM, Deep learning, CNN.

Cite this article

Abbas ZK, Al-Ani AA

Refference

[1][1]Ullah W, Ullah A, Haq IU, Muhammad K, Sajjad M, Baik SW. CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks. Multimedia Tools and Applications. 2021; 80(11):16979-95.

[2][2]Öztürk Hİ, Can AB. ADNet: temporal anomaly detection in surveillance videos. In international conference on pattern recognition 2021 (pp. 88-101). Springer, Cham.

[3][3]Ullah W, Ullah A, Hussain T, Khan ZA, Baik SW. An efficient anomaly recognition framework using an attention residual LSTM in surveillance videos. Sensors. 2021; 21(8):1-17.

[4][4]Morais R, Le V, Tran T, Saha B, Mansour M, Venkatesh S. Learning regularity in skeleton trajectories for anomaly detection in videos. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 (pp. 11996-2004).

[5][5]Dos SFP, Ribeiro LS, Ponti MA. Generalization of feature embeddings transferred from different video anomaly detection domains. Journal of Visual Communication and Image Representation. 2019; 60:407-16.

[6][6]Fan Y, Wen G, Li D, Qiu S, Levine MD, Xiao F. Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder. Computer Vision and Image Understanding. 2020.

[7][7]Ren J, Xia F, Liu Y, Lee I. Deep video anomaly detection: opportunities and challenges. In international conference on data mining workshops 2021 (pp. 959-66). IEEE.

[8][8]Abbas ZK, Al-Ani AA. A Comprehensive review for video anomaly detection on videos. In international conference on computer science and software engineering 2022 (pp. 1-1). IEEE.

[9][9]Nouripayam M, Sheikhipoor N. HEVC (H. 265) Intra-Frame prediction implementation using MATLAB. 2014.

[10][10]Khaire P, Kumar P. A semi-supervised deep learning based video anomaly detection framework using RGB-D for surveillance of real-world critical environments. Forensic Science International: Digital Investigation. 2022.

[11][11]Sharfuddin AA, Tihami MN, Islam MS. A deep recurrent neural network with bilstm model for sentiment classification. In international conference on Bangla speech and language processing 2018 (pp. 1-4). IEEE.

[12][12]Chaudhary S, Khan MA, Bhatnagar C. Multiple anomalous activity detection in videos. Procedia Computer Science. 2018; 125:336-45.

[13][13]Bhagyalakshmi P, Indhumathi P, Bhavadharini LR. Real time video surveillance for automated weapon detection. International Journal of Trend in Scientific Research and Development. 2019; 3(3).

[14][14]Sultani W, Chen C, Shah M. Real-world anomaly detection in surveillance videos. In proceedings of the IEEE conference on computer vision and pattern recognition 2018 (pp. 6479-88).

[15][15]Shine L, CV J. Automated detection of helmet on motorcyclists from traffic surveillance videos: a comparative analysis using hand-crafted features and CNN. Multimedia Tools and Applications. 2020; 79(19):14179-99.

[16][16]Shreyas DG, Raksha S, Prasad BG. Implementation of an anomalous human activity recognition system. SN Computer Science. 2020; 1(3):1-10.

[17][17]Ramchandran A, Sangaiah AK. Unsupervised deep learning system for local anomaly event detection in crowded scenes. Multimedia Tools and Applications. 2020; 79(47):35275-95.

[18][18]Anala MR, Makker M, Ashok A. Anomaly detection in surveillance videos. In 26th international conference on high performance computing, data and analytics workshop (HiPCW) 2019 (pp. 93-8). IEEE.

[19][19]Liu K, Ma H. Exploring background-bias for anomaly detection in surveillance videos. In proceedings of the 27th ACM international conference on multimedia 2019 (pp. 1490-9).

[20][20]Zhang J, Qing L, Miao J. Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection. In international conference on image processing 2019 (pp. 4030-4). IEEE.

[21][21]Zhu Y, Newsam S. Motion-aware feature for improved video anomaly detection. arXiv preprint arXiv:1907.10211. 2019.

[22][22]Zhong JX, Li N, Kong W, Liu S, Li TH, Li G. Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 (pp. 1237-46).

[23][23]Hao W, Zhang R, Li S, Li J, Li F, Zhao S, et al. Anomaly event detection in security surveillance using two-stream based model. Security and Communication Networks. 2020.

[24][24]Venkatesh SV, Anand AP, Gokul SS, Ramakrishnan A, Vijayaraghavan V. Real-time surveillance based crime detection for edge devices. In VISIGRAPP (4: VISAPP) 2020 (pp. 801-9).

[25][25]Cheng M, Cai K, Li M. RWF-2000: an open large scale video database for violence detection. In 25th international conference on pattern recognition 2021 (pp. 4183-90). IEEE.

[26][26]Zaheer MZ, Mahmood A, Astrid M, Lee SI. Claws: clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In European conference on computer vision 2020 (pp. 358-76). Springer, Cham.

[27][27]Dubey S, Boragule A, Gwak J, Jeon M. Anomalous event recognition in videos based on joint learning of motion and appearance with multiple ranking measures. Applied Sciences. 2021; 11(3):1-21.

[28][28]Wu G, Guo Z, Wang M, Li L, Wang C. Video abnormal event detection based on CNN and multiple instance learning. In twelfth international conference on signal processing systems 2021 (pp. 134-9). SPIE.

[29][29]Aziz Z, Bhatti N, Mahmood H, Zia M. Video anomaly detection and localization based on appearance and motion models. Multimedia Tools and Applications. 2021; 80(17):25875-95.

[30][30]Boekhoudt K, Matei A, Aghaei M, Talavera E. HR-crime: human-related anomaly detection in surveillance videos. In international conference on computer analysis of images and patterns 2021 (pp. 164-74). Springer, Cham.

[31][31]Wan B, Jiang W, Fang Y, Luo Z, Ding G. Anomaly detection in video sequences: a benchmark and computational model. IET Image Processing. 2021; 15(14):3454-65.

[32][32]Zaheer MZ, Lee JH, Astrid M, Mahmood A, Lee SI. Cleaning label noise with clusters for minimally supervised anomaly detection. arXiv preprint arXiv:2104.14770. 2021.

[33][33]Majhi S, Das S, Brémond F, Dash R, Sa PK. Weakly-supervised joint anomaly detection and classification. In IEEE international conference on automatic face and gesture recognition 2021 (pp. 1-7). IEEE.

[34][34]Wu J, Zhang W, Li G, Wu W, Tan X, Li Y, et al. Weakly-supervised spatio-temporal anomaly detection in surveillance video. arXiv preprint arXiv:2108.03825. 2021.

[35][35]Tian Y, Pang G, Chen Y, Singh R, Verjans JW, Carneiro G. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In proceedings of the IEEE/CVF international conference on computer vision 2021 (pp. 4975-86).

[36][36]Cao C, Zhang X, Zhang S, Wang P, Zhang Y. Adaptive graph convolutional networks for weakly supervised anomaly detection in videos. arXiv preprint arXiv:2202.06503. 2022.

[37][37]Maqsood R, Bajwa UI, Saleem G, Raza RH, Anwar MW. Anomaly recognition from surveillance videos using 3D convolution neural network. Multimedia Tools and Applications. 2021; 80(12):18693-716.

[38][38]https://www.dropbox.com/sh/75v5ehq4cdg5g5g/AABvnJSwZI7zXb8_myBA0CLHa?dl=0. Accessed 15 April 2022.

[39][39]Kumari P, Bedi AK, Saini M. Multimedia datasets for anomaly detection: a review. arXiv preprint arXiv:2112.05410. 2021.

[40][40]Hassan KH, Butt SA. Motion estimation in HEVC/H. 265: metaheuristic approach to improve the efficiency. Engineering Proceedings. 2021; 12(1):1-4.

[41][41]Mathew A, Amudha P, Sivakumari S. Deep learning techniques: an overview. In international conference on advanced machine learning technologies and applications 2020 (pp. 599-608). Springer, Singapore.

[42][42]Zhang A, Lipton ZC, Li M, Smola AJ. Dive into deep learning. arXiv preprint arXiv:2106.11342. 2021.