UNUSUAL EVENT DETECTION AND LOCALIZATION USING OPTICAL FLOW TECHNIQUE

--Video-surveillance systems are becoming more and more independent in the finding and the exposure of anomalous events. The safety of mankind in crowds under disaster situation is one of the most essential concerns of researchers. This paper presents an approach to notice abnormal situations in crowded scenes by analysing the movement aspect and localize the unusual event using clusterization method . Experiments on UMN datasets show that the proposed method is promising.

shown a novel, probabilistic technique for background subtraction. This technique deals with slow lighting changes by slowly adapting the values of the Gaussians and multimodal distributions caused by shadows, secularities, swaying branches, computer monitors, and other troublesome features of the real world which are not frequently mentioned in computer vision. Limitation of their method is that it works for selected areas and cannot classify the objects. Helbing et al. [7] used a model of pedestrian behaviour to investigate the mechanisms of panic and jamming by uncoordinated motion in crowds. Kamijo et al. [8] presents HMM-based event model was proposed for bumping, passing, jamming, illegal U-turn, and reckless driving detection using track features. Although the algorithm achieves such reliable tracking, it requires only gray-scale images. Lin et al. [9] developed a system goes one step further to estimate the number of people in crowded scenes in a complex background by using a single image. The detection tool can work well in some cases, it is still limited to some complex situations such as when the overlap among people is great and the contours of the heads are not clear. Rudolf Mester [10] shows how prior knowledge on the video signal can be integrated into a motion estimation procedure. Relations between different classes of motion algorithms are discussed and perspectives for a fusion and improvement of such procedures are presented. Niu et al. [12] proposed an algorithm for activity modelling and recognition from video clips captured at arbitrary views. They compared the use of only motion features, only shape features and both motion and shape features to build HMMs. It may not be capable to recognizing complex activities. Andrade et al. [13] applies a video modelling technique to a surveillance scenario where pedestrians are monitored to detect unusual events. It demonstrates that patterns derived from optical flow and encoded by a Hidden Markov Model are able to capture the dynamic evolution of normal behaviour allowing the classification of abnormal events. Pankaj Kumar et al. [14] present a rule-based framework for behaviour and activity detection in traffic videos obtained from stationary video cameras. These are classified into different categories using a novel Bayesian network approach, which makes use of image features and image-sequence-based tracking results for robust classification. It is capable of handling uncertainties that arise due to errors in visual signal processing. This system's performance can be further improved by having context images made more specific to the target. Armin Seyfried [15] presents the findings for the single-file movement with literature data for the movement in a plane. In the analysis presented they conclude the fundamental diagram for the single-file movement of pedestrians under laboratory conditions. The data are suitable to test, if microscopic models are able to reproduce the experiential relation between velocity and density in the simplest system. Andrade et al. [16] characterise crowd behaviour by observing the crowd optical flow and use unsupervised feature extraction to encode normal crowd behaviour. The experiments show that the bank of models is useful in quickly detecting the simulated emergency situation in a dense crowd. Andrade et al. [17] presented two different detection methods are implemented to detect global and local disaster scenarios. The hypothesis of having a dense crowd to detect enough flow changes can be lifted if the video sequences are modelled with banks of HMMs trained for normal motion in different density scenarios. This would be applicable only from modelling of dense crowds to pedestrian traffic. Blunsden et al. [18] presented an automatic technique for detection of abnormal events in crowds. Using projections of the eigenvectors in a sub-space spanned by the normal crowd scene as an input feature. The investigation of the relation between the number of eigenvectors and the model likelihood variations indicates that all configurations present a significant drop relative to the normal case and are able to correctly detect the emergencies. Masaaki et al. [19] describe a way for estimating optical flow from coloured images. In their approach, RGB data in the captured colour images are converted to the colour vectors. Since the colour vector has arbitrary orientation, the analysis of the optimal angle of the vector is reported in this paper. Min et al. [20] propose an algorithm that uses instantaneous motion field of the video instead of long-term motion tracks for learning the motion patterns. The motion field is a set of independent flow vectors identified in each frame of the video. Xiang et al. [21] developed a new visual behaviour modelling approach that performs incremental and adaptive model learning for online anomaly recognition in a visual surveillance scene. The approach has the following key features that make it useful over previous ones: (1) fully unsupervised learning. (2) Robust abnormality detection (3) Online and incremental model construction (4) Model adaptation to reflect changes in visual context. Antoni et al. [22] compared with state-of-the-art techniques in motion segmentation, together with temporal texture methods and conventional representations (e.g. optical flow or other localized motion representations), the combination of dynamic textures achieves superior performance in the problems of clustering and segmenting video of such processes. Nacim et al. [23] Identified events where local motion variation is significant compared to earlier events. Optical flow methods are used to extract information such as compactness, direction and velocity. It is likely to extend the estimation of the motion variations with factors such as acceleration by tracking the Points Of Interest (POIs) over various frames. Amit et al. [24] proposed a new algorithm for recognition of certain types of abnormal events. In this algorithm multiple local monitors to collect low-level statistics and produces an alert these alerts are incorporated to an ultimate decision regarding the existence of an unusual event. The main limitation of their algorithm is the lack of sequential monitoring. Ramin et al. [26] proposed a technique to identify and localize unusual behaviours in crowd videos using Social Force model. A grid of particles is placed over the image and it is advected with the space-time average of optical flow. They categorize frames as usual and unusual by using a bag of words technique. The regions of anomalies in the unusual frames are localized using communication forces. Tian et al. [27] proposed a technique for the detection of anomaly in busy scenes based on the crowd movement description. These description include the crowd kinetic energy and the movement directions. This estimates the crowd kinetic energy and the motion directions using the optical flow techniques. Wang et al. [28] presented for abnormal sections recognition in video sequences using the histogram to illustrate the colour alteration in the segment, and the histograms of the frames identified from the section create the histogram matrix. To improve the procedure effectiveness, the principal components analysis (PCA) is used to decrease size of the histogram matrix. The information of frames differences is used to illustrate the motion variation information, and the information twice frame differentiation is used to illustrate the rate of change. WALI et al. [29] proposed a new method for event recognition from video surveillance information using optical flow histogram without previous information of the motion pattern. They begin by estimating the motion from images series by optical flow technique. Second, followed by classification using the histogram of the optical flow vectors and using a chain coding algorithm. At last, they extract a high-level feature from any frame for use in the learning and investigate events by SVM and HMM. Sukeshini et al. [30] presented optical flow motion vector estimation using iterative Lucas-Kanade Pyramidal implementation for both large & small motion. The velocity & displacement at every pixel is obtained by using Lucas-Kanade equations. The original image is recovered by warping reference frame towards current frame by flow vectors i.e. velocity & displacement by using image warping techniques. Shu Wang et al. [31] proposed a method to automatically identify anomalous behaviour in crowd scene. They separated whole frame into small blocks, and movement pattern in each chunk is encoded by the distribution of motion vectors in it. Similar motion patterns are grouped into pattern model in an unsupervised way, and they categorize motion pattern into usual or unusual group according to the variation between motion pattern and trained model. It is required to make the descriptor stronger in poor lighting scene. Julio et al. [32] presented a study on crowd analysis based on computer vision. This study focused on three important problems in crowd analysis: people counting/density estimation, tracking in crowded scenes, and crowd behaviour understanding in a higher-level analysis, like the temporal evolution, main directions, velocity estimations, and recognition of abnormal situations. Abhijit et al. [33] proposed a real time moving entity recognition algorithm from a single freely moving monocular camera. An online visual SLAM algorithm working at the same time estimates the camera egomotion. Christoph et al. [34] used a twostage mean-shift algorithm for detection of all moving objects in the scene by an image-based mean-shift which utilises a grey value histogram as a target model and refinement of the object pose largely along the depth axis. The prediction step in the tracking system relies on flow-based, kinematic, and trajectory-based prediction methods. Cen Rao et al. [35] present a view-invariant illustration of movement having dynamic instants and intervals that is computed using spatiotemporal curvature of a path. Then this illustration is used by their method to learn human behaviour without any training. The system automatically segments video into individual behaviour, and computes view invariant illustration for each action. Lili Cui et al. [37] presented an anomalous event recognition technique using local features for traffic video surveillance. Each forefront region's area, shape factors and pixel affecting velocity vector are extracted. Using these character, regions are classified into dissimilar groups as pedestrian, vehicle or noise region, and their actions is classified using trained local features' distribution map. Finally, a simple classifier is used to decide objects' states of usual or unusual. Su Yang et al. [38] presented a case study of detecting unusual traffic flows and find out new resolution by presentation the coevolving data streams as high-dimensional patterns and applying various learning techniques to capture the description of such patterns. Xiaobai Liu et al. [39] proposed a flexible structure that combines the spatiotemporal contextual knowledge with a novel deformable pattern matching process. In this process, the background modelling was scene-specific and thus not flexible enough for portable visual surveillance system. Duan-Yu Chen et al. [40] estimated optical flows and used to find out cluster of human crowds in unconfirmed manner using their proposed scheme of adjacency-matrix based clustering (AMC). Based on clusters of human crows, their characters with properties, direction, location and crowd size, are considered by a model of force field. Finally, they predicted the characters of human crowds and then detect if any abnormalities of human crowd there in the scene. Mahfuzul et al. [41] proposed a common event recognition method in which the extracted low-level characters signify the global features of the target scene instead of any context-specific information. In this method, supervised training and thorough feature extraction are necessary only once while construction the target event models. Hajananth et al. [42] presented a model using a Semi-2D Hidden Markov Model (HMM) to the usual actions of people. Their Semi-2D HMM is intended to model both the temporal and spatial causalities of the crowd activities by assuming the present state of the Hidden Markov Model depends not only on the preceding state in the of time direction, but also on the earlier states of the nearby spatial locations. Tian Wang et al. [43] proposed an algorithm to identify unusual events based on histograms of the orientation of optical flow illustrator and one-class SVM classifier. They presented grids of Histograms of the Orientation of Optical Flow (HOFs) as the descriptors for movement information of the colossal video frame. Hiral Raveshiya et al. [44] presented summary of some fundamental concepts related to motion estimation using optical flow estimation processes and activity estimation techniques are attractive research area which is described here in brief. Sharmin et al. [45] presented an image filtering scheme as a pre-processing stage for the Lucas-Kanade pyramidal optical flow algorithm. Based on a study of dissimilar types of filtering techniques and applied on the Iterative Refined Lucas-Kanade, resulted on the best filtering exercise. As the Gaussian smoothing filter was selected, an experiential approach for the Gaussian variance estimation was introduced. Oluwatoyin et al. [46] focuses on relative unusual human behaviour recognition in particular in video surveillance applications. The main reason of this review is to widely recognize existing techniques and distinguish the literature in a method that brings key challenges to notice. Hajananth et al. [47] studied the performance of different state of the art features to identify the occurrence of the unusual objects in the scene. These comprise optical flow vectors to detect motion related abnormalities, textures of optical flow and image textures to identify the occurrence of anomalous objects. Hiba et al. [48] presented a novel automated algorithm for the Detection of Abnormal activities in Dynamic Crowded Gatherings (DADCG) that has diminished processing speed, understanding to noise, and enhanced precision. Firstly, the temporal features of the scenes are extracted using Motion History Image (MHI) technique. Then the Optical Flow (OF) vectors are calculated for each MHI image using Lucas-Kanade method to get the spatial character. This Optical flow image is segmented into four equal-sized segments. Finally, a two dimensional histogram is calculated with motion track and motion scale for each chunk. Weiguo et al. [49] studied the scheme and technology for executing and analyzing two critical controlled experiments. Based on the study and some typical characteristics of the singlefile experiment, a one dimensional continuous distance model (CDM) was projected in their earlier work and in this study it is improved by introducing the needed moving direction so as to describe the two-dimensional bottleneck movement. Nguyen et al. [50] presented a method that include Brox's optical flow algorithm with background subtraction with a new method to get better optical flow in the traffic surveillance system. Malavika et al. [51] presented an implementation of image subtraction algorithm to identify a moving object and the velocity has been calculated. Dhara et al. [52] presented general idea of some elementary concepts associated to motion estimation and presented the Lucas kanade method which is broadly used in differential method for optical flow estimation and computer vision. Soheil et al. [53] proposed a comparative study of Cepstrum, principal component analysis (PCA) and symbolic dynamic filtering (SDF) as feature extractors for target detection and classification. These methods have been tested on two data sets, consisting of passive infrared (PIR) and seismic sensors, collected from different fields. The underlying algorithms of feature extraction have been executed in conjunction with three different classification algorithms, namely, support vector machines (SVM), k-nearest neighbour (k-NN), and sparse representation classifier (SRC). Crossvalidation has been used to assess the performance of Cepstrum, PCA and SDF as feature extractors for both PIR and seismic sensor data sets. Jorge et al. [54] presented a scheme for unusual activity investigation that ensures efficiency in the inferences with less computational effort. They use a region-based model that uses a mobile grid of sub regions constructed from scene's ROI (region of interest). Rajanayaki et al [55] presented model to identify the global actions in crowd through observing motion patterns such as flow, velocity and direction. Background subtraction as a pre processing step is performed to extract the foreground blobs and optical flow is calculated to find the velocity and direction of motion. The human crowds are then clustered based on alike direction and nearness using Adjacency Matrix based Clustering (AMC). Chunyu et al. [56] concentrated on the challenges of recognition and localization of crowd escape abnormal behaviours in video surveillance process. This scheme identifies the unusual events as well as identifies the potential location of unusual events. Tian et al. [57] introduced a histogram of optical flow orientation as a descriptor encoding and nonlinear one-class support vector machine classification algorithm, subsequent a learning period characterizing the usual behaviour of training frames, recognizes unusual events in the present frame. Aravinda et al. [58] presented crowd activity recognition and classification in optical flow methods (OFMs) to identify walking and running events, which uses optical flow vector lengths in OFMs. Prabuwono et al.
[59] presented a two-layer bucket method using moment-based feature extraction algorithm (MFEA). A moment represents the coherent intensity of pixels and motion estimation is a motion pixel intensity measurement, this research used this relation to develop the proposed algorithm.Their approach contributes to the detection of abnormal events in crowd flows by the fact that it doesn't need learning process and training data, it also considers simultaneously density, direction and velocity and focuses analysis on specific regions where the density of motions is high. Weixin et al.
[60] proposed a detector based on a video representation that accounts for both appearance and dynamics, using a set of mixture of dynamic textures models. A data set of densely crowded pedestrian walkways is introduced and used to evaluate the proposed anomaly detector.

III. PROPOSED APPROACH
In this section, different processing components of the proposed event detection framework are presented.

A. Lucas-Kanade Method for Optical Flow Measurement
The Lucas-Kanade method is broadly used in differential method for optical flow estimation and computer vision. This method solves the fundamental optical flow equations for all the pixels in that neighbourhood, by the least squares criterion. It is a purely local process because it cannot offer flow knowledge in the interior of consistent regions of the image. Here assumes that the flow is essentially constant in a local neighbourhood of the pixel under consideration. The Lucas-Kanade process assumes that the displacement of the image contents between two consecutive frames is small and approximately constant within a neighbourhood of the point p.

I x (p 1 ) V x + I y (p 1 ) V y = -I t (p 1 ) I x (p 2 ) V x + I y (p 2 ) V y = -I t (p 2 )
.
. I x (p n ) V x + I y (p n ) V y = -I t (p n ) Based on the least squares principle to find out the solution of Lucas-kanade method. Now A T AV=A T b, this equation can also be written in this form, so we get: Where n=1 to N The matrix A T is often called the structure tensor of the image at the point p. Weighted window: The plain least squares solution above gives the similar importance to all N pixels p n in the window. Practically it is generally better to assign more weight to the pixels that are nearer to the central pixel p. For that, one uses the weighted version of the least squares equation. A T WAV=A T Wb (5) We can also write this equation in this form, so we get:

V= (A T WAV) -1 A T Wb
Where W is an nxn diagonal matrix having the weight Wii=wi to be given to the equation of pixel p n. It computes: The weight w i is generally set to a Gaussian function of the distance between p n and p. Lucas-Kanade and others implemented a weighted least squares (LS) fit of local first-order constraints to a constant model for v in every small spatial neighbourhood by minimizing , , , , , Where w(x, y) indicates a window function that gives additional influence to constraints at the centre of the neighbourhood.

. Abnormality Detection Based on the Energy of the Optical Flow
Abnormal behaviours generally have features such as big motion magnitudes, fast velocities, and various directions and so on. While the motion magnitudes of the usual behaviours are not big and the motion directions are roughly same, consequential to the motion energy is low in one direction. In the case of anomalous behaviours, due to the litter of motion direction and large speed in the process of movement, the kinetic energy of speed must be higher than the usual behaviours. Thus the kinetic energy of optical flow is useful to reaction the behavioural characteristic. In this paper, the kinetic energy of n th frame optical flow is extracted using the following equation: E(n) depicts the kinetic energy of the optical flow in the movement region. Parameter V i,j(n) is the velocity of the n th frame image pixels pix(i,j) and coefficient w i,j (n) is the weight of the optical flow energy. W denotes the width of the movement area and H denotes the height of the movement area. The velocity of each pixel in the movement area is calculated using following equation: The offsets of horizontal direction and vertical direction are ⧍X i,j and ⧍Y i,j respectively. The angle between the moving direction and the horizontal direction is denoted as: The optical flow includes instantaneous speed as well as movement direction of each pixel. The magnitude of velocity shows the intensity of movement, and the motion direction angle can used to find the direction of moving target. In the case of the usual behaviour, the motion directions are generally same and the difference between pixels is not large. On the contrary, the directions of movement are not in order and not steady, the movement with a high and strong magnitudes for the unusual behaviours. Related with the characteristics of the optical flow, this point outs the variation of pixels speeds and the variation of the directions of motion. Therefore, for the non-weighted optical flow, it does not differentiate the unusual behaviours and the usual behaviours properly only by using the knowledge of the optical flow speed. As per the above discussion, we describe the weight of the optical flow energy: ∆Angel |∠AngelCurrent ∠AngelAvg| ∆AngelMax |∠AngelCurrent ∠AngelMax| (13) Where ∠AngelCurrent indicates the angel between the present pixel and the horizontal direction. The angle of the max speed pixel is ∠AngelMaxin the movement region. The average angle between moving target and horizontal direction: Where Sum (P) indicates the sum of all pixels in the movement region.

IV. EXPERIMENTAL RESULTS
In this section, we test our algorithm on the real data is the public dataset: the UMN dataset which have been widely used for performance evaluation. Most of the parameters associated with the proposed method are set to fixed values. The video frame size is 240×320 pixels, data rate 4500 kbps and frame rate 30 frame/second. In our experiment, we tested our algorithm on Intel Pentium 2.66 GHz quad core processor with 2 GB RAM. We show the results based on the energy of motion using optical flow estimation on the UMN dataset. We compare results with the Horn-Schunck Method for determining the unusual events. We estimate optical flow using Lucas-Kanade derivative of Gaussian method. First we have converted the RGB video in to gray scale video so that it occupies low space on disk. Due to this conversion the processing cost of the video become very low and system will be very efficient. From Figure 2 and fig. 4, the proposed approach successfully detected the trend of the abnormality. However, in order to get an accurate output, it is necessary to choose the suitable threshold.

B. Unusual Event Localization:
A few researches have been studied for the localization of anomalous crowd behaviours in videos. Some literatures about the localization are almost only study the individual anomalies. However, the consequence caused by the anomalous crowd behaviours is more severe, so determining the location of the anomalous crowd behaviours is very important to take action timely when the unusual events happen. For the abnormal crowd events localization, literature has been mentioned that the Markov Chain Monte Carlo method is used to address the localization issue. We consider the problem of abnormal crowd events localization, the notion of the divergent centre is inspired by the idea of literature [60]. We proposed a clusterazation method to determine the divergent centre. We test our approach on public database and some realistic video recorded by ourselves: public database including the UMN database. UMN database includes 3 videos in 3 different scenes: lawn, indoor and square. All these 3 videos depict people wandering in the beginning and suddenly evacuate quickly at a time. Every video of UMN database video has a relative long time of normal situation in the beginning and short abnormal situation in the end. In the proposed method first starting frame of unusual event is detected using optical flow technique. Then optical flow vector is divided into small clusters. On the bases of velocity magnitude of each cluster it can detect the location of unusual event. As shown in fig 5(a), fig 6(a), and fig 7(a) In table-1, table-2, table-3 Corresponding to fig-7   1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31

V. Conclusions
People usually instinctively escape from a place when abnormal or dangerous events occur, we consider crowd escape behaviour as an indicator of an abnormal event. We proposed a method to detect anomalous crowd behaviours and its initial location. The proposed method can detect and localize the abnormal behaviours happen. We conducted the experiments on the UMN dataset. We have made a comparative analysis with other methods and found that our method is more efficient to detect the abnormal events in crowed and its location. Finally, we can detect and localize if any anomalous crowd behaviours present in the scene based on the energy of the optical flow.