Activity modelling and unusual event detection in a network of cameras is challenging particularly when the camera views are not overlapped. We show that it is possible to detect unusual events in multiple disjoint cameras as context-incoherent patterns, through incremental learning of time delayed dependencies between distributed local activities observed within and across camera views. Specifically, we model multi-camera activities using a Time Delayed Probabilistic Graphical Model (TD-PGM) with different nodes representing activities in different decomposed regions from different views and the directed links between nodes encoding their time delayed dependencies. To deal with visual context changes, we formulate a novel incremental learning method for modelling time delayed dependencies that change over time. We validate the effectiveness of the proposed approach using a synthetic dataset and videos captured from a camera network installed at a busy underground station.
The underground station layout, the camera views, and the scene decomposition results for our dataset. Entry and exit points are shown in red bars.
An activity global dependency graph learned using the proposed two-stage structure learning method with BIC scoring function. Edges are labelled with the associated time delays discovered using the Time Delayed Mutual Information analysis. Regions and nodes with discovered inter-camera dependencies are highlighted.
Example frames from detection output using the proposed approach on analysing unusual events caused by atypical long queues. The plot depicts the associated cumulative abnormality scores produced by different methods over the period. In ground truth, unusual events occurred at frames (5741-5853) and (5915-6376).
Global unusual event due to faulty train, which first occurred in Cam 6 and 7, and later propagated to Cam 5, 4, and 3. The plot depicts the cumulative abnormality scores in Region 55 produced by different methods over the period. In ground truth, this unusual event occurred at frames (15340-15680).
MATLAB code for computing time delayed mutual information (TDMI) between time series.
Time Delayed Mutual Information analysis is employed in our work for learning pairwise time delayed association between regional activity patterns in a multi-camera network.
Reference:
C. C. Loy, T. Xiang, and S. Gong, Incremental Activity Modelling in Multiple Disjoint Cameras, TPAMI 2012
MATLAB code for computing cross canonical correlation (xCCA) between time series.
Reference:
C. C. Loy, T. Xiang, and S. Gong, Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding, IJCV 2010
MATLAB code for segmenting a scene into regions based on activity patterns. C++ codes for spectral clustering by L. Zelnik-Manor and P. Perona are included.
Reference:
C. C. Loy, T. Xiang, and S. Gong, Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding, IJCV 2010
MATLAB and C++ codes for background modelling and subtraction.
A mean-shift based method for robust background subtraction in video with sudden global intensity change. A static background image is first generated based on minimum cut, the method then adapts the background image to the intensity level of current frame prior to actual background subtraction.
Reference:
C. C. Loy, T. Xiang, and S. Gong, Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding, IJCV 2010