Incremental Activity Modelling in Multiple Disjoint Cameras TPAMI'12 ICCV'09

Introduction

Activity modelling and unusual event detection in a network of cameras is challenging particularly when the camera views are not overlapped. We show that it is possible to detect unusual events in multiple disjoint cameras as context-incoherent patterns, through incremental learning of time delayed dependencies between distributed local activities observed within and across camera views. Specifically, we model multi-camera activities using a Time Delayed Probabilistic Graphical Model (TD-PGM) with different nodes representing activities in different decomposed regions from different views and the directed links between nodes encoding their time delayed dependencies. To deal with visual context changes, we formulate a novel incremental learning method for modelling time delayed dependencies that change over time. We validate the effectiveness of the proposed approach using a synthetic dataset and videos captured from a camera network installed at a busy underground station.

Contribution Highlights

• This work is the first study on modelling time delayed activity dependencies for real- time detection of global unusual events across distributed multi-camera views of busy public scenes.
• Existing studies generally assume activity model that remain static once learned; the problem of incremental global activity modelling in multiple disjoint cameras have not been addressed before. To cope with the inevitable visual context changes over time, a novel incremental two-stage structure learning method is pro- posed to discover and quantify optimised time delayed dependency structure globally.

Citation

1. Incremental Activity Modelling in Multiple Disjoint Cameras
   C. C. Loy, T. Xiang, and S. Gong
   IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1799-1813, 2012 (TPAMI)
   DOI PDF
2. Modelling Activity Global Temporal Dependencies using Time Delayed Probabilistic Graphical Model
   C. C. Loy, T. Xiang, and S. Gong
   in Proceedings of International Conference on Computer Vision, pp. 120-127, 2009 (ICCV, Oral)
   PDF

Images

Underground station:

The underground station layout, the camera views, and the scene decomposition results for our dataset. Entry and exit points are shown in red bars.

Activity global dependency graph:

An activity global dependency graph learned using the proposed two-stage structure learning method with BIC scoring function. Edges are labelled with the associated time delays discovered using the Time Delayed Mutual Information analysis. Regions and nodes with discovered inter-camera dependencies are highlighted.

Example:

Example frames from detection output using the proposed approach on analysing unusual events caused by atypical long queues. The plot depicts the associated cumulative abnormality scores produced by different methods over the period. In ground truth, unusual events occurred at frames (5741-5853) and (5915-6376).

Example:

Global unusual event due to faulty train, which first occurred in Cam 6 and 7, and later propagated to Cam 5, 4, and 3. The plot depicts the cumulative abnormality scores in Region 55 produced by different methods over the period. In ground truth, this unusual event occurred at frames (15340-15680).

Datasets and Codes

Related Projects

• Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding
• Comparing Visual Feature Coding for Learning Disjoint Camera Dependencies