Detecting anomalous behaviour in audio-video sensor networks

This project investigates the identification and classification of behaviours as normal or abnormal, safe or threatening, from an irregular and often heterogeneous sensor network. Although some general principles may be applied, there is unlikely to be a unified approach that can be applied across different sensor domains; for example, tracking moving ‘blobs’ in radar or sonar data is quite distinct in dimension, mathematical and applied treatment from tracking human subjects in CCTV image data. In this work project we focus specifically on the problems of using electro-optic (video, IR, LiDAR) and audible data to monitor behaviour and detect anomalies, therefore addressing a number of specified areas of interest and challenges. Our current work addresses the problem of statistical anomaly detection in video pattern recognition. This can be applied to tracking people and objects in surveillance networks; learning statistical representations of normality which evolve to produce pattern-of-life estimates for targets; detecting deviations from normality and using the spatial context of behaviour to filter out spurious anomalies. The same context driven approach can be applied to various domains for example, airport or city surveillance. Initial progress has focused on developing algorithms to incorporating contextual understanding into low-level tracking algorithms. Specifically, we have focused on concourse protection where the challenge is to identify a threat before it reaches the entry point of an asset (e.g. checkpoint, forward operating base). We have developed a novel head-pose classifier for surveillance video that is able to identify where a person is looking. Our approach has shown encouraging results on two public video datasets (Benfold and Caviar), as shown below:

By integrating head-pose information from our classifier into a Kalman Filter our ‘Intentional Tracker’ uses this contextual prior when making predictions about target movement. We have demonstrated that using this intentional prior allows us to make better predictions of expected behaviour. This theory generalises to different types of target (e.g. vehicles using indicators) and other spatio-temporal priors, which will become the focus of our work over the coming months. Using contextual information our algorithms construct better models of normal behaviour. This enables them to make more accurate predictions of expected behaviour through which anomalous activity can be identified by its low probability.



Dr. Rolf H Baxter
Dr. Neil M Robertson

Useful links: