Abnormal Crowd Behavior Detection using Social Force Model
One of the most challenging tasks in computer vision is analysis of human activity in crowded scenes. In addition, research in sociology and behavioral sciences provide mathematical models of pedestrian behavior patterns such as Social Force Model. In this paper, we introduce a computer vision method based on particle advection to detect and localize abnormal crowd behavior using the Social Force model.
(a) The block diagram of the abnormal behavior detection algorithm
- Challenges of crowd behavior analysis
- Low resolution
- Ignoring social interaction of pedestrians
- Considering a crowd as a global entity using particle advection method.
- Using Social Force Model to model crowd behaviors.
- The holistic approach provides robustness to occlusion and clutter.
- Social Force Model endows the way to analyze crowd behaviors based on interaction forces.
In Social Force Model an individual is subject to long-ranged forces and his/her dynamics follow the equation of motion, similar to Newtonian mechanics. The velocity of an individual is described as the result of a personal desire force and interaction forces.
(b) A visualization of the forces and velocities in the social force model
2. Dynamic Model
In this model, the individual dynamics of pedestrians is modeled as:
3. Gernalized Social Force Model
Estimating the interaction forces in the a crowd is a daunting task because of the occlusion and clutter. The holistic approach of particle advection provides an alternative way to compute these forces.
- Particle Advection
(c) An example of the particle advection (Left). The computed social forces for the particles yellow arrows (Right).
- The motion of crowd is captured by moving a grid of particles using the optical flow (O).
- In a crowd, people tend to move with the average flow therefore, the spatio-temporal average of optical flow (Oave) is used for particle advection.
(d) The optical flow and interaction forces between a gentle man walking across a crowd: The computed interaction forces (red), the optical flow (yellow)
Regarding particles as individuals in the crowd, Social Force Model can be adapted for particles:
The value of the interaction forces are not enough to understand the dynamics of the crowd.
- The pattern of forces in a scene represent different behaviors in the crowd.
- Force flow: The interaction forces in a scene is computed by mapping the magnitude of forces of particles to the frame plain.
(e) The flow of interaction forces in the video frame and the definition of visual words
- Abnormal Event DetectionLDA: Thresholding the Likelihood of a clip to distinguish normal and abnormal set of frame.
- The location of abnormal events are highlighted in the areas of high interaction forces in the abnormal clips.
- UMN Dataset
- Crowd Escape Panic, 11 Videos, 3 Scenes, Videos: a normal starting section and an abnormal ending section
- Trained on normal section of 5 videos of scene 1, Tested on all
(e) The ROC on UMN dataset
(f) Abnormal Behavior Detection on UMN Dataset
2. Experiments on Web Dataset
- 8 Videos of real-life Escape panic, clash, fight
- 12 Videos of normal pedestrians
- Trained and tested in a 2-fold validation manner
(g) Web dataset example results
(h) The ROC on Web dataset
- UMN Dataset: Unusual Crowd Activity
- Thought Equity Stock Footage Videos: Crowd Panic, Normal Crowd (New York).
Ramin Mehran, Alexis Oyama, and Mubarak Shah, Abnormal Crowd Behavior Detection using Social Force Model, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, 2009