Abnormal Crowd Behavior Detection using Social Force Model

• Introduction
  • Challenges of crowd behavior analysis
  • The solution
  • Advantages of the proposed method
• Social Force Model
  • Description
  • Dynamic Model
  • Generalized Model
• Estimation of Interaction Forces
  • Particle Advection
  • Computing Social Forces
• Event Detection
• Results
  • UMN Dataset
  • Web Dataset
• Download Dataset
• Related Links
• Related Publication

Introduction

One of the most challenging tasks in computer vision is analysis of human activity in crowded scenes. In addition, research in sociology and behavioral sciences provide mathematical models of pedestrian behavior patterns such as Social Force Model. In this paper, we introduce a computer vision method based on particle advection to detect and localize abnormal crowd behavior using the Social Force model.


(a) The block diagram of the abnormal behavior detection algorithm

1. Challenges of crowd behavior analysis

   • Occlusion
   • Clutter
   • Low resolution
   • Ignoring social interaction of pedestrians
   
   

2. Solution

   • Considering a crowd as a global entity using particle advection method.
   • Using Social Force Model to model crowd behaviors.
   
   

3. Advantages

   • The holistic approach provides robustness to occlusion and clutter.
   • Social Force Model endows the way to analyze crowd behaviors based on interaction forces.

Social Force Model

1. Description

   In Social Force Model an individual is subject to long-ranged forces and his/her dynamics follow the equation of motion, similar to Newtonian mechanics. The velocity of an individual is described as the result of a personal desire force and interaction forces.
   (b) A visualization of the forces and velocities in the social force model
   
   

2. Dynamic Model

   In this model, the individual dynamics of pedestrians is modeled as:
   
   
   

3. Gernalized Social Force Model

   

Estimation of Interaction Forces in Crowds

Estimating the interaction forces in the a crowd is a daunting task because of the occlusion and clutter. The holistic approach of particle advection provides an alternative way to compute these forces.

1. Particle Advection

   
   (c) An example of the particle advection (Left). The computed social forces for the particles yellow arrows (Right).
   
   The Motion of individuals in a dense crowd resembles the gradual motion of particles in a fluid. The optical flow in a crowd scene represents the flow of pedestrians.
   
   
   • The motion of crowd is captured by moving a grid of particles using the optical flow (O).
   • In a crowd, people tend to move with the average flow therefore, the spatio-temporal average of optical flow (Oave) is used for particle advection.
     
     
   
   (d) The optical flow and interaction forces between a gentle man walking across a crowd: The computed interaction forces (red), the optical flow (yellow)
   
   

2. Computing Social Forces

   Regarding particles as individuals in the crowd, Social Force Model can be adapted for particles:
   
   

Event Detection

The value of the interaction forces are not enough to understand the dynamics of the crowd.

• The pattern of forces in a scene represent different behaviors in the crowd.
• Force flow: The interaction forces in a scene is computed by mapping the magnitude of forces of particles to the frame plain.
  
  

(e) The flow of interaction forces in the video frame and the definition of visual words

1. Abnormal Event Detection

   LDA: Thresholding the Likelihood of a clip to distinguish normal and abnormal set of frame.
   
   
   
   • The location of abnormal events are highlighted in the areas of high interaction forces in the abnormal clips.
     
     

Experiments

1. UMN Dataset

   • Crowd Escape Panic, 11 Videos, 3 Scenes, Videos: a normal starting section and an abnormal ending section
   • Trained on normal section of 5 videos of scene 1, Tested on all
     
     

(e) The ROC on UMN dataset


(f) Abnormal Behavior Detection on UMN Dataset



2. Experiments on Web Dataset

   • 8 Videos of real-life Escape panic, clash, fight
   • 12 Videos of normal pedestrians
   • Trained and tested in a 2-fold validation manner
     
     

(g) Web dataset example results


(h) The ROC on Web dataset

Downloads

1. Web Dataset: Abnormal/Normal Crowds
2. Extra crowd videos

Links

1. UMN Dataset: Unusual Crowd Activity
2. Thought Equity Stock Footage Videos: Crowd Panic, Normal Crowd (New York).

Related Publication

Ramin Mehran, Alexis Oyama, and Mubarak Shah, Abnormal Crowd Behavior Detection using Social Force Model, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, 2009


Back to Crowd Analysis Projects

---

© Copyright 2011 CRCV
University of Central Florida
Orlando, Florida 32816-2365
Phone: (407) 823-1119 | Fax: (407) 823-0594