Multi-agent event recognition by preservation of spatiotemporal relationships between probabilistic models
Introduction
The goal of this paper is to model and recognize multi-agent activity that consists of smaller sub-activities taking place in a structured manner. The specific
application that we have targeted is the recognition of American football plays. We perform activity modeling using simple optical flow computed on clips of
football plays, circumventing the need for difficult tracking in such a scenario. We build a comprehensive activity representation that encodes not only all the
events taking place within the activity but also their spatial and temporal inter-relationships, using graphical modeling over probabilistic sub-activity
representations.
Problem Description
Our sub-activity representation is a motion pattern (ref: Saleemi, CVPR 2010) or a Gaussian Mixture Model over optical flow data. We also determine the spatial and
temporal relationships between sub-activities and collectively model one single activity as a complete graph. The nodes of this graph are the sub-activities that
make up one activity and the edges are their spatial and temporal relationships. Please see the paper for complete details. The figure below provides examples of
our motion representation versus hand drawn schematic diagrams of corresponding plays.
Results
Our method manages to out-perform the state of the art methods on this particular problem easily. We test the method using subsets of the cues we use in the paper.
The results for these experiments are given below. For complete results and comparisons please see the paper.
Presentation
Slides
Click Here
Video
Related Publication
Salman Khokhar, Imran Saleemi, and Mubarak Shah,
Multi-agent event recognition by preservation of spatiotemporal relationships between probabilistic models,
Image and Vision Computing, Volume 31, Issue 9, Pages 603-615, September 2013.
Back to Motion Patterns Projects
