Multi-agent Event Recognition by Preservation of Spatiotemporal Relationships Between Probabilistic Models
| Paper: | Multi-agent event recognition by preservation of spatiotemporal relationships between probabilistic models |
| Contact: | S. Khokhar, I. Saleemi, M. Shah |
The goal of this paper is to model and recognize multi-agent activity that consists of smaller sub-activities taking place in a structured manner. The specific application that we have targeted is the recognition of American football plays. We perform activity modeling using simple optical flow computed on clips of football plays, circumventing the need for difficult tracking in such a scenario. We build a comprehensive activity representation that encodes not only all the events taking place within the activity but also their spatial and temporal inter-relationships, using graphical modeling over probabilistic sub-activity representations.
Our sub-activity representation is a motion pattern (ref: Saleemi, CVPR 2010) or a Gaussian Mixture Model over optical flow data. We also determine the spatial and temporal relationships between sub-activities and collectively model one single activity as a complete graph. The nodes of this graph are the sub-activities that make up one activity and the edges are their spatial and temporal relationships. Please see the paper for complete details. The figure below provides examples of our motion representation versus hand drawn schematic diagrams of corresponding plays.
Our method manages to out-perform the state of the art methods on this particular problem easily. We test the method using subsets of the cues we use in the paper. The results for these experiments are given below. For complete results and comparisons please see the paper.
Slides
Video
Salman Khokhar, Imran Saleemi, and Mubarak Shah, Multi-agent event recognition by preservation of spatiotemporal relationships between probabilistic models, Image and Vision Computing, Volume 31, Issue 9, Pages 603-615, September 2013.