Video Action Understanding: Advancing Action Recognition, Temporal Localization and Detection

Dissertation

Video action understanding involves comprehending human actions in videos, a challenging task for computer vision systems due to inter-class variations and complex scenes. Central to this task are four fundamental questions: What, When, Where, and Who. This dissertation addresses these by proposing methods for multi-view action classification (What?), modeling relationships for precise temporal localization (When?) of actions, introducing Gabriella for real-time action detection (Where?) in security videos, and developing joint action and actor recognition (Who?) for robotic care.

Enhanced multi-view action classification significantly advances surveillance and sports analysis by integrating multiple camera perspectives for a comprehensive understanding of activities. Precise temporal localization within videos enhances video analytics and content moderation by accurately identifying specific moments of interest. Gabriella, an advanced video surveillance system, excels in monitoring large fields-of-view and detect diverse activities, enabling detection of multiple subjects and recognition of the activities performed with high precision. Joint action and actor recognition capabilities empowers robots to provide personalized care, thereby improving the quality of life for users through tailored support and interactions.