Avalanche Ski-Resort Snow-Clad Mountain Moving Vistas: Exploiting Motion for Describing Scenes Nitesh Shroff, Pavan Turaga, Rama Chellappa University of Maryland, College Park Problem Definition and Motivation Contributions Dynamic Attributes  Dynamic Attributes  motion information from a global perspective.  Characterize the unconstrained dynamics of scenes using Chaotic Invariants.  Does not require localization or tracking of scene elements.  Unconstrained real world Dynamic Scene dataset.  Dynamic Scene Recognition  Dynamics of scene reveals further information !!  Motion of scene elements improve or deteriorate classification?  How to expand the scope of scene classification to videos? What makes it difficult?  Scenes are unconstrained and ‘in-the-wild’ -- Large variation in scale, view, illumination, background  Underlying physics of motion -- too complicated or very little is understood of them. Ray of hope !!!  Underlying process not entirely random but has deterministic component Can we characterize motion at a global level ??  Yes using dynamic attributes and chaotic invariants Modeling Dynamics  Requires No assumptions  Purely from the sequence of observations.  Fundamental notion -- all variables in a influence one another.  Constructs state variables from given time series  Estimate embedding dimension and delay Chaotic Invariants[2,4] Class LDS[3] (GIST) Bag of Words Mean (GIST) Dynamics (Chaos) Statics+ Dynamics Toranado Waves Chaotic Traffic Whirlpool Total [1] A.Oliva and A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV, 2001 [2] M. Perc. The dynamics of human gait. European journal of physics, 26(3):525–534, 2005 [3] G. Doretto, A. Chiuso, Y. Wu, and S. Soatto. Dynamic textures, IJCV, 2003 [4]S. Ali, A. Basharat, and M. Shah. Chaotic Invariants for Human Action Recognition. ICCV, References Degree of Busyness: Amount of activity in the video.  Highly busy: Sea-waves or Traffic scene --high degree of detailed motion patterns.  Low busyness: Waterfall -- largely unchanging and motion typically in a small portion Degree of Flow Granularity of the structural elements that undergo motion. Degree of Regularity Degree of Busyness  Reconstruct the phase space.  Characterize it using invariants  Lyapunov Exponent: Rate of separation of nearby trajectories.  Correlation Integral: Density of phase space.  Correlation Dimension: Change in the density of phase space  Coarse: falling rocks in a landslide.  Fine: waves in an ocean D egree of Regularity of motion of structural elements.  Irregular or random motion: chaotic traffic  Regular motion: smooth traffic Algorithmic Layout GIST [1] for each frame Each dimension as time series Chaotic Invariants Classification & Learn Attributes  Unconstrained YouTube videos  Large Intra-class variation  Available at Dynamic Scene Dataset Recognition Accuracy Linear Separation using Attributes 18 out of 20 correctly classified WhirlpoolWaves Busyness