3D Motion Data Mining Multimedia Project Multimedia and Network Lab, Department of Computer Science Introduction 3D Motion Capture 2UTD Multimedia and Networking Lab2/19/2016 Integration Gait Analysis 3D Motion Capture Motion correlation and error modeling Humanoid robotics Game Control Disease diagnostic Motion Clacification 3UTD Multimedia and Networking Lab2/19/2016 Gait? Study of Human Walk (Lower Limbs) Terminology Gait cycle: Begins when one foot contacts the ground and ends when that foot contacts the ground again 4UTD Multimedia and Networking Lab2/19/2016 3D Input Data (MoCap & EMG) 3D Mocap data M x 54 Matrix ( M is the total num of Frames ) 5UTD Multimedia and Networking Lab2/19/2016 EMG data Input Data Format Each Motion is represented by set of joint vectors Use sliding Windows for feature extractions TibiaFootToe Windows (Time Frame) 6UTD Multimedia and Networking Lab2/19/2016 Motion Data Analysis Flow 7 Motion capture Data Collection Preprocessing Feature Extraction Data Analysis Gait Cycle Cross-Pair 24 Feature Point Geometric Trans. Project I Content-Based Indexing(HS Grid). Project II Motion Classification using PMK Project III Motion Animation using 3D Human Locomotion Data. 8UTD Multimedia and Networking Lab2/19/2016 Project I: Content-Based Indexing WSVD IAV MoCapEMG FCMFCM Reduced dimensional vector where, c - pre-determined number of clusters M Total number of overall windows center - center/median points for all c clusters in k-d space U - degree of membership for each M points with respect to each cluster. + Input Data Hierarchical Spatial Grid(HSG) 10UTD Multimedia and Networking Lab2/19/2016 HSG (2D) 1 2 3 Even distribution transformation Transforming the points into corresponding rank index on each dimension rank index (A(i))= r where, array A with non-zero values, non-increasing ordered array A_RI, A_RI(r)=A(i) Similarity search Given a query Q( using IAV, WSVD, and FCM) Q [q j,min, q j,max] Where Distance Project Goal Goal: Building Disk base Input/output functions for Hierarchical Spatial Grid Indexing Structure Input: Spatial data(2D,3D,) Input query : Spatial data(2D,3D,) Output: related Spatial data(2D,3D,) Requirement: Disk based input/output Indexing Structure Language option: C or JAVA (prefer C) Reference : G. Pradhan and B. Prabhakaran, " Indexing 3D Human Motion Repositories For Content-based Retrieval," IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 13, NO. 5,, SEPTEMBER 2009 14UTD Multimedia and Networking Lab2/19/2016 Project II: Pyramid Matching Kernel Place multi-dimensional, multi-resolution grid over point sets Consider points matched at finest resolution where they fall into same grid cell Approximate similarity between matched points with worst case similarity at given level No explicit search for matches! Pyramid match kernel measures similarity of a partial matching between two sets: Project II: Pyramid Matching Kernel Pyramid Match Kernel? measures similarity of a partial matching between two sets Place multi-dimensional, multi-resolution grid over point sets Consider points matched at finest resolution where they fall into same grid cell Approximate similarity between matched points with worst case similarity at given level 2/19/2016UTD Multimedia and Networking Lab16 Pyramid match 2/19/2016UTD Multimedia and Networking Lab17 PMK : Approximate partial match similarity 2/19/2016UTD Multimedia and Networking Lab18 Number of newly matched pairs at level i Measure of difficulty of a match at level i PMK Weights inversely proportional to bin size Normalize kernel values to avoid favoring large sets measure of difficulty of a match at level i histogram pyramids number of newly matched pairs at level i Project II (Cont.) Goal: Motion Classification using Pyramid Matching Kernel (PMK) for fast matching motion features. Feature Extraction 3D Geometric feature extractions(Detail points are provided Later) Classification: HMM or GMM Programming Language: C, JAVA, or Matlab Reference : K. Grauman and T. Darrell. The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features, In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Beijing, China, October C++ code:20UTD Multimedia and Networking Lab2/19/2016 Project III Goal: Motion animation Tool with selected frames. e.g., 21UTD Multimedia and Networking Lab2/19/2016 Question? Thank You ! 22UTD Multimedia and Networking Lab2/19/2016