AIR UNIVERSITY
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Intelligent Media Center (IMC)

Robust human activity recognition from depth video using spatiotemporal multi-fused features
Duration
[2015 – 2017], Designation = Research Associate
Supported by
This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the “ICT Consilience Creative Program” (IITP-2015-R0346–15-1007) supervised by the IITP (Institute for Information & communications Technology Pro- motion), and was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP)(B0101–15-0552, Development of Predictive Visual Intelligence Technology).
Project Description
The recently developed depth imaging technologies have provided new directions for human activity recognition (HAR) without attaching optical markers or any other motion sensors to human body parts. In this paper, we propose novel multi-fused features for online human activity recognition (HAR) system that recognizes human activities from continuous sequences of depth map. The proposed online HAR system segments human depth silhouettes using temporal human motion information as well as it obtains human skeleton joints using spatiotemporal human body information. Then, it extracts the spatiotemporal multi-fused features that concatenate four skeleton joint features and one body shape feature. Skeleton joint features include the torso-based distance feature (DT), the key joint-based distance feature (DK), the spatiotemporal magnitude feature (M) and the spatiotemporal directional angle feature (θ). The body shape feature called HOG DDS represents the projections of the depth differential silhouettes (DDS) between two consecutive frames onto three orthogonal planes by the histogram of oriented gradients (HOG) format. The size of the proposed spatiotemporal multi fused feature is reduced by a code vector in the codebook which is generated by vector quantization method. Then, it trains the hidden Markov model (HMM) with the code vectors of the multi-fused features and recognizes the segmented human activity by the forward spotting scheme using the trained HMM-based human activity classifiers. The experimental results on three challenging depth video datasets such as IM-Daily-Depth Activity, MSR Action 3D and MSR Daily Activity 3D demonstrate that the proposed online HAR method using the proposed multi-fused features outperforms the state-of-the-art HAR methods in terms of recognition accuracy.
Robust human activity recognition from depth video using spatiotemporal multi-fused features