This paper addresses the issues of 3D human pose estimation, tracking and recognition from RGB-D video sequences using a generative structured framework. Most existing approaches focus on these issues using discriminative models. However, a discriminative model has certain drawbacks: a) it requires expensive training steps and large amount of training samples for covering inherently wide pose space, and (b) not suitable for real-time applications due to its slow algorithmic inferences. In this work, a real-time tracking system has been proposed for human pose recognition utilizing ridge body parts features. Initially, depth silhouettes extract ridge data inside the binary edges and initialize each body joints information using predefined pose. Then, body parts tracking incorporates appearance learning to handle occlusions and manage body joints features. Lastly, Support Vector Machine is used to recognize different poses. Experimental results demonstrate that the proposed method is reliable and efficient in recognizing human poses at different realistic scenes.