This paper presents spatiotemporal hybrid features, human tracking, and activity recognition into a single framework from video sequences captured by a RGB-D sensor. Initially, we received a sequence of depth maps to extract human silhouettes from the noisy background and track them using temporal human motion information from each frame. Then, hybrid features as optical flow motion features and distance parameters features are extracted from the depth silhouette region and used in an augmented form to work as spatiotemporal features. In order to represent each activity in a better way, the augmented features are being clustered and symbolized by self-organization maps. Finally, these features are then processed by hidden Markov models to train and recognize human activities based on transition and emission probabilities values. The experimental results show the superiority of the proposed method over the state-of-the-art methods using two challenging depth images datasets.