Recognizing human interaction is an important factor of distinguishing human’s feeling among multiple person’s dealing and promote major role in society as public dealing, violence detection, robot’s perception and virtual entertainments in today’s world. In this paper, we propose novel hybrid spatio-temporal features for human interaction recognition (HIR) system that recognizes human-human interaction from sequences of digital images. The proposed HIR system segments human silhouettes using neighboring data points observation and extracts co-occurring robust spatio-temporal features having full body and key body points techniques. These features technically deal with local descriptions, point-to-point distances, time based associations, and intensity measurements. Then, artificial neural network is used to measure six basic interactions taken from UT-Interaction dataset.