Human Activity Recognition (HAR) is a time series categorization challenge that requires data from a number of timesteps in order to correctly classify the activities that are carried out. In recent times, the usage of image datasets for activity recognition has increased, however good classification cannot be done with just one frame. To increase recognition accuracy, multiple frames of data and the context of environment are required. It is known that a video is made up of a number of still images (frames) that are quickly updated to create the illusion of motion. The hybrid models of Deep Learning (DL) algorithms like Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) are proposed for recognising the human activity from video dataset. The hybrid models, Convolutional Long Short-Term Memory (ConvLSTM) and Long-term Recurrent Convolutional Network (LRCN) are introduced to improve the accuracy of HAR on video dataset. The models will be evaluated on standard video datasets, and the outcomes will show how HAR has the potential to significantly influence a number of industries. It has many applications in the fields of security, sports, and healthcare.