Tracking of emotional states is very important for building intelligent systems, to have an efficient human-machine interaction and have a better understanding of human behavior. Nevertheless, most of the state-of-the-art works in emotion recognition employ complex algorithms, which are difficult to implement in real-time on devices with low computational resources. This work emphasizes on fast and effective techniques in order to efficiently recognize facial expressions. In this regard, we propose a fast face recognition model based on Local Binary Pattern operator and a straightforward Convolutional Neural Network for emotion classification. The results of our recognition system show an accuracy of 66.5% in the FER2013 wild dataset, which is near to state-of-the-art techniques but uses by far less computational resources.