The recognition of hand gestures is a very interesting research topic due to the growing demand in recent years in robotics, virtual reality, autonomous driving systems, human-machine interfaces and in other new technologies. Despite several approaches for a robust recognition system, gesture recognition based on visual perception has many advantages over devices such as sensors, or electronic gloves. This paper describes the implementation of a visual-based recognition system on a embedded computer for 10 hand poses recognition. Hand detection is achieved using a tracking algorithm and classification by a light convolutional neural network. Results show an accuracy of 94.50%, a low power consumption and a near real-time response. Thereby, the proposed system could be applied in a large range of applications, from robotics to entertainment.