The present work facilitates the monitoring of the nutritional composition of the cultivation soil by identifying nutrient deficiencies through image recognition of banana leaves using a convolutional neural network trained with transfer learning and fine tuning. An original dataset of photos was used in this research, which is composed of healthy banana leaves images, and leaves with known deficiencies of nitrogen, potassium, and phosphorus. Subsequently, an augmentation is performed to this dataset through linear transformations and the resulting images were pre-processed in different color spaces to be used as inputs to the neural network. It was possible to obtain a model with high precision that could be validated through different metrics. Finally, a prototype of a web platform was developed so that the system could be accessed by farmers.