The automatic characterization and classification of plant species is an important task for plant taxonomists. On this work, we propose the use of well-known pre-trained Deep Convolutional Neural Networks (DCNN) for the characterization of plants based on their leaf midrib. The samples studied are microscope images of leaf midrib cross-sections taken from different specimens under varying conditions. Results with traditional handcrafted image descriptors demonstrate the difficulty to effectively characterize these samples. Our proposal is to use DCNN as a feature extractor through Global Average Pooling (GAP) over the raw output of its last convolutional layers without the application of summarizing functions such as ReLU and local poolings. Results indicate considerably performance improvements over previous approaches under different scenarios, varying the image color-space (gray-level or RGB) and the classifier (KNN or LDA). The highest result is achieved by the deeper network analyzed, ResNet (101 layers deep), using the LDA classifier, with 99.20% of accuracy rate. However, shallower networks such as AlexNet also provide good classification results (97.36%), which is still a significant improvement over the best previous result (83.67% of combined fractal descriptors).