Evaluating the physical defects of green coffee beans are an important process in defining their quality. This evaluation is normally carried out by visual inspection or using traditional instruments which have some limitations. This work is focused on the implementation of a computer vision system combining a hardware prototype and a software module. The hardware was developed to capture the images of coffee beans, the software uses a White-Patch algorithm as a image enhancement procedure, color histograms as feature extractor and SVM for the classification task, a database of 1930 images was collected, we used 13 categories of defects described in the SCAA standard of evaluation. Results of classification achieved a 98.8% of overall detection accuracy, therefore the proposed system proved to be effective in classifying physical defects of green coffee beans. Finally a set of conclusions and future works are presented.