In digital archaeology, a large research area is concerned with the computer-aided analysis of 3D captured ancient pottery objects. A key aspect thereby is the analysis of motifs and patterns that were painted on these objects' surfaces. In particular, the automatic identification and segmentation of repetitive patterns is an important task serving different applications such as documentation, analysis and retrieval. Such patterns typically contain distinctive geometric features and often appear in repetitive ornaments or friezes, thus exhibiting a significant amount of symmetry and structure. At the same time, they can occur at varying sizes, orientations and irregular placements, posing a particular challenge for the detection of similarities. A key prerequisite to develop and evaluate new detection approaches for such repetitive patterns is the availability of an expressive dataset of 3D models, defining ground truth sets of similar patterns occurring on their surfaces. Unfortunately, such a dataset has not been available so far for this particular problem. We present an annotated dataset of 82 different 3D models of painted ancient Peruvian vessels, exhibiting different levels of repetitiveness in their surface patterns. To serve the evaluation of detection techniques of similar patterns, our dataset was labeled by archaeologists who identified clearly definable pattern classes. Those given, we manually annotated their respective occurrences on the mesh surfaces. Along with the data, we introduce an evaluation benchmark that can rank different recognition techniques for repetitive patterns based on the mean average precision of correctly segmented 3D mesh faces. An evaluation of different incremental sampling-based detection approaches, as well as a domain specific technique, demonstrates the applicability of our benchmark. With this benchmark we especially want to address the geometry processing community, and expect it will induce novel approaches for pattern analysis based on geometric reasoning like 2D shape and symmetry analysis. This can enable novel research approaches in the Digital Humanities and related fields, based on digitized 3D Cultural Heritage artifacts. Alongside the source code for our evaluation scripts we provide our annotation tools for the public to extend the benchmark and further increase its variety.