Since the device performance depends on the structure of the organic semiconductor blend that makes up the active layer, understanding structure-function relationships is crucial for the development of next-generation polymer solar cells. Through a combination of grazing incidence X-ray diffraction and X-ray rocking scans, we have quantified the crystallinity of poly(3-hexylthiophene) (P3HT) and [6,6]-phenyl-C61-butyric acid methyl ester (PCBM) in the active layer of bulk heterojunction polymer solar cells. Contrary to common belief that the device JSC is only dictated by P3HT crystallinity, our study - varied over a wide range of processing conditions - shows that the device JJC is limited, to first order, by the PCBM crystallinity. However, when the PCBM crystallinity is greater than 50% of the maximum crystallinity achievable in our samples through thermal annealing, the extent of P3HT out-of plane p-stacking is directly correlated with the device JSC. By quantifying the relationship between the crystallinity of both organic semiconductors of the active layer and device performance, we have developed a description of the complex interplay between structure and performance in polymer solar cells.