The non-polymorphic nature of CD1 proteins creates a situation in which T cells with invariant T cell receptors (TCRs), like CD1d-specific NKT cells, are present in all humans. CD1b is an abundant protein on human dendritic cells that presents M. tuberculosis (Mtb) lipid antigens to T cells. Analysis of T cell clones suggested that semi-invariant TCRs exist in the CD1b system, but their prevalence in humans is not known. Here we used CD1b tetramers loaded with mycolic acid or glucose monomycolate to study polyclonal T cells from 150 Peruvian subjects. We found that CD1b tetramers loaded with mycolic acid or glucose monomycolate antigens stained TRAV1-2+ GEM T cells or TRBV4-1+ LDN5-like T cells in the majority of subjects tested, at rates ~10-fold lower than NKT cells. Thus, GEM T cells and LDN5-like T cells are a normal part of the human immune system. Unlike prior studies measuring MHC- or CD1b-mediated activation, this large-scale tetramer study found no significant differences in rates of CD1b tetramer-mycobacterial lipid staining of T cells among subjects with Mtb exposure, latent Mtb infection or active tuberculosis (TB) disease. In all subjects, including “uninfected” subjects, CD1b tetramer+ T cells expressed memory markers at high levels. However, among controls with lower mycobacterial antigen exposure in Boston, we found significantly lower frequencies of T cells staining with CD1b tetramers loaded with mycobacterial lipids. These data link CD1b-specific T cell detection to mycobacterial exposure, but not TB disease status, which potentially explains differences in outcomes among CD1-based clinical studies, which used control subjects with low Mtb exposure.