Recent theories suggest that low frequency dynamic intramolecular and intermolecular motions in organic semiconductors (OSCs) are critical to determining the hole mobility. So far, however, it has not been possible to probe these motions directly experimentally and therefore no unequivocal and quantitative link exists between molecular-scale thermal disorder and macroscale hole mobility in OSCs. Here we use inelastic neutron scattering to probe thermal disorder directly by measuring the phonon spectrum in six different small molecule OSCs, which we accurately reproduce with first principles simulations. We use the simulated phonons to generate a set of electron-phonon coupling parameters. Using these parameters, the theoretical mobility is in excellent agreement with macroscopic measurements. Comparison of mobility between different materials reveals routes to improve mobility by engineering phonon and electron-phonon coupling.