Black phosphorus (BP) has gained renewed attention due to its singular anisotropic electronic and optical properties that might be exploited for a wide range of technological applications. In this respect, the thermal properties are particularly important both to predict its room temperature operation and to determine its thermoelectric potential. From this point of view, one of the most spectacular and poorly understood phenomena is indeed the BP temperature-induced band gap opening; when temperature is increased, the fundamental band gap increases instead of decreases. This anomalous thermal dependence has also been observed recently in its monolayer counterpart. In this work, based on ab initio calculations, we present an explanation for this long known and yet not fully explained effect. We show that it arises from a combination of harmonic and lattice thermal expansion contributions, which are in fact highly interwined. We clearly narrow down the mechanisms that cause this gap opening by identifying the peculiar atomic vibrations that drive the anomaly. The final picture we give explains both the BP anomalous band gap opening and the frequency increase with increasing volume (tension effect).