Decoherence in quantum bits (qubits) is a major challenge for realizing scalable quantum computing. One of the primary causes of decoherence in qubits and quantum circuits based on superconducting Josephson junctions is the critical current fluctuation. Many efforts have been devoted to suppressing the critical current fluctuation in Josephson junctions. Nonetheless, the efforts have been hindered by the defect-induced trapping states in oxide-based tunnel barriers and the interfaces with superconductors in the traditional Josephson junctions. Motivated by this, along with the recent demonstration of 2D insulator h-BN with exceptional crystallinity and low defect density, we fabricated a vertical NbSe2/h-BN/Nb Josephson junction consisting of a bottom NbSe2 superconductor thin layer and a top Nb superconductor spaced by an atomically thin h-BN layer. We further characterized the superconducting current and voltage (I-V) relationships and Fraunhofer pattern of the NbSe2/h-BN/Nb junction. Notably, we demonstrated the critical current noise (1/f noise power) in the h-BN-based Josephson device is at least a factor of four lower than that of the previously studied aluminum oxide-based Josephson junctions. Our work offers a strong promise of h-BN as a novel tunnel barrier for high-quality Josephson junctions and qubit applications.