Two-particle pseudorapidity correlations are measured in sNN=2.76TeVPb+Pb, sNN=5.02TeVp+Pb, and s=13TeVpp collisions at the Large Hadron Collider (LHC), with total integrated luminosities of approximately 7μb-1, 28 nb-1, and 65 nb-1, respectively. The correlation function CN(η1,η2) is measured as a function of event multiplicity using charged particles in the pseudorapidity range |η|<2.4. The correlation function contains a significant short-range component, which is estimated and subtracted. After removal of the short-range component, the shape of the correlation function is described approximately by 1+(a12)1/2η1η2 in all collision systems over the full multiplicity range. The values of (a12)1/2 are consistent for the opposite-charge pairs and same-charge pairs, and for the three collision systems at similar multiplicity. The values of (a12)1/2 and the magnitude of the short-range component both follow a power-law dependence on the event multiplicity. The short-range component in p + Pb collisions, after symmetrizing the proton and lead directions, is found to be smaller at a given η than in pp collisions with comparable multiplicity.