The structural and magnetic properties of a series of ordered double perovskites with the formula Sr2Co1-xMgxTeO6 (x = 0.1, 0.2 and 0.5) are investigated by X-ray diffraction, low temperature neutron diffraction, electron paramagnetic resonance and magnetic susceptibility. The progressive substitution of the paramagnetic Co2+ high spin ion by the diamagnetic Mg2+, of about the same size, induces changes in the room temperature crystal structure, from a distorted P21/n phase for the undoped Sr2CoTeO6 oxide to the I4/m of the end member (Sr2MgTeO6). These perovskites experience structural transitions on heating, the temperature at which the transitions occur being smaller as x increases. The novel approach of mode-crystallography is used for the analysis. All oxides show antiferromagnetic exchange interactions between Co2+ ions but the long range antiferromagnetic order is not achieved for the phase with x = 0.5. The low temperature neutron diffraction data have been evaluated using a full symmetry analysis. Results are consistent with an unquenched orbital contribution of a high spin Co2+ ion.