We study the spatio-temporal formation and spreading of the low-spin state (LS) during the thermal spin transition and the cooperative relaxation of the photo-induced metastable high spin (HS) state at low temperature, in the presence of a structural defect. The model is made of a two-dimensional rectangular-shaped lattice with discrete spins coupled by springs. The investigations are performed for a perfect lattice and a lattice with a hole (simulating the defect) with a fixed size. We found that the presence of the defect affects the thermal equilibrium by reducing the size of the thermal hysteresis at the transition, although the transition temperature remains unchanged. The study of the low-temperature relaxation of the defect-free lattice from HS to LS state indicated the existence of three different regimes of the growth process: (i) a first regime of growth from one corner of the rectangle along the width, then followed by (ii) a second regime of longitudinal propagation at almost constant velocity, and (iii) a third rapid regime when the system feels the surface or the border of the crystal. When a hole is injected inside the lattice, it results in (i) the deformation of the HS/LS interface’s shape when it approaches the defect position; and (ii) the slowing down of its propagation velocity. These results, which are in good agreement with available experimental data, are discussed in terms of elastic energy stored in the system during the relaxation process.