The luminescence due to rare earth ions in Al and Ga nitride thin layers can be increased by proper structural tailoring. In this contribution, we report on the photoluminescence of the rare earth ion Tb3+ that is contained in films prepared by DC magnetron co-sputtering of Ga or Al targets with additional Tb pellets in a nitrogen atmosphere. These results are typical also for other rare earth ions, such as Eu3+, Sm3+, Tm3+ and Ce3+. Thermal annealing of the samples results in significant improvement of the photoluminescence by factors up to several tens in intensity. Interestingly enough, optimum intensity is achieved with annealing temperatures in a window between approximately 500 and 750 °C in all cases. This finding indicates similar excitation and emission processes based on structural properties of the layers.