[Background] The performance of emotion recognition systems depends heavily on datasets used in their training, validation, or testing stages. [Aims] This research aims to evaluate the extent to which public available physiological datasets created for emotion recognition systems meet a set of reference requirements. [Method] Firstly, we analyze the applicability of some reference requirements proposed for stress datasets and adjust the corresponding evaluation criteria. Secondly, nine public physiological datasets were identified from a previous survey. [Results] None of the evaluated datasets satisfy all the reference requirements in order to be considered as a reference dataset for being used in the construction of reliable emotion recognition systems. [Conclusion] Although the evaluated datasets do not support the whole reference requirements, they provide a baseline for further development. Also, a greater effort is needed to establish specific reference requirements that can appropriately guide the creation of physiological datasets for emotion recognition systems.