Gamification has been applied in software engineering contexts, and more recently in requirements engineering with the purpose of improving the motivation and engagement of people performing specific engineering tasks. But often an objective evaluation that the resulting gamified tasks successfully meet the intended goal is missing. On the other hand, current practices in designing gamified processes seem to rest on a try, test and learn approach, rather than on first principles design methods. Thus empirical evaluation should play an even more important role.We combined gamification and automated reasoning techniques to support collaborative requirements prioritization in software evolution. A first prototype has been evaluated in the context of three industrial use cases. To further investigate the impact of specific game elements, namely point-based elements, we performed a quasi-experiment comparing two versions of the tool, with and without pointsification. We present the results from these two empirical evaluations, and discuss lessons learned.