Real-time electricity prices and volatile workload patterns have a major impact on the operational costs of distributed data centers. A smart grid enables large consumers to regulate their usage patterns with the knowledge of real-time prices at different locations, and their energy consumption through the demand response (DR) program. However, the price inclined workload assignment may lead to an increase in peak demand, and decrease the benefits of workload offloading. Moreover, energy-efficient service management is challenged by unpredictable network conditions that can cause service performance degradation. Using an actor-critic approach, we developed a system that distributes a service's load among distributed sites to take advantage of the spatial variations in energy pricing. Our experimental results on the CloudLab testbed have proved with 45% reduction in running costs without impacting the response time using the Wikipedia request traces considering peak-based pricing.