Reducing the environmental impact of cloud computing requires efficient workload distribution across geographically dispersed Data Center Clusters (DCCs). In this paper, we introduce Green-DCC, which proposes Reinforcement Learning-based hierarchical controller techniques to dynamically optimize temporal and geographical workload distribution between data centers that belong to the same DCC. The environment models non-uniform external weather, carbon intensity, computing resources, cooling capabilities, and dynamic bandwidth costs, which provide constraints and interdependencies. We adapted and evaluated various reinforcement learning approaches, comparing their aggregate carbon emissions across the DCC, demonstrating Green-DCC's effectiveness for controlling and testing advanced data center control algorithms for sustainability.