Network load balancers (LBs) are one of the key components in data centers (DCs). They distribute workloads across multiple servers and help offer scalable services. However, operating in dynamic network environments with limited observations, modern LBs rely on heuristic algorithms and require manual configurations for fairness optimization. As reinforcement learning (RL) helps achieve performance gains in dynamic systems, this paper proposes a distributed asynchronous RL mechanism to improve LBs’ workload distribution fairness with limited observations. The performance of proposed mechanism is evaluated and compared with state-of-the-art LB algorithms in a simulator, under configurations with progressively increasing difficulties. Preliminary results show promise in RL-based LB algorithms, and cast light on more challenges for future research, including reward function design and model scalability.