Implementability improvement of deep reinforcement learning based congestion control in cellular network

The application of deep reinforcement learning to improve the adaptability of congestion control is promising. However, the state-of-the-art method indicates a high packet loss and requires a high CPU to handle a flow. Those hinder the implementation of deep reinforcement learning-based congestion control in the production network. Therefore, we propose modifications in the agent's deployment design, specifically in the monitoring component, interval, and transport protocol to reduce packet loss and CPU usage. Unfortunately, those agent modifications yield a tradeoff in throughput performance. In order to compensate for the tradeoff, we re-train the policy model using ns-3 (network-simulator-3) as a gym environment and custom reward function to improve the throughput. Our work shows that the proposed method evaluated in cellular networks is able to reduce packet loss by up to 50.7×, suppress CPU (central processing unit) usage by up to 4.13×, and increase the throughput by up to 6.94%. We hope our contribution can drive the adoption of deep reinforcement learning-based congestion control to the production network.