Ablation Study of Deep Reinforcement Learning Congestion Control in Cellular Network Settings

The application of deep reinforcement learning for congestion control (DRL-CC) is promising. It improves the learnability of congestion control to adapt with the changes of networking condition. However, it inherits the challenges posed by the machine learning approach such as generalization, huge action space, and interpretability. Understanding the mechanics of DRL-CC is the initial step to find the appropriate solution. Hence, this paper presents an ablation study to DRL-CC, specifically in the cellular network settings. It shows that clipping and scaling in training process contribute positively to networking performance, changing the initialization method may improve the performance, and higher reward achievement may not directly correlated to better networking performance.