DDPG-Based Load-Aware QoS Guaranteed SDN Controller Placement for Internet of Vehicles

Networks in the 5G and beyond era can use software-defined networks (SDNs) to achieve network slicing (NS), so as to meet the extremely diverse service requirements of diverse applications in the Internet of Vehicles (IoV). However, the flow fluctuations in the highly dynamic IoV make it difficult to provide reliable, flexible, and scalable services for the IoV by the SDN control plane. Careful SDN controller placement can be a feasible solution to achieve its robustness and flexibility to deal with the changes in network status. Thus, this article studies a dynamic controller placement problem (CPP) to improve the performance of IoV services. To be specific, a hierarchical SDN control plane for the IoV is considered, with the SDN controllers placed at the edge of networks. Under this architecture, we model the dynamic controller placement by Markov decision process (MDP). To efficiently solve the formulated NP-hard problems, we develop an algorithm based on deep deterministic policy gradient (DDPG) because of its advantages in solving problem with multidimensional action and a large solution space. Furthermore, we incorporate a random process into the action selection strategy of DDPG to prevent it from getting trapped in a local optimum. Simulation results show that the proposed DDPG-based controller placement approach can adapt to a highly dynamic IoV environment with outstanding performance.