A Multi-Agent Deep Reinforcement Learning based Spectrum Allocation Framework for D2D Underlay Communications
share
Device-to-device (D2D) underlay communication improves spectrum efficiency but causes severe interference, which requires an effective resource management solution without requiring global information. In this paper, a distributed spectrum allocation framework based on multi-agent deep reinforcement learning is proposed, named Neighbor-Agent Actor Critic (NAAC), to improve the D2D sum rate, while ensuring the quality of transmission of cellular users (CUEs). Each D2D pair is supported by an agent, which automatically selects a reasonable spectrum for transmission. For full cooperation between users, the proposed framework uses neighbor users' information for centralized training but is executed distributedly without that information, which not only ensures the convergence of the training but also improves the system performance. The simulation results show that the proposed method can effectively guarantee the transmission quality of the CUEs and improve the sum rate of D2D links, as well as have better convergence.
READ FULL TEXT