Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications

Jiajia Liu, Zhenjiang Shi, Shangwei Zhang, Nei Kato

Research output: Contribution to journalArticlepeer-review

Abstract

The explosive growth of machine-type communications (MTC) devices poses critical challenges to the existing cellular networks. Therefore, how to support massive MTC devices with limited resources is an urgent problem to be solved. Bursty traffic is an important characteristic of MTC devices, which makes it difficult for agents to learn useful experience and has a negative impact on model convergence. However, most existing reinforcement learning-based literatures assume that devices have saturate data. Towards this end, we propose two distributed Q-learning aided uplink grant-free non-orthogonal multiple access (NOMA) schemes (including all-devices distributed Q-learning (ADDQ) scheme and portion-devices distributed Q-learning (PDDQ) scheme) to maximize the number of accessible devices, where the bursty traffic of massive MTC devices is carefully considered. In order to reduce the dimension of scheduling space and mitigate the impact of bursty traffic, the idea of grouping devices as well as transmission resources and the intermittent learning mode are adopted in our schemes. Extensive numerical results demonstrate the advantages of proposed schemes from multiple perspectives.

Original languageEnglish
Article number9427159
Pages (from-to)2029-2041
Number of pages13
JournalIEEE Journal on Selected Areas in Communications
Volume39
Issue number7
DOIs
Publication statusPublished - 2021 Jul

Keywords

  • Bursty traffic
  • distributed Q-learning
  • grant-free
  • machine-type communications
  • non-orthogonal multiple access

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications'. Together they form a unique fingerprint.

Cite this