Controlling Action Space of Reinforcement-Learning-Based Energy Management in Batteryless Applications

Junick Ahn, Daeyong Kim, Rhan Ha, Hojung Cha

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Duty cycle management is critical for the energy-neutral operation of batteryless devices. Many efforts have been made to develop an effective duty cycling method, including machine-learning-based approaches, but existing methods can barely handle the dynamic harvesting environments of batteryless devices. Specifically, most machine-learning-based methods require the harvesting patterns to be collected in advance, as well as manual configuration of the duty-cycle boundaries. In this article, we propose a configuration-free duty cycling scheme for batteryless devices, called CTRL, with which energy harvesting nodes tune the duty cycle themselves adapting to the surrounding environment without user intervention. This approach combines reinforcement learning (RL) with a control system to allow the learning algorithm to explore all possible search space automatically. The learning algorithm sets the target State of Charge (SoC) of the energy storage, instead of explicitly setting the target task frequency at a given time. The control system then satisfies the target SoC by controlling the duty cycle. An evaluation based on the real implementation of the system using publicly available trace data shows that CTRL outperforms state-of-the-art approaches, resulting in 40% less frequent power failures in energy-scarce environments while achieving more than ten times the task frequency in energy-rich environments.

Original languageEnglish
Pages (from-to)9928-9941
Number of pages14
JournalIEEE Internet of Things Journal
Volume10
Issue number11
DOIs
Publication statusPublished - 2023 Jun 1

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Controlling Action Space of Reinforcement-Learning-Based Energy Management in Batteryless Applications'. Together they form a unique fingerprint.

Cite this