Time series forecasting with multi-headed attention-based deep learning for residential energy consumption

Seok Jun Bu, Sung Bae Cho

Research output: Contribution to journalArticlepeer-review

29 Citations (Scopus)


Predicting residential energy consumption is tantamount to forecasting a multivariate time series. A specific window for several sensor signals can induce various features extracted to forecast the energy consumption by using a prediction model. However, it is still a challenging task because of irregular patterns inside including hidden correlations between power attributes. In order to extract the complicated irregular energy patterns and selectively learn the spatiotemporal features to reduce the translational variance between energy attributes, we propose a deep learning model based on the multi-headed attention with the convolutional recurrent neural network. It exploits the attention scores calculated with softmax and dot product operation in the network to model the transient and impulsive nature of energy demand. Experiments with the dataset of University of California, Irvine (UCI) household electric power consumption consisting of a total 2,075,259 time-series show that the proposed model reduces the prediction error by 31.01% compared to the state-of-the-art deep learning model. Especially, the multi-headed attention improves the prediction performance even more by up to 27.91% than the single-attention.

Original languageEnglish
Article number4722
Issue number18
Publication statusPublished - 2020 Sept

Bibliographical note

Funding Information:
Meatnhwrohugilhea, gegvreengatsti otnhaptr aacrtieceosuwtshidiceh stmhoe oltohnogu-tttehremeffbeechtoafvsiuocrh cmains -pbree dciocntisoind,earneddt reaas tpitowssitihbilne events happeningand failure tocorrectly predicting them within accuracy, whichthe rest ofthe prediction profiles present, can beattributed as outlier points. Asa future work, wewilltake care ofthis issue through aggregationpractices which smooth out theeffectofsuch mis-prediction, andtreat it within the tolerable error by the provided flexibilities of the hybrid approach. Funding: This work was partly supported by the Institute of Information and Communications Technology Planning and Evaluation (IITP) grant funded by the Korean government (MSIT) (No. 2020-0-01361, Author CAorntitfriciibaulIntitoenllsig:enCceonGrcaedputautealSiczhaotioolPnr,ogSra.-mB.(CY.o; nsFeoirUmnivale rsaitnya)l)yasnidst, heSK.-oJ.rBe.a;ElFeuctnridciPnogw eracCqouripsiotriatoionn, S.-B.C.; Investigation, S.-J.B.; Methodology, S.-J.B. and S.-B.C.; Supervision, S.-B.C.; Visualization, S.-J.B.; Writing— review & editing, S.-B.C. All authors have read and agreed to the published version of the manuscript.

Publisher Copyright:
© 2020 by the authors. Licensee MDPI, Basel, Switzerland.

All Science Journal Classification (ASJC) codes

  • Renewable Energy, Sustainability and the Environment
  • Energy Engineering and Power Technology
  • Energy (miscellaneous)
  • Control and Optimization
  • Electrical and Electronic Engineering


Dive into the research topics of 'Time series forecasting with multi-headed attention-based deep learning for residential energy consumption'. Together they form a unique fingerprint.

Cite this