Improving Resource Utilization in Data Centers using an LSTM-based Prediction Model

Kundjanasith Thonglek, Kohei Ichikawa, Keichi Takahashi, Hajimu Iida, Chawanat Nakasan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

Data centers are centralized facilities where computing and networking hardware are aggregated to handle large amounts of data and computation. In a data center, computing resources such as CPU and memory are usually managed by a resource manager. The resource manager accepts resource requests from users and allocates resources to their applications. A commonly known problem in resource management is that users often request more resources than their applications actually use. This leads to the degradation of overall resource utilization in a data center. This paper aims to improve resource utilization in data centers by predicting the required resource for each application. We designed and implemented a neural network model based on Long Short-Term Memory (LSTM) to predict more efficient resource allocation for a job based on historical data. Our model has two LSTM layers each of which learns the relationship between: (1) allocation and usage, and (2) CPU and memory. We used Googles cluster-usage trace, which contains a trace of resource allocation and usage for each job executed on a Google data center, to train our neural network. Googles cluster scheduler simulator was used to evaluate our proposed method. Our simulation indicated that the proposed method improved the CPU utilization and memory utilization by 10.71% and 47.36%, respectively, compared to a conventional resource manager. Moreover, we discovered that increasing the memory cell size of our LSTM model improves the accuracy of the prediction in return for longer training time.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728147345
DOIs
Publication statusPublished - 2019 Sept
Externally publishedYes
Event2019 IEEE International Conference on Cluster Computing, CLUSTER 2019 - Albuquerque, United States
Duration: 2019 Sept 232019 Sept 26

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Volume2019-September
ISSN (Print)1552-5244

Conference

Conference2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
Country/TerritoryUnited States
CityAlbuquerque
Period19/9/2319/9/26

Keywords

  • Computing Resources
  • Long Short-Term Memory
  • Resource Management
  • Resource Utilization

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Signal Processing

Fingerprint

Dive into the research topics of 'Improving Resource Utilization in Data Centers using an LSTM-based Prediction Model'. Together they form a unique fingerprint.

Cite this