Failure Prediction in Datacenters Using Unsupervised Multimodal Anomaly Detection

Minglu Zhao, Reo Furuhata, Mulya Agung, Hiroyuki Takizawa, Tomoya Soma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Predicting hard drive failures in datacenters can help avoid wasting resources and waiting time for recovery. Anomaly detection from sensing data is commonly used for predicting failures. Usually, conventional threshold-based anomaly detection methods consider each sensor independently. However, deciding an optimal threshold for each type of sensors is not trivial, especially for large-scale systems in datacenters. To detect failures that cannot conventionally be detected, multimodal anomaly detection becomes crucial integrating sensing data from different types of sensors. This work proposes a correlation-based multimodal anomaly detection approach. This approach is applied to a Network-Attached Storage (NAS) system with multiple hard disk drives (HDDs) and three sensors, which are a thermal camera, a microphone, and system performance logs. The unimodal results show that the auditory and system performance model can detect temporal anomalies, and the thermal model can detect spatial anomalies. The multimodal results show that even with a simple filter and detection algorithms, the multimodal approach was able to detect failure signs before the real failure and also earlier than the auditory unimodal approach.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE International Conference on Big Data, Big Data 2020
EditorsXintao Wu, Chris Jermaine, Li Xiong, Xiaohua Tony Hu, Olivera Kotevska, Siyuan Lu, Weijia Xu, Srinivas Aluru, Chengxiang Zhai, Eyhab Al-Masri, Zhiyuan Chen, Jeff Saltz
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3545-3549
Number of pages5
ISBN (Electronic)9781728162515
DOIs
Publication statusPublished - 2020 Dec 10
Event8th IEEE International Conference on Big Data, Big Data 2020 - Virtual, Atlanta, United States
Duration: 2020 Dec 102020 Dec 13

Publication series

NameProceedings - 2020 IEEE International Conference on Big Data, Big Data 2020

Conference

Conference8th IEEE International Conference on Big Data, Big Data 2020
Country/TerritoryUnited States
CityVirtual, Atlanta
Period20/12/1020/12/13

Keywords

  • HDD failure
  • multimodal approach
  • unsupervised anomaly detection

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Failure Prediction in Datacenters Using Unsupervised Multimodal Anomaly Detection'. Together they form a unique fingerprint.

Cite this