SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Jonghyun Bae; Hakbeom Jang; Jeonghun Gong; Wenjing Jin; Shine Kim; Jaeyoung Jang; Tae Jun Ham; Jinkyu Jeong; Jae W. Lee

doi:10.1109/MM.2019.2930497

SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Jonghyun Bae, Hakbeom Jang, Jeonghun Gong, Wenjing Jin, Shine Kim, Jaeyoung Jang, Tae Jun Ham, Jinkyu Jeong, Jae W. Lee

Department of Computer Science

Research output: Contribution to journal › Article › peer-review

3 Citations (Scopus)

Abstract

This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

Original language	English
Article number	8770099
Pages (from-to)	73-81
Number of pages	9
Journal	IEEE Micro
Volume	39
Issue number	5
DOIs	https://doi.org/10.1109/MM.2019.2930497
Publication status	Published - 2019 Sept 1

Bibliographical note

Publisher Copyright:
© 1981-2012 IEEE.

All Science Journal Classification (ASJC) codes

Software
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/MM.2019.2930497

Cite this

@article{986b65940a6e4b7b903c7f75da6efc9a,

title = "SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning",

abstract = "This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.",

keywords = "Distributed Systems, Machine Learning, Storage Management",

author = "Jonghyun Bae and Hakbeom Jang and Jeonghun Gong and Wenjing Jin and Shine Kim and Jaeyoung Jang and Ham, {Tae Jun} and Jinkyu Jeong and Lee, {Jae W.}",

note = "Publisher Copyright: {\textcopyright} 1981-2012 IEEE.",

year = "2019",

month = sep,

day = "1",

doi = "10.1109/MM.2019.2930497",

language = "English",

volume = "39",

pages = "73--81",

journal = "IEEE Micro",

issn = "0272-1732",

publisher = "IEEE Computer Society",

number = "5",

}

TY - JOUR

T1 - SSDStreamer

T2 - Specializing I/O Stack for Large-Scale Machine Learning

AU - Bae, Jonghyun

AU - Jang, Hakbeom

AU - Gong, Jeonghun

AU - Jin, Wenjing

AU - Kim, Shine

AU - Jang, Jaeyoung

AU - Ham, Tae Jun

AU - Jeong, Jinkyu

AU - Lee, Jae W.

PY - 2019/9/1

Y1 - 2019/9/1

N2 - This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

AB - This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

KW - Distributed Systems

KW - Machine Learning

KW - Storage Management

UR - http://www.scopus.com/inward/record.url?scp=85072190310&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072190310&partnerID=8YFLogxK

U2 - 10.1109/MM.2019.2930497

DO - 10.1109/MM.2019.2930497

M3 - Article

AN - SCOPUS:85072190310

SN - 0272-1732

VL - 39

SP - 73

EP - 81

JO - IEEE Micro

JF - IEEE Micro

IS - 5

M1 - 8770099

ER -

SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this