ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management

William F. Godoy; Norbert Podhorszki; Ruonan Wang; Chuck Atkins; Greg Eisenhauer; Junmin Gu; Philip Davis; Jong Choi; Kai Germaschewski; Kevin Huck; Axel Huebl; Mark Kim; James Kress; Tahsin Kurc; Qing Liu; Jeremy Logan; Kshitij Mehta; George Ostrouchov; Manish Parashar; Franz Poeschel; David Pugmire; Eric Suchyta; Keichi Takahashi; Nick Thompson; Seiji Tsutsumi; Lipeng Wan; Matthew Wolf; Kesheng Wu; Scott Klasky

doi:10.1016/j.softx.2020.100561

ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management

William F. Godoy, Norbert Podhorszki, Ruonan Wang, Chuck Atkins, Greg Eisenhauer, Junmin Gu, Philip Davis, Jong Choi, Kai Germaschewski, Kevin Huck, Axel Huebl, Mark Kim, James Kress, Tahsin Kurc, Qing Liu, Jeremy Logan, Kshitij Mehta, George Ostrouchov, Manish Parashar, Franz PoeschelDavid Pugmire, Eric Suchyta, Keichi Takahashi, Nick Thompson, Seiji Tsutsumi, Lipeng Wan, Matthew Wolf, Kesheng Wu, Scott Klasky

Research output: Contribution to journal › Article › peer-review

66 Citations (Scopus)

Abstract

We present ADIOS 2, the latest version of the Adaptable Input Output (I/O) System. ADIOS 2 addresses scientific data management needs ranging from scalable I/O in supercomputers, to data analysis in personal computer and cloud systems. Version 2 introduces a unified application programming interface (API) that enables seamless data movement through files, wide-area-networks, and direct memory access, as well as high-level APIs for data analysis. The internal architecture provides a set of reusable and extendable components for managing data presentation and transport mechanisms for new applications. ADIOS 2 bindings are available in C++11, C, Fortran, Python, and Matlab and are currently used across different scientific communities. ADIOS 2 provides a communal framework to tackle data management challenges as we approach the exascale era of supercomputing.

Original language	English
Article number	100561
Journal	SoftwareX
Volume	12
DOIs	https://doi.org/10.1016/j.softx.2020.100561
Publication status	Published - 2020 Jul 1
Externally published	Yes

Keywords

Data science
Exascale computing
High-performance computing (HPC)
In-situ
Luster GPFS file systems
RDMA
Scalable I/O
Staging

ASJC Scopus subject areas

Software
Computer Science Applications

Access to Document

10.1016/j.softx.2020.100561

Cite this

Godoy, W. F., Podhorszki, N., Wang, R., Atkins, C., Eisenhauer, G., Gu, J., Davis, P., Choi, J., Germaschewski, K., Huck, K., Huebl, A., Kim, M., Kress, J., Kurc, T., Liu, Q., Logan, J., Mehta, K., Ostrouchov, G., Parashar, M., ... Klasky, S. (2020). ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management. SoftwareX, 12, [100561]. https://doi.org/10.1016/j.softx.2020.100561

Godoy, WF, Podhorszki, N, Wang, R, Atkins, C, Eisenhauer, G, Gu, J, Davis, P, Choi, J, Germaschewski, K, Huck, K, Huebl, A, Kim, M, Kress, J, Kurc, T, Liu, Q, Logan, J, Mehta, K, Ostrouchov, G, Parashar, M, Poeschel, F, Pugmire, D, Suchyta, E, Takahashi, K, Thompson, N, Tsutsumi, S, Wan, L, Wolf, M, Wu, K & Klasky, S 2020, 'ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management', SoftwareX, vol. 12, 100561. https://doi.org/10.1016/j.softx.2020.100561

@article{f761da86ee1b48379ec69a26065fff74,

title = "ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management",

abstract = "We present ADIOS 2, the latest version of the Adaptable Input Output (I/O) System. ADIOS 2 addresses scientific data management needs ranging from scalable I/O in supercomputers, to data analysis in personal computer and cloud systems. Version 2 introduces a unified application programming interface (API) that enables seamless data movement through files, wide-area-networks, and direct memory access, as well as high-level APIs for data analysis. The internal architecture provides a set of reusable and extendable components for managing data presentation and transport mechanisms for new applications. ADIOS 2 bindings are available in C++11, C, Fortran, Python, and Matlab and are currently used across different scientific communities. ADIOS 2 provides a communal framework to tackle data management challenges as we approach the exascale era of supercomputing.",

keywords = "Data science, Exascale computing, High-performance computing (HPC), In-situ, Luster GPFS file systems, RDMA, Scalable I/O, Staging",

author = "Godoy, {William F.} and Norbert Podhorszki and Ruonan Wang and Chuck Atkins and Greg Eisenhauer and Junmin Gu and Philip Davis and Jong Choi and Kai Germaschewski and Kevin Huck and Axel Huebl and Mark Kim and James Kress and Tahsin Kurc and Qing Liu and Jeremy Logan and Kshitij Mehta and George Ostrouchov and Manish Parashar and Franz Poeschel and David Pugmire and Eric Suchyta and Keichi Takahashi and Nick Thompson and Seiji Tsutsumi and Lipeng Wan and Matthew Wolf and Kesheng Wu and Scott Klasky",

note = "Funding Information: This research was supported by the Exascale Computing Project ( 17-SC-20-SC ), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725 . Publisher Copyright: {\textcopyright} 2020 The Authors",

year = "2020",

month = jul,

day = "1",

doi = "10.1016/j.softx.2020.100561",

language = "English",

volume = "12",

journal = "SoftwareX",

issn = "2352-7110",

publisher = "Elsevier BV",

}

TY - JOUR

T1 - ADIOS 2

T2 - The Adaptable Input Output System. A framework for high-performance data management

AU - Godoy, William F.

AU - Podhorszki, Norbert

AU - Wang, Ruonan

AU - Atkins, Chuck

AU - Eisenhauer, Greg

AU - Gu, Junmin

AU - Davis, Philip

AU - Choi, Jong

AU - Germaschewski, Kai

AU - Huck, Kevin

AU - Huebl, Axel

AU - Kim, Mark

AU - Kress, James

AU - Kurc, Tahsin

AU - Liu, Qing

AU - Logan, Jeremy

AU - Mehta, Kshitij

AU - Ostrouchov, George

AU - Parashar, Manish

AU - Poeschel, Franz

AU - Pugmire, David

AU - Suchyta, Eric

AU - Takahashi, Keichi

AU - Thompson, Nick

AU - Tsutsumi, Seiji

AU - Wan, Lipeng

AU - Wolf, Matthew

AU - Wu, Kesheng

AU - Klasky, Scott

N1 - Funding Information: This research was supported by the Exascale Computing Project ( 17-SC-20-SC ), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725 . Publisher Copyright: © 2020 The Authors

PY - 2020/7/1

Y1 - 2020/7/1

N2 - We present ADIOS 2, the latest version of the Adaptable Input Output (I/O) System. ADIOS 2 addresses scientific data management needs ranging from scalable I/O in supercomputers, to data analysis in personal computer and cloud systems. Version 2 introduces a unified application programming interface (API) that enables seamless data movement through files, wide-area-networks, and direct memory access, as well as high-level APIs for data analysis. The internal architecture provides a set of reusable and extendable components for managing data presentation and transport mechanisms for new applications. ADIOS 2 bindings are available in C++11, C, Fortran, Python, and Matlab and are currently used across different scientific communities. ADIOS 2 provides a communal framework to tackle data management challenges as we approach the exascale era of supercomputing.

AB - We present ADIOS 2, the latest version of the Adaptable Input Output (I/O) System. ADIOS 2 addresses scientific data management needs ranging from scalable I/O in supercomputers, to data analysis in personal computer and cloud systems. Version 2 introduces a unified application programming interface (API) that enables seamless data movement through files, wide-area-networks, and direct memory access, as well as high-level APIs for data analysis. The internal architecture provides a set of reusable and extendable components for managing data presentation and transport mechanisms for new applications. ADIOS 2 bindings are available in C++11, C, Fortran, Python, and Matlab and are currently used across different scientific communities. ADIOS 2 provides a communal framework to tackle data management challenges as we approach the exascale era of supercomputing.

KW - Data science

KW - Exascale computing

KW - High-performance computing (HPC)

KW - In-situ

KW - Luster GPFS file systems

KW - RDMA

KW - Scalable I/O

KW - Staging

UR - http://www.scopus.com/inward/record.url?scp=85087590183&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85087590183&partnerID=8YFLogxK

U2 - 10.1016/j.softx.2020.100561

DO - 10.1016/j.softx.2020.100561

M3 - Article

AN - SCOPUS:85087590183

SN - 2352-7110

VL - 12

JO - SoftwareX

JF - SoftwareX

M1 - 100561

ER -

ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this