Rucio: Scientific Data Management

Date

2019

Authors

Barisits, Martin
Beermann, Thomas
Berghaus, Frank
Bockelman, Brian
Bogado, Joaquin
Cameron, David
Christidis, Dimitrios
Ciangottini, Diego
Dimitrov, Gancho
Elsing, Markus

Journal Title

Journal ISSN

Volume Title

Publisher

Computing and Software for Big Science

Abstract

Rucio is an open-source software framework that provides scientific collaborations with the functionality to organize, manage, and access their data at scale. The data can be distributed across heterogeneous data centers at widely distributed locations. Rucio was originally developed to meet the requirements of the high-energy physics experiment ATLAS, and now is continuously extended to support the LHC experiments and other diverse scientific communities. In this article, we detail the fundamental concepts of Rucio, describe the architecture along with implementation details, and report operational experience from production usage.

Description

Keywords

Data organization, Data management, Data access, Distributed computing, Exascale

Citation

Barisits, M., Beermann, T., Berghaus, F., Bockelman, B., Bogado, J., Cameron, D., … Wegner, T. (2019). Rucio: Scientific Data Management. Computing and Software for Big Science, 3(1). https://doi.org/10.1007/s41781-019-0026-3