Title
Scidac-Data: Enabling Data Driven Modeling of Exascale Computing
Date Issued
23 November 2017
Access level
open access
Resource Type
conference paper
Author(s)
Mubarak M.
Ding P.
Tsaris A.
Norman A.
Lyon A.
Ross R.
Fermi National Accelerator Laboratory, Batavia
Publisher(s)
Institute of Physics Publishing
Abstract
The SciDAC-Data project is a DOE-funded initiative to analyze and exploit two decades of information and analytics that have been collected by the Fermilab data center on the organization, movement, and consumption of high energy physics (HEP) data. The project analyzes the analysis patterns and data organization that have been used by NOvA, MicroBooNE, MINERvA, CDF, D0, and other experiments to develop realistic models of HEP analysis workflows and data processing. The SciDAC-Data project aims to provide both realistic input vectors and corresponding output data that can be used to optimize and validate simulations of HEP analysis. These simulations are designed to address questions of data handling, cache optimization, and workflow structures that are the prerequisites for modern HEP analysis chains to be mapped and optimized to run on the next generation of leadership-class exascale computing facilities. We present the use of a subset of the SciDAC-Data distributions, acquired from analysis of approximately 71,000 HEP workflows run on the Fermilab data center and corresponding to over 9 million individual analysis jobs, as the input to detailed queuing simulations that model the expected data consumption and caching behaviors of the work running in high performance computing (HPC) and high throughput computing (HTC) environments. In particular we describe how the Sequential Access via Metadata (SAM) data-handling system in combination with the dCache/Enstore-based data archive facilities has been used to develop radically different models for analyzing the HEP data. We also show how the simulations may be used to assess the impact of design choices in archive facilities.
Volume
898
Issue
6
Language
English
OCDE Knowledge area
Ingeniería de sistemas y comunicaciones Hardware, Arquitectura de computadoras
Scopus EID
2-s2.0-85038425976
Source
Journal of Physics: Conference Series
ISSN of the container
17426588
Sponsor(s)
The research at Argonne is based upon work supported by the U.S. Department of Energy, Office of Science, under Contract No. DE-AC02-06CH11357. Fermilab is operated by Fermi Research Alliance, LLC under Contract No. De-AC02-07CH11359 with the U.S. Department of Energy. The research at Argonne is based upon work supported by the U.S. Department of Energy, Office of Science, under Contract No. DE-AC02-06CH11357.
Sources of information: Directorio de Producción Científica Scopus