Title
Open and scalable analytics of large Earth observation datasets: From scenes to multidimensional arrays using SciDB and GDAL
Date Issued
01 April 2018
Access level
open access
Resource Type
journal article
Author(s)
Imperial College London
Publisher(s)
Elsevier B.V.
Abstract
Earth observation (EO) datasets are commonly provided as collection of scenes, where individual scenes represent a temporal snapshot and cover a particular region on the Earth's surface. Using these data in complex spatiotemporal modeling becomes difficult as soon as data volumes exceed a certain capacity or analyses include many scenes, which may spatially overlap and may have been recorded at different dates. In order to facilitate analytics on large EO datasets, we combine and extend the geospatial data abstraction library (GDAL) and the array-based data management and analytics system SciDB. We present an approach to automatically convert collections of scenes to multidimensional arrays and use SciDB to scale computationally intensive analytics. We evaluate the approach in three study cases on national scale land use change monitoring with Landsat imagery, global empirical orthogonal function analysis of daily precipitation, and combining historical climate model projections with satellite-based observations. Results indicate that the approach can be used to represent various EO datasets and that analyses in SciDB scale well with available computational resources. To simplify analyses of higher-dimensional datasets as from climate model output, however, a generalization of the GDAL data model might be needed. All parts of this work have been implemented as open-source software and we discuss how this may facilitate open and reproducible EO analyses.
Start page
47
End page
56
Volume
138
Language
English
OCDE Knowledge area
Ciencias del medio ambiente
Ingeniería ambiental y geológica
Subjects
Scopus EID
2-s2.0-85042181249
Source
ISPRS Journal of Photogrammetry and Remote Sensing
ISSN of the container
09242716
Sponsor(s)
Horizon 2020 Framework Programme - 776242 -H2020
Natural Environment Research Council - NE/I022558/1, NE/J016578/1 - NERC
Sources of information:
Directorio de Producción Científica
Scopus