Title
Web technologies for environmental Big Data
Date Issued
01 January 2015
Access level
open access
Resource Type
review
Author(s)
Imperial College London
Publisher(s)
Elsevier Ltd
Abstract
Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources.
Start page
185
End page
198
Volume
63
Language
English
OCDE Knowledge area
Ciencias del medio ambiente
Ciencias de la información
Subjects
Scopus EID
2-s2.0-84910036454
Source
Environmental Modelling and Software
ISSN of the container
13648152
Sponsor(s)
Finally, DataOne 66 66 is another cyber-infrastructure, supported by the U.S. National Science Foundation, to access Earth observational data ( Michener et al., 2011 ). Its first prototype implements client libraries in Java, Python and R and aims to enable users to mount the entire DataONE cloud infrastructure as a file system.
This work was supported by the Natural Environment Research Council pilot projects on Environmental Virtual Observatory technologies NE/I002200/1 and NE/I004017/1 .
Sources of information:
Directorio de Producción Científica
Scopus