GDAM_abstract.tex

\begin{abstract}
Multi-sensor platforms like buoys and gliders produce one or more readings per sensor on varying, discrete time frequencies.  The resulting dataset is a sparse matrix with rows containing readings from sensors that reported at a moment in time and NaN for missing readings from sensors that did not report at that moment.  To assist with data retrieval and analysis, datasets are inserted into relational schemas.  These schemas must either be normalized with replicate data (e.g., time, latitude, longitude, and depth) across tables to minimize the number of joins necessary for a data query or be built with large tables containing mostly NULL values in every row.  NoSQL databases provide a schema-less alternative for storing and retrieving sparse datasets.

Glider Database Alternative with Mongo (GDAM) is a data management system for gliders built on the MongoDB NoSQL database engine.  It is live in production at the University of South Florida Center for Ocean Technology (USF COT).  GDAM is a collection of scripts which parse, process and store real-time glider datasets.  Data is parsed as soon as it is transmitted via satellite to our shore-based servers.  A collection exists for each glider.  Each document in a collection is a moment in time on the glider when a subset of available sensors reported.  The system has been tested during two Slocum G1 glider deployments in September and October of 2012.  Archival datasets dating back to March of 2009 have also been uploaded into this system.  As of May 2013, there are 685,977 records available for three different gliders and 22 deployments.  Records are indexed by time, GPS, and depth with the ability to add more indexes as necessary.

The paper discusses the design decisions made in selecting MongoDB over a traditional relational database management system used for glider operations.  It outlines the structure of glider datasets and common queries run on them.  The common queries guide a comparison between possible relational database schemas and the GDAM implementation.  Database management system implementations are compared on query complexity, storage efficiency, flexibility, and scalability.
\end{abstract}