Techniques for Exploring Cluster Compressed Geospatial-Temporal Satellite Datasets
Date
2013-08
Authors
Ashley, John M.
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
NASA satellite data products are part of the recent big data explosion. An example of this are the individual physically referenced and processed footprints of data from the AIRS satellite (L2 Data Product), Each 2.3 MB data file covers a 6 minute period. Daily data volumes are 0.552GB/day and the collection of data products now spans over a decade. This research addressed NASA's L3Q Data Products. NASA has developed the L3Q Entropy Constrained Vector Quantization (ECVQ) cluster compressed dataset to provide a compact representation of the detailed data that retains much of the original multi-variate, altitudinally indexed information content summarized to a 5o x 5o Earth grid cell over a period of one month. The monthly summary files are- roughly 5.5MB in size, so the compression factor is about 3000 to 1. These multivariate L3Q monthly summaries differ from the NASA's L3 products which contain univariate statistics (means and standard deviations) for 1 x 1 degree earth grid cells.
Description
Keywords
Computer science, Atmospheric sciences, Statistics, Climate change, Data Compression, Data Reduction, Quantization, Remote Sensing, Visualization