Target-Independent Mining For Scientific Data: Capturing Transients and Trends for Phenomena Mining

All Contents

This paper describes a data mining approach for extracting enriched data from scientific data archives such as NASA’s Earth Observing System Data and Information System (EOSDIS) that are stored on slow access tertiary storage. This enriched data has significantly smaller volume than the original data, yet preserves sufficient properties of this data such that over time, many different users can repeatedly mine it for different Earth-science phenomena. This enriched data captures daily trends and significant deviation from trends for each bin of gridded data from an equal-degree grid covering the Earth’s surface. A feature of this enriched data is that it is independent of any particular target phenomena, although it assumes that such phenomena are either transient in nature or characterized by trends in the data. The enriched data can be stored in a database on fast secondary storage where it can be used repeatedly by many users to rapidly mine for phenomena of interest. Our research effort with SSM/I data shows that the approach gives anticipated results and has many potential applications in the mining of transient and long-term phenomena.