SolarSPIRE: A content-based retrieval engine for temporal sequences of solar imagery

Abstract

In this paper, we present an application designed to permit specification of, and search for spatio-temporal phenomenon in image sequences of the solar surface acquired via satellite. The application is designed to permit space scientists to search archives of imagery for well-defined solar phenomenon, including solar flares, search tasks that are not practical if performed manually due to the large data volumes. Data for this application comes from the SOHO solar-observing satellite and consists of images acquired every 17 minutes over a period of a number of months. Images are preprocessed to identify bright and dark spots. Temporally persistent objects are identified and both time-variant attributes (such as size and luminance), and time-invariant attributes (such as average luminance) are extracted. Time-variant attributes are stored using a novel multi-resolution representation that assigns semantic labels for each resolution. Objects of interest are defined by the user via a web-based Java interface. The interface provides an object-definition language, which permits specification of a set of constraints to be used to filter the pre-extracted image objects. Time-variant attributes are constrained using semantic labels and temporal relationships. For example, a new object type might be defined as `bright spots, which have sharply rising intensity for 6 to 12 hours followed by slowly falling intensity for more than 24 hours'. Compound objects are specified using spatial and temporal constraints between simpler object attributes. Sharp constraints such as `within 100 pixels' and fuzzy constraints such as `between' are both supported. Both simple and compound object definitions are added to a library of object types, which can be used to formulate queries, or as building blocks for new object definitions. Object searches return key frames for each object identified, and these can be used to subsequently retrieve more detailed images or image sequences. Using this application, we have defined and successfully retrieved complex solar phenomenon such as Coronal Mass Ejections. We anticipate applying the framework to other non-video, time-varying data sets such as sequences of earth-observing satellite images, or medical imagery (e.g., PET).