Discovering and exploiting statistical properties for query optimization in relational databases: A survey

Abstract

Discovering and exploiting statistical features in relational datasets is key to query optimization in a relational database management system (rdbms), and is also needed for database design, cleaning, and integration. This paper surveys a variety of methods for automatically discovering important statistical features such as correlations, functional dependencies, keys, and algebraic constraints. We discuss proactive approaches in which the data is scanned or sampled (periodically, at optimization time or at query time), or in which exploratory queries are executed. Also discussed are reactive approaches that monitor the results of the query processing. Finally, we discuss methods for dealing with the practical challenges of maintaining statistical information in the face of heavy system utilization, and of dealing with inconsistencies that arise from incomplete cardinality models, use of multiple discovery methods, or changes in the underlying data over time. © 2009 Wiley Periodicals, Inc.