A predicate-based incremental refresh method for a data warehouse

Abstract

With the increasing popularity of data warehouses and data marts, the ability to refresh data in a timely fashion is more important than ever. In this paper, a new approach is proposed to incrementally update materialized summary tables. The advantage of the approach is that a) this method retrieves changes on a predicate level; b) the method utilizes time information hidden in user data. Therefore, the method does not introduce extra cost while the other current approaches do introduce overhead for time tag; c) our method applies changes to summary tables by merge. The merge only updates changed values and inserting new rows into summary table, and it is much more efficient than union. By making use of the above advantages, this method is much faster than other methods, and the same accurateness as state-of-the-art approaches can be achieved. By comparing this method with the state-of-the-art over IBM DB2 workstation version, it showed that the proposed method outperforms the state-of-the-art in terms of elapse time.