Distributed data streams indexing using content-based routing paradigm

Abstract

In recent years, we have seen a dramatic increase in the use of data-centric distributed systems such as global grid infrastructures, sensor networks, network monitoring, and various puhlish-subscribe systems. The realization of this potential requires adequate support from middleware that could be used to deploy and support such systems. In this regard, we propose an integrated distributed indexing architecture that supports scalable handling of intense dynamic information flows. The architecture is geared towards providing timely responses to queries of different types while minimizing the use of network and computational resources. The underlying communication framework ensures scalability and load balancing of communication as well as adaptivity in presence of dynamic changes. We elaborate on database and content-based routing methodologies used in the integrated solution as well as non-trivial interaction between them, and thereby provide a valuable feedback to the designers of these techniques. We demonstrate the effectiveness of our architecture with performance results that we obtained using our prototype implementation on top of the Chord system simulator.