Scale-up strategies for processing high-rate data streams in system S

Abstract

High performance stream processing is critical in sense-and-respond application domains - from environmental monitoring to algorithmic trading. In this paper, we focus on language and runtime support for improving the performance of sense-and-respond applications in processing data from highrate streams. The central tenet of this work is the definition of a streaming architectural pattern for these application domains and the programming model and the code generation framework to support it. Using IBM Research's System S middleware and the SPADE language, we demonstrate how to scale up a financial trading application. © 2009 IEEE.