A dynamic assignment scheduling algorithm for big data stream processing in mobile Internet services

Abstract

As the huge number of mobile devices (e.g., smart phones, tablets and netbooks) increases, more and more people choose to use the Internet services financed by mobile Internet service providers (MISPs). To provide better services, it is quite necessary for MISPs to analyze the information hidden in the big data stream generated by users. Therefore, processing the real-time big data stream efficiently has become increasingly important. However, traditional static data storage technology fails to meet the demands of real-time data processing. To improve processing capacity, many parallel processing structures are proposed, which brings up the problem about how the parallel devices can be scheduled to maximize their efficiency. Accordingly, a dynamic assignment scheduling algorithm for big data stream processing in mobile Internet services is proposed, and a stream query graph is built to calculate the weight of every edge. The edge with the minimum weight is selected to send tuples. Simulation results show that the proper number of the logic devices can dramatically reduce system response time. Furthermore, system context switching is reduced by increasing the number of tuples sent each time.