HparC: A mixed nested shared memory and message passing programming style intended for grid

Abstract

A set of clusters (Grid) of multicore machines form an attractive platform for executing parallel programs. However, it is not clear how to program a single coherent parallel program over such a Grid of multicore machines (MCs). The main problem derives from the fact that the shared memory of each single MC machine cannot be scaled to run over a cluster of machines or a Grid. This is due to the fact that shared memory in a multicore machine is handled in hardware using a special cache coherency protocol, whereas over a cluster, the shared memory must be implemented in software using message passing. It is therefore, imperative for a Grid to devise a unified and coherent programming language that would combine both shared memory and message passing constructs, e.g., a language that combines the abilities of both OpenMP and MPI into one coherent language. The proposed language must produce parallel programs that are portable and can support the ability to scale the cluster without the need for re-compilation. In this work we consider extending the parallel-for loop programming style to include message-passing constructs in such a way that: • The message-passing and the shared memory constructs can be uniformly combined creating a coherent nested structure of a parallel program. • The parallel programs that are written in the proposed programming style are automatically scaled to run over different topologies of Grids. The proposed programming style (called HparC) combines nested parallel loops of shared memory with message passing to create efficient and simple parallel servers. The expressive power of the programming language is demonstrated via a "filtering-server" example implemented as a tree of threads. The coding of this server is compared to coding in Java and to an implementation using OpenMP together with MPI. This work extends a previous version of HparC. Copyright 2010 ACM.