The Future of Computing Systems for AI & HPC: Applications & Architecture

Abstract

Many recent breakthroughs in AI and science were only possible due to the availability of ever-more powerful computing systems. The architecture of the systems used to run classic HPC applications at exascale and the systems training trillion-parameter AI models is converging, but the explosion of the amounts of data, the power & energy limitations, the slow-down of Moore’s law, and the operational complexity (including security, data management, development environment, etc) need to be addressed. This demands a fresh look at system architecture, where data takes the center stage and defines key architectural elements in a data-driven approach. As a result, we propose a system architecture that combines a data and service oriented core with high-performance, energy efficient, domain-specific compute HW, which gets integrated, e.g., as chiplets into the overall system. The increased complexity of these system also requires a co-design approach, where the programming environment (languages, compilers, run-time components) is developed and adopted together with the new technologies.