Containers and the Truth of HPC & Cloud System Software Convergence

As the code complexity of HPC applications expands, development teams continually rely on detailed software operation workflows to enable automation of building and testing their application. HPC application development can become increasingly complex and, as a result, difficult to maintain when the target HPC platforms’ environments are increasingly diverse and continually changing. Recently, containers have demonstrated a new software deployment mechanism and the latest support for containers in HPC environments makes them now attainable for scientific computing teams. 

This talk will discuss the dichotomy of container utilization in both HPC and the cloud.  Specifically, we will showcase how DOE ECP is taking a multi-level approach to accelerate adoption of container technologies for exascale systems. A goal of the effort is to ensure that container runtimes are well poised to take advantage of future HPC systems, including efforts to ensure container images can be scalable, interoperable, and well integrated into the first exascale supercomputers and across the DOE. However, much of these container deployment features already exist in the cloud and hyperscaler computing industry, although at a lower scale. This raises the question of whether container utilization in exascale will match that of the cloud, or will the cloud industry demonstrate scalability in their respective domain first?