With the increasing complexity and diversity of the software stack and system
architecture of high performance computing (HPC) systems, the traditional HPC
community is facing a huge productivity challenge in software building,
integration and deployment for multiple exascale computing systems that will be
deployed in year 2020 and after. Recently, this challenge has been addressed by
new software build management tools such as Spack that enable seamless software
building and integration. Container based solutions provide a versatile way to
package software and are increasingly being deployed on HPC systems. The DOE
Exascale Computing Project (ECP) Software Technology focus area is developing
an HPC software ecosystem that will enable the efficient and performant
execution of exascale applications. Through the Extreme-scale Scientific
Software Stack (E4S) [https://e4s.io], it is developing a comprehensive and
coherent software stack that will enable application developers to productively
write highly parallel applications that can portably target diverse exascale
architectures. E4S provides both source builds through the Spack platform and a
set of containers that feature a broad collection of HPC software packages. E4S
exists to accelerate the development, deployment, and use of HPC software,
lowering the barriers for HPC and AI users. It provides container images, build
manifests, and turn-key, from-source builds of popular HPC software packages
developed as Software Development Kits (SDKs). This effort includes a broad
range of areas including programming models and runtimes (MPICH, Kokkos, RAJA,
OpenMPI), development tools (TAU, PAPI), math libraries (PETSc, Trilinos), data
and visualization tools (Adios, HDF5, Paraview), and compilers (LLVM), all
available through the Spack package manager. It will describe the community
engagements and interactions that led to the many artifacts produced by E4S. It
will introduce the E4S containers are being deployed at the HPC systems at DOE
national laboratories using Singularity, Shifter, and Charliecloud container
runtimes. In this minisymposium, we will discuss the recent efforts and
techniques to improve software integration and deployment for HPC platforms. We
will cover the issues on software development practice, use of containers and
automation of performance testing, performance optimization using TAU, and
software building.