What is OpenHPC?

Community-based effort provides tools, documentation, and more to accelerate innovation and broaden access to HPC.

3 mistakes to avoid when learning to code in Python

Image by:

Opensource.com

High performance computing (HPC)—the aggregation of computers into clusters to increase computing speed and power—relies heavily on the software that connects and manages the various nodes in the cluster. Linux is the dominant HPC operating system, and many HPC sites expand upon the operating system's capabilities with different scientific applications, libraries, and other tools.

As HPC began developing, that there was considerable duplication and redundancy among the HPC sites compiling HPC software became apparent, and sometimes dependencies between the different software components made installations cumbersome. The OpenHPC project was created in response to these issues. OpenHPC is a community-based effort to solve common tasks in HPC environments by providing documentation and building blocks that can be combined by HPC sites according to their needs.

The project's mission and vision are:

Mission: To provide a reference collection of open source HPC software components and best practices, lowering barriers to deployment, advancement, and use of modern HPC methods and tools.

Vision: OpenHPC components and best practices will enable and accelerate innovation and discoveries by broadening access to state-of-the-art, open source HPC methods and tools in a consistent environment, supported by a collaborative, worldwide community of HPC users, developers, researchers, administrators, and vendors.

OpenHPC history

OpenHPC started at International Supercomputing 2015 (ISC'15) with a birds of a feather (BoF) discussion on the merits of a community effort towards a HPC software repository and management framework. At Supercomputing 2015 (SC'15), OpenHPC 1.0 was released and the Linux Foundation announced the Open HPC Collaborative Project's technical leadership, founding members, and formal governance structure.

Since then, the growing number of OpenHPC project members have collaborated on regular updates of OpenHPC's software packages and documentation, with v1.1.1 released in June 2016 at ISC'16. The newest, v1.3.2, was released in September 2017.

What is OpenHPC?

OpenHPC provides a collection of prebuilt software components common in HPC environments, such as provisioning tools, resource managers, I/O libraries, development tools, and a variety of scientific libraries. It supports different Linux distributions and architectures and provides a package repository for CentOS 7 and SUSE Linux Enterprise Server (SLES) 12.

One key tenet of the project is that no one is ever required to install the entire OpenHPC stack; rather, users can pick and choose the components that are relevant to their HPC needs.

OpenHPC v1.2 and later include builds for x86_64 and aarch64 (tech preview); v.1.3.2 adds CentOS 7.3 and SLES 12 SP2 builds. In addition to the provided software packages, OpenHPC includes installation guides for different provisioning systems (Warewulf, xCAT) as well as different resource managers (Slurm, PBS Professional).

The software and documentation are maintained by the OpenHPC project members and external collaborators in a Git repository hosted at GitHub. To ensure that the software and documentation works as described, OpenHPC has a continuous integration system that verifies the software using bare metal installs and integration test suites.

OpenHPC takes into consideration that HPC sites often rely on multiple compilers that may have partially compatible or even incompatible Application Binary Interfaces (ABIs) and Message Passing Interfaces (MPIs). With the adoption of hierarchical build configurations, OpenHPC delivers unique builds of software packages for each desired compiler/MPI permutation. Software packages that have specific compiler and MPI dependencies have those added to the package name in the following manner:

package-<compiler>-<mpi>-ohpc

By using environment modules to load the corresponding compiler and MPI family, the environment variables will point to the user's compiler and MPI choices.

Supported compiler families are different versions of the GNU Compiler Collection or the Intel compiler, and supported MPI families are Open MPI, MPICH, MVAPICH2, and Intel MPI. Using those options, the resulting package name could be:

fftw-gnu7-openmpi-ohpc

OpenHPC software packages

OpenHPC divides its software packages into different functional areas, and each includes multiple components:

Provisioning: Warewulf, xCAT
Resource management: Slurm, Munge, PBS Professional
Runtimes: OpenMP, OCR, Singularity
Numerical/scientific libraries: GSL, FFTW, Metis, PETSc, Trilinos, Hypre, SuperLU, Mumps, OpenBLAS, Scalapack
I/O libraries: HDF5, NETCDF, Adios
Compilers: GCC, LLVM
MPI families: MVAPICH2, Open MPI, MPICH
Performance tools: PAPI, IMB, mpiP, pdtoolkit, TAU, Scalasca, ScoreP, SIONLib

The next release (1.3.3) is planned for November 2017. As with every new release, new packages will be added and existing packages will be updated. Latest version can always be obtained from openhpc.community/downloads/.

Additional resources

OpenHPC is a collaborative, community effort that is open for contributions. If you would like to learn more about OpenHPC and how you can contribute, please visit the following resources: