HPC combines all aspects of computationally intensive tasks required for conducting simulations in the area of CSE. This typically refers to utilizing more than one computing system to achieve this goal. In HPC hardware and software aspects with respect to conducting performance critical applications are considered. This fact introduces a highly diverse ecosystem of research areas. Typical HPC areas are parallel computing, network architecture, supercomputers, compilers, debuggers, operating systems, visualization, and data storage. In the following, hardware and software aspects of parallel computing are investigated, which we consider to be the basis for HPC.
Historically the CPU’s sole core was continually scaled down to increase the processing power by adding more and more transistors. Consequently, each new generation of CPUs automatically improved application performance. However, around the mid-2000s this approach gave way to introducing additional cores on the CPU die but with stagnating - or even decreased - clock frequencies to handle the thermal budget [100][128][129]. Clock frequencies for high-end consumer-level workstation CPUs saturated at approximately 3-4 GHz. Overall, this conceptual change was forced by physical limitations imposed by scaling limits, such as issues with heat dissipation, power consumption, and leakage-currents. Today’s computing platforms entirely use multi-core CPUs. This development forces the software - driving the actual computation - to harness the parallel computational power to achieve an appropriate computational speed-up with new generations of hardware [130].
The ongoing scaling of semiconductor devices towards the low nanometer regime introduced the ability to provide multi-core CPUs as well as accelerators, such as graphics adaptors, to single workstations and even mobile devices. Therefore, parallel computing environments are no longer confined to highly expensive supercomputers, but are already available in consumer products. This introduces a dire need for software developers to have a solid grasp on parallel computing and the supported ways of harnessing the computational capabilities.
This section provides a detailed overview of parallel computing, more specifically: