High Performance Computing (HPC) in the cloud has the potential to improve a wide variety of industries, but this trend is not impressive simply based on its potential—there are already real-world cloud applications leveraging HPC today that have led to incredible accomplishments. HPC in the cloud has enabled research into the effects of Climate Change, helped run scenarios to find cures for rare diseases and aided global energy companies in maximizing the power output of tens of thousands of wind turbines around the world for affordable, renewable energy. HPC has many different advantages depending on the specific use case, but one aspect that these implementations have in common is their use of RDMA-based fabrics to improve compute performance and reduce latency.
The low latency and high performance benefits that Remote Direct Memory Access (RDMA) fabrics, such as InfiniBand and RDMA over Converged Ethernet (RoCE), offer are crucial to the development of HPC applications. RDMA’s ultra-low latency allows minimal processing overhead and exponentially increases overall system performance, which applications such as Artificial Intelligence (AI) and Machine Learning demand while transferring significant amounts of data, exchanging messages and computing results.
-Scot Schultz, 2019
Government agencies, research institutions and universities used to be part of the select few that leveraged HPC for scientific breakthroughs and academic studies. However, over the past few years, private enterprises such as cloud service providers, such as Microsoft Azure, IBM Cloud, Profit Bricks and others saw the many benefits that HPC can bring to their business and have begun making investments to take advantage of everything that High Performance Computing has to offer.