Understanding Why AI and Analytics Have to Run on HPC Clusters

Data science is now a first-class citizen in the HPC data center, in the cloud, and on the workstation. The converse is also true, High Performance Computing (HPC) is now a high priority for data scientists who confront issues of time-to-model when training, inference latency, and of course, the ever-present need to manipulate large amounts of data. HPC clusters must support these combined workloads – or more specifically “converged” HPC, AI, and High-Performance Data Analytic (HPC-AI-HPDA) workloads.

This convergence of HPC-AI-HPDA reflects a golden age for data science as people expect to run with supercomputer performance on their HPC systems be they workstations, on their local clusters, and in the cloud. The reason is that workload optimized servers for AI provide optimized software stacks that run industry-standard tools like TensorFlow* with support for vectorized low-precision arithmetic along with vector AVX-512 floating point operations with high performance. Higher core count processors deliver greater parallelism in high memory bandwidth configurations. These capabilities deliver performance and eliminate the need for accelerators for many organizations.

AI and cloud innovation have helped stimulate a golden age of HPC as well. The mass adoption of AI has introduced new thinking where AI is part of a simulation and modeling process along with a new ecosystem of industry-standard tools and workload-optimized, high performance hardware. New 2nd Generation Intel® Xeon® Scalable processors with Intel® Deep Learning Boost demonstrate a real convergence at the hardware level while facilitating better convergence for workloads.

Read the full article, by Esther Baldwin, on HPCwire.com.

Comments are closed.