How Intel Omni-Path® delivers industry-leading performance and improves as systems scale up

The future of high-performance computing looks different depending on who you ask. But one thing nearly everyone is in agreement about: Scale will always be a key factor. So future-proofing a cluster should involve technologies that scale well to ever-larger numbers of nodes — and ideally improve their performance as the total number of nodes in a cluster grows.

In other words, one crucial goal in HPC should be to deploy components whose performance only improve as the system scales. The Intel® Omni-Path Architecture (OPA) is just such a fabric, one that scales as well as or better than other leading interconnects.

Consider the simulation software package ANSYS® Fluent®. Engineering teams facing difficult simulation environments often must trim back their simulations in order to meet production deadlines and beat the competition to market. Unfortunately, the simulations they will be doing in the quarters and years ahead will only be demanding more of their HPC systems. They’ll be needing more and greater scale. Their system’s interconnect should be up to the challenge.

Benchmark testing performed by Intel in cooperation with ANSYS has established that OPA provides comparable performance to competing interconnects in systems with 1, 2, 4, or 8 nodes. However, as the node count increases, OPA establishes a lead on the competition that only grows with size.

Running ANSYS Fluent on OPA systems with 64 nodes, for instance, achieves a 19% speedup compared to a competing interconnect. While an OPA cluster with 96 nodes runs the same Fluent job 33% faster.

Once at scale, OPA also achieves further efficiencies with lightweight PSM2 libraries that reduce communications overhead. This becomes particularly useful in an environment like Fluent that generates many small MPI messages. Contrast the above to other fabrics that use the VERBS libraries, whose millions of lines of code were developed originally for storage connectivity.

As discussed at length in another article, OPA achieves such impressive ANSYS performance in part due to its efficient interconnect topology.

Its 48 port radix switches can support all 24 servers in an HPE Apollo 6000 Gen10 chassis at full bi-sectional bandwidth given its design for 24 downlinks and 24 uplinks. In contrast, alternative switches are limited to only 24 downlinks and 12 uplinks, requiring over-subscription and subsequently taking up to 2X bandwidth reduction to utilise all 24 servers in a chassis. Further, given that Intel OPA can support 33% more ports per switch this means that users will need to buy fewer switches, cables, and utilise less power which translates into significant CAPEX and OPEX costs savings, especially as users grow their cluster. In fact, Intel OPA can support up to 1152 servers with full bi-sectional bandwidth in a 2-Tier Fat Tree, 2.5X more than alternative fabric.

In the words of Dr. Wim Slagter, ANSYS Director of HPC and cloud marketing, “Thanks to Intel® Omni-Path Architecture and the latest Intel® Xeon® processor E5-2600 v4 product family, ANSYS® Fluent® is able to achieve performance levels beyond our expectations. Its unrivalled performance enables our customers to simulate higher-fidelity models without having to expand their cluster nodes.”

OPA’s impressive performance specs and benchmark performance speak for themselves. In even-handed apples-to-apples comparisons, OPA performs as well as or better than the other leading HPC interconnects. Whether the metric is MPI message rate, Natural Order Ring (NOR) latency, Random Order Ring (ROR) latency, or just overall latency, OPA stands up to scrutiny.

The HPC industry is adopting OPA too. Both end customers and system vendors respond to OPA’s performance and cost effectiveness. They find utility in OPA’s advanced features like Traffic Flow Optimisation, Packet Integrity Protection and Dynamic Lane Scaling. Moreover, 63% of the fastest supercomputers in the world using 100Gb fabric are taking advantage of Intel® OPA.

To scale your HPC’s performance and future-proof its interconnect, explore HPE and Intel® OPA HPC solutions today.


Contact us


Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedIn

Leave a Reply