CoreWeave Makes NVIDIA HGX H100 Generally Available, NovelAI Among First Users

Second H100 offering from CoreWeave

According to the company's announcement, dated March 21, 2023, the HGX H100 instances are CoreWeave's second NVIDIA H100 offering. They follow the launch of H100 PCIe GPU instances in January. CoreWeave began offering the new instances to select customers in February.

CoreWeave CEO and co-founder Michael Intrator said it "means a lot to be the first provider to make NVIDIA HGX H100 platforms generally available." He called this "a testament to our agility and efficiency in deploying infrastructure." The announcement comes only months after CoreWeave said in November that it would be among the first to offer cloud instances with NVIDIA HGX H100 supercomputing.

How the clusters are built

CoreWeave says it has taken a unique approach to building its HGX H100 clusters in order to optimize performance for model training. The clusters run on CoreWeave's Kubernetes-native infrastructure and use a rail-optimized design based on the NVIDIA Quantum-2 InfiniBand networking platform. According to the company, this provides 3.2Tbps of bandwidth per node.

• The infrastructure can scale up to 16,384 H100 SXM5 GPUs under the same InfiniBand Fat-Tree Non-Blocking fabric.
• CoreWeave Cloud integrates NVIDIA BlueField data processing units (DPUs) to enable secure and elastic provisioning of H100 instances.

Ian Buck, vice president of hyperscale and high performance computing at NVIDIA, said that collaboration with innovators such as CoreWeave "enables developers with cutting-edge NVIDIA HGX H100 GPUs to supercharge large language models (LLMs) and other AI workloads."

NovelAI and Anlatan as early adopters

NovelAI, launched in June 2021, is a monthly subscription service for AI-assisted authorship, storytelling, text-adventure games and virtual companionship. It also serves as a GPT-powered sandbox for creators and developers. According to the announcement, its algorithms are trained on actual literature and generate text based on users' writing styles. CoreWeave says the service combines AI storytelling with full encryption.

The HGX H100 platforms will be a key component in Anlatan's product development and deployment. CoreWeave says the cluster will let the developers be more flexible with model design, iterate more quickly on training, and serve their models through NovelAI to millions of users every month. CoreWeave has worked with Anlatan for more than a year, according to Intrator.

Anlatan CEO Eren Doğan said the GPUs give the company "a significant competitive advantage within the machine learning industry" for applications from model training to inference. He added that CoreWeave's Kubernetes-first, cloud native ecosystem "frees us from infrastructure worries and saves us time."

What it means for customers

For AI developers, the announcement means HGX H100 capacity is now generally available from CoreWeave rather than limited to select customers. The company positions the large-scale InfiniBand fabric and the Kubernetes-native setup as aimed at teams training and serving large language models. NovelAI's planned use shows one example: a consumer-facing service that needs both training flexibility and inference capacity for millions of monthly users.

CoreWeave, founded in 2017, describes itself as a specialized cloud provider and claims its solutions for compute-intensive use cases are up to 35 times faster and 80% less expensive than large, generalized public clouds. These are the company's own claims, and the announcement does not provide pricing for the H100 instances.