Artificial intelligence news.
Straight from the original announcements.

CoreWeave launches one-click GPT-J inference service with scale-to-zero

November 13, 2026, 22:09By the AI News Bites Editorial TeamSource: CoreWeave

CoreWeave says it is making EleutherAI’s GPT-J available through a one-click inference service, combining the open-source model with access to its GPU infrastructure. The company is targeting organizations that want to deploy GPT-J with faster scaling and lower infrastructure overhead.

Key points
  • CoreWeave offers one-click deployment for GPT-J inference.
  • GPT-J has 6B parameters and was trained on The Pile.
  • The service supports scale-to-zero with zero resource consumption when scaled down.
  • AI Dungeon reported 75% lower costs and 50% lower latency.
  • NovelAI reported 3x faster requests and 75% lower cloud costs.

A smaller model designed for practical deployment

GPT-J was trained by EleutherAI on The Pile, described by CoreWeave as an 825 GiB language modelling data set. The company says GPT-J is one of the largest open-source natural language processing models available, with 6B parameters.

CoreWeave says the model can compete with, and at times produce higher quality results than, GPT-3 and its 175B parameters. Its smaller size means GPT-J requires less memory and horsepower to serve at scale, according to the company.

EleutherAI said its aim was to create “the world's largest open-source language model” and described diverse data as important for improving the quality of large models. The organization said GPT-J-6B is “proving to be incredibly powerful given its size and quality.”

One-click deployment and responsive scaling

CoreWeave’s GPT-J Inference Service is available through CoreWeave Apps, where clients can deploy an API inference service with one click. CoreWeave says this removes infrastructure overhead while allowing customers to use its range of GPUs, including NVIDIA A5000s.

The service uses CoreWeave’s Kubernetes-native infrastructure and supports scale-to-zero. When workloads are scaled down, the service consumes zero resources and incurs zero billing, according to CoreWeave. The company says its autoscaling is responsive and that GPT-J can be served more efficiently by selecting from a broad range of GPUs.

CoreWeave also says its fleet includes approximately 50,000 GPUs. Customers can deploy containerized workloads through Kubernetes, while CoreWeave engineers provide help with DevOps, workload optimization and GPU selection for training and serving models.

Early customer results and open-source NLP support

AI Dungeon CEO Nick Walton said the company previously faced planning, cost and latency problems with GPT-3. After moving its compute infrastructure to CoreWeave, AI Dungeon lowered costs by 75% and reduced latency by 50%, according to CoreWeave. The company expects further performance gains as it moves production traffic to GPT-J on CoreWeave.

AI Dungeon selected the A5000 after benchmarking CoreWeave’s available compute and said it delivered the best performance-adjusted cost for its workloads. NovelAI is also serving its Sigurd storytelling model, which is based on GPT-J and fine tuned on novels and short stories. NovelAI CEO Eren Doğan said requests were served 3x faster after the move to CoreWeave, while cloud costs fell by 75%.

CoreWeave says it is working with EleutherAI on open-source NLP using a DGX A100 distributed training cluster with full InfiniBand connectivity between nodes. For users and developers, the announcement offers a managed route to deploy GPT-J, with the company’s reported cost and scaling benefits aimed at making production NLP workloads easier to operate.

Related news

CoreWeave launches managed PostgreSQL inside AI Kubernetes clusters
Products and toolsNovember 11, 2026, 17:05

CoreWeave launches managed PostgreSQL inside AI Kubernetes clusters

CoreWeave Database is now generally available, giving teams managed PostgreSQL inside their CoreWeave Kubernetes Service clusters. The service places databases alongside…

CoreWeave reports $3.7 billion revenue as AI cloud demand accelerates
BusinessNovember 10, 2026, 22:11

CoreWeave reports $3.7 billion revenue as AI cloud demand accelerates

CoreWeave reported $3.7 billion in revenue and a revenue backlog of approximately $133 billion, as the AI cloud provider expanded its active power capacity beyond 1.7…

CoreWeave Joins CNCF as a Platinum Member
BusinessNovember 10, 2026, 18:03

CoreWeave Joins CNCF as a Platinum Member

CoreWeave has become a Platinum member of the Cloud Native Computing Foundation, expanding its support for Kubernetes and the AI-native cloud community. The company…