Baseten launches joey-1, a 2B decision model with 20 ms latency
Baseten has released joey-1, a 2B-parameter decision model that, according to the company, serves an average 2,000-token request at a P50 end-to-end latency of 20 ms. It costs $0.01 per million tokens and is available now on Baseten Model APIs, with open weights on Hugging Face.
- joey-1 is a 2B decision model with a P50 end-to-end latency of 20 ms on an average 2,000-token request, according to Baseten.
- It costs $0.01 per million tokens, a tenth of mammoth-1.
- It is post-trained from Qwen3.5-2B and available on Baseten Model APIs, with open weights on Hugging Face.
- Dedicated deployments offer the full 262k context window, while Model APIs support 16k.
- Baseten suggests joey-1 as a first-line model, escalating uncertain cases to elephant-1 or mammoth-1.
What joey-1 does
Many LLM tasks do not need generated text. They need a decision: where a support ticket should go, whether a message should be flagged, or whether a request should be approved. Baseten argues that LLMs have historically answered such questions by generating text, which adds cost and latency to every call, and that the overhead compounds with each decision in a chain.
Decision models work differently. According to Baseten, they return the answer directly as a single token plus a confidence score, in one forward pass. That is meant to cut cost and latency sharply.
joey-1 is the smallest and fastest of the Base Labs decision models. Baseten says it is 5x faster than comparable decision models. In internal testing, with details still to be published, the company calls joey-1 the strongest decision model with 3B or fewer parameters.
Speed, price and the model family
Baseten gives the following figures for production-scale serving of the average 2,000-token request:
- P50 end-to-end latency of 20 ms for joey-1, about 2.5x faster than elephant-1 at 49 ms
- A price of $0.01 per million tokens, a tenth of the cost of mammoth-1
The release comes eight days after mammoth-1 (27B) and elephant-1 (9B), which Baseten describes as the #1-ranked decision models. With joey-1, the Base Labs family now spans three sizes. All three accept text and images, were trained on the same data mixture and share the same API, so switching between them is a one-line change, according to the company.
Baseten positions mammoth-1 for decisions where accuracy matters most, elephant-1 as the balanced default, and joey-1 for high-volume, latency-sensitive calls. It also suggests using joey-1 as a first-line model, with elephant-1 or mammoth-1 as an escalation path when joey-1 is uncertain.
When to move a workload
Baseten says joey-1 is worth considering in two cases: when decision volume is high, so cost per call determines whether a project is economic, and when the decision is a prerequisite for something else, so its latency is added to every response.
As an example, the company calculates that classifying 10 million support tickets a month at about 500 tokens each comes to 5 billion tokens. That is roughly $50 a month on joey-1 versus $500 on mammoth-1.
Deployment, fine-tuning and availability
joey-1 is post-trained from Qwen3.5-2B by Base Labs, Baseten's in-house research lab, using the same data and recipe as elephant-1 and mammoth-1 with Baseten Training Jobs. On Model APIs, the models support a 16k context window and are optimized for low latency. Dedicated deployments unlock the full 262k context window, support fine-tuned weights, and can deliver faster TTFT and TPS through workload-specific optimizations and co-location with a customer's LLM workloads.
Developers who want custom weights can post-train elephant-1 and joey-1 on Training Jobs, and Baseten points to a fine-tuning cookbook. joey-1 is live on Baseten Model APIs with zero data retention by default.
For teams running classification, routing or moderation at scale, the release offers a cheaper and faster option for such calls, though the benchmark details behind the company's claims have not yet been published.
