Scale AI builds real-time insurance call copilot on Nemotron, cutting costs 70 to 75%
Scale AI says it has post-trained NVIDIA's Nemotron open model into a real-time copilot for a global insurance company's customer service representatives, lifting accuracy while cutting inference cost by roughly 70 to 75% compared with the insurer's production model. The copilot runs alongside live French-language roadside assistance calls.
- Scale AI post-trained NVIDIA's Nemotron into a real-time copilot for a global insurer's French-language roadside assistance calls.
- Fact extraction accuracy rose to 93.8% from 76.1% before post-training; the production model scored 62.6%.
- Coverage recommendation reached 95.6%, versus 85.4% for the production model and 92.2% for the best frontier model tested.
- Inference cost fell by roughly 70 to 75% versus the production model, with under $1,000 in training compute per task.
- Results come from synthetic calls; real-call evaluation and guardrail work remain before production.
What the copilot does
When a driver calls for roadside assistance, the representative must understand what happened, work out which benefits the policy covers and arrange help, all before the call ends. According to Scale, the relevant facts emerge over several turns, so decisions have to be made with incomplete information.
Scale split the copilot into three connected tasks and evaluated each separately, so that a gain in one could not hide a failure in another:
- Extract case facts: identify the event type, the breakdown reason and the number of passengers, updating answers as the call continues and leaving a field blank when the customer has not said enough.
- Recommend coverage: identify which benefit families may apply, such as roadside repair, towing, mobility or accommodation. If a deciding fact is missing, it asks for it rather than assuming coverage.
- Guard against over-promising: check whether a representative's commitment exceeds the policy, for example a promise to reimburse a €60 taxi fare that the policy does not cover.
Scale notes that the copilot must also respond with low latency, because representatives rely on it while still on the line.
How it was trained and measured
The insurer's call data needs legal approval before it can be used for training, so Scale generated synthetic French conversations from sampled case facts and policy scenarios. Scoring is strict. Fact extraction is correct only when all three fields match exactly, and coverage recommendation is correct only when the full set of recommended benefits matches. The guardrail counts a call as correct only when every check matches the ground truth. Violations occur in only about 4% of checks, so it must catch rare cases without raising false alerts.
For each task, Scale first applied supervised fine-tuning (SFT) on the model's own correct reasoning and tool-use trajectories, selected from multiple attempts. It then applied reinforcement learning using group sequence policy optimization (GSPO), rewarding exact matches with the verified answer. According to Scale, SFT widened the range of cases the model could solve, while RL made it more consistent on the first try.
Training was inexpensive, the company says. The fact-extraction model took about 16 GPU hours on 8 H100s (about $890), and the coverage model about 4 GPU hours (about $230).
Results and costs
Scale reports that fact extraction reached 93.8% accuracy, up from 76.1% for base Nemotron and 31 points above the production model's 62.6%. That is in line with the strongest frontier models Scale tested, which scored 95.8% and 94.4%. Coverage recommendation reached 95.6%, compared with 85.4% for the production model and 92.2% for the best frontier model tested. Base Nemotron had already scored 93.6% on this task.
On cost, Scale says Nemotron 3.5 Super VL uses a mixture-of-experts architecture that activates only 12B parameters out of 120B total. Fact extraction costs about $3.00 per 1,000 queries, against $12.10 for the production model. At a lower reasoning setting it still scores 91.9% at $1.80 per 1,000 queries, and latency drops from 26 seconds to 11. Coverage recommendation costs about $1.40 per 1,000 queries, against $4.80 for the production model.
Scale also says its engineers contributed improvements back to NVIDIA's open post-training tools to support reinforcement learning on Nemotron's hybrid mixture-of-experts architecture.
Limits and what comes next
Scale stresses that the results come from synthetic calls in a controlled setting. The next steps are evaluating on approved real calls, measuring latency in the live workflow and finishing the guardrail work before the copilot goes into production.
The project is one of several early deployments of Scale Post-Training, alongside work with Skydance on finance automation and EY on M&A due diligence. Scale Post-Training is available now for enterprises that want to customize open models, including the Nemotron families, for their own workflows. The post was published alongside an announced collaboration with NVIDIA on customizing Nemotron models.
Scale's own takeaway is that post-training does not guarantee improvement and that inference settings matter as much as training. A cheaper, faster configuration gave up less than two points of accuracy on fact extraction.
