Salesforce Releases Agents That Improve AI Models and Code

Agents propose, test and retain improvements

SFR-AutoR&D receives a goal, a system to work on and a compute budget. Its agents then formulate hypotheses, implement methods and run controlled experiments against established baselines. Code or model improvements are retained only when they are supported by evidence and shown to be accurate and safe, according to Salesforce.

The system uses two agents. AutoR&D-Engineer searches for algorithmic and systems improvements in existing code, while TrainForge explores training methods and evaluates candidates against held-out benchmarks that are not used during training. Both run on AutoInfra, Salesforce's training backbone for experiments across shared GPU capacity.

Salesforce AI Research has used SFR-AutoR&D on public codebases including pandas, hnswlib and USearch. The company says the agents found further gains in all three projects and packaged the changes with benchmark evidence.

Improving retrieval in about 80 hours

Salesforce first used SFR-AutoR&D to improve SFR-Embedding V3, the model it uses for retrieval in Data Cloud. The team then directed the system at the open source model EmbeddingGemma 2, producing SFR-Embedding v4, which Salesforce says was measurably better in both quality and latency.

Evaluated on 10 public retrieval tasks in MTEB, the system improved the model's nDCG@10 score by 6.13 points. The metric measures the relevance of the top 10 results, giving greater weight to results ranked higher.

In a second session lasting roughly nine hours, the system focused on operating speed and cost. Salesforce says the resulting model delivered 36% more throughput and 17% lower latency, with almost no loss in quality. The complete improvement cycle took about 80 hours, compared with the several weeks that human-led embedding model research has historically required.

Embedding models convert text into numerical representations that help systems retrieve records, documents and case histories. In Salesforce's description, they determine what information an Agentforce agent retrieves through Data Cloud before answering a question grounded in a customer's data.

Human-set constraints remain part of the process

Salesforce says trust and safety requirements are built into the system's operating limits. Researchers set the goal, budget, evaluation and data constraints, while models intended for products use only datasets that have passed ethical and legal review. A model that performs well in evaluation must still meet production requirements before customers see it.

“The scientific method still governs, and the loop runs faster,” Silvio Savarese, Salesforce's EVP and chief scientist, said. Shafiq Joty, senior research director at Salesforce AI Research, said the agents are designed to “back those discoveries with rigorous experimental evidence.”

The company describes SFR-AutoR&D as an early example of enterprise systems that can propose and prove improvements before deployment. Salesforce also says its researchers are developing CodeEvolve, a related effort that uses evolutionary search with large language models to improve performance and code quality in Java and Salesforce Apex.