ZenML Pro introduces Bots for automated ML and AI engineering work
ZenML has introduced Bots in ZenML Pro, a way for teams to delegate investigations, fixes and experiments to AI engineers working across repositories, ZenML runs and Kitaru agent sessions. Bots can be started on demand or scheduled for recurring reviews, with results prepared for human review.
- Bots can investigate failed runs and prepare draft pull requests with supporting evidence.
- Kitaru integrations let Bots compare agent changes against recorded sessions and baselines.
- Automations schedule recurring pipeline, agent and cost investigations.
- Each bot uses a separate ZenML Pro service account and configurable permissions.
- ZenML reports a 3.3 percentage point balanced-accuracy advantage in one claims-agent benchmark.
From failed runs to reviewable fixes
Bots are designed to handle work that often follows a failed pipeline run or an underperforming AI workflow. A team can ask a bot to compare a failed training run with the last successful run, inspect pipeline metadata, artifacts, configuration and repository code, then open a draft pull request with a proposed fix. With an authorized rerun, the bot can continue the investigation through a new run that tests the change.
ZenML says the output includes both the code change and the execution evidence behind it. Results are recorded as artifacts such as code diffs, pull requests, reports, charts, comparisons and links to relevant runs and model versions. For document-processing workflows, a bot can also explain why two runs produced different outputs alongside a targeted code change.
Testing agent behavior with Kitaru
The Bots workflow also covers AI agent improvement through Kitaru. Bots can inspect recorded agent sessions, investigate repeated tool calls or unnecessary model requests, test a proposed change and compare it with a baseline. Teams can ask for an experiment that reports quality, cost and latency trade-offs using an agreed evaluator.
This approach can help assess whether a cheaper model is suitable for a particular workload. A team can select sessions and define quality criteria, then review the comparison before deciding whether to roll out a model or prompt change.
ZenML describes one benchmark involving a claims-review agent across eight batches of 100 real California health appeals. The test case files were generated from public appeal decisions. Across the 800 appeals, the AI-engineer-improved agent scored 3.3 percentage points higher in balanced accuracy than the Codex-improved agent. After the agent switched to a cheaper open model, the AI engineer used recorded session endings to find appeals left undecided because the agent ran out of tokens or model requests, then raised the limits and tested the change.
The company cautions that the comparison used different setups. The AI engineer used Kitaru and a specific instruction to count session endings, while Codex received a general improvement request. ZenML says an earlier run without that instruction found no clear difference, and describes the result as one benchmark rather than a general ranking of coding agents.
Recurring investigations and controlled access
Automations let teams run a configured bot with a fixed prompt on a schedule. A daily task could review the previous day’s failed pipeline runs, investigate recurring failures, link affected runs and open draft pull requests when a code fix is clear. A weekly task could review AI workflows and agent sessions for avoidable work, rank proposed improvements and identify a bounded experiment.
Teams define the scope and actions each recurring task is authorized to take. ZenML presents this as a practical FinOps workflow for finding waste, proposing a change and comparing cost and quality before adoption. Each scheduled execution has its own chat and recorded outputs. Work started in Slack or GitHub returns to its associated thread, and bots can send Slack notifications when attention is needed.
Each bot uses its own ZenML Pro service account, with roles determining which ZenML and Kitaru workspaces and resources it can access. Separate permissions govern who can use and manage the bot. Bots run in isolated sandboxes configured with repositories, setup commands, team instructions and skills. Connectors support services including GitHub, Notion, Context7, Firecrawl and Tavily, while custom MCP servers are also supported.
What Bots mean for ZenML Pro teams
ZenML is positioning Bots as a repeatable way to prepare investigations, proposed fixes and evaluation results for engineers to review. The same configuration can be reused across engineers, scheduled tasks and requests from the ZenML Pro dashboard, Slack, GitHub, SDK or API.
The company recommends starting with one recurring responsibility, such as investigating failed ML pipelines, reviewing AI workflow regressions or finding an agent improvement worth testing in Kitaru. Teams can configure the relevant repositories, skills and permissions, review the first results and schedule the task once its scope is clear. The model keeps decisions about what to ship with engineers while shifting the preparation and evidence-gathering work to an always-on bot.
