LangChain lifts coding agent from Top 30 to Top 5 by changing only the harness

What LangChain changed and how it measured it

In a post by Vivek Trivedy, LangChain describes harness engineering as building tooling around a model to mold its "inherently spiky intelligence" for specific tasks. Design choices include the system prompt, tool selection and execution flow, with goals such as task performance, token efficiency and latency.

The team used Terminal Bench 2.0, which has 89 tasks across areas such as machine learning, debugging and biology. Runs were orchestrated with Harbor, which starts sandboxes through Daytona, interacts with the agent loop and handles verification and scoring. Every agent action was stored in LangSmith, along with latency, token counts and costs.

Of the many possible knobs, LangChain limited itself to three: the system prompt, tools, and middleware, its term for hooks around model and tool calls. The baseline, with a default prompt and standard tools and middleware, scored 52.8% with GPT-5.2-Codex, according to LangChain.

To find what to fix, the team built a Trace Analyzer Skill. It fetches experiment traces from LangSmith, spawns parallel error analysis agents whose findings a main agent synthesizes, and then aggregates the feedback into targeted harness changes. LangChain compares this to boosting, since it focuses on mistakes from earlier runs. A human can review proposed changes, though this is not required. The company warns that changes overfit to one task can cause regressions elsewhere, and says it plans to publish the skill.

Self-verification, environment context and loop detection

The most common failure LangChain found was that the agent wrote a solution, re-read its own code, judged it fine and stopped. To counter this, the team added system prompt guidance for a four-step approach: planning and discovery, build, verify, and fix. The prompt tells the agent to write tests if none exist, cover both happy paths and edge cases, and compare test output against what was asked rather than against its own code.

A PreCompletionChecklistMiddleware intercepts the agent before it exits and reminds it to run a verification pass against the task spec. LangChain likens this to a Ralph Wiggum Loop, where a hook forces the agent to keep working on exit.

Other changes focused on giving the agent knowledge of its environment:

• Directory context and tooling: a LocalContextMiddleware maps the working directory and related directories at start, and runs bash commands to find tools such as Python installations.
• Testable code: prompts tell the agent its work will be measured by programmatic tests, and that file paths in task specs must be followed exactly.
• Time budgeting: injected warnings nudge the agent to finish and move to verification, since agents are, in LangChain's words, "famously bad at time estimation".

A LoopDetectionMiddleware tracks per-file edit counts through tool call hooks. After N edits to the same file it adds context suggesting the agent reconsider its approach. LangChain says some traces showed 10+ small variations of the same broken approach. It notes the model can still continue down the same path, and calls the guardrail a heuristic that will likely become unnecessary as models improve.

The reasoning sandwich

LangChain also tuned how much compute the model spends on reasoning. gpt-5.2-codex has four modes: low, medium, high and xhigh. More reasoning helps evaluate each step but, according to the post, can burn over 2x more tokens and time, which matters under Terminal Bench timeouts.

The team chose an xhigh-high-xhigh "reasoning sandwich" as a baseline, spending more on planning and on final verification. Running only at xhigh scored 53.9% because of agent timeouts, compared with 63.6% at high. LangChain says trial runs showed no large differences between reasoning budget splits, so it kept the sandwich, which pushed the score to 66.5%.

The post adds that adaptive reasoning, where the model decides its own compute spend, is the natural approach and appears in Claude and Gemini models. In a multi-model harness, it suggests, a large model could plan and hand off implementation to a smaller one.

What it means for agent builders

The results suggest that, for agentic coding, gains can come from the system around a model as well as from the model itself. LangChain's stated view is that the harness engineer's job is to prepare and deliver context so agents can complete work autonomously. Its findings point to traces as a practical source of improvement, with self-verification and tracing singled out as especially helpful. The source text is cut off before the post's full list of practical takeaways.