LangChain: Open Models GLM-5 and MiniMax M2.7 Now Rival Closed Rivals on Agent Tasks
LangChain says open weight models such as GLM-5 and MiniMax M2.7 now perform similarly to closed frontier models on core agent tasks like file operations, tool use and instruction following, at a fraction of the cost and latency. The claim rests on early results from the company's Deep Agents harness evaluations.
- LangChain says GLM-5 and MiniMax M2.7 score similarly to closed frontier models on core agent tasks in its Deep Agents evals.
- GLM-5 scored 0.64 correctness (94 of 138) versus 0.68 for Claude Opus 4.6, 0.65 for Gemini 3.1 Pro Preview and 0.61 for GPT-5.4.
- LangChain estimates 10M output tokens per day costs roughly $250 on Opus 4.6 versus about $12 on MiniMax M2.7.
- OpenRouter data cited shows GLM-5 on Baseten at 0.65s latency and 70 tokens/second, against 2.56s and 34 tokens/second for Claude Opus 4.6.
- The Deep Agents CLI can switch models mid-session, for example frontier for planning and open for execution.
What the evals show
Over several weeks, LangChain ran open weight large language models through its Deep Agents evaluations. According to the company, GLM-5 (from z.ai) and MiniMax M2.7 each score similarly to closed frontier models on file operations, tool use and instruction following. LangChain describes tool calling as reliable and instruction following as consistent, and says this makes real-world production workflows more viable.
The test suite covers seven categories: file operations, tool use, retrieval, conversation, memory, summarization and unit tests. LangChain reports four metrics: correctness, solve rate, step ratio and tool call ratio. The company stresses that these are early results and that it is expanding the eval set.
On overall correctness, GLM-5 on Baseten scored 0.64 (94 of 138 tests passed) and MiniMax M2.7 scored 0.57 (85 of 138). Among closed models, Claude Opus 4.6 scored 0.68 (100 of 138), Gemini 3.1 Pro Preview 0.65 (96 of 138) and GPT-5.4 0.61 (91 of 138). So GLM-5 landed between Gemini 3.1 Pro Preview and GPT-5.4 on this measure, while MiniMax M2.7 trailed the closed group.
Conversation was the weakest category across the board. GLM-5 scored 0.38 and MiniMax M2.7 0.14, while the closed models ranged from 0.05 for Claude Opus 4.6 to 0.29 for GPT-5.4. File operations were at or near 1 for all models tested.
Cost and latency gap
LangChain argues that cost and latency are the main reasons teams look at open models. It says closed frontier models can run 8 to 10 times more expensive for high-throughput workloads and are often too slow for interactive products.
The pricing table in the post lists these rates per million tokens:
- Claude Opus 4.6: $5.00 input, $25.00 output
- Claude Sonnet 4.6: $3.00 input, $15.00 output
- GPT-5.4: $2.50 input, $15.00 output
- GLM-5 (Baseten): $0.95 input, $3.15 output
- MiniMax M2.7 (OpenRouter): $0.30 input, $1.20 output
LangChain calculates that an application outputting 10M tokens per day costs roughly $250 per day on Opus 4.6, versus about $12 per day on MiniMax M2.7, a difference of about $87k per year.
On speed, the company cites OpenRouter data showing GLM-5 on Baseten averaging 0.65s latency and 70 tokens per second, compared with 2.56s and 34 tokens per second for Claude Opus 4.6. LangChain notes that providers such as Groq, Fireworks and Baseten optimize for latency and throughput beyond what most teams could achieve themselves.
Using open models in Deep Agents
According to LangChain, switching to an open model in the Deep Agents SDK takes a one-line change. For GLM-5, developers pass the model name baseten:zai-org/GLM-5 to create_deep_agent. For MiniMax M2.7, they use openrouter:minimax/minimax-m2.7. The same model is often available from several providers, for example GLM-5 through Baseten, Fireworks, or Ollama for self-hosting. The providers tested for this release are Baseten, Fireworks, Groq, OpenRouter and Ollama (cloud).
The harness adapts to each model. It injects the model's name, provider, context limit and supported modalities into the system prompt, and it adjusts compression, offloading and summarization thresholds to the model's actual context window. LangChain gives the example that a model with a 4K context gets more aggressive compaction than Opus with 1M.
Developers can also reproduce the evals locally. LangChain's CI runs the suite across 52 models in groups, including an open group of GLM-5, MiniMax M2.7 and nemotron-3-super, and a pytest command can target a model group or a single model.
Mixing frontier and open models
The Deep Agents CLI, which LangChain calls its open-source coding agent and an alternative to Claude Code, supports runtime model swapping through a new ConfigurableModelMiddleware. Users can switch models mid-session with the /model slash command. LangChain says this allows patterns such as planning a task with a frontier model and then handing execution to a cheaper open model.
For teams building agents, the post's message is that open models are a practical option alongside closed ones, particularly where cost and response time matter. The results are early, and the weaker conversation scores and the gap between MiniMax M2.7 and the closed models show that the picture varies by task and model.
LangChain: Open Models have crossed a threshold
