LangChain Adds ScaleDown Models to Cut Cost of Long-Running Agents

Why context management is the bottleneck

According to LangChain, building long-running agents comes down to deciding which information goes into the context window, and when. That means summarizing old history, compacting large tool results and extracting the right data from long inputs. Each of these is a decision, and LangChain says such decisions have historically been made with LLMs, which is slow and expensive at the volume a long-running agent needs.

ScaleDown's constrained decoding models (CDMs) are meant to replace that step. A general-purpose LLM writes any string one token at a time. A CDM is trained for a single task and only returns answers that task allows, such as a label from a list or spans of the input. That lets the models be much smaller, faster and cheaper. ScaleDown reports its models are 200x faster and 467x cheaper than frontier models on these narrow tasks.

Four models, three middlewares

ScaleDown offers four models, each trained for one task:

• Summarize writes a short summary of long text.
• Compress cuts a block of context down to the parts relevant to a question.
• Extract pulls described fields out of text, each with a confidence score and its position in the source.
• Classify scores text against labels the user defines and returns a probability for each, with an optional natural language reason.

Every model accepts up to 1M tokens of input, so a full contract or an hour of agent history fits in one call without chunking. Extract, Summarize and Classify are also multimodal and take images and scanned PDFs directly. On ScaleDown's own benchmarks, measured against the average of GPT-5.5, GPT-5.4 Mini and GPT-5.4 Nano on public datasets, Summarize is 41x cheaper and 2.7x faster, and Compress is 138x cheaper and 1.8x faster, while scoring higher on accuracy for both.

The integration ships with three middlewares for LangChain and Deep Agents. The agent's own model still plans and writes code, while ScaleDown handles the context work around it.

How the middlewares work

Summarization. ScaledownSummarizationMiddleware replaces the built-in summarization middleware in Deep Agents. It uses the same trigger and keep options. Once history crosses the trigger, ScaleDown replaces older messages with one summary and recent messages stay as they are. By default the summary uses the same sections as LangChain's summary prompt: intent, summary, artifacts and next steps. If a ScaleDown call fails, the middleware leaves history untouched and the agent keeps running. It also works with LangChain's create_agent.

Tool result compression. ScaledownToolCompressionMiddleware adds an optional compression_query parameter to tools. When the model needs only part of a result, it fills in the query, for example "refund policy for EU orders". The middleware strips it before the tool runs, then ScaleDown compresses the output before the model sees it. Tools do not need to change, so this works with any tool, including MCP tools. In one trace cited by LangChain, a long docs page was cut by almost 90%. The compressed result starts with a note so the model can call the tool again without compression_query to get the full output.

Extraction. ScaledownExtractionMiddleware lets developers describe fields in plain language, such as an order number, an error message or a customer email. Before the first model call of a run, it sends the user's messages to Extract and saves the fields in agent state under scaledown_extraction, where tools can read them. A field can also be a set of labels answered by Classify, for example to route a ticket by issue type. LangChain says this avoids cases where the main model paraphrases an error code or mistypes an email address.

What it means for developers

ScaleDown is now supported in the LangSmith LLM Gateway, so its calls are traced and count toward a team's spend and rate limits. LangChain also compares ScaleDown Classify to the decision model Jev: Jev takes text only, up to 32K tokens of state per question, while Classify takes up to 1M tokens, includes images and adds a one-sentence reason to each decision. According to LangChain, that reason makes it easier to debug an agent that routes a ticket or stops a run based on a label.

For teams running agents over long sessions, the pitch is to keep a frontier model for reasoning and offload repetitive context chores to smaller specialized models. The speed and cost figures are ScaleDown's own claims from its benchmarks, so results in production workloads may differ.