AI21 Labs launches Teel, a Python runtime for faster agent workflows
AI21 Labs has introduced Teel, an open source Python runtime designed to overlap independent, IO-bound calls in ordinary sequential Python code. The company says Teel reduced runtime by 83% on average in an unmodified Tree of Thoughts workload.
- Teel is an open source Python runtime from AI21 Labs.
- It overlaps independent IO-bound calls in sequential code.
- AI21 reports an 83% average runtime reduction on Tree of Thoughts.
- Teel currently requires Python 3.12.13 exactly.
- The project uses @cacheable, @jit and optional @inline decorators.
A runtime that finds parallel work
Teel is designed to reduce the amount of concurrency plumbing developers, and AI agents generating code, have to write. Developers mark expensive functions with @cacheable and orchestration functions with @jit. Inside a with Teel(): context, the runtime reads ahead through a function's Python bytecode and looks for calls whose arguments are already known.
When it finds an independent call, Teel dispatches it on a background thread before the main thread reaches that line. The approach is intended for workloads involving model calls, retrievals, web requests, tool calls and shell execution, where much of the elapsed time can be spent waiting on external systems.
AI21 says Teel does this without requiring async, await, futures or gather. Outside the Teel context, decorated functions run as plain Python.
Performance results from two agent workloads
AI21 applied Teel to Princeton's Tree of Thoughts implementation on Game of 24 samples with GPT-4, without editing the repository. The company says average runtime fell by 83%, from 148s to 24s at the low end and from 307s to 38s at the high end. The longest run therefore showed an 8.07x speedup.
In a separate test, AI21 forked LangChain's Deep Agents and replaced its asynchronous JavaScript execution with synchronous Python running under Teel. Across ten mixed web-search-and-LLM prompts, the company reports an average runtime of 58s, compared with 71s for asynchronous JavaScript and 150s for the same Python run sequentially. AI21 says the comparison covers distributions of generated programs because the same prompt can produce different code each time.
AI21 also describes a code-review agent example. Its sequential version took 7.1s, while a manually rewritten asyncio version took 3.0s and the original code with Teel's decorators took 2.9s. The company says both faster versions begin possible follow-up actions before a verdict is available, but Teel identifies and manages that behavior automatically.
API and current requirements
Teel provides three decorators and a context manager. @cacheable marks expensive, parallelizable steps and memoizes results using a key built from json.dumps of the arguments, so those arguments should be JSON-serializable. @jit identifies orchestration functions that Teel may analyze, while optional @inline marks cheap helpers that can be executed during lookahead.
AI21 says Teel currently requires Python 3.12.13 exactly. The project is open source, and the company directs developers to the repository at github.com/AI21Labs/teel. For users building agents, the stated benefit is that fan-out across independent calls can be added by marking existing sequential code, rather than rewriting the call chain around asynchronous functions.
