Langfuse launches open source AI Gateway in public beta
Langfuse has launched its AI Gateway in public beta, offering companies a central entry point for model traffic, usage tracking and access governance. The MIT-licensed gateway can record requests as Langfuse traces, while allowing organizations to choose how much data is logged.
- Langfuse AI Gateway is now available in public beta.
- The core gateway is MIT-licensed and can be self-hosted.
- OpenAI and Anthropic are supported in the beta.
- Logging can include metadata, costs, prompts and responses, or nothing.
- Langfuse reports less than 3ms median latency with tracing enabled.
A central layer for company-wide AI use
Langfuse says the way companies use AI has expanded beyond a single chatbot managed by one team. Coding agents, internal assistants, workflow tools and custom AI applications can create model traffic across engineering and business teams, making it harder to identify who uses which models, what that usage costs and whether it meets security requirements.
AI Gateway is designed to address that problem by placing a central layer in front of model calls. Every key can be linked to a person or service and can include a team or cost center, allowing spending to be broken down according to a company's organization. Administrators can also approve models centrally, set spend limits per key and receive alerts before unexpected bills occur.
The gateway is available through Langfuse Cloud or can be run on a company's own infrastructure. Langfuse says its core gateway is MIT-licensed and can be deployed as a single 22MB Docker image alongside an existing Langfuse instance, with no extra dependencies.
Tracing, logging and supported providers
Unlike a gateway focused only on token and cost accounting, Langfuse turns calls into full traces when tracing is enabled. Teams can search, evaluate and annotate those traces, and can use Langfuse data through MCP and CLI access. OpenTelemetry distributed tracing can merge additional traces into Langfuse, including details such as tool-call executions from complex agents.
Logging is configurable. By default, the gateway records metadata, usage and cost, but not prompt or response content. Users can choose to record usage and cost only, full prompts and responses, or nothing at all. Pass-through mode records nothing, while self-hosting keeps data on the organization's infrastructure.
The public beta supports OpenAI and Anthropic. Langfuse says OpenRouter, AWS, GCP, Azure and custom inference APIs will follow soon. Requests and responses remain in the provider's native format, and the company says users only need to change the base URL and use a gateway key.
Performance, pricing and the roadmap
Langfuse says the gateway is built in Rust and delivers less than 3ms median latency with tracing enabled. The company also reports support for more than 1,000 concurrent streams on a single 2 vCPU node, with memory usage scaling in line with load.
Self-hosting is free, according to Langfuse. In Langfuse Cloud, the company currently charges only when customers choose to log requests, while non-logged requests are free. Customers keep their provider contracts, rates and compliance agreements, and Langfuse says it does not resell tokens.
The announced roadmap includes spend limits by team, user or API key, provider fallbacks, guardrails and PII redaction, SCIM user provisioning, additional providers and automatic model routing. Langfuse says the gateway is available to try today in public beta. Administrators can set it up for an organization, create a gateway key in Organization settings, and direct an application or coding agent to a regional gateway host.
Langfuse: Langfuse launches AI Gateway - Langfuse
