Red Hat Details EnMaaS Pilot for Open Source AI Inference

A single gateway for enterprise model access

EnMaaS was created to give Red Hat engineers governed access to a broad mix of models without requiring separate provider accounts and onboarding processes. Through a streamlined, SSO-authenticated portal, developers can self-enroll and generate one unified API key for commercial frontier APIs and internally hosted open weight models.

Red Hat says thousands of developers route their daily coding harnesses through the platform. Administrators can configure model availability for specific teams, including access to frontier models from Anthropic and OpenAI alongside open weight models such as GLM, served through llm-d in partnership with IBM Research.

The gateway is designed to support commercial tools, open source harnesses and emerging agent frameworks. It also provides token tracking, quotas and rate limits across users, teams and models, giving administrators a way to manage consumption and assigned budgets.

Praxis and llm-d form the open source architecture

Praxis is a framework for building proxy servers that also provides an AI gateway. Red Hat built it in Rust and says it runs as a single process on the data plane. Rather than chaining multiple proxies or external sidecar containers, Praxis consolidates request processing, including prompt inspection, body-based routing, API translation and content guardrails.

A policy engine contributed by IBM researchers enforces organizational security and safety policies. Praxis was contributed to the Cloud Native Computing Foundation in September 2026.

Behind the gateway, llm-d provides distributed inference for open weight models. The Kubernetes-native framework, co-founded by Red Hat and contributed to the CNCF in March 2026, works with vLLM to coordinate model servers. Its capabilities include runtime scheduling based on live load, predicted latency and prefix affinity, along with KV-cache tiering across GPU, CPU and NVMe storage.

Red Hat says the infrastructure served a dozen open weight models across hundreds of GPUs. It also supports prefill and decode disaggregation and multi-node expert parallelism for mixture-of-experts models.

Internal usage shifted toward open weight models

Red Hat describes itself as its own “customer zero”, using EnMaaS internally to test and refine the architecture before bringing components to the open source community. The combined environment processed more than 1 trillion tokens in a single 30-day window, according to the company.

Red Hat says consumption initially favored commercial frontier APIs by 70/30, but later shifted to 30/70 in favor of internally hosted open weight models. The company attributes that change to intelligent routing and frictionless access to hosted models.

According to Red Hat, moving workloads from frontier models to self-hosted models running on llm-d reduced effective token spend by 40% to 70%, depending on prompt caching efficiency and model routing. Praxis can also automatically fall back to hosted open weight models when a team or developer reaches a commercial monthly token budget.

Security controls extend to agent workflows

Red Hat says its development teams configured agents to access models through Praxis and then secured them with OpenShell, an open source agent runtime. OpenShell provides an isolated runtime boundary around proxy operations and additional controls over agents’ access to local files, credentials and external networks.

The EnMaaS pilot shows how Red Hat is combining a centralized gateway, distributed open weight model serving and policy controls for internal software development. For users, the approach promises a single access path and uninterrupted workflows when commercial limits are reached. For organizations, Red Hat presents it as a way to increase open weight model adoption while improving visibility and control over inference costs.