Twilio Launches Video Media SDK in Public Beta to Connect Live Video to AI

How the SDK works

The Twilio Video Media SDK lets an application join a Twilio Video Room as a secure, virtual participant. Twilio says this design allows a backend to receive individual audio and video tracks directly, without complex workarounds or client-side rendering.

The company lists three main benefits:

• Secure backend access to media: individual tracks can be routed to a secure backend, which keeps sensitive information private and can support security and privacy requirements, including use cases subject to HIPAA and GDPR.
• AI logic across channels: teams can reuse the same AI logic, such as transcriptions or sentiment analysis, across chat, messaging, voice and video, so customers and support teams do not lose context when switching channels.
• More room for experiment: developers can test new AI models, prompts and workflows without needing to rebuild their video systems.

Why Twilio says video still matters

Twilio argues that many customer conversations are now handled by AI agents over phone or chat, but some moments still call for a human connection. When someone needs to see what is happening, show something or fully explain a problem, the company says video is the most personal and effective way to help.

The SDK is positioned as a way to keep those video calls personal while adding AI features, keeping private information secure and making support teams more effective. Twilio also says customers need no extra setup.

Use cases and an early adopter

Twilio describes four types of applications that become possible when a backend acts as a virtual participant in a call:

• Multimodal AI assistants: conversational agents that connect live audio and video to frontier AI models, so the AI can both see and hear users and keep context across SMS, voice and video.
• Ambient healthcare documentation: streaming consultation audio to clinical transcription engines in real time to generate HIPAA-compliant clinical notes directly in EHR systems.
• Remote verification and field inspection: feeding live video to custom computer vision models for real-time KYC identity checks, damage assessments or object detection.
• Server-side quality and compliance checks: running visual inspections, interpreter appearance audits and content moderation server-side, without draining device battery or affecting call performance.

Early adopter GLOBO Language Solutions, which works on language access in healthcare, uses the SDK for real-time quality, compliance and AI integration in patient care. Francesca Mayr, Senior Vice President of Innovation and Experience at GLOBO, said the SDK lets the team "securely leverage on-demand video in highly demanding regulatory environments like healthcare, and experiment with the latest AI tools."

What it means for developers and businesses

Twilio calls the Video Media SDK "a foundational step" toward customer journeys where AI agents, human experts and real-time communications work together across every channel. For teams already building on Twilio Video, the SDK offers a route to send call media to their own AI systems while keeping it on a secure backend.

The product is in Public Beta. Twilio points interested developers to its documentation and quickstart, and invites feedback, use cases and partnership requests.