Databricks AI Search Now Indexes Streaming Tables, Materialized Views and Iceberg

Three new sources for AI Search

Databricks says AI Search now natively supports three table types beyond Delta tables: Streaming Tables, Materialized Views, and Managed tables using Iceberg v3+. The company positions the change as a way to support real-time intelligence, personalized experiences and cross-engine AI applications.

Each source maps to a different kind of workload, according to the company:

• Delta tables (existing): data and analytical workloads, with a broad range of analytics and AI applications.
• Streaming Tables: continuously arriving events, such as fraud detection, security monitoring and real-time IoT analytics.
• Materialized Views: custom aggregated and joined data, for personalization, session-aware search and agent assist.
• Managed tables using Iceberg: open-format data across engines, for cross-engine agents, unified knowledge search and open-format data intelligence.

How each source is used

For Streaming Tables, Databricks describes continuously changing data such as new listings, inventory, availability and user activity flowing into an append-only table. AI Search indexes new records as they arrive and keeps the search index up to date. The company gives examples such as helping users discover newly available restaurants through a search bar, or generating real-time recommendations based on recent trends.

Materialized Views can aggregate events, filter records and join data from multiple sources to build customized state. As the underlying data changes, the view is incrementally maintained and AI Search propagates those changes to the index. Databricks says this enables session-aware recommendations, product and content discovery based on recent activity, and results tailored to a user's evolving interests and preferences.

Managed Iceberg tables can serve as AI Search sources directly, without conversion or migration to another table format. Databricks says this allows search across product catalogs managed by different systems, unified business or customer data across Managed Iceberg tables, and search over large, distributed datasets without moving the underlying data.

No more intermediate Delta layer

Before this update, customers had to create a separate Delta representation of Streaming Tables, Materialized Views or Managed Iceberg tables and build pipelines to keep it synchronized with the source. Databricks says this added an extra data layer, custom update logic and ongoing operational overhead.

Now AI Search connects directly to these sources. Users create an index from the source and sync it when needed, and AI Search propagates the changes to the index. According to Databricks, this eliminates the intermediate Delta table and the custom synchronization pipeline, with the platform handling the plumbing behind the scenes.

Same workflow, more data

The setup stays the same whether users work in the UI or through the API. Building an AI Search index follows the workflow already used for Delta table sources: create a Delta Sync index and point it at the source table. Databricks says the same workflow now extends to Streaming Tables, Materialized Views and Iceberg tables, so there is no new workflow to learn.

For teams already running AI Search on Delta tables, the announcement suggests they can bring event streams, precomputed personalized state and open-format data into search without extra pipeline work. Details are in the AI Search documentation, which Databricks points readers to.