Lance and DuckDB Bring Vector and Hybrid Search Into SQL

Lance has introduced a DuckDB extension that brings vector, full-text and hybrid retrieval directly into SQL queries. The integration is designed to keep multimodal datasets, search results and analysis workflows within the same lakehouse environment.
- The Lance × DuckDB extension adds SQL access to Lance datasets.
- SQL table functions support vector, full-text and hybrid search.
- Users can join retrieval results with DuckDB tables and metadata.
- Search results can be written back as new Lance datasets.
- Cloud-hosted Lance data can be queried through configured DuckDB secrets.
Retrieval without a separate analysis pipeline
LanceDB says the main difficulty in building scalable retrieval systems is the fragmentation of the end-to-end data workflow. Embeddings may be stored separately from text, metadata and evaluation slices, while analysis can require copying data into another format.
Lance is designed as an open lakehouse format for multimodal AI. It spans the file format, table format and catalog specification layers, with the goal of providing a durable data plane for AI and machine learning on object storage.
DuckDB provides the compute layer in this setup. The new extension allows developers to query, scan and write Lance datasets directly from SQL, while keeping retrieval and analysis in DuckDB.
Vector, full-text and hybrid search as SQL functions
After installing and loading the extension, DuckDB can scan a Lance dataset by referencing a .lance path. The integration adds three SQL table functions: lance_vector_search, lance_fts and lance_hybrid_search.
These functions support vector search, full-text search and hybrid search. According to LanceDB, the operations push work into the Lance table layer, combining SQL access with Lance retrieval capabilities.
The hybrid search example combines a full-text query with a vector query through a linear combination of scores. Results include a _hybrid_score, along with distance and text-search score fields, which can be used for ordering.
The extension also supports joins between retrieval results and regular DuckDB tables. In the example, search results are joined with metadata containing data splits and quality scores, then filtered to an evaluation slice.
From cloud datasets to materialized evaluation slices
The extension supports both reading and writing. Teams can materialize the top results from a retrieval query and write them to a new Lance dataset. LanceDB presents this as a way to organize data slices, hard negatives and reranked features alongside source data.
Namespaces provide another organizational option. When a Lance namespace is attached, tables created with CREATE TABLE are managed by that namespace, similar to catalogs in other systems.
Cloud access uses DuckDB secrets. After configuring a scoped secret for an object-storage location such as an S3 URI, users can query a Lance dataset by its path. LanceDB describes the workflow as configuring access once and then treating the dataset path as a queryable input.
For users of the earlier Python and Apache Arrow integration, LanceDB says that workflow remains suitable for Python-first exploration and notebooks. The company says the new extension avoids the Arrow interface boundary, leaving more room for improvements in pushdown, parallelism and diagnostics.


