LeRobot Now Trains on Lance Data Straight From Object Storage

LeRobot training without a local copy

LanceDB's newsletter, published October 8, 2026, leads with a robotics training result. LeRobot now reads Lance datasets natively, so training streams frames from object storage instead of requiring a download first. The company says the setup provides a true global shuffle.

The test used all of DROID, which comprises 27.6M frames and 369 GB. Running 10k SmolVLA steps on 8 H100s took 1h27m with Lance, against 2h00m for the standard reader on a local NVMe copy. LanceDB reports identical loss in both runs. GPUs waited on data 1.7% of the time with Lance, compared with 37.4% for the standard reader.

Mining fleet data and scaling vector search

A second post, by Lei Xu, addresses the data mining problem in physical AI. Rare failures sit in the long tail of fleet logs, and the signal needed to find them is rarely stored as a column. The walkthrough works on a single LanceDB table: it derives an is_left_turn flag from yaw rate and speed, counts pedestrians per clip with a Faster R-CNN detector, then runs a clip-embedding vector search filtered by city, hour, turn and pedestrian count.

The third featured post covers vector search growth from 10 million to 10 billion vectors. According to LanceDB, five-bit RaBitQ reached 96.2% recall at 1,614 QPS on 10M 768d vectors, while PQ384 reached 91.5% at 1,047 QPS. At 10B vectors, distributed search held an 18.05 ms p50 and reached 1,066 QPS at 256 concurrent requests.

The company also says a one-bit distance bound prunes over 99% of candidates before full scoring. Switching from fast to normal at query time lifts recall from 75.0% to 93.1% with no index rebuild.

Enterprise updates and open source releases

The newsletter lists several performance changes in LanceDB Enterprise:

• Indexing workers now preload up to 512MiB of shuffle offsets by default. On a 1B-row IVF_RQ shuffle replay, merge time fell from 61 to 13.4 minutes and peak memory from 14.2GiB to 9.6GiB.
• Function worker cold start improved: a 10,000-row column refresh with a 3.0 GB torch + CUDA image fell from 409 s to 54 s with an empty cache on ext4.
• A NumPy multiply over a nullable float64 column ran 1.61x faster per batch at 1,048,576 rows in Python UDFs.
• Table discovery in large databases now lists each namespace once per pass, rather than once per 1,000-table page.

New features include Secrets for Functions, native vector deduplication in SQL through vector_dedup and vector_duplicate_pairs, grouped materialized views, PAUSE JOB and RESUME JOB commands, and primary-key SQL DML on WAL-backed tables.

On the open source side, Lance v12.0.0 and LanceDB v0.39.0 both include breaking changes. In Lance, namespace merge_insert takes a list of key columns for composite-key upserts, and json_extract predicates no longer route to JSON indices. In LanceDB, get_job and job_history are replaced by open_job, describe_job and query_job_events. Lance also lists IVF index and full-text search improvements, and fragment reuse indexes that open 10, 2200× faster.

Talks, events and community

At Ray Summit 2026, Lei Xu and Hubert Yuan of Exa presented exa-d, Exa's framework built on Lance and Ray Data for reprocessing embeddings, extracted text and search signals across hundreds of billions of web pages. A recording is available. LanceDB also published a benchmark of 19 reranker configurations on five datasets.

Upcoming appearances include the Open Intelligence Summit on October 12-13, 2026, Long Horizon on October 20-21, 2026, and Reverie on November 5, 2026, all in San Francisco. The newsletter thanks contributors from Bytedance, Baidu, Adobe, Huawei, Tencent, DeepL and LumaLabs.

What it means

For teams training robot policies, the DROID result, as reported by LanceDB, suggests that streaming from object storage can match or beat a local copy while keeping GPUs far busier. The vector search and mining posts point to the same data layer being used for curation and retrieval at larger scale. These figures come from the company's own tests.