Baseten Launches BDN Sealed Volumes to Keep Model Data Consistent

The problem: model files that drift from their source

Baseten says that when there is no single source of truth for model files, data drifts from its source. Its example: a team fine-tunes a model and saves the checkpoint to cloud storage. Someone downloads a copy to a laptop to test it, then uploads it elsewhere to serve it. Each deployment, such as staging, production and testing, gets its own config file pointing to the checkpoint. If one person pins another revision, nobody notices until the outputs differ.

The company says the same issue affects LoRA adaptors, vector indexes, tokenizers, embedding indexes, lookup tables and any other model or deployment artifact. Baseten compares this to container images, which it says solved the problem for code a decade ago: build once, get a digest, run that digest anywhere. Sealed Volumes aim to do the same for model data.

How Sealed Volumes work

A volume is a named, versioned set of files delivered to deployments by the Baseten Delivery Network (BDN). Volumes live in namespaces, such as weights, and hold a series of versions. When baseten volume push or baseten volume sync finishes, BDN commits the files as a new version and seals it. A sealed version never changes and is identified by a digest of its contents, for example bdn:weights/qwen3-8b@b3:9f3c….

Publishers can add a mutable name with the --tag parameter. Pushing with --tag prod points prod at the new version. Baseten says the digest, not the tag, is what a deployment should pin, mirroring the split between a tag and an image digest in container registries. Because the digest sits in config files, version mismatches show up in a diff.

Baseten lists three benefits:

• Identical bytes: every deployment that references a sealed version mounts the same files read-only. To roll back, point config.yaml at an earlier digest.
• Incremental updates: BDN splits files into chunks and stores each once by content hash, so a new checkpoint sharing most bytes with the previous one uploads only the changed chunks.
• No download step at startup: the volume is mounted at a path set in config.yaml, and bytes stream in from the BDN cache tiers as the model reads them. According to Baseten, startup time no longer depends on volume size.

Publishing, sources and rollbacks

For a Hugging Face model, the sync runs as a durable background job on Baseten and nothing downloads to the user's machine. The example in the post syncs hf://Qwen/Qwen3-8B into bdn:weights/qwen3-8b:prod. When the job reports READY, it prints the sealed version's reference, which is then added under bdn.mounts in the deployment config. Deployment uses baseten model push. Baseten says there is no weights block to maintain and no credentials in the deployment.

Sync works the same way for Hugging Face, S3, GCS, Azure Blob, Cloudflare R2 and CoreWeave buckets. Private sources take a Baseten secret, and for S3 and GCS access can be granted with OIDC instead of stored credentials. A local directory can be published with baseten volume push, which uploads chunks in parallel and skips any BDN already holds.

Moving a tag does not change what running replicas serve, but new and scaled-up replicas mount the tag's new target. A deployment that mounts a tag can therefore serve two versions at once after a push. Deployments that pin a digest keep serving the old version until the source is changed and redeployed. A deleted version stays restorable for 7 days via baseten volume restore.

Beyond model weights and into training

Baseten says anything a model reads but does not write belongs in a volume. Examples include LoRA adapters, prebuilt engines such as a TensorRT-LLM engine, embedding indexes, vocabulary files, tokenizer configs, lookup tables, safety classifiers, evaluation data and prompt templates. The BDN Startup Cache, the first feature built on volumes, publishes a replica's compiled kernels and captured graphs as a Sealed Volume and restores it in later replicas.

For training, a job on a team's own cluster can publish its output with the CLI and an API key. For jobs trained on Baseten, checkpoints already sync to Baseten storage and can be synced into a volume without managing credentials. Because only changed chunks are uploaded, Baseten says a run that publishes a checkpoint every few hundred steps moves only a fraction of the model each time, and every version stays addressable.

On speed, Baseten says Sealed Volumes are read through the same three cache tiers as BDN weights: node disk, the in-cluster cache and the mirrored origin. A weights block pulls every file before the container starts, so startup grows with model size. A Sealed Volume, the company says, mounts in seconds, and the model reads from it at the same rate as from a weights mount.

What it means for users

For teams serving models on Baseten, the feature offers a single versioned source for checkpoints and other artifacts, with digests in config files making changes visible and rollbacks a matter of editing one line. It requires Baseten CLI 1.1.0 or newer.