本页尚未翻译

本页内容仍为英文原文，源自官方文档 ingestion/import。中文翻译正在分批进行中。

Import Datasets

Create a table directly from a HuggingFace dataset or an existing Lance dataset in object storage, with one CREATE TABLE AS SELECT.

Applies to: Embedded ❌, Enterprise ✅

A CREATE TABLE ... AS SELECT (CTAS) reads directly from a HuggingFace dataset or an existing Lance dataset in object storage, once a storage integration covers that location -- the integration is never named in the statement, LanceDB matches it by URI prefix.

Import a HuggingFace dataset

CREATE TABLE imported AS
SELECT * FROM 'hf://datasets/owner/repo/data/train-*.parquet'

See Inspect Datasets for the full workflow, including querying a dataset directly without importing it first.

Import a Lance dataset from object storage

CREATE TABLE events_copy AS
SELECT * FROM 's3://customer-data/raw/events.lance'

An s3://, gs:// or az:// prefix reads an existing Lance dataset the same way as a managed table -- use this to copy a dataset another process already wrote in Lance format, not an arbitrary file format. See Storage Integration for GCS and Azure examples.

How the import runs

See Distributed Write for how an import's writes divide across plan executors, and when a small import instead runs as one local task.

说明

Importing requires a native database with database_manifest_enabled. Combining a HuggingFace source with a WAL-backed destination table in one CTAS is rejected, since the import cannot publish a base-only read that omits the destination's unflushed data.

What to read next

Define Storage IntegrationSet up the credential an import reads through.Inspect DatasetsQuery and import HuggingFace or object-storage datasets in full.Distributed WriteHow an import's writes divide across plan executors.Storage IntegrationThe complete SQL reference for every provider.