本页尚未翻译

本页内容仍为英文原文，源自官方文档 integrations/embedding/nvidia。中文翻译正在分批进行中。

NVIDIA Nemotron

Text embeddings with Nemotron 3.5 Embed 1B and 8B in the Python SDK.

The Python SDK registers nvidia for client-side text embedding with NVIDIA-hosted and self-hosted NIM endpoints. Documents use input_type="passage"; searches use input_type="query". This provider supports text only, with floating-point output. Null and empty text values get a null vector without a request; pass on_bad_vectors="null" or "drop" to add to keep or skip those rows. Image and combined text/image inputs are not supported by this integration.

Install the optional client and set your NVIDIA API key:

pip install openai
export NVIDIA_API_KEY="your-api-key"

from lancedb.embeddings import get_registry
from lancedb.pydantic import LanceModel, Vector

func = get_registry().get("nvidia").create(
    name="nvidia/nemotron-3.5-embed-1b",
)

class Document(LanceModel):
    text: str = func.SourceField()
    vector: Vector(func.ndims()) = func.VectorField()

db = lancedb.connect("./nemotron-db")
table = db.create_table("documents", schema=Document)
table.add([{"text": "Paris is the capital of France."}])
results = table.search("What is the capital of France?").limit(3).to_list()

For a LanceDB Enterprise table, connect as usual, for example lancedb.connect("db://my-database", api_key=..., region=...). The same schema, add, and search calls compute embeddings in the Python client. The NVIDIA key is separate from the LanceDB database key. Use a fresh table name when creating a table; reopening an existing table restores its embedding configuration.

Models and configuration

The hosted API documents these exact IDs and dimensions:

Model ID	Default dimensions	Supported dim
nvidia/nemotron-3.5-embed-1b	2048	128–2048
nvidia/nemotron-3.5-embed-8b	4096	1–4096

Parameter	Default	Description
name	nvidia/nemotron-3.5-embed-1b	Model ID sent to the endpoint.
dim	Model default above	Output dimensions; required for other served model IDs.
base_url	https://integrate.api.nvidia.com/v1	API base including /v1.
api_key	NVIDIA_API_KEY for the hosted API	Alternatively, a registry reference such as $var:nvidia-key.
batch_size	32	Maximum inputs per request. Reduce for your deployment's limits.
truncate	END	Truncate overlong text at the end, or use NONE to reject it.
max_retries	7	LanceDB's retry policy, applied to each request during automatic document/query embedding.

Requests specify modality="text" and encoding_format="float". Returned vectors are checked for count, input indices, dimensions, and finite numeric values. The hosted input limit is 10,240 tokens per item; self-hosted limits depend on the served profile. This provider batches by item count, not token count.

Self-hosted NIM

Set name to the exact ID returned by your deployment's GET /v1/models and dim to an output dimension that deployment supports:

func = get_registry().get("nvidia").create(
    name="nvidia/Nemotron-3.5-Embed-1B-BF16",
    base_url="http://localhost:8000/v1",
    dim=2048,
)

Authenticated endpoints

Custom endpoints may run without a key. NVIDIA_API_KEY is only sent to the hosted API, so for authenticated endpoints use get_registry().set_var("nvidia-key", key) with api_key="$var:nvidia-key". Registry references, rather than secret values, are stored in the schema. Restore registry variables in each process. Before using a custom endpoint restored from table metadata in a new process, call get_registry().allow_endpoint("http://localhost:8000/v1").

Output dimensions

The NIM 2.4 support matrix lists nvidia/Nemotron-3.5-Embed-8B-BF16 at 2048 dimensions, whereas the hosted 8B reference specifies 4096. For self-hosted model IDs, this provider requires an explicit dim and checks the response instead of assuming the hosted defaults.

SDK support

The NVIDIA provider is available in the Python SDK only. TypeScript, Rust and Java don't have an NVIDIA embedding provider.

References

NVIDIA documentation: hosted 1B API, hosted 8B API, NIM request/response format, NIM support matrix.