本页尚未翻译

本页内容仍为英文原文，源自官方文档 curation/vend-secret。中文翻译正在分批进行中。

Vend a Secret to a Function

Bind a Secret to a Function by reference, and how its value reaches running code during SQL evaluation and refresh.

Applies to: Embedded ❌, Enterprise ✅

Early access

A Function that calls an external service needs a credential without putting it in the function's source -- see Create a Function for how to register one. A Secret is a database-scoped named credential: you create it once, then bind it by reference -- by name, not by value -- when you register the Function that needs it.

Specify the Secret when you register the Function

Store the credential, then bind it to an environment variable the function's code already reads:

import os
from lancedb import EnvVarSecret, udf

db.create_secret("embedding-api-key", os.environ["EMBEDDING_API_KEY"])

@udf
class embed_remote:
    def __init__(self, model: str = "text-embedding-3-small"):
        import os
        from openai import OpenAI

        self.client = OpenAI(api_key=os.environ["EMBEDDING_API_KEY"])
        self.model = model

    def __call__(self, text: str) -> list[float]:
        response = self.client.embeddings.create(input=text, model=self.model)
        return response.data[0].embedding

version = db.create_function_async(
    embed_remote,
    secrets=[
        EnvVarSecret(secret_name="embedding-api-key", env_variable="EMBEDDING_API_KEY")
    ],
).wait()

The function's own code never names the Secret -- it reads EMBEDDING_API_KEY the same way it would outside LanceDB. The binding, in secrets=[...], is what connects that variable to the stored credential. It is checked for shape at registration (valid variable name, not reserved, not repeated) but not resolved yet: the Secret does not have to exist until a column actually uses this Function version, so you can create the Function and its Secret in either order.

Hugging Face models

For a model loaded from the Hugging Face Hub, store the token as a Secret and bind it to the variable the Hub client reads. Use a class Function so the model is loaded once per remote instance rather than once per row:

import os

from lancedb import EnvVarSecret, col, udf

db.create_secret("hf-token", os.environ["HF_TOKEN"])


@udf(
    name="embed_text",
    pip=["sentence-transformers==3.3.1"],
)
class EmbedText:
    def __init__(self, revision: str):
        from sentence_transformers import SentenceTransformer

        self.model = SentenceTransformer(
            "sentence-transformers/all-MiniLM-L6-v2",
            revision=revision,
        )

    def __call__(self, text: str) -> list[float]:
        return self.model.encode(text).tolist()


embed_fn = db.create_function(
    EmbedText,
    secrets=[EnvVarSecret(secret_name="hf-token", env_variable="HF_TOKEN")],
)

table = db.open_table("documents")
table.add_columns(
    {
        "embedding": embed_fn(
            text=col("text"),
            revision="<full-40-character-commit-sha>",
        )
    }
)
table.refresh_column("embedding")

A decorated class calls __init__ once per remote instance and __call__ for each row or batch. The class's __init__ parameters are binding initialization values, so revision is fixed for this binding and the model is reused for all rows handled by that instance. Instances are not shared across workers.

Use a full 40-character commit SHA for revision. A branch or tag can move; a commit pins the model version. See the official Hugging Face documentation for HF_TOKEN and downloading a specific revision.

HF_TOKEN authenticates Hub requests, but a cached model and a pinned revision are not a guarantee of offline execution. The Hub client may still request metadata or missing files, and authentication does not guarantee that requests will avoid rate limits.

How the value reaches a running evaluation

The Secret's value is resolved fresh for each job, never read once and cached in the Function's definition:

Step	What happens
add_columns with this Function version	Every bound Secret must now exist. The column pins this exact version, not a value.
A refresh starts -- refresh_column, or a materialized view's refresh	Every bound Secret must still exist, checked before any row is computed.
The job captures each bound Secret's current value	LanceDB looks up the value once, when the job starts, and sets it as that environment variable for every row the job evaluates, including its retries. No other environment variable carries over.

This is why rotating a Secret needs no new Function version and no column change: alter_secret changes the value a future lookup returns. The agent captures that value once, when a job starts, and reuses its own captured copy for that job's full run and any of its retries -- a refresh already running keeps the value it started with for every row, and only a job that starts after the rotation sees the new one.

Rotate and revoke

alter_secret and drop_secret never interrupt work in progress, so a routine rotation cannot fail a backfill halfway through:

Action	Running jobs	Next job	New Function version?
alter_secret	A running job and its retries keep the value captured at start; only a new job gets the latest value	Uses the latest value	No
drop_secret	Run to completion on the last value	Refused, naming the Secret	No
Revoke the key at its provider	Fails on their next call to the provider	Fails until a new value is set	No
Re-create a dropped name	Keeps the dropped Secret's last value	Picks up the new Secret	No

Because a rotation creates no new Function version, it also reaches columns pinned to a version registered before it. Binding a Function to a different Secret or variable does register a new version, and existing columns keep the binding they were declared with.

注意

To stop a credential from being used now, revoke it at its provider, for example by deleting the key in the OpenAI console. A value already handed to a running job cannot be recalled.

drop_secret cannot be undone, and it does not check for Functions still bound to the Secret.

说明

Reading a column this Function already filled needs only SELECT on the table. No Function runs on the read path, so a reader needs no grant on the Function or its Secrets -- only someone triggering a refresh does.

Vending during SQL evaluation specifically

A computed column's expression and a CREATE MATERIALIZED VIEW query both plan and execute as SQL, so a Function called from either one is vended its Secrets the same way: at the evaluation that fills the column or refreshes the view, not at declaration time.

CREATE MATERIALIZED VIEW frames_captioned AS
SELECT f.frame_id, caption(f.jpeg) AS caption
FROM frames f

caption here is a Function bound to a Secret through the Python SDK when it was registered -- SQL has no syntax for the binding itself, only for calling the already-registered Function. The CREATE MATERIALIZED VIEW statement's initial population, and every REFRESH MATERIALIZED VIEW after it, each look up the bound Secret's current value independently.

Troubleshooting

403 when adding or refreshing a Function column: you need USAGE on the Function and on every Secret it binds, in addition to the table privilege. The error names the object you are missing.

The Function runs but its provider rejects the key: a misspelled environment variable cannot be caught at registration, because nothing knows which variables a Function reads. Check that env_variable matches what the Function's code reads, such as EMBEDDING_API_KEY above.

Related guides

Create a FunctionWrite the function that reads the credential.SecretCreate, rotate, list and drop the credential itself.PrivilegesGrant and revoke privileges on a Secret.Refresh a computed columnRunning the evaluation that vends the Secret.