本页尚未翻译

本页内容仍为英文原文，源自官方文档 search/hybrid-search。中文翻译正在分批进行中。

Hybrid Search

Combine vector similarity and keyword matches in a single ranked result set.

Applies to: Embedded ✅, Enterprise ✅

Hybrid search combines vector similarity with keyword matching. It is useful when a query needs both: for example, finding products with a particular model number whose descriptions match what the customer is looking for.

LanceDB runs a vector search and a full-text search, then merges their results with a reranker. Reciprocal Rank Fusion (RRF) is the default. It combines the positions of records in the two ranked lists.

Before you query, store vectors and create an FTS index on the text column.

Run a complete hybrid search example

These examples use three sample records and hand-written vectors, so no embedding service is needed. Each program creates a temporary local table and removes it when finished.

• Python: Install with pip install lancedb, save as hybrid_search.py, then run python hybrid_search.py.
• TypeScript: Install with npm install @lancedb/lancedb, save as hybrid_search.mts, then run node hybrid_search.mts with Node.js 22.18 or later.
• Rust: Download and extract the complete Rust examples, then run cargo run --locked --bin hybrid_search from the extracted directory.

table = db.create_table(
    "documents",
    [
        {
            "id": 1,
            "text": "A puppy playing in the park",
            "category": "animals",
            "vector": [1.0, 0.0],
        },
        {
            "id": 2,
            "text": "A dog resting at home",
            "category": "animals",
            "vector": [0.9, 0.1],
        },
        {
            "id": 3,
            "text": "A guide to growing tomatoes",
            "category": "gardening",
            "vector": [0.0, 1.0],
        },
    ],
)
from lancedb.rerankers import RRFReranker

table.create_index("text", config=FTS())
rows = (
    table.search(
        query_type="hybrid", vector_column_name="vector", fts_columns="text"
    )
    .vector([1.0, 0.0])
    .text("puppy")
    .rerank(RRFReranker())
    .limit(2)
    .to_list()
)
print(rows)

const lancedb = await import("@lancedb/lancedb");
const { mkdtemp, rm } = await import("node:fs/promises");
const { tmpdir } = await import("node:os");
const { join } = await import("node:path");
const uri = await mkdtemp(join(tmpdir(), "lancedb-search-"));
try {
  const db = await lancedb.connect(uri);
  const table = await db.createTable("documents", [
    {
      id: 1,
      text: "A puppy playing in the park",
      category: "animals",
      vector: [1.0, 0.0],
    },
    {
      id: 2,
      text: "A dog resting at home",
      category: "animals",
      vector: [0.9, 0.1],
    },
    {
      id: 3,
      text: "A guide to growing tomatoes",
      category: "gardening",
      vector: [0.0, 1.0],
    },
  ]);
  await table.createIndex("text", { config: lancedb.Index.fts() });
  const reranker = await lancedb.rerankers.RRFReranker.create();
  const rows = await table
    .vectorSearch([1.0, 0.0])
    .fullTextSearch("puppy", { columns: ["text"] })
    .rerank(reranker)
    .limit(2)
    .toArray();
  console.log(rows);

} finally {
  await rm(uri, { recursive: true, force: true });
}

use arrow_array::types::Float32Type;
use arrow_array::{FixedSizeListArray, Int64Array, RecordBatch, RecordBatchIterator, StringArray};
use arrow_schema::{DataType, Field, Schema};
use futures::TryStreamExt;
use lance_index::scalar::FullTextSearchQuery;
use lancedb::index::{Index, scalar::FtsIndexBuilder};
use lancedb::query::{ExecutableQuery, QueryBase};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let directory = tempfile::tempdir()?;
    let db = lancedb::connect(directory.path().to_str().unwrap())
        .execute()
        .await?;
    let schema = Arc::new(Schema::new(vec![
        Field::new("id", DataType::Int64, false),
        Field::new("text", DataType::Utf8, false),
        Field::new("category", DataType::Utf8, false),
        Field::new(
            "vector",
            DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), 2),
            true,
        ),
    ]));
    let batch = RecordBatch::try_new(
        schema.clone(),
        vec![
            Arc::new(Int64Array::from(vec![1, 2, 3])),
            Arc::new(StringArray::from(vec![
                "A puppy playing in the park",
                "A dog resting at home",
                "A guide to growing tomatoes",
            ])),
            Arc::new(StringArray::from(vec!["animals", "animals", "gardening"])),
            Arc::new(
                FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(
                    [[1.0, 0.0], [0.9, 0.1], [0.0, 1.0]]
                        .into_iter()
                        .map(|v| Some(v.into_iter().map(Some).collect::<Vec<_>>())),
                    2,
                ),
            ),
        ],
    )?;
    let reader = RecordBatchIterator::new(vec![Ok(batch)].into_iter(), schema);
    let table = db
        .create_table(
            "documents",
            Box::new(reader) as Box<dyn arrow_array::RecordBatchReader + Send>,
        )
        .execute()
        .await?;
    table
        .create_index(&["text"], Index::FTS(FtsIndexBuilder::default()))
        .execute()
        .await?;
    let query = table
        .vector_search(&[1.0_f32, 0.0])?
        .full_text_search(FullTextSearchQuery::new("puppy".into()).with_column("text".into())?)
        .rerank(Arc::new(lancedb::rerankers::rrf::RRFReranker::default()))
        .limit(2);
    let rows: Vec<RecordBatch> = query.execute().await?.try_collect().await?;
    println!("{rows:?}");

    Ok(())
}

The puppy record ranks first because both searches place it first. The query returns _relevance_score, with higher values first. This small example shows the mechanics; evaluate retrieval quality with embeddings and queries from your own application.

Generate query embeddings automatically

1. Setup

The following walkthrough uses Sentence Transformers in Python and OpenAI in TypeScript. Install the corresponding provider packages before starting. Python downloads a local model; TypeScript calls OpenAI. Install lancedb sentence-transformers pandas for Python, or @lancedb/lancedb apache-arrow openai for TypeScript. Set your OpenAI API key as described in the OpenAI embedding setup. Run the steps for your chosen language in order, choosing one connection option.

from lancedb.index import FTS
from lancedb.rerankers import RRFReranker
from lancedb.embeddings import get_registry
from lancedb.pydantic import LanceModel, Vector

import * as lancedb from "@lancedb/lancedb";
import "@lancedb/lancedb/embedding/openai";
import { Utf8 } from "apache-arrow";

2. Connect to LanceDB

Choose a LanceDB Embedded connection or use your Enterprise connection details.

Embedded

uri = "data/sample-lancedb"
db = lancedb.connect(uri)

const databaseDir = "data/sample-lancedb";
const db = await lancedb.connect(databaseDir);

Enterprise

Replace the placeholders with your Enterprise database URI, API key, region and endpoint. In application code, load credentials from environment variables or a secrets manager.

import lancedb

db = lancedb.connect(
    uri="db://your-database",
    api_key="your-api-key",
    region="us-east-1",
    host_override="https://your-endpoint",
    sql_host_override="grpc+tls://your-sql-endpoint:10026",
)

const db = await lancedb.connect("db://your-database", {
  apiKey: "your-api-key",
  region: "us-east-1",
  hostOverride: "https://your-endpoint",
});

3. Configure Embedding Model

Configure the model used for both document and query embeddings.

embeddings = get_registry().get("sentence-transformers").create()

const embedFunc = lancedb.embedding.getRegistry().get("openai")?.create({
  model: "text-embedding-ada-002",
}) as lancedb.embedding.EmbeddingFunction;

4. Create Table and Schema

Define the data structure for your documents, including both the text content and its vector representation.

class Documents(LanceModel):
    category: str
    text: str = embeddings.SourceField()
    vector: Vector(embeddings.ndims()) = embeddings.VectorField()

table_name = "hybrid_search_example"
table = db.create_table(table_name, schema=Documents, mode="overwrite")

const documentSchema = lancedb.embedding.LanceSchema({
  category: new Utf8(),
  text: embedFunc.sourceField(new Utf8()),
  vector: embedFunc.vectorField(),
});

const tableName = "hybrid_search_example";
const table = await db.createEmptyTable(tableName, documentSchema, {
  mode: "overwrite",
});

5. Add Data

Insert sample documents into your table, which will be used for both semantic and keyword search.

data = [
    {"text": "rebel spaceships striking from a hidden base", "category": "film"},
    {"text": "have won their first victory against the evil Galactic Empire", "category": "film"},
    {"text": "during the battle rebel spies managed to steal secret plans", "category": "film"},
    {"text": "to the Empire's ultimate weapon the Death Star", "category": "film"},
]
table.add(data=data)

const data = [
  { text: "rebel spaceships striking from a hidden base", category: "film" },
  { text: "have won their first victory against the evil Galactic Empire", category: "film" },
  { text: "during the battle rebel spies managed to steal secret plans", category: "film" },
  { text: "to the Empire's ultimate weapon the Death Star", category: "film" },
];
await table.add(data);
console.log(`Created table: ${tableName} with ${data.length} rows`);

6. Build Full Text Index

Create a full-text search index on the text column to enable keyword-based search capabilities.

table.create_index("text", config=FTS())
table.wait_for_index(["text_idx"])

console.log("Creating full-text search index...");
await table.createIndex("text", {
  config: lancedb.Index.fts(),
});
await table.waitForIndex(["text_idx"], 300);

7. Set Reranker [Optional]

RRF is the default. The examples below configure it explicitly. Other rerankers and their language support are described in Reranking.

reranker = RRFReranker()

const reranker = await lancedb.rerankers.RRFReranker.create();

8. Hybrid Search

Perform a hybrid search query that combines semantic similarity with keyword matching, using the specified reranker to merge and rank the results.

results = (
    table.search(
        "rebel spaceships",
        query_type="hybrid",
        vector_column_name="vector",
        fts_columns="text",
    )
    .rerank(reranker)
    .limit(10)
    .to_pandas()
)

print("Hybrid search results:")
print(results)

console.log("Performing hybrid search...");
const queryVector = await embedFunc.computeQueryEmbeddings("rebel spaceships");
const hybridResults = await table
  .query()
  .fullTextSearch("rebel spaceships")
  .nearestTo(queryVector)
  .rerank(reranker)
  .select(["text"])
  .limit(10)
  .toArray();

console.log("Hybrid search results:");
console.log(hybridResults);

9. Supply a vector and text separately

You can supply the vector and text separately. The vector must use the same model and dimensions as the stored embeddings. This example reuses the model configured above; the complete example at the top also demonstrates this pattern in TypeScript and Rust.

text_query = "rebel spaceships"
vector_query = embeddings.compute_query_embeddings(text_query)[0]
(
    table.search(query_type="hybrid")
    .vector(vector_query)
    .text(text_query)
    .limit(5)
    .to_pandas()
)

Control the candidate results

Use a result limit, metadata filters and vector distance bounds to control which candidates are available to the reranker. The examples below show each option on the query builder.

说明

Set limit() to the number of results your application needs. The default is 10. A larger candidate set can help a reranker find relevant records, but adds work.

Returning row IDs

Pass with_row_id(True) (Python) or withRowId() (TypeScript) to include the internal _rowid column in the results. This is useful for joining hybrid results back to a primary table, or for deduping across multiple queries:

results = (
    table.search("rebel spaceships", query_type="hybrid")
    .with_row_id(True)
    .limit(10)
    .to_pandas()
)
# results now contains a `_rowid` column alongside `_relevance_score`

const results = await table
    .query()
    .fullTextSearch("rebel spaceships")
    .nearestTo(queryVector)
    .withRowId()
    .limit(10)
    .toArray();

Bounding vector distance

distance_range(lower, upper) (Python) and distanceRange(lower, upper) (TypeScript) constrain the vector half of the hybrid query to the half-open interval [lower, upper). This is helpful when you want to cap how far semantic candidates can drift from the query vector before reranking:

results = (
    table.search("rebel spaceships", query_type="hybrid")
    .distance_range(lower_bound=0.0, upper_bound=0.4)
    .limit(10)
    .to_pandas()
)

const results = await table
    .query()
    .fullTextSearch("rebel spaceships")
    .nearestTo(queryVector)
    .distanceRange(0.0, 0.4)
    .limit(10)
    .toArray();

Either bound can be omitted to leave that side unbounded.

Prefilter vs. postfilter

A hybrid query applies its metadata filter to both the vector and full-text searches. Prefiltering, the default, restricts eligible rows before retrieval. Postfiltering removes candidates after retrieval and can leave fewer results than requested. See Filtering.

# Prefilter (default): filter applied before scoring
table.search("rebel spaceships", query_type="hybrid") \
    .where("category = 'film'", prefilter=True) \
    .limit(10) \
    .to_pandas()

# Postfilter: filter applied after the sub-queries return top-k
table.search("rebel spaceships", query_type="hybrid") \
    .where("category = 'film'", prefilter=False) \
    .limit(10) \
    .to_pandas()

// Prefilter (default): just call .where(...)
await table.query()
    .fullTextSearch("rebel spaceships")
    .nearestTo(queryVector)
    .where("category = 'film'")
    .limit(10)
    .toArray();

// Postfilter: chain .postfilter() after .where(...)
await table.query()
    .fullTextSearch("rebel spaceships")
    .nearestTo(queryVector)
    .where("category = 'film'")
    .postfilter()
    .limit(10)
    .toArray();

In Python, use explain_plan on a hybrid query to see whether the filter pushed into the scan or ran as a separate FilterExec step.

Choose and evaluate a reranker

Start with RRF, then compare alternatives on representative queries. Review which relevant rows reach the candidate set before evaluating how a reranker orders them. Record latency as well as relevance: increasing candidate counts or applying a model-based reranker adds work.

Reranking runs in the application for both Embedded and Enterprise connections. See Reranking for model-free fusion, model-based rerankers, custom implementations and score interpretation.

In synchronous Python, .rerank(...) accepts a reranker and a normalize option ("score" or "rank"). Normalization matters for methods that blend raw scores; RRF uses rank positions. These are Python query options, not a shared signature across every SDK.