本页尚未翻译

本页内容仍为英文原文，源自官方文档 enterprise/distributed-search。中文翻译正在分批进行中。

Distributed Search

How LanceDB Enterprise distributes index execution, data reads, and caching across query nodes and executors.

Applies to: Embedded ❌, Enterprise ✅

LanceDB Enterprise separates query planning from index execution and data reads so that search can use the memory, CPU, and caches of multiple machines. A query node plans and coordinates a query. Executors search indexes, read table data, and return partial results for the query node to combine.

For the service components and overall request paths, see Architecture.

Search architecture

LanceDB Enterprise prioritizes horizontal scalability. Index execution on a single node can be limited by the memory available to cache the index for fast search and by the CPU available to execute queries. Dividing one logical index into multiple index segments lets each segment be distributed to an executor, cached independently, and searched in parallel with other segments.

The two serving components have different responsibilities:

Component	Responsibility
Query node (query planning + routing fleet)	Accept requests, plan queries, distribute work, and combine results.
Executor (executor + cache fleet)	Cache index structures and table data, execute subplans from the query node, and return partial results.

Eligible scalar index operations can also execute on executors. See scalar indexes and filtering for index types and query syntax.

For example, consider a vector index with six segments served by three executors. Each executor searches its two segments and returns its best 10 candidates. The query node compares the candidates returned by all three executors and selects the final 10. This example assumes enough matching rows and the default search path without additional reranking.

%%{init: {"theme": "base", "themeVariables": {
  "background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19",
  "primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19",
  "clusterBkg": "#f4ebe8", "clusterBorder": "#ad9e95", "edgeLabelBackground": "#ffffff",
  "fontFamily": "Inter, sans-serif", "fontSize": "18px"
}}}%%
flowchart TB
    Client["Application: vector search, limit 10"]
    QueryNode["Query node<br/>Plan, distribute, combine"]
    Index["One logical vector index<br/>Six index segments"]
    PlanExecutor1["Executor 1<br/>Cache and search segments A and B"]
    PlanExecutor2["Executor 2<br/>Cache and search segments C and D"]
    PlanExecutor3["Executor 3<br/>Cache and search segments E and F"]
    Merge["Query node<br/>Select final top 10 from up to 30 candidates"]

    Client --> QueryNode
    Index -.-> PlanExecutor1
    Index -.-> PlanExecutor2
    Index -.-> PlanExecutor3
    QueryNode -->|Search subplan| PlanExecutor1
    QueryNode -->|Search subplan| PlanExecutor2
    QueryNode -->|Search subplan| PlanExecutor3
    PlanExecutor1 -->|Up to 10 candidates| Merge
    PlanExecutor2 -->|Up to 10 candidates| Merge
    PlanExecutor3 -->|Up to 10 candidates| Merge
    Merge --> Result["Application receives up to 10 results"]
    classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19
    classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19
    class Merge accent

Tiered index cache

Executors cache index structures in memory, and extend that cache onto local disk when the disk tier is enabled. See Tiered Index Cache for how the two tiers work together and how the disk tier's budget is shared and sized; ask your deployment operator about enabling it on your cluster.

Create and query an IVF-RQ index

Use the regular table API. These examples assume table is an open Enterprise table with a vector column and the query vector is compatible with that column. For connection setup, see connect to LanceDB.

from lancedb.index import IvfRq

table.create_index("vector", config=IvfRq(num_bits=5, distance_type="cosine"))

# Run after the background index build is complete.
results = table.search(query_vector).distance_type("cosine").limit(10).to_arrow()

import { Index } from "@lancedb/lancedb";

await table.createIndex("vector", {
  config: Index.ivfRq({ numBits: 5, distanceType: "cosine" }),
});

// Run after the background index build is complete.
const results = await table
  .vectorSearch(queryVector)
  .distanceType("cosine")
  .limit(10)
  .toArray();

use futures::TryStreamExt;
use lancedb::index::{vector::IvfRqIndexBuilder, Index};
use lancedb::query::{ExecutableQuery, QueryBase};
use lancedb::DistanceType;

table
    .create_index(
        &["vector"],
        Index::IvfRq(
            IvfRqIndexBuilder::default()
                .num_bits(5)
                .distance_type(DistanceType::Cosine),
        ),
    )
    .execute()
    .await?;

// Run after the background index build is complete.
let results: Vec<arrow_array::RecordBatch> = table
    .vector_search(&query_vector)?
    .distance_type(DistanceType::Cosine)
    .limit(10)
    .execute()
    .await?
    .try_collect()
    .await?;

Enterprise builds indexes in the background. Submitting index creation does not mean the index is ready; check index status before evaluating indexed search. These examples explicitly select 5 bits per dimension. See the shared docs for RaBitQ and num_bits, index creation parameters, and vector search.

For the fast, normal, and accurate latency/recall modes, see tuning approx_mode. The Rust SDK currently exposes that query setting; the examples above use the default mode.

Distributed full-text search

FTS work is organized by inverted index. Compatible operations targeting the same table and text column can execute together on executors over that index's segments. Operations targeting different indexed columns can execute separately, with their results combined on the query node according to the query.

For example, suppose an articles table has separate FTS indexes on title and body. A query asks for rows whose title matches the exact term vector or whose body contains the phrase nearest neighbor:

1. The query node plans the title and body index searches separately.
2. Executors search the relevant segments of each index.
3. The query node combines the scored rows according to the cross-column query's semantics and returns the requested results.

Local and global scoring

Distributed FTS defaults to local scoring. Each executor uses statistics from its assigned segments to calculate BM25 scores. This avoids a separate statistics round, but document frequencies and average document lengths can differ between executors. Merging their scores can therefore produce a different ranking from scoring with statistics across the entire index.

Global scoring adds a statistics round before search. The query node collects and aggregates document counts, token counts, and query-term document frequencies across the target FTS index. Executors then score with the shared statistics before the query node combines their results. The statistics are global to that index, not blended across all text columns in the table.

Both modes combine results. The difference is the statistics used during scoring on executors; global mode does not rescore the returned rows on the query node. It trades an additional statistics round for consistent BM25 statistics across executors.

When term frequencies and document lengths are similarly distributed across index segments, local scoring can produce rankings comparable to global scoring. If segments contain substantially different document populations, prefer global scoring for consistent statistics across the index, at the cost of an additional statistics round and higher query latency.

Global scoring is a query node setting, configured at the deployment level.

See full-text search for query syntax and FTS indexes for index configuration.

Distributed data search

A table can be larger than any single node's local storage. Enterprise keeps the durable table in object storage and distributes data reads and caches across executors. An executor needs capacity for its cached working set rather than a complete local copy of the table.

With fragment-based data routing, reads for a data fragment go to a serving executor. Different fragments can be served and cached on different executors. The following example shows six fragments served by three executors:

%%{init: {"theme": "base", "themeVariables": {
  "background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19",
  "primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19",
  "clusterBkg": "#f4ebe8", "clusterBorder": "#ad9e95", "edgeLabelBackground": "#ffffff",
  "fontFamily": "Inter, sans-serif", "fontSize": "18px"
}}}%%
flowchart TB
    QueryNode["Query node<br/>Plan data reads and assemble results"]
    PlanExecutor1["Executor 1<br/>Read fragments 1 and 2<br/>Local NVMe disk cache"]
    PlanExecutor2["Executor 2<br/>Read fragments 3 and 4<br/>Local NVMe disk cache"]
    PlanExecutor3["Executor 3<br/>Read fragments 5 and 6<br/>Local NVMe disk cache"]
    Store[("Object storage<br/>Durable table fragments")]

    QueryNode -->|Read subplan| PlanExecutor1
    QueryNode -->|Read subplan| PlanExecutor2
    QueryNode -->|Read subplan| PlanExecutor3
    PlanExecutor1 -->|Cache miss| Store
    PlanExecutor2 -->|Cache miss| Store
    PlanExecutor3 -->|Cache miss| Store
    classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19
    classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19
    class Store accent

NVMe data caching

The data page cache stores pages read from table fragments on local disk, including NVMe SSDs. On a cache hit, the executor reads the page from local disk; on a miss, it fetches the page from object storage and fills the cache for subsequent reads. The cache holds accessed pages, not necessarily every byte of each fragment assigned to an executor. See Data Page Cache for how it's structured and sized.

Inspect execution

Use analyze_plan() (analyzePlan() in TypeScript) to understand how a query executes and troubleshoot slow searches. It runs the query and returns the execution plan with runtime metrics. Using the same table and query vector as above:

query = table.search(query_vector).distance_type("cosine").limit(10)
print(query.analyze_plan())

const query = table.vectorSearch(queryVector).distanceType("cosine").limit(10);
console.log(await query.analyzePlan());

use lancedb::query::{ExecutableQuery, QueryBase};
use lancedb::DistanceType;

let query = table
    .vector_search(&query_vector)?
    .distance_type(DistanceType::Cosine)
    .limit(10);
println!("{}", query.analyze_plan().await?);

Inspect the plan to check which indexes are used, which operations run on query nodes or executors, and where execution time is spent. Compare index search with data retrieval to identify expensive work. Repeating the query can also help distinguish cold-cache reads from warm-cache performance.