本页尚未翻译

本页内容仍为英文原文，源自官方文档 enterprise/distributed-compaction。中文翻译正在分批进行中。

Distributed Compaction

How LanceDB Enterprise distributes compaction across a fleet of job workers.

Applies to: Embedded ❌, Enterprise ✅

Distributed compaction splits a table's fragments into independent, fragment-disjoint tasks and fans them out across job workers in LanceDB Enterprise. Unlike an index build, a compaction plan needs no bin-packing: Lance's own planner already produces disjoint tasks, so workers claim and execute them directly, with no coordination needed to decide who does what. See Architecture.

%%{init: {"theme": "base", "themeVariables": {
  "background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19",
  "primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19",
  "edgeLabelBackground": "#ffffff", "fontFamily": "Inter, sans-serif", "fontSize": "18px"
}}}%%
flowchart LR
    L["Lance's planner<br/>splits fragments into<br/>disjoint tasks"]
    L --> W1["Worker<br/>claims fragments 1-2"]
    L --> W2["Worker<br/>claims fragments 3-4"]
    L --> W3["Worker<br/>claims fragments 5-6"]
    W1 --> C["Fleet-wide eligibility gate:<br/>one worker claims the commit"]
    W2 --> C
    W3 --> C
    classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19
    classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19
    class C accent

No task depends on another's result, so a worker that finishes early simply claims another task rather than waiting on its peers. Committing is where the fleet does coordinate: finished tasks go through a shared eligibility gate, and only the one worker that claims the commit actually writes it, rather than each worker committing its own task independently.

Whether a failed task can still let the rest commit depends on the remap mode. Inline remap is all-or-nothing: one failed task fails the whole job, since the commit updates every affected index inline and can't skip a task's share of that. Deferred remap can commit a subset -- a failed task's fragments are abandoned and replanned in a later round, while the rest of the job's tasks commit without waiting on it. See Advanced Conflict Resolution for when compaction defers a remap at all.

Compaction interfaces

• Compaction for triggering and configuring a compaction.
• Garbage Collection for removing files compaction and other operations leave behind.
• Job Framework for the job types a deployment runs on its own schedule.