本页尚未翻译

本页内容仍为英文原文，源自官方文档 enterprise/intelligent-auto-maintenance。中文翻译正在分批进行中。

Intelligent Auto-Maintenance

How Enterprise continuously tracks table activity in the background and decides what maintenance -- compaction, cleanup, indexing, remap catch-up -- a table needs next.

Applies to: Embedded ❌, Enterprise ✅

Keeping a table in good condition takes several different kinds of work -- compacting files, catching indexes up, cleaning old versions, folding a RTE write tier into the base table -- each with its own trigger. In LanceDB Enterprise, a fleet of agent processes decides when each of these needs to run, continuously, per table, in the background across your cluster -- nothing to schedule, trigger, or monitor by hand unless you want to.

%%{init: {"theme": "base", "themeVariables": {
  "background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19",
  "primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19",
  "edgeLabelBackground": "#ffffff", "fontFamily": "Inter, sans-serif", "fontSize": "18px"
}}}%%
flowchart LR
    E["Table event notifications"] --> P
    S["Periodic sweep<br/>every table, on a timer"] --> P
    P["Planner<br/>evaluates one table"]
    P --> W["Possible next step:<br/>WAL index catch-up"]
    P --> I["Possible next step:<br/>Index operations"]
    P --> C["Possible next step:<br/>Compaction"]
    P --> CL["Possible next step:<br/>Cleanup"]
    P --> R["Possible next step:<br/>Remap catch-up"]
    classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19
    classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19
    class P accent

Tracking activity

An agent's long-running process watches two independent sources for a table: table events (Job Registry notifications, so it learns about a write, a schema change, or a new index almost immediately) and a periodic sweep that revisits every table it owns on a timer -- 600 seconds by default -- so a table that stops producing events still gets re-evaluated rather than going stale. A hot table producing a flood of events does not get to monopolize evaluation: notifications and the periodic sweep share a bounded per-tick budget, and reads for a given table run at most once at a time.

Each evaluation reopens the table and reads its manifest, indexes and in-flight jobs fresh, rather than deciding from a cached guess. Those are separate reads rather than one atomic snapshot, so a concurrent write between them is possible; the next evaluation -- triggered by that same write's own event notification -- is what catches anything the current one missed.

Deciding what to run

A table is not reduced to one "next best action" -- each evaluation walks a fixed, priority-ordered sequence of independent decisions, launching whichever ones are due:

1. WAL index catch-up. Checked first and gated by nothing else, because it is correctness-driven rather than discretionary: while an index lags behind the compacted generation, the RTE write tier cannot trim its log. See RTE Compaction.
2. Index operations. Reindexing a column, unless a compaction that cannot defer its remap is about to run this same tick -- deferred-remap compaction lets index work proceed alongside it instead of waiting. See Incremental Indexing.
3. Compaction. Whether enough small fragments or deleted rows have accumulated to merge. See Compaction and Distributed Compaction.
4. Cleanup. Independent of the other four -- it can always run in parallel with them. See below for exactly when it's due.
5. Deferred-remap catch-up. The batched, amortized follow-up for index entries a deferred compaction didn't remap synchronously. See Advanced Conflict Resolution.

More than one of these can fire for the same table in the same evaluation -- a compaction and a cleanup are unrelated and can both launch together -- but some gate each other where running them at the same time would be unsafe, like index operations waiting on a compaction that can't defer its remap.

When cleanup is due

Cleanup prunes old table versions and the files only they referenced. In LanceDB Embedded, that always happens because you asked for it -- a call to optimize(). LanceDB Enterprise does not support a manual optimize() call at all; cleanup there is entirely this planner's own decision, off by default unless the deployment sets a minimum version age -- with no minimum age configured, nothing is ever old enough to clean up automatically, and old versions simply accumulate until an operator enables it. A table can also disable it individually even when the deployment allows it, overriding the deployment-wide setting for just that table.

Once enabled, a table becomes due for cleanup when any one of these is true:

Trigger	Condition
Enough new versions	The current version has advanced far enough past the version the last successful cleanup started from
Enough time has passed	The last successful cleanup finished long enough ago -- the default is 24 hours -- so an idle table whose versions simply age past the retention window still gets revisited
Version went backward	The current version is lower than the version the last successful cleanup recorded, which re-triggers a run immediately

A table that has never had a successful cleanup run is held to its own initial version threshold first, so a brand-new table is not claimed for cleanup before it has accumulated enough versions to be worth the pass. A failed attempt is retried, but not immediately -- a minimum delay (60 seconds by default) must pass after the failure before the table becomes eligible again, even if one of the triggers above would otherwise fire.

Confirming it's actually running

Rather than asking you to trust that evaluation is happening, the agent exposes coverage metrics scoped to successful evaluations specifically -- a failed or timed-out attempt does not count:

Metric (lance_agent_ prefix)	What it tells you
coverage_never_evaluated	Tables this agent owns that have never completed an evaluation
coverage_oldest_success_age_seconds	How long it's been since the longest-overdue table was last successfully evaluated
coverage_cycle_target / coverage_cycle_pending	How much work the current full sweep has left
coverage_cycles_completed_total	How many full sweeps have finished

A cycle freezes its target set of tables up front; new tables join the next cycle rather than resetting progress on the current one, and a table that disappears or changes ownership is dropped from the target rather than counted as covered.

What to read next

CleanupWhat a cleanup pass actually removes, and the retention window it respects.Advanced Conflict ResolutionWhy compaction and index maintenance need the deferred-remap decision at all.RTE CompactionThe WAL-tier compaction this page's Decision 1 depends on.Incremental IndexingWhat an index operation actually does once scheduled.OpenTelemetryViewing the `lance_agent_*` coverage metrics above.