LanceDB 中文文档

外观

本页尚未翻译

本页内容仍为英文原文，源自官方文档 geneva/index。中文翻译正在分批进行中。

Multimodal Feature Engineering with Geneva ​

Learn how to do multimodal feature engineering in LanceDB Enterprise to transform raw data into meaningful features for AI models.

Enterprise-only

When working with multimodal data at scale, LanceDB Enterprise makes it easy to define, extract, and transform raw data into useful information and features for your AI applications. LanceDB Enterprise's Multimodal Feature Engineering package is designed to improve the productivity of AI engineers operating at immense scale.

With an API designed to leverage LanceDB's optimized data storage and retrieval, it streamlines prototyping extraction and transformation tasks, performing experiments, exploring your data, scaling up execution, and moving to production.

LanceDB Multimodal Feature Engineering enables researchers to seamlessly transition from experiments in local notebooks to fully-managed distributed job execution on datasets with billions of rows.

[card]

The geneva Python package is currently only available as part of LanceDB Enterprise. Please contact us if you're interested in scaling up your feature engineering workloads for your AI and multimodal use cases.

Feature engineering itself is not Enterprise-only: LanceDB Embedded has computed columns and materialized views, which need no extra package. See Feature engineering to choose between them and Geneva.

The geneva package uses Python User Defined Functions (UDFs) to define features as columns in a Lance dataset. Adding a feature is straightforward:

Step 1

Prototype your Python function in your favorite environment.

Step 2

Wrap the function with a small UDF decorator (see UDFs).

Step 3

Register the UDF as a virtual column using Table.add_columns().

Step 4

(Optional, advanced) Override where the job runs — see Advanced Execution Contexts. On LanceDB Enterprise, distributed job execution is fully managed, so most users can skip this step.

Step 5

Trigger a backfill operation (see Backfilling).

提示

You can build your Python feature generator function in an IDE or a notebook using your project's Python versions and dependencies. geneva will automate much of the dependency and version management needed to move from prototype to scale and production.

Ready to write your first feature? Head to Getting Started.

Continue learning ​

Visit the following pages to learn more about featuring engineering in LanceDB Enterprise:

API Reference ​

  • geneva.connect() — connect to a Geneva database
  • Connection — manage tables, views, jobs, clusters, and manifests
  • Table — add columns, backfill, search, and manage table data
  • UDF — define user-defined functions for feature computation