Starburst AI agent-based data product enrichment#

Starburst supports data product enrichment using the Starburst AI agent. This feature automatically generates descriptions for data products and datasets using metadata and relationships between data elements.

Note

Starburst Enterprise AI agent-based feature is available as a private preview. Contact your Starburst account team for further information.

Requirements#

To use the data products enrichment feature, you need:

  • A valid AI_WORKFLOWS license.

  • Access to at least one configured language AI model.

  • Write permissions for data products.

Configuration#

To configure the enrichment feature, add the following property to your coordinator configuration file:

starburst.agent.enabled=true

Enrich a data product#

After confirming access and configuration, follow these steps to enrich a data product:

  1. In the Data products tab of the Starburst Enterprise web UI, select an existing data product.

  2. Click the wand_stars Enrich with AI button to open the Datasets section of the Enrich data product dialog.

  3. From the drop-down menu, choose an AI model.

  4. Click wand_stars AI generate all datasets to enrich each dataset within the data product:

    • Column descriptions are generated based on their names, types, any existing descriptions, and the surrounding context such as dataset and data product names.

    • Dataset descriptions are generated based on column descriptions, existing dataset descriptions (if available), and context.

    To enrich a specific dataset, select it, then choose wand_stars AI generate this dataset from the wand_stars AI generate all drop-down menu and click wand_stars AI generate this dataset.

  5. Click Next.

  6. In the Data product details section, click wand_stars Generate values for all fields.

    This generates a description for the data product, based on its name, current description (if any), and its datasets and their descriptions. The Summary and Tags are based on the generated description.

  7. Click Save.

Enrich a specific field#

To enrich a specific dataset field using the AI agent:

  1. Navigate to the Data products tab and select a data product.

  2. Click wand_stars Enrich with AI.

  3. Click the AI generate button for the field you want to enrich. You can enrich an individual column, all columns, or the dataset description.

Resetting dataset fields#

You can reset metadata fields during the initial AI generation, either for all datasets or for specific ones.

To reset a single field within a dataset, navigate to the field and click the corresponding autorenew reset button.

To reset all fields in a specific dataset:

  1. Navigate to the Data products tab and select a data product.

  2. Select the dataset you want to reset.

  3. Click wand_stars Enrich with AI.

  4. From the wand_stars AI generate all datasets drop-down menu, select Reset this dataset.

  5. Click autorenew Reset this dataset.

  6. Click Next.

  7. In the Data product details section, select Reset values for all fields from the wand_stars AI generate all datasets drop-down menu.

  8. Click autorenew Reset values for all fields.

  9. Click Save.

To reset all fields across all datasets:

  1. Navigate to the Data products tab and select a data product.

  2. Click wand_stars Enrich with AI.

  3. From the wand_stars AI generate all datasets drop-down menu, select Reset all.

  4. Click autorenew Reset all to resets all fields across all datasets in the selected data product.

  5. Click Next.

  6. In the Data product details section, select Reset values for all fields from the wand_stars AI generate all datasets drop-down menu.

  7. Click autorenew Reset values for all fields.

  8. Click Save.