Data engineering #

Data engineers deliver data to data consumers in a performant and suitable format, and in a timely manner with an expectation of a given level of data quality.

These pages cover different topics related to a data engineer.

Working with data lakes #

Follow the links to learn about the different ways Starburst Galaxy provides you to work with your data lakes.

• Storage
• Metastores
• File formats

Table formats #

Starburst Galaxy supports the following table formats.

• Iceberg table format
• Delta Lake table format
• Hive table format
• Hudi table format
• Table format migration

Optimization, performance, and quality #

Follow the links for more information about optimizing your cluster performance and data quality:

• Using materialized views
• Partitioning
• Catalog indexing

Observability #

• Data quality
• Data lineage

Workload optimization #

This section provides information about different cluster execution mode.

• Warp speed-enabled clusters
• Clusters with fault tolerant execution
• Query result caching