Insights usage metrics#

The usage metrics tab is accessed in Starburst Insights by using the Usage metrics item in the left-hand navigation.

It provides an overview of the cluster usage over a period of time and a cost estimation tool. You can also download the usage data or send it to Starburst from this tab.

Setup#

To access usage metrics through this tab, you must have event logger configured and Insights enabled to use persistent data by setting:

insights.persistence-enabled=true

Date filter#

Usage metrics

The time range for which the usage metrics are shown can be selected using a filter that provides the following options:

  • Today

  • Current month

  • Last month

  • Current year

  • Last year

  • All available dates

  • Custom range

Selecting the custom range option allows specifying a start and end date:

Usage metrics date range

A summary of the usage for the selected time range is shown in the pane along with a chart depicting cumulative usage over time. The data for the selected time interval can be downloaded or sent to Starburst.

Cost estimation tool#

The cost estimation tool allows you to optionally input the cost factor (price per vCPU per hour) for the instance type used in your SEP cluster. The estimated cost of usage per filtered period of time is displayed.

Usage metrics cost estimation tool

Automatic usage metrics upload#

Usage metrics are automatically uploaded to Starburst every hour. This upload consists of usage metrics collected since the previous upload. Note that if insights-persistence-enabled is set to false, usage metrics are still collected and stored in memory for up to 24 hours to support this upload.

You can configure this automatic metric upload in config.properties, but the default values should suffice for most deployments. The following snippet shows these configuration properties and their default values:

insights.data-upload-automatic.enabled=true
insights.data-upload-automatic.memory-store-interval=24h
insights.data-upload-automatic.interval=1h
insights.data-upload-protocol=https
insights.data-upload-endpoint=https://telemetry.eng.starburstdata.net/usage-metrics

Manual upload#

If the network upstream of your SEP cluster blocks the automatic usage metric upload, disable the automatic upload by setting insights.data-upload-automatic.enabled to false. You can then manually upload usage metrics data by following these steps:

  1. Download the usage metrics file from the usage-metrics view in Insights.

  2. On a device with internet access, upload the metrics file to Starburst.