3.3. Data Source Management#
Data sources define the connection of Presto to the various external databases and other systems, that are queried by the Presto user. With Mission Control the creation and management of these data sources is available via a simple user interface.
The data source management is suitable to create, update, and delete these data sources. Any user can access the data source management by clicking on the Data Sources item in the left hand panel.
The data source management view displays a grid of all existing data sources.
Press the + symbol on the bottom right corner to add a new data source.
The following data sources can be configured with a rich user interface for the necessary properties.
- Amazon S3
- Amazon Redshift
- Apache Kafka
- Azure Storage
- Delta Lake
- Microsoft SQL Server
Creation and configuration of the data source and the related properties is performed in a tabbed user interface for basic information, connection details, advanced configuration and others. Specific sections differ based on the data source type.
If you can not find your desired data source in the list, try the custom data source configuration.
Data source configuration creates a catalog property file, which configures a specific connector and the relevant properties.
Clicking on a data source name, or using the Edit link in the actions drop down, allows you to edit the data source.
As a next step you can move on to create a cluster, in which the data source is used and available for Presto users to query against.
Data sources are defined by the following properties:
Defines the type of external data source to connect to. Type is closely related to the connector used to access the data in the external data source.
The name of the data source. A well chosen name is important, since it details the data source to the user in Presto. For example, your data source is configured to access a specific database on a specific PostgreSQL server instance. Choosing a name
postgresqlis therefore not really sufficient. It can be more useful for your users to choose a name of
postgresql-customerdb. Potentially, if your users do not really have to know anything about the underlying database, it can also be appropriate to just use a name such as
The description allows you to compose a short paragraph that provides more details about the data source than the name alone.
After choosing the data source type the most important, required properties to connect to the data sources are displayed. These include properties required by connector as well as properties that enable connection to the specified data source, such as access keys.
Data source types define the connector Presto uses for the specific data source. The connector in turn, defines the configuration properties to use.
Refer to the following table to determine what connector configuration properties are used for a specific data source type:
|Data source type||Connector|
|Amazon S3||Starburst Hive Connector|
|Amazon Redshift||Redshift Connector|
|Apache Kafka||Starburst Kafka Connector|
|Azure Storage||Azure Storage Support for the Hive Connector|
|Delta Lake||Starburst Delta Lake Connector|
|HDFS||Starburst Hive Connector|
|Microsoft SQL Server||Starburst SQL Server Connector|
|MySQL||Starburst MySQL Connector|
|Oracle||Starburst Oracle Connector|
|PostgreSQL||Starburst PostgreSQL Connector|
|Teradata||Starburst Teradata Connector|
Custom data sources allow you to configure a catalog with a connector that Mission Control does not support. Only use this feature if you have expert knowledge of Presto connectors and catalog configuration.
You can use this interface in the following situations:
- Configure a catalog using a connector with no related data source support in Mission Control.
- Manually control the content of the catalog file as an alternative configuration method.
- Configure a catalog for a custom connector.
Access the configuration with the following steps:
- Press the + symbol on the bottom right corner to add a new data source.
- Click on the Don’t see your data source? in the notes section beside the list of data source types
- Click on the Add one manually link.
- Confirm that you understand the risk.
- Configure catalog name, description and connector name.
- Add the content property definitions for the catalog file in the text input.
- Press Create to finish.
The supported properties differ for each connector and are identical to the properties used in the data source configuration file, documented with the connectors.