Starburst MaxCompute connector#

The MaxCompute connector allows users to query data in MaxCompute databases.

Requirements#

To use the MaxCompute connector, you need:

Network access from the coordinator and workers to the MaxCompute server.
A valid Starburst Enterprise license.

Configuration#

Create a catalog properties file in etc/catalog named example.properties to access the configured MaxCompute database in the example catalog (replace example with your database name or some other descriptive name of the catalog). Configure the usage of the connector by specifying the name maxcompute and replace the connection properties as appropriate for your setup.

connector.name=maxcompute
maxcompute.project.name=max_compute
maxcompute.access.id=access id
maxcompute.access.key=access key
maxcompute.endpoint=http://service.cn-example.maxcompute.aliyun.com/api

General configuration properties#

The following table describes catalog configuration properties for the connector:

Property name	Description
`maxcompute.project.name`	Name of the MaxCompute project. Required.
`maxcompute.access.id`	Unique identifier used to access MaxCompute resources securely. Required.
`maxcompute.access.key`	Access key used to authenticate access to MaxCompute. Required.
`maxcompute.endpoint`	Endpoint used to communicate with MaxCompute. Required.
`maxcompute.tunnel.endpoint`	Endpoint where the tunneling protocol should connect, used to improve performance.
`maxcompute.additional-projects`	Comma separated list of additional MaxCompute projects to be exposed as SEP schemas.
`maxcompute.input.split.size`	Maximum size for each split of the input data.

Optionally, configure maxcompute.tunnel.endpoint to improve performance:

maxcompute.tunnel.endpoint=http://dt.cn-example.maxcompute.aliyun.com

Type mapping#

Because Trino and MaxCompute each support types that the other does not, this connector modifies some types when reading data. Data types may not map the same way between SEP and the data source. Refer to the following section for type mapping.

MaxCompute to Trino type mapping#

The connector maps MaxCompute types to the corresponding Trino types following this table:

MaxCompute to Trino type mapping#
MaxCompute type	Trino type	Notes
`BOOLEAN`	`BOOLEAN`
`TINYINT`	`TINYINT`
`SMALLINT`	`SMALLINT`
`INT`	`INT`
`BIGINT`	`BIGINT`
`BINARY`	`VARBINARY`
`FLOAT`	`REAL`
`DOUBLE`	`DOUBLE`
`DECIMAL`	`DECIMAL`
`VARCHAR`	`VARCHAR`
`CHAR`	`CHAR`	Special characters in `CHAR` columns cannot be read.
`STRING`	`VARCHAR`
`DATE`	`DATE`
`DATETIME`	`TIMESTAMP(3)`
`TIMESTAMP`	`TIMESTAMP`

No other types are supported.

SQL support#

The connector provides globally available and read operation statements to access data and metadata in MaxCompute.

Performance#

The connector includes a number of performance improvements, detailed in the following sections.

Pushdown#

The connector supports partition pushdown.