Starburst SAP HANA connector#

The SAP HANA connector allows querying and creating tables in an external database. Connectors let Starburst Enterprise platform (SEP) join data provided by different databases, like SAP HANA and Hive, or different database instances.
Requirements#
To connect to SAP HANA, you need:
SAP HANA version 2.0 or higher.
Network access from the coordinator and workers to the SAP HANA server. Port 30015 the default port for instance 00.
A valid Starburst Enterprise license.
SAP HANA JDBC driver, acquired from SAP.
Configuration#
Before configuring a catalog with the SAP HANA connector, install the JDBC driver on your SEP nodes:
Add the SAP HANA JDBC driver JAR file to the SEP
plugin/sap-hana
directory on all nodes.Restart SEP on every node.
To configure the SAP HANA connector as the myhanadb
catalog, create a file
named myhanadb.properties
in etc/catalog
:
connector.name=sap-hana
connection-url=jdbc:sap://Hostname:Port/?optionalparameters
connection-user=USERNAME
connection-password=PASSWORD
Refer to the SAP HANA for more information about format and parameters of the JDBC URL supported by the SAP HANA JDBC driver.
General configuration properties#
The following table describes general catalog configuration properties for the connector:
Property name |
Description |
Default value |
---|---|---|
|
Support case insensitive schema and table names. |
|
|
|
|
|
Path to a name mapping configuration file in JSON format that allows Trino to disambiguate between schemas and tables with similar names in different cases. |
|
|
Frequency with which Trino checks the name matching configuration file for changes. |
|
|
Duration for which metadata, including table and column statistics, is cached. |
|
|
Cache the fact that metadata, including table and column statistics, is not available |
|
|
Maximum number of objects stored in the metadata cache |
|
|
Maximum number of statements in a batched execution. Do not change this setting from the default. Non-default values may negatively impact performance. |
|
Type mapping#
Because SEP and SAP HANA each support types that the other does not, this connector modifies some types when reading or writing data.
SAP HANA to SEP read type mapping#
The following read type mapping applies when data is read from existing tables in SAP HANA, or inserted into existing tables in SAP HANA from SEP.
SAP HANA database type |
SEP type |
Notes |
---|---|---|
BOOLEAN |
BOOLEAN |
|
TINYINT |
TINYINT |
|
SMALLINT |
SMALLINT |
|
INTEGER |
INTEGER |
|
BIGINT |
BIGINT |
|
REAL |
REAL |
|
DOUBLE |
DOUBLE |
|
FLOAT(p) |
REAL for p <= 24, DOUBLE otherwise |
|
DECIMAL(p, s) |
DECIMAL(p, s) |
|
DECIMAL |
DOUBLE |
SAP HANA’s DECIMAL with precision and scale not specified represents a floating-point decimal number |
SMALLDECIMAL |
DOUBLE |
SAP HANA’s DECIMAL with precision and scale not specified represents a floating-point decimal number |
NCHAR |
CHAR |
|
VARCHAR(n) |
VARCHAR(n) |
|
NVARCHAR(n) |
VARCHAR(n) |
|
ALPHANUM(n) |
VARCHAR(n) |
|
SHORTTEXT(n) |
VARCHAR(n) |
|
CLOB |
VARCHAR (unbounded) |
|
NCLOB |
VARCHAR (unbounded) |
|
TEXT |
VARCHAR (unbounded) |
|
BINTEXT |
VARCHAR (unbounded) |
|
VARBINARY(n) |
VARBINARY |
|
BLOB |
VARBINARY |
|
DATE |
DATE |
|
TIME |
TIME(0) |
|
SECONDDATE |
TIMESTAMP(0) |
|
TIMESTAMP |
TIMESTAMP(7) |
All other types aren’t supported.
SEP to SAP HANA write type mapping#
The following write type mapping applies when tables are created in SAP HANA from SEP.
SEP type |
SAP HANA database type |
Notes |
---|---|---|
BOOLEAN |
BOOLEAN |
|
TINYINT |
TINYINT |
|
SMALLINT |
SMALLINT |
|
INTEGER |
INTEGER |
|
BIGINT |
BIGINT |
|
REAL |
REAL |
|
DOUBLE |
DOUBLE |
|
DECIMAL(p, s) |
DECIMAL(p, s) |
|
CHAR |
CHAR or NCLOB |
|
VARCHAR |
NVARCHAR or CLOB |
|
VARBINARY |
BLOB |
|
DATE |
DATE |
|
TIME(p) |
TIME |
|
TIMESTAMP(p) |
SECONDDATE for p = 0, TIMESTAMP otherwise |
All other types aren’t supported.
Type mapping configuration properties#
The following properties can be used to configure how data types from the connected data source are mapped to Trino data types and how the metadata is cached in Trino.
Property name |
Description |
Default value |
---|---|---|
|
Configure how unsupported column data types are handled:
The respective catalog session property is |
|
|
Allow forced mapping of comma separated lists of data types to convert to
unbounded |
SQL support#
The connector provides read and write access to data and metadata in SAP HANA. In addition to the globally available and read operation statements, the connector supports the following features:
Views#
The connector can read data from views, including SAP HANA calculation views.
Performance#
The connector includes a number of performance improvements, detailed in the following sections.
Table statistics#
The SAP HANA connector can use table and column statistics for cost based optimizations, to improve query processing performance based on the actual data in the data source.
The statistics are collected by SAP HANA and retrieved by the connector.
You have to use the CREATE STATISTICS
command in SAP HANA to initiate
creation and ongoing collection and update of the relevant statistics. You can
find more information about statistics collection in the SAP HANA documentation.
The connector and SEP support the statistic types HISTOGRAM
, SIMPLE
,
and TOPK
.
Note
The collection in SAP HANA can take considerable time and depends on the data size. You can use the MERGE DELTA command to affect availability of the statistics.
Pushdown#
The connector supports pushdown for a number of operations:
Aggregate pushdown for the following functions:
Additionally, for the aggregate functions below, pushdown is only supported
for DOUBLE
type columns:
Cost-based join pushdown#
The connector supports cost-based Join pushdown to make intelligent decisions about whether to push down a join operation to the data source.
When cost-based join pushdown is enabled, the connector only pushes down join operations if the available Table statistics suggest that doing so improves performance. Note that if no table statistics are available, join operation pushdown does not occur to avoid a potential decrease in query performance.
The following table describes catalog configuration properties for join pushdown:
Property name |
Description |
Default value |
---|---|---|
|
Enable join pushdown. Equivalent catalog
session property is
|
|
|
Strategy used to evaluate whether join operations are pushed down. Set to
|
|
Dynamic filtering#
Dynamic filtering is enabled by default. It causes the connector to wait for dynamic filtering to complete before starting a JDBC query.
You can disable dynamic filtering by setting the dynamic-filtering.enabled
property in your catalog configuration file to false
.
Wait timeout#
By default, table scans on the connector are delayed up to 20 seconds until dynamic filters are collected from the build side of joins. Using a large timeout can potentially result in more detailed dynamic filters. However, it can also increase latency for some queries.
You can configure the dynamic-filtering.wait-timeout
property in your
catalog properties file:
dynamic-filtering.wait-timeout=1m
You can use the dynamic_filtering_wait_timeout
catalog session
property in a specific session:
SET SESSION catalogname.dynamic_filtering_wait_timeout = 1s;
Compaction#
The maximum size of dynamic filter predicate, that is pushed down to the
connector during table scan for a column, is configured using the
domain-compaction-threshold
property in the catalog
properties file:
domain-compaction-threshold=100
You can use the domain_compaction_threshold
catalog
session property:
SET SESSION domain_compaction_threshold = 10;
By default, domain-compaction-threshold
is set to 32
.
When the dynamic predicate for a column exceeds this threshold, it is compacted
into a single range predicate.
For example, if the dynamic filter collected for a date column dt
on the
fact table selects more than 32 days, the filtering condition is simplified from
dt IN ('2020-01-10', '2020-01-12',..., '2020-05-30')
to dt BETWEEN
'2020-01-10' AND '2020-05-30'
. Using a large threshold can result in increased
table scan overhead due to a large IN
list getting pushed down to the data
source.
Metrics#
Metrics about dynamic filtering are reported in a JMX table for each catalog:
jmx.current."com.starburstdata.presto.plugin.jdbc.dynamicfiltering:name=catalogname,type=dynamicfilteringstats"
Metrics include information about the total number of dynamic filters, the number of completed dynamic filters, the number of available dynamic filters and the time spent waiting for dynamic filters.
JDBC connection pooling#
When JDBC connection pooling is enabled, each node creates and maintains a connection pool instead of opening and closing separate connections to the data source. Each connection is available to connect to the data source and retrieve data. After completion of an operation, the connection is returned to the pool and can be reused. This improves performance by a small amount, reduces the load on any required authentication system used for establishing the connection, and helps avoid running into connection limits on data sources.
JDBC connection pooling is disabled by default. You can enable JDBC connection
pooling by setting the connection-pool.enabled
property to true
in your
catalog configuration file:
connection-pool.enabled=true
The following catalog configuration properties can be used to tune connection pooling:
Property name |
Description |
Default value |
---|---|---|
|
Enable connection pooling for the catalog. |
|
|
The maximum number of idle and active connections in the pool. |
|
|
The maximum lifetime of a connection. When a connection reaches this lifetime it is removed, regardless of how recently it has been active. |
|
|
The maximum size of the JDBC data source cache. |
|
|
The expiration time of a cached data source when it is no longer accessed. |
|
Starburst Cached Views#
The connectors supports table scan redirection to improve performance and reduce load on the data source.
Security#
The connector includes a number of security-related features, detailed in the following sections.
Password credential pass-through#
The connector supports password credential pass-through. To enable it, edit the catalog properties file to include the authentication type:
sap-hana.authentication.type=PASSWORD_PASS_THROUGH
For more information about configurations and limitations, see Password credential pass-through.