Running SEP on Kubernetes requires Kubernetes cluster version 1.14 or higher.
The installation typically requires usage of kubectl.
Support for network policies#
Network policies are implemented by the network plugin, you must be using a networking solution which supports Kubernetes Network Policies.
Worker and coordinator pods require significant amount of memory and CPU to work efficiently.
Presto requires an exclusive node for each coordinator and each worker.
Resource sharing on a node or pod is not supported. Each node should only host one coordinator or worker pod, and nodes or pods can not be shared with other applications. Presto performs best with exclusive access to the underlying hardware resources, since it requires significant resources on each worker. It performs best when they are equal for each worker to allow for query planning and processing optimizations to work with a predictable resource pool and processing power.
By default, each worker pod requires 100GB of memory and 16 CPU cores which provide good overall performance. The requirements can be changed with worker-specific properties.
By default, the coordinator pod requires 60GB of memory and 16 CPU cores. Thee requirements can be changed with coordinator-specific properties.
The minimal production requirement for a worker pod is 64GB of memory and 8 CPU cores. The minimal production requirement for a coordinator is 48GB of memory and 8 CPU cores. Using lower memory and CPU settings is recommended only for testing and development purposes.
Internal metastore pods#
You can use a built-in Hive metastore to keep schema and table metadata, when querying data in object storage with the Hive connector. The needed separate pod requires 8GB of memory and 2 CPU cores.
Additionally, the Metastore requires a PostgreSQL database to store its data. It is possible to use a built-in ephemeral PostgreSQL instance for that purpose, however it is not recommended for production use.
This PostgreSQL pod requires 8GB of memory and 2 CPU cores. Additionally, a persistent storage is required with at least 10GB of disk space. For more information, refer to the detailed documentation.