Today, we’re excited to announce that customers can now submit Apache Spark jobs to Amazon EMR on EKS using Apache Livy, in addition to using the StartJobRun API, Spark Operator, Spark Submit, and Interactive Endpoints. Masu. With this release, customers can now easily submit Spark jobs and Spark code snippets using a REST interface and retrieve results synchronously or asynchronously, while also using an EMR-optimized Spark runtime. You can continue to enjoy all the benefits of SSL-secured Amazon EMR on EKS. Livy endpoints, programmatic setup experience, and more.
Apache Livy is a service that allows you to easily interact with Spark clusters through a REST interface. Prior to today’s release, if a customer wanted to run a batch or interactive workload from their local environment through their REST interface, they had to use either the StartJobRun API or the interactive endpoint. As a result, customers had to modify their applications. Today’s release simplifies the experience for a customer to easily create a Livy endpoint and use that endpoint to submit Apache Spark jobs to his Amazon EMR on his EKS cluster via his REST interface it was done. Additionally, for added security, you can also restrict access to Livy endpoints by configuring authentication through one of the supported protocols, such as Kerberos or custom authentication.
To learn more and get started, see the documentation in the Apache Livy section. Amazon EMR on EKS with Apache Livy is available with the EMR 7.1 release in regions where Amazon EMR on EKS is currently available.
