webinar register page

CNCF Member Webinar: Improving Data Locality for Analytics Jobs on Kubernetes Using Alluxio
In the on-prem days, one key performance optimization for Apache Hadoop or Apache Spark workloads is to run tasks on nodes with local HDFS data. However, while adoption of the Cloud & Kubernetes makes scaling compute workloads exceptionally easy, HDFS is often not an option. Effectively accessing data from cloud-native storage services like AWS S3 or even on-premises HDFS becomes harder as data locality is lost.

Originated from UC Berkeley AMPLab, the open source project Alluxio approaches this problem in a new way by helping to move data closer to compute workloads efficiently and on-demand, and unify data across multiple or remote clouds, and many more. This webinar will describe the concept and internal mechanism using the stack of Spark+Alluxio in Kubernetes to enhance data locality even when the storage service is outside or remote.

Particularly, we will go over:
Why Spark is able to make a locality-aware schedule when working with Alluxio in K8s environment using the host network;
Why a pod running Alluxio can share data efficiently with a pod running Spark on the same host using domain socket and host path volume;
The roadmap of Alluxio to further improve running analytics jobs like Spark and Presto, including the on-going closer integration with Presto

Jan 21, 2020 10:00 AM in Pacific Time (US and Canada)

Webinar logo
* Required information
Loading

Speakers

Gene Pang
PMC Maintainer @Alluxio
Gene Pang is the PMC Maintainer of the Alluxio open source project and the Head of Architecture at Alluxio, Inc. He graduated with a Ph.D. from the AMPLab at UC Berkeley, working on distributed database systems. Before starting at Berkeley, he worked at Google and has an M.S. from Stanford University, and a B.S. from Cornell University.
Adi Madan
Core Engineer @Alluxio
Adit Madan is a core engineer at Alluxio. His experience is in distributed systems, storage systems, and large-scale data analytics. He has a M.S. from Carnegie Mellon University, and a B.S. from IIT.