Research Guides: Data Services Class Descriptions: Machine Learning with Spark

This tutorial provides a basic understanding of Apache Spark and its usage on the HPC. There will be hands-on examples on how to use Apache Spark and step-by-step instructions on how to run Spark jobs.

Software:	None
Duration:	120 min
Room description:	Some tutorials are held remotely and require NYU sign on to access, while others are held in person, without a remote component. Please note the correct modality and location of the tutorial when registering
Prerequisites:	HPC user account
Skills Taught / Learning Outcomes:	Brief overview of the Spark ecosystem RDD, transformation and action Running spark-shell, pyspark Compiling Java code with Maven Spark-submit Spark SQL Accessing Hive database in Spark
Class Materials:	Link to Class Materials
Related Classes:	Introduction to HPC Advanced HPC Globus and Data Transfers
Additional Training Materials:	Available via LinkedIn Learning (NYU NetID required): Apache Spark Essential Training Apache Spark Essential Training: Big Data Engineering Spark for Machine Learning and AI
Feedback:	bit.ly/feedbackds

Data Services Class Descriptions

Upcoming sessions for this tutorial