Course Description
In this course, you will learn about various ways to monitor Spark applications such as web UIs, metrics, and other monitoring tools. You will also learn about memory tuning.
Learning Objectives
Monitoring Spark Applications
- start the course
- access the web user interface
- use the Spark environment configuration parameters
- use JSON to query monitoring tools for Spark
- set JVM fractional memory amounts for Spark
Tuning Spark Applications
- modify speculation controls for Spark tasks
- describe data serialization and the role it plays in the performance of Spark applications
- describe memory management and consumption
- determine executor memory allocation
- describe garbage collection tuning
- set the level of parallelism
- use the broadcast functionality
- use query execution plan explainer
- implement data compression on parquet storage
Practice: Monitoring Spark Applications
- monitor Spark applications
Framework Connections
Feedback
If you would like to provide feedback for this course, please e-mail the NICCS SO at NICCS@hq.dhs.gov.