• Online, Self-Paced
Course Description

Analytic model management ensures that models are not only superior to alternatives, but they also meet or exceed current business needs. Explore the process of building, validating, and evaluating a predictive analytics model.

Learning Objectives

Model Building

  • start the course
  • recognize key phases in developing a model
  • recognize important aspects of data preparation for model development
  • identify key functions in the model creation process

Model Considerations and Validation

  • identify key considerations of model validation
  • balance model complexity with overfitting
  • identify features of two-fold validation

Model Evaluation

  • recognize key features of Occam's Razor
  • calculate mean squared error measures
  • recognize prediction variation measures
  • recognize evaluation measures for classification models
  • interpret lift and gain charts
  • interpret ROC curves and AUC

Practice: Classification Model Performance

  • determine the superior classification model

Framework Connections

The materials within this course focus on the Knowledge Skills and Abilities (KSAs) identified within the Specialty Areas listed below. Click to view Specialty Area details within the interactive National Cybersecurity Workforce Framework.

Feedback

If you would like to provide feedback for this course, please e-mail the NICCS SO at NICCS@hq.dhs.gov.