pipelines/components/aws/sagemaker/README.md

Amazon SageMaker Components for Kubeflow Pipelines

Summary

With Amazon SageMaker Components for Kubeflow Pipelines (KFP), you can create and monitor training, tuning, endpoint deployment, and batch transform jobs in Amazon SageMaker. By running Kubeflow Pipeline jobs on Amazon SageMaker, you move data processing and training jobs from the Kubernetes cluster to Amazon SageMaker’s machine learning-optimized managed service. The job parameters, status, and outputs from Amazon SageMaker are still accessible from the Kubeflow Pipelines UI.

Components

Amazon SageMaker Components for Kubeflow Pipelines offer an alternative to launching compute-intensive jobs in Kubernetes and integrate the orchestration benefits of Kubeflow Pipelines. The following Amazon SageMaker components have been created to integrate key SageMaker features into your ML workflows from preparing data, to building, training, and deploying ML models. You can create a Kubeflow Pipeline built entirely using these components, or integrate individual components into your workflow as needed. The components are available in one or two versions. Each version of a component leverages a different backend. For more information on those versions, see SageMaker Components for Kubeflow Pipelines versions.

For an end-to-end tutorial using these components, see Using Amazon SageMaker Components.

For more example pipelines, see Sample AWS SageMaker Kubeflow Pipelines.

There is no additional charge for using Amazon SageMaker Components for Kubeflow Pipelines. You incur charges for any Amazon SageMaker resources you use through these components.

List of Components

Note: We encourage users to utilize Version 2 of a SageMaker component wherever it is available. You can find the version of an AWS SageMaker Components in the docker image tag used in the component specification file component.yaml.

Ground Truth components

• Ground Truth The Ground Truth component enables you to submit SageMaker Ground Truth labeling jobs directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker Ground Truth Kubeflow Pipelines component version 1	X

• Workteam The Workteam component enables you to create Amazon SageMaker private workteam jobs directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker create private workteam Kubeflow Pipelines component version 1	X

Data processing components

• Processing The Processing component enables you to submit processing jobs to Amazon SageMaker directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker Processing Kubeflow Pipeline component version 1	X

Training components

• Training The Training component allows you to submit Amazon SageMaker Training jobs directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker Training Kubeflow Pipelines component version 1	SageMaker Training Kubeflow Pipelines component version 2

• Hyperparameter Optimization The Hyperparameter Optimization component enables you to submit hyperparameter tuning jobs to Amazon SageMaker directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker Hyperparameter Optimization Kubeflow Pipeline component version 1	X

• RLEstimator The RLEstimator component allows you to submit RLEstimator (Reinforcement Learning) SageMaker Training jobs directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker RLEstimator Kubeflow Pipelines component version 1	X

Inference components

• Hosting Deploy The Hosting components allow you to deploy a model using SageMaker hosting services from a Kubeflow Pipelines workflow.

  Version 1 of the component

  Version 2 of the component

  SageMaker Hosting Services - Create Endpoint Kubeflow Pipeline component version 1

  Version 2 of the Hosting components consists of the three sub-components needed to create a hosting deployment on SageMaker. A SageMaker Model Kubeflow Pipelines component version 2 responsible for the model artifacts and the model image registry path that contains the inference code. A SageMaker Endpoint Config Kubeflow Pipelines component version 2 responsible for defining the configuration of the endpoint such as the instance type, models, number of instances, and serverless inference option. A SageMaker Endpoint Kubeflow Pipelines component version 2 responsible for creating or updating the endpoint on SageMaker as specified in the endpoint configuration.

• Batch Transform component The Batch Transform component enables you to run inference jobs for an entire dataset in Amazon SageMaker from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  SageMaker Batch Transform Kubeflow Pipeline component version 1	X

• Model Monitor components The Model Monitor components allow you to monitor the quality of SageMaker machine learning models in production from a Kubeflow Pipelines workflow.

  Version 1 of the component

  Version 2 of the component

  X

  The Model Monitor components consist of four sub-components for monitoring drift in a model. A SageMaker Data Quality Job Definition Kubeflow Pipelines component version 2 responsible for monitoring drift in data quality. A SageMaker Model Quality Job Definition Kubeflow Pipelines component version 2 responsible for monitoring drift in model quality metrics. A SageMaker Model Bias Job Definition Kubeflow Pipelines component version 2A responsible for monitoring bias in a model's predictions. A SageMaker Model Explainability Job Definition Kubeflow Pipelines component version 2 responsible for monitoring drift in feature attribution. Additionally, for on-schedule monitoring at a specified frequency, a fifth component, SageMaker Monitoring Schedule Kubeflow Pipelines component version 2, is responsible for monitoring the data collected from a real-time endpoint on a schedule.

RoboMaker components

• Create Simulation Application The Create Simulation Application component allows you to create a RoboMaker Simulation Application directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  RoboMaker Create Simulation app Kubeflow Pipelines component version 1	X

• Simulation Job The Simulation Job component allows you to run a RoboMaker Simulation Job directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  RoboMaker Simulation Job Kubeflow Pipelines component version 1	X

• Simulation Job Batch The Simulation Job Batch component allows you to run a RoboMaker Simulation Job Batch directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  RoboMaker Simulation Job Batch Kubeflow Pipelines component version 1	X

• Delete Simulation Application The Delete Simulation Application component allows you to delete a RoboMaker Simulation Application directly from a Kubeflow Pipelines workflow.

  Version 1 of the component	Version 2 of the component
  RoboMaker Delete Simulation app Kubeflow Pipelines component version 1	X