Amazon SageMaker Model Governance - Model Cards

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

This notebook walks you through the features of Amazon SageMaker Model Cards. For more information, see Model Cards in the Amazon SageMaker Developer Guide.

Amazon SageMaker Model Cards give you the ability to create a centralized, customizable fact-sheet to document critical details about your machine learning (ML) models. Use model cards to keep a record of model information, such as intended uses, risk ratings, training details, evaluation metrics, and more for streamlined governance and reporting.

In this example, you create a binary classification model along with a model card to document model details along the way. Learn how to create, read, update, delete, and export model cards using the Amazon SageMaker Python SDK.

Contents

1. Setup
2. Prepare a Binary Classification Model
3. Create Model Card
4. Update Model Card
5. Load Model Card
6. List Model Card History
7. Export Model Card
8. Cleanup

Setup

To begin, you must specify the following information:

• The IAM role ARN used to give SageMaker training and hosting access to your data. The following example uses the SageMaker execution role.
• The SageMaker session used to manage interactions with Amazon SageMaker Model Card API methods.
• The S3 URI (bucket and prefix) where you want to store training artifacts, models, and any exported model card PDFs. This S3 bucket should be in the same Region as your Notebook Instance, training, and hosting configurations. The following example uses the default SageMaker S3 bucket and creates a default SageMaker S3 bucket if one does not already exist.
• The S3 session used to manage interactions with Amazon S3 storage.

Next, import the necessary Python libraries.

Prepare a Model

The following code creates an example binary classification model trained on a synthetic dataset. The target variable (0 or 1) is the second variable in the tuple.

1. Prepare the training data

The code will upload example data to your S3 bucket.

2. Train a model

Train a binary classification model with the training data from the previous step.

3. Create a model

Create Model Card

Document your binary classification model details in an Amazon SageMaker Model Card using the SageMaker Python SDK.

1. Auto-collect model details

Automatically collect basic model information like model ID, training environment, and the model output S3 URI. Add additional model information such as a description, problem type, algorithm type, model creator, and model owner.

2. Auto-collect training details

Automatically collect basic training information like training ID, training environment, and training metrics. Add additional training information such as objective function details and training observations.

3. Collect evaluation details

Add evaluation observations, datasets, and metrics.

(Optional) 3.1 Parse metrics from existing evaluation report

If you have existing evaluation reports generated by SageMaker Clarify or SageMaker Model Monitor, upload them to S3 and provide an S3 URI to automatically parse evaluation metrics. To add your own generic model card evaluation report, provide a report in the evaluation results JSON format. See the example JSON files in the ./example_metrics folder for reference.

Collect metrics from a JSON format evaluation report

Collect metrics from S3

4. Collect additional details

Add the intended uses of your model and business details and any additional information that you want to include in your model card. For more information on intended uses and business details, see Model Cards in the Amazon SageMaker Developer Guide.

5. Initialize a model card

Initialize a model card with the information collected in the previous steps.

Update Model Card

After creating a model card, you can update the model card information. Updating a model card creates a new model card version.

Load Model Card

Load an existing model card with the model card name.

List Model Card History

Track the model card history by listing historical versions.

Export Model Card

Share the model card by exporting it to a PDF file.

1. Create an export job

(optional) List export jobs

Check all the export jobs for this model card.

2. Download the exported model card PDF

The downloaded PDF is stored in the same directory as this notebook by default.

Parse the bucket and key of the exported PDF

Download

Cleanup

Delete the following resources:

1. The model card
2. The exported model card PDF
3. The example binary classification model

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.