Time-Series Forecasting with Amazon SageMaker Autopilot

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Contents

1. Introduction
2. Setup
3. Model Training
4. Real-Time Predictions (Inference)
5. Batch Predictions (Inference)

1. Introduction

This notebook uses Amazon SageMaker Autopilot to train a time-series model and produce predictions against the trained model. At the top-level, customers provide a set of tabular historical data on S3 and make an API to train a model. Once the model has been trained, you can elect to produce prediction as a batch or via a real-time endpoint. As part of the training process, SageMaker Autopilot manages and runs multiple time series models concurrently. All of these models are combined into a single ensembled model which blends the candidate models in a ratio that minimizes forecast error. Customers are provided with metadata and models for the ensemble and all underlying candidate models too. SageMaker Autopilot orchestrates this entire process and provides several artifacts as a result.

These artifacts include:

• backtest (holdout) forecasts per base model over multiple time windows,
• accuracy metrics per base model,
• backtest results and accuracy metrics for the ensembled model,
• a scaled explainability report displaying the importance of each covariate and static metadata feature.
• all model artifacts are provided as well on S3, which can be registered or use for batch/real-time inference

2. Setup

We provide a sample set of data to accompany this notebook. You may use our synthetic dataset, or alter this notebook to accommodate your own data. As a note, the next cell will copy a file to your S3 bucket and prefix defined in the last cell. As an alternate, we provide a method to copy the file to your local disk too.

IMPORTANT: When training a model, your input data can contain a mixture of covariate and static item metadata. Take care to create future-dated rows that extend to the end of your prediction horizon. In the future-dated rows, carry your static item metadata and expected covariate values. Future-dated target-value (y) should be empty. Please download the example synthetic file using the S3 copy command in the next cell. You can observe the data programmatically or in a text editor as an example.

The structure of the CSV file provided is as follows:

• product_code (required: ItemIdentifierAttributeName)
• product_category (static, categorical feature describing product_code)
• product_subcategory (static, categorical feature describing product_code)
• location_code (GroupingAttributeNames column to get predictions at product_code + location_code)
• scaled_price (covariate)
• promotion_email (covariate)
• promotion_homepage (covariate)
• timestamp (required, TimestampAttributeName)
• unit_sales (required: TargetAttributeName)

3. Model Training

Establish an AutoML training job name

Define training job specifications. More information about create_auto_ml_job_v2 can be found in our SageMaker documentation.This JSON body leverages the built-in sample data schema. Please consult the documentation to understand how to alter the parameters for your unique schema.

With parameters now defined, invoke the training job and monitor for its completion. You can expect the training to take about 1 hour.

Next, we demonstrate a looping mechanism to query (monitor) job status. When the status is Completed, you may review the accuracy of the model and decide whether to perform inference on a batch or real-time API basis as described in this notebook. Please consult documentation for describe_auto_ml_job_v2 as needed.

Once training is completed, you can use the describe function to iterate over model leaderboard results. Below is an example to use the best candidate in the subsequent inference phase. Please consult our documentation on create_model as needed.

4. Real-Time Predictions (Inference)

If you want to perform real-time inference, review this section. If you want to perform batch processing, you may skip the real-time inference section and move to Batch Predictions (Inference).

Define a model, endpoint configuration and endpoint name using the candidate metadata. Consult create_endpoint_config documentation for more detail. Additionally, please adjust the InstanceType and InitialInstanceCount according to need.

IMPORTANT: The data you supply for inference must have at least four valid historical values for each time-series.

Next, you can deploy a real-time endpoint using the create_endpoint API. See the documentation for more details and options.

Poll for endpoint to become ready to serve (InService)

The next cells help demonstrate opening a local CSV file for inference. Alternately, this data could come from S3, a database query or live application. In this example, the data is loaded into a Python memory object.

Method to instantiate SageMaker runtime client and invoke endpoint. Please note the guidance given in the documentation page which says the response must be within 60 seconds of invocation. Response times are a function of payload size. You should take care to provide history for single time-series, and carefully testing the ability to perform predictions for more than one time-series at a time.

At this point, the results from the real-time API call are loaded into a variable called prediction. You have several options on what you do with the results. A few are given here.

Optional: Example of saving the resulting real-time predictions to a local filesystem. Carefully plan your naming convention.

Optional: Example of saving the resulting real-time predictions to a S3 object. Carefully plan your naming convention.

Optional: Example of loading predictions into a Panda dataframe

Finally, you may also elect to place predictions in a SQS queue, post them to a stream, or post them to another real-time API according to need.

Cleanup Real-time Endpoint Resources

As needed, you can stop the endpoint and related billing costs as follows. When you need the endpoint again, you can follow the deployment steps again. Ideally, at a future time, another newer model is trained and able to be deployed as well.

Caution: Do not delete the model if you intend on testing batch transformation too. If you do delete the model, you may redeploy it as long as the model artifact exists on S3. The next cell provides the container and S3 location for your best candidate model.

5. Batch Predictions (Inference)

Please review service limits with batch transform. At the time of writing, the documentation says the maximum size of the input data per invocation is 100 MB. Translated, when working with datasets over 100MB, you will need to prepare your data by splitting/sharding into multiple files. Take care to ensure each file contains whole time series. One potential way to do this is to use a function that splits data on the item key, or similar.

Launch Batch Transformation Job using create_transform_job. The runtime of the job is a function of the size of your data and the InstanceType and InstanceCount provided. Once the task is complete, results are available on S3 at the declared S3OutputPath location. From there, you can use an event handler or other mechanism to consume the results.

The next cell downloads a dataset once again and this time places in a batch_transform/input folder. Ideally, this input dataset can be all of your time-series, or a fraction thereof. Please take care to ensure the dataset is within the limits described.

IMPORTANT: The data you supply for inference must have at least four valid historical values for each time-series.

Poll for batch transformation job to complete. Once completed, resulting prediction files are available at the URI shown in the prior cell, S3OutputPath. We use the API method describe_transform_job to complete this step.

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.