Sagemaker Core Feature Store Introduction

Amazon SageMaker Feature Store: Introduction to Feature Store

This notebook demonstrates how to get started with Feature Store, create feature groups, and ingest data into them. These feature groups are stored in your Feature Store.

Feature groups are resources that contain metadata for all data stored in your Feature Store. A feature group is a logical grouping of features, defined in the feature store to describe records. A feature group’s definition is composed of a list of feature definitions, a record identifier name, and configurations for its online and offline store.

Overview

1. Set up
2. Creating a feature group
3. Ingest data into a feature group

Prerequisites

This notebook uses sagemaker_core SDK and Python 3 (Data Science) kernel. This notebook works with Studio, Jupyter, and JupyterLab.

Library dependencies:

• sagemaker_core
• numpy
• pandas

Role requirements:

IMPORTANT: You must attach the following policies to your execution role:

• AmazonS3FullAccess
• AmazonSageMakerFeatureStoreAccess

Set up

Inspect your data

In this notebook example we ingest synthetic data. We read from ./data/feature_store_introduction_customer.csv and ./data/feature_store_introduction_orders.csv.

Below is an illustration on the steps the data goes through before it is ingested into a Feature Store. In this notebook, we illustrate the use-case where you have data from multiple sources and want to store them independently in a feature store. Our example considers data from a data warehouse (customer data), and data from a real-time streaming service (order data).

Create a feature group

We first start by creating feature group names for customer_data and orders_data. Following this, we create two Feature Groups, one for customer_data and another for orders_data

Instantiate a FeatureGroup object for customers_data and orders_data.

Append EventTime feature to your data frame. This parameter is required, and time stamps each data point.

Load feature definitions to your feature group.

Below we call create to create two feature groups, customers_feature_group and orders_feature_group respectively

To confirm that your FeatureGroup has been created we use wait_for_status functions to wait for the feature group to be created successfully.

Add metadata to a feature

We can add searchable metadata fields to FeatureGroup features by using the FeatureMetadata class. The currently supported metadata fields are description and parameters.

To view feature metadata, we can use get method to display that feature.

Feature metadata fields are searchable. We use search API to find features with metadata that matches some search criteria.

Ingest data into a feature group

We can put data into the FeatureGroup by using the PutRecord API. It will take < 1 minute to ingest data.

Creating IngestData function to ingest all dataframe records using PutRecord API call.

Using an arbitrary customer record ID, 573291 we use get_record to check that the data has been ingested into the feature group.

We use batch_get_record to check that all data has been ingested into two feature groups by providing customer IDs.

Add features to a feature group

If we want to update a FeatureGroup that has done the data ingestion, we can use the Update function and then re-ingest data by using the updated dataset.

Inspect the new dataset.

Append EventTime feature to your data frame again.

Ingest the new dataset.

Use batch_get_record again to check that all updated data has been ingested into customers_feature_group by providing customer IDs.

Clean up

Here we remove the Feature Groups we created.