Automatic Speech Recognition - OpenAI Whisper Models

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Welcome to Amazon SageMaker Jumpstart! You can use Amazon SageMaker Jumpstart to solve many Machine Learning tasks through one-click in SageMaker Studio, or through SageMaker Python SDK.

In this demo notebook, we demonstrate how to use the SageMaker Python SDK for Automatic Speech Recognition. Automatic Speech recognition or speech-to-text, is a capability which enables a program to process human speech into a written format. Here, we show how to use state-of-the-art pre-trained openai whisper models for automatic speech recognition (ASR). The following OpenAI whisper models are available currently in the SageMaker Jumpstart.

1. Set Up
2. Select a pre-trained model
3. Deploy an Endpoint
4. Query endpoint and parse response
5. Supported Parameters
6. Clean up the endpoint

1. Set Up

Before executing the notebook, there are some initial steps required for set up

2. Select a pre-trained model

3. Deploy an Endpoint

Using SageMaker, we can perform inference on the pre-trained model.

4. Query endpoint and parse response

We will download one of the audio file for the automatic speech recognition. We will pass this file to the predictor for inference.

5. Supported Parameters

This model supports many parameters while performing inference. They include:

• max_length: Model generates text until the output length. If specified, it must be a positive integer.
• language and task: We specify the output language and task here. The model supports the task of transcription or translation.
• max_new_tokens: The maximum numbers of tokens to generate.
• num_return_sequences: Number of output sequences returned. If specified, it must be a positive integer.
• num_beams: Number of beams used in the greedy search. If specified, it must be integer greater than or equal to num_return_sequences.
• no_repeat_ngram_size: Model ensures that a sequence of words of no_repeat_ngram_size is not repeated in the output sequence. If specified, it must be a positive integer greater than 1.
• temperature: Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If temperature -> 0, it results in greedy decoding. If specified, it must be a positive float.
• early_stopping: If True, text generation is finished when all beam hypotheses reach the end of sentence token. If specified, it must be boolean.
• do_sample: If True, sample the next word as per the likelihood. If specified, it must be boolean.
• top_k: In each step of text generation, sample from only the top_k most likely words. If specified, it must be a positive integer.
• top_p: In each step of text generation, sample from the smallest possible set of words with cumulative probability top_p. If specified, it must be a float between 0 and 1.

We may specify any subset of the parameters mentioned above while invoking an endpoint. Next, we show an example of how to invoke endpoint with these arguments

6. Clean up the endpoint

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.