Notebooks
M
Meta Llama Getting Started Llama 3 On Amazon Bedrock

Getting Started Llama 3 On Amazon Bedrock

llamaAIvllmmachine-learning3p-integrationsawsllama2LLMllama-cookbookPythonfinetuningpytorchlangchain
alph-notebooks/llama-cookbook / getting_started_llama_3_on_amazon_bedrock.ipynb

Using Amazon Bedrock with Llama

Open In Colab

Use this notebook to quickly get started with Llama on Bedrock. You can access the Amazon Bedrock API using the AWS Python SDK.

In this notebook, we will give you some simple code to confirm to get up and running with the AWS Python SDK, setting up credentials, looking up the list of available Meta Llama models, and using bedrock to inference.

Resources

Set up the Amazon Bedrock API - https://docs.aws.amazon.com/bedrock/latest/userguide/api-setup.html

To connect programmatically to an AWS service, you use an endpoint. Amazon Bedrock provides the following service endpoints:

  • bedrock – Contains control plane APIs for managing, training, and deploying models.
  • bedrock-runtime – Contains runtime plane APIs for making inference requests for models hosted in Amazon Bedrock.
  • bedrock-agent – Contains control plane APIs for creating and managing agents and knowledge bases.
  • bedrock-agent-runtime – Contains control plane APIs for managing, training, and deploying models.

Prerequisite

Before you can access Amazon Bedrock APIs, you will need an AWS Account, and you will need to request access to the foundation models that you plan to use. For more information on model access - https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html

Setting up the AWS CLI (TBD)

https://docs.aws.amazon.com/bedrock/latest/userguide/api-setup.html#api-using-cli-prereq

Setting up an AWS SDK

https://docs.aws.amazon.com/bedrock/latest/userguide/api-setup.html#api-sdk

Using SageMaker Notebooks

https://docs.aws.amazon.com/bedrock/latest/userguide/api-setup.html#api-using-sage

For more information on Amazon Bedrock, please refer to the official documentation here: https://docs.aws.amazon.com/bedrock/

[2]

Security Note

For this notebook, we will use getpass() to reference your AWS Account credentials. This is just to help you get-started with this notebook more quickly. Otherwise, the we recommend that you avoid using getpass for your AWS credentials in a Jupyter notebook. It's not secure to expose your AWS credentials in this way. Instead, consider using AWS IAM roles or environment variables to securely handle your credentials.

[15]
[16]
[17]
[18]
[19]
[20]
Llama 2 Chat 13B : meta.llama2-13b-chat-v1:0:4k
Llama 2 Chat 13B : meta.llama2-13b-chat-v1
Llama 2 Chat 70B : meta.llama2-70b-chat-v1:0:4k
Llama 2 Chat 70B : meta.llama2-70b-chat-v1
Llama 2 13B : meta.llama2-13b-v1:0:4k
Llama 2 13B : meta.llama2-13b-v1
Llama 2 70B : meta.llama2-70b-v1:0:4k
Llama 2 70B : meta.llama2-70b-v1
[21]
.
Llamas are domesticated mammals that are native to South America. They are known for their distinctive long necks, ears, and legs, as well as their soft, woolly coats. Llamas are members of the camel family, and they are closely related to alpacas and vicuñas.

Here are some interesting facts about llamas:

1. Llamas are known for their intelligence and curious nature. They

'.\nLlamas are domesticated mammals that are native to South America. They are known for their distinctive long necks, ears, and legs, as well as their soft, woolly coats. Llamas are members of the camel family, and they are closely related to alpacas and vicuñas.\n\nHere are some interesting facts about llamas:\n\n1. Llamas are known for their intelligence and curious nature. They'
[ ]
[ ]