Introduction

This notebook shows how to replace the VolatileMemoryStore memory storage used in a previous notebook with a WeaviateMemoryStore.

WeaviateMemoryStore is an example of a persistent (i.e. long-term) memory store backed by the Weaviate vector database.

About Weaviate

Weaviate is an open-source vector database designed to scale seamlessly into billions of data objects. This implementation supports hybrid search out-of-the-box (meaning it will perform better for keyword searches).

You can run Weaviate in 5 ways:

• SaaS – with Weaviate Cloud Services (WCS).

  WCS is a fully managed service that takes care of hosting, scaling, and updating your Weaviate instance. You can try it out for free with a sandbox that lasts for 14 days.

  To set up a SaaS Weaviate instance with WCS:

  1. Navigate to Weaviate Cloud Console.
  2. Register or sign in to your WCS account.
  3. Create a new cluster with the following settings:
     • Subscription Tier – Free sandbox for a free trial, or contact hello@weaviate.io for other options.
     • Cluster name – a unique name for your cluster. The name will become part of the URL used to access this instance.
     • Enable Authentication? – Enabled by default. This will generate a static API key that you can use to authenticate.
  4. Wait for a few minutes until your cluster is ready. You will see a green tick ✔️ when it's done. Copy your cluster URL.

• Hybrid SaaS

  If you need to keep your data on-premise for security or compliance reasons, Weaviate also offers a Hybrid SaaS option: Weaviate runs within your cloud instances, but the cluster is managed remotely by Weaviate. This gives you the benefits of a managed service without sending data to an external party.

  The Weaviate Hybrid SaaS is a custom solution. If you are interested in this option, please reach out to hello@weaviate.io.

• Self-hosted – with a Docker container

  To set up a Weaviate instance with Docker:

  1. Install Docker on your local machine if it is not already installed.

  2. Install the Docker Compose Plugin

  3. Download a docker-compose.yml file with this curl command:

     curl -o docker-compose.yml "https://configuration.weaviate.io/v2/docker-compose/docker-compose.yml?modules=standalone&runtime=docker-compose&weaviate_version=v1.19.6"
     

     Alternatively, you can use Weaviate's docker compose configuration tool to generate your own docker-compose.yml file.

  4. Run docker compose up -d to spin up a Weaviate instance.

     To shut it down, run docker compose down.

• Self-hosted – with a Kubernetes cluster

  To configure a self-hosted instance with Kubernetes, follow Weaviate's documentation.|

Setup

First, we instantiate the Weaviate memory store. Uncomment ONE of the options below, depending on how you want to use Weaviate:

• from a Docker instance
• from WCS

Then, we register the memory store to the kernel:

Manually adding memories

Let's create some initial memories "About Me". We can add memories to our weaviate memory store by using save_information_async

Searching is done through SearchAsync:

Let's see the results of the functions:

Here's how to use the weaviate memory store in a chat application:

Adding documents to your memory

Create a dictionary to hold some files. The key is the hyperlink to the file and the value is the file's content:

Use SaveInformationAsync to save the file:

Use SearchAsync to ask a question: