Weaviate Quickstart

1. Load API Key with .env

2. Initialize Weaviate client

Next, use your API key to initialize your client. Weaviate is open source so you can deploy your clusters and collections locally or use their managed cloud clusters. Just make sure you're pointing to the correct cluster endpoint.

3. Prepare language model for vector encoder

We use a small transformers language model to create 364-dimensional embeddings. You can out models for generating embeddings

4. Create a Weaviate collection

This creates a collection named "quickstart" that performs similarity search with your vectors.

5. Generate vector values from wikipedia text

We retrieve a wikipedia based dataset with Hugging Face's datasets library. Note that this dataset contains Cohere's vectors, but we're generating our own in this notebook.

6. Upsert vectors

Now that you’ve created your collection and the vector embeddings of your wikipedia data, you can upsert these vectors into your collection.

7. Check the that vectors were inserted to the collection

8. Run a similarity search

9. Deploy an app to port forward and share publically

10. Clean up

When you no longer need the collection, call drop_collection and specify the name to shut it down.