Readthedocs Openai Emb3
ReadtheDocs Retrieval Augmented Generation (RAG) using Zilliz Free Tier
In this notebook, we are going to use Milvus documentation pages to create a chatbot about our product. The chatbot is going to follow RAG steps to retrieve chunks of data using Semantic Vector Search, then the Question + Context will be fed as a Prompt to a LLM to generate an answer.
Many RAG demos use OpenAI for the Embedding Model and ChatGPT for the Generative AI model. In this notebook, we will demo a fully open source RAG stack.
Using open-source Q&A with retrieval saves money since we make free calls to our own data almost all the time - retrieval, evaluation, and development iterations. We only make a paid call to OpenAI once for the final chat generation step.
Let's get started!
Download Milvus documentation to a local directory.
The data we’ll use is our own product documentation web pages. ReadTheDocs is an open-source free software documentation hosting platform, where documentation is written with the Sphinx document generator.
The code block below downloads the web pages into a local directory called rtdocs.
I've already uploaded the rtdocs data folder to github, so you should see it if you cloned my repo.
Start up a Zilliz free tier cluster.
Code in this notebook uses fully-managed Milvus on Ziliz Cloud free trial.
- Choose the default "Starter" option when you provision > Create collection > Give it a name > Create cluster and collection.
- On the Cluster main page, copy your
API Keyand store it locally in a .env variable. See note below how to do that. - Also on the Cluster main page, copy the
Public Endpoint URI.
💡 Note: To keep your tokens private, best practice is to use an env variable. See how to save api key in env variable.
👉🏼 In Jupyter, you need a .env file (in same dir as notebooks) containing lines like this:
- ZILLIZ_API_KEY=f370c...
- OPENAI_API_KEY=sk-H...
- VARIABLE_NAME=value...
Type of server: Zilliz Cloud Vector Database(Compatible with Milvus 2.3)
model_name: text-embedding-3-small EMBEDDING_DIM: 1536
model_name: text-embedding-3-small EMBEDDING_DIM: 512
Successfully created collection: `MilvusDocs_text_embedding_3_small`
{'collection_name': 'MilvusDocs_text_embedding_3_small', 'auto_id': True, 'num_shards': 1, 'description': '', 'fields': [{'field_id': 100, 'name': 'id', 'description': '', 'type': 5, 'params': {}, 'element_type': 0, 'auto_id': True, 'is_primary': True}, {'field_id': 101, 'name': 'vector', 'description': '', 'type': 101, 'params': {'dim': 512}, 'element_type': 0}], 'aliases': [], 'collection_id': 447376431941003159, 'consistency_level': 3, 'properties': {}, 'num_partitions': 1, 'enable_dynamic_field': True}
loaded 8 documents
chunk_size: 511, chunk_overlap: 51.0
chunking time: 0.01361989974975586
docs: 8, split into: 8
split into chunks: 161, type: list of <class 'langchain_core.documents.base.Document'>
Looking at a sample chunk...
pymilvus latest Table of Contents Installation Installing via pip Installing in a virtual environmen
{'h1': 'pymilvus latest Table of Contents Installation Installing via pip Installing in a virtual environmen', 'h2': 'Installing via pip', 'source': '../RAG/rtdocs/pymilvus.readthedocs.io/en/latest/install.html'}
pymilvus latest Table of Contents Installation Installing via pip Installing in a virtual environmen
{'h1': 'pymilvus latest Table of Contents Installation Installing via pip Installing in a virtual environmen', 'h2': 'Installing via pip', 'source': 'https://pymilvus.readthedocs.io/en/latest/install.html'}
Start inserting entities
100%|██████████| 1/1 [00:01<00:00, 1.77s/it]
Milvus Client insert time for 161 vectors: 1.803873062133789 seconds
query length: 75
Milvus Client search time for 161 vectors: 0.059777021408081055 seconds type: <class 'list'>, count: 2
Length context: 509, Number of contexts: 2
Retrieved result #1
distance = 0.5799391269683838
Context: count. HNSW¶ HNSW (Hierarchical Navigable Small World Graph) is a graph-based indexing algorithm. It builds a multi-layer navigation structure for an
Metadata: {'h1': 'pymilvus latest Table of Contents Installation Tut', 'h2': 'Milvus support to create index to accelerate vecto', 'source': 'https://pymilvus.readthedocs.io/en/latest/param.html'}
Retrieved result #2
distance = 0.5569350719451904
Context: HNSW client create_index collection_name IndexType HNSW "M" 16 # int. 4~64 "efConstruction" 40 # int. 8~512 search parameters: ef: Take the effect in
Metadata: {'h1': 'pymilvus latest Table of Contents Installation Tut', 'h2': 'Milvus support to create index to accelerate vecto', 'source': 'https://pymilvus.readthedocs.io/en/latest/param.html'}
Question: What do the parameters for HNSW mean?
('Answer: The parameters for HNSW in Milvus refer to the configuration options '
'for the Hierarchical Navigable Small World index. These parameters include '
'"M" for the number of bi-directional links created for each element during '
'construction, "efConstruction" for the size of the dynamic list for the '
'nearest neighbors search, and "ef" for the size of the dynamic list for the '
'nearest neighbors search during query. These parameters help optimize the '
'performance of the index for efficient similarity search operations. \n'
'Source: https://pymilvus.readthedocs.io/en/latest/param.html')
Author: Christy Bergman Python implementation: CPython Python version : 3.11.7 IPython version : 8.21.0 pymilvus : 2.3.6 langchain: 0.1.5 openai : 1.11.1 conda environment: py311