Milvus Agent Llamaindex
Milvus + Llama-Index: Enhancing OpenAI Assistant Agent with a Custom Retriever
This shows how to enhance Llama-Index's agent built on top of the OpenAI Assistant API with retriever tool customized by Milvus.
Preparation
1. Install dependencies
2. Start Milvus Service
There are 2 options to start a Milvus service:
- Zilliz Cloud: Zilliz provides cloud-native service for Milvus. It simplifies the process of deploying and scaling vector search applications by eliminating the need to create and maintain complex data infrastructure. Get Started Free!
- Open Source Milvus: You can install the open source Milvus using either Docker Compose or on Kubernetes.
Here, we use Milvus Lite to start with a lightweight version of Milvus, which works seamlessly with Google Colab and Jupyter Notebook.
3. Download example data
You can use any file(s) to build the knowledge base. We will use a SEC file uber_2021.pdf as an example.
Getting Started
1. Set up Environment
You need to set up some environment variables, for example, passing your OpenAI API Key. Please note that your OpenAI account should have the accessibility and enough quota available for the model GPT-4 Turbo.
2. Customize Strategies
In this step, we will define some strategies to be used:
- Chunking: configure the text splitter (e.g.
chunk_size) - Embedding: choose embedding model (e.g.
BAAI/bge-small-en) and its provider (e.g. HuggingFace, OpenAI). - LLM: select LLM model (e.g.
gpt-4-1106-preview) and set up model parameters (e.g.temperature).
/home/mengjia.gu/anaconda3/envs/develop/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
3. Ingest Document(s)
4. Define Agent & Tool(s)
In order integrate the vector store index with agent, we need to define the index as a Retriever Tool. The agent will be able to recognize the retriever via the tool's name and description in metadata.
Then let's define the agent powered by OpenAI's Assistants API. To create a agent, we will define its role, give instructions, and provide tools. Here we will make LLM thinking itself a SEC analyst, with Milvus retriever as an available tool.
Try it out!
Now the agent is ready as a SEC analyst. It is able to respond to users based off documents loaded into Milvus.
With verbose=True, you are able to what information are retrieved when the agent's answering your question.
=== Calling Function ===
Calling function: CustomRetriever with args: {"input":"Uber Technologies, Inc. annual revenue growth 2021"}
Got output: page_label = 77
file_name = uber_2021.pdf
file_path = uber_2021.pdf
UBER TECHNOLOGIES, INC.CONSOLIDATED STATEMENTS OF
OPERATIONS(In millions, except share amounts which are ref
lected in thousands, and per share amounts)Year Ended December 31,
2019
2020 2021 Revenue
$ 13,000 $ 11,139 $ 17,455 Costs and expenses
Cost of revenue, exclusive of dep
reciation and amortization shown separately below6,061 5,154 9,351 Operations and support
2,302 1,819 1,877 Sales and marketing
4,626 3,583 4,789 Research and development
4,836 2,205 2,054 General and administrative
3,299 2,666 2,316 Depreciation and amortization
472 575 902 Total costs and expenses
21,596 16,002 21,289 Loss from operations
(8,596) (4,863) (3,834) Interest expense
(559) (458) (483) Other income (expense), net
722 (1,625) 3,292 Loss before income taxes and loss from equity me
thod investments(8,433) (6,946) (1,025) Provision for (benefit fro
m) income taxes45 (192) (492) Loss from equity method invest
ments(34) (34) (37) Net loss including non-controlling interests
(8,512) (6,
page_label = 57
file_name = uber_2021.pdf
file_path = uber_2021.pdf
(61) %(3) % Totals of percentage of
revenues may not foot due to rounding.Comparison of the Years Ended December 31, 2020 and 2021
Revenue
Year Ended December 31,
2020 to 2021 % Change
(In millions, except percentages) 2020 2021 Revenue
$ 11,139 $ 17,455 57 %2021
Compared to 2020Revenue
increased $6.3 billion, or 57%, primarily attributable to an increase in Gross Bookings of 56%, or 53% on a constant currency basis. The increase inGross
Bookings was primarily driven by an increase in Delivery Gross Bookings of 71%, or 66% on a constant currency basis, due to an increase in food deliveryorders
and higher basket sizes as a result of stay-at-home order demand related to COVID-19, as well as continued expansion across U.S. and internationalmarkets.
The increase was also driven by Mobility Gross Bookings growth of 38%, or 36% on a constant currency basis, due to increases in Trip volumes as thebusiness
recovers from the impacts of COVID-19.
page_label = 82
file_name = uber_2021.pdf
file_path = uber_2021.pdf
UBER TECHNOLOGIES, INC.CONSOLIDATED STATEMENTS OF
CASH FLOWS(In millions)
Year Ended December 31,
2019
2020 2021 Cash flows from operating activities
Net loss including non-controll
ing interests$ (8,512) $ (6,788) $ (570) Adjustments to reconcile n
et loss to net cash used in operating activities:Depreciation and amortization
472 575 902 Bad debt expense
92 76 109 Stock-based compensation
4,596 827 1,168 Gain on extinguishment of conver
tible notes and settlement of derivatives(444) — — Gain from sale of investm
ents— — (413) Gain on business divestitures, net
— (204) (1,684) Deferred income taxes
(88) (266) (692) Impairment of debt and
equity securities— 1,690 — Impairments of goodwill, long
-lived assets and other assets— 404 116 Loss from equity method invest
ments34 34 37 Unrealized (gain) loss on deb
t and equity securities, net(2) 125 (1,142) Unrealized foreign cur
rency transactions16 48 38 Other
15 2 4 Change in assets and liabili
ties, net of impact of business acquisitions and disposals:Accounts receivable
(407) 142 (597) Prepaid expenses and other asse
ts(478) 94 (236) Collateral held by insurer
(1,
========================
Check the agent's answer:
Uber's revenue growth in 2021 was 57 percent compared to 2020, with revenue increasing from $11,139 million in 2020 to $17,455 million in 2021.
Optional
For Milvus-Lite, stop the service at the end.