Responses Api Tool Orchestration
chatgptopenairesponses_apigpt-4examplesopenai-apiopenai-cookbook
Export
Multi-Tool Orchestration with RAG approach using OpenAI's Responses API
This cookbook guides you through building dynamic, multi-tool workflows using OpenAI's Responses API. It demonstrates how to implement a Retrieval-Augmented Generation (RAG) approach that intelligently routes user queries to the appropriate in-built or external tools. Whether your query calls for general knowledge or requires accessing specific internal context from a vector database (like Pinecone), this guide shows you how to integrate function calls, web searches in-built tool, and leverage document retrieval to generate accurate, context-aware responses.
For a practical example of performing RAG on PDFs using the Responses API's file search feature, refer to this notebook.
This example showcases the flexibility of the Responses API, illustrating that beyond the internal file_search tool—which connects to an internal vector store—there is also the capability to easily connect to external vector databases. This allows for the implementation of a RAG approach in conjunction with hosted tooling, providing a versatile solution for various retrieval and generation tasks.
[2]
[notice] A new release of pip is available: 24.0 -> 25.0.1 [notice] To update, run: pip install --upgrade pip Note: you may need to restart the kernel to use updated packages.
/Users/shikhar/openai_projects/github_repos/success-git/success_new/success/oneoffs/shikhar/responses_rag_cookbook/env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
In this example we use a sample medical reasoning dataset from Hugging Face. We convert the dataset into a Pandas DataFrame and merge the “Question” and “Response” columns into a single string. This merged text is used for embedding and later stored as metadata.
[3]
Example merged text: Question: A 61-year-old woman with a long history of involuntary urine loss during activities like coughing or sneezing but no leakage at night undergoes a gynecological exam and Q-tip test. Based on these findings, what would cystometry most likely reveal about her residual volume and detrusor contractions? Answer: Cystometry in this case of stress urinary incontinence would most likely reveal a normal post-void residual volume, as stress incontinence typically does not involve issues with bladder emptying. Additionally, since stress urinary incontinence is primarily related to physical exertion and not an overactive bladder, you would not expect to see any involuntary detrusor contractions during the test.
[4]
Create a Pinecone Index Based on the Dataset
Use the dataset itself to determine the embedding dimensionality. For example, compute one embedding from the merged column and then create the index accordingly.
[5]
Embedding dimension: 1536
[9]
Index stats: {'dimension': 1536,
'index_fullness': 0.0,
'metric': 'dotproduct',
'namespaces': {},
'total_vector_count': 0,
'vector_type': 'dense'}
Upsert the Dataset into Pinecone index
Process the dataset in batches, generate embeddings for each merged text, prepare metadata (including separate Question and Answer fields), and upsert each batch into the index. You may also update metadata for specific entries if needed.
[10]
Upserting to Pinecone: 100%|██████████| 4/4 [00:06<00:00, 1.64s/it]
Query the Pinecone Index
Create a natural language query, compute its embedding, and perform a similarity search on the Pinecone index. The returned results include metadata that provides context for generating answers.
[11]
[12]
Query Results: 0.70: A 45-year-old man with a history of alcohol use, who has been abstinent for the past 10 years, presents with sudden onset dysarthria, shuffling gait, and intention tremors. Given this clinical presentation and history, what is the most likely diagnosis? - Considering the clinical presentation of sudden onset dysarthria, shuffling gait, and intention tremors in a 45-year-old man with a history of alcohol use who has been abstinent for the past 10 years, the most likely diagnosis is acquired hepatocerebral degeneration. This condition is associated with chronic liver disease, which can often be a consequence of long-term alcohol use. Despite the patient's abstinence from alcohol for a decade, previous alcohol use may have led to underlying liver dysfunction. This dysfunction, even if subclinical, can cause encephalopathy due to the accumulation of neurotoxic substances that affect the brain. The sudden onset of these neurological symptoms aligns with how acquired hepatocerebral degeneration can manifest, making it a probable diagnosis in this scenario. 0.55: A 45-year-old man presents with symptoms including a wide-based gait, a blank facial expression, hallucinations, memory issues, a resting tremor that resolves with movement, and bradykinesia. Based on these clinical findings, what is most likely to be observed in the histological specimen of his brain? - Based on the clinical findings presented—wide-based gait, blank facial expression, hallucinations, memory issues, resting tremor that resolves with movement, and bradykinesia—it is likely that the 45-year-old man is experiencing a condition related to Parkinsonism, possibly Parkinson's disease or dementia with Lewy bodies. Both of these conditions are associated with the presence of Lewy bodies in the brain. Lewy bodies are abnormal aggregates of protein, primarily alpha-synuclein, which can cause both the motor and cognitive symptoms observed in this patient. Therefore, in the histological specimen of his brain, you would most likely observe the presence of Lewy bodies. 0.53: A 73-year-old man is evaluated for increasing forgetfulness, getting lost while walking, irritability, and difficulty recalling recent events while retaining detailed memories from over 20 years ago. On examination, he is oriented to person and place but disoriented to time, and an MRI of the brain reveals significant changes. Considering these symptoms and the imaging findings, what is the most likely underlying pathological process contributing to the patient's condition? - The symptoms and MRI findings of this 73-year-old man suggest the most likely underlying pathological process is the buildup of amyloid-beta plaques and tau protein tangles, which are characteristic of Alzheimer's disease. These changes often begin in brain regions involved in memory, such as the hippocampus and temporal lobes, leading to the gradual memory decline, disorientation, and personality changes observed in the patient. 0.42: A 2-day-old male newborn delivered at 36 weeks presents with generalized convulsions, lethargy, feeding difficulties, icterus, purpura, posterior uveitis, and failed auditory screening. Cranial ultrasonography shows ventricular dilatation and hyperechoic foci in multiple brain areas. Considering these clinical signs and history, what is the most likely diagnosis? - The symptoms and findings you've described in this 2-day-old newborn point towards congenital Toxoplasmosis. The combination of neurological symptoms (such as convulsions and ventricular dilatation with hyperechoic foci), the presence of posterior uveitis, and the skin manifestations like purpura, all fit into the classic presentation of a TORCH infection. Toxoplasmosis, specifically, is known to cause widespread calcifications in the brain, not limited to the periventricular areas, which matches the ultrasound findings. Additionally, while hearing loss is more traditionally associated with CMV, it can also occur in Toxoplasmosis. Thus, the most likely diagnosis given this clinical picture is congenital Toxoplasmosis. 0.42: A 45-year-old male patient experiences double vision specifically when walking upstairs. Considering his well-controlled history of Type-II diabetes, which cranial nerve is most likely involved in his symptoms? - Based on the symptoms described, the cranial nerve most likely involved in the double vision experienced by this patient while walking upstairs is the trochlear nerve, or cranial nerve IV. This nerve controls the superior oblique muscle, which plays a role in stabilizing the eye during certain movements, including the coordination required when looking upwards while walking upstairs. Given the patient's history of diabetes, cranial neuropathies can occur, and CN IV involvement can lead to vertical diplopia that becomes noticeable during specific activities like walking up stairs. Therefore, the trochlear nerve is a likely candidate for the involvement in these symptoms.
{'matches': [{'id': '1',
, 'metadata': {'Answer': 'Considering the clinical presentation of '
, 'sudden onset dysarthria, shuffling gait, '
, 'and intention tremors in a 45-year-old '
, 'man with a history of alcohol use who '
, 'has been abstinent for the past 10 '
, 'years, the most likely diagnosis is '
, 'acquired hepatocerebral degeneration.\n'
, '\n'
, 'This condition is associated with '
, 'chronic liver disease, which can often '
, 'be a consequence of long-term alcohol '
, "use. Despite the patient's abstinence "
, 'from alcohol for a decade, previous '
, 'alcohol use may have led to underlying '
, 'liver dysfunction. This dysfunction, '
, 'even if subclinical, can cause '
, 'encephalopathy due to the accumulation '
, 'of neurotoxic substances that affect the '
, 'brain. The sudden onset of these '
, 'neurological symptoms aligns with how '
, 'acquired hepatocerebral degeneration can '
, 'manifest, making it a probable diagnosis '
, 'in this scenario.',
, 'Question': 'A 45-year-old man with a history of '
, 'alcohol use, who has been abstinent '
, 'for the past 10 years, presents with '
, 'sudden onset dysarthria, shuffling '
, 'gait, and intention tremors. Given '
, 'this clinical presentation and '
, 'history, what is the most likely '
, 'diagnosis?'},
, 'score': 0.697534442,
, 'values': []},
, {'id': '2',
, 'metadata': {'Answer': 'Based on the clinical findings '
, 'presented—wide-based gait, blank facial '
, 'expression, hallucinations, memory '
, 'issues, resting tremor that resolves '
, 'with movement, and bradykinesia—it is '
, 'likely that the 45-year-old man is '
, 'experiencing a condition related to '
, "Parkinsonism, possibly Parkinson's "
, 'disease or dementia with Lewy bodies. '
, 'Both of these conditions are associated '
, 'with the presence of Lewy bodies in the '
, 'brain. Lewy bodies are abnormal '
, 'aggregates of protein, primarily '
, 'alpha-synuclein, which can cause both '
, 'the motor and cognitive symptoms '
, 'observed in this patient. Therefore, in '
, 'the histological specimen of his brain, '
, 'you would most likely observe the '
, 'presence of Lewy bodies.',
, 'Question': 'A 45-year-old man presents with '
, 'symptoms including a wide-based gait, '
, 'a blank facial expression, '
, 'hallucinations, memory issues, a '
, 'resting tremor that resolves with '
, 'movement, and bradykinesia. Based on '
, 'these clinical findings, what is most '
, 'likely to be observed in the '
, 'histological specimen of his brain?'},
, 'score': 0.55345,
, 'values': []},
, {'id': '19',
, 'metadata': {'Answer': 'The symptoms and MRI findings of this '
, '73-year-old man suggest the most likely '
, 'underlying pathological process is the '
, 'buildup of amyloid-beta plaques and tau '
, 'protein tangles, which are '
, "characteristic of Alzheimer's disease. "
, 'These changes often begin in brain '
, 'regions involved in memory, such as the '
, 'hippocampus and temporal lobes, leading '
, 'to the gradual memory decline, '
, 'disorientation, and personality changes '
, 'observed in the patient.',
, 'Question': 'A 73-year-old man is evaluated for '
, 'increasing forgetfulness, getting lost '
, 'while walking, irritability, and '
, 'difficulty recalling recent events '
, 'while retaining detailed memories from '
, 'over 20 years ago. On examination, he '
, 'is oriented to person and place but '
, 'disoriented to time, and an MRI of the '
, 'brain reveals significant changes. '
, 'Considering these symptoms and the '
, 'imaging findings, what is the most '
, 'likely underlying pathological process '
, "contributing to the patient's "
, 'condition?'},
, 'score': 0.526201367,
, 'values': []},
, {'id': '38',
, 'metadata': {'Answer': "The symptoms and findings you've "
, 'described in this 2-day-old newborn '
, 'point towards congenital Toxoplasmosis. '
, 'The combination of neurological symptoms '
, '(such as convulsions and ventricular '
, 'dilatation with hyperechoic foci), the '
, 'presence of posterior uveitis, and the '
, 'skin manifestations like purpura, all '
, 'fit into the classic presentation of a '
, 'TORCH infection. Toxoplasmosis, '
, 'specifically, is known to cause '
, 'widespread calcifications in the brain, '
, 'not limited to the periventricular '
, 'areas, which matches the ultrasound '
, 'findings. Additionally, while hearing '
, 'loss is more traditionally associated '
, 'with CMV, it can also occur in '
, 'Toxoplasmosis. Thus, the most likely '
, 'diagnosis given this clinical picture is '
, 'congenital Toxoplasmosis.',
, 'Question': 'A 2-day-old male newborn delivered at '
, '36 weeks presents with generalized '
, 'convulsions, lethargy, feeding '
, 'difficulties, icterus, purpura, '
, 'posterior uveitis, and failed auditory '
, 'screening. Cranial ultrasonography '
, 'shows ventricular dilatation and '
, 'hyperechoic foci in multiple brain '
, 'areas. Considering these clinical '
, 'signs and history, what is the most '
, 'likely diagnosis?'},
, 'score': 0.422916651,
, 'values': []},
, {'id': '31',
, 'metadata': {'Answer': 'Based on the symptoms described, the '
, 'cranial nerve most likely involved in '
, 'the double vision experienced by this '
, 'patient while walking upstairs is the '
, 'trochlear nerve, or cranial nerve IV. '
, 'This nerve controls the superior oblique '
, 'muscle, which plays a role in '
, 'stabilizing the eye during certain '
, 'movements, including the coordination '
, 'required when looking upwards while '
, "walking upstairs. Given the patient's "
, 'history of diabetes, cranial '
, 'neuropathies can occur, and CN IV '
, 'involvement can lead to vertical '
, 'diplopia that becomes noticeable during '
, 'specific activities like walking up '
, 'stairs. Therefore, the trochlear nerve '
, 'is a likely candidate for the '
, 'involvement in these symptoms.',
, 'Question': 'A 45-year-old male patient experiences '
, 'double vision specifically when '
, 'walking upstairs. Considering his '
, 'well-controlled history of Type-II '
, 'diabetes, which cranial nerve is most '
, 'likely involved in his symptoms?'},
, 'score': 0.420719624,
, 'values': []}],
, 'namespace': '',
, 'usage': {'read_units': 6}} Generate a Response Using the Retrieved Context
Select the best matching result from your query results and use the OpenAI Responses API to generate a final answer by combining the retrieved context with the original question.
[15]
Final Answer: The presentation of confusion, ataxia, and ophthalmoplegia in a 45-year-old man with a history of alcohol use is suggestive of Wernicke's encephalopathy. This condition is caused by thiamine (vitamin B1) deficiency, often associated with chronic alcohol use. The recommended treatment is the immediate administration of thiamine, typically given intravenously or intramuscularly, to prevent progression to more severe neurological damage or Korsakoff syndrome.
Orchestrate Multi-Tool Calls
Now, we'll define the built-in function available through the Responses API, including the ability to invoke the external Vector Store - Pinecone as an example.
Web Search Preview Tool: Enables the model to perform live web searches and preview the results. This is ideal for retrieving real-time or up-to-date information from the internet.
Pinecone Search Tool: Allows the model to query a vector database using semantic search. This is especially useful for retrieving relevant documents—such as medical literature or other domain-specific content—that have been stored in a vectorized format.
[19]
[20]
[21]
🌟--- Processing Query ---🌟
🔍 **User Query:** Who won the cricket world cup in 1983?
✨ **Initial Response Output:**
[ResponseOutputMessage(id='msg_67e6e7a9f7508191a9d18c3ff25310290811a0720cf47168', content=[ResponseOutputText(annotations=[], text='India won the Cricket World Cup in 1983.', type='output_text')], role='assistant', status='completed', type='message')]
💡 **Final Answer:**
India won the Cricket World Cup in 1983.
🌟--- Processing Query ---🌟
🔍 **User Query:** What is the most common cause of death in the United States according to the internet?
✨ **Initial Response Output:**
[ResponseFunctionWebSearch(id='ws_67e6e7aad0248191ab974d4b09b460c90537f90023d2dd32', status='completed', type='web_search_call'), ResponseOutputMessage(id='msg_67e6e7ace08081918f06b5cac32e8c0e0537f90023d2dd32', content=[ResponseOutputText(annotations=[AnnotationURLCitation(end_index=363, start_index=225, title='10 Leading Causes of Death in the U.S.', type='url_citation', url='https://www.usnews.com/news/healthiest-communities/slideshows/top-10-causes-of-death-in-america?slide=11&utm_source=openai'), AnnotationURLCitation(end_index=753, start_index=625, title='Top causes of death in the US — see the CDC’s latest list - Rifnote', type='url_citation', url='https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai'), AnnotationURLCitation(end_index=1014, start_index=886, title='Top causes of death in the US — see the CDC’s latest list - Rifnote', type='url_citation', url='https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai'), AnnotationURLCitation(end_index=1216, start_index=1061, title='US deaths are down and life expectancy is up, but improvements are slowing', type='url_citation', url='https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai'), AnnotationURLCitation(end_index=1394, start_index=1219, title='A Mysterious Health Wave Is Breaking Out Across the U.S.', type='url_citation', url='https://www.theatlantic.com/ideas/archive/2024/12/violence-obesity-overdoses-health-covid/681079/?utm_source=openai')], text='According to the Centers for Disease Control and Prevention (CDC), heart disease was the leading cause of death in the United States in 2023, accounting for 680,980 deaths, which is approximately 22% of all deaths that year. ([usnews.com](https://www.usnews.com/news/healthiest-communities/slideshows/top-10-causes-of-death-in-america?slide=11&utm_source=openai))\n\nThe top 10 causes of death in the U.S. for 2023 were:\n\n1. Heart disease\n2. Cancer\n3. Unintentional injury\n4. Stroke\n5. Chronic lower respiratory diseases\n6. Alzheimer’s disease\n7. Diabetes\n8. Kidney disease\n9. Chronic liver disease and cirrhosis\n10. COVID-19\n\n([rifnote.com](https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai))\n\nNotably, COVID-19, which was the fourth leading cause of death in 2022, dropped to the tenth position in 2023, with 76,446 deaths. ([rifnote.com](https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai))\n\n\n## Recent Trends in U.S. Mortality Rates:\n- [US deaths are down and life expectancy is up, but improvements are slowing](https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai)\n- [A Mysterious Health Wave Is Breaking Out Across the U.S.](https://www.theatlantic.com/ideas/archive/2024/12/violence-obesity-overdoses-health-covid/681079/?utm_source=openai) ', type='output_text')], role='assistant', status='completed', type='message')]
💡 **Final Answer:**
According to the Centers for Disease Control and Prevention (CDC), heart disease was the leading cause of death in the United States in 2023, accounting for 680,980 deaths, which is approximately 22% of all deaths that year. ([usnews.com](https://www.usnews.com/news/healthiest-communities/slideshows/top-10-causes-of-death-in-america?slide=11&utm_source=openai))
The top 10 causes of death in the U.S. for 2023 were:
1. Heart disease
2. Cancer
3. Unintentional injury
4. Stroke
5. Chronic lower respiratory diseases
6. Alzheimer’s disease
7. Diabetes
8. Kidney disease
9. Chronic liver disease and cirrhosis
10. COVID-19
([rifnote.com](https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai))
Notably, COVID-19, which was the fourth leading cause of death in 2022, dropped to the tenth position in 2023, with 76,446 deaths. ([rifnote.com](https://rifnote.com/health/2024/08/11/top-causes-of-death-in-the-us-see-the-cdcs-latest-list/?utm_source=openai))
## Recent Trends in U.S. Mortality Rates:
- [US deaths are down and life expectancy is up, but improvements are slowing](https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai)
- [A Mysterious Health Wave Is Breaking Out Across the U.S.](https://www.theatlantic.com/ideas/archive/2024/12/violence-obesity-overdoses-health-covid/681079/?utm_source=openai)
🌟--- Processing Query ---🌟
🔍 **User Query:** A 7-year-old boy with sickle cell disease is experiencing knee and hip pain, has been admitted for pain crises in the past, and now walks with a limp. His exam shows a normal, cool hip with decreased range of motion and pain with ambulation. What is the most appropriate next step in management according to the internal knowledge base?
✨ **Initial Response Output:**
[ResponseFunctionToolCall(arguments='{"query":"7-year-old sickle cell disease knee hip pain limp normal cool hip decreased range of motion"}', call_id='call_ds0ETZbYtX71U2bQZXTBEWxN', name='PineconeSearchDocuments', type='function_call', id='fc_67e6e7b03ee48191bb400c13c359c35e0aeeec60d0806312', status='completed')]
🔧 **Model triggered a tool call:** PineconeSearchDocuments
🔍 **Invoking PineconeSearchDocuments tool...**
Query Results:
0.87: A 7-year-old boy with sickle cell disease is experiencing knee and hip pain, has been admitted for pain crises in the past, and now walks with a limp. His physical exam shows a normal and cool hip to the touch, with decreased range of motion at the hip and pain with ambulation. Given these findings, what is the most appropriate next step in the management of this patient's hip pain? - In managing the hip pain of a 7-year-old boy with sickle cell disease, who presents with knee and hip pain, a limp, and decreased range of motion in the hip, the most appropriate next step is to obtain an X-ray of the hip. This will help evaluate the possibility of avascular necrosis (AVN) or other structural abnormalities. X-rays are typically the first-line imaging technique in such cases due to their accessibility and ability to reveal gross pathological changes. If the X-ray does not provide conclusive information and clinical suspicion of AVN remains high, an MRI may subsequently be considered for a more detailed assessment.
0.55: What is the most likely diagnosis for a 2-year-old 70 kg child who presents with limitation of abduction and internal rotation, tenderness in Scarpa's triangle, and abduction of the limb upon flexing the hip? - Based on the described symptoms and the unusual weight for a 2-year-old child, the most likely diagnosis is Slipped Capital Femoral Epiphysis (SCFE). Even though SCFE typically occurs in older children, mainly adolescents, the combination of excessive weight, limited hip abduction and internal rotation, tenderness in the hip area, and the characteristic limb movement (abduction upon hip flexion) strongly points towards SCFE as the most plausible diagnosis in this scenario.
0.48: A 5-year-old boy has recurrent abdominal pain primarily occurring during school hours, with no significant findings on physical examination and normal stool characteristics. His symptoms resolve at home, and his laboratory tests and abdominal exam are unremarkable. Considering the psychological factors involved, what is the most appropriate next step in managing his condition? - Given the symptoms and the context you've provided, it seems quite possible that the boy's recurrent abdominal pain is linked to psychological stressors related to school. Since all medical tests and examinations have returned normal results, this suggests that the pain might be stress-induced, possibly due to anxiety or stress at school.
The most appropriate next step is to focus on addressing any potential psychological or emotional factors. Consulting a psychologist or school counselor would be beneficial. They can work with the boy to explore any underlying emotional issues or anxieties about school. Through conversation, play, or other therapeutic techniques suitable for his age, they can help identify and manage any stressors he might be facing. This approach could not only help alleviate his abdominal pain but also improve his overall well-being by addressing the source of his anxiety.
0.44: In a patient who, five days post-open colectomy for colon cancer, develops severe pain and swelling of the left calf along with necrotic lesions, a fever, and thrombocytopenia while on unfractionated heparin, what is the most appropriate next step in management? - In this clinical scenario, the presentation of severe pain and swelling in the calf, necrotic skin lesions, fever, and thrombocytopenia in a patient receiving unfractionated heparin strongly suggests heparin-induced thrombocytopenia (HIT). HIT is a prothrombotic disorder caused by antibodies against heparin-platelet factor 4 complexes, leading to platelet activation, thrombocytopenia, and an increased risk of thrombosis.
The most appropriate next step in management is to immediately discontinue the unfractionated heparin to prevent further complications related to thrombosis. Simultaneously, it's crucial to initiate an alternative anticoagulant that does not cross-react with HIT antibodies to manage the thrombotic risk. Argatroban or fondaparinux are commonly used anticoagulants in this context as they are safe and effective for patients with HIT. Direct-acting oral anticoagulants (DOACs) are also potential options, but argatroban is often preferred initially due to its intravenous route and ability to be titrated easily in acute care settings. This dual approach addresses both the cause and the risk effectively.
0.44: In a patient with sickle cell anaemia presenting with multiple non-suppurative osteomyelitic dactylitis, what is the most likely causative organism? - In a patient with sickle cell anemia presenting with multiple non-suppurative osteomyelitic dactylitis, the most likely causative organism is Salmonella species. In individuals with sickle cell disease, Salmonella is particularly notorious for causing osteomyelitis. The relationship between sickle cell anemia and Salmonella infections, especially in the bone, is well-documented, and their presentations can often be less typical and less suppurative than those caused by other common bacteria like Staphylococcus aureus.
✅ **PineconeSearchDocuments tool invoked successfully.**
💡 **Final Answer:**
The most appropriate next step in the management of this 7-year-old boy with sickle cell disease and hip pain is to obtain an X-ray of the hip. This will help evaluate for potential avascular necrosis or other structural issues. If the X-ray is inconclusive and there is still a high suspicion of avascular necrosis, further imaging with an MRI may be considered.
As shown above, depending on the query, appropriate tool is invoked in order to determine the optimal response.
For instance, looking at the third example, when the model triggers the tool named "PineconeSearchDocuments", the code calls query_pinecone_index with the current query and then extracts the best match (or an appropriate context) as the result. For non health related inqueries or queries where explicit internet search is asked, the code calls the web_search_call function and for other queries, it may choose to not call any tool and rather provide a response based on the question under consideration.
Finally, the tool call and its output are appended to the conversation, and the final answer is generated by the Responses API.
Multi-tool orchestration flow
Now let us try to modify the input query and the system instructions to the responses API in order to follow a tool calling sequence and generate the output.
[22]
🌟--- Processing Query ---🌟
🔍 **User Query:** What is the most common cause of death in the United States
🔧 **Calling Responses API with Tools Enabled**
🕵️♂️ **Step 1: Web Search Call**
- Initiating web search to gather initial information.
📚 **Step 2: Pinecone Search Call**
- Querying Pinecone to find relevant examples from the internal knowledge base.
input_messages [{'role': 'user', 'content': 'What is the most common cause of death in the United States'}]
✨ **Initial Response Output:**
[ResponseFunctionWebSearch(id='ws_67e6e83241ac81918f93ffc96491ec390fdddafaeefcefc1', status='completed', type='web_search_call'), ResponseOutputMessage(id='msg_67e6e833a2cc8191a9df22f324a876b00fdddafaeefcefc1', content=[ResponseOutputText(annotations=[AnnotationURLCitation(end_index=698, start_index=613, title='Products - Data Briefs - Number 521 - December 2024', type='url_citation', url='https://www.cdc.gov/nchs/products/databriefs/db521.htm?utm_source=openai'), AnnotationURLCitation(end_index=984, start_index=891, title='US deaths are down and life expectancy is up, but improvements are slowing', type='url_citation', url='https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai'), AnnotationURLCitation(end_index=1186, start_index=1031, title='US deaths are down and life expectancy is up, but improvements are slowing', type='url_citation', url='https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai')], text="As of 2023, the leading causes of death in the United States are:\n\n1. **Heart Disease**: 680,981 deaths\n2. **Cancer**: 613,352 deaths\n3. **Unintentional Injuries**: 222,698 deaths\n4. **Stroke**: 162,639 deaths\n5. **Chronic Lower Respiratory Diseases**: 145,357 deaths\n6. **Alzheimer's Disease**: 114,034 deaths\n7. **Diabetes**: 95,190 deaths\n8. **Kidney Disease**: 55,253 deaths\n9. **Chronic Liver Disease and Cirrhosis**: 52,222 deaths\n10. **COVID-19**: 49,932 deaths\n\nNotably, COVID-19 has dropped from the fourth leading cause in 2022 to the tenth in 2023, reflecting a significant decrease in related deaths. ([cdc.gov](https://www.cdc.gov/nchs/products/databriefs/db521.htm?utm_source=openai))\n\nOverall, the U.S. experienced a decline in total deaths and a modest increase in life expectancy in 2023, attributed to reductions in deaths from COVID-19, heart disease, and drug overdoses. ([apnews.com](https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai))\n\n\n## Recent Trends in U.S. Mortality Rates:\n- [US deaths are down and life expectancy is up, but improvements are slowing](https://apnews.com/article/be061f9f14c883178eea6dddc9550e60?utm_source=openai) ", type='output_text')], role='assistant', status='completed', type='message'), ResponseFunctionToolCall(arguments='{"query":"most common cause of death in the United States","top_k":3}', call_id='call_6YWhEw3QSI7wGZBlNs5Pz4zI', name='PineconeSearchDocuments', type='function_call', id='fc_67e6e8364e4c819198501fba5d3f155b0fdddafaeefcefc1', status='completed')]
[23]
[24]
ResponseFunctionWebSearch(id='ws_67e6e83241ac81918f93ffc96491ec390fdddafaeefcefc1', status='completed', type='web_search_call')
ws_67e6e83241ac81918f93ffc96491ec390fdddafaeefcefc1
ResponseFunctionToolCall(arguments='{"query":"most common cause of death in the United States","top_k":3}', call_id='call_6YWhEw3QSI7wGZBlNs5Pz4zI', name='PineconeSearchDocuments', type='function_call', id='fc_67e6e8364e4c819198501fba5d3f155b0fdddafaeefcefc1', status='completed')
call_6YWhEw3QSI7wGZBlNs5Pz4zI
[166]
[{'role': 'user', 'content': 'What is the most common cause of death in the United States'}, ResponseFunctionToolCall(arguments='{"query":"most common cause of death in the United States"}', call_id='call_8Vzsn4RwMOgXyX98UpZY8hls', name='PineconeSearchDocuments', type='function_call', id='fc_67e348f36f7c81919d0aeef1855df3f20d0bd7f2a5744b88', status='completed')]
[{'role': 'user', 'content': 'What is the most common cause of death in the United States'}, ResponseFunctionToolCall(arguments='{"query":"most common cause of death in the United States"}', call_id='call_8Vzsn4RwMOgXyX98UpZY8hls', name='PineconeSearchDocuments', type='function_call', id='fc_67e348f36f7c81919d0aeef1855df3f20d0bd7f2a5744b88', status='completed'), {'type': 'function_call_output', 'call_id': 'call_8Vzsn4RwMOgXyX98UpZY8hls', 'output': "**Question:** A 7-year-old boy with sickle cell disease is experiencing knee and hip pain, has been admitted for pain crises in the past, and now walks with a limp. His physical exam shows a normal and cool hip to the touch, with decreased range of motion at the hip and pain with ambulation. Given these findings, what is the most appropriate next step in the management of this patient's hip pain?\n**Answer:** In managing the hip pain of a 7-year-old boy with sickle cell disease, who presents with knee and hip pain, a limp, and decreased range of motion in the hip, the most appropriate next step is to obtain an X-ray of the hip. This will help evaluate the possibility of avascular necrosis (AVN) or other structural abnormalities. X-rays are typically the first-line imaging technique in such cases due to their accessibility and ability to reveal gross pathological changes. If the X-ray does not provide conclusive information and clinical suspicion of AVN remains high, an MRI may subsequently be considered for a more detailed assessment."}]
[25]
🔧 **Calling Responses API for Final Answer**
Response(id='resp_67e6e886ac7081918b07224fb1ed38ab05c4a598f9697c7c', created_at=1743186054.0, error=None, incomplete_details=None, instructions=None, metadata={}, model='gpt-4o-2024-08-06', object='response', output=[ResponseOutputMessage(id='msg_67e6e8872ddc81918e92c9e4508abbe005c4a598f9697c7c', content=[ResponseOutputText(annotations=[], text='The most common cause of death in the United States is heart disease.', type='output_text')], role='assistant', status='completed', type='message')], parallel_tool_calls=True, temperature=1.0, tool_choice='auto', tools=[], top_p=1.0, max_output_tokens=None, previous_response_id=None, reasoning=Reasoning(effort=None, generate_summary=None), status='completed', text=ResponseTextConfig(format=ResponseFormatText(type='text')), truncation='disabled', usage=ResponseUsage(input_tokens=37, input_tokens_details=InputTokensDetails(cached_tokens=0), output_tokens=15, output_tokens_details=OutputTokensDetails(reasoning_tokens=0), total_tokens=52), user=None, store=False)
[26]
The most common cause of death in the United States is heart disease.
Here, we have seen how to utilize OpenAI's Responses API to implement a Retrieval-Augmented Generation (RAG) approach with multi-tool calling capabilities. It showcases an example where the model selects the appropriate tool based on the input query: general questions may be handled by built-in tools such as web-search, while specific medical inquiries related to internal knowledge are addressed by retrieving context from a vector database (such as Pinecone) via function calls. Additonally, we have showcased how multiple tool calls can be sequentially combined to generate a final response based on our instructions provided to responses API.
As you continue to experiment and build upon these concepts, consider exploring additional resources and examples to further enhance your understanding and applications
Happy coding!