Using Tool Required For Customer Service
chatgptopenaigpt-4examplesopenai-apiopenai-cookbook
Export
Using Tool Required for Customer Service
The ChatCompletion endpoint now includes the ability to specify whether a tool must be called every time, by adding tool_choice='required' as a parameter.
This adds an element of determinism to how you build your wrapping application, as you can count on a tool being provided with every call. We'll demonstrate here how this can be useful for a contained flow like customer service, where having the ability to define specific exit points gives more control.
The notebook concludes with a multi-turn evaluation, where we spin up a customer GPT to imitate our customer and test the LLM customer service agent we've set up.
[1]
Config definition
We will define tools and instructions which our LLM customer service agent will use. It will source the right instructions for the problem the customer is facing, and use those to answer the customer's query.
As this is a demo example, we'll ask the model to make up values where it doesn't have external systems to source info.
[2]
[3]
Example
To test this we will run an example for a customer who has experienced fraud, and see how the model handles it.
Play the role of the user and provide plausible next steps to keep the conversation going.
[4]
User: Hi, I have had an item stolen that was supposed to be delivered to me yesterday. Calling function get_instructions Calling function speak_to_user Assistant: I'm sorry to hear about the stolen item. Could you please provide me with more details about the fraudulent activity, including the date and the items involved? This information will help us to investigate the issue further and proceed with the necessary actions, including offering you a refund.
[5]
User: For sure, it was a shirt, it was supposed to be delivered yesterday but it never arrived. Calling function speak_to_user Assistant: Thank you for providing the details. I will now proceed to report this incident to our security team for further investigation and arrange a refund for the stolen shirt. Please confirm if you would like me to go ahead with the refund. Calling function speak_to_user Assistant: Thank you for contacting us about this issue. Please don't hesitate to reach out if you have any more questions or need further assistance in the future.
[6]
User: Yes I would like to proceed with the refund. Calling function get_instructions Calling function speak_to_user Assistant: Thank you for confirming. I have processed the refund for the stolen shirt. The amount should be reflected in your account within 5-7 business days. If you have any more questions or need further assistance, please feel free to contact us.
[7]
User: Thanks very much. Calling function speak_to_user Assistant: You're welcome! If you need any more help in the future, don't hesitate to reach out. Have a great day!
Evaluation
Now we'll do a simple evaluation where a GPT will pretend to be our customer. The two will go back and forth until a resolution is reached.
We'll reuse the functions above, adding an execute_conversation function where the customer GPT will continue answering.
[8]
[9]
[10]
User: I want to get a refund for the suit I ordered last Friday. Calling function get_instructions Calling function speak_to_user Assistant: I understand you'd like a refund for the suit you ordered last Friday. Could you please provide more details about the issue with the suit? This will help us process your refund request accurately. User: The suit I received is not the color I ordered. I ordered a navy blue suit, but the one I received is black. Calling function speak_to_user Assistant: Thank you for providing the details. I will proceed with the refund for the navy blue suit that was incorrectly sent as black. Please confirm if you would like me to go ahead with the refund. User: Yes, please go ahead with the refund. Calling function speak_to_user Assistant: The refund for the incorrectly colored suit has been processed. You should see the amount credited back to your original payment method within 5-7 business days. Thank you for contacting us, and if you have any more questions or need further assistance, please feel free to reach out. Achieved objective, closing conversation User: Can you tell me what your policy is for returning damaged goods? Calling function get_instructions Calling function speak_to_user Assistant: It seems there was a misunderstanding in my previous request. I'm looking for information on our policy for returning damaged goods. Could you please provide me with the details on how to handle returns for damaged items? User: Yes, I'd appreciate that. Could you please tell me the steps I need to follow to return a damaged item? Calling function get_instructions Calling function speak_to_user Assistant: I apologize for the confusion earlier. Here's the correct information regarding our policy for returning damaged goods: 1. Please provide a description of the damage and the item involved. 2. Include the date of purchase and your order number if available. 3. You can choose to return the item by mail or in person at one of our stores. Please let us know which method you prefer, and we will provide the necessary details for the return process. 4. Once we receive the item, we will inspect it and process a refund or exchange based on your preference and our return policy guidelines. Please let me know if you need further assistance with this process or if there's anything else I can help you with! User: I would like to return the item by mail. Could you please provide me with the details on how to do that? Calling function speak_to_user Assistant: Certainly! To return the damaged item by mail, please follow these steps: 1. Pack the item securely in its original packaging, if possible. 2. Include all original materials such as manuals, accessories, and receipts. 3. Address the package to our returns department at the following address: [Your Company's Return Address] 4. Use a trackable shipping service and keep the tracking number for your records. 5. Once we receive and inspect the item, we will process your refund or exchange as per your request. Please let me know if you need a prepaid shipping label or any other assistance with your return! User: Yes, please, I would need a prepaid shipping label. Calling function speak_to_user Assistant: I will arrange for a prepaid shipping label to be sent to your email address. Please ensure that the email address we have on file is correct. You should receive the label shortly, which you can print and attach to your return package. Once you've shipped the item, please provide us with the tracking number so we can monitor the return's progress. If there's anything else you need help with, feel free to ask! User: That sounds great, thank you! My email address is correct on file. I'll look out for the prepaid shipping label and send the package back as soon as I receive it. Calling function speak_to_user Assistant: You're welcome! I'm glad I could assist you. Once you receive the prepaid shipping label, please proceed with the return as discussed. If you have any more questions or need further assistance in the future, don't hesitate to reach out. Have a great day! Achieved objective, closing conversation User: Please tell me what your complaint policy is Calling function get_instructions Calling function speak_to_user Assistant: It seems there was a misunderstanding. I will provide you with information about our complaint policy. Please give me a moment to gather the correct details for you. User: Sure, I'll wait. Thank you. Calling function speak_to_user Assistant: Our complaint policy is designed to ensure that any issues you encounter are addressed promptly and fairly. Here’s a brief overview: 1. **Submission**: You can submit your complaint through our website, by email, or by calling our customer service hotline. 2. **Acknowledgment**: We will acknowledge receipt of your complaint within 24 hours. 3. **Investigation**: We will investigate your complaint thoroughly. This process typically takes 3-5 business days. 4. **Resolution**: After the investigation, we will contact you with the outcome and any steps we will take to resolve the issue. 5. **Follow-up**: If you are not satisfied with the resolution, you can request a review of the decision. Please let me know if you need more detailed information or if there's anything else I can assist you with! User: That covers everything I needed to know, thank you! Calling function speak_to_user Assistant: You're welcome! I'm glad I could help. If you have any more questions in the future or need further assistance, feel free to reach out. Have a great day! Achieved objective, closing conversation
Conclusion
You can now control your LLM's behaviour explicitly by making tool use mandatory, as well as spin up GPT testers to challenge your LLM and to act as automated test cases.
We hope this has given you an appreciation for a great use case for tool use, and look forward to seeing what you build!