Note: To answer questions based on text documents, we recommend the procedure in Question Answering using Embeddings. Some of the code below may rely on deprecated API endpoints.

3. Train a fine-tuning model specialized for Q&A

This notebook will utilize the dataset of context, question and answer pairs to additionally create adversarial questions and context pairs, where the question was not generated on that context. In those cases the model will be prompted to answer "No sufficient context for answering the question". We will also train a discriminator model, which predicts whether the question can be answered based on the context or not.

We will add hard adversarial examples as well, which will be based either on semantically similar sections, or neighbouring sections, originating from the same article.

Split the sections into a training and testing set

we check that the separator we intend to use isn't present within the contexts

3.1 Create the fine-tuning datasets for Q&A and discriminator models

The fine-tuning dataset is created in the following way. For every corresponding question, answer and context pair we create:

• Positive example: correct question, answer, context pair
• Negative examples:
  • random negative example, where the random context is paired with the question
  • two hard negative examples
    • one originating from the same wikipedia article
    • another, which is most similar to the correct context

This process is noisy, as sometimes the question might be answerable given a different context, but on average we hope this won't affect the performance too much.

We apply the same process of dataset creation for both the discriminator, and the Q&A answering model. We apply the process separately for the training and testing set, to ensure that the examples from the training set don't feature within the test set.

We apply the same process of dataset creation for both the discriminator, and the Q&A answering model. We apply the process separately for the training and testing set, to ensure that the examples from the training set don't feature within the test set.

We formatted the data according to the recommendations from the fine-tuning tool, which is available using

openai tools fine_tunes.prepare_data -f qa_train.jsonl

We highly recommend that you use this tool, which suggests improvements in your data formatting for fine-tuning.

3.2 Submit the datasets for fine-tuning

3.3 Using the fine-tuned models

We will now use the fine-tuned discriminator and the fine-tuned Q&A model. By requesting logprobs, we can see how certain the discriminator is in a yes vs no answer.

We can see that the model can generalize well to different contexts and questions.

We can see that the model can answer the question, when the context is appropriate.

We can see that the model knows when to answer the question, and when to say that insufficient context is present to answer the question.

We can also combine a discriminator and a base model, or a fine-tuned Q&A model. Discriminator can essentially serve as a decision whether the question can be answered given the context or not.

The above function illustrates how to potentially combine a discriminator and a fine-tuned Q&A model. This gives a more fine-grained control over how certain we want the model to be before it answers the question.

We'll now take a look on how answers endpoint works - combining search to retrieve the relevant context from a knowledge base, and then using the fine-tuned Q&A model to answer the question.

3.4 Answering the question based on a knowledge base

Finally we can use a logic similar to the /answers endpoint, where we first search for the relevant context, and then ask a Q&A model to answer the question given that context. If you'd like to see the implementation details, check out the answers_with_ft.py file.