Falcon 7b Instruction Domain Adaptation Finetuning
Introduction to SageMaker JumpStart - Text Generation with Falcon models
In this demo notebook, we demonstrate how to use the SageMaker Python SDK to fine-tuning and deploy Falcon 7B models for text generation. For fine-tuning, we include two types of fine-tuning: instruction fine-tuning and domain adaption fine-tuning.
The Falcon model is a permissively licensed (Apache-2.0) open source model trained on the RefinedWeb dataset.
Below is the content of the notebook.
Install latest SageMaker and dependencies.
1. Instruction fine-tuning
Now, we demonstrate how to instruction-tune huggingface-llm-falcon-7b-instruct-bf16 model for a new task. As mentioned in Section 1.3 About the models, Falcon-7b-instruct/Falcon-40B-instruct models are instruction base falcon models fine-tuned on a mixture of chat and instruction datasets.
In this task, given a piece of context, the model is asked to generate questions that are relevant to the text, but cannot be answered based on provided information. Examples are given in the inference section of this notebook.
1.1. Preparing training data
We will use a subset of SQuAD2.0 for supervised fine-tuning. This dataset contains questions posed by human annotators on a set of Wikipedia articles. In addition to questions with answers, SQuAD2.0 contains about 50k unanswerable questions. Such questions are plausible, but cannot be directly answered from the articles' content. We only use unanswerable questions for our task.
Citation: @article{rajpurkar2018know, title={Know what you don't know: Unanswerable questions for SQuAD}, author={Rajpurkar, Pranav and Jia, Robin and Liang, Percy}, journal={arXiv preprint arXiv:1806.03822}, year={2018} }
License: Creative Commons Attribution-ShareAlike License (CC BY-SA 4.0)
The training data must be formatted in JSON lines (.jsonl) format, where each line is a dictionary representing a single data sample. All training data must be in a single folder, however it can be saved in multiple jsonl files. The .jsonl file extension is mandatory. The training folder can also contain a template.json file describing the input and output formats.
If no template file is given, the following default template will be used:
{
"prompt": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{context}`,
"completion": "{response}",
}
In this case, the data in the JSON lines entries must include instruction, context, and response fields.
Different from using the default prompt template, in this demo we are going to use a custom template (see below).
Next, we are going to reformat the SQuAD 2.0 dataset. The processed data is saved as task-data.jsonl file. Given the prompt template defined in above cell, each entry in the task-data.jsonl file include context and question fields. For demonstration purpose, we limit the number of training examples to be 2000.
Upload the prompt template (template.json) and training data (task-data.jsonl) into S3 bucket.
1.2. Prepare training parameters
Overwrite the hyperparameters
Validate hyperparameters
1.3. Starting training
Note. The parameter load_best_model_at_end (Whether or not to load the best model found during training at the end of training. When this option is enabled, the best checkpoint will always be saved) is set as "True" by default. During loading the best model checkpoints at the end of training (HuggingFace will load the best model checkpoints before saving it), there is overhead of memory usage which can lead to Out-Of-Memory error.
If setting load_best_model_at_end, we recommend to use ml.g5.48xlarge; if not, we recommend to use ml.g5.12xlarge.
Extract Training performance metrics. Performance metrics such as training loss and validation accuracy/loss can be accessed through cloudwatch while the training. We can also fetch these metrics and analyze them within the notebook.
1.4. Deploying inference endpoints
We also deploy the pre-trained model for comparision in Section 2.5.
1.5. Running inference queries and compare model performances
We examine three examples as listed in variable test_paragraphs. The prompt as defined in variable prompt asks the model to ask a question based on the context and make sure the question cannot be answered from the context.
We compare the performance of pre-trained Falcon instruct 7b (huggingface-llm-falcon-7b-instruct-bf16) that we deployed in Section 1 and fine-tuned Falcon instruct 7b.
The pre-trained model was not specifically trained to generate unanswerable questions. Despite the input prompt, it tends to generate questions that can be answered from the text. The fine-tuned model is generally better at this task, and the improvement is more prominent for larger models
1.6. Clean up the endpoint
2. Domain adaptation fine-tuning
We also have domain adaptation fine-tuning enabled for Falcon models. Different from instruction fine-tuning, you do not need prepare instruction-formatted dataset and can directly use unstructured text document which is demonstrated as below. However, the model that is domain-adaptation fine-tuned may not give concise responses as the instruction-tuned model because of less restrictive requirements on training data formats.
In this demonstration, we use falcon text generation model huggingface-llm-falcon-7b-bf16. This is not an instruction-tuned version of Falcon 7B model. You can also conduct domain adaptation finetuning on top of instruction-tuned model like huggingface-llm-falcon-7b-instruct-bf16. However, we generally do not recommend that.
We will use financial text from SEC filings to fine tune huggingface-llm-falcon-7b-bf16 for financial applications.
Here are the requirements for train and validation data.
- Input: A train and an optional validation directory. Each directory contains a CSV/JSON/TXT file.
- For CSV/JSON files, the train or validation data is used from the column called 'text' or the first column if no column called 'text' is found.
- The number of files under train and validation (if provided) should equal to one.
- Output: A trained model that can be deployed for inference.
Below is an example of a TXT file for fine-tuning the Text Generation model. The TXT file is SEC filings of Amazon from year 2021 to 2022.
This report includes estimates, projections, statements relating to our
business plans, objectives, and expected operating results that are “forward-
looking statements” within the meaning of the Private Securities Litigation
Reform Act of 1995, Section 27A of the Securities Act of 1933, and Section 21E
of the Securities Exchange Act of 1934. Forward-looking statements may appear
throughout this report, including the following sections: “Business” (Part I,
Item 1 of this Form 10-K), “Risk Factors” (Part I, Item 1A of this Form 10-K),
and “Management’s Discussion and Analysis of Financial Condition and Results
of Operations” (Part II, Item 7 of this Form 10-K). These forward-looking
statements generally are identified by the words “believe,” “project,”
“expect,” “anticipate,” “estimate,” “intend,” “strategy,” “future,”
“opportunity,” “plan,” “may,” “should,” “will,” “would,” “will be,” “will
continue,” “will likely result,” and similar expressions. Forward-looking
statements are based on current expectations and assumptions that are subject
to risks and uncertainties that may cause actual results to differ materially.
We describe risks and uncertainties that could cause actual results and events
to differ materially in “Risk Factors,” “Management’s Discussion and Analysis
of Financial Condition and Results of Operations,” and “Quantitative and
Qualitative Disclosures about Market Risk” (Part II, Item 7A of this Form
10-K). Readers are cautioned not to place undue reliance on forward-looking
statements, which speak only as of the date they are made. We undertake no
obligation to update or revise publicly any forward-looking statements,
whether because of new information, future events, or otherwise.
...
SEC filings data of Amazon is downloaded from publicly available EDGAR. Instruction of accessing the data is shown here.
2.1. Preparing training data
The training data of SEC filing of Amazon has been pre-saved in the S3 bucket.
2.2. Prepare training parameters
Validate hyperparameters
2.3. Starting training
Extract Training performance metrics. Performance metrics such as training loss and validation accuracy/loss can be accessed through cloudwatch while the training. We can also fetch these metrics and analyze them within the notebook
2.4. Deploying inference endpoints
We deploy the domain-adaptation fine-tuned and pretrained models separately, and compare their performances.
We firstly deploy the domain-adaptation fine-tuned model.
Next, we deploy the pre-trained huggingface-llm-falcon-7b-bf16.
2.5. Running inference queries and compare model performances
As you can, the fine-tuned model starts to generate responses that are more specific to the domain of fine-tuning data which is relating to SEC report of Amazon.