Generate Doctor Who Synopses With GPT 3 And Weights & Biases(Video)
Fine-tune GPT-3 with Weights & Biases to Generate Doctor Who Episode Synopses
OpenAI’s API gives practitioners access to GPT-3, an incredibly powerful natural language model that can be applied to virtually any task that involves understanding or generating natural language.
If you use OpenAI's API to fine-tune GPT-3, you can now use the W&B integration to track experiments, models, and datasets in your central dashboard.
All it takes is one line: openai wandb sync
Set up your API key
WARNING: Remove the API key after running the cell and clear output so it does not get logged to wandb in case you sync code (see settings)
Install dependencies
Using Weights & Biases Artifacts to Download a .CSV dataset file with episode title > synopsis pairs
Using OpenAI Tool to preprocess the data
Splitting the data into train and val sets
Let's define our GPT-3 fine-tuning hyper-parameters.
Time to train the model!
Sync fine-tune jobs to Weights & Biases
We can log our fine-tunes with a simple command.
Our fine-tunes are now successfully synced to Weights & Biases
Anytime we have new fine-tunes, we can just call openai wandb sync to add them to our dashboard.
Log inference samples
The best way to evaluate a generative model is to explore sample predictions.
Let's generate a few inference samples and log them to W&B.
We can easily retrieve all config parameters from a job file.
Job files are logged to W&B as artifacts and can be accessed with run.use_artifact('USERNAME/PROJECT/job_details:VERSION') where VERSION is either:
- a version number such as
v2 - the fine-tune id such as
ft-xxxxxxxxx - an alias added automatically such as
latestor manually
You can explore them in your artifacts dashboard.
Let's take advantage to add metadata into our eval run config.
We can easily access model id from any job.
Loading validation data as dataframe
We'll perform the inference on all 30 validation examples.
We create and log a W&B Table to easily explore, query & compare model predictions.
You can open the link to your run page down below.