Rag Paper Titles
Getting Started with RAG using Fireworks Fast Inference LLMs
While large language models (LLMs) show powerful capabilities that power advanced use cases, they suffer from issues such as factual inconsistency and hallucination. Retrieval-augmented generation (RAG) is a powerful approach to enrich LLM capabilities and improve their reliability. RAG involves combining LLMs with external knowledge by enriching the prompt context with relevant information that helps accomplish a task.
This tutorial shows how to getting started with RAG by leveraging vector store and open-source LLMs. To showcase the power of RAG, this use case will cover building a RAG system that suggests short and easy to read ML paper titles from original ML paper titles. Paper tiles can be too technical for a general audience so using RAG to generate short titles based on previously created short titles can make research paper titles more accessible and used for science communication such as in the form of newsletters or blogs.
Before getting started, let's first install the libraries we will use:
Let's download the dataset we will use:
--2023-11-10 15:04:22-- https://raw.githubusercontent.com/dair-ai/ML-Papers-of-the-Week/main/research/ml-potw-10232023.csv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.110.133, 185.199.108.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 664158 (649K) [text/plain] Saving to: ‘ml-potw-10232023.csv’ ml-potw-10232023.cs 100%[===================>] 648.59K --.-KB/s in 0.06s 2023-11-10 15:04:22 (10.7 MB/s) - ‘ml-potw-10232023.csv’ saved [664158/664158] mkdir: cannot create directory ‘data’: File exists
Before continuing, you need to obtain a Fireworks API Key to use the Mistral 7B model.
Checkout this quick guide to obtain your Fireworks API Key: https://readme.fireworks.ai/docs
Getting Started
Let's define a function to get completions from the Fireworks inference platform.
Let's first try the function with a simple prompt:
" Robert and I'm Italian living in Mexico. I have a passion for photography and movies. I hope you will enjoy them too. You can also notice that I'm a huge fan of James Bond 007, Michael Bay"
Now let's test with Mistral-7B-Instruct:
' Jane. Today, I would like to share with you some interesting facts about London.\n\nLondon is the capital of England and the United Kingdom, and it is one of the oldest cities in the world. The city has a rich history, dating'
The Mistral 7B Instruct model needs to be instructed using special instruction tokens [INST] <instruction> [/INST] to get the right behavior. You can find more instructions on how to prompt Mistral 7B Instruct here: https://docs.mistral.ai/llm/mistral-instruct-v0.1
'!\n1: Why don’t scientists trust atoms? Because they make up everything! \n\n2: Did you hear they are republishing that Simple Mathematics study guide? It’s the revised addition.'
" Sure, here are two jokes for you:\n\n1. Why don't scientists trust atoms? Because they make up everything!\n2. What do you call a fake noodle? An impasta!"
Now let's try with a more complex prompt that involves instructions:
" Dear John Doe,\n\nWe, Tom and Mary, are writing to thank you for attending our wedding. We were thrilled to have you there and share such a special occasion with us. We couldn't help but be impressed by your amazing accomplishment of climbing Mount Everest in 2020, and we are grateful for the insight and experiences you brought to our wedding. Thank you again for being a wonderful guest and for being a part of our special day.\n\nSincerely,\nTom and Mary"
RAG Use Case: Generating Short Paper Titles
For the RAG use case, we will be using a dataset that contains a list of weekly top trending ML papers.
The user will provide an original paper title. We will then take that input and then use the dataset to generate a context of short and catchy papers titles that will help generate catchy title for the original input title.
Step 1: Load the Dataset
Let's first load the dataset we will use:
{'Title': 'Llemma',
, 'Description': 'an LLM for mathematics which is based on continued pretraining from Code Llama on the Proof-Pile-2 dataset; the dataset involves scientific paper, web data containing mathematics, and mathematical code; Llemma outperforms open base models and the unreleased Minerva on the MATH benchmark; the model is released, including dataset and code to replicate experiments.',
, 'PaperURL': 'https://arxiv.org/abs/2310.10631',
, 'TweetURL': 'https://x.com/zhangir_azerbay/status/1714098025956864031?s=20',
, 'Abstract': 'We present Llemma, a large language model for mathematics. We continue pretraining Code Llama on the Proof-Pile-2, a mixture of scientific papers, web data containing mathematics, and mathematical code, yielding Llemma. On the MATH benchmark Llemma outperforms all known open base models, as well as the unreleased Minerva model suite on an equi-parameter basis. Moreover, Llemma is capable of tool use and formal theorem proving without any further finetuning. We openly release all artifacts, including 7 billion and 34 billion parameter models, the Proof-Pile-2, and code to replicate our experiments.'} We will be using SentenceTransformer for generating embeddings that we will store to a chroma document store.
We will now generate embeddings for batches:
0%| | 0/9 [00:00<?, ?it/s]
Now we can test the retriever:
[['LLMs for Software Engineering', 'Communicative Agents for Software Development']]
Now let's put together our final prompt:
Model Suggestions: 1. S3Eval: A Comprehensive Evaluation Suite for Large Language Models 2. Synthetic and Scalable Evaluation for Large Language Models 3. Systematic Evaluation of Large Language Models with S3Eval 4. S3Eval: A Synthetic and Scalable Approach to Evaluating Large Language Models 5. S3Eval: A Synthetic and Scalable Evaluation Suite for Large Language Models Prompt Template: [INST] Your main task is to generate 5 SUGGESTED_TITLES based for the PAPER_TITLE You should mimic a similar style and length as SHORT_TITLES but PLEASE DO NOT include titles from SHORT_TITLES in the SUGGESTED_TITLES, only generate versions of the PAPER_TILE. PAPER_TITLE: S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models SHORT_TITLES: Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling ChemCrow: Augmenting large-language models with chemistry tools A Survey of Large Language Models LLaMA: Open and Efficient Foundation Language Models SparseGPT: Massive Language Models Can Be Accurately Pruned In One-Shot REPLUG: Retrieval-Augmented Black-Box Language Models LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Auditing large language models: a three-layered approach Fine-Tuning Language Models with Just Forward Passes DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents SUGGESTED_TITLES: [/INST]
As you can see, the short titles generated by the LLM are somewhat okay. This use case still needs a lot more work and could potentially benefit from finetuning as well. For the purpose of this tutorial, we have provided a simple application of RAG using open-source models from Firework's blazing-fast models.
Try out other open-source models here: https://app.fireworks.ai/models
Read more about the Fireworks APIs here: https://readme.fireworks.ai/reference/createchatcompletion