06 Precognition Thinking Step By Step
Chapter 6: Precognition (Thinking Step by Step)
Setup
Run the following setup cell to load your API key and establish the get_completion helper function.
Lesson
If someone woke you up and immediately started asking you several complicated questions that you had to respond to right away, how would you do? Probably not as good as if you were given time to think through your answer first.
Guess what? Claude is the same way.
Giving Claude time to think step by step sometimes makes Claude more accurate, particularly for complex tasks. However, thinking only counts when it's out loud. You cannot ask Claude to think but output only the answer - in this case, no thinking has actually occurred.
Examples
In the prompt below, it's clear to a human reader that the second sentence belies the first. But Claude takes the word "unrelated" too literally.
To improve Claude's response, let's allow Claude to think things out first before answering. We do that by literally spelling out the steps that Claude should take in order to process and think through its task. Along with a dash of role prompting, this empowers Claude to understand the review more deeply.
Claude is sometimes sensitive to ordering. This example is on the frontier of Claude's ability to understand nuanced text, and when we swap the order of the arguments from the previous example so that negative is first and positive is second, this changes Claude's overall assessment to positive.
In most situations (but not all, confusingly enough), Claude is more likely to choose the second of two options, possibly because in its training data from the web, second options were more likely to be correct.
Letting Claude think can shift Claude's answer from incorrect to correct. It's that simple in many cases where Claude makes mistakes!
Let's go through an example where Claude's answer is incorrect to see how asking Claude to think can fix that.
Let's fix this by asking Claude to think step by step, this time in <brainstorm> tags.
If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the Example Playground.
Exercise 6.1 - Classifying Emails
In this exercise, we'll be instructing Claude to sort emails into the following categories:
- (A) Pre-sale question
- (B) Broken or defective item
- (C) Billing question
- (D) Other (please explain)
For the first part of the exercise, change the PROMPT to make Claude output the correct classification and ONLY the classification. Your answer needs to include the letter (A - D) of the correct choice, with the parentheses, as well as the name of the category.
Refer to the comments beside each email in the EMAILS list to know which category that email should be classified under.
❓ If you want a hint, run the cell below!
Still stuck? Run the cell below for an example solution.
Exercise 6.2 - Email Classification Formatting
In this exercise, we're going to refine the output of the above prompt to yield an answer formatted exactly how we want it.
Use your favorite output formatting technique to make Claude wrap JUST the letter of the correct classification in <answer></answer> tags. For instance, the answer to the first email should contain the exact string <answer>B</answer>.
Refer to the comments beside each email in the EMAILS list if you forget which letter category is correct for each email.
❓ If you want a hint, run the cell below!
Congrats!
If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!
Example Playground
This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses.