It's sometimes useful when you're talking to an LLM, when you want it to do something that's relatively complex, to get it to plan its work ahead of time. This was popularized by the idea of chain of thought, where you get the LLM to "think step by step".
In practical terms, what this means is the LLM will:
The task we're going to try this out on is explaining TypeScript code.
(For those who don't know, a decent chunk of my career has been spent teaching TypeScript, so I'm sort of training an LLM to do my job. Whoops)
We've provided it some detailed task context here:
const result = streamText({model: google('gemini-2.0-flash-lite'),prompt: `<task-context>You are a helpful TypeScript expert that can explain complex TypeScript code for beginner TypeScript developers. You will be given a complex TypeScript code and you will need to explain it in a way that is easy to understand.</task-context>
We're passing in some COMPLEX_TS_CODE into the background data as code, and I'm also passing in an article about the immediately indexed map type pattern.
If you don't know what this is, don't worry. The idea here is for the LLM to produce some text that's going to explain it to you.
We've got a couple of rules down here, and we've got the ask, which is to explain the code using the article as a reference.
Now, if we run this, we notice it generates an answer for a little bit and then writes to an output.md file.
Generating answer....................
This produces a pretty decent article, really. I advise that you read through it and try to get a sense for the basics.
From experience, the way that you optimally teach a complex piece of TypeScript like this is you break it down into its individual components. You then explain every single piece of code with different code samples, showing how every little bit works on its own, and then you bring it all together.
This is why I want the LLM to burn a few tokens planning this ahead of time. Ideally, it would figure out the optimal path for the user to understand the code, including all of the syntax.
The way we're going to do this is we're going to add some instructions telling the model to think about its answer first before it responds. That's our first TODO.
// TODO: Add some instructions telling the model to think about its answer first before it responds. Consider the optimal path for the user to understand the code, including each individual piece of syntax.
Now, if we just add the instruction to think harder and consider the optimal path, it will return that thinking as a block at the very start of the article:
A better way to do that would be for it to wrap its thinking in thinking XML tags:
This way, we could make them visually distinct in the frontend.
So, we're going to specify an output format telling the model to return two sections, a thinking XML block and an answer.
Here's where we need to make the changes, at the TODOs in the code:
So that's the goal: get the model to break down the individual pieces of information that the user needs to know and return that in a thinking XML tag before the actual answer. And hopefully the eventual answer that gets produced will be of better quality than the one we initially got.
Good luck, and I will see you in the solution.
OK, let me think about the output. Thinking...Code explanation begins here.
<thinking>OK, let me think about the output. Thinking...</thinking>Code explanation begins here.
// TODO: Add an output format telling the model to return two sections - a <thinking> block and an answer. The answer should NOT be wrapped in an <answer> tag.
