Streaming Objects Solution

So if I run this, we should be able to see a paragraph being generated here and then a bunch of objects being logged. streamObject is doing its thing. Let's take a look at the code down here for streamObject.

const factsResult = streamObject({
  model,
  prompt: `Give me some facts about the imaginary planet. Here's the story: ${finalText}`,
  schema: z.object({
    facts: z
      .array(z.string())
      .describe(
        "The facts about the imaginary planet. Write as if you are a scientist."
      ),
  }),
})

We are passing it a model, we are passing it a prompt saying

Give me some facts about the imaginary planet. Here's the story:

And then we're passing it a schema here of z.object with facts, which is an array of strings.

One cool thing here is that we can use the describe property just to give it a bit more context.

We zoom back a bit here, we can see that it starts out as an empty object, then we get three facts as a starter. Notice how the third one here is just half generated and then we end up with

the surface of Xilus is covered in fungal forests.

With the next chunk here we get a few more facts, then we get a couple more facts, and then it's just done in three chunks.

You can hopefully start to imagine as I can just the power of this API. Being able to stream this stuff to the front end in a structured object is so, so nice. And later on, we're going to look at how to combine this with really nice APIs that stream this data down. The way this works under the hood is that the Zod object here is converted into JSON schema.

schema: z.object({
  facts: z
    .array(z.string())
    .describe(
      "The facts about the imaginary planet. Write as if you are a scientist."
    ),
}),

for await (const chunk of factsResult.partialObjectStream) {
  console.log(chunk)
}

That JSON schema is then passed to the LLM using its structured outputs API. Another cool thing about using Zod here is that as it's being streamed down, Zod is used to check the output. And if the LLM's output differs from this shape here, then it will throw an error.

/*
const object: {
  facts: string[]
}
*/
const object = await factsResult.object

It's actually inferred from the shape of the Zod schema.

So streamObject gives us this wonderful API we can use to construct type safe objects from LLMs.

Nice work and I will see you in the next one.