Workflow Solution

Okay, we are writing a Slack message to our boss complaining about a colleague's hygiene. We get a little bit of waiting time and then it streams in. Let's run that through just one more time. You can see the waiting time, time to first token, pretty poor waiting, and then it streams in.

Let's take a look at the code here.

We are calling generateText with a model with a system prompt and passing in the conversation history by formatting the message history of the messages.

Our next step is we evaluate the Slack message with another generateText call, this time passing in the conversation history and passing in the Slack message from the result before.

Finally, we call it one more time and we stream it down. We streamText and we return toUIMessageStreamResponse(), which of course turns it into a stream and then turns that stream into a response. Here we're passing in the first draft and the previous feedback and also the entire conversation history.

This process then acts like a workflow. We end up with a deterministic pipeline of three different LLM calls. We write, we evaluate, and then we rewrite. And while this is relatively simplistic, your mind should be buzzing on the different things you could do with these workflows. What we're going to focus on is taking this very basic workflow and making it better, especially in improving our time to first token because those generateText calls are giving me a headache.

Nice work and I will see you in the next one.