HTTP streaming is not new at all, but somehow it is all the rage these days, and for a good reason. At Vercel, we recently introduced streaming over serverless/lambda functions, which wasn't done anywhere else at that point, as far as I'm aware of. Nowadays, AWS has the same capability and I guess more and more providers will have them.
You might wonder: why this change is so important and why people are that excited about it?
When Next.js introduced the App Router (using React Server Components), they said it supports streaming. The streaming is actually out-of-order, which let's you stream updates to your UI after rendering it. That means your comment box can start with a loading spinner, but as the page renders (in that original request), you will get the data and replace the contents with the rendered format, no extra JavaScript necessary in the client.
How does that work? and what the hell is out-of-order streaming?
When streaming content over HTTP, the data is sent in order. That means that if you have the following HTTP server:
require("http").createServer(async (req, res) => {
res.write("hello,");
await new Promise((res) => setTimeout(res, 2000));
res.write(" world.");
res.end();
});
what you will see is "hello," -- followed by 2 seconds delay -- " world", sent to the client. This enables things like streaming big files: you don't need to load the entire file to memory. But that does not give us the ability to manipulate already-sent data.
So how does React and Next.js solve it? This is where out-of-order streaming comes to play.
Let's revisit our web server, and now we will send some HTML that renders a page with a list of todos:
async function getTodosHtml() {
await new Promise((res) => setTimeout(res, 2000));
return html`
<ol>
<li>Write a blog post</li>
<li>Drink water</li>
</ol>
`;
}
require("http").createServer(async (req, res) => {
const todosHtml = await getTodosHtml();
res.write(html`
<html>
<body>
<h1>Todos:</h1>
${todosHtml}
</body>
</html>
`);
res.end();
});
This page works. However, we get a pretty slow time-to-first-byte here. It takes more than 2 seconds to load our page, even though we can show the "shell" of the app pretty quickly.
Prior to server components, we would've set it as loading and then kick off a separate HTTP request when the client is hydrated. So how can we fix it using out of order streaming? Check out the following code, then we discuss:
require("http").createServer(async (req, res) => {
const todosHtmlPromise = getTodosHtml();
res.write(html`
<html>
<body>
<h1>Todos:</h1>
<div id="loading-todos">loading...</div>
</body>
</html>
`);
const todosHtml = await todosHtmlPromise;
res.write(html`
<template id="for-loading-todos">${todosHtml}</template>
<script>
(() => {
const template = document.querySelector("#for-loading-todos");
document.querySelector("#loading-todos").outerHTML = template.innerHTML;
template.remove();
})();
</script>
`);
res.end();
});
That's pretty intense, I understand. Let's go it bit by bit.
We start off with calling the promise, but we don't await it: this allows us to kick it off and await later. Then we send the "shell": an app that has the entire website, other than the slow part (the todos). For the slow part, we render a <div id="loading-todos">. This div will be replaced later on. Keep in mind that frameworks like React have better ways to replace nodes and data, this is just for demonstration purposes and simplicity.
Then we await the promise, and send yet another chunk. This chunk contains the template that we want to apply to our app (which is the todos html like before), and a script tag.
The script tag takes the template, and replaces #loading-todos with its content, then removes the template altogether from our page. This leads us to the final result. Yes, it's exactly like what we had before, with much more code and complexity involved, but the perceived performance is much better. The website now gradually enhances over time, instead of a blank page for two seconds. When you use modern frameworks like Next.js, you get this optimization for free:
Check the sandbox below for a runnable example:
So yeah. Like Guillermo mentioned on 𝕏: streaming is inevitable for performance reasons. But the real hero here is out-of-order streaming that helps us build these fine interactivity using streaming.