Netlify swaps V8 isolates for microVMs in its Edge Functions

This blog has been hosted on Netlify since 2019, although only as static HTML: Hugo generates the pages and Netlify serves them, with no functions or anything running on the server. Even so, the latest article on their blog, 5x faster Edge Functions: V8 isolates to Firecracker MicroVMs, describes a change I think is important and that goes somewhat against the grain: they’ve stopped running their Edge Functions in V8 isolates, and now each function runs in its own Firecracker microVM.

The curious part is that for years isolates have been the answer for running code at the edge, precisely because virtual machines were considered too heavy. Netlify says that with well-tuned microVMs they’re five times faster and isolate better too.

What Edge Functions are and what has changed

Edge Functions are small JavaScript or TypeScript functions that run in front of the site, on the node closest to the user, on every request matching their route. They’re used to personalize pages, redirect based on a cookie, handle authentication… Netlify says it runs about a billion a day.

Until now, when a request needed an Edge Function, it left Netlify’s network for an external execution service, ran there and came back. The article doesn’t name it, but since their launch in 2022 that service was Deno Deploy. Now the request stays inside: it goes to a compute node in Netlify’s own network and runs in a microVM. They built it together with Unikraft, which handles the virtual machines’ lifecycle.

The numbers

Cold invocations, when a node in a region has never seen that function and has to download the images, happen in about 1.2% of cases and add around 9 ms on average.

All the figures are Netlify’s, of course. But they make sense: a good part of the improvement comes simply from not leaving their network on every request.

The path of a request

What I liked most about the article is that it explains step by step what happens to each request. In short:

flowchart LR
    A["Request"] --> B["Edge node · TLS and routes"]
    B -->|"no function"| C["Cache · origin"]
    B -->|"with function"| D["Spec · runtime, platform, function"]
    D --> E["Compute node · rendezvous hashing"]
    E --> F["MicroVM · snapshot or boot"]
    F --> G["Response"]

Some design details I find particularly good:

• A specification travels with every request. It names three images (the runtime, Netlify’s platform and the function code) and the CPU, memory and connection limits. Its hash is the service identifier, so two deploys with different code or environment variables never share a microVM.
• The same service always goes to the same node thanks to rendezvous hashing. That keeps the microVM warm and the code already on disk. When a function gets too much traffic, they relax that rule and spread it across several nodes to avoid a hot spot that hurts everyone else.
• Boots in milliseconds. The microVMs are created in under a millisecond and boot in about 2 ms at p99, because they load a minimal Linux rather than a full operating system. The function code is mounted as an EROFS image and memory-mapped, so only what’s used gets read.
• Snapshots and scale to zero. When the microVM boots and the JavaScript server starts listening, they take a snapshot. If the function gets no traffic, the microVMs shut down, and the next invocation starts from that snapshot, also memory-mapped.
• Each microVM handles a limited number of requests and then shuts down, so none keeps running forever. Before it does, they start another one in parallel.

And one line that made me smile: among the problems they’ve hit at this scale they mention running out of ports on virtual switches and DNS, “we were surprised, but it wasn’t always DNS”. Just in case, every compute node now runs its own local DNS resolver.

Isolates versus microVMs

The security side is what interests me most. In February I wrote about filesystem isolation in containers and about Vercel Sandbox, which also uses Firecracker, and the underlying idea was the same: when you run other people’s code, how much isolation do you need.

A V8 isolate separates each customer’s code inside the same process, with the JavaScript engine as the boundary. It’s very light and very fast, but if someone finds a bug in V8, they’re in the same process as other customers’ code. Netlify says it bluntly: “V8 isolates, no matter their name, do not provide this level of isolation”.

With Firecracker, each function gets its own virtual machine with its own kernel. To reach another customer you’d first have to escape the runtime and then the virtualization. In Netlify’s words, a compromised deploy “cannot poison other customers or the compute layer itself”.

For years, the choice seemed forced: isolates for speed, virtual machines for isolation. What’s interesting about this article is that, with snapshots, memory-mapped images and a minimal Linux, microVMs get both.

What changes for users

Nothing, and that’s probably the best part of the announcement. URL imports, npm packages, Node built-ins, netlify.toml configuration and local development all work the same. It’s already in production, at the same price and with no migration step.

What does change is what comes next. With a real virtual machine and a real filesystem, Netlify points to three improvements:

• Taking npm package support out of beta, which currently has limitations with native binaries and importing files at runtime.
• Revisiting the current limits (50 ms of CPU per request, 512 MB of memory and 20 MB of compressed code), which came from the isolate model.
• Building services that depend on controlling the network, now that compute lives inside their own network.

What I take away

As I said at the start, this doesn’t affect this blog at all: it’s static HTML. But I think the article is a good example of something we’re seeing more and more: microVMs are becoming the default option for running third-party code. Lambda, Vercel, agent sandboxes and now Netlify’s Edge Functions are all heading the same way.

And I appreciate the detailed explanation, which isn’t that common. Describing how a request travels, what gets cached, how they avoid hot spots and what went wrong along the way is far more useful than “it’s now five times faster”. For anyone designing a system that runs other people’s code, even one with nothing to do with the edge, it’s worth reading in full.