· firecracker · mangofaas · architecture
Firecracker, part three: a request's journey through MangoFaaS
Gateway to Kafka to a microVM and back over vsock — with the first cold-start numbers.
Previously
An API gateway mapping HTTP routes to (FunctionId, FunctionVersion), disk bloat from giving every microVM its own full rootfs, and the fix: OverlayFS with one shared read-only base and a per-VM sparse writable ext4 layer.
Enough preparation. Let’s walk through MangoFaaS from the point of view of everyone involved.
I’m an HTTP request. My journey starts in MangoFaaS.Gateway. I carry a Host and a Path; the gateway uses them to work out the tenant and which function version to invoke.
I’m a function owner. My journey starts in MangoFaaS.Functions. I register a function by uploading a ZIP of my .NET application to S3 with metadata: entrypoint and runtime. A background job discovers new version ZIPs in the bucket, builds a function overlay image, and pushes it to an images bucket.
Worth noting: those images are sparse files. To avoid shipping all the zero-filled regions we compress them — currently with GNU tar, because .NET’s Tar implementation doesn’t preserve sparseness. So we shell out to the lovely GNU tar.
I’m the HTTP request again. I’ve been routed and I’m sitting in Kafka. I move toward the first free MangoFaaS.Firecracker.Node. (Future improvement: prefer nodes that already have my function cached.)
I’m MangoFaaS.Firecracker.Node. I:
- put the request in a pending list;
- download, if not cached, the latest kernel, the runtime base image and the function’s overlay image;
- launch or prepare a Firecracker microVM and wait for the function runtime to connect to my Kestrel server over vsock and ask for work.
I’m the function runtime. I connect and poll the Kestrel server over vsock, process the request, and send the response back down the same channel.
The response then travels back through the node and out via the gateway that took the inbound request.
Early timings
| Case | Time |
|---|---|
| Very cold — image downloads required | ~10 s |
| Not-so-cold — images cached locally | ~6 s |
| Warm — VM already running | negligible |
Next: snapshots, prewarming, smarter scheduling, cache affinity.