Not Speedy

This gives you some idea how absurdly slow microservices are.

So, we are around ~30 developers who maintain ~5 micro-services which run on ~1000 EC2 instances that serve around 100k requests/minute1.

Thatโ€™s around a thousand virtual machines to handle 1,666 requests per second. By contrast, at my previous employer we had single machines that could handle 10,000 requests per second, or 600,000 requests per minute. Sure, those were extremely beefy machines with 6TB of memory (yeah, you read that right), but thereโ€™s no way they cost even 1/10 as much as running 1,000 EC2 instances. And each machine handled roughly six times the number of requests with far less complexity.

The post itself is about โ€œobservability.โ€ Observability is whatโ€™s required when youโ€™ve built a system so complex that you canโ€™t actually troubleshoot it. Itโ€™s a way of saying, โ€œWe donโ€™t know what the fuck is going on or why something breaks, so we have to watch it with also-expensive and very slow tools and then guess or just change things nearly-randomly and see what happens.โ€

Microservices arenโ€™t all bad. But the above is what you get when you turn developers loose on infrastructure. Since they donโ€™t understand it or why/how it works, extreme weirdness like this occurs.