A System that Never Stops

As of 2015, New Relic received half a trillion data points from customers daily. All of these need to be processed, stored and made available for charts, queries and alerts as soon as possible. Getting from humble roots to querying more than a billion events a second has meant radically reinventing and re-architecting the whole platform multiple times. Matthew Flaming and Nic Benders shared a journey designing scalable services, optimizing for predictability, creating software architecture through team structure, and what it takes to continuously rebuild a system that can never stop. youtube

We are drawn back to this talk somewhat often because it captures a truth about modern software engineering. Any company with software running in the cloud faces this fundamental challenge of continually updating a living system.

This is much more like a biology or ecosystem problem than what we widely accept as true for computer problems.