Crash-Restart Patterns

Fred Hebert writes, Over the years I found that a similar pattern makes a lot of sense for Erlang processes that need to do crash-restart patterns, with the idea of "rolling back to a known stable state". In Erlang and elixir, specifically with OTP, we have access to these architectural patterns that most languages only have when you're dealing with hardware machines or virtual machines and not within the language itself. See the second half of The Hitchhiker's Guide to the Unexpected blog youtube 19m27s

The supervisor itself should contain only that immutable data that requires no dynamism

The processes themselves should be able to bootstrap themselves from having data that loads once and is sorta dynamic (what is the guarantee you offer on boot?)

The rest of the transient state is just moving around doing whatever. If you want to keep it safe, you have to make a copy of it to another area that is more stable

Part of the thing you encode in a supervision structure is that gradient from "we got nothing" and how to progressively rebuild state. A connection to a database is extremely dynamic state but you can rebuild it from less dynamic state (an IP address, which you get on DNS), which comes from something even more static (a configuration string somewhere). Being able to define that gradient of what type of dynamism you have in your state becomes really useful to properly define failure and recovery domains.