4.8 What happens when DNS fails?

In your phone's contacts: Your phone works, Ravi's phone works, but your contacts app has crashed and you don't remember his number. You can't call him, even though nothing is wrong with either phone.

In the system: If users can't turn your name into an IP address, your site is down for them, even if every server is healthy. Two well-documented examples:

• October 2016, Dyn: a large DDoS attack, driven by the Mirai botnet of infected internet-connected devices, hit the managed DNS provider Dyn. Many well-known sites that used Dyn for DNS became unreachable for many users, though their own servers were running.
• October 2021, Facebook: a mistaken command during maintenance disconnected Facebook's backbone network. Its DNS servers were built to withdraw themselves from the internet when they couldn't reach the data centres, so they did, and the rest of the internet could no longer find Facebook, Instagram or WhatsApp. Because Facebook's internal tools depended on the same systems, fixing it took hours.

How to protect yourself:

• Use a reliable managed DNS provider, and for critical services consider two independent providers serving the same zone, so one provider's outage doesn't take you down.
• Keep reasonable TTLs. Very short TTLs mean resolvers ask more often, so an outage of your DNS hurts users sooner. A balance like 5 minutes to an hour, depending on how fast you need to change things, is common.
• Don't let your recovery tools depend on the thing that's broken. Keep a way to reach your systems (and a list of IP addresses) that doesn't need your own DNS.
• Watch it: monitor resolution of your main names from several places in the world (Chapter 30).

Memory code: "If DNS is down, you're down, even when everything is up."

---