1. Analyzing the Costs and Benefits of DNS, DoT, and DoH for the Modern Web
Austin Hounsel* Kevin Borgolte* Paul Schmitt*
Jordan Holland* Nick Feamster†
Princeton University* University of Chicago†
Should I disclose that I’m also an intern at Mozilla?

2. DNS Privacy Has Become a Significant Concern
On-path network observers can spy on traditional DNS traffic (Do53)
Two protocols have been proposed to encrypt DNS traffic
DNS-over-TLS (DoT)
DNS-over-HTTPS (DoH)
When you visit a coffee shop, every website that you visit can be seen by passive eavesdroppers.
For the remainder of the talk, I will refer to plain-ol DNS as Do53.
DoT was proposed in RFC 7858 in 2016.
DoH was proposed in RFC 8484 in Available to use in Firefox.

3. Contributions Extensive performance study of Do53, DoT, and DoH
Query response times
Page load times
Emulated network conditions
Measurements from five global vantage points
We performed measurements from Ohio and California, Frankfurt, Seoul, and Sydney on Amazon EC2.
Each machine was configured with Debian, and they ran a measurement suite in Docker that we built.
Due to time constraints, I will focus on our measurements from Frankfurt. We also show our results for Seoul in the full paper.

4. Unexpected Finding
Despite higher response times, page load times with encrypted DNS transports can be faster than Do53

5. DNS Responses from Cloudflare at Frankfurt
DoH outperforms Do53 in tail
DoH outperforms 50% of DoT queries
This is a CDF of DNS response times with each protocol.
By “Default Do53”, I’m referring to the local resolver provided by Amazon EC2. It does not support DoT or DoH.
Cloudflare DoH outperforms DoT for 50% of queries.
Cloudflare DoH actually outperforms local Do53 and Cloudflare Do53 in the tail.

6. DNS Responses from Google at Frankfurt
DoH outperforms Do53/DoT in tail
DoH catches up to DoT
Google DoH outperforms university Do53 and Google Do53 in the tail.

7. DNS Responses from Quad9 at Frankfurt
DoH catches up to Do53 in the tail
DoH outperforms DoT for almost all queries
Quad9 DoH completely outperforms Quad9 DoT.
Quad9 DoH catches up to Quad9 Do53 in the tail.

8. Takeaway: DoH Can Outperform Do53
DoH outperforms Do53 in the tail of response times
Higher mean but lower variance
HTTP caching of the wire format may help
We also think wire-format caching generally improves DoH response times. Firefox appears to use a hard-coded transaction ID of 0 for its DoH implementation, which is recommended in RFC 8484.
This enables a recursor to put the wire-format in an HTTP cache, rather than having to cache responses that differ by transaction IDs.
It also means that the recursor can just pull the answer from an HTTP cache, rather than having to parse and construct a DNS response from scratch.
It’s unclear how much time HTTP caching saves, but we think it’s worth exploring.

9. Emulated Cellular Conditions
We performed measurements with emulated cellular conditions
DoH and DoT are starting to be offered on phones
Performance may be significantly different
DoT is implemented in Android 9, and DoH is implemented in Cloudflare app ( )
We traffic shaped our measurements with settings provided by an OpenSignal report.

10. Page Loads with Cloudflare at Frankfurt
DoT was 1ms slower than Do53
DoH was 16ms slower than Do53
These are CDFs that show the difference in page load times between DoH, DoT, and Do53.
The vertical line shows the median.
Cloudflare DoT was 1ms slower than Do53.
Cloudflare DoH was 20ms slower than Do53.

11. Page Loads with Cloudflare at Frankfurt (4G)
DoT was 11ms faster than Do53
DoH was 58ms slower than Do53
We traffic shaped our network to emulate 4G traffic by adding 50s of latency and 0.5% loss, and 8ms jitter.
We also shape our uplink rate to 7.44 MB/s and our downlink rate to 22.1 MB/s.
For the sake of time, we do not show response time plots. They are in the full paper.
DoT gets slightly faster than Do53, but DoH gets slower. It’s not clear why DoH gets slower.

12. Page Loads with Cloudflare at Frankfurt (Lossy 4G)
DoT was 101ms faster than Do53
DoH was 33ms faster than Do53
We traffic shaped our network to emulate a lossy 4G network by adding 50ms latency, 1.5% loss, and 8ms jitter.
DoT and DoH get significantly faster than Do53 on the lossy 4G network.

13. Page Loads with Cloudflare at Frankfurt (3G)
DoT was 156ms slower than Do53
DoH was 310ms slower than Do53
Lastly, we traffic shaped our network to emulate 3G traffic by adding 150ms of latency, 2.5% loss, and 8ms jitter.
We shape our uplink and downlink rates to 1 MB/s.
As shown, DoT and DoH collapse on a 3G network.
The link capacity may explain it, as DoT and DoH have a higher overhead than Do53.

14. Takeaway: TCP Helps Page Load Times
TCP packets can be retransmitted after 2x RTT
Timeout of Do53 is set to 5 seconds by default in Linux

15. Summary
Measured Do53, DoT, and DoH performance from five vantage points
Future work: performance analyses over diverse networks
Residential ISPs