1. How do we decide where to deploy to next?
Marty Strong
NIX meeting - Oslo
4th February 2016

2. What is CloudFlare?
CloudFlare makes websites faster and safer using our globally distributed network to deliver essential services to any website
Performance
Content
Optimisation
Security
3rd party services
Analytics
NIX - How do we decide where to deploy to next? - Marty Strong

3. How does CloudFlare work?
CloudFlare works at the network level
Once a website is part of the CloudFlare platform, its web traffic is routed through our global network of 75+ data centres.
At each edge node, CloudFlare manages DNS, caching, bot filtering, web content optimisation and third party app installations.
NIX - How do we decide where to deploy to next? - Marty Strong

4. How does CloudFlare work?
Amsterdam
Frankfurt
London
ISP DNS server
Visitor
How does it work?
DNS Query - to anycast DNS address
DNS result returned with Anycast IP
Client makes connection to returned IP
CloudFlare replies, session established
What happens in the event of an outage?
Anycast prefixes are withdrawn from problematic PoP
Traffic re-routes to next closest PoP
TCP session resets at this point
NIX - How do we decide where to deploy to next? - Marty Strong

5. CloudFlare works globally
CloudFlare protects globally
DDoS attack traffic is localised and lets other geographic areas continue to operate
NIX - How do we decide where to deploy to next? - Marty Strong

6. Step 1: Where geographically?
NIX - How do we decide where to deploy to next? - Marty Strong

7. Which city? Where are we missing dots on the map?
Where are the regional or national connectivity hubs?
Where do we have traffic destined for?
Where are the highest density cities?
Where do we have significant customers?
NIX - How do we decide where to deploy to next? - Marty Strong

8. Which city? Case study: Norway Population of ~5 million
Traffic destined to users in multiple cities across the country
Served mostly from our PoPs in Copenhagen and Stockholm, plus a little in Amsterdam
Main connectivity hub in Oslo for international transit
Latency from current PoPs to Oslo: ~7ms (ARN), ~8ms (CPH), ~20ms (AMS)
NIX - How do we decide where to deploy to next? - Marty Strong

9. Which city? Sources of information: Netflow Carrier looking glasses
Traceroute/MTR
Map of the world
NIX - How do we decide where to deploy to next? - Marty Strong

10. Which city?
Netflow:
NIX - How do we decide where to deploy to next? - Marty Strong

11. Step 2: How do we reach people?
NIX - How do we decide where to deploy to next? - Marty Strong

12. Which ISPs?
Who are the main ISPs in the country? (home, mobile & enterprise)
How can we reach them?
Do they peer?
Which transit providers can we connect to?
Where can we reach them?
Do transit providers interconnect locally?
NIX - How do we decide where to deploy to next? - Marty Strong

13. Which ISPs? Sources of information:
BGP country report (e.g.
PeeringDB
Transit provider network maps
IRR databases (RADb, RIPE, ARIN, AFRINIC, APNIC, LACNIC)
Route collector (e.g. Oregon Route views)
RIPE Atlas probes (
NIX - How do we decide where to deploy to next? - Marty Strong

14. Which ISPs? IRR example: $ whois -h whois.radb.net “AS50304”
remarks: AS50304 is represented at the following IXPs
remarks:
remarks: AMSIX, Amsterdam
remarks: NL-ix, Amsterdam
remarks: FIXO, Oslo
remarks: NIX, Oslo
remarks: STHIX, Stockholm
remarks: SOLIX, Stockholm
remarks: DE-CIX NY, New York City
remarks: RO Communities
remarks: :1000 Route is learned from a customer
remarks: :1300 Route is local
remarks: :11CC Route is learned from a peer
remarks: CC 01 USA
remarks: CC 31 Netherlands
remarks: CC 45 Denmark
remarks: CC 46 Sweden
remarks: CC 47 Norway
IRR example:
$ whois -h whois.radb.net “AS50304”
NIX - How do we decide where to deploy to next? - Marty Strong

15. Which ISPs? Route views: route-views>show ip bgp regexp _2119$
Network Next Hop Metric LocPrf Weight Path
* / i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
* i
*> i
* i
Route views:
route-views>show ip bgp regexp _2119$
NIX - How do we decide where to deploy to next? - Marty Strong

16. Which ISPs?
NIX - How do we decide where to deploy to next? - Marty Strong

17. Step 3: Which building should we colo at?
NIX - How do we decide where to deploy to next? - Marty Strong

18. Which building should we colo at?
Are there any colo companies we already use elsewhere?
Which facilities host an IXP?
Which building do our transit providers have a presence at?
Which buildings do significant ISPs have a presence at?
Which buildings can accommodate our power and space needs?
Which buildings will give us room to grow our footprint?
NIX - How do we decide where to deploy to next? - Marty Strong

19. Which building should we colo at?
Do buildings that share a business park allow cross connects between them?
What if there is not a single facility that can cover all of our requirements?
What if the local ISPs refuse to peer?
Can we source local transport between buildings?
NIX - How do we decide where to deploy to next? - Marty Strong

20. Which building should we colo at?
Sources of information:
PeeringDB
Colo provider websites
Euro-IX website (
NIX - How do we decide where to deploy to next? - Marty Strong

21. Which building should we colo at?
NIX - How do we decide where to deploy to next? - Marty Strong

22. Which building should we colo at?
NIX - How do we decide where to deploy to next? - Marty Strong

23. Which building should we colo at?
NIX - How do we decide where to deploy to next? - Marty Strong

24. Which building should we colo at?
NIX - How do we decide where to deploy to next? - Marty Strong

25. Step 4: Are there hurdles to overcome?
NIX - How do we decide where to deploy to next? - Marty Strong

26. Are there hurdles to overcome?
Are there any local laws around certain content types?
Are there any regulations on country of origin for imports of equipment?
What is the import duty/VAT situation?
Are there likely to be long customs delays?
Do we have to hire a third party to rack our equipment at the data centre?
NIX - How do we decide where to deploy to next? - Marty Strong

27. Are there hurdles to overcome?
Sources of information:
Government ministry websites
Colo provider account manager
Friendly local network contacts
Courier account manager
NIX - How do we decide where to deploy to next? - Marty Strong

28. Thank you! Questions? AS13335 Marty Strong, Network Engineer
@martystronguk
AS13335
NIX - How do we decide where to deploy to next? - Marty Strong