Internet Monitoring and Results for the Digital Divide Les Cottrell SLAC, Aziz Rehmatullah NIIT, Jerrod Williams SLAC, Akbar Khan NIIT Presented at the.

Slides:



Advertisements
Similar presentations
EC workshop on Support to RENs in SE-Africa Session 2 Chair: W. von Rüden, CERN.
Advertisements

1 PingER End to End Internet measurements: what we learn Les Cottrell SLAC, Presented at the OARC/TechDay for the ICANN San Francisco March 7 th, 2011.
Geolocation Les Cottrell – SLAC University of Helwan / Egypt, Sept 18 – Oct 3, 2010 Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end.
Quantifying the Digital Divide: Latin America, S. Asia, Africa Prepared by: Les Cottrell SLAC, Shahryar Khan NIIT/SLAC, Jared Greeno SLAC ICFA Workshop.
1 Quantifying the Digital Divide from Within and Without Les Cottrell, SLAC Internet2 Members Meeting SIG on Hard to Reach Network Places, Washington,
MAGGIE NIIT- SLAC On Going Projects Measurement & Analysis of Global Grid & Internet End to end performance.
Quantifying the need for Improved Network Performance for S. Asia Prepared by: Les Cottrell SLAC & Shahryar Khan NIIT For the Internet2 Special Interest.
1 Effects of Mediterranean Fibre Cuts seen by PingER, Jan Prepared by: Les Cottrell SLAC, Qasim Lone NIIT/SLAC
1 Network Monitoring for SCIC Les Cottrell, SLAC For ICFA meeting September, 2005 Initially funded by DoE Field Work proposal. Currently partially funded.
1 PingER: Methodology, Uses & Results Les Cottrell SLAC, Warren Matthews GATech Extending the Reach of Advanced Networking: Special International Workshop.
Quantifying the Digital Divide from an Internet Point of View Les Cottrell SLAC, Aziz Rehmatullah NIIT, Jerrod Williams SLAC, Akbar Khan NIIT Presented.
Stanford University, SLAC, NIIT, the Digital Divide & Projects Prepared by Les Cottrell, SLAC for the NIIT Under Graduate Students, March 15, 2007.
1 Prepared by: Les Cottrell SLAC, Umar Kalim SEECS,NUST/SLAC European Geosciences Union General Assembly 2009 session on African Cyberinfrastructures,
Quantifying the Digital Divide from an Internet Point of View Les Cottrell SLAC, Aziz Rehmatullah NIIT, Jerrod Williams SLAC, Akbar Khan NIIT Presented.
The Digital Divide Prepared by: Les Cottrell SLAC Shahryar Khan NIIT, Akbar Mehdi NIIT
1 ICFA/SCIC Network Monitoring Prepared by Les Cottrell, SLAC, for ICFA
1 Monitoring Internet connectivity of Research and Educational Institutions Les Cottrell – SLAC/Stanford University Prepared for the workshop on “Developing.
PingER: Research Opportunities and Trends R. Les Cottrell, SLAC University of Malaya.
Internet View of the Digital Divide, especially for Sub- Saharan Africa Prepared by: Les Cottrell SLAC, Shahryar Khan NIIT/SLAC, Jared Greeno SLAC 2 nd.
Monitoring the World’s Networks SCIC Monitoring Group (R. Cottrell et al.) Mapping the Digital Divide.
Quantitative Measurement of the Digital Divide Prepared by: Les Cottrell SLAC with Shahryar Khan NIIT
A View from the Stoep: Tracking the evolution of local hosting in ZA and beyond Joe Abley, Director of Architecture Jim Cowie, Chief Scientist iWeek 2015.
PingER Project Arguably the world’s most extensive active end-to-end Internet Performance Project –Digital Divide emphasis –Partially funded by MoST, US.
Sub-Saharan Africa is a Dark Zone for World Internet: Sounding an Alarm Les Cottrell SLAC, Aziz Rehmatullah NIIT, Jerrod Williams SLAC, Akbar Khan NIIT.
Quantifying the Digital Divide: A scientific overview of the connectivity of South Asian and African Countries Les Cottrell SLAC, Aziz Rehmatullah NIIT,
Setting a Target for Maternal Mortality Marjorie Koblinsky, USAID Thomas Pullum, MEASURE DHS Tessa Wardlaw, Danzhen You, UNICEF Lale Say, Doris Chou (WHO)
1 Quantifying the Digital Divide from Within and Without Les Cottrell, SLAC International ICFA Workshop on HEP Networking, Grid and Digital Divide Issues.
UbuntuNet Alliance Information for Change: Securing affordable high speed connectivity and efficient ICT access and usage for African.
Sub-Saharan Africa is a Dark Zone for World Internet: Sounding an Alarm Prepared by: Les Cottrell SLAC, presented by Warren Matthews GATech Presented at.
LAN and WAN Monitoring at SLAC Connie Logg September 21, 2005.
1 New E. Coast of Africa Fibre Prepared by: Les Cottrell SLAC, Umar Kalim SEECS,NUST/SLAC Presented.
Report from ICFA Digital Divide Workshop Daegu, Korea, May ‘05 Les Cottrell, Presented at the Internet2 Fall members Meeting, SIG on Supporting International.
P AKISTAN E DUCATION & R ESEARCH N ETWORK PERN2 – Looking into the Future.
1 connect communicate collaborate FEAST Feasibility Study for African – European Research and Education Network 27th October 2009 Michael Nowlan, DANTE.
Connecting People, Reaching Out Partnering with National Research and Education Networks (NRENs) Michael Foley, SARDE, World Bank.
Measurement & Analysis of Global Grid & Internet End to end performance (MAGGIE) Network Performance Measurement.
SCIC in the WSIS Stocktaking Report (July 2005): uThe SCIC, founded in 1998 by ICFA, is listed in Section.
1 The PingER Project: Measuring the Digital Divide PingER Presented by Les Cottrell, SLAC At the SIS Show Palexpo/Geneva December 2003.
1 Quantifying the Digital Divide from Within and Without Les Cottrell, SLAC International ICFA Workshop on HEP Networking, Grid and Digital Divide Issues.
1 Quantifying the Digital Divide Les Cottrell – SLAC Prepared for the ICFA-SCIC video meeting, May 2003
1 Network Monitoring for SCIC Les Cottrell, SLAC ICFA/SCIC meeting August 24, aug05.ppt Initially.
1 Measurements of Internet performance for NIIT, Pakistan Jan – Feb 2004 PingER From Les Cottrell, SLAC For presentation by Prof. Arshad Ali, NIIT.
Locating hosts by TULIP (Trilateration Utility for Locating IP hosts) Prepared by: Les Cottrell SLAC, Faran Javed NIIT, Shahryar Khan NIIT,Umar Kalim NIIT.
Global trends in telecom development Havana, 22 October 2001 The original document was elaborated by Dr Tim Kelly, ITU/SPU. It has been completed by Saburo.
1 Measuring The Digital Divide Prepared by: Les Cottrell SLAC, Shahryar Khan NIIT/SLAC, Jared Greeno SLAC, Qasim Lone NIIT/SLAC Presentation to Princess.
Global trends in telecom development Saburo TANAKA Councellor ITU/TSB The original document is elaborated by Dr Tim Kelly, ITU/SPU. It has completed by.
1 Quantifying the Digital Divide: focus Africa Prepared by Les Cottrell, SLAC for the NSF IRNC meeting, March 11,
Quantifying the need for Improved Network Performance for S. Asia Prepared by: Les Cottrell SLAC & Shahryar Khan NIIT For the Internet2 Special Interest.
Cyberinfrastructure in Africa, Survey on Internet capability Monique Petitdidier IPSL, Paris, France.
ICFA Standing Committee on Interregional Connectivity (SCIC) ICFA Standing Committee on Interregional Connectivity (SCIC) Harvey B. Newman Harvey B. Newman.
Internet Connectivity and Performance for the HEP Community. Presented at HEPNT-HEPiX, October 6, 1999 by Warren Matthews Funded by DOE/MICS Internet End-to-end.
Visualizing the Digital Divide from an Internet Point of View & Challenges Prepared by: Les Cottrell SLAC Umar Kalim NIIT, Shahryar Khan NIIT, Akbar Mehdi.
The Current Status of Research Networking in Pakistan. Arshad Ali (NIIT) and Warren Matthews (Georgia Tech).
1 PingER performance to Bangladesh Prepared by Les Cottrell, SLAC for Prof. Hilda Cerdeira May 27, 2004 Partially funded by DOE/MICS Field Work Proposal.
Health and development: a global comparison. Key knowledge Definitions of developed and developing countries according to the WHO, including high/low.
1 IEPM / PingER project & PPDG Les Cottrell – SLAC Presented at the NGI workshop, Berkeley, 7/21/99 Partially funded by DOE/MICS Field Work Proposal on.
1 Quantifying the Digital Divide Prepared by Les Cottrell, SLAC for the Internet2/World Bank meeting, Feb 7,
Mini Presentations: Economic & Social & Composite Indicators of Development.
Pinger and IEPM-BW activity at FNAL By Frank Nagy FTP/CCF Computing Division Fermilab.
Implementing National Research and Education Networks (NRENs) in land locked African countries: critical success factors 1 D. Kunda, Director ICT B. Khunga,
Growing Economies 4.1 Globalisation.
The stakes of Development: from development to sustainable development
End to end Internet Performance today
End to end Internet Performance today
The PingER Project: Measuring the Digital Divide
PingER: An Effort to Quantify the Digital Divide
Human Development Index
MAGGIE NIIT- SLAC On Going Projects
Quantifying the Global Digital Divide
The PingER Project: Measuring the Digital Divide
Presentation transcript:

Internet Monitoring and Results for the Digital Divide Les Cottrell SLAC, Aziz Rehmatullah NIIT, Jerrod Williams SLAC, Akbar Khan NIIT Presented at the Internationmal ICFA Workshop on Grid Activities Within Large Scale International Collaborations Sinaia, Romania October 13-18,

Introduction PingER project originally (1995) for measuring network performance for US, Europe and Japanese HEP community Extended this century to measure Digital Divide Last year added monitoring sites in S. Africa, Pakistan & India Will report on network performance to these regions from US and Europe – trends, comparisons Plus early results within and between these regions

PingER Methodology Internet 10 ping request packets each 30 mins Remote Host (typically a server) Monitoring host > ping remhost Ping response packets Measure Round Trip Time & Loss Data SLAC Once a Day

PingER coverage ~120 countries (99% world’s connected population), 35 monitor sites in 14 countries New monitoring sites in Cape Town, Rawalpindi, Bangalore Monitor 25 African countries, contain 83% African population

Finding Hosts Best via contacts Also use Google to provide hosts for country (eg.ly) –Found 844 hosts => 702 unique names => 600 ping –88 unique IP addresses –6 in Libya according to Geo IP Tool Verified with TULIP geolocator –Locates hosts using RTT from multiple landmarks to target –Also see Octant for US

TULIP geolocator –Java applet (needs Java Webstart) –Friendly client, easy vizualization –Need landmarks around world Landmarks Target Enter target Pings min/avg/max from landmarks to targets Traceroutes from landmarks to targets

Minimum RTT from US Maps show increased coverage Min RTT indicates best possible, i.e. no queuing >600ms probably geo-stationary satellite Between developed regions min-RTT dominated by distance –Little improvement possible Only a few places still using satellite, mainly Africa & Central Asia –E. African Submarine System (EASSy)

Effect of Losses Losses critical, cause multi-second timeouts Typically depend on a bad link, so ~distance independent > 4-6% video-conf irritating, non-native language speakers unable to communicate > 4-5% irritating for interactive telnet, X windows >2.5% VoIP annoying every 30 seconds or so Burst losses of > 1% slightly annoying for VoIP

Losses from SLAC to world >=12% >=5% <12% >=2.5% < 5% >=1% < 2.5% < 1% # hosts monitored increased seven-fold Increase in fraction with good loss –Despite adding more hosts in developing world

Loss Improvement by Population Loss by country weighted by population of country

Unreachability from SLAC All pings of a set fail ≡ unreachable Shows fragility, ~ distance independent Developed regions US, Canada, Europe, Oceania lead –Factor of 10 improvement in 8 years Africa, S. Asia followed by L. America worst off

World thruput seen from US Behind Europe 6 Yrs: Russia, Latin America 7 Yrs: Mid-East, SE Asia 10 Yrs: South Asia 11 Yrs: Cent. Asia 12 Yrs: Africa South Asia, Central Asia, and Africa are in Danger of Falling Even Farther Behind Throughput ~ 1460Bytes / (RTT*sqrt(loss))

S. Asia & Africa from US Data v. noisy but there are noticeable trends Separate India & Pak: India may be gaining Africa & Pakistan are falling behind

Compare to US residence Sites in many countries have bandwidth< US residence –“10 Meg is Here”, Africa: $5000/Mb/mo –Ghana $10K/mo/2Mbps –Togo $450/mo/128Kbps

India to India Monitoring host in Bangalore from Oct ’05 –Too early to tell much, also need more sites, have some good contacts 3 remote hosts (need to increase): –R&E sites in Mumbai, Pune & Hyderabad –Government site in AP Lot of difference between sites, Gov. site sees heavy congestion

Pakistan to Pakistan 3 monitoring sites in Islamabad/Rawalpindi –NIIT via NTC, NIIT via Micronet, NTC (PERN supplier) –All monitor 7 Universities in ISB, Lahore, KHI, Peshawar Careful: many University sites have proxies in US & Europe Minimum RTTs: best NTC 6ms, NIIT/NTC 10ms - extra 4ms for last mile, NIIT/Micronet 60ms – slower links different routes Queuing = Avg(RTT)-Min(RTT) –NIIT/NTC heavily congested ms queuing –Better when students holiday –NIIT/Micronet & NTC OK –Outages show fragility NIIT Holiday

Pakistan Network Fragility NIIT/Micronet NIIT/NTC NTC NIIT/NTC heavily congested Other sites OK NIIT outage Remote host outages

Pakistan International fragility Infrastructure appears fragile Losses to QEA & NIIT are 3-8% averaged over month RTT ms Loss % Feb05 Jul05 Fiber cut off Karachi causes 12 day outage Jun- Jul ’05, Huge losses of confidence and business Another fiber outage, this time of 3 hours! Power cable dug up by excavators of Karachi Water & Sewage Board Typically once a month losses go to 20%

Routing from S Africa Seen from ZA Only Botswana & Zimbabwe are direct Most go via Europe or USA Wastes costly international bandwidth Many systemic factors: Electricity, Import duties, Skills, disease 915M people 14% world population, 2.2% of world Internet users

Satellites vs Terrestrial Terrestrial links via SAT3 & SEAMEW (Mediterranean) Terrestrial not available to all within countries PingER min-RTT measurements from S. African TENET monitoring station EASSy fibre for E. Africa Will it share sorry experience of SAT3 for W. Africa?

2006

Between Regions Red ellipses show within region Blue = min(RTT) Red = min-avg RTT India/Pak green ellipses ZA heavy congestion –Botswana, Argentina, Madascar, Ghana, BF India better off than Pak

Overall (Aug 06) ~ Sorted by Average throughput Within region performance better (black ellipses) Europe, N. America, E. Asia generally good M. East, Oceania, S.E. Asia, L. America acceptable Africa, C. Asia, S. Asia poor

UNDP Human Development Index (HDI) A long and healthy life, as measured by life expectancy at birth Knowledge, as measured by the adult literacy rate (with two-thirds weight) and the combined primary, secondary and tertiary gross enrolment ratio (with one-third weight) A decent standard of living, as measured by GDP per capita.

UNDP Technology Achievement Index (TAI) Creation of technology (e.g. patents, royalties); diffusion of recent innovations (Internet hosts/capita, high & medium tech export); Diffusion of old innovations (log phones/capita, log of electric consumption/capita); Human skills (years of schooling, enrollment in tertiary level in science, math & engineering). Less coverage (50 countries vs. 96 HDI ) Linear fit (both variables technology related) Better fit, fewer outliers S Asia better thruput vs TAI

Why does it matter: Science Scientists cannot collaborate as equal partners unless they have connectivity to share data, results, ideas etc. Distance education needs good communication for access to libraries, journals, educational materials, video, access to other teachers and researchers.

Why does it matter: Business G8 specifically pledged support for African higher education and research by “Helping develop skilled professionals for Africa's private and public sectors, through supporting networks of excellence between African's and other countries' institutions of higher education and centres of excellence in science and technology institutions” Saturating western markets High growth IT markets: BRIC NOT business as usual –New business models –Distinct needs –Dearth of distribution channels Traditional MNC Business Model >$20K per year 75 to 100 million people Some MNCs >$1, K per year 1.5 to 1.75 billion people Local Firms <$1,500 per year 4 billion people Future Opportunity? Prahalad and Hart Karen Coppock RDVP, Stanford

What can we do? The worldwide science and education community is in a unique position to facilitate persistent, non-threatening dialog and increased cooperation between nations that have often been at odds. Has a track record: –first permanent Internet connection to mainland China[1];[1] –initiating the "Silk Road" satellite system[2] to bring connectivity to central Asia;[2] –upgrading connectivity to Brazil; leading the installation and demonstrating the first 622 Mbps connection to India; –the efforts of the International Committee for Future Accelerators (ICFA) Standing Committee on Inter-regional Connectivity (SCIC[3]);[3] –and the free eJournals delivery service[4] of the Abdus Salam International Centre for Theoretical Physics (ICTP) etc[4] [1] “Networking with China”, R. L. A. Cottrell, C. Granieri, L. Fan, R. Xu, Y. Karita, CHEP04, Japan, also SLAC-PUB-6478, Aug 1994[1] [2] See [3] See [4] See Extend PingER coverage, contacts for more monitoring & remote sites,

Need contacts, can you help? Need PingER monitoring sites in Africa (only have S. Africa) Need PingER remote sites in: –Africa: All central African countries E. Africa: Ethiopia, Somalia, Tanzania, Zimbabwe N Africa: Libya W Africa: Gambia, Guinea, Ivory Coast, Liberia, Sierra Leone, Togo S Africa: Swaziland –L America Bolivia, Chile, Columbia, Costa Rica, Cuba, Honduras, Nicaragua, Paraguay –Mid East Iraq, Palestine, Syria –SE Asia Cambodia, Myanmar, Singapore, Thailand, Vietnam –S. Asia Bangladesh Need TULIP landmark/traceroute servers (especially in Africa): – –

Conclusions S. Asia and Africa ~ 10 years behind and falling further behind creating a Digital Divide within a Digital Divide India appears better than Africa or Pakistan Last mile problems, and network fragility Decreasing use of satellites, still needed for many remote countries in Africa and C. Asia –EASSy project will bring fibre to E. Africa Growth in # users % Africa, 4000% Pakistan networks not keeping up Need more sites in developing regions –Especially Africa, S. America Funding for operations would help (0.5 FTE)

More information Acknowledgements: –Harvey Newman and ICFA/SCIC for a raison d’etre, ICTP for contacts and education on Africa, NIIT/Pakistan, Maxim Grigoriev (FNAL), Warren Matthews (GATech) for ongoing code development for PingER, USAID MoST/Pakistan for development funding, SLAC for support for ongoing management/operations PingER –www-iepm.slac.stanford.edu/pinger Human Development –

Prolog: Internet growth Internet use, performance & coverage exploding –> 1Billion users –In 2004 users in China 6 => 78Million –Traffic through Amsterdam increased fourfold in 2005 –CERN-US connection 9.6kbsp ’85 to 10Gbits/s today –Typical backbone bandwidths (including transoceanic) 2.5 – 10Gbits for developed world

Prolog: New Technologies The transition to the use of "dense wavelength division multiplexing" (DWDM) to support multiple optical links on a single fiber has made these links increasingly affordable, and this has resulted in a substantially increased number of these links coming into service. At the end nodes the commoditization of Gigabit and 10 Gigabit Ethernet, new buses, and faster cpus are driving performance higher and costs lower.

Prolog: Developing world The Global Ring Network for Advanced Applications Development (GLORIAD[5]) project is providing high speed connectivity especially for Russia and China 10GBps around globe by Mar ’07);[5] The Trans-Eurasia Information Network (TEIN2[6]) is improving the connectivity of the Asia Pacific region;[6] The Latin America Cooperation of Advanced Networks (CLARA[7]) and the Western Hemisphere Research and Education Networks (WHREN[8]) Links Interconnecting Latin America (LILA) projects are bringing Gbits/s to Latin America;[7][8] EUMEDConnect[9] is improving connectivity to the Mediterranean;[9] The East African Submarine System (EASSy[10]) is bringing fibre to the E. coast of Africa;[10] Four Southern African National Research and Education Networks (NRENS) in Kenya, Malawi, Mozambique, Rwanda and South Africa have come together to found the Ubuntunet[11] Alliance for Research and Education Networking with the goal of delivering Gigabits/s connectivity to their countries and the rest of the world.[11]