Energy Efficient Digital Networks Lawrence Berkeley National Laboratory September 22, 2009 efficientnetworks.LBL.gov EEDN PAC Meeting

Agenda 10:30Welcome, Introductions, Project Overview 11:15Energy Efficient Ethernet 11:35Network Connectivity Proxying 12:15Energy Efficiency Specs for Network Equipment 12:45Break / Pick up lunch 1:00Consumer Electronic Data/Network Links 1:15Consumer Electronic Inter-device Power Control 1:45Set-Top Boxes 2:00Builder-Installed Miscellaneous 2:10Energy Star Perspective 2:20General Discussion 3:00Next Steps (discuss Building Networks, if time) 3:30Adjourn

Electronics as an End Use Electronics are an end use of electricity “Devices whose primary function is Information (obtain, store, manage, present)” –Includes both Information Technology (IT) and Consumer Electronics (CE) –Much of this digitally networked already Conventional end uses (HVAC, lighting, appliances, …) all based in physics Electronics based in information

Network Structure Edge devices: PCs, servers - Displays, storage, phones, … Network equipment: switches, and routers

Networks and Energy Network equipment …. Routers, switches, modems, wireless APs, … … vs networked equipment PCs, printers, set-top boxes, … How networks drive energy use Direct –Network interfaces (NICs) –Network products Induced in Networked products –Increased power levels –Increased time in higher power modes (to maintain network presence) Product Network Int. Network Product Link

Network electricity use in context All Electricity: ~3,700 TWh Buildings Electricity: ~2,700 TWh Commercial Residential Telecom Electronics: ~290 TWh Networked: ~150 TWh ? Network Eqt.: ~ 20 TWh One central baseload power plant (about 7 TWh/year) U.S. only Annual figures circa 2008 All approximate NOT to scale

Network electricity use in context, cont. Buildings Electricity: ~2,700 TWh Commercial Residential Electronics Networked ~150 TWh ? Net. Eqt. ~ 20 TWh U.S. only Annual figures circa 2008 All approximate One central baseload power plant (about 7 TWh/year) ~290 TWh This time to scale Tel.

Some questions worth asking How much energy does all network equipment use? … telecom equipment? … edge devices? How much energy does network connectivity induce in edge devices? [ How much energy does IT avoid ] Where is all this headed? How much can we reasonably save in network eqt.? … in edge devices? What are research and implementation priorities?

Efficiency Approaches Product Focus Network Product Focus Interface Focus Protocol / Application Focus Need all approaches ProxyingEnergy StarCE Examples:

Finding Energy Savings Opportunities Sample approaches Relax assumptions commonly made about networks –when feasible (rarely in core); mine wireless technology –these assumptions drive systems to peak performance average conditions require less energy many assumptions tied to latency Design for average condition, not just peak –rely on data about typical use Use Network to gather info about savings opportunities Use Network to enable edge device savings

Project Tasks - as proposed (% budget) - Information Technology Networks – Power-efficient Ethernet Links (13%) – Reducing Network-induced Consumption (17%) – Energy Efficiency Specs for Network Equipment (13%) Consumer Electronics Networks – Power-efficient Firewire Links (9%) – Consumer Electronics Inter-device Power Control (17%) – The Energy-efficient Set-top Box (24%) – Reducing Energy User of Hard-wired and Builder- installed Equipment in New Homes (9%)

Project Partners (named in proposal) U.S. EPA Energy Star Cisco Systems Broadcom Force 10 Networks EFI University of South Florida

Market Connection Need pathway for widespread adoption of PIER- developed technologies; for EEDN primarily: – Industry standards – Energy Star specifications Many invited talks, including: – Internet Engineering Task Force tutorial – IEEE (Ethernet) Committee tutorial – 2008 ACEEE Summer Study on EE in Bldgs. – Cisco Green Research Symposium – HP Labs Sustainability Innovation Workshop – CA Emerging Technologies Summit

Project Timeline and Status Summer Original proposal January Signed contract March Scheduled end date (plan to extend) Work is about 2/3 complete Seek input on mid-course corrections for remaining tasks Considering follow-on projects to build on accomplishments of EEDN

IT-focused EEDN projects Energy Efficient Ethernet (EEE) –Reducing power consumption of network links Network Connectivity “Proxying” –Reducing induced consumption of networked devices Efficiency Specifications for Network Equipment (Specs) –Providing market pull for more efficient network products

Agenda 10:30Welcome, Introductions, Project Overview 11:15Energy Efficient Ethernet 11:35Network Connectivity Proxying 12:15Energy Efficiency Specs for Network Equipment 12:45Break / Pick up lunch 1:00Consumer Electronic Data/Network Links 1:15Consumer Electronic Inter-device Power Control 1:45Set-Top Boxes 2:00Builder-Installed Miscellaneous 2:10Energy Star Perspective 2:20General Discussion 3:00Next Steps (discuss Building Networks, if time) 3:30Adjourn

EEE: Observations (1) Most links are mostly idle most of the time Actual traffic is bursty Most of time, full link capacity not needed Notebooks already dropped link rate in sleep Upper: file server link (Bennett, 2006) Lower: Snapshot of a typical 100 Mb/s Ethernet link (Singh)

EEE: Observations (2) Data networks are lightly utilized, and will stay that way, A. M. Odlyzko, Review of Network Economics, 2003 NetworkUtilization AT&T switched voice33% Internet backbones15% Private line networks3~5% LANs 1% Low utilization is norm in life — e.g. cars Average U.S. car ~12,000 miles/year = 1.5 miles/hour If capacity is 75 mph, this is 2% utilization

EEE: Observations (3) Source: METI, 2006 Maximum throughput (Mbit/s) Power consumption (W) Routers Throughput capacity is a function of links: Energy cost is a function of capacity, not throughput Measured power of various computer NICs (averaged) Source: Christensen, 2005

EEE: Savings Opportunity (1) 1 Gbps (and lower) copper Ethernet links in the U.S. number in the 100s of millions - # increasing each year –2 NICs for each link –Each 1 Gbps NIC requires about 1 W Servers and many network links will migrate to 10 Gbps copper –Each 10 Gbps NIC requires 5 W, maybe more With Audio/Video Bridging, Ethernet aims to penetrate the A/V market (# of possible links very large) New devices getting Ethernet (e.g. TVs)

EEE: Scope and Plan Review power consumption of several Ethernet technologies, technical approaches to changing speeds, and energy savings of these approaches Present these to IEEE, and if they take up the topic, work with them to create a standard However…. Even before we started, things changed: November, IEEE created the Energy Efficient Ethernet Study Group We adapted our role as project advanced

EEE: Overall Timeline Sometime, 2004: Bruce and Ken Christensen come up with ALR idea July, 2005: Bruce and Ken propose ALR to IEEE 802 November, 2006: approves Call For Interest, renaming effort “Energy Efficient Ethernet” January, 2007: First EEE Study Group Meeting July, 2007: approves move to Task Force status –Gets name IEEE 802.3az July, 2009: approves first working draft August, 2009: First EEE NIC announced (Infineon) September, 2010: Anticipated FINAL approval

EEE: IEEE Timeline Source: Mike Bennett, 802.3az TF Chair

EEE: Savings Opportunity (2) August 2009, Infineon announces first Gigabit EEE PHY Power drops 90% when little data traffic

Original proposal: Switch speeds quickly: << 2 seconds This became: “Rapid PHY Selection” - switching in milliseconds Later, proposal for “Low Power Idle” (LPI): promised Easier implementation Greater power savings Quicker transitions (switching in microseconds) After much discussion, EEE SG adopted LPI Also added use of LLDP (Link Layer Discovery Protocol) Enables optimal timing of transitions to maximize potential savings of hardware beyond PHY EEE: Technical Approach TqTr Tw TsTd Quiet Active Refresh Wake Sleep Active Low-PowerActive

EEE: LBNL Roles Chair of IEEE 802.3az Task Force (and Energy Efficient Ethernet Study Group): Mike Bennett, LBNL Network Group Helping to guide process through key transitions Providing savings estimates Providing guidance on policy interest in EEE Educating energy community about EEE potential Posting materials to EEE web site

EEE: Next Steps EEDN Project Prepare deliverables Beyond EEDN Continue existing roles Monitor introduction of EEE components Provide policy guidance (e.g. role of LLDP) Help Energy Star incorporate as requirement

EEE: Summary Ethernet link utilization very low Energy can be made to track utilization –Savings in PHY 90% Standards process on track to realize this Products beginning to become available Policy in U.S. ready to react EEE could penetrate 100% of market

Agenda 10:30Welcome, Introductions, Project Overview 11:15Energy Efficient Ethernet 11:35Network Connectivity Proxying 12:15Energy Efficiency Specs for Network Equipment 12:45Break / Pick up lunch 1:00Consumer Electronic Data/Network Links 1:15Consumer Electronic Inter-device Power Control 1:45Set-Top Boxes 2:00Builder-Installed Miscellaneous 2:10Energy Star Perspective 2:20General Discussion 3:00Next Steps (discuss Building Networks, if time) 3:30Adjourn

Proxying: Observations (1) LBNL Report: 1997 USF paper: 1998 Wake-on-LAN introduced: 1994 This is not a new topic:

Proxying: Observations (2) Core Fact: Most PC energy use occurs when no one present All time for year sorted by power level Most of time when idle, could be asleep PC savings potential is most of current consumption Similar patterns apply to set-top boxes, printer, game consoles, …

Proxying: Observations (3) Network connectivity a key reason for systems to be on continuously –Enterprise: Backups, IT admin access, remote access –Home: Media sharing, communications, remote access Role of network connectivity in applications increasing Game consoles and set-top boxes getting PC-like functionality

Proxying: Scope and Plan Review –PC usages for network issues –Limitations of Wake-on-LAN (WOL) –Other relevant standards (e.g. DMTF, UPnP) –Savings potentials Develop proxy specification (in part from traces) Collect comments and refine “Sell” idea to industry, standards orgs., utilities, Energy Star, CEE, etc. (possibly including prototype)

Desktop PC use nearly 70 TWh/year (U.S. only) Idle time when no one present easily half of this Goals –Enable large majority of PC users to use sleep without breaking their own or IT admin applications At least 80%. > 90% better. > 95% or > 98% even better. –Enable both current and emerging common applications –Enable standard to be used directly in (or easily adapted for) printers, set-top boxes, game consoles, etc. Savings from these also significant Proxying: Savings

Proxying: Operation LAN or Internet PC Proxy PC awake; becomes idle PC transfers network presence to proxy on going to sleep Proxy responds to routine network traffic for sleeping PC Proxy wakes up PC as needed Proxy operation Proxy can be internal (NIC), immediately adjacent switch, or “third-party” device elsewhere on network Proxy does: ARP, DHCP, TCP, ICMP, SNMP, SIP, ….

Proxying: Process Standard Ecma TC32-TG21 Prototypes Microsoft Research “Somniloquy” ??? Use Cases In development Trace Analysis Intel Research Berkeley TG21 participants Microsoft, Apple, Intel, AMD, Sony, Realtek, Oce, Hitachi, Lexmark, Terra Novum, LBNL

Proxying: Functionality Components of the standard Basic Architecture Basic Frameworks (IPv4, IPv6, 802.3, ) SNMP Teredo Remote wake UPnP mDNS/Bonjour Keepalives

Proxying: Timeline (partial) 1998: Ken Christensen publishes Proxy Server paper 2001/2003: LBNL surveys find most desktop PCs on 24/7 in comm. bldgs. 2003: Bruce and Ken begin discussions September, 2004: Energy Star announces at IDF intention to address the “Network Problem” July, 2005: Bruce and Ken present proxying to IEEE 802 January, 2007: EEDN project officially commences September, 2007: Ethernet Alliance publishes White Paper December, 2007: Bruce presents proxying to IETF January, 2008: Intel commits to helping May, 2008: Initial discussions with Ecma International September, 2008: Ecma creates TC32-TG21 on proxying October, 2008: First TG21 phone meeting January, 2009: First TG21 Face-to-Face meeting June, 2009: Apple announces external proxying for mDNS/Bonjour July, 2009: Energy Star computer spec V5.0 includes proxying September, 2009: Fifth TG21 Face-to-Face meeting November, 2009: Anticipated last F2F November/09-March/10: Standard published Shortly thereafter: Energy Star recognize Ecma standard

Proxying: LBNL’s role Pull idea from academic obscurity (work of Ken Christensen) Create interest in idea Work to put into Energy Star specification Work with Intel Research on trace analysis Identify best standards organization (Ecma International) Secure creation of Ecma TC32-TG21 “Encourage” industry participation in TG21 Define overall architecture of standard and several components Secretary for TG21 Subcontract to Terra Novum (Tom Bolioli) –TG21 contributor, convenor, Energy Star advisor

Proxying: Next Steps EEDN project Help finish standard Create deliverables Beyond EEDN –All working with industry and others Test internal proxying at LBNL and elsewhere Test external proxying at LBNL and elsewhere Refine Ecma standard based on testing Widely deploying initial proxy implemenations Create standard for communication with external proxy Create standard for monitoring proxy success Help extend proxying to printers, game consoles, and set-top boxes (and TVs and phones and …)

Proxying: Summary Most PC energy use occurs when no one present Network connectivity a key barrier to using sleep Technology can enable network connectivity in sleep Ecma standard will define much of this Energy Star ready to respond Additional steps needed to fully launch proxying

Agenda 10:30Welcome, Introductions, Project Overview 11:15Energy Efficient Ethernet 11:35Network Connectivity Proxying 12:15Energy Efficiency Specs for Network Equipment 12:45Break / Pick up lunch 1:00Consumer Electronic Data/Network Links 1:15Consumer Electronic Inter-device Power Control 1:45Set-Top Boxes 2:00Builder-Installed Miscellaneous 2:10Energy Star Perspective 2:20General Discussion 3:00Next Steps (discuss Building Networks, if time) 3:30Adjourn

Equipment energy use changes little with load Utilization is very low Exaggerated estimates of network energy use Specs: Observations on Energy and Network Equipment 367 W 376 W

Specs: Savings Opportunity Market differentiation exists in switch energy use Dialog with industry to better estimate savings Power use is a new design parameter –Previous design paradigm: reliability –Savings estimates vary from 25% to 75% –Achievable savings unknown

Estimate the annual energy use of IP networks Estimate how energy use may change Focus specifications efforts on the products with the most potential impact Develop procedures and specifications –In collaboration with industry –In collaboration with Energy Star Specs: Scope of Study Annual Energy Use (2008) Gig Switches 10/100 Switches Residential Network Equipment Enterprise Switches

Specs: Scope of Study Focus on network equipment that primarily carries IP traffic –LAN switches, routers –Home modems, routers –WiFi access points –Service provider equipment Lots of things not covered –POTS switching equipment –IP phones –Servers & desktop computer –Other end use and non-IP switching equipment Product Network Int. Network Equipment

Specs: Plan OriginalRevised All equipment together Focus on small equipment first Large equipment spread over time Identify market interest, opportunities and barriers Evaluate equipment power consumption and estimate typical power Develop test procedures & specifications Work with Energy Star on a program for network equipment

Specs: Small & Large Equipment Small equipment –Unmanaged switches < 9 ports –WiFi Access Points and Routers –Integrated home access devices –Optical network terminals Large equipment –Managed & modular switches –Dedicated security appliances In support of Energy Star Specifications Process –Small in late 2009 –Large in 2010

Specs: Rough Energy Use Estimates Energy in USA: 19 TWh/yr (0.7% of US bldg total, 2008) Grew 16% between 2007 and 2008 Forecast growth rate ~10% annually Sources: Infonetics Market Data, FCC Broadband Market Data Tolly Group Power Measurements LBNL Power Measurements AT&T Market Estimates Industry Data Sheets LBNL Market Research

Specs: Rough Energy Use Estimates Customer Premises Equip (Small Equipment): 5.9 TWh Switching Products: 8.0 TWh 19 TWh Total

Specs: Products of Interest Small Network Equipment (SNE) –WiFi routers and access points –Cable modems –DSL integrated access devices –Cable integrated access devices –Unmanaged wired switches Large Network Equipment (LNE) –Managed switches –Modular core switches Line card vs box rating

Specs: SNE Market Interest and Barriers SNE dominated by ISP provided hardware Manufacturers of ISP provided SNE are not very interested –“Only if the service providers tell us to do it.” –Cost and reliability reign supreme –Integrated access devices are a moving target Other SNE manufacturers are interested –At least for marketing purposes –Some actual energy use reductions

Specs: LNE Market Interest and Barriers The “large equipment” market is interested –“But the real energy is in the data center (or PCs)” –Industry is addressing energy use ATIS network equipment test procedures, metrics Marketing flaunts green credentials Companies considering power in current redesigns

Utilization is very low (throughput / capacity) –Need to test at realistic work loads Specs: What to Test Uplink utilization (10 days) % Capacity 1% 0.5% Modular switch with 144 GigE, 2x10GigE

Most ports are inactive & unassigned –Incentivize power reduction when ports unassigned Specs: What to Test Port Utilization (ports assigned / total powers N = 183 switches

Throughput has small impact on power (<10% typical) –Incentivize stronger throughput vs power relationship Specs: What to Test Startup Device under test: 6U modular switch with 96 GigE ports, 2x10 GigE (fiber), no PoE load, 320 Gbps fabric Tested at LBNL, August 2009

Specs: Spec Development Progress Plan to use industry standard ATIS test procedures –Some modifications may be required Attended ATIS meeting on network equipment test procedure development –Procedures under development for most areas of interest –Working on a more formal relationship where we can contribute to the process directly Internal procedures for testing network equipment –Provide a basis for our comments to ATIS

Specs: Next Steps Revise energy use estimates –Industry vetting –Values changing with time Complete market assessment Specification development –Relationship with ATIS –Work with Energy Star on modifications Beyond EEDN: Continue Energy Star process Revisit procedures in light of data & technology

Specs: Summary Current network energy consumption ~ 19TWh annually Major consumers –Small network equipment –Enterprise switches Test procedure development is underway In a good position to assist Energy Star process