Session Title Smart Grid Communication Architecture Speaker: Doug McGinnis Company: Exelon Session Title Smart Grid Communication Architecture

2 Smart Grid at Exelon Smart Home / Business Real-time usage and pricing statistics Home Area Network (HAN) composed of smart devices and appliances that know the price of energy Smart Meters (AMI) A method to enable two-way information flow System status, customer outage status, usage and pricing signals delivered to and from location Smart Distribution System Real-time reporting of status and outages Automated controls of relays and reclosers. Efficient field force management Effective interconnection of renewable energy sources Smart Utility More efficient data collection, processing and back office functions Asset Monitoring Leveraging integrated communication systems and information processing is critical Customer End-Use Advanced Pricing & Billing In-home Devices PowerCo Many working definitions of Smart Grid Electricity Advisory Committee to U.S. Department of Energy defines Smart Grid as “a broad range of solutions that optimize the energy value change” Smart Grid leverages advancements in communication and information processing technologies to enable a highly automated integrated intelligent network Integrated communication systems Information processing Enhanced operator controls, decision support, demand response, asset condition assessment Advanced components (Automated switches, intelligent meters, microprocessor relays…) Advanced sensing and measurement Time of use pricing, outage detection and notification, energy management systems, power quality monitoring, etc. Advanced control Automatic restoration and isolation, control of distributed resources, etc. Facilitates the interconnection, operation, and grid management of renewable energy sources (wind, solar, PHEVs). There are 900MWs of wind generation currently connected to the ComEd system with pending interconnection requested for over an additional 20,000MWs. Current grid technologies can only absorb so much of these intermittent energy resources without jeopardizing the stability of the grid. The Smart Grid Home Area Network (HAN) is a network of smart devices within the home which provides for communications between these devices and may include: utility meters, appliances and in-home monitoring and control devices, e.g., the smart thermostat. The HAN is envisioned to empower customers with information and the ability to control their energy costs and improve our environment. 1234 1234567 Vehicle Electrification Plug-in Hybrid Electric Vehicles Customer Gateway Advanced Metering Infrastructure (AMI) Grid Modernization Distribution Automation Smart Substations Renewable Interconnection

Fundamental Design Principles Security – Robust end-to-end, aligned with NISTIR 7628 Deterministic – Smart Grid applications will share a logically isolated deterministic communications infrastructure. Interoperable – Industry standard protocols will be utilized with a focus on migrating to IP/Ethernet consistent with industry direction Privately Owned – Privately owned communications is preferred. Eliminate telecom circuits, O&M cost savings. No Unanalyzed Single Points of Failure (Self Healing) – The communication architecture will be designed with no unanalyzed single points of failure. Communications Maintenance Management & Monitoring – Inherent to the communications Architecture will be Communications Maintenance Management & Monitoring, i.e. the ability to maintain, monitor and control network devices. Tiered Architecture – Unit/Distribution substations are linked to larger Transmission substations in a hierarchical design through new wireless technologies Relay Protection Communications – Highest level of reliability and availability including diverse backup paths.

Architectural Framework Substation Application Portfolio – 6 application groups exist in the substation and are logically partitioned over common transport Telemetry – RTU/IED – EMS/DMS communications (encrypted) CIP Telemetry – NERC CCA sites (encrypted) DA – Field Distribution Automation traffic aggregated by TGB (encrypted) Enterprise – Business applications (email, VoIP, surveillance video) Security – Card readers AMI – TGB backhaul handoff to Core PoP Tier 1 – Substation will be a point-of-presence to the SONET backbone. SONET infrastructure provides Ethernet and TDM provisioning Routing capability to permit application provisioning over Ethernet (Layer 3) Quality of service management, data prioritization Tier 2 – RF bridge between TGB’s and non-PoP substations WiMax/Ethernet transport VLAN partitioning per application for separation

SONET Design Relay Protection Communications Parallel dual SONET ring configuration (OC3 & OC48) with PoP’s at each substation No relay channel failure with loss of a ring Path diversity – loop topology with fast switching to protected path OC3 ring – Primary Relay communications No other applications will use OC3 Static environment OC48 ring – Backup Relay, TDM & Ethernet service GigE provisioned to support Ethernet services TDM circuits for relay protection, voice & serial communications as required

Substation SONET Design

Substation Communications Architecture Substation LAN Access switch – VLAN provisioned (Layer 2) No inter-application routing will be permitted Telemetry network access/authentication will be through core SCADA Firewall – NERC CIP compliant TGB’s and other substation IP devices will be connected to switch partitioned in their respective VLAN’s Substation WAN Router (layer 3) will interface with switch and will provision Virtual Route Forwarding Tunnels (VRF) 6 VRF tunnels will be created for logical separation VRF tunnels will be encrypted using Dynamic Multipoint VPN (DMVPN) IP addressing schema will be defined for entire substation population based on application requirements

Substation LAN – WAN Architecture Network Core Work Station VRF Tunnels Substation Telemetry CIP Telemetry Field DA Enterprise Security AMI VoIP VLAN extended to switch per Application SCADA Gigabit Ethernet Camera CardReader Switch Firewall Router Switch Enterprise Core Router Firewall RTU Security DA TGB Incorporates Layer 3 VRF Tunneling and Dynamic Multipoint VPN AMI/RNI AMI TGB Ethernet based devices

Middle Mile (Tier 2) Architecture Requirements Transport the Smart Grid application portfolio AMI backhaul – (200kbps/TGB @ 70 Aggregation pts) Distribution Automation (100kbps/TGB @ Aggregation pts) Substation Telemetry (56kbps/substation) Voice/Video (~1Mbps per video stream) Application Traffic Considerations Bandwidth consumption (5-20Mbps) Latency sensitivity (QoS tagging) Security (PKI) Logical provisioning of applications (VLAN tagging)

Middle Mile Design Considerations Spectrum Options – Licensed vs. Unlicensed Unlicensed (ISM) Free Limited Power Uncontrolled Noise Floor Uncontrolled Interference 900 MHz – Hi Noise 2.4 GHz – Hi Noise 3.65 GHz (Lite License) 5.X GHz Licensed High cost – Spectrum Market Strict FCC deployment rules High Power Operation Low/controlled Noise Floor Interference Remedy 700 MHz Public Safety 900 MHz 2.3 GHz

What Spectrum to Use? What characteristics need to be considered? Free-space path loss is proportional to the square of the distance between the transmitter and receiver, and proportional to the square of the frequency of the radio signal. Higher the frequency – Lower the signal propagation What type of coverage are you planning for? Blanket umbrella (target lower frequencies with minimal interference characteristics) Surgical microcell (target higher frequencies) Urban (target lower frequencies with minimal interference characteristics) Suburban/rural (depends on the type of coverage, Blanket vs. Surgical)

Spectrum Evaluation Frequencies Requirements Overall Ranking 2 6 5 1 3 700Mhz 900Mhz 2.3Ghz 3.65GHZ 5.8Ghz 6-11Ghz Risk High Medium Low Cost Coverage Excellent Adequate Good Equipment Availability Limited Growing Licensed √ No  Unlicensed Lightly No   Availability – PECO area Point-to-Point Point-to-Multi Point Overall Ranking 2 6 5 1 3 4 Ranking: 1 high - 6 low

Tier II Integration (Conceptual) Design

Two prevailing backhaul Standards Technology Standards Two prevailing backhaul Standards WiMax 802.16 Long Distance 5-10 Miles P2P & P2MP Time Slotted Access Mobility (802.16.e) 5-10 MHz Channels 802.1Q (VLAN, QoS) Security EAP/TLS/AES WiFi 802.11 Shorter range 1-2 Miles Meshing capability Contention Access DSSS 22 MHz Channels 802.1Q (VLAN, QoS) Security EAP/TLS/WPA 2

Bringing it together Backhaul 100 AMI/DA collector sites Provide backhaul of non-fiber substations Requires surgical deployment, not blanket coverage Point-to-point links Microcell canopies Decided WiMax using 3.65 GHz Lite-License Minimal noise at this time Do not require full territory coverage Point-to-point links where necessary

Tier II Conceptual Design