Smart Home Cybersecurity: Threat and Defense in a Cyber-Physical System Professor Shiyan Hu Director, Michigan Tech Cyber-Physical System Research Group Department of Electrical and Computer Engineering Michigan Technological University

Michigan Tech CPS Research Group Currently consists of 11 faculty members and more than 50 graduate students across departments of ECE, CS, ME and School of Technology. 2

International Advisory Board Member of U.S. National Academy of Engineering, IEEE Fellow, ACM Fellow Former Editor-In-Chief, IEEE Transactions on Computer Aided Design, IEEE Fellow Editor-In-Chief, IEEE Transactions on Computers, IEEE Fellow, IET Fellow, AAAS Fellow Editor-In-Chief, ACM Transactions on Design Automation of Electronic Systems, IEEE Fellow Deputy Editor-In-Chief, IEEE Transactions on Computer Aided Design, IEEE Fellow Editor-In-Chief, IEEE Transactions on Circuits and Systems, IEEE Fellow Editor-In-Chief, IEEE Transactions VLSI, Editor-In-Chief, ACM Journal on Emerging Technologies in Computing Systems, and Former Editor-In-Chief, IEEE Design & Test of Computers, IEEE Fellow, ACM Fellow Founding Chair, IEEE Smart Cities Initiative 3

Smart Home: Industrial Perspective 4

Emerging Smart Devices 5

Smart Switch to Traditional Appliances 6

Many Sensors To Maintain 7

Smart Home: Academic Perspective 8

The Power System 5% energy efficiency improvement in residential users leads to carbon emission reduction equivalent to removing 53 million cars in U.S. 9

Why we schedule? The Single User Smart Home 10 Power flow Internet Control flow 10

Varying Energy Consumption Typical summer energy load profile in State of Ontario, Canada. One can see the peak load around 7:00pm which usually involves a lot of human activities. Source: Ontario Energy Board 11

Dynamic Electricity Pricing Set high prices at peak energy hours to discourage the energy usage there for energy load balancing Hourly Price from Ameren Illinois 12

Energy Scheduling for a Single Smart Home Given the electricity pricing, to decide when to launch a home appliance at what power level for how long subject to scheduling constraints Targets Reduce monetary cost of each user Reduce peak to average ratio of grid energy usage The smart home scheduler computes the scheduling solutions for future, so it needs the future pricing. How? 13

Two Pricing Models: Guideline and Realtime Pricing Guideline price: utility publishes it one day ahead to guide customers to schedule their appliances, through providing the predicted pricing in the next 24 hours. Real time price: utility uses it to bill customers, e.g., it obtains the total energy consumption in the past hour, computes the total bill as a quadratic function of the total energy, and then distributes the bill to each customer proportionally. 14

Dynamic Pricing + Game Theory = U.S. Solution Multiple Users? Dynamic Pricing + Game Theory = U.S. Solution Customer 1 Customer 2 Customer n ............. Game theory is used to handle the interactions among customers. 15

Decentralized Scheduling at Community level Each user schedules their own appliances separately Initialize Customer 𝑛 All users share information with each other Through the dynamic programming based algorithm Maximize 𝑃 𝐱 𝑛 | 𝐱 −𝑛 Share information 𝑙 −𝑛,ℎ Each user reschedules their own appliances separately No Converge? No Converge Yes Yes End Schedule 16

Case Study 5 communities in which each one contains 400 customers, and 2 utilities. Simulation time horizon is 24 hours from the current time, which is divided into 15-minutes time slots. 17

Average Energy Consumption and Bill Many issues beyond energy and bill Impact to electricity market Architecture Community level, city level Centralized, decentralized, hierarchical Reliability Privacy Cybersecurity 18

What Will be Discussed? Electricity Price Hack the input of a smart meter (pricing cyberattack) Hack the smart meter (hardware security) Embedded Software Purposes of hacking Individual level: bill reduction Local community level: load increase/fluctuation Larger area level: cascading effect Energy Load Hack the output of a smart meter (energy theft) 19

Vulnerability in Pricing Propagation in AMI Utility Utility Pricing Fiber Cable TI SoC Based Smart Meter w/ Remote Upgrade WiMAX Base Station Access Point Aggregator In Advanced Metering Infrastructure (AMI), WiMAX is used for the communication with smart meters. The smart meter of the customers connect to the base station of aggregator through the access point. WiMAX is able to operate on different frequency bands, primarily 2.3, 2.5, 3.65 and 5GHz. It has a throughput of 25MBps (in practice). Each access point can serve 200 smart meters at the same time. 20

Hacking Google Nest (Backdoor) Set high voltage and reboot from USB 21

Hacking Belkin Wemo (Accessible Programming Port) Remote switch How to hack? Connecting a UART adapter with “57600,8N1” Run the command “kill -9 $(ps | grep 'reboot'|sed -r -e 's/^ ([0-9]+) [0-9]+/\1/')” Root shell can be accessed Company Response New firmware adds SSL encryption and validation to prevent a malicious firmware attack. 22

Hacking Amazon TV Stick (Accessible Programming Port) Functionality Stream media to the TV using HDMI Exploitation and Payload A modified SD card is used to interface e-MMC flash device The user will mount the file system and copy a superuser binary file to the “/xbin” directory User can gain root access Company Response No response yet 23

Advanced Hacking: Secure Key Localization Input1 State1 Input2 State2 Input3 State3 . . ASIC Chip Encryption Communications Smart device communication is encrypted, but the secure key is typically in the flash but not ASIC. We can potentially locate the secure key. 24

Media Reports 25

Pricing Cyberattack For Reducing Hacker’s Bill With Attack, $4.12 paid by each customer Create a low price period. The attacker can schedule his energy consumption there with bill reduction by 34.3%, while the bill of other customers are increased by 7.9% on average. Without Attack, $3.82 paid by each customer Hacker wants to schedule here but it is expensive Fake Guideline Price Authentic Guideline Price Now it is much cheaper Actual Price Actual Energy Load 26

Pricing Cyberattack For Forming a Peak Load (Overloading) Create a peak energy load and the peak to average ratio is increased by 35.7%. The real time electricity price from 7:00 pm to 9:00 pm is increased by 43.9%. Without Attack With Attack Hacker wants to create a peak on the energy load here Expected Energy Load Fake Guideline Price Peak Energy Load Actual Energy Load 27

Cascading Impacts on a 5-Bus System Line 1 Line 2 Bus 1 Bus 2 Bus 3 Pricing cyberattack can increase the load and power flow. If the power flow on a line exceeds the capacity, the line is tripped. Line 4 Line 5 Line 3 Line 7 Line 6 Bus 4 Bus 5 28

Defense Technology For Pricing Cyberattack Detection of cyberattacks Hacker changes the guideline pricing, so the key is to detect anomaly in guideline pricing. The electricity price trends to be similar in short term. Customers can use machine learning technique to predict energy price from recent historical data. Compare the predicted guideline price with the received guideline price. Support Vector Regression is a good choice as it provides robust training result. Electricity Price from 06/11 to 06/13 from Ameren Illinois 29

The Guideline Electricity Price Prediction The electricity price of the last T days. H is the number of time slots per day. Predicted guideline electricity price is computed as Kernel Function 30

Anomaly Detection? The First Idea How to set the threshold? Set it to 0, then all manipulation could be found but too much false detection. Set it to a large value, then few false alarm with few cyberattacks detected. If one can tolerate up to an impact (e.g., 2% bill increase) due to cyberattack, then what is the right threshold? Cyberattack is detected if ||𝒂 𝑝 −𝑎|| ∞ >𝛿 31

The Second Idea: Alert if Impact is Signifcant Predicted Price Average Bill: 𝐵 𝑝 PAR: 𝑃 𝑝 Received Price Average Bill: 𝐵 PAR: 𝑃 Δ𝐵= 𝐵− 𝐵 𝑝 𝐵 𝑝 Δ𝑃= 𝑃− 𝑃 𝑝 𝑃 𝑝 32

Simulation Result (Detection with 𝛿 𝐵 =5% and 𝛿 𝑝 =2%) Predicted Guideline: Average Bill $3.83, PAR 1.17 Unattacked Guideline: Average Bill $3.82, PAR 1.153 Difference: Average Bill -0.26%, PAR -1.45% Predicted Guideline: Average Bill $3.83, PAR 1.17 Attacked Guideline: Average Bill $4.09, PAR 1.203 Difference: Average Bill 6.79%, PAR 2.82% 33

Limitation? The above technique is a point solution, with no memory on the past and no prediction to the future. If 𝛿 𝐵 =2% is used, then the hacker could simply manipulate guideline pricing with 1.9% bill increase at each time slot. Minor impact for each time slot, but cumulative impact over a long time could be significant. Need long term monitoring and detection technique. 34

Long Term Detection Last hour a smart meter hacked, and this hour it is hacked again, so will it be hacked in the next hour? ? Last hour 4 smart meters are hacked and this hour 7 smart meters hack, so what will be the next hour? ? 35

POMDP Based Long Term Defense What is POMDP? Partially Observable Markov Decision Process Why good for long term defense? Belief state, model training and probabilistic long term reward to account for the cumulative impact Three layer architecture Observation, State, Action POMDP models the interactions among them Observation 𝑜 State 𝑠 Action 𝑎 36

A Simple Example of POMDP 𝑠 0 , 𝑜 0 : No hacking, 𝑠 1 , 𝑜 1 : Smart meter 1 is hacked, 𝑠 2 , 𝑜 2 : Smart meter 2 is hacked. 𝑠 3 , 𝑜 3 : Both smart meters are hacked. 𝑆={ 𝑠 0 , 𝑠 1 , 𝑠 2 , 𝑠 3 } 𝑂={ 𝑜 0 , 𝑜 1 , 𝑜 2 , 𝑜 3 } 𝐴={ 𝑎 0 , 𝑎 1 } 𝑎 0 : No or negligible cyberattack, 𝑎 1 : Check and fix the hacked smart meters 37

Output of POMDP: Policy Transfer Graph 𝑒 0 𝑎 0 𝑒 1 𝑎 1 𝑜 0 𝑜 1 , 𝑜 2 , 𝑜 3 38

Step 1: Probabilistic State Transition Diagram 0.5| 𝑎 0 , 1| 𝑎 1 𝑠 0 Learn from historical observation data Calibrate the mapping from observation to state Apply conditional probability (Bayesian rule) 0| 𝑎 0 , 1| 𝑎 1 0| 𝑎 0 , 1| 𝑎 1 0| 𝑎 0 , 1| 𝑎 1 0.2| 𝑎 0 , 0| 𝑎 1 0.1| 𝑎 0 , 0| 𝑎 1 0.2| 𝑎 0 , 0| 𝑎 1 𝑠 3 0| 𝑎 0 , 0| 𝑎 1 0| 𝑎 0 , 0| 𝑎 1 0.5| 𝑎 0 , 0| 𝑎 1 0.5| 𝑎 0 , 0| 𝑎 1 0| 𝑎 0 , 0| 𝑎 1 1| 𝑎 0 , 0| 𝑎 1 0.1| 𝑎 0 , 0| 𝑎 1 𝑠 1 𝑠 2 0.5| 𝑎 0 , 0| 𝑎 1 0.5| 𝑎 0 , 0| 𝑎 1 39

Step 2: Probabilistic Transition to Policy Transfer Graph We need to account for the future impact 𝑒 0 :𝑎 0 𝑒 1 :𝑎 1 𝑜 0 𝑜 1 , 𝑜 2 , 𝑜 3 40

Model Future and Discount It Associate a reward to each action and weight it differently at different time slot. Find a series of actions leading to the maximum reward for the future k time slots. 𝑅 0 Discount Factor: 0.5 ×1 for 2pm 𝑎 0 𝑅 1 ×0.5 for 3pm < 𝑎 0 𝑎 1 𝑅 2 > 𝑎 0 𝑎 1 × 0.25 for 4pm 𝑎 0 < 𝑎 1 × 0.125 for 5pm 𝑎 0 > 𝑎 1 𝑎 0 < 𝑎 1 𝑎 0 < 𝑎 1 𝑎 0 < 𝑎 1 𝑅 3 41

Computing Long Term Expected Reward 𝑉 ∗ 𝑏,𝑡 80% 𝑠 0 20% 𝑠 1 𝑎 0 𝑎 1 𝑅 𝑠 0 , 𝑎 0 𝑅 𝑠 0 , 𝑎 1 𝑅 𝑠 1 , 𝑎 0 𝑅 𝑠 1 , 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 1 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 0 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑎 0 𝑎 1 𝑉 ∗ 𝑏,𝑡 = max {0.8𝑅 𝑠 0 , 𝑎 0 +0.2𝑅 𝑠 1 , 𝑎 0 ,0.2𝑅 𝑠 1 , 𝑎 1 +0.8𝑅 𝑠 0 , 𝑎 1 } 42

The POMDP Formulation A POMDP problem is formulated as 𝑆,𝐴,𝑇,𝑅,Ω,𝑂 𝑆: The system state space. 𝐴: The action space. 𝑂: The observation of the system state. 𝑇( 𝑠 ′ ,𝑎,𝑠): The state transition function, defined as the probability that the system transits from state 𝑠 to 𝑠 ′ when action 𝑎 is taken. Ω(𝑜,𝑎,𝑠): The observation function, defined as the probability that the observation is 𝑜 when the state and action are 𝑠 and 𝑎 respectively. 𝑅( 𝑠 ′ ,𝑎,𝑠): The reward function, defined as the reward achieved by the decision maker taking action 𝑎 and the state transits from 𝑠 to 𝑠′. 43

Key 1: Observation, Action → State Using the belief state, the POMDP problem is reduced to 𝐵,𝐴,𝜌,𝜏 𝐵: The space of belief state Given a new observation, the belief state is updated as 𝑏 𝑠 ′ =𝑃( 𝑠 ′ |𝑜,𝑎,𝑠)= Ω(𝑜,𝑎, 𝑠 ′ ) 𝑠∈𝑆 𝑇 𝑠 ′ ,𝑎,𝑠 𝑏(𝑠) 𝑃(𝑜|𝑎,𝑏) 𝜌(𝑎,𝑏): The intermediate reward for taking action 𝑎 in the belief state 𝑏 𝜌 𝑎,𝑏 = 𝑠∈𝑆 𝑠 ′ ∈𝑆 𝑏(𝑠)𝑅( 𝑠 ′ ,𝑎,𝑠)𝑇( 𝑠 ′ ,𝑎,𝑠) (1) 𝜏( 𝑏 ′ ,𝑎,𝑏): The transition function between the belief states 𝜏 𝑏 ′ ,𝑎,𝑏 =𝑃 𝑏 ′ 𝑎,𝑏 = 𝑜∈𝑂 𝑃 𝑏 ′ |𝑏,𝑎,𝑜 𝑃 𝑜|𝑎,𝑏 (2) 𝑏 𝑠 ′ =𝑃( 𝑠 ′ |𝑜,𝑎,𝑠)= Ω(𝑜,𝑎, 𝑠 ′ ) 𝑠∈𝑆 𝑇 𝑠 ′ ,𝑎,𝑠 𝑏(𝑠) 𝑃(𝑜|𝑎,𝑏) , given the previous belief state, previous action and current observation. Thus, 𝑃 𝑏 ′ |𝑏,𝑎,𝑜 = 1, 𝑖𝑓 (𝑏,𝑎,𝑜)⇒𝑏′ 0,𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 Note that even if we have exactly the same observations in two steps, we could have different belief states. 44

Key 2: State Transition Probability Computation When 𝑎 1 is taken, all the hacked smart meters are fixed. 𝑇 𝑠 𝑖 , 𝑎 1 , 𝑠 𝑗 = 1, 𝑖𝑓 𝑠 𝑖 = 𝑠 0 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (3) Ω 𝑜 𝑖 , 𝑎 1 , 𝑠 𝑗 = 1, 𝑖𝑓 𝑜 𝑖 = 𝑜 0 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (4) 45

From 𝑇( 𝑜 𝑖 , 𝑎 0 , 𝑜 𝑗 ) to 𝑇( 𝑠 𝑖 , 𝑎 0 , 𝑠 𝑗 ) Compute 𝑇( 𝑠 𝑖 , 𝑎 0 , 𝑠 𝑗 ) directly? In general, we cannot since we do not know the state. The action 𝑎 0 does not change the state, so we can obtain the state transition from the observation transition. Define observation transition function 𝑇( 𝑜 𝑖 , 𝑎 0 , 𝑜 𝑗 ) Training for 𝑇( 𝑜 𝑖 , 𝑎 0 , 𝑜 𝑗 ): In the past, 𝑜 0 appears 10 times before 𝑎 0 is taken. When 𝑎 0 is taken, there are 8 times it transits to 𝑜 0 and 2 times transits to 𝑜 1 . Thus, 𝑇 𝑜 0 , 𝑎 0 , 𝑜 0 =80%, 𝑇 𝑜 1 , 𝑎 0 , 𝑜 0 =20% 𝑇 𝑠′, 𝑎 0 ,𝑠 = 𝑜∈𝑂 𝑜∈𝑂′ 𝑇 𝑜′, 𝑎 0 ,𝑜 𝑃 𝑠 𝑎 0 ,𝑜 𝑃 𝑠′ 𝑎 0 ,𝑜′ (5) 𝑃 𝑠 𝑎 0 ,𝑜 = 𝑃 𝑜 𝑎 0 ,𝑠 𝑃 𝑠 𝑠′∈𝑆 𝑃 𝑜 𝑎 0 ,𝑠′ 𝑃 𝑠′ (6) 𝑃 𝑠 𝑖 is approximated by 𝑃 𝑜 𝑖 46

Key 3: State → Action? POMDP aims to maximize the expected long term reward 𝐸 𝑡=0 ∞ 𝑟 𝑡 𝛾 𝑡 (Bellman’s Optimality), where 𝛾 is a discount factor to reduce the importance of the future events, and 𝑟 𝑡 is the reward achieved in step 𝑡. 𝑉 ∗ 𝑏,𝑡 =max 𝐸 𝑡=0 ∞ 𝑟 𝑡 𝛾 𝑡 =max 𝑎∈𝐴 𝜌 𝑎,𝑏 +𝛾 𝑏 ′ ∈𝐵 𝜏 𝑏 ′ ,𝑎,𝑏 𝑉 ∗ 𝑏 ′ ,𝑡+1 Reward for each action 𝑅 𝑠 𝑖 , 𝑎 0 , 𝑠 𝑗 = − 𝐶 𝐿 1 , 𝑖𝑓 𝑆 1 ∗ ≤𝑖< 𝑆 2 ∗ − 𝐶 𝐿 1 − 𝐶 𝐿 2 , 𝑖𝑓 𝑆 2 ∗ ≤𝑖 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (7) 𝑅 𝑠 𝑖 , 𝑎 1 , 𝑠 𝑗 =− 𝐶 𝐼 −(𝑗−𝑖) 𝐶 𝑅 (8) System loss when there is an undetected cyberattack Labor cost due to detection 47

Compute Prob. Transition and Optimal Series of Actions Leading to Maximum Reward 48

𝑎 ∗ = 𝑎 1 ? No IYes 49 Obtain the training data Obtain the Observation 𝑜 Map the observation to belief state 𝑏 Compute the belief state transition 𝜏( 𝑏 ′ ,𝑎,𝑏) according to Eqn. (2) Compute the intermediate reward function 𝜌(𝑎,𝑏)according to Eqn. (1) Solve the optimization problem P to get the optimal action 𝑎 ∗ Obtain the training data Estimate the state transition probability 𝑇( 𝑠 𝑖 , 𝑎 0 , 𝑠 𝑗 ) for action 𝑎 0 using 𝑇 𝑜′, 𝑎 0 ,𝑜 according to Eqn. (5) and Eqn. (6) Reset state transition probability 𝑇( 𝑠 𝑖 , 𝑎 1 , 𝑠 𝑗 ) and observation probability Ω( 𝑜 𝑖 , 𝑎 1 , 𝑠 𝑗 ) for 𝑎 1 from Eqn. (3) and Eqn. (4) respectively. Obtain the reward functions according to Eqn. (7) and Eqn. (8) respectively. 𝑎 ∗ = 𝑎 1 ? Apply single event defense technique on each smart meter to check the hacked smart meters and fix them. IYes No 49

Simulation Setup We conduct 2 simulations on a small testcase and a large testcase. . Parameter 𝑆 ∗ 1 𝑆 ∗ 2 𝐶 𝑅 𝐶 𝐼 𝐶 𝐿 1 𝐶 𝐿 2 𝜸 5-customer 4 5 $50 $200 $500 0.9 500-customer 150 250 $2000 $25000 $100000 Compare with Heuristic method (repeatedly using single event defense technique). No defense technique. We show The impact including PAR increase, bill increase and labor cost. The observation accuracy defined as 1− 𝑖−𝑗 𝑗 , where 𝑗 is the number of hacked smart meters and 𝑖 is the observed number of hacked smart meters. 50

Observation Accuracy for The 500-Customer Testcase 51

Comparison on The 500-Customer Testcase Method No Defense Heuristic Method Proposed Method PAR Bill Labor Cost 31.3% 1 8.40% 0.313 3.42% 0.118 1.0813 Comparing with the results without defense technique, the PAR increase and bill increase are reduced by 1− 3.42% 31.3% =89.1% and 1− 0.118 1 =88.2%, respectively. Comparing with the heuristic method, our proposed method can further reduce the PAR and bill increase by 1− 3.42% 8.40% =59.3% and 1− 0.118 0.313 =62.3%, respectively at the expense of increasing the labor cost by 1.0813−1 1 =8.13%. 52

First Pricing Cyberattacks in Smart Home CPS Guideline price changes We explore interdependance between the power system (energy load) and the communication system (the transmitted price values). Actual price changes Energy usage changes 53

Energy Theft: Detection w/ Machine Learning? A smart meter is hacked such that it transmits the reading of 100kWh but actually 1000kWh is measured. Detectable through the statistical data analysis technique such as bollinger band. Energy consumption 2:00pm – 2:15pm over 100 days 54

Critical to distinguish tampered anomaly and non-tampered anomaly Problem of This Idea Critical to distinguish tampered anomaly and non-tampered anomaly False positive Anomaly data do not necessarily mean meter tampering They could be due to occasional user behavior change

Use Machine Learning and Deploy Sensors Together Feeder Remote Terminal Unit (FRTU) A device installed in the primary distribution network Monitor the power flow of the distribution system Communicate with smart meters Communicate with Distribution Dispatching Center (DDC) Perform some basic operation such as opening the switch We propose to use it for cybersecurity FRTU

Using FRTU in Tampering Detection Industrial Consumer Node Residential Consumer Distribution Transformer Feeder head Level 4 1 Level 3 2 3 4 Primary Network 10 Level 2 5 6 7 8 9 Level 1 20 11 12 13 14 15 16 17 18 19 21 Secondary Network 22 23 24 25 26 27 28 29 30 31 32 33 34 35 57

Impact of Different FRTU Deployment Insert FRTU everywhere? Please limited number of FRTUs such that the system can well detect smart meter tampering Industrial Consumer Node Residential Consumer Distribution Transformer Feeder head Mismatch detected Level 4 1 Level 3 2 3 4 Let’s go there to check… Primary Network 10 Level 2 5 6 7 8 9 Level 1 20 11 12 13 14 15 16 17 18 19 21 Tampering Secondary Network 22 23 24 25 26 27 28 29 30 31 32 33 34 35

Motivation 1 2 3 4 Primary Network 10 5 6 7 8 9 20 11 12 13 14 15 16 Probability that any of the 4 smart meters can have anomaly is 28.9% Can narrow down to 4 smart meters with 100% probability Probability that any of the 4 smart meters can have anomaly is 14.5% 2 3 4 Primary Network 10 5 6 7 8 9 20 11 12 13 14 15 16 17 18 19 21 These historical anomaly rates are changing Secondary Network 22 23 24 25 26 27 28 29 30 31 32 33 34 59 10% 0% 0% 5% 0% 0% 5% 35% 7% 0% 15% 7% 10%

Stochastic Problem Formulation Minimize FRTU usage Can narrow down to ≤ k meters with ≥ w% chance Considering future load growth We propose a stochastic optimization technique based on cross entropy optimization technique and conditional random field method 60

Theoretical Foundation of Cross Entropy Optimization ? 61

Estimating δ(a) f(X) a a Importance Sampling 62

Importance Sampling Each node is associated with a PDF indicating the probability to insert an FRTU Generate a set of samples using these PDFs Choose a set of top performance samples Update the corresponding PDF Repeat the above process until convergence 63

Our FRTU Deployment

Ongoing International Collaboration Our group is currently collaborating with 9 groups internationally, spanning both industry and academia, on the topic of smart home cybersecurity. 65

Collusive Energy Theft Attack a group of smart meters. For example, reduce mine by 1000kwh while increasing neighbors by 1000kwh. Interferes the electricity billing system leading to overloading without being sensed by the detection techniques. 66

Machine Learning Based Defense Technique Historical Data Hacked? Current Data Comparing with the historical measurement of a smart meter can indicate if it is hacked. Machine learning is suitable to find the anomaly pattern. 67

Challenge #1: EV Energy If some EVs move from a local community to the other community, since EV charging is a large load the community energy profile is significantly changed which impacts the electricity pricing. 68

Challenge #2: Renewable Energy and Net Metering Due to the renewable energy, the grid energy demand changes which impacts the electricity pricing. According to net metering, the customers are allowed to sell the generated renewable energy back to power grid. What is the right pricing? Behavior modelling? 69

Smart Building and HVAC The accurate HVAC modeling in a building can provide better energy and pricing prediction. This can help improve the cyberattack detection accuracy. 70

Hardware Security and Crosslayer Defense Electricity Price Embedded Software Energy Load Part of detection code is implemented at a smart meter, but the smart meter itself can be hacked. We need the crosslayer defense. 71

Chain of Hack Just check Java code? What if VM is hacked? Java Virtual Machine What if VM is hacked? What if OS is hacked? OS What if firmware is hacked? Firmware Hardware 72

An Example Typically, the code jumps to the beginning of a routine. A potential solution is to add some specific registers in the hardware architecture to monitor where a code jumps. The detection algorithm needs to consider both the software security analysis and the runtime readings from those specific registers. This is a crosslayer security solution, which aims to establish a chain of trust. Typically, the code jumps to the beginning of a routine. The hacker can manipulate the binary code to jump to the middle of a routine which contains malicious code. 73

Developing POMDP Based Crosslayer Defense Hierarchical Decomposition of the State Space Cross Entropy Based State Minimization Kernelized Approximate Dynamic Programming

Privacy: Obfuscation by Proxy Mapping Central Computer Central Computer Customer A Customer B Customer C Proxy Customer 1 Customer 2 Customer 3 Customer 1 Customer 2 Customer 3 75

Homomorphic Encryption Arithmetic on Encrypted Data 𝐸 𝑚,𝑟 = 𝑔 𝑚 𝑟 𝑛 𝑚𝑜𝑑 𝑛 2 𝑚=𝐿( 𝑐 𝜆 𝑚𝑜𝑑 𝑛 2 ) 𝐷 𝐸 𝑚, 𝑟 1 ∙𝐸 𝑚, 𝑟 2 𝑚𝑜𝑑 𝑛 2 = 𝑚 1 + 𝑚 2 Encryption Encryption Encryption Encrypt both communication and computation 76

Conclusion 77 Distribution Dispatching Center Primary Distribution Network with Feeder Remote Terminal Units (FRTUs) Secondary Distribution Network with Smart Meters Customer Billing Center Network 77

Thanks 78