Arslan Munir and Ann Gordon-Ross+

Slides:



Advertisements
Similar presentations
Mobile Wireless Sensor Network (mWSN) at Nokia
Advertisements

Min Song 1, Yanxiao Zhao 1, Jun Wang 1, E. K. Park 2 1 Old Dominion University, USA 2 University of Missouri at Kansas City, USA IEEE ICC 2009 A High Throughput.
1 University of Southern California Keep the Adversary Guessing: Agent Security by Policy Randomization Praveen Paruchuri University of Southern California.
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Solving POMDPs Using Quadratically Constrained Linear Programs Christopher Amato.
Optimization of intrusion detection systems for wireless sensor networks using evolutionary algorithms Martin Stehlík Faculty of Informatics Masaryk University.
ANDREW MAO, STACY WONG Regrets and Kidneys. Intro to Online Stochastic Optimization Data revealed over time Distribution of future events is known Under.
Kai Li, Kien Hua Department of Computer Science University of Central Florida.
An Introduction to Markov Decision Processes Sarah Hickmott
Planning under Uncertainty
Neeraj Jaggi ASSISTANT PROFESSOR DEPT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE WICHITA STATE UNIVERSITY 1 Rechargeable Sensor Activation under Temporally.
Energy-Efficient Target Coverage in Wireless Sensor Networks Mihaela Cardei, My T. Thai, YingshuLi, WeiliWu Annual Joint Conference of the IEEE Computer.
*Sponsored in part by the DARPA IT-MANET Program, NSF OCE Opportunistic Scheduling with Reliability Guarantees in Cognitive Radio Networks Rahul.
Bayesian Reinforcement Learning with Gaussian Processes Huanren Zhang Electrical and Computer Engineering Purdue University.
Context Compression: using Principal Component Analysis for Efficient Wireless Communications Christos Anagnostopoulos & Stathes Hadjiefthymiades Pervasive.
Planning in MDPs S&B: Sec 3.6; Ch. 4. Administrivia Reminder: Final project proposal due this Friday If you haven’t talked to me yet, you still have the.
1 University of Freiburg Computer Networks and Telematics Prof. Christian Schindelhauer Wireless Sensor Networks 5th Lecture Christian Schindelhauer.
1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.
Autonomic Wireless Sensor Networks: Intelligent Ubiquitous Sensing G.M.P. O’Hare, M.J. O’Grady, A. Ruzzelli, R. Tynan Adaptive Information Cluster (AIC)
Maximum Network lifetime in Wireless Sensor Networks with Adjustable Sensing Ranges Mihaela Cardei, Jie Wu, Mingming Lu, and Mohammad O. Pervaiz Department.
More RL. MDPs defined A Markov decision process (MDP), M, is a model of a stochastic, dynamic, controllable, rewarding process given by: M = 〈 S, A,T,R.
Yanyan Yang, Yunhuai Liu, and Lionel M. Ni Department of Computer Science and Engineering, Hong Kong University of Science and Technology IEEE MASS 2009.
Yi Wang, Bhaskar Krishnamachari, Qing Zhao, and Murali Annavaram 1 The Tradeoff between Energy Efficiency and User State Estimation Accuracy in Mobile.
Dimitrios Konstantas, Evangelos Grigoroudis, Vassilis S. Kouikoglou and Stratos Ioannidis Department of Production Engineering and Management Technical.
Exploring the Tradeoffs of Configurability and Heterogeneity in Multicore Embedded Systems + Also Affiliated with NSF Center for High- Performance Reconfigurable.
1 of 22 Markov Modeling of Fault-Tolerant Wireless Sensor Networks Arslan Munir and Ann Gordon-Ross + Department of Electrical and Computer Engineering.
Deciding when to intervene: A Markov Decision Process approach Xiangjin Zou(Rho) Department of Computer Science Rice University [Paolo Magni, Silvana Quaglini,
1 Secure Cooperative MIMO Communications Under Active Compromised Nodes Liang Hong, McKenzie McNeal III, Wei Chen College of Engineering, Technology, and.
Steady and Fair Rate Allocation for Rechargeable Sensors in Perpetual Sensor Networks Zizhan Zheng Authors: Kai-Wei Fan, Zizhan Zheng and Prasun Sinha.
M-GEAR: Gateway-Based Energy-Aware Multi-Hop Routing Protocol
OPTIMAL SERVER PROVISIONING AND FREQUENCY ADJUSTMENT IN SERVER CLUSTERS Presented by: Xinying Zheng 09/13/ XINYING ZHENG, YU CAI MICHIGAN TECHNOLOGICAL.
1 of 29 A One-Shot Dynamic Optimization Methodology for Wireless Sensor Networks Arslan Munir 1, Ann Gordon-Ross 1+, Susan Lysecky 2, and Roman Lysecky.
Research Projects in the Mobile Computing and Networking (MCN) Lab Guohong Cao Department of Computer Science and Engineering The Pennsylvania State University.
SoftCOM 2005: 13 th International Conference on Software, Telecommunications and Computer Networks September 15-17, 2005, Marina Frapa - Split, Croatia.
Tracking with Unreliable Node Sequences Ziguo Zhong, Ting Zhu, Dan Wang and Tian He Computer Science and Engineering, University of Minnesota Infocom 2009.
Prediction Assisted Single-copy Routing in Underwater Delay Tolerant Networks Zheng Guo, Bing Wang and Jun-Hong Cui Computer Science & Engineering Department,
Maximum Network Lifetime in Wireless Sensor Networks with Adjustable Sensing Ranges Cardei, M.; Jie Wu; Mingming Lu; Pervaiz, M.O.; Wireless And Mobile.
Optimal Selection of Power Saving Classes in IEEE e Lei Kong, Danny H.K. Tsang Department of Electronic and Computer Engineering Hong Kong University.
Energy-Efficient Signal Processing and Communication Algorithms for Scalable Distributed Fusion.
College of Engineering Grid-based Coordinated Routing in Wireless Sensor Networks Uttara Sawant Major Advisor : Dr. Robert Akl Department of Computer Science.
A Distributed Coordination Framework for Wireless Sensor and Actor Networks Tommaso Melodia, Dario Pompili, Vehbi C.Gungor, Ian F.Akyildiz (MobiHoc 2005)
REECH ME: Regional Energy Efficient Cluster Heads based on Maximum Energy Routing Protocol Prepared by: Arslan Haider. 1.
A Passive Approach to Sensor Network Localization Rahul Biswas and Sebastian Thrun International Conference on Intelligent Robots and Systems 2004 Presented.
A Power Assignment Method for Multi-Sink WSN with Outage Probability Constraints Marcelo E. Pellenz*, Edgard Jamhour*, Manoel C. Penna*, Richard D. Souza.
ELECTIONEL ECTI ON ELECTION: Energy-efficient and Low- latEncy sCheduling Technique for wIreless sensOr Networks Shamim Begum, Shao-Cheng Wang, Bhaskar.
Secure In-Network Aggregation for Wireless Sensor Networks
Dynamic Phase-based Tuning for Embedded Systems Using Phase Distance Mapping + Also Affiliated with NSF Center for High- Performance Reconfigurable Computing.
CUHK Learning-Based Power Management for Multi-Core Processors YE Rong Nov 15, 2011.
By Naeem Amjad 1.  Challenges  Introduction  Motivation  First Order Radio Model  Proposed Scheme  Simulations And Results  Conclusion 2.
1 of 21 Online Algorithms for Wireless Sensor Networks Dynamic Optimization Arslan Munir 1, Ann Gordon-Ross 2+, Susan Lysecky 3, and Roman Lysecky 3 1.
Syed Hassan Ahmed Syed Hassan Ahmed, Safdar H. Bouk, Nadeem Javaid, and Iwao Sasase RIU Islamabad. IMNIC’12, RIU Islamabad.
Analysis of Cache Tuner Architectural Layouts for Multicore Embedded Systems + Also Affiliated with NSF Center for High- Performance Reconfigurable Computing.
Thermal-aware Phase-based Tuning of Embedded Systems + Also Affiliated with NSF Center for High- Performance Reconfigurable Computing This work was supported.
Maximizing Lifetime per Unit Cost in Wireless Sensor Networks
PANEL Ubiquitous Systems Ubiquity for Everyone: What is Missing? Ann Gordon-Ross Department of Electrical and Computer Engineering University of Florida,
Performance of Adaptive Beam Nulling in Multihop Ad Hoc Networks Under Jamming Suman Bhunia, Vahid Behzadan, Paulo Alexandre Regis, Shamik Sengupta.
DISTIN: Distributed Inference and Optimization in WSNs A Message-Passing Perspective SCOM Team
Link Layer Support for Unified Radio Power Management in Wireless Sensor Networks IPSN 2007 Kevin Klues, Guoliang Xing and Chenyang Lu Database Lab.
Bing Wang, Wei Wei, Hieu Dinh, Wei Zeng, Krishna R. Pattipati (Fellow IEEE) IEEE Transactions on Mobile Computing, March 2012.
Smart Sleeping Policies for Wireless Sensor Networks Venu Veeravalli ECE Department & Coordinated Science Lab University of Illinois at Urbana-Champaign.
Chance Constrained Robust Energy Efficiency in Cognitive Radio Networks with Channel Uncertainty Yongjun Xu and Xiaohui Zhao College of Communication Engineering,
Learning for Physically Diverse Robot Teams Robot Teams - Chapter 7 CS8803 Autonomous Multi-Robot Systems 10/3/02.
Exploiting Dynamic Phase Distance Mapping for Phase-based Tuning of Embedded Systems + Also Affiliated with NSF Center for High- Performance Reconfigurable.
Energy Efficient Data Management in Sensor Networks Sanjay K Madria Web and Wireless Computing Lab (W2C) Department of Computer Science, Missouri University.
Wireless Access and Networking Technology (WANT) Lab. An Efficient Data Aggregation Approach for Large Scale Wireless Sensor Networks Globecom 2010 Lutful.
Wireless Sensor Network: A Promising Approach for Distributed Sensing Tasks.
Marilyn Wolf1 With contributions from:
In the name of God.
Wireless Sensor Networks 5th Lecture
Tosiron Adegbija and Ann Gordon-Ross+
Markov Decision Problems
Presentation transcript:

An MDP-based Application Oriented Optimal Policy for Wireless Sensor Networks Arslan Munir and Ann Gordon-Ross+ Department of Electrical and Computer Engineering University of Florida, Gainesville, Florida, USA + Also affiliated with NSF Center for High-Performance Reconfigurable Computing This work was supported by National Science Foundation (NSF) grant CNS-0834080

Introduction and Motivation Wireless Sensor Network (WSN) Network Application manager Sensor nodes Gateway node Sensor field Sink node

Introduction and Motivation WSN Applications Ever Increasing Ambient conditions monitoring e.g. forest fire detection Security and Defense Systems Industrial Automation Health Care Logistics

Introduction and Motivation WSN Design Forest fire could spread uncontrollably in the case of a forest fire detection application Failure to meet Catastrophic Consequences Challenges Meeting application requirements e.g. reliability, lifetime, throughput, delay (responsiveness), etc. Loss of life losses in the case of health care application Application requirements change over time Major disasters in the case of defense systems Environmental conditions (stimuli) change over time

Introduction and Motivation Commercial off-the-shelf sensor nodes Characteristics Generic Design Not Application Specific Few Tunable Parameters Tunable Parameters Processor Frequency Processor Voltage Radio Transmission Power Sensing Frequency Crossbow Mica2 mote

Introduction and Motivation Parameter Tuning Determine appropriate parameter values to meet application requirements Challenges Application managers typically non-experts e.g. agriculturist, biologist, etc. Cumbersome and time consuming task Optimal parameter value selection given a large design exploration space

Introduction and Motivation WSN Design Challenges Dynamic Optimization What solutions assist application manager??? Processor Voltage Processor Voltage Processor Frequency Processor Frequency Sensing Frequency Sensing Frequency High Values High Values Dynamically tune/change sensor node parameter values Adapts to application requirements and environmental stimuli Low Values Low Values Tunable Parameters Tunable Parameters Application manager I have hide the next slide so that you can keep whichever looks better. In this one tortoise is a bit faster but the movement is very smooth. In the next one, the movement is not that smooth =( although it took me a long time to make that one.

Introduction and Motivation WSN Design Challenges Dynamic Optimization Solution to Assist Application Manager ??? Processor Voltage Processor Voltage Processor Frequency Processor Frequency Sensing Frequency Sensing Frequency High Values High Values Dynamically tune sensor node parameter values Adapts to application requirements and environmental stimuli Low Values Low Values Tunable Parameters Tunable Parameters Application manager Hidden but complete, you can use either this one or previous one whichever looks good to u =)

Introduction and Motivation Dynamic Optimization Crossbow Mica2 mote Processor Voltage Processor Frequency Sensing Frequency Radio Transmission Power Challenges How to perform dynamic optimization? Which optimization technique to select? Formulate an optimization to perform dynamic optimization Optimal tunable parameter values selected

Contributions Dynamic Optimization Models and solves For WSNs dynamic decision making problems MDP – Markov Decision Process MDP –based Dynamic Optimization Discrete Stochastic Dynamic Programming Gives an optimal policy that performs dynamic voltage, frequency, and sensing frequency scaling (DVFS2) Optimal in any situation Adapts to changing application requirements and environmental stimuli

MDP-based Tuning Methodology for WSNs

Application Characterization Domain Application Metrics Tolerable power consumption Tolerable throughput Tolerable delay Weight Factors Signify the weight or importance of each application metric Network Sink node Gateway node Application manager Sensor nodes Sensor field MDP Reward Function Parameters (to Communication Domain) Profiling Statistics (from Communication Domain) Application Requirements Reward Function Parameters (Application Metrics & Weight Factors) Wireless Sensor Network Application Application Manager

Communication Domain Sink Node (from Application Characterization Network Sink node Gateway node Application manager Sensor nodes Sensor field MDP Reward Function Parameters (to Sensor Node Tuning Domain) (from Application Characterization Domain) Sink Node Profiling Statistics (from Sensor Node Tuning Domain) (to Application Characterization Domain)

Sensor Node Tuning Domain MDP Reward Function Parameters (from Communication Domain) MDP-based Optimal Policy MDP Reward Function Parameters Sensor Node MDP Controller Module Sensor Node Identify Sensor Node Operating State Action a Stay in same state OR Transition to some other state Sensor node state Processor voltage Processor frequency Sensing frequency Sensor Node Dynamic Profiler Module Profiles statistics Radio transmission power Packet loss Remaining battery Profiling Statistics (to Communication Domain) Find an Action a Execute Action a

MDP-based Tuning Methodology for WSNs 10 minutes.

MDP Overview With Respect to WSNs Markov Decision Process Markovian: Transition probabilities and rewards depend on the past only through the current state MDP Basic Elements Decision Epochs States State Transition Probabilities Actions Rewards

MDP Basic Elements Decision epochs State Actions Points of time at which sensor nodes make decisions Discrete time divided into periods Decision epochs correspond to the beginning of a period State Combination of sensor node parameter values Processor voltage Vp Processor frequency Fp Sensing frequency Fs Sensor node operates in a particular state at each decision epoch and period Actions Allowable actions in each state Continue operating in the current state Switch to some other state

MDP Basic Elements Transition probability Reward Policy Probability of being in a state given an action Reward Reward (income or cost) received in given state at a given time Specified by reward function Captures application requirements application metrics weight factors Policy Prescribes actions for all decision epochs MDP optimization objective Determine optimal policy that maximizes reward sequence

Application Specific Tuning Formulation as an MDP – State Space We define state space as such that where = cartesian product = total number of available sensor node state tuples [Vp, Fp, Fs ] = power for state i = throughput for state i = delay for state i

MDP Formulation – Decision Epochs The sequence of decision epochs is such that where = random variable (related to sensor node lifetime) Assumption: geometrically distributed with parameter λ Geometric distribution mean =

MDP Formulation – Action Space Determines the next state to transition to given the current state where = action taken at time t that causes transition to state j at time t+1 given current state is i action taken action not taken

MDP Formulation – State Dynamics We formulated our problem as deterministic dynamic program (DDD) Choice of an action determines next state with certainty Transfer function provides mapping useful in determining next state Transition probability function determines state dynamics where = probability that action a taken at time t dictates transitions to state j at time t+1 given current state is s

MDP Formulation – Policy and Performance Criterion Policy π that maximizes the expected total discounted reward performance criterion where = reward received at time t = discount factor (present value of one unit of reward received one unit in future) = expected total discounted reward value obtained using policy π

MDP Formulation – Reward Function Captures application metrics, weight factors, and sensor node characteristics We define reward function r(s,a) given current sensor node state s and sensor node selected action a as We define where = power reward function = throughput reward function = delay reward function = transition cost function = power weight factor = throughput weight factor = delay weight factor

MDP Formulation – Reward Function Example: Throughput Reward Function We define throughput reward function as where = throughput of the current state given action a taken at time t = minimum tolerated throughput = maximum tolerated throughput = maximum throughput in state i

MDP Formulation – Optimality Equations and Policy Iteration Algorithm Optimality equations or Bellman’s equations for expected total discounted reward criterion are where = maximum expected total discounted reward Policy Iteration algorithm MDP iterative algorithm to solve optimality equations Solves optimality equations to give MDP-based optimal policy

Numerical Results WSN Platform WSN Application eXtreme Scale Motes (XSMs) Two AA alkaline batteries – average lifetime = 1000 hours Atmel ATmega128L microcontroller Chipcon CC1000 radio – operating frequency = 433 MHz Sensors Infra red Magnetic Acoustic Photo Temperature WSN Application Security/defense system Verified for other applications Health care Ambient conditions monitoring

Numerical Results Fixed heuristic policies for comparison with πMDP πPOW = policy which always selects the state with lowest power consumption πTHP = policy which always selects the state with highest throughput πEQU = policy which spends an equal amount of time in each of the available states πPRF = policy which spends an unequal amount of time in each of the available states based on specified preference E.g. given a system with four states, it spends 40% of time in first state, 20% of time in second state, 10% of time in third state, and 30% of time in fourth state i2 20% i1 40% i3 10% i4 30%

Numerical Results – MDP Specifications Parameters for sensor node states Parameter values are based on XSM motes We consider four sensor node states i.e. I = 4 Each state tuple is given by Vp in volts, Fp in MHz, Fs in KHz Parameters specified as multiple of a base unit One power unit equal to 1 mW One throughput unit equal to 0.5 MIPS One delay unit equal to 50 ms Parameter i1=[2.7,2,2] i2=[3,4,4] i3=[4,6,6] i4=[5.5,8,8] pi 10 units 15 units 30 units 55 units ti 4 units 8 units 12 units 16 units di 26 units 14 units 6 units pi = power consumption in state i ti = throughput in state i di = delay in state i

Numerical Results – MDP Specifications Each sensor node state has allowable actions Stay in the same state Transition to any other state Transition cost Hi,j=0.1 if i ≠ j Sensor Node lifetime Mean lifetime = 1/(1-λ) E.g. when λ = 0.999 Mean lifetime = 1/(1-0.999)=1000 hours ≈ 42 days

Numerical Results – MDP Specifications Reward Function Parameters Minimum L and Maximum U reward function parameter values and application metric weight factors for a security/defense system Notation Parameter Description Value LP Minimum acceptable power consumption 12 units UP Maximum acceptable power consumption 35 units LT Minimum acceptable throughput 6 units UT Maximum acceptable throughput LD Minimum acceptable delay 7 units UD Maximum acceptable delay 16 units ωp Power weigh factor 0.45 ωt Throughput weight factor 0.2 ωd Delay weight factor 0.35

Results – Effects of Discount Factor Magnitude Difference in expected total discounted reward provides relative comparison between policies πMDP results in highest expected total discounted reward The effects of different discount factors on the expected total discounted reward for a security/defense system. Hi,j=0.1 if i ≠ j, ωp=0.45, ωt=0.2, ωd=0.35.

Results – Percentage Improvement Gained by πMDP πMDP shows significant percentage improvement over all heuristic policies Percentage improvement in expected total discounted reward for πMDP for a security/defense system. Hi,j=0.1 if i ≠ j, ωp=0.45, ωt=0.2, ωd=0.35.

Results – Effects of State Transition Cost πMDP results in highest expected total discounted reward for all state transition costs πEQU mostly affected by state transition costs due to its high state transition rate The effects of different state transition costs on the expected total discounted reward for a security/defense system. λ=0.999, ωp=0.45, ωt=0.2, ωd=0.35.

Results – Effects of Weight Factors πMDP results in highest expected total discounted reward for all weight factors The effects of different reward function weight factors on the expected total discounted reward for a security/defense system. λ=0.999, Hi,j=0.1 if i ≠ j .

Conclusions We propose an application-oriented dynamic tuning methodology based on MDPs Our proposed methodology is adaptive Dynamically determines new MDP-based optimal policy when application requirements change in accordance with changing environmental stimuli Our proposed methodology outperforms heuristic policies Discount factors (sensor node lifetimes) State transition costs Application metric weight factors

Future Work Enhancement of our MDP model to incorporate additional high-level application metrics Reliability Scalability Security Accuracy Incorporate additional sensor node tunable parameters Radio transmission power Radio sleep states Packet size Enhancement of our dynamic tuning methodology Reaction to environmental stimuli without the need for application manger’s feedback Exploration of light-weight dynamic optimizations for WSNs