Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Emmanuel Fernandez Associate Professor

Slides:

Advertisements

Similar presentations

CPSC 422, Lecture 9Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 9 Jan, 23, 2015.

Advertisements

Practical and Theoretical Issues on Adaptive Security Alexander Shnitko Novosibirsk State Technical University.

1 University of Southern California Keep the Adversary Guessing: Agent Security by Policy Randomization Praveen Paruchuri University of Southern California.

Partially Observable Markov Decision Process (POMDP)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Solving POMDPs Using Quadratically Constrained Linear Programs Christopher Amato.

1 SRC/ISMT Factory Operations Research Center SRC/ISMT FORCe:Factory Operations Research Center Task NJ-877 Michael Fu, Director Emmanuel Fernandez Steven.

1 Investing the LANCS Resources Edmund Burke Nottingham / Stirling Sanja Petrovic LANCS Initiative Scientific and Industrial Advisory Board November

Planning under Uncertainty

Neeraj Jaggi ASSISTANT PROFESSOR DEPT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE WICHITA STATE UNIVERSITY 1 Rechargeable Sensor Activation under Temporally.

Integrated MEMS and Advanced Technologies for the Next Generation Power Distribution System Arizona State University Tempe, AZ Research Team Esma Gel,

1 Chapter 12: Decision-Support Systems for Supply Chain Management CASE: Supply Chain Management Smooths Production Flow Prepared by Hoon Lee Date on 14.

ISM 206 Optimization Theory and Applications Fall 2005 Lecture 1: Introduction.

ISM 206 Optimization Theory and Applications Spring 2005 Lecture 1: Introduction.

Genetic Algorithms for multiple resource constraints Production Scheduling with multiple levels of product structure By : Pupong Pongcharoen (Ph.D. Research.

Zhu, Song-Chun and Mumford, David. A Stochastic Grammar of Images. Foundations and Trends in Computer Graphics and Vision 2(4), (2006) Hemerson.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Optimal Fixed-Size Controllers for Decentralized POMDPs Christopher Amato Daniel.

Reinforcement Learning Yishay Mansour Tel-Aviv University.

1 Supply Chain Decision Support Systems ISyE3103.

Instructor: Vincent Conitzer

The Systems Modeling & Information Technology Laboratory 1 Intelligent Preventive Maintenance Scheduling In Semiconductor Manufacturing Fabs Preventive.

MAKING COMPLEX DEClSlONS

Distributed control and Smart Grids

Sérgio Ronaldo Barros dos Santos (ITA-Brazil) Sidney Nascimento Givigi Júnior (RMC-Canada) Cairo Lúcio Nascimento Júnior (ITA-Brazil) Autonomous Construction.

ECES 741: Stochastic Decision & Control Processes – Chapter 1: The DP Algorithm 1 Chapter 1: The DP Algorithm To do:  sequential decision-making  state.

European Network of Excellence in AI Planning Intelligent Planning & Scheduling An Innovative Software Technology Susanne Biundo.

1 Management Information Systems Transaction Processing Systems (TPS) –Support operation –Management and control –Routine, normal operations Management.

Artificial Intelligence

LECTURE 8-9. Course: “Design of Systems: Structural Approach” Dept. “Communication Networks &Systems”, Faculty of Radioengineering & Cybernetics Moscow.

Supply Chain Management AN INITIATIVE BY: VAINY GOEL BBA 1 MODI COLLEGE.

1 ECE-517 Reinforcement Learning in Artificial Intelligence Lecture 7: Finite Horizon MDPs, Dynamic Programming Dr. Itamar Arel College of Engineering.

INVESTIGATORS R.E. King S-C. Fang J.A. Joines H.L.W. Nuttle STUDENTS P. Yuan Y. Dai Y. Ding Industrial Engineering Textile Engineering, Chem. and Science.

CPSC 422, Lecture 9Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 9 Sep, 28, 2015.

INVESTIGATORS R. King S. Fang J. Joines H. Nuttle STUDENTS N. Arefi Y. Dai S. Lertworasirikul Industrial Engineering Textiles Engineering, Chem. and Science.

Model-based Bayesian Reinforcement Learning in Partially Observable Domains by Pascal Poupart and Nikos Vlassis (2008 International Symposium on Artificial.

Introduction to Reinforcement Learning Dr Kathryn Merrick 2008 Spring School on Optimisation, Learning and Complexity Friday 7 th.

Tactical Planning in Healthcare with Approximate Dynamic Programming Martijn Mes & Peter Hulshof Department of Industrial Engineering and Business Information.

CPSC 7373: Artificial Intelligence Lecture 10: Planning with Uncertainty Jiang Bian, Fall 2012 University of Arkansas at Little Rock.

Privacy-Preserving Bayes-Adaptive MDPs CS548 Term Project Kanghoon Lee, AIPR Lab., KAIST CS548 Advanced Information Security Spring 2010.

Decision Making Under Uncertainty Lec #8: Reinforcement Learning UIUC CS 598: Section EA Professor: Eyal Amir Spring Semester 2006 Most slides by Jeremy.

© TRESETarget Industry TRESE Group Department of Computer Science University of Twente P.O. Box AE Enschede, The Netherlands

Reinforcement Learning Yishay Mansour Tel-Aviv University.

Hidden Markov Model Multiarm Bandits: A Methodology for Beam Scheduling in Multitarget Tracking Presented by Shihao Ji Duke University Machine Learning.

Instructor: Spyros Reveliotis homepage: IE7201: Production & Service Systems Engineering Fall.

CPS 570: Artificial Intelligence Markov decision processes, POMDPs

1 ECE 517: Reinforcement Learning in Artificial Intelligence Lecture 21: Dynamic Multi-Criteria RL problems Dr. Itamar Arel College of Engineering Department.

4 th International Conference on Service Oriented Computing Adaptive Web Processes Using Value of Changed Information John Harney, Prashant Doshi LSDIS.

Generalized Point Based Value Iteration for Interactive POMDPs Prashant Doshi Dept. of Computer Science and AI Institute University of Georgia

Euro-Par, HASTE: An Adaptive Middleware for Supporting Time-Critical Event Handling in Distributed Environments ICAC 2008 Conference June 2 nd,

Tabu Search Applications Outlines: 1.Application of Tabu Search 2.Our Project with Tabu Search: EACIIT analytics.

1 Item 3 - Research and Development of High Security Remote Authentication Technology Item 3 - Research and Development of High Security Remote Authentication.

A Knowledge-Based Tool for Planning of Military Operations: the Coalition Perspective Larry Ground Alexander Kott Ray Budd BBN Technologies Presented by.

ISM 206 Optimization Theory and Applications Fall 2011 Lecture 1: Introduction.

Keep the Adversary Guessing: Agent Security by Policy Randomization

Introduction to Linear Programs

Interconnected Distribution Strategy

PETRA 2014 An Interactive Learning and Adaptation Framework for Socially Assistive Robotics: An Interactive Reinforcement Learning Approach Konstantinos.

Markov Decision Processes

UAV Route Planning in Delay Tolerant Networks

Markov Decision Processes

Management Information Systems

CS 188: Artificial Intelligence Fall 2007

13. Acting under Uncertainty Wolfram Burgard and Bernhard Nebel

Markov Decision Problems

Hidden Markov Models (cont.) Markov Decision Processes

Managing the information systems function

Reinforcement Learning Dealing with Partial Observability

Management Information Systems

Management Information Systems

MSE 606A Engineering Operations Research

Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 7

Presentation transcript:

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Emmanuel Fernandez Associate Professor

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati INTERESTS Stochastic Models, Decision & Control Processes, Dynamic Programming Telecommunications Information Technology Operations & Logistics: Semiconductor fabs Basic Methodology Algorithms, Software Tools

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati OVERVIEW Phase 1: : Learning and Adaptive Systems, Models with Partial Information, Average Optimality Criteria. Phase 2: : Non-standard Optimality Criteria, Modeling Applications, Algorithms & Software Tools. Phase 3: 1998-Present: Risk-Sensitive Models, Security & Fault Management in Telecommunication Networks, Operational Methods in Semiconductor Manufacturing. Over 61 refereed publications(6 b, 18+ j, 37 c) Four Ph.D.s, 3 M.Sc., 18+ undergrad. RA’s. Honors: –Tau Beta Pi Professor of the Year, David Rist Prize MORS, IEEE Life Member Fund Research Initiation Award (Eng. Foundation).

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati OUTLINE Motivation: Applications –Semiconductor manufacturing operations –Logistics –Information Networks oFault & Security Management in communication networks oRouting in the Intelligent Network Stochastic Decision & Control Models: Optimality Criteria: Why Risk-Sensitivity? Basic Research Risk sensitive results: –Optimality equations & the Vanishing Discount Approach (AC). –Modular functions & structured policies (DC).

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati APPLICATIONS Semiconductor Manufacturing: –Capacity expansion & allocation, –Preventive maintenance scheduling (AMD). Information Networks: –Routing in the Intelligent Network (AT&T); –Security & fault management.. Operations & Logistics: –Workforce management; –Scheduling military training resources (Army).

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Semiconductor Manufacturing: Capacity Expansion & Allocation NSF/SRC Project at U. Maryland (PI’s: M. Fu & S. Marcus) EF Sabbatical project (begun Fall 98) EF liaison with industry (AMD) during 99 Integrate transient product dynamics over entire fab life cycle: Markov Decision Process (MDP) models –allocating/adding tool and process capacity –dynamic uncertain demands (e.g., market shifts) –transient dynamics (e.g., technology shrinks/shifts) Computational Investigation & Cost Modeling Tool: SYSCODE (University of Arizona software) –Stochastic Systems Control and Decision Algorithms Software Laboratory Find optimal policy for different parameters : –demand distribution –inventory cost and/or backlogging cost Simple policies vs. optimal policy Infinite horizon results vs. finite horizon A Markov Decision Process Model for Capacity Expansion and Allocation: IEEE Conf. Decision & Control, 1999.

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Industry Interaction: Advanced Micro Devices Joint effort UA & ISR On-site visits Preventive maintenance –Within allowed window, when to do PM? Information Technology: –“Torrents” of information! –Inefficient “manual” methods –Do not use available information –No models Develop basic models & solution SRC/ISMT

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Information Technology & Telecommunication Networks Routing calls in the Intelligent Network Security and Fault Management Software and Web tools: –SYSCODE –Computations & MATLAB Web course.

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati The Intelligent Network: Routing Toll-free Calls (AT&T) AT&T - UA project Route 800- traffic to call centers State information: –Workload at call centers –Incomplete information –Periodic updates Solution: –POMDP model –Heuristic Policy Iteration Algorithm R. Milito & E. Fernandez: (a) IEEE TAC 1995, (b) IEEE Conf. Decision & Control 1995

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Information Networks: Security and Fault Management Joint project with M. Shayman, U. Maryland. Searching for faults in a given domain: –Scheduling tests Single/Multiple faults Test sequence constraints Risk-sensitive criterion Interchange argument: –Explicit scheduling rules Qualitative analysis Security intrusions: –Similar to fault management 1999 Allerton Conference IEEE TAC 2001 Proposals

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Operations & Logistics: Scheduling Army Training Resources LTC M. McGinnis: Ph.D. UA Thousands of recruits/year Many installations/bases Decisions: –Company size –Length of training period –Number of companies to activate/retire each week. Model: Inventory-type Solution: Heuristic Policy Iteration Algorithm Decision support software (in use by Army). Journal Military Op. Res (Winner of David Rist Prize)

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Operations & Logistics: Scheduling Army Training Resources

Emmanuel Fernandez ECECS Dept. Univ. Cincinnati Logistics: Workforce Management Recruit-retain-dismiss individuals Intrinsic individual’s potential –Unobservable state Random productivity –Bayesian stochastic model The firm’s lifetime is long: –Average cost criterion Adaptive control through Bayesian learning Qualitative analysis of case studies Fdez, Jain, Lee, Rao, Rao: Management Science 1995.