Research Challenges of Autonomic Computing Author: Jeffrey O. Kephart Presented by: Djuradj Babich

Research Challenges of Autonomic Computing Introduction Characteristics of IT environments: Complex Heterogeneous (hardware, middleware, software) Difficult to integrate, install, configure, tune, and maintain Within few years, will become impossible to administer 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Complexity Solution Systems comprised of self-managing components. Self: Configuring, Healing, Optimizing, Protecting Managing done in accordance with high-level objectives specified by humans. 11/27/2018 Research Challenges of Autonomic Computing

The Purpose of The Paper Introduce IBM’s Autonomic Computing Initiative (ACI). Decompose the grand challenge of autonomic computing into several of its constituent scientific and engineering challenges. Provide pointers to initial efforts to address these challenges. 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Objectives Describe a research framework Present a subset of challenges: Autonomic Element Challenges Autonomic System Challenges Human-Computer Challenges Conclusion 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Research Framework Helpful in defining, describing, and growing IBM’s current autonomic computing research program Research space divided into 3 parts: Autonomic Elements Autonomic Systems Human-Computer Interactions 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Autonomic Elements Basic building blocks of autonomic systems Their mutual interactions produce self-managing behavior of autonomic systems. Three sub-branches distinguished within: Specific Autonomic Elements Generic Autonomic Element Technologies Generic Autonomic Element Architectures, Tools, and Prototypes. 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Autonomic Systems Composed of autonomic elements Three sub-branches distinguished within: Autonomic System Technologies Autonomic System Architectures and Prototypes Autonomic System Science 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Human Interaction Managing done in accordance with high-level objectives specified by humans. Two sub-branches distinguished within: Human Studies Policy 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Objectives Describe a research framework Present a subset of challenges: Autonomic Element Challenges Autonomic System Challenges Human-Computer Challenges Conclusion 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Challenges Specific Autonomic Elements Improving the self-managing capability of specific components Generic Autonomic Element Technologies Applicable technologies (monitoring, forecasting, event correlation, rule execution, optimization, planning) Generic Autonomic Element Architectures, Tools, and Prototypes. Internal structure, helpful tools, and implementations 11/27/2018 Research Challenges of Autonomic Computing

Specific Autonomic Elements More self-managing components Emphasis: servers, DBMS, storage systems Challenges numerous, but well-understood New challenge introduced by IBM’s ACI: Achieve effective interoperation by the autonomic elements 11/27/2018 Research Challenges of Autonomic Computing

Element Interoperation Elements situated in autonomic systems Cooperative intercommunication and interaction between elements essential Requirements: Standard interface (system architecture) Ability to generate and supply needed info to other components Ability to request and use info from other components 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Challenges Specific Autonomic Elements Improving the self-managing capability of specific components Generic Autonomic Element Technologies Applicable technologies (monitoring, forecasting, event correlation, rule execution, optimization, planning) Generic Autonomic Element Architectures, Tools, and Prototypes. Internal structure, helpful tools, and implementations 11/27/2018 Research Challenges of Autonomic Computing

Monitoring and Analysis Common methods for collecting and representing monitored data and log files. Challenge: standardization Solution effort: Common Base Event (OASIS Web Services Distributed Management Technical Committee) Rule and correlation engines for analyzing monitored data and log files. Challenge: set of rules and/or correlation expressions for condition description Complication: correlation across multiple components Solution effort: modeling, machine learning 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Event Correlation Correlation extraction between low-level system measurements and high-level Service Level Objectives is based on modeling: Map potential actions into probable outcomes Forecast future demand and plan accordingly Access models of other system components Challenges: Learn and readjust the models continually on the fly. Adjust quickly to observations with a minimum of data and training time. Adapt to conditions that are noisy and prone to fluctuations. 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Rule Execution Rule authoring should be based on some form of machine learning, based on system-level goals coupled with historical observation Challenges: Random exploration Several hundred tunable parameters Learning process convergence 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Optimization Optimization challenges overlap a good deal with those of learning. Challenge: Nonstationarity Landscape changes due to effects such as: Changing workload Adaptive behavior by other autonomic elements 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Planning Autonomic elements must conduct their planning in open environments that are not completely understood. Challenge: Need for planning techniques that handle incomplete domain specification Need for continual assessment of plan completion progress Need for generating and manipulating plan metadata 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Challenges Specific Autonomic Elements Improving the self-managing capability of specific components Generic Autonomic Element Technologies Applicable technologies (monitoring, forecasting, event correlation, rule execution, optimization, planning) Generic Autonomic Element Architectures, Tools, and Prototypes. Internal structure, helpful tools, and implementations 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Architecture Support the need to manage and coordinate multiple threads of execution within the element. Example: single task of honoring the terms of a service agreement with another element might entail: Monitoring the variables of interest Analyzing the extent to which they are being kept within agreed-upon bounds Invoking a planner to determine a sequence of actions to improve performance Use monitored data to improve internal model Revise forecast of future workload Revise model upon which the forecast is based 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Architecture (cont.) Challenge: Develop an architecture (and functionality) for detecting and resolving conflicts that are bound to occur when an autonomic element cannot meet all of its obligations. Example: Autonomic Database Element (Optimizer/Planner conflict) 11/27/2018 Research Challenges of Autonomic Computing

Autonomic Element Tools and Prototypes Tools provide a set of autonomic element technologies, means for interconnecting them in accordance to architecture, and interfaces that permit appropriate interaction between elements. Building and refining prototypes is essential for developing architectures and tools for autonomic elements. Effort to offer some of the services: IBM’s Autonomic Computing toolkit 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Objectives Describe a research framework Present a subset of challenges: Autonomic Element Challenges Autonomic System Challenges Human-Computer Challenges Conclusion 11/27/2018 Research Challenges of Autonomic Computing

Autonomic System Challenges Autonomic System Technologies Generic technologies that entail interactions among multiple autonomic elements to achieve system level goals (problem determination and remediation, automated provisioning, installation, and configuration). Autonomic System Architectures and Prototypes System-level architectures that effectively govern interactions among autonomic elements. Autonomic System Science Fundamental science of large-scale autonomic computing systems (learning, stability, control, emergent behavior). 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Technologies Self-Configuration Self-Healing Self-Optimization Self-Protection All four entail interactions among multiple autonomic elements 11/27/2018 Research Challenges of Autonomic Computing

System Self-Configuration Challenges: A new autonomic element integration Automated deployment of large-scale application Solution Effort: Bootstrapping (registry interaction) Integration of capacity planning, use of human expert knowledge, and schedulling technologies that generate workflows describing installation and configuration choreography 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing System Self-Healing Monitoring and aggregation of component metrics Low-level events processed into higher-level assessments of the health Rule and correlation technologies Problem localization Probe signals (overlapping) Information theoretic techniques Problem remediation Database (symptom – action) Recursive micro-rebooting 11/27/2018 Research Challenges of Autonomic Computing

System Self-Optimization Workload management Automated provisioning Both based on objectives specified by humans Performance models Machine learning Challenge: coordination Incompatible goals and procedures 11/27/2018 Research Challenges of Autonomic Computing

System Self-Protection Monitor system for compliance to security policies Protect actively against threats as they occur Challenge: Intercommunication 11/27/2018 Research Challenges of Autonomic Computing

Autonomic System Challenges Autonomic System Technologies Generic technologies that entail interactions among multiple autonomic elements to achieve system level goals (problem determination and remediation, automated provisioning, installation, and configuration). Autonomic System Architectures and Prototypes System-level architectures that effectively govern interactions among autonomic elements. Autonomic System Science Fundamental science of large-scale autonomic computing systems (learning, stability, control, emergent behavior). 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing System Architecture Autonomic Elements must share Common behaviors Common interfaces Common interaction patterns 11/27/2018 Research Challenges of Autonomic Computing

Autonomic System Challenges Autonomic System Technologies Generic technologies that entail interactions among multiple autonomic elements to achieve system level goals (problem determination and remediation, automated provisioning, installation, and configuration). Autonomic System Architectures and Prototypes System-level architectures that effectively govern interactions among autonomic elements. Autonomic System Science Fundamental science of large-scale autonomic computing systems (learning, stability, control, emergent behavior). 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing System Science Control and exploitation of emergent behavior Economic mechanisms Auctions and bilateral negotiations Benchmarking 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Objectives Describe a research framework Present a subset of challenges: Autonomic Element Challenges Autonomic System Challenges Human-Computer Challenges Conclusion 11/27/2018 Research Challenges of Autonomic Computing

Interfacing with Humans Develop new languages and metaphors Specify objectives and goals in natural manner Sufficiently expressive/structured Intuitive interfaces 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Policies Forms of expressing objectives Low-level action-based High-level, based on: goals Utility functions Lingua franca Difficult to specify 11/27/2018 Research Challenges of Autonomic Computing

Research Challenges of Autonomic Computing Conclusion Autonomic computing presents a challenge Requires advances in several fields of science and technology An open-platform autonomic computing prototype needed to support research 11/27/2018 Research Challenges of Autonomic Computing