Download presentation
Presentation is loading. Please wait.
1
24 February 2015 Ryota Mibu, NEC
Fault Management (Doctor) 24 February 2015 Ryota Mibu, NEC
2
Questions How big are the telco systems?
How many applications and VMs will be run on NFV platform? How many physical machines will be deployed and managed? How could you manage faults to keep all critical services up? 24 February 2015 OPNFV Prague Hackfest
3
Use Case 24 February 2015 OPNFV Prague Hackfest
4
High Level Architecture
Applications VIM User and Administrator App App App Virtualized Infrastructure Virtualized Infrastructure Manager (VIM) = OpenStack Virtual Compute Virtual Storage Virtual Network Virtualization Layer Hardware Resources 24 February 2015 OPNFV Prague Hackfest
5
Fault Management Sequence
Detection Reaction Applications VIM User and Administrator App App App Virtualized Infrastructure Virtualized Infrastructure Manager (VIM) = OpenStack Virtual Compute Virtual Storage Virtual Network Virtualization Layer Hardware Resources Doctor Initial Focus 24 February 2015 OPNFV Prague Hackfest
6
Key Requirements as VIM
Immediate notification to VIM user and administrator Fault notification of affected virtual resources Configurable notification by VIM admin and user Catch all faults in NFVI 24 February 2015 OPNFV Prague Hackfest
7
TO-BE: Functional Blocks
Applications VIM User and Administrator App App App Virtualized Infrastructure VIM Virtual Compute Virtual Storage Virtual Network Controller Notifier Virtualization Layer Monitor Inspector Hardware Resources 24 February 2015 OPNFV Prague Hackfest
8
Fault Management Scenarios (1/2)
Applications 0. Set Alarm User-side Manager Admin-side Manager 6-. Action 5. Notify Error Virtualized Infrastructure 4. Notify all Controller Controller Notifier Controller Resource Map Alarm Conf. 3. Update State 2. Find Affected 4. (alt) Notify Monitor Monitor Inspector Monitor Failure Policy 1. Raw Failure 24 February 2015 OPNFV Prague Hackfest
9
Fault Management Scenarios (2/2)
Applications User-side Manager Admin-side Manager 0. Set Alarm 6-. Action 5. Notify Error Virtualized Infrastructure 4. Notify all Controller Controller Notifier Controller Resource Map Alarm Conf. 3. Update State 2. Find Affected 4. (alt) Notify Monitor Monitor Inspector Monitor Failure Policy 1. Raw Failure 24 February 2015 OPNFV Prague Hackfest
10
AS-IS: OpenStack Juno (1/3)
How can you find faults as a tenant user? Keep-a-live check to each VM Polling VM state to Nova API Set alarm on metering service 24 February 2015 OPNFV Prague Hackfest
11
AS-IS: OpenStack Juno (2/3)
How does the metering service work? Resource controller such as Nova monitors usage of resource [Periodically] Get samples from resource controller and register them to DB [Periodically] Evaluate alarm definition on samples [Periodically] Raise alarm depend on result of the evaluation VM Hypervisor Nova Ceilometer (Heat) 1. 2. 4. 3. Machine Samples 24 February 2015 OPNFV Prague Hackfest
12
AS-IS: OpenStack Juno (3/3)
Notification OpenStack components post events to messaging queue Ceilometer collects, transform and publish those events which can be used for billing NFVI Nova Queue Neutron Ceilometer (Billing) Cinder Samples 24 February 2015 OPNFV Prague Hackfest
13
Implementation Plan in OpenStack
Applications VIM User and Administrator Virtualized Infrastructure Queue Alarm Set Notifier Nova Immediate Notificatioin Error Injection Zabbix Inspector Plugin ? 24 February 2015 OPNFV Prague Hackfest
14
Demo (1/3) User Scenario Public Net Private Net Web Server
HTTP Clients HTTP Clients Load Balancer Web Server HTTP Clients Web Server Launch New VM Public Net Private Net 24 February 2015 OPNFV Prague Hackfest
15
Demo (2/3) Demo 1 Demo 2 3. Create New Web Server VM Ceilometer
Hypervisor Nova Agent Alarm (Heat) Machine 1. Collect CPU time samples 2. Alarm Heat if CPU runtime = 0 Samples VM Ceilometer Hypervisor Nova Agent Alarm (Heat) 2. Notify as Event Machine 1. Hook 3. Alarm Heat 24 February 2015 OPNFV Prague Hackfest
16
Scenario 1: Ceilometer Normal
Demo 1 Demo 2 90 sec 26 sec 24 February 2015 OPNFV Prague Hackfest
17
Backup 24 February 2015 OPNFV Prague Hackfest
18
Fault Management Sequence (Optional)
Detection Reaction Applications VIM User and Administrator App App App Virtualized Infrastructure Virtualized Infrastructure Manager (VIM) = OpenStack Auto Reaction Virtual Compute Virtual Storage Virtual Network Virtualization Layer Hardware Resources 24 February 2015 OPNFV Prague Hackfest
19
Fault Management Scenarios (Optional)
Applications 0. Set Alarm User-side Manager Admin-side Manager 5. Notify Error 6-. Action Auto Reaction Virtualized Infrastructure 4. Notify all Controller Controller Notifier Controller Resource Map Alarm Conf. 3. Update State 2. Find Affected 4. (alt) Notify Monitor Monitor Inspector Monitor Failure Policy 1. Raw Failure 24 February 2015 OPNFV Prague Hackfest
Similar presentations
