1. Copyright © GENIVI Alliance 2014
GENIVI Lifecycle Webinar
09th Dec 2014
David Yates
GENIVI System Architect and Lifecycle Topic Owner
Continental Automotive Gmbh
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
)GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

2. Copyright © GENIVI Alliance 2014
9/22/2018
Scope of Presentation
The aim of this presentation is to provide an overview of the Lifecycle architecture within GENIVI detailing where we believe the Automotive world requires extensions to existing open source solutions.
The following topics will be covered:
Welcome & Introduction
Lifecycle Domain Overview
Component Overview
Startup/Shutdown Concept
Introduction to NSM (session management)
Introduction to Resource Management
Roadmap
Location of further information (AMM presentations)
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18
David Yates

3. Lifecycle Overview Supply Management Node State Management
1*: Get internal states
State change notification
Events:
Phone session
Diag,SWL,Coding session
State change protocol
(register for shutdown, get states,handshake for state changes)
Ctrls
State chart
Plug in for:
ADC,
PMIC
State chart
Supply Management
Plug in for power handling
HMI, Phone,…
SWL/Update
Diagnostics
Node State Management
Reaction on conditions
Events:
Good
Poor
Bad
Turn off display, drives, mute audio,…
State chart 1*
Plug in for:
Sensors, Devices
Events:
Full operational
Error startup
Get states
Node observing for CPU load, memory, appl. crash
Thermal Management
Node
Resource
config
Reaction on conditions
Set LUC
Last-User-Context
Turn on fan, reduce audio volume,…
Resource Management
Boot config
Plug in for:
Wakeup reason, node / vehicle network
Boot Management
Power Management
Config
22-Sep-18

4. Lifecycle Manifest Node State Manager Node State Machine
Package
Product Component
Node State Manager
Node State Machine
Node Startup Controller
Platform Component
Supply
Manager
systemd
Supply Management
Node State Management
Boot Management
Thermal Management
Power Management
Resource Management
cgroup service
Thermal
Manager
Node Health Monitor
Power Event Collector
Node Resource Mgr
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

5. Startup/Shutdown Management
takes care about
Boot Management
Startup Management
takes care about
Node State Management
Shutdown Management
Why do we have this split?
systemd stops and unloads all components during its shutdown concept. This requires a lot of time to make them functional again in the event of a cancel shutdown event being received
An IVI system must be able to resume operation without losing any context and without the need for a reboot. Therefore Node State Management will only call registered consumers in the shutdown phase. This event notification will drive the components into a stable state and persistent data written to disk. The processes are not killed or unloaded from memory.
Traditional shutdown will still be required for some legacy components and system services (filesystems, etc.) Therefore, once registered components have been notified and have gone into their quiet state, systemd will be used to shut down selected components, unmount filesystems and halt the system.
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

6. Shutdown preparation in Startup Phase
Before systemd
Runlevel replacement
kernel
GENIVI extensions
initrd
Start NSM via systemd
Mandatory targets (Base System & Early Features)
Node State
Manager A B C
BASE_RUNNING
focussed.target
(last user context)
(during NSC init)
Registration of Shutdown Consumers
LUC_RUNNING
unfocussed.target(s)
FULLY_RUNNING
lazy.target J
FULLY_ OPERATIONAL
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

7. Shutdown Execution Node State Manager app1.service app2.service
Consumer I
Consumer H
Writing LUC
Node Startup
Controller
app1.service
Consumer G
systemd
Consumer F
Consumer E
Calling registered Shutdown Consumers in reverse order
Node Startup
Controller
app2.service
Consumer D
systemd
Consumer C
Writing LUC
Consumer B
Enables:
Shutdown activities are trigger able without unloading the components.
Legacy components can be shut down in their traditional way.
Full flexibility on where to integrate systemd based shutdown units.
Consumer A
Node State
Machine
Unmount FFS
NodeState = NsmNodeState_Shutdown
Turn power off
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

8. Use Cases Vehicle Node State Machine Node State Manager Data Phone
PhoneSession
SWLSession
…..
set method
Thermal
Mgmt ….
events/data
SWL
request system restart
Supply
Mgmt ….
LucRunning
FullyOperational ….
Audio
Signal
HMI
Vehicle
Bus ….
events/data
Shutdown
Phone
Audio
HMI
Navigation
Navigation
lifecycle requests
22-Sep-18

9. Resource Management - Goals
Resource management contains the functionality to ensure that the node runs in a stable and defined manner.
To do this, it will monitor and limit different aspects of SW component behavior including system resources (i.e. CPU load and memory) and critical run-time observation.
Resource allocation will be configurable on a component basis through the use of cgroups.
Health management will provide a configurable escalation strategy defining actions to be taken in the case of system failures.
The Resource Management component does not aim to implement access control. Access control strategy must be implemented when setting the final system architecture.
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

10. forward NHM heartbeat externally or to internal HW Watchdog
Health Management
Health Management will ensure that the node runs in a stable and defined manner. To do this it is planned to have the following multi layered observation system and escalation strategy:
register failure & attempt recovery
Platform components
and integrated
applications
monitoring of userland
Platform components
and integrated
applications
Boot
Management
request app/service restart
Recovery Clients
Recovery Clients
notify alive
start/ restart
execute recovery
request node restart
NHM
NSM
start/ restart
notify alive
systemd
notify alive
/dev/watchdog
forward NHM heartbeat externally or to internal HW Watchdog
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

11. Concepts for the System Health Management - NHM
The Node Health Monitor will work in conjunction with systemd to monitor component failures in the system. It will be responsible for :
monitoring systemd to automatically record and track failures per component (i.e. application, service)
providing an interface with which components can register failures when not using the systemd monitoring
maintaining failure statistics over multiple lifecycles for the system and components
the service name will be used to identify and track component failures
statistics on number of failures in number of lifecycles will be maintained (i.e. 3 failures in last 32 lifecycles)
monitoring the wakeup and shutdown events to catch unexpected system restarts
provide an interface for components to read system and component error counts
provide an interface for recovery clients to request a node restart
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

12. Concepts for the System Health Management – NHM cont..
Additionally the Node Health Monitor will test a number of product defined criteria with the aim to ensure that userland is stable and functional. For instance it will be able to validate that :
there is enough free system memory
the CPU is not reporting an excessively high load for a sustained period
defined file accessibility is possible
defined processes are still running
communication is possible (D-Bus)
a user defined process can be executed with an expected result
If the NHM believes that there is an issue with user land then it will be capable to initiate a system restart
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

13. Concepts for the System Health Management – Recovery Client
A “Recovery Client” is a component that is executed when a failure has been detected in the system. There can be a one to one relationship between apps and recovery clients or one client can handle multiple apps. It should contain enough functionality to be able to :
request the error status count from the NHM (based on name of the service file failing)
based on the error count, escalate the recovery action, for instance:
file system mount failure, recovery action could be to format the file system and request a node restart
if it is an application that has failed multiple times then we may want to delete that applications persistency data and restart the application
when possible, request that the SW is uninstalled or rolled back to a previous version
request systemd to restart the application
request a node restart via the NHM
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

14. Resource Management - Goals
Resource management contains the functionality to ensure that the node runs in a stable and defined manner.
To do this, it will monitor and limit different aspects of SW component behavior including system resources (i.e. CPU load and memory) and critical run-time observation.
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

15. Resource Management Node State Mgmt Resource Management P3 22-Sep-18
<<refine>>
<<refine>>
<<refine>>
cgroups
(access via sysfs or systemd wrapper)
Node Resource Manager
Node State Manager
systemd
Starts services
Configure cgroups
Control system resources
Report/Handle resource allocation errors
Monitor system resources
Kill resource abusers (policy dependent)
Evaluate node restart requests
Handle node restart requests
Application Component
Supply Control Logic P3
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

16. NRM Interfaces
The Node Resource Manager (NRM) provides the following D-Bus interface
org.genivi.NodeResourceManager.Info
which contains the following methods:
GetMemoryStatus -> provides the memory status of a particular cgroup or of the complete node
GetProcessorUsage -> read the current CPU usage for either a particular cgroup or for the complete node
and the following two signals
CgroupMemoryStatus -> triggers when the memory status of a particular cgroup is changed. The client can register for notification about a specific cgroup through the use of the GroupName parameter
NodeMemoryStatus -> triggered when the memory status of the node is changed
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

17. Status and Roadmap Horizon Apr 2014 Intrepid Oct 2014 Jupiter Apr 2015
Kronos
Oct 2015
Cgroup
(Kernel)
Adopted comp., provided by the OSS community
specific
specific
specific
specific
systemd
GENIVI funded OSS component (implemented by Codethink)
specific
specific
specific
specific
Node Startup Controller
Node
State Manager
OSS Component (implemented and maintained by Continental)
specific
specific
specific
specific
Node
State Machine
n/a
n/a
n/a
n/a
Product specific library
Node
Resource Mgr
OSS Release upcoming (implemented by Continental)
placeholder
placeholder
abstract
specific
Node
Health Monitor
OSS Component (implemented and maintained by Continental)
abstract
abstract
specific
specific
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

18. Links Link to project pages :
Links to git repositories :
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18

19. Copyright © GENIVI Alliance 2014
Questions
Thanks for your time and attention. Any questions??
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 (CC BY-SA 4.0)
GENIVI is a registered trademark of the GENIVI Alliance in the USA and other countries
Copyright © GENIVI Alliance 2014
22-Sep-18