1. Digital management of the railway sector through OSGi
TRACK project

2. What is IBBT?
IBBT introduction

3. IBBT Research

4. IBBT: Entrepreneurship

5. IBBT: Develop & Test

6. Information on an OSGi based product solution
TRACK framework

7. Problem description Depot Train A into depot for update
Train A from depot back to riding
Depot
Transfer dependent libraries
Update train A with software X

8. Problem description Depot Train B into depot for update
Train B from depot back to riding
Depot
Change upload files, update train B
Update train B with software X
Transfer dependent libraries
Library already present, blue library missing

9. Problem description Custom IO handler
Custom IO handler initiates connection
over technology of choice
Need diagnostics information from train…
Use 3rd party IO components
Custom IO handler
Diagnostics
Application

10. Problem description For every other application: custom IO handlers
preferred connection type

11. Solution – goal Functional requirements Extra requirements (research)
No wasted time spent in depot for updates
Immediate updates
Software version management
Extra requirements (research)
Modular system
Remotely accessible
Solve unreliable connections

12. GPSLocation 2.4.1 LogService 1.0.0 Timetables 8.5.0 GPSLocation 2.5.3
21/09/2018
GPSLocation
2.4.1
LogService
1.0.0
Timetables
8.5.0
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
MediaService
1.2.1
LogService
1.0.0
CateringInfo
3.5.0
Timetables
8.5.0
Our first goal is to provide support for applications, services and content for devices in the fleet.
In this example we have two trains with:
an application GPSLocation that periodically captures the GPS coördinates of the train
a service LogService that can be used by application to record log data
a content pack that contains the timetables for this train
Our support platform provides applications and services with a runtime environment and a number of supportive features such as life cycle management.
This platform must be able to support a wide range of devices, which can all run a different software configuration. E.g. an additional train type, which doesn’t have the GPS application, but runs a MediaService that streams video and has an application that provides catering info.
These devices can be connected to a backend server, but the available connections will range from GPRS to satellite and WiFi and can be unreliable depending on the situation.
Another supportive feature of the platform is then e.g. an event system that deals with these unreliable connectivity and allows the GPSLocation application to send the GPS coördinates to the backend management system.
21/09/2018

13. GPSLocation 2.4.1 LogService 1.0.0 Timetables 8.5.0 GPSLocation 2.5.3
21/09/2018
GPSLocation
2.4.1
LogService
1.0.0
Timetables
8.5.0
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
CCTVDriver
5.1.0
GPSLocation
2.4.1
Undeploying
2.5.3
Deploying
LogService
1.0.0
Deployed
Timetables
8.5.0
MediaService
1.2.1
CCTVDriver
5.1.0
GPSLocation
2.4.1
Undeploying
2.5.3
Deploying
LogService
1.0.0
Deployed
Timetables
8.5.0
MediaService
1.2.1
MediaService
1.2.1
LogService
1.0.0
CateringInfo
3.5.0
Timetables
8.5.0
MediaService
1.2.1
LogService
1.0.0
CateringInfo
3.5.0
Timetables
8.5.0
CCTVDriver
5.1.0
A second goal is a system that allows the remote management and deployment of application/services and content in an environment such as displayed here.
On the backend we then need a sort of registry that keeps track of the deployed application or content.
When a new application needs to be deployed (e.g. a CCTVDriver that captures the stream from a CCTV device and transfer the data back to the backend server), it is added to this registry. The remote deployment and management system is then able to deploy this application to a set of target devices in an intelligent way. In this example the new application is installed to all trains in a certain area.
21/09/2018

14. GPSLocation 2.4.1 LogService 1.0.0 Timetables 8.5.0 GPSLocation 2.5.3
21/09/2018
GPSLocation
2.4.1
LogService
1.0.0
Timetables
8.5.0
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
CCTVDriver
5.1.0
GPSLocation
2.4.1
Undeploying
2.5.3
Deploying
LogService
1.0.0
Deployed
Timetables
8.5.0
MediaService
1.2.1
CCTVDriver
5.1.0
GPSLocation
2.4.1
Undeploying
2.5.3
Deploying
LogService
1.0.0
Deployed
Timetables
8.5.0
MediaService
1.2.1
CCTVDriver
5.1.0
GPSLocation
2.4.1
Undeploying
2.5.3
Deploying
LogService
1.0.0
Deployed
Timetables
8.5.0
MediaService
1.2.1
MediaService
1.2.1
LogService
1.0.0
CateringInfo
3.5.0
Timetables
8.5.0
CCTVDriver
5.1.0
MediaService
1.2.1
LogService
1.0.0
CateringInfo
3.5.0
Timetables
8.5.0
The system should be flexible and be able to cope with new configuration on the fly.
E.g. when a new device is added, it should suffice to send a message to the backend upon activation, after which the necessary software can then be deployed and activated.
GPSLocation
2.5.3
LogService
1.0.0
Timetables
8.5.0
CCTVDriver
5.1.0
21/09/2018
21/09/2018 14

15. Overview Technical requirements Why OSGi? TRACK overview
Feature highlights
Challenges
Way forward

16. Technical requirements
Two types of customers
Railway stakeholders…
… but also …
TRACK
… 3rd party developers

17. Technical requirements
Very dynamic environments: unreliable connections

18. Technical requirements
Inter-node communication
Other research group:
network aspects
Remote communication (inter-node)

19. Technical requirements
Legacy application support: existing (native) applications

20. Why OSGi? Only Java runtime Modular by design Standard JAVA language
Dynamic
Platform independent !
Foto’s van more limited devices dan de typische trein.

21. TRACK overview: node layout
TRACK Container
The TRACK software architecture is centred around the concept of a generic TRACK node. This node will act as a uniform base for all participating devices (such as the trains and the backend of the previous slides).
To accommodate for the various device types, we’ve put an abstraction layer on to of the device specific hardware and operating system. On top of this we run the actual TRACK platform, which provides a runtime-environment for the application and content modules, called the TRACK container.
These modules can communicate with each other through a number of APIs of which the availability depends on the node configuration:
A basic node on a train can be limited to an implementation of the Core API, which provides the minimal platform features including communication and local management of the installed software.
The node on the backend server implements all APIs, including the deployment and management system.

22. TRACK overview: framework layout
21/09/2018
TRACK overview: framework layout
RST Infrastructure
Central System
Fleet
Node modules
System + Op. services
Node modules
TRACK Platform
TRACK Platform
TRACK Platform
Reliable link
WiFi/GPRS/3G
Node modules
TRACK Platform
A concrete deployment of this architecture can look like this. The backend server is the central system of the framework. 3rd party providers can connect to the central system to upload and manage their software.
Devices participating in this framework can include both railway stakeholder infrastructure (such as displays in station) and the actual fleet devices such as trains. All these devices run the basic TRACK platform which allows them to be managed from and communicate with the central system. Additional modules are then installed on top to provide the functionality that is specific to the device.
3rd party providers
21/09/2018

23. Feature highlights Fleet management Content management
Module deployment
Monitoring
Remote service discovery
Job scheduling
File transfer: BitTorrent
Vaadin UI
Legacy application support
Communication

24. Fleet management DeviceProfile Definition of device type/family
ID + properties definition
Can inherit from other DeviceProfiles
OPERATOR_X/train
hardwareVersion
A first important aspect is fleet management. When e.g. a new version of our GPSLocation application is to be deployed, we will need some mechanism to determine to which devices of the fleet the update applies.
The trivial solution is to maintain a list of all devices and their properties, but we chose an approach that is more scalable and easier to query.
We introduced an extra level of indirection in the form of a DeviceProfile, which defines a device type or a family of devices. E.g. a profile for all trains of a certain operator X. Profiles can define properties, e.g. a hardwareVersion.
Profiles can have inheritance relations, allowing a certain hierarchy in the model. E.g. the train profile of operator X inherits from a generic train profile which defines additional properties such the build year and operation area.
inherits from
GENERIC/train
buildYear, operationArea

25. Profile: OPERATOR_X/train
Fleet management
TrackNode
Represents a node in the framework
Name
Host (IP or ID)
Is assigned a DeviceProfile
TrainX
Profile: OPERATOR_X/train
buildYear
2000
operationArea
ParisNord
hardwareVersion
4.2
Our model is completed by a TrackNode object, which represents an actual device in fleet. A tracknode has a name (e.g. TrainX), an IP or logical address and is assigned a device profile, e.g. the train of operator x profile as used on the previous slide. The tracknode can then contain values for each of the properties that were defined in the profile.

26. Fleet management Querying
GENERIC/train (&(operationArea=ParisNord)(buildYear>1999))
All devices adhering to this profile or subtypes…
…which meet these conditions.
The goal is to perform queries on this model, e.g. we want to update all trains which operate in the north of Paris and are build after 1999, return me the applicable devices.
The query to do this is shown here and consists of a profile name (meaning we will look at all devices with this profile or a profile that inherits from this profile) and a filter expression in terms of the defined properties, which allows us to narrow down the results.

27. Content management TRACK module Application binaries or content
21/09/2018
Content management
TRACK module
Track Module
Application binaries or content
As shown in the introduction, we needed a component on the central server that allows us to manage the framework content and its state.
This content can range from applications to services and static content such as multimedia. The content management system stores the content as deployable artifacts called TRACK modules.
A TRACK module is an application binary or a collection of content, annotated with meta-data and packaged as a JAR or ZIP.
The meta-data contains the necessary information for the platform to perform its supportive features, e.g.:
Import-Package which specifies the dependencies
(Track-Module-Activation which specifies when the module should be activated)
Track-Module-Target which contains query for the fleet management and thus tells the platform to which devices this module should be deployed.
Meta-data
Packaged as JAR/ZIP
21/09/2018

28. Content management Versioned Item meta-data versions Version Version
21/09/2018
Content management
Versioned Item
versions
meta-data
Version
meta-data
Track Module
Version
meta-data
Track Module
The content management system adds structure to the TRACK modules by grouping all the different versions of a single application or piece of content in an object called Versioned item. The meta-data of a versioned item can hold additional info such as the organization that is responsible for the module, update strategy, etc…
The versioned item holds these versions in a Version object which contains the actual module and meta-data which includes the data from the manifest and additional data such as the deployment state of the module.
Version
meta-data
Track Module
21/09/2018

29. Module deployment Central System Content Manager Global Update Mgr.
21/09/2018
Module deployment
Central System
Content Manager
Global Update Mgr.
update request
INIT -> DEPLOYING
Fleet Manager
File Sharing
This leads us to the module deployment process.
e.g. When we deploy a new version of our GPSLocation application, the underlying state changes form its initial state to deploying.
This will trigger the GlobalUpdateManager to schedule an update job.
The update job first queries the FleetManager to obtain the target devices for the update. The module data is then shared using the FileSharing component. The update job will also register itself with the event system in order to receive notifications from the target devices.
Finally an update request is sent to each of the target devices. An update request contains information on the module that is updated and the operation, a deployment in this case.
Event System
21/09/2018

30. Target Node Local Update Mgr. Application Manager file transfer
21/09/2018
Target Node
Local Update Mgr.
Application Manager
file transfer
Download Manager
Update Handler
The update request is received by the LocalUpdateManager, which acts as an agent for the update system on all TRACK nodes.
The request is processed and a DownloadManager is called to begin the file transfer. During the transfer, the EventSystem can be used to send progress information. Note that the connection between the device and the central server is unreliable, so the DownloadManager needs to deal with transmission errors and connection drops.
When the download is complete, the module is sent to the appropiate UpdateHandler, which by default installs the module on the local platform using the ApplicationManager.
The LocalUpdateManager completes its work by sending feedback to the central server using the EventSystem.
Event System
21/09/2018

31. Module deployment Central System Content Manager Global Update Mgr.
21/09/2018
Module deployment
Central System
Content Manager
Global Update Mgr.
Fleet Manager
File Sharing
The feedback is delivered to the update job running on the GlobalUpdateManager, which then completes the update by stopping the sharing of the module file and by setting the state of the module in the ContentManager to DEPLOYED.
Event System
21/09/2018

32. Module updates Central System Target Node Δ DiffGenerator
21/09/2018
Module updates
Central System
Target Node
DiffGenerator
Global Update Mgr.
Local Update Mgr.
DiffBuilder
version info?
1.2
generateDiff 1.2->1.4 Δ
The update system also allows modules to be updated differentially. By only sending the files that are new or were changed, we can drastically reduce the network traffic for large updates. A typical example is a multimedia library of several gigabytes to which new content is added from time to time.
To perform this type of updates, we require an additional communication step between the central server and the device that needs to be updated. This gives us information on the current version of the installed module.
A component called the DifferenceGenerator is then contacted to generate the difference or module delta. This delta contains the newly added and changed files and a build script.
The target node then downloads this delta instead of the complete module and contacts the DifferenceBuilder to build the new module version. This is done by applying the build script to the installed module version using the files supplied with the delta.
Archive containing:
changed/new files
build script
buildModule
Track Module
21/09/2018

33. Monitoring: job status
21/09/2018
Monitoring: job status
The last feature we’d like to mention is the Monitoring component which gives a detailed view on the progress of the update operations. This component is available through the TRACK web application, but also as a Web service.
21/09/2018

34. Monitoring: remote OSGi console
21/09/2018
Monitoring: remote OSGi console
Finally, the web application also allows to open a console session to each connected device in order to check its status (e.g. a list of the installed and running applications)..
21/09/2018

35. Remote service discovery
Custom solution for OSGi 4.1 (no RSA available): R-OSGi
Producer
Node A
Node B
Consumer
Remoting is transparent
TRACK framework
TRACK framewrok
Register service
Expose service
Generate proxy
Automaticly adding remote services locally
Transparent as if local service
R-OSGi
R-OSGi
SLP
SLP
Discovery: advertise
R-OSGi specific communication

36. Job scheduling Updates are scheduled as jobs Concurrent update jobs
Scheduler allows
Prioritization of individual jobs
Pauze / resume of individual jobs
Eventueel slides Lander

37. File transfer: BitTorrent
Downloads are divided in small parts
Clients download small parts in random order
Client stitches everything together
Unreliability solved:
In case of failure, only small piece lost
Immediately start downloading piece from other source

38. Vaadin UI Webpage UI done with vaadin
Modularity of OSGi bundles visible in UI
Easily integrated with the webserver (Jetty) hosted in OSGi
All fleet and content interfaces done in Vaadin

39. Legacy application support
Installing, uninstalling, updating …
Custom update handler
TRACK container
External System
Controller
install
uninstall
Update Mechanism
updatehandler
Track Module
Extension possible
TRACK update handler

40. Legacy application support
Life cycle management: start, stop …
OSGi
TRACK Container
External legacy application
MyApplication
EntryPoint
JVM
JNI
bridge OS
Implement Application interface (TRACK Activator)
through JNI (Java Native Interface)

41. Challenges Remote communication  R-OSGi
(Common) external libraries in OSGi context
Simple: wrap as OSGi bundle
Nasty: fix class loading issues (class loaders…)
Last resort: * DynamicImport-Package *
OSGi 4.x (.1 of .0) geen Remote Services API dus zelf toevoegen met behulp van R-OSGi| Transparantie services of ze nu remote of lokaal zijn (zelf geimplementeerd)

42. Way forward (RAILS) Decentralizing the current framework
Allows for more robustness, server may fail, others can resolve issues. => No single point of failure
Replace R-OSGi with Remote Service Admin
Own implementation: focus on open standards => Better support for 3rd party
Discovery: JGroups
Service communication: HTTP/JSON
Question

43. Thank you for your attention
Questions?