1. DataHub Product Overview Cover example with photo inside the shape
Insert new photo: Click on Insert photo-icon on slide and Insert new photo
Use Crop Tool under Picture Tools to pan and zoom
Beware: when you add or replace a picture, it may come to the front. Right click on it and select ‘Send to back’ to re-arrange it.
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
Please insert the other company’s logo in the same size as the Amadeus logo, with no background and reverse out colour
© Amadeus IT Group and its affiliates and subsidiaries
Jorge Jarrin; Lead Developer, AODB
Clinton Aurin; Solution Design Engineer
Amadeus Airport IT Americas, Inc. – 2017 User Conference

2. What is the DataHub and what motivated its development.
Major Features.
How does the system work?
Design Goals.
System Overview.
FAA Feeds.
Airline Feeds.
Airlines.
Other Data Sources.
© Amadeus IT Group and its affiliates and subsidiaries

3. DataHub: Definition and Motivation
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

4. DataHub: Definition and Motivation
Past strategy consists of interfacing directly with operational data sources.
Costly and time consuming to maintain as our software install base grows.
Linking different feeds (FAA and airlines) into one ‘hub’ provides useful aggregated data.
This system consumption and dissemination of operational data from a centralized location.
In a nutshell: It consumes data from N number of external sources, processes and persists data, and provides an enriched fusion of useful data.
© Amadeus IT Group and its affiliates and subsidiaries

5. 2.
Major Features
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

6. Major Features
Centralized processing of major airline and flight data sources (FAA).
Extremely fast on-ramping of new installation sites.
Centralized monitoring, and command and control.
High performance and scalable data processing.
Rapid addition of new input and output nodes.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

7. 3. How does it work? Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

8. How does it work?
Consuming input data: Raw data in any format from any transport.
Pre-processing data: Build unique keys (identifiers) from raw data.
Filtering data: What data is relevant for processing based on criteria.
Data normalization: Relevant information is reformed into a coherent model.
Data cache: Semi-persistent In-memory cache of data for fast queries.
Outbound data: Live subscription mechanism for interested consumers.
Dataset requests: Bulk requests from the central cache.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

9. How does it work?
Volatile high volume data: High throughput data feeds (5000+ transaction/s)
System administration: CLI based administration console.
System monitoring: Health and status of topologies deployed.
Permanent Storage: Traditional SQL database storage for reporting and BI.
Watchdog mechanisms: To monitor hung processes/nodes.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

10. 4. Design Goals Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

11. Design Goals
The system should be one thing about all others: FAST.
Typical transaction should not take more than 10 milliseconds.
Each component in the system has an error recovery strategy.
Horizontally scalable across N-number of nodes.
Utilization of HTTPS/SSL standards where available.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

12. 5. System Overview Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

13. Centralized Storage/Cache
System Overview
External Data Sources
Inbound Node
Inbound Node
Inbound Node
Inbound Data
Centralized Storage/Cache
Central Cache
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Outbound Node
Outbound Node
Data Delivery
External Consumer
External Consumer
© Amadeus IT Group and its affiliates and subsidiaries
Data Query
Dataset Servicer

14. Centralized Storage/Cache
System Overview
External Data Sources
Inbound Node
Inbound Node
Inbound Node
Inbound Data
Centralized Storage/Cache
Central Cache
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Outbound Node
Outbound Node
Data Delivery
External Consumer
External Consumer
© Amadeus IT Group and its affiliates and subsidiaries
Data Query
Dataset Servicer

15. System Overview Inbound Data Route Ingress Hash Filter Data Enrichment
Variety of transports.
VFS, JMS, CSV, etc.
Hot-cache mapping.
Used for records diffs.
Record aggregation and versioning.
Key Extraction
Data Extraction
Serialization
Unique identifying key for records.
Dynamic filtering.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Raw data to normalized data.
JSON format for storage in in-memory database.
© Amadeus IT Group and its affiliates and subsidiaries

16. Centralized Storage/Cache
System Overview
External Data Sources
Inbound Node
Inbound Node
Inbound Node
Inbound Data
Centralized Storage/Cache
Central Cache
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Outbound Node
Outbound Node
Data Delivery
External Consumer
External Consumer
© Amadeus IT Group and its affiliates and subsidiaries
Data Query
Dataset Servicer

17. System Overview Central Cache Inbound Pipe Key Calculation
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
Outbound Pipe

18. Centralized Storage/Cache
System Overview
External Data Sources
Inbound Node
Inbound Node
Inbound Node
Inbound Data
Centralized Storage/Cache
Central Cache
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Outbound Node
Outbound Node
Data Delivery
External Consumer
External Consumer
© Amadeus IT Group and its affiliates and subsidiaries
Data Query
Dataset Servicer

19. System Overview Data Delivery Subscriber Channels Arrival Channel
Departure Channel
Read Data from Channel
Fetch Data Payload from Key
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Invoke Subscriber Endpoint
© Amadeus IT Group and its affiliates and subsidiaries
VPN/External Network
ESB/Consumer

20. Centralized Storage/Cache
System Overview
External Data Sources
Inbound Node
Inbound Node
Inbound Node
Inbound Data
Centralized Storage/Cache
Central Cache
To update the text, just click on the text
More slides with this layout: Copy slide and paste
Outbound Node
Outbound Node
Data Delivery
External Consumer
External Consumer
© Amadeus IT Group and its affiliates and subsidiaries
Data Query
Dataset Servicer

21. System Overview
Data Query
Capable of limiting requests to prevent excessive resource utilization.
Capable of producing a set of keys to produce relevant data from the in- memory database.
Queries are limited by potential key-space in order to manage memory.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

22. System Overview Control mechanism: Command line interface (CLI)
Command and Control
Control mechanism: Command line interface (CLI)
Capability to dynamically start and stop routes.
Monitor the simple status of routes and runtime components.
Manage cluster priorities and route assignments.
Manage and modify subscription information.
Provide remote access to logs and error messages.
Provide basic data query mechanisms.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

23. System Overview
Monitoring and Alarming
See fine grained statistics per route per interface, including message/second metrics and error counts.
Defines alarms and alert outcomes based on the emitted metrics.
Provides a simple real-time graph to show system load and throughput.
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

24. System Overview Monitoring and Alarming
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

25. System Overview Monitoring and Alarming
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

26. 6. FAA Feeds Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

27. FAA Feeds Positional Track Updates (1/min.)
TFMS (Traffic Flow Management System)
Positional Track Updates (1/min.)
Position/Speed/Altitude (We Calculate Heading)
Airspace ETA, ATA, ATD
Callsign, Departure/Arrival Airport
Routes (Airspace Routes)
Flight Cancellations
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

28. FAA Feeds TFMS (Traffic Flow Management System): Example "metadata": {
"aircraftId": "JBU148",
"airlineCode": "JBU",
"arrivalAirport": "OAK",
"departureAirport": "LGB",
"flightRef": "LGB:OAK: ",
"operationTime": " ",
"source": "TFMS" },
"data": {
"id": "LGB:OAK: ",
"eta": " T15:04: ",
"lon": ,
"speed": "266",
"alt": "080",
"bearing": 0.0,
"timeAtPosition": " T14:13: ",
"type": "HCS_TRACK_MSG",
"lat": 33.95,
"airline": "JBU" }
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

29. FAA Feeds Same as TFMS, but Update 1/10 sec. Registrations
FDPS (Flight Data Publication Service)
Same as TFMS, but Update 1/10 sec.
Registrations
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

30. FAA Feeds FDPS (Flight Data Publication Service) : Example
"metadata": {
"aircraftId": "SWA1538",
"arrivalAirport": "KSNA",
"departureAirport": "KOAK",
"operationTime": " ",
"source": "FDPS" },
"data": {
"eta": " T14:21: ",
"ractCode": "B737",
"etd": " T13:10: ",
"registration": "N952WN",
"originDate": " ",
"atd": " T13:18: " }
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

31. FAA Feeds Update Rate 1/sec.
STDDS (SWIM Terminal Data Distribution System)
Update Rate 1/sec.
‘Spot Out’ – Ramp to Movement Area Transition
‘Spot In’ – Movement Area to Ramp Transition
ATA, ATD, Mode-S Address, Callsign, Aircraft Type
Runway Assignment
Wake Turbulence Category
Position/Speed/Heading/Altitude
Aircraft Type
RVR Status
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

32. FAA Feeds TRACON Precipitation (Graphical Data Run Length Encoded)
ITWS (Integrated Terminal Weather System)
TRACON Precipitation (Graphical Data Run Length Encoded)
To update the text, just click on the text
More slides with this layout: Copy slide and paste
© Amadeus IT Group and its affiliates and subsidiaries

33. 7. Airline Feeds Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

34. Airline Feeds Airline Name* Flight Number* STA* Registration* ETA*
Data Types
Airline Name*
STA*
ETA*
ATA
Onblock*
STD*
ETD*
ATD
Offblock*
Flight Number*
Registration*
Origin Date*
Aircraft Type*
Departure Airport*
Arrival Airport*
Arrival Leg Status
Departure Leg Status
© Amadeus IT Group and its affiliates and subsidiaries
* Preferred Data

35. Airline Feeds Departure Gate Codeshares Aircraft Sub Type Arrival Gate
Data Types (cont.)
Departure Gate
Arrival Gate
Departure Terminal
Arrival Terminal
Fleet ID
Departure Belt
Arrival Bag Claim
Trip Number
Codeshares
Aircraft Sub Type
Service Type
Departure Spot
Arrival Spot
Passenger Information
Weight Class
Estimated Touchdown
International Status
© Amadeus IT Group and its affiliates and subsidiaries

36. Airline Feeds Emaples "metadata": { "sta": "2017-04-08T00:31:00+0000",
"airline": "AS",
"arrivalAirport": "OAK",
"departureAirport": "SEA",
"originDate": " ",
"source": "AS",
"tripNumber": "538" },
"data": {
"aircraftSubType": "9WX",
"eta": " T00:31: ",
"tripNumber": "538",
"tailNumber": "303",
"aircraftType": "739",
"sta": " T00:31: ",
"etd": " T22:35: ",
"std": " T22:35: ",
"flightNumber": "AS538",
"registration": "N303AS",
"serviceType": "J",
"originDate": " ",
"internationalStatus": "D",
"departureAirport": "SEA",
"arrivalAirport": "OAK",
"airline": "AS",
"arrivalGate": "14",
"arrivalBelt": "1",
"arrivalSpot": "14" }
© Amadeus IT Group and its affiliates and subsidiaries

37. 8. Airlines Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

38. Airlines Airlines on DataHub
© Amadeus IT Group and its affiliates and subsidiaries

39. Airlines Airlines in the works
© Amadeus IT Group and its affiliates and subsidiaries

40. 9. Other Data Sources Divider example with no photo
You can change the colour to Amadeus blue, Light Blue and Pale Blue
To update the text, just click on the text
To get more slides with this layout just copy this slide and paste
© Amadeus IT Group and its affiliates and subsidiaries
© Amadeus IT Group and its affiliates and subsidiaries

41. Other Data Sources All Other Data Sources
© Amadeus IT Group and its affiliates and subsidiaries

42. Thank you! AmadeusITgroup amadeus.com amadeus.com/blog
© Amadeus IT Group and its affiliates and subsidiaries
amadeus.com amadeus.com/blog
You can follow us on:
AmadeusITgroup