1. Kernel HW KABI Tools Tony Camuso Aug 20, 2015 INTERNAL ONLY
EXERCISE - STEP 1
INTRO:
I'm here today to talk to you about the value of having a relationship with Red Hat.
[ Whiteboard Exercise ]
Before we get started, I want to capture a few thoughts you may have about what Red Hat is today. Possibly some of the following words:
[ Write examples in a single column as you say them ]
Linux
Open source
Jboss
Low-cost
Performance
Freedom
This is all true, and it's part of what of we offer, but but what we want to talk to you about today is something more compelling that Red Hat has to offer from a value and relationship perspective.
INTERNAL ONLY

2. What's the problem?
No easy way to determine KABI dependencies kernel wide.
Need to view the dependency tree from exported symbols down to all the compound data types they depend on.
Need to view the dependency tree from compound data types up to their dependent exported symbols.
Need to determine if changes to a subordinate compound data type can affect a whitelisted symbol.
[ EXERCISE - STEP 2 ]
But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange.
[PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ]
Performance
Mission critical
Brainstorming
Our best minds, their best minds together
Technology innovation
Good interoperability
Advising us the best way to use it
We can count on
Leader
Support
Extremely strategic
What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies.
It's about working closely with a trusted advisor to understand technology and the impact in your organization.
About creating a relationship that combines your expertise and Red Hat's knowledge and leadership.
Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer.
Internal Only

3. Proposed Solution
Create a database of all exported symbols and their dependencies.
Create the database as part of the build process or independently as an ancillary process.
Create a tool to query the data base for exported symbols and the data structures in their hierarchy.
[ EXERCISE - STEP 2 ]
But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange.
[PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ]
Performance
Mission critical
Brainstorming
Our best minds, their best minds together
Technology innovation
Good interoperability
Advising us the best way to use it
We can count on
Leader
Support
Extremely strategic
What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies.
It's about working closely with a trusted advisor to understand technology and the impact in your organization.
About creating a relationship that combines your expertise and Red Hat's knowledge and leadership.
Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer.
Internal Only

4. Proposed Solution (cont)
Uses preprocessor output files, which include all headers with all typedef and macro expansions for each corresponding source file.
The generated data files comprise a comprehensive graph of all exported symbols and all their dependencies.
Internal Only

5. Why use THIS tool?
Able to relate lowest level data structures to any exported symbols that depend on them.
Can detail the entire dependency tree of any exported symbol or any data structure.
Provides command line options to limit search scope to directories, files, and the kabi whitelists.
Provides a verbose option to display entire hierarchies.
Imposes a 'dups' policy to limit output when encountering duplications.
Internal Only

6. Internal Only

7. Internal Only

8. Show exported symbols affected by data structure
-slw : struct, limited to whitelist, whole word search (exact match)
-m fs/bio : limit search to path having substring “fs/bio”
Internal Only

9. Another struct search
Show exported symbols affected by structs with 'struct device' as part of their name.
Limit search to files and directories with 'pci' substring, and limit output to first 20 lines.
Search for declaration of 'struct class' and grep for 'struct device'.
Internal Only

10. Search for exported symbol
Search for exported 'struct bio_set' and see its top level dependencies.
See if 'struct bio_set' is in the kabi whitelist.
Internal Only

11. Find whitelisted function pci_iomap
Then show its dependency tree
Internal Only

12. RHEL7 Kernel Compile Times
System
Without kabi-parser
With kabi-parser
% change
intel workstation
core 2.8GHz
4 cores (8 HT)
16 G DDR3
15:49 (15.82)
17:07 (17.12)
8.2
amd dinar
opteron 1.4GHz
32 cores
32 G DDR3
4:25 (4.42)
4:57 (4.95)
12.1
Internal Only

13. Search Times
Substring search for popular struct (generates lots of output)
Exact match search for same takes about 2 seconds less.
Internal Only

14. Search Times (cont)
Longest search times are typically less than a minute.
At this time, searches are disk intensive, so cpu cores and memory have little impact on performance.
SSD should improve search times considerably.
Slurping the whole database for each search is impractical, though it might be useful for an interactive version of the tool.
Internal Only

15. Improving Search Performance
Under consideration
Second level database that maps symbols to the files that contain them.
Threaded search
Memory mapped database files
Internal Only

16. Implementation kabi-parser
Uses sparse semantic parser to tokenize precompiled sources that contain exported symbols.
Used by kernel make for source checking (C=1, C=2)
Maintained upstream.
Uses boost to serialize the sparse output for precompiled source files.
Invoked by kernel make for each source file.
makei.sh and kabi-data.sh are scripts for creating the precompiled sources and data files outside of the kernel build.
[ EXERCISE - STEP 2 ]
But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange.
[PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ]
Performance
Mission critical
Brainstorming
Our best minds, their best minds together
Technology innovation
Good interoperability
Advising us the best way to use it
We can count on
Leader
Support
Extremely strategic
What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies.
It's about working closely with a trusted advisor to understand technology and the impact in your organization.
About creating a relationship that combines your expertise and Red Hat's knowledge and leadership.
Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer.
Internal Only

17. Implementation (cont)
kabi-lookup
Uses boost to deserialize the data files into a container (std::map).
Partial or whole word searches.
Searches can be limited to symbols in the kabi whitelists.
Searches can be limited to specific directories or files by using a path mask option.
[ EXERCISE - STEP 2 ]
But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange.
[PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ]
Performance
Mission critical
Brainstorming
Our best minds, their best minds together
Technology innovation
Good interoperability
Advising us the best way to use it
We can count on
Leader
Support
Extremely strategic
What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies.
It's about working closely with a trusted advisor to understand technology and the impact in your organization.
About creating a relationship that combines your expertise and Red Hat's knowledge and leadership.
Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer.
Internal Only

18. Where is it? Latest rpms: http://people.redhat.com/tcamuso/kabitools/
Git repo:
Report problems and feature requests to
Internal Only