Kernel HW KABI Tools Tony Camuso Aug 20, 2015 EXERCISE - STEP 1 INTRO: I'm here today to talk to you about the value of having a relationship with Red Hat. [ Whiteboard Exercise ] Before we get started, I want to capture a few thoughts you may have about what Red Hat is today. Possibly some of the following words: [ Write examples in a single column as you say them ] Linux Open source Jboss Low-cost Performance Freedom This is all true, and it's part of what of we offer, but but what we want to talk to you about today is something more compelling that Red Hat has to offer from a value and relationship perspective.

What's the problem? No easy way to determine KABI dependencies kernel wide. Need to view the dependency tree from exported symbols down to all the compound data types they depend on. Need to view the dependency tree from compound data types up to their dependent exported symbols. Need to determine if changes to a subordinate compound data type can affect a whitelisted symbol. Need to determine potential impact of customer requests to add new symbols to the KABI whitelist. [ EXERCISE - STEP 2 ] But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange. [PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ] Performance Mission critical Brainstorming Our best minds, their best minds together Technology innovation Good interoperability Advising us the best way to use it We can count on Leader Support Extremely strategic What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies. It's about working closely with a trusted advisor to understand technology and the impact in your organization. About creating a relationship that combines your expertise and Red Hat's knowledge and leadership. Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer. 22 Internal Only

Proposed Solution Creates a database of all exported symbols and their dependencies. Creates the database as part of the build process or independently as an ancillary process. Provides a tool to query the data base for exported symbols and the data structures in their hierarchy. Uses preprocessor output files, which include all headers with all typedef and macro expansions for each corresponding source file. The generated data files comprise a comprehensive graph of all exported symbols and all their dependencies. [ EXERCISE - STEP 2 ] But rather than have me alone tell this story. I want to take a few moments have you hear from one of our customers, Steve Rubinow from the New York Stock Exchange. [PLAY NYSE VIDEO. - While the video is playing, write key words or themes spoken by Rubinow, but write them in a second, column next to the first list. ] Performance Mission critical Brainstorming Our best minds, their best minds together Technology innovation Good interoperability Advising us the best way to use it We can count on Leader Support Extremely strategic What you heard from Steve is that Red Hat to the New York Stock Exchange is more than just particular products or particular technologies. It's about working closely with a trusted advisor to understand technology and the impact in your organization. About creating a relationship that combines your expertise and Red Hat's knowledge and leadership. Ultimately, this is what we'd like you to understand about the type of relationship we'd like to have with you as a customer. 33 Internal Only

Why use THIS tool? Able to relate lowest level data structures to any exported symbols that depend on them. Can detail the entire dependency tree of any exported symbol or any data structure. Provides command line options to limit search scope to directories, files, and the kabi whitelists. Provides a verbose option to display entire hierarchies. Imposes a 'dups' policy to limit output when encountering duplications. 44 Internal Only

kabi-lookup tool options 55 Internal Only

kabi-lookup tool options (cont) 66 Internal Only

Search for exported symbols affected by a struct -slw : struct, limited to whitelist, whole word search (exact match) -m fs/bio : limit search to path having substring “fs/bio” 77 Internal Only

Another struct search Show exported symbols affected by structs with 'struct device' as part of their name. Limit search to files and directories with 'pci' substring, and limit output to first 20 lines. Search for declaration of 'struct class' and grep for 'struct device'. 88 Internal Only

Yet another struct search Search for whitelisted symbols affected by 'struct kernel_param' in directories and files with 'pci' in their name. 99 Internal Only

Show details of struct kernel_param 1010 Internal Only

Search for exported symbol Search for exported 'struct bio_set' and see its top level dependencies. See if 'struct bio_set' is in the kabi whitelist. 1111 Internal Only

Find whitelisted function pci_iomap Then show its dependency tree 1212 Internal Only

RHEL7 Kernel Compile Times System Without kabi-parser With kabi-parser % change intel workstation core 860 @ 2.8GHz 4 cores (8 HT) 16 G DDR3 15:49 (15.82) 17:07 (17.12) 8.2 amd dinar opteron 6386 @ 1.4GHz 32 cores 32 G DDR3 4:25 (4.42) 4:57 (4.95) 12.1 1313 Internal Only

Search Times Substring search for popular struct (generates lots of output) Exact match search for same takes about 2 seconds less. 1414 Internal Only

Search Times (cont) Longest search times are typically less than a minute. At this time, searches are disk intensive, so cpu cores and memory have little impact on performance. SSD should improve search times considerably. Slurping the whole database for each search is impractical, though it might be useful for an interactive version of the tool. 1515 Internal Only

Improving Search Performance Under consideration Second level database that maps symbols to the files that contain them. Threaded search Memory mapped database files 1616 Internal Only

Known Problems Patches to inhibit sparse error reporting have been posted upstream. Until these patches are incorporated, you must apply them before building the libsparse.a in order to build kabitools from scratch. _Static_assert (gcc 4.6+) is not yet recognized by sparse sparse processing bails as soon as it encounters _Static_assert Running kabi-lookup from a directory other than the top of the kernel tree hasn't been fully implemented yet. 1717 Internal Only

Where is kabitools? Latest rpms: http://people.redhat.com/tcamuso/kabitools/ Git repo: https://github.com/camuso/kabitools Report problems and feature requests to https://github.com/camuso/kabitools/issues 1818 Internal Only

Where is sparse? sparse git repo: http://git.kernel.org/cgit/devel/sparse/sparse.git/ sparse mailing list: linux-sparse@vger.kernel.org 1919 Internal Only

Summary Find the hierarchy of any exported symbol. See how a data structure affects any exported symbols. Augur the effects of adding new symbols to a whitelist. 2020 Internal Only

Thank you! Tony Camuso <tcamuso@redhat.com> https://home.corp.redhat.com/user/tcamuso 2121 Internal Only