Taming the Lint Monster A personal perspective of the PC-Lint code analysis tool, and how to use it effectively Anna-Jayne Metcalfe Riverblade Limited http://www.riverblade.co.uk

An All Too Common Story Partway through a project, the company starts to become anxious about the number of defects that are being identified in the product, and how long they are taking to fix. Even worse, customers are beginning to notice.   Something must be done. Additional resources are thrown at the problem, but somehow it doesn't ever seem to be enough. The codebase is large, complex and hard to understand and maintain. It is - for all intents and purposes - a "Big Ball of Mud“. After several months of throwing additional firefighters at the problem, someone has the bright idea™ to find out just how much hidden nastiness is lurking in the code base waiting for the right moment to let loose it's wrath on the unsuspecting team. An appropriate tool is identified and procured, and then the real fun starts - actually using it. Inevitably, it never quite turns out the way the team (or their managers) expect. Not only does it turn out to be an absolute nightmare to configure and use but when the team do finally get it working to their satisfaction the results it produces are so volumous that nobody quite knows what to do with them. Worse, they contain some really bad news ™. As all too often happens, dealing with the issues the tool raises is deemed to be a) too expensive, b) too risky and c) not as much fun as writing new copy-paste code (though nobody is ever quite honest enough to admit to the latter). The team conveniently forget about the whole experience and go back to compiling at warning level 3 as they always have done. The installation disk for the offending tool is quietly hidden away in a desk draw and forgotten...and of course, the Big Ball of Mud grows ever bigger until the inevitable "lets just re-write it in language "X" event a year or two later….

What Can We Learn From This? Very few teams seem to use code analysis tools effectively There is a credibility gap - especially. among “the 80%”: Perceptions such as “too hard to set up”, “too much noise” “bad news” or “telling us how to do our job” can be hard to shift Nevertheless, analysis tools can uncover real problems in a codebase ...if you invest the time to learn how to use them effectively So what can we learn from this experience? Unfortunately, very few teams use code analysis tools effectively. If you ask around in organisations you encounter you will hear many reasons (and more than a few excuses...) for this. In many cases, developers are simply unaware that such tools exist, or are unwilling to invest the time in integrating them with their development processes. Of course you can hear similar excuses if you ask about unit testing, profiling, etc... However, code analysis tools like PC-Lint can uncover real problems in a codebase if you use them correctly. You do however have to invest the time and effort to learn how to do this effectively – any organisation which expects a code analysis tool to provide “a quick fix” is likely to be very disappointed.

So What is PC-Lint, Anyway? C/C++ code analysis tool first introduced in 1985 Available in both Windows (PC-Lint) and Unix/Linux (Flexelint) variants Command line only Extremely thorough and very flexible Can be difficult to set up correctly Analysis runs can be (very) slow on large projects PC-Lint is a C/C++ static code analysis tool published by Gimpel Software, and first introduced way back in 1985. Both Windows (PC-Lint) and Unix/Linux (Flexelint) variants are available, with the latter delivered as obfuscated source code (and much more expensive).   PC-Lint is a command line tool with a range of options rivaling those of a full featured C++ compiler. Of course, all of that configurability comes at a price – PC-Lint is far from easy to use, can be slow to run (depending on how your codebase is structured) and the analysis results it produces can seem verbose and cryptic at first. Nevertheless, it is very thorough, and more than capable of exposing potentially serious hidden flaws in your codebase – if you give it the time to do so.

Example Analysis Results --- Module: CJFlatHeaderCtrl.cpp } CJFlatHeaderCtrl.cpp(160): error 1401: (Warning -- member 'CCJFlatHeaderCtrl::m_bSortAsc' (line 146, file ..\Include\CJFlatHeaderCtrl.h) not initialized by constructor) CJFlatHeaderCtrl.cpp(166): error 1740: (Info -- pointer member'CCJFlatHeaderCtrl::m_pParentWnd' (line 150, file ..\Include\CJFlatHeaderCtrl.h) not directly freed or zero'ed by destructor  -- Effective C++ #6) PC-Lint analysis results are usually presented in textual form (I refer to this as “raw analysis results”). The format of the analysis results can be configured using a suitable indirect file. With the appropriate configuration, most development environments can understand enough to provide "double click to go to issue location“ functionality for analysis results piped to its output window.   Gimpel provide a number of such environment options files in the PC-Lint installation (for example env-vc8.lnt for Visual Studio 2005), and others can be downloaded from the support page at http://www.gimpel.com/html/ptch80.htm (PC-Lint 8.00) or http://www.gimpel.com/html/ptch90.htm (PC-Lint 9.00). A special mention must be made of env-xml.lnt, which allows PC-Lint to easily generate XML output – the usefulness of which should be obvious!

Online PC-Lint Demonstrator http://www.gimpel-online.com/OnlineTesting.html and http://www.gimpel-online.com/bugsLinkPage.html Get a volunteer to choose which one to run!

A Few PC-Lint Capabilities Detection of dangling/uninitialised pointers Variable initialisation/value tracking Variable scoping Type mismatches and dodgy casts Assignment operator/copy constructor checking Detection of potential memory leaks Analysis of thread behaviour (new to PC-Lint 9.0) MISRA C/C++ rule validation PC-Lint is a incredibly capable tool (version 9.0 alone added no fewer than 146 new messages) so in practice it is impossible to do its capabilities justice in just one two slides.

Other C/C++ Static Analysis Tools Splint (C only, but free) PreFAST QA C++ Klockwork Insight Coverity Parasoft etc... The C++ code analysis toolspace is a crowded one. Although there are a number of other similar tools on the market, most are part of a much larger integrated toolset with an enterprise price tag to match. The only open source contender I'm aware of (Splint- another derivative of the original Unix lint) is limited to C and its authors unfortunately seem to have no intention of adding C++ support (however, as the source code is freely available, I will happily leave that as an exercise for the reader...).

Be Prepared! The first time you analyse a codebase, expect both difficulties and surprises Configuration issues Lots of unwelcome (at least initially) “noise” Potentially bad news in places you really don’t want to look Management and co-workers may not want to know

A (Very) Simple PC-Lint Command Line lint-nt std.lnt filea.cpp Single file (“unit checkout”) analysis But what is “std.lnt”? At its absolute simplest, a PC-Lint command line to analyse a single file and output the results to the console looks something like this:   lint-nt std.lnt <filename> where lint-nt.exe is the PC-Lint executable and std.lnt is a configuration file (Gimpel call them "indirect files") describing the compiler and framework configuration (preprocessor symbols etc.), include paths and warning policy. [HANDS ON: Navigate to lint folder and run command...]

The Global “Indirect file” std.lnt Conventionally holds the “global” PC-Lint configuration High level configuration options Global include folder specification References to other indirect files (compiler options, warning policy etc.) Include folder specifications can be machine dependent Std.lnt does not usually define the full warning policy std.lnt usually consists of a set of references to other indirect files, together with a handful of options and a set of include folder specifications. [SHOW EXAMPLE] Of the contents within the file, the only section which is not straightforward is the Include folder specification. This must match whichever IDE you use EXACTLY, or you will receive analysis errors. It follows that this needs to be customised for each development system, which can be a significant issue in itself a large team which is trying to integrate PC-Lint into its development process. When PC-Lint encounters the name of another indirect file within std.lnt, it will attempt to open and process it (failing with a fatal error if it cannot). By the way, if you use Visual C++ you can download sample std.lnt and options.lnt files for all Visual Studio versions from Visual C++ 6.0 to Visual Studio 2008 and eMbedded Visual C++ 4.0 from the Riverblade website.

Warning Policy: options.lnt Defines specific messages to globally enable/disable std.lnt actually defines the high level policy, but options.lnt adjusts it to your needs Usually comprised of just a list of –e directives Options.lnt usually defines the warning policy, together with (possibly) additional issue suppression options. Our own warning policy is actually pretty simple - it consists of the full set of Scott Meyers recommendations (activated by the inclusion of au-sm123.lnt in std.lnt), with a handful of issues suppressed by -e directives.

Basic IDE Integration Conventionally via custom tools within the IDE Analysis results sent to the output window May be blocking, so analysis time can be an issue. Especially in whole project analysis (more on that later) Detailed analysis configuration can be an issue Results are not usually persistent Some of the env-*.lnt files also contain instructions on how to perform a basic integration of PC-Lint within the corresponding IDE. This usually takes the form of a custom tool, as I will now demonstrate using Visual Studio 2005. [SHOW IDE TOOL CONFIG – draw attention to $(ItemPathName) ] The advantage of this approach is simplicity – the instructions are straightforward to follow, and the analysis results appear in the Output Window where developers expect them to be. There are however several downsides (see slide). None of these issues are insurmountable of course. But taken together they do illustrate why it is often commonplace to run batch analysis runs on a central build server rather than locally on the developer machines (which is where they are really needed, of course). [DEMONSTRATE IN VS2005 (using a simple lint example file, then one which requires a project config)]

A (Not So) Simple PC-Lint Command Line lint-nt.exe -iC:\Lint -background -b --u SourceVersioner_vs71_Debug_Win32.lnt –u std_vs71.lnt env-vc7.lnt -t4 +ffb +linebuf +macrobuf –iDebug Shared\FileUtils.cpp The above command line defines a single file (or “unit checkout”) analysis using the configuration files project.lnt, std.lnt and env-vc7.lnt. It is notable that the options given in each .lnt file are applied in order – so those specified later in the command line can override directives contained within (for example) std.lnt. This can be very useful for message suppression purposes. SourceVersioner_vs71_Debug_Win32.lnt is a project specific indirect file - an indirect file which specifies the PC-Lint options for a particular project configuration and platform. This example also shows several PC-Lint options which are worthy of mention:   -i specifies a folder for include and/or indirect files. In this particular case it is being used to specify the location of PC-Lint indirect files referenced within std.lnt, and any .tlh and .tli intermediate files located within the "Debug" subfolder under the project. -background instructs PC-Lint to run analysis at a low priority (useful if you are analysing several files at once). -b suppresses the banner line --u instructs PC-Lint to ignore any files listed in the following project.lnt file (we will come back to this later).Usually used with -u. -u instructs PC-Lint to perform a "unit checkout" analysis - i.e. analyse a single source file. -t4 tells PC-Lint that it should use a tab size of 4 spaces while parsing source. +ffb instructs PC-Lint that it should assume ANSI compliant for loop scoping. Each occurrence of +linebuf doubles the size of the PC-Lint input buffer (the default size is 620 characters). Each occurrence of +macrobuf doubles the size of the PC-Lint macro buffer.

Coping with Project Configurations For analysis to work effectively, the PC-Lint configuration must match that of the compiler Any mismatches will lead to analysis errors C++ project configurations can be very complex PC-Lint can write suitable configuration (“project.lnt”) files for most Visual C++ projects directly For other platforms, you are (unfortunately) on your own It is critically important in ensuring that the lint tool uses the same project analysis configuration as the compiler itself. Any mismatch in preprocessor directives or include paths is likely to result in a deluge of angry (and misleading) error messages. As C++ project configurations can be very complex, this can be a major undertaking in itself. In the case of Visual C++ projects, PC-Lint can however write suitable project.lnt files in most cases by parsing the project (.dsp or .vcproj) file itself. For example, to generate a project.lnt file for the Unicode Debug configuration and Win32 platform of project CoreLib you could use the command line:   lint-nt.exe CoreLib.vcproj +d"Win32|Unicode Debug" >CoreLib_Win32_Unicode_Debug.lnt Unfortunately this technique only works for Visual C++ project files; for other compilers you have to write the project.lnt files manually. In addition, PC-Lint cannot at present handle Visual Studio environment variables or include folder/preprocessor specifications within inherited property sheet (.vsprops) files. If you have projects structured in this way, you will have to either write the .lnt files manually or use a third party tool to do so.

What is in a project.lnt file? Basically a subset of the compiler configuration for lint purposes Preprocessor definitions Additional include folder paths A list of files in the project, relative to the project folder [SHOW EXAMPLE PROJECT.LNT FILE]

Whole Project Analysis lint-nt.exe -iC:\Lint -background -b std_vs71.lnt env-vc7.lnt -t4 +ffb +linebuf –iDebug SourceVersioner_vs71_Debug_Win32.lnt Can identify functions, enums etc. which are not used in that project Single threaded, and can be slow on large projects The list of files in the project.lnt file shown in the previous slide previously is used in "whole project analysis", which involves analysing all of the files in a project together. This method requires more memory than analysing each source file individually, but has the advantage of allowing PC-Lint to identify issues (e.g. unreferenced functions Or declarations) which only become apparent in a wider context than an individual compilation unit.   By comparison with the previous example, the -u and --u options are not used and a specific source file need not be specified since the project.lnt file contains a list of files to analyse. [SHOW EXAMPLE OUTPUT]

PC-Lint Message Categories Five categories, of varying severity: Elective Notes Informational Warnings Errors Fatal Errors Individual categories and messages can be selectively enabled via –w and +e/-e options PC-Lint organises its messages into five categories of varying severity:   Elective Notes e.g. 953 ("Variable could be declared as const") Informational - e.g. 1924 ("C-Style cast) Warnings - e.g 534 ("Ignoring return value of function") or 1401 ("Member symbol not initialised by constructor") Errors – e.g. 1083 (“1083 - Ambiguous conversion between 2nd and 3rd operands of conditional operator”) Fatal Errors – e.g. 322 (“322 - Unable to open include file”) Conveniently, PC-Lint has a -w option which allows the warning level to be set globally. For example, -w3 (the default) will enable only messages of level “Warning” and above – so all Elective Notes and Informational Messages will be suppressed. Similarly, –w4 will enable all messages except Elective Notes. If you need to enable a specific message below the current warning level, you can simply add a +e directive for the message in question after the –w option. If you wanted to detect any unused include files in a project, adding the options –w0 +e766 to your command line (after all other .lnt files) will enable only message 766 ("Header file not used in module").

Common Analysis Failures Fatal Error 314: Previously used .lnt file Fatal Error 307: Can’t open indirect file Fatal Error 322/Error 7: Unable to open include file Error 91: Line exceeds Integer characters (use +linebuf) Error 303: String too long (try +macros)

Analysis Speed Influenced by CPU/disk speed and project structure Include dependencies can be very significant PC-Lint 9.0 adds precompiled and bypass headers Can potentially cut analysis time by 3-4 times PC-Lint is currently single threaded Adding more cores won’t help unless you run multiple analysis tasks simultaneously Single file analysis is amenable to parallelisation

Some issues to look out for 429 (Custodial pointer not freed or returned) 578 (Declaration of symbol hides another) 716 (while(1)) 717 (do...while(0)) 777 (Testing floats for equality) 795 (Conceivable division by zero) Since we originally started using PC-Lint we have noticed a number of issues which, if found in a codebase, may just indicate potential trouble ahead. The following are examples of the sort of issues look for the first time we analyse a new codebase (interestingly, very few of them are warnings – most are actually informational):   429 Custodial pointer 'Symbol' (Location) has not been freed or returned (Warning) 578 Declaration of symbol 'Symbol' hides symbol 'Symbol' (Warning) 716 while(1) ... (Informational) 717 do ... while(0) (Informational) 777 Testing float's for equality (Informational) 795 Conceivable division by 0 (Informational) If you are running PC-Lint on a codebase for the first time with a (consequently) relaxed warning policy, it may just be worth turning on these particular issues specifically (using a +e directive in options.lnt) to see if any of them manifest themselves. As many of these issues are below warning level, you may not see them otherwise.

Some issues to look out for (cont.) 801 (Use of goto is deprecated) 825 (Control flows into case/default) 1506 (Call to virtual function in constructor or destructor) 1725 (Class member is a reference) 1735 (Virtual function has default parameter) 1773 (Attempt to cast away const or volatile) 801 Use of goto is deprecated (Informational) 825 Control flows into case/default without -fallthrough comment (Informational) 1506 Call to virtual function 'Symbol' within a constructor or destructor (Warning) 1725 Class member 'Symbol' is a reference (Informational) 1735 Virtual function 'Symbol' has default parameter (Informational) 1773 Attempt to cast away const (or volatile) (Informational)  

Tuning Out Issues in Libraries Issues in library header files can cause “noise” elseware in a project These can be dealt with in several ways: Reduce the warning level while including library headers Modify the library to fix it or add lint directives Create an indirect file containing “tuning” directives (e.g. -etype(1746, boost::shared_ptr<*>) Regardless of which libraries you use in your projects, you are likely to encounter PC-Lint issues in class or macro definitions within those libraries. When you don't have any control over the implementation of the libraries concerned, such "noise" can be irritating to say the least (although it could of course be indicative of a real problem with the library implementation you should be aware of…).   Fortunately, using some of PC-Lint's error suppression directives in most cases it is relatively easy to write lint directives to prevent this happening. [SHOW ISSUE SUPRESSION FILE EXAMPLE]    In our case, we have collected the "tuning" directives we've identified within the Win32 libraries (Win32 API, ATL, WTL and MFC) into specific indirect files so we don't have to specify them directly in each project or in our global warning policy. You can download them from http://www.riverblade.co.uk/products/visual_lint/downloads if needed.

Turning Down the Volume How can I cope with this deluge of analysis results? (the “noise” issue again) Define your initial warning policy carefully Either start with a reasonably relaxed warning policy and gradually make it more aggressive, or: Start with an aggressive policy and carefully analyse the results to determine which ones you don’t care about I am pretty sure that most people who run PC-Lint do so using either pre-configured scripts or some form of rudimentary integration into whichever IDE they are using.   Regardless of the method used, the issue of how to deal with the volume of analysis results it can produce remains. Common strategies include turning off all but the most critical issues (with the intention of gradually enabling others as the issues are addressed - not that that always happens of course), grepping the raw analysis results, exporting to a spreadsheet and filtering there....the list goes on. Which method you use doesn’t matter. What matters is that you find an approach which works well for you and your organisation.

Tools/Techniques Which May Help Aloa LintProject Grep XSLT transformations highlighting issues you care about (and potentially ignoring the ones you don’t) SourceMonitor (or other complexity measuring tools) If you code is “noisy”, you probably have architectural issues too There are quite a few tools and techniques you can use to collect or post-process analysis results. Ralph Holly's Aloa (http://www.ddj.com/cpp/184401810) or our own LintProject analysis/HTML reporting tool (http://www.codeproject.com/KB/applications/lintproject.aspx) are just two examples, but you may of course find grepping, transforming XML output and analysing it in a speadsheet or something else entirely is more convenient. Find a solution that works for you, and use it effectively. [SHOW ALOA/LINTPROJECT EXAMPLES, if time permits]

Summary Analysis tools such as PC-Lint can uncover real problems in your codebase There is no “Quick Fix” for poor code quality Be prepared to invest significant time (at least at first) in: Configuring the tool to work well with your codebase Developing your warning policy Interpreting analysis results Consider also analysing complexity and design

Any (more) questions? I hope you have found this presentation useful. Any questions?

Taming the Lint Monster A personal perspective of the PC-Lint code analysis tool, and how to use it effectively Anna-Jayne Metcalfe Riverblade Limited http://www.riverblade.co.uk