Download presentation
Presentation is loading. Please wait.
Published byJerome Ames Modified over 9 years ago
1
© 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. Fact From Fiction: Demystifying Credit Score Features Why the FICO ® Score Looks the Way It Does Frederic Huynh Senior Principal Scientist, Analytic Development—Scores FICO
2
© 2014 Fair Isaac Corporation. Confidential. Much care and thought is put into the design of the FICO ® Score. With each subsequent development we challenge our prior design conventions and meticulously evaluate the merits of any challenging approach. 2
3
Agenda © 2014 Fair Isaac Corporation. Confidential. ► Scoring More Consumers Is Easy… ► Engineering Robustness in the FICO ® Score ► Why You Want Three Unique Score Algorithms 3
4
© 2014 Fair Isaac Corporation. Confidential. Scoring More Consumers Is Easy… ► …But Scoring More Consumers Responsibly Is Hard 4
5
© 2014 Fair Isaac Corporation. Confidential. ► As lenders focus on growing their portfolios, the credit underserved population is becoming of interest again ► Segments that could be evaluated for potential inclusion in the scoreable universe for the FICO ® Score: ► Files with no trade line history ► Collection and public record only files ► Inquiry only files ► Collections, public records, inquiries ► “Stale” files (no trade line updates within the last 6 months) ► Very young files (less than 6 months trade line history) The Challenge to Score More Consumers 5
6
© 2014 Fair Isaac Corporation. Confidential. Key Considerations in Evaluating Unscoreable Segments Is the credit data available sufficiently predictive of future repayment risk? Is there sufficient credit repayment history from which to develop models? Will the score continue to effectively rank-order risk? ? 6
7
© 2014 Fair Isaac Corporation. Confidential. Categories of Predictive Variables in the FICO ® Score ► Inquiry Only Files: Only New Credit predictors are available ► Collection/Public Record files: Only Payment History and/or Amounts Owed predictors are available Is the Available Data Sufficiently Predictive? 7
8
© 2014 Fair Isaac Corporation. Confidential. Tepid Prediction Available for Very Thin Credit Files ► Risk discrimination attainable from the extremely sparse data is much weaker than what lenders currently expect from the FICO ® Score Example 8
9
© 2014 Fair Isaac Corporation. Confidential. Bureau Snapshot A Scoring date Bureau Snapshot B Performance date Sufficient Performance History Is Needed for Model Development 9 2 years Only consumers who had measurable credit repayment history between Snapshots A and B (i.e., “classifiable performance”—can be included in the model development sample)
10
© 2014 Fair Isaac Corporation. Confidential. ► Current FICO ® Scoreable Universe ► ~85% of current scoreable universe have “classifiable performance” in the following 24 month window ► Test segment: “Stale” files (last update 24+ months ago) ► Only 4.6% of test segment have “classifiable performance” in the following 24 month window ► Risk patterns observed on 4.6% of sample with ‘classifiable performance’ must be assumed to hold for the other 95.4% of the segment ► Like trying to draw the whole puppy based on its tail Challenge #1: Truncation 10
11
© 2014 Fair Isaac Corporation. Confidential. ► Test segment: Collection and Public Record only files ► Only 6.5% of test segment have “classifiable performance” in the following 24 month window ► The 6.5% of sample with ‘classifiable performance’ likely to be “cherry-picked” by lender—based on non-CB qualifications such as high income, high assets, VIP status, etc. ► Repayment rates for this small, favored, non-representative slice of the segment likely to be much better than would be observed for the segment as a whole ► The make-up of data available for model development is impacted by previous lender decisions ► The bias in these segments can over-estimate the credit quality of the “new scoreable” segment Challenge #2: Bias 11
12
© 2014 Fair Isaac Corporation. Confidential. ► If you are building a model to predict good and bad payment behavior, you need enough information to determine if a consumer is a good or bad payer ► Young Exclusions: <6 Months Time on File ► Kind of Stale: Months Since Most Recent Report Date between 7–23 ► Really Stale: Months Since Most Recent Report Date >= 24 Is There Sufficient Credit Repayment History from Which to Develop Models? % of Potential Scorable% with Perf Young Exclusions5.4%56.2% Kind of Stale Exclusions19.9%8.3% Really Stale Exclusions40.0%4.3% No Trade Files34.8%7.0% 12
13
© 2014 Fair Isaac Corporation. Confidential. Test Segment: “Stale” Files ► Rank ordering weakens as staleness increases ► Degradation in slope as staleness increases is observed across a wide variety of industry performance variables (bankcard, auto, mortgage, etc.) Challenge #3: Robustness in Rank Ordering Slope of Odds-to-Score Line Months Since Reported SlopePDO 0–60.020235 7–110.015944 12–230.015046 24–470.015545 48+0.011560 13
14
© 2014 Fair Isaac Corporation. Confidential. ► Building models on populations with low classifiable performance rates is likely to: ► Over-estimate the credit quality of the “new scoreable” segment ► Be much less robust—across industries, across applications (originations vs. account management), and over time ► A credit score returned from a sparse data file may be unreliable or inaccurate. This may yield: ► Credit lines/loans higher than safe, or lower than necessary ► Interest rates higher than necessary ► Declination where there should have been acceptance, or vice versa ► No one benefits when faulty credit risk assessments yield credit offerings (or declinations) inconsistent with the consumers’ true repayment ability A Misleading Score May be Worse than No Score 14
15
© 2014 Fair Isaac Corporation. Confidential. ► Limited predictive signal you can extract from a traditional credit bureau report ► Solutions incorporating alternative credit data are essential to growing portfolios responsibly when traditional credit data is unavailable or sparse ► Selection of the best combination of alternative credit sources is critical ► Coverage ► Regulatory compliance ► Predictive power ► Modeling techniques need to be incorporated to mitigate truncation and bias ► Stay tuned for our latest research in alternative data solutions Alternative Data and More Robust Modeling Techniques Are the Solution 15
16
© 2014 Fair Isaac Corporation. Confidential. Engineering Robustness in the FICO ® Score ► Designing Scores to be Effective Across the Economic Cycle 16
17
© 2014 Fair Isaac Corporation. Confidential. ► A key value of the FICO ® Score—designed to be a robust rank-ordering tool ► For over 25 years FICO ® Scores have been proven to hold up well across wide variety of economic conditions ► The FICO ® Score is developed using a single pair of snapshots ► Blending multiple pairs of snapshots together to build credit scores is sometimes used for model development ► For FICO ® Score 9, we revisited our development convention to assess the merit of developing using blended samples Engineering Robustness in the FICO ® Score 17
18
© 2014 Fair Isaac Corporation. Confidential. ► Conducted study into relative predictive strength and through-the-cycle robustness of three approaches: ► FICO ® Score status quo ► Based on 2010–2012 sample ► “Light” blend: Dual timeframe sample, with samples closely related in time and economic characteristics ► 2009–2011 and 2010–2012 ► “Premium” blend: Six-timeframe sample, covering a seven year period of economic boom, bust, and initial recovery ► 2005–2007, 2006–2008, 2007–2009, 2008–2010, 2009–2011, and 2010–2012 ► Validated these approaches on out-of-time samples from 2010, 2011, and 2012 ► Both 12 and 24-month performance windows FICO’s Blended Snapshot Research Overview 18
19
© 2014 Fair Isaac Corporation. Confidential. FICO’s Blended Snapshot Research Findings Limited Support for Light Blend Score ► Light blend score consistently less predictive (up to 1 KS point) than “status quo” score built on single snapshot ► Light blend score shows slightly less stability in odds to score alignment across segments compared to “status quo” score Premium Blend Score ► Performs better than light blend score ► In some cases performs better than “status quo” score 19
20
© 2014 Fair Isaac Corporation. Confidential. Additional Consequences of Blended Samples 10/2003– 10/2005 10/2005– 10/2007 10/2007– 10/2009 10/2009– 10/2011 10/2011– 10/2013 Fields to identify medical collections were introduced in September 2011 for one CRA The CARD Act of 2009 impacted the underwriting of credit cards Diluting your development sample with older data can prevent innovation from leveraging newly reported fields and deemphasize current market realities 20
21
© 2014 Fair Isaac Corporation. Confidential. ► FICO ® Score 9 was developed on October 2011–October 2013 data, a time period where many vintages were performing exceptionally well ► To address concerns about the model’s ability to hold up during a more stressed economic period, the model’s predictiveness was assessed on April 2006–April 2008 data ► This represents one of the FICO ® Score 8 development databases ► Across all key industries and applications, FICO ® Score 9 is highly competitive with FICO ® Score 8 on FICO ® Score 8 development data despite the fact that FICO ® Score 8 has “home court advantage” ► Strong out of time performance results is testament to the robustness of the score and advancements in predictive modeling incorporated in FICO ® Score 9 How Well Does FICO ® Score 9 Perform in a More Turbulent Time Period? 21
22
© 2014 Fair Isaac Corporation. Confidential. Despite the Radical Change in the Mortgage Industry FICO ® Score 9 Performs Extremely Well 22
23
© 2014 Fair Isaac Corporation. Confidential. ► There are pros and cons with any analytic approach ► With a blended sample approach, much thought needs to be put into the selection of the actual timeframes used for development ► Selecting two samples very close to each other: ► Defeats the whole purpose of building more variation into the development data and does meet the objective of covering a broader spectrum of the economic cycle ► Delays/waters down introducing enhancements if necessary data is available in current time periods but not in late time periods ► In stress testing the model, FICO ® Score 9 was assessed on FICO ® Score 8 development data and proven to be extremely competitive in a drastically different environment ► The FICO ® Score 9 out of time performance analysis underscores the value of innovation and advancements in predictive modeling Conclusions 23
24
© 2014 Fair Isaac Corporation. Confidential. Why You Want Three Unique Score Algorithms ► Balancing Score Consistency with Leveraging Unique Data Elements 24
25
© 2014 Fair Isaac Corporation. Confidential. ► Key driver of material score differences: material differences in the data ► FICO ® Scores employ a unique algorithm tailored to each CRA ► This is done to leverage the unique aspects of the data captured by each bureau ► Voice-of-customer feedback: “leverage the unique data available at each bureau” ► Identical FICO ® Score design blueprint applied across the bureaus ► Same segmentation ► Same performance window (24 months) ► Same performance definition (90+ days past due) ► Same candidate set of 500+ predictors ► To facilitate greater consistency with FICO ® Score 9 each model was developed using data from 10/2011–10/2013 Score Variation Across Bureaus 25
26
© 2014 Fair Isaac Corporation. Confidential. ► One bureau does not report negative information associated with authorized user accounts ► One bureau reports the date of first delinquency associated with a collection agency account as opposed to reporting only when the collection agency account was assigned ► The minimum score criteria is the same at all three bureaus—yet the score exclusion rate varies from 15% to 25% depending on the bureau ► Some bureaus provide more granular information on fields than others ► One byte industry “kind of business” code ► Each bureau highlights unique sources of information. For example, one bureau highlights their inclusion of rental data from a national network of property management companies Data Formats/Storage Can Differ Across Bureaus 26
27
© 2014 Fair Isaac Corporation. Confidential. ► The data available at each CRA can be different ► FICO ® Scores are built with a consistent design blueprint across CRAs and has maintained alignment between CRAs and prior versions ► FICO ® Scores maintains the competitive advantage of each CRA’s unique data to provide the best predictive value Conclusions 27
28
© 2014 Fair Isaac Corporation. Confidential. This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent. Frederic Huynh fhuynh@fico.com (415) 492-5323 Thank You! 28
29
© 2014 Fair Isaac Corporation. Confidential. Learn More at FICO World Related Sessions ► The FICO ® Score: 25 Years of Democratizing Access to Credit ► Leveraging Alternative Credit Data to Make Better Risk Decisions Products in Solution Center ► Unveiling FICO ® Score 9—Your Credit Score for the New Normal Experts at FICO World ► Frederic Huynh ► Tommy Lee ► Julie Wooding White Papers Online ► To Score or Not to Score? #70 Blogs ► www.fico.com/blog 29
30
© 2014 Fair Isaac Corporation. Confidential. Please rate this session online! Frederic Huynh fhuynh@fico.com 30
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.