CFI GROUP WORLDWIDE ANN ARBOR ATLANTA BEIJING LONDON MADRID MILAN PARIS SHANGHAI STOCKHOLM REPRESENTATIVE OFFICES BUENOS AIRES KUALA LUMPUR PORTO ALEGRE NASA Earth Observing System Data and Information System Customer Satisfaction Results November 17, 2008 Data Center Managers November 18, 2008 User Services Working Group

2 © CFI Group 2008 Today’s Discussion Background Overview Key Results Detailed Analysis Summary

3 © CFI Group 2008 Background

4 © CFI Group 2008 Project Background Objectives Measure customer satisfaction with the NASA Earth Observing System Data and Information System at a national level and for each Data Center –Alaska Satellite Facility (ASF DAAC) –Goddard Space Flight Center Earth Sciences Data and Information Services Center (GSFC DISC DAAC) –Global Hydrology Resource Center (GHRC) –MODIS Data Processing System (MODAPS/LAADS) –NASA Langley Atmospheric Science Data Center (ASDC DAAC – LaRC) –Land Processes Distributed Active Archive Center (LP DAAC) –National Snow and Ice Data Center (NSIDC DAAC) –Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC/FLUXNET) –Physical Oceanography Distributed Active Archive Center (PO DAAC - JPL) –Socioeconomic Data and Applications Center (SEDAC) –*Crustal Dynamics Data and Information Systems (CDDIS) –*Ocean Biology Processing Group (OBPG) Assess the trends in satisfaction with NASA EOSDIS specifically in the following key areas: –Product Search –Product Selection and Order –Delivery –Product Quality –Product Documentation –Customer Support Identify the key areas that NASA can leverage across the Data Centers to continuously improve its service to its users *Measured for first time in 2008

5 © CFI Group 2008 Project Background Measurement timetable Finalized questionnaireAugust 1, 2008 Data collection via webAugust 28, 2008 – September 22, 2008 Topline resultsSeptember 30, 2008 Results briefingNovember 3, 2008

6 © CFI Group 2008 Project Background Data collection Respondents 2,763 responses were received 2,601 responses were used for modeling Those who answered for more than one data center: Two: 62 Three: 9 Four: 2 Five: 1

7 © CFI Group 2008 Project Background Respondent information For which disciplines do you need or use Earth science data? Demographics remain fairly consistent with 2007 * Multi-select; **Modeling was asked as a separate question prior to 2008

8 © CFI Group 2008 Project Background Respondent information Demographics remain fairly consistent with 2007 *Asked in previous years ** Asked separately

9 © CFI Group 2008 Overview Key Results

10 © CFI Group 2008 NASA EOSDIS Customer satisfaction results 2006 ATTRIBUTES Ideal How close does [DAAC] come to the ideal organization? Overall satisfaction How satisfied are you with the data products and services provided by [DAAC]? Expectations To what extent have the data products and services provided by [DAAC] fallen short of or met your expectations? ACSI (+/-) 0.9(+/-) 0.7 (+/-) 0.5 (+/-) 0.6 (+/-) 0.5 N=1016 N=1263 N=2857N=2291 N=2601

11 © CFI Group 2008 NASA EOSDIS Benchmarks Strong performance continues …

12 © CFI Group 2008 Customer Satisfaction Index Future Use Recommend NASA EOSDIS Model Product Search/Selection/Documentation and Customer Support most critical Sample Size: The performance of each component on a 0 to 100 scale. Component scores are made up of the weighted average of the corresponding survey questions. Scores Customer Support Product Search Product Quality Product Documentation Product Selection and Order The change in target variable that results from a five point change in a component score. For example, a 5-point gain in Product Search would yield a 0.7-point improvement in Satisfaction. Impacts Delivery

13 © CFI Group 2008 NASA EOSDIS Significant improvements from 2007 =Significant Difference vs (+/-) 0.5 (+/-) 0.9 (+/-) 0.6 (+/-) 0.7 (+/-) 0.6

14 © CFI Group 2008 Areas of Opportunity for NASA EOSDIS Remain consistent year over year Top Improvement Priority Product Search (75) Product Selection and Order (77) Product Documentation (75)

15 © CFI Group 2008 Detailed Analysis

16 © CFI Group 2008 Score Comparison Higher satisfaction continues outside of the USA Respondents outside the USA continue to have a higher overall Satisfaction score with EOSDIS (76 outside vs. 74 USA in 2007), the gap remains 2 points. 68% of respondents are outside of the USA in 2008 vs. 65% in 2007.

17 © CFI Group 2008 CSI by Data Centers Most Data Centers trend positively or stay same =Significant Difference vs. 2007

18 © CFI Group 2008 Product Search Key driver of satisfaction 41% used EOS Data Gateway to search for data and products (52% in 2007); 12% selected Internet Search Tool, a new category in 2008 =Significant Difference vs Product Search How well the search results met your needs Ease of finding data Ease of using search capability Impact=0.7

19 © CFI Group 2008 Product Search Score Comparison By method for most recent search How did you search for the data products or services you were seeking? =Significant Difference vs (+/-) 1.0 (+/-) 1.3 (+/-) 0.8 (+/-) 2.7 (+/-) 4.1 (+/-) 6.6 (+/-) 1.7

20 © CFI Group 2008 Product Search Scores by Data Center; most trend positively =Significant Difference vs. 2007

21 © CFI Group 2008 Product Selection and Order Also a top opportunity for continuous improvement 95% said that they are finding what they want in terms of type, format, time series, etc. Please think about your most recent request/order/download from the Data Center. Did you use a subsetting tool? 25% said No, 39% said Yes, by geographic area, 7% said Yes, by geophysical parameter, and 29% said Yes, by both geographic area and geophysical parameter. Impact=0.7 =Significant Difference vs Product Selection and Order Ease of requesting/ordering data products Ease of selecting data products Description of data products

22 © CFI Group 2008 Product Selection and Order Scores by Data Center; most trend positively =Significant Difference vs. 2007

23 © CFI Group 2008 Product Documentation Data product description and product format most sought after What documentation did you use or were you looking for?* Data product description 66%; Product format 53% Science algorithm 46%; Instrument specifications 33% Tools 32%; Science applications 30% Production code 10% Impact=0.6 *Multi-select Was the documentation… Delivered with the data (16% vs. 15% in ‘07), Available online (70% vs. 69% in ‘07), Not found (14% vs. 16% in ‘07). CSI for those whose documentation was not found is 69 vs. those who got it delivered with the data (77) or online (78) N/A Product Documentation Extent to which the data documentation helped you use the data Overall quality of the document (i.e., technical level, organization, clarity) 2005

24 © CFI Group 2008 Product Documentation Scores by data center Impact=0.6 =Significant Difference vs. 2007

25 © CFI Group 2008 Customer Support Maintain strong performance Did you request assistance from the Data Center’s user services staff during your most recent search or order? No=64%, Yes, by phone=2%, Yes, by =29%, Yes, by phone and =5% 91% (91% in 2007) were able to get help on first request. These respondents continue to have a significantly higher CSI (80) than those who did not (64). Impact= Customer Support Professionalism Technical knowledge Accuracy of information provided Helpfulness in selecting/finding data or products Timeliness of response Helpfulness in correcting a problem

26 © CFI Group 2008 Product Quality Preferences in line with actual for the most part In 2007, 73% said products were provided in HDF- EOS and HDF and 41% said they were their preferred method. *Multiple responses allowed

27 © CFI Group 2008 Product Quality Improves this year Impact= N/A Product Quality Ease of using the data product in the delivered format

28 © CFI Group 2008 Delivery Improves this year Over half said their data came from MODIS; 18% said ASTER (24% in 2007). Impact=0.4 =Significant Difference vs Instrument the data came from 2008* MODIS (Land) 49% MODIS (Atmosphere) 26% MODIS (Ocean) 26% =Significant Difference vs Delivery Convenience of delivery method Timeliness of delivery method *Multi-Select

29 © CFI Group 2008 Delivery Methods for receiving … How long did it take to receive your data products? 26% immediate retrieve (20% in 2007) CSI=79 30% less than a day (29% in 2007) CSI=78 30% 1-3 days CSI=76 8% 4-7 days CSI=73 3% 8-14 days CSI=71 2% more than 14 days CSI=68 76% said FTP was their preferred method in 2007

30 © CFI Group 2008 Customers over multiple years Who have answered the survey multiple years … For those answering the survey over multiple year, key areas show improvement from 2005.

31 © CFI Group 2008 Customers over the past two years Who answered the survey in 2007 and 2008 For those answering the survey in 2007 and 2008, scores remained similar.

32 © CFI Group 2008 Customers over the past three years Who answered the survey in 2006, 2007 and 2008 For those answering the survey in 2006, 2007 and 2008 overall satisfaction is the same.

33 © CFI Group 2008 Summary

34 © CFI Group 2008 Summary  NASA EOSDIS has made significant improvements versus last year in multiple areas (Product Selection/Order, Search Quality and Delivery)  Most Data Centers trend positively or stay same  Product Search, Selection and Order continue to be the top opportunities for improvement Documentation continues to have a higher impact this year  Customer Support continues to be high impact for those who require it. Imperative to maintain the strong level of service. Ensure those who are providing it realize how it affects satisfaction

35 © CFI Group 2008 Appendix

36 © CFI Group 2008 ACSI National, Sector and Industry Scores: Q – Q Hotels 81Full Service Restaurants 77Limited- Service Restaurants 66Newspapers 70Motion Pictures 67Network/Cable TV News 73Computer Software 70Fixed Line Telephone Service 68Wireless Telephone Service 70Cellular Telephones 62Cable & Satellite TV 73Energy Utilities 75Supermarkets 71Gasoline Stations 74Department & Discount Stores 75Specialty Retail Stores 78Health & Personal Care Stores 77Banks 79Life Insurance 72Health Insurance 78Property & Casualty Insurance 63Airlines 73U.S. Postal Service 81Express Delivery 65.9Local Government 71.3Federal Government Accommodation & Food Services 75.7 Information 68.3 Utilities 72.9 Finance & Insurance 76.0 Transportation & Warehousing 71.1 Public Administration/ Government Retail 78Auctions 78Brokerage 76 Travel E-Commerce Hospitals Health Care & Social Assistance 76.8 Source: Manufacturing/ Durable Goods 80.1 E-Business Personal Computers 80 Electronics (TV/VCR/DVD) 81Major Appliances 81Automobiles & Light Vehicles 73News & Information 76Portals 79Search Engines Manufacturing/ Nondurable Goods Food Manufacturing 83Pet Food 84Soft Drinks 82Breweries 78Cigarettes 80Apparel 76Athletic Shoes 84Personal Care & Cleaning Products Retail Trade

37 © CFI Group 2008 x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x 1 x 3 x 4 x 5 x 6 y1y1 y2y2 y3y3 y 3 y 2 y 1 11 22  x i xit i , for i=1,2,3 t=1,2 y jyjj  1, for j=1,2,3     x 2 The Math Behind the Numbers A discussion for a later date…or following this presentation for those who are interested.

38 © CFI Group 2008 A Note About Score Calculation Attributes (questions on the survey) are typically answered on a 1-10 scale –Social science research shows 7-10 response categories are optimal –Customers are familiar with a 10 point scale Before being reported, scores are transformed from a 1-10 to a scale –The transformation is strictly algebraic; e.g. –The scale simplifies reporting: Often no need to report many, if any, decimal places scale is useful as a management tool

39 © CFI Group 2008 Deriving Impacts Remember high school algebra? The general formula for a line is: y = mx + b The basic idea is that x is a “cause” and y is an “effect”, and m represents the slope of the line – summarizing the relationship between x & y CFI Group uses a sophisticated variation of the advanced statistical tool, Partial Least Squares (PLS) Regression, to determine impacts when many different causes (i.e., quality components) simultaneously effect an outcome (e.g., Customer Satisfaction)