NASA Land Atmosphere Near real-time Capability for EOS 2014 Customer Satisfaction Results January 2015

2 © 2014 CFI Group. All rights reserved. Today’s Discussion  Background  Objectives  Data Collection  Respondent Information  Overview of Key Results  Detailed Analysis  Summary

Background

4 © 2014 CFI Group. All rights reserved. Data collection via invitation NASA LANCE provided a list of addresses for people who have used NASA LANCE data and/or imagery. After cleaning the sample list, CFI Group sent out 10,611 invitations. Project Background - Objectives Measure customer satisfaction with NASA Land Atmosphere Near real-time Capability for EOS (LANCE) Identify the key areas that NASA can leverage to continuously improve its service to its customers Assess the trends in satisfaction with NASA EOSDIS specifically in the following areas: Usage Delivery Customer Support Search & Download Data Search & Download Visualization WHAT HOW

5 © 2014 CFI Group. All rights reserved. Finalized questionnaire: October 2014 Data collection: October 21 – November 17, 2014 Topline Results: December 2014 Results Briefing: January 2015 Of the 10,611 invitations sent, 749 individuals completed the survey resulting in a response rate of 7%. Project Background – Data Collection MEASUREMENT TIMETABLE SAMPLE SIZE

6 © 2014 CFI Group. All rights reserved. Questionnaire Design – New in 2014  Respondents asked to focus on one of five major LANCE service areas: > Data download > FIRMS (Fire Information for Resource Management System) > Rapid Response image subsets, gallery or swath images > Worldview > GIBS (Global Imagery Browse Services) e.g., Web Map Tile Services (WMTS), KML (Keyhole Markup Language)

7 © 2014 CFI Group. All rights reserved. Data Download (58%) and FIRMS (51%) were the most cited LANCE NRT services used. 22% of respondents were from the US. Fires (60%), vegetation (40%) and weather (39%) are the most cited areas of need for NRT products. Data Download and FIRMS most used; Fire area of most need Respondent Information ~ Total percentage may exceed 100 due to multiple responses

8 © 2014 CFI Group. All rights reserved. NRT data was accessed by users in 112 different countries. The countries with at least 1% response are listed here and on the next slide. User Country of Origin Breakout Respondent Information

9 © 2014 CFI Group. All rights reserved. The most common non-US countries are:  Spain-6%  Argentina-4%  Canada-4%  Indonesia-4%  Australia- 3%  India-3 %  Italy-3%  Mexico-3% which account for 30% of all users User Country of Origin Breakout (cont) Respondent Information

10 © 2014 CFI Group. All rights reserved. Conservation (31%) and Answering Science Questions (25%) are areas with most interest. Nearly half (48%) find the timeframe of less than 3 hours within spacecraft acquisition most useful. Most Interested in Natural Resource Mgmt/Answering Science Questions; 48% Want Data Within 3 Hours Respondent Information ~ Total percentage may exceed 100 due to multiple responses

11 © 2014 CFI Group. All rights reserved. LANCE websites (67%) and Internet searches (50%) are most used methods to search for products. 91% find the information needed. LANCE Website or Internet most often used for product search Respondent Information ~ Total percentage may exceed 100 due to multiple responses

12 © 2014 CFI Group. All rights reserved. Just under one-quarter request assistance from Customer Support. A large majority (90%) get the help they need on the first request. The availability of data is improving as the percentage of those that never have experienced a data unavailability rose 7 points (35%) from last year. Most Get Help Needed on First Request; Fewer Experience Data Unavailability Respondent Information ~ Total percentage may exceed 100 due to multiple responses

Overview Key Results

14 © 2014 CFI Group. All rights reserved. NASA LANCE outscores federal government average by 11 points and outscores the aggregate score for internet news and information providers by 3. The score of 77 is within the range of other federal agency information providers. Baseline score is on par with other information providers Benchmarking

15 © 2014 CFI Group. All rights reserved. 15 Customer Support Search & Download Visualization Data & Imagery Delivery Format & Method Search & Download Data Usage – Use of LANCE NRT Products Future Use Recommend SATISFACTION DRIVERSFUTURE BEHAVIORS Scores represent your performance as rated by your customers Driver Impacts show you which driver has the most/least leverage – where improvements matter most/least to your customers Future Behavior Impacts represent the impact of CSI on the future behaviors of your customers Overall Satisfaction 80 Compared to Expectations 77 Compared to Ideal 75 Customer Satisfaction Index NASA LANCE Customer Satisfaction Model (n=740)

16 © 2014 CFI Group. All rights reserved. Priorities for NASA LANCE

Detailed Analysis

18 © 2014 CFI Group. All rights reserved. Drivers of satisfaction indicate solid performance across all areas. Most drivers hold steady as all Year over Year changes are not statistically significant. Customer Support remains area of greatest strength CSI & Component Scores

19 © 2014 CFI Group. All rights reserved. US users tend to have higher satisfaction than users outside of the US. Significant differences exist in Customer Support, Search & Download Visualization and Data & Imagery Delivery US Customers slightly more satisfied than other locations CSI & Component Scores by Location * Significant difference at 90% confidence level

20 © 2014 CFI Group. All rights reserved. Although Search & Download Visualization holds steady from last year, Ease of Downloading Imagery climbs 5 points*. Search & Download Visualization has the most impact on satisfaction this year. * Significant difference at 90% confidence level Search & Download Visualization is highest impact area Search & Download Visualization (Impact = 2.2)

21 © 2014 CFI Group. All rights reserved. Data and Imagery Delivery Format and Method also has a high impact. Convenience rates slightly higher than timeliness. Delivery Format and Method rates higher for Convenience Data and Imagery Delivery Format and Method (Impact = 1.8)

22 © 2014 CFI Group. All rights reserved. Customer Support is a high-impact, high- performing area. Just over one-fifth (22%) of customers contacted Customer Support. Support rates highest for professionalism and technical knowledge. Customer Support is highly professional and technically knowledgeable Customer Support (Impact = 1.5)

23 © 2014 CFI Group. All rights reserved. Search and Download Data has a lower impact on satisfaction. Ratings are slightly higher for ease of downloading data compared to ease of finding data. Search & Download Data meeting users’ needs Search & Download Data (Impact = 1.2)

24 © 2014 CFI Group. All rights reserved. Ease of Using experiences a 3 point significant increase so that Ease of Using is now close to on par with Usefulness of Products. Easing of Using experiences increase Usage – Use of LANCE NRT Products (Impact = 1.1)

25 © 2014 CFI Group. All rights reserved. The table shows component and attribute scores for customers surveyed in Customer support experiences a drop in Professionalism (-4) and How Well the Issue was Handled (-3). Ease of Downloading Imagery (84) and Ease of Using LANCE Products (82) experience increases. Customer Support dips while Ease of Downloading and Using rise Model Attribute Table * Significant difference at 90% confidence level

26 © 2014 CFI Group. All rights reserved. Almost all service attributes score between the mid 70s and mid 80s with one major exception. Rapid Response Real Time Ease of Finding Imagery and Image Availability Timeliness both score in the high 50s. Users are generally satisfied with most LANCE service attributes LANCE Service Attributes Table

27 © 2014 CFI Group. All rights reserved. 63% of users download data at least once a month or more. Most (90%) are able to download in their preferred format. Almost all users (94%) prefer either FTP or HTTPS format. Over 60% download data at least once a month; FTP HTTPS most preferred Data Service Respondent Information

28 © 2014 CFI Group. All rights reserved. Most (84%) use the alerts in FIRMS. 82% of users browse or receive FIRMS data at least weekly. Most (94%) are able to download FIRMS data in their preferred format. Over 80% of users browse or receive FIRMS data at least weekly FIRMS Service Respondent Information

29 © 2014 CFI Group. All rights reserved. 76% browse or receive data at least once a week. Just under half (43%) have tried Worldview and 21% would consider switching. 76% browse or receive Rapid Response data at least once a week Rapid Response Service Respondent Information

30 © 2014 CFI Group. All rights reserved. 83% of the Worldview service respondents access near real-time imagery at least once a month. Most (88%) were able to download imagery data in their preferred format. Worldview accessed at least once a month by over 80% of users Worldview Service Respondent Information

31 © 2014 CFI Group. All rights reserved. GIBS is accessed at least once a week by 70% of the respondents and is most often accessed via Google. 70% access GIBS at least once a week GIBS Service Respondent Information ~ Total percentage may exceed 100 due to multiple responses

32 © 2014 CFI Group. All rights reserved. Most (92%) are able to request GIBS imagery in their preferred web service. Just under one-third (31%) redistribute the imagery in some manner. Over 90% request GIBS imagery in their preferred format GIBS Service Respondent Information

33 © 2014 CFI Group. All rights reserved. All of the rated questions for each LANCE service area were averaged to come up with an overall service area score. There is a 6 point difference between the highest and lowest rated service areas with GIBS at 83 and Rapid Response at 77. GIBS highest rated among LANCE service areas Summary

34 © 2014 CFI Group. All rights reserved. Respondents chose the service they wanted to rate. FIRMS and Data Download were the most rated. Those who rated FIRMS services had the lowest overall CSI score. CSI Score by Service Area Rated

35 © 2014 CFI Group. All rights reserved. Rapid Response Search and Download Visualization rates lower than the other two service areas. GIBS Search and Download Visualization rates highest Search and Download Visualization Service Area Comparison

36 © 2014 CFI Group. All rights reserved. Search and Download Data ratings are in the high 70s for both the Data and FIRMS service areas. Data and FIRMS rate closely Search and Download Data Service Area Comparison

37 © 2014 CFI Group. All rights reserved. All three service areas are meeting the data delivery format needs of users with scores in the low 80s. All service areas meet user needs for data delivery format Data Delivery Service Area Comparison

38 © 2014 CFI Group. All rights reserved. FIRMS and Data service area users are similarly satisfied with both Search and Delivery attributes. Scores range from the high 70s to mid 80s. FIRMS and Data service areas on par with each other for Search and Delivery Data Search and Delivery Service Area Attribute Comparison

39 © 2014 CFI Group. All rights reserved. Ease of finding and downloading imagery scores in the low to mid 80s for all three service areas. Rapid Response real-time scores were much lower than the Wordview. Some attribute scores are not available for GIBS due to low sample. Users are satisfied with both Ease of Finding and Download of image Rapid Response Worldview and GIBS Visualization Search Comparison

40 © 2014 CFI Group. All rights reserved. Data service respondents rate Search and Download capabilities in the high 70s/low 80s. Search and Download capabilities meet Data service respondents’ needs Data Service Respondents – Search and Download

41 © 2014 CFI Group. All rights reserved. Data delivery capabilities for Data service respondents all score in the low 80s. Data service respondents are satisfied with data delivery capabilities Data Service Respondents – Data Delivery

42 © 2014 CFI Group. All rights reserved. Search and download capabilities for FIRMS users range from a high of 81 (Ease of Downloading Data) to a low of 76 (Timeliness of Browse Availability). Ease of Downloading Data scores highest for FIRMS users FIRMS Service Respondents – Search and Download

43 © 2014 CFI Group. All rights reserved. Data delivery capabilities for FIRMS service respondents users all score in the low to mid 80s. FIRMS service respondents are satisfied with data delivery capabilities FIRMS Service Respondents – Data Delivery

44 © 2014 CFI Group. All rights reserved. Rapid Response users are quite satisfied with ease of finding and downloading imagery. Real time ease of finding and image timeliness score much lower (high 50s). Real time Image Availability and Ease of Finding rate lowest Rapid Response Service Respondents – Search and Download

45 © 2014 CFI Group. All rights reserved. With scores in the high 70s to high 80s, users are generally satisfied with Worldview Search and Download capabilities. Worldview users were particularly pleased with the Ease of Finding Imagery (88). Ease of Finding Imagery scores highest for Worldview users Worldview Service Respondents – Search and Download

46 © 2014 CFI Group. All rights reserved. GIBS’ Search and Download capabilities mostly score in the mid 80s with Quality of Service Documentation at 79. GIBS users are satisfied with Search and Download capabilities GIBS Service Respondents – Search and Download

47 © 2014 CFI Group. All rights reserved. Data delivery capabilities for GIBS users all score in the low 80s. GIBS service respondents are satisfied with data delivery capabilities GIBS Service Respondents – Data Delivery

Summary and Recommendations

49 © 2014 CFI Group. All rights reserved.  The measure of Customer Satisfaction with NASA LANCE is 77, down just one point from This is 11 points above the current federal government average.  Scores for Quality Components or drivers of satisfaction were in the high 70s to mid 80s. Customers were highly likely to recommend products from NASA LANCE, as well as highly likely to use them again. Both areas had ratings in the high 80s.  Search & Download Visualization had the highest impact on satisfaction. This component measures the ease of finding and downloading imagery. Ease of Downloading Imagery (84) rose 5 significant points from last year.  Data & Imagery Delivery Format and Customer Support are the next highest impact areas. Data & Imagery Delivery Format measures the convenience and timeliness of image delivery. Convenience (83) rated slightly higher than Timeliness (81) again this year. Nearly one-quarter (22%) of customers used Customer Support and 90% of the time they received the help needed on the first request. Customer Support was still the highest rated area with a score of 85 despite significant drops in Professionalism (-4) and How Well the Issue was Handled (-3).  Although Search & Download Data and Usage of LANCE Products had the lowest impact scores among all the drivers, both still had a sizeable impact on satisfaction. Both Search & Download Data and Usage of LANCE Products were relatively flat at 78 and 82 respectively. Summary and Recommendations

50 © 2014 CFI Group. All rights reserved. The survey also collected background information about NASA LANCE users  LANCE websites (67%) and Internet Searches (50%) were the most common methods to search for products and services.  Fires (60%) vegetation (40%) and weather (39%) were the most common areas of need.  Data downloading, (58%) and FIRMS (51%) were the most commonly used LANCE NRT services.  In terms of the useful timeframe for LANCE NRT products, nearly half (48%) indicated a timeframe of less than 3 hours.  In most cases (91%), users were able to find the information they needed. Just over a quarter (28%) download data on a daily basis.  Most of the time (90%) users are able to download in their preferred format which is split between FTP (51%) and HTTPS (43%).  About one-quarter (25%) of respondents replied that they were not informed when data was not available, but the percentage that Never Experienced Unavailability (35%) rose 7 points from Summary and Recommendations

51 © 2014 CFI Group. All rights reserved. Summary and Recommendations Based on the findings from the 2014 survey, we make the following recommendations to NASA for LANCE NRT:  Search and Download Visualization is the highest impact area. Scores are good, but there may be some area to improve by increasing the ease of finding and downloading imagery. The ability to set up download criteria to enable automatic downloads would greatly streamline the Search/Download process and may improve Timeliness. This desire is best described by the following user, “It could be great if you could define your own area of interest, e.g. 4 corner coordinates, and then automatically get that NRT cut- out available for download every day.” It should be noted that Timeliness scores were much lower for Non-US locations. This may be due to lower connectivity/internet standards in some countries.  Data and Imagery Delivery Format and Method is the second highest area of impact and has a very solid rating of 82. It appears that most users are satisfied with the download format, as 90% of respondents are able to download in their preferred format. There does seem to be some room for improving the quality of images delivered however. While most users were thankful for the image, they also expressed a desire for sharper images. Any improvement in image quality would be received favorably.

52 © 2014 CFI Group. All rights reserved.  Customer Support is used by nearly one-quarter of customers (22%) and most (90%) got resolution on the first request. While Customer Support is the highest rated driver at 85, it has experienced some declines in both Professionalism (85) and How Well the Issue was Handled (83). While customer support is still an area of strength, it should be monitored to ensure the drops in Professionalism and Issue Handling are not symptoms of an underlying larger issue that could erode satisfaction in the near future. This will help ensure a high level of customer support for LANCE NRT users.  Search and Download Data and Use of LANCE NRT Products are also still high areas of impact. While scores in these areas are good, satisfaction can be positively influenced if NASA can improve in the area of ease of use. Ease of using products rates rather low (63) for those individuals who cannot download in their preferred format. Only 10% were not able to do so, but providing more formatting options (or helping convert these users to FTP or HTTPS formats) for data and imagery may be an area to improve scores for this segment of users. Summary and Recommendations

Appendix

54 © 2014 CFI Group. All rights reserved. The Math Behind the Numbers x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x 1 x 3 x 4 x 5 x 6 y1y1 y2y2 y3y3 y 3 y 2 y 1 11 22  x i xit i , for i=1,2,3 t=1,2 y jyjj  1, for j=1,2,3     x 2 A discussion for a later date…or following this presentation for those who are interested.

55 © 2014 CFI Group. All rights reserved.  Attributes (questions on the survey) are typically answered on a 1-10 scale > Social science research shows 7-10 response categories are optimal > Customers are familiar with a 10 point scale  Before being reported, scores are transformed from a 1-10 to a scale > The transformation is strictly algebraic; e.g. > The scale simplifies reporting: Often no need to report many, if any, decimal places scale is useful as a management tool A Note About Score Calculation

56 © 2014 CFI Group. All rights reserved. Deriving Impacts Remember high school algebra? The general formula for a line is: y = mx + b The basic idea is that x is a “cause” and y is an “effect”, and m represents the slope of the line – summarizing the relationship between x & y CFI Group uses a sophisticated variation of the advanced statistical tool, Partial Least Squares (PLS) Regression, to determine impacts when many difference causes (i.e., quality components) simultaneously effect an outcome (e.g., Customer Satisfaction)

Thank you CFI GROUP 625 Avis Drive Ann Arbor, MI (tel) (fax) CFI GROUP WORLDWIDE MICHIGAN (USA) - Ann Arbor ENGLAND (UK) – London SWEDEN - Stockholm ITALY - Milan CHINA - Shanghai BRAZIL - Porto Alegre