1. American Customer Satisfaction Index (ACSI) Survey Results - 2009 HDF/HDF-EOS Spring Meeting Carol.L.Boquist@nasa.gov 04/01/2010 1

2. Contents 2009 EOSDIS Survey Flow ACSI Responses * Comments 2 Legend on survey flow charts: Blue boxes designate survey areas White boxes contain rating questions Yellow boxes contain the number of respondents Arrows extending outside blue boxes denote skips Boxes outlined in red or red text denote suggested changes Green outlines and boxes added for HDF/HDF-EOS * Note: The purpose of the survey is to assess user satisfaction with the data and services of the Earth Observing System Data and Information data centers. The survey is conducted by CFI Group under a contract with the Federal Consulting Group (FCG) located within the Department of Interior's National Business Center. The Federal Consulting Group, Executive Agent in government for the ACSI, works with OMB to ensure adherence to applicable Federal regulations such as the Privacy Act and the Paperwork Reduction Act.

3. Finalized questionnaireJuly 30, 2009 Data collection via webSeptember 3, 2009 – October 13, 2009 * Topline resultsOctober 23, 2009 Results briefingNovember 23, 2009 December 1, 2009 December 8, 2009 Measurement Timetable * After a spam issue was identified with receipt of email at one of the data centers, email invitations were resent and data collection was extended.

4. 3,842 responses were received 3,842 responses were used for modeling Those who answered for more than one data center: Two: 108 Three: 15 Four: 2 Data Collection

5. User background and interests How did you become aware that you could acquire Earth science data from NASA? (select any that apply) Colleague ECHO Ware House Inventory Tool (WIST)/ EOS Data Gateway Global Change Master Directory (GCMD) NASA or Data Center Web Site NASA Sponsored Research/Data Provider/Affiliate Science Conference/Workshop/Meeting Scientific Literature University (Graduate School, Course work, Classroom, Professor, Lecture, etc.) Web Search Other (please specify) How did you become aware that you could acquire Earth science data from NASA? (select any that apply) Colleague ECHO Ware House Inventory Tool (WIST)/ EOS Data Gateway Global Change Master Directory (GCMD) NASA or Data Center Web Site NASA Sponsored Research/Data Provider/Affiliate Science Conference/Workshop/Meeting Scientific Literature University (Graduate School, Course work, Classroom, Professor, Lecture, etc.) Web Search Other (please specify) Where are you currently located? drop down list Use ISO 3166, http://www.iso.org/iso/english_country _names_and_code_elements http://www.iso.org/iso/english_country _names_and_code_elements Included USA, “Taiwan, Province of China” changed to “Taiwan” Where are you currently located? drop down list Use ISO 3166, http://www.iso.org/iso/english_country _names_and_code_elements http://www.iso.org/iso/english_country _names_and_code_elements Included USA, “Taiwan, Province of China” changed to “Taiwan” For which general areas do you need or use Earth science data and services? (select any that apply) Atmosphere Cryosphere Land Human dimensions Ocean Space Geodesy Calibrated radiance Other (please specify) For which general areas do you need or use Earth science data and services? (select any that apply) Atmosphere Cryosphere Land Human dimensions Ocean Space Geodesy Calibrated radiance Other (please specify) 3842 Continue to User background and interests 2 52009 EOSDIS Survey Flow - CLB 3842

6. Respondents outside the USA have a slightly lower overall Satisfaction score with EOSDIS (76 outside vs. 77). Compared to last year there was a score increase for those respondents within the USA and a score decrease for those respondents outside the USA. Respondents outside the USA have a slightly lower overall Satisfaction score with EOSDIS (76 outside vs. 77). Compared to last year there was a score increase for those respondents within the USA and a score decrease for those respondents outside the USA. 71% of respondents are outside of the USA in 2009 vs. 68% in 2008. Satisfaction – within USA vs. outside USA

7. User background and interests (cont.) For which areas do you need or use Earth science data and services? (select any that apply) Agriculture Air quality Atmospheric Composition Carbon Cycle Climate Climate Change Cryosphere – sea ice Cryosphere – snow Ecological forecasting Ecosystems Land Cover Land Use Modeling (please specify) Natural Disasters Natural Hazards Ocean Color Ocean (sea surface height, sea surface temperature, etc.) Population Public Health Resources (Forestry, Mining, etc.) Socioeconomics Solid Earth Space Geodesy Space Weather Sun-Earth Connections Sustainability Water resources Water & energy Weather Other (please specify) For which areas do you need or use Earth science data and services? (select any that apply) Agriculture Air quality Atmospheric Composition Carbon Cycle Climate Climate Change Cryosphere – sea ice Cryosphere – snow Ecological forecasting Ecosystems Land Cover Land Use Modeling (please specify) Natural Disasters Natural Hazards Ocean Color Ocean (sea surface height, sea surface temperature, etc.) Population Public Health Resources (Forestry, Mining, etc.) Socioeconomics Solid Earth Space Geodesy Space Weather Sun-Earth Connections Sustainability Water resources Water & energy Weather Other (please specify) Have you done any of the following : searched, requested, ordered, visualized, and/or downloaded data or data products? No: Skip to Documentation Yes: Continue to Search 3675 3842 167 72009 EOSDIS Survey Flow - CLB 3842

8. For which specific areas do you need or use Earth science data and services? Demographics (when comparable) remain fairly consistent with 2008 * Multi-select question; Language to question was changed slightly in 2009; Modeling was asked as a separate question prior to 2008. Respondent Information

9. Search How did you search for the data products or services you were seeking? Data center’s or data-specific specialized search, online holdings or datapool (Please specify, e.g., DADDI, Giovanni, GISMO, GLOVIS, HyDRO, LAADS, Mercury, Mirador, MISR, MIST, NOESIS, OPENDAP, POET, SEADAS, SNOWI, URSA, WebGIS, etc.,,…) Direct interaction with user services personnel Global Change Master Directory Internet search tool (e.g., Google Earth, Google) Warehouse Inventory Search Tool (WIST)/EOS Data Gateway (EDG) Other (please specify) Did not search (skip to Order) How did you search for the data products or services you were seeking? Data center’s or data-specific specialized search, online holdings or datapool (Please specify, e.g., DADDI, Giovanni, GISMO, GLOVIS, HyDRO, LAADS, Mercury, Mirador, MISR, MIST, NOESIS, OPENDAP, POET, SEADAS, SNOWI, URSA, WebGIS, etc.,,…) Direct interaction with user services personnel Global Change Master Directory Internet search tool (e.g., Google Earth, Google) Warehouse Inventory Search Tool (WIST)/EOS Data Gateway (EDG) Other (please specify) Did not search (skip to Order) Please comment on your experience with the search method you used. Please include whether you need to search for multiple datasets occurring at the same time or place. Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate the tool … Ease of finding data Ease of using search capability How well the search results met your needs Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate the tool … Ease of finding data Ease of using search capability How well the search results met your needs Were you searching for multiple datasets occurring at the same time or place? Yes No Were you searching for multiple datasets occurring at the same time or place? Yes No 3505 170 Continue to Order 92009 EOSDIS Survey Flow - CLB 3675 3505

10. Order Did you use a subsetting tool as part of the process of requesting/ordering/downloading the data or was a subsetting tool part of the subscription process? Yes, by band Yes, by channel Yes, by geographic area Yes, by geophysical parameter Yes, by both geographic area and geophysical parameter Did not use a subsetting tool Did you use a subsetting tool as part of the process of requesting/ordering/downloading the data or was a subsetting tool part of the subscription process? Yes, by band Yes, by channel Yes, by geographic area Yes, by geophysical parameter Yes, by both geographic area and geophysical parameter Did not use a subsetting tool Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate … Ease of selecting data products Description of data products Ease of requesting/ordering data products Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate … Ease of selecting data products Description of data products Ease of requesting/ordering data products How frequently during a year do you request/order/download data products? If the order is an automatic subscription, please choose the frequency. Once Daily Weekly Monthly Quarterly As needed (please specify approximately how many times numerically) Did not order (skip to Documentation) How frequently during a year do you request/order/download data products? If the order is an automatic subscription, please choose the frequency. Once Daily Weekly Monthly Quarterly As needed (please specify approximately how many times numerically) Did not order (skip to Documentation) 74 3601 3675 Continue to Delivery 102009 EOSDIS Survey Flow - CLB 3601 Skip to Documentation

11. Delivery How was your data delivered? FTP immediate retrieval from online holdings FTP retrieved after order FTP via subscription http-based download from Web http-based batch download from Web (wget) Web-based visualization tool Other (Please specify) How was your data delivered? FTP immediate retrieval from online holdings FTP retrieved after order FTP via subscription http-based download from Web http-based batch download from Web (wget) Web-based visualization tool Other (Please specify) Which method of data delivery do you prefer? FTP immediate retrieval from online holdings FTP retrieved after order FTP via subscription http-based download from Web http-based batch download from Web (wget) Web-based visualization tool Other (Please specify) Which method of data delivery do you prefer? FTP immediate retrieval from online holdings FTP retrieved after order FTP via subscription http-based download from Web http-based batch download from Web (wget) Web-based visualization tool Other (Please specify) Using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Convenience of delivery method Timeliness of delivery method Using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Convenience of delivery method Timeliness of delivery method How long did it take for you to receive your data products? Immediate retrieve Less than an hour Less than a day 1-3 days 4-7 days More than 7 days How long did it take for you to receive your data products? Immediate retrieve Less than an hour Less than a day 1-3 days 4-7 days More than 7 days 3601 Continue to Format 112009 EOSDIS Survey Flow - CLB

12. Over half said their data came from MODIS (same in 2008) 27% said ASTER (18% in 2008) Impact=0.5 =Significant Difference vs. 2008 *Multi-Select Delivery

13. Format Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Ease of using the data product in the delivered format Overall quality of the data product Overall usability of the data product Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Ease of using the data product in the delivered format Overall quality of the data product Overall usability of the data product In what format(s) were your data products provided to you? (select any that apply) HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services (WMS, WCS, WFS, etc.) GIS (e00, shp, etc.) KML, KMZ CEOS Don’t know Other (please specify and/or comment) In what format(s) were your data products provided to you? (select any that apply) HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services (WMS, WCS, WFS, etc.) GIS (e00, shp, etc.) KML, KMZ CEOS Don’t know Other (please specify and/or comment) What format(s) would/do you prefer? (select any that apply) HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services (WMS, WCS, WFS, etc.) GIS (e00, shp, etc.) CEOS KML, KMZ Other (Please specify another format or comment on specific version, etc.) What format(s) would/do you prefer? (select any that apply) HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services (WMS, WCS, WFS, etc.) GIS (e00, shp, etc.) CEOS KML, KMZ Other (Please specify another format or comment on specific version, etc.) 3601 Continue to Usage 1 132009 EOSDIS Survey Flow - CLB HDF-EOS/HDF: 2217 GeoTIFF: 1193 HDF-EOS/HDF&GeoTIFF: 596 HDF-EOS/HDF: 2217 GeoTIFF: 1193 HDF-EOS/HDF&GeoTIFF: 596 HDF-EOS/HDF: 1686 GeoTIFF: 1707 HDF-EOS/HDF: 1686 GeoTIFF: 1707

14. In 2008, 74% said products were provided in HDF- EOS and HDF and 41% said they were their preferred method. *Multiple responses allowed Format data products were providedFormat preferred HDF-EOS/HDF58%HDF-EOS/HDF44% GeoTIFF31%GeoTIFF44% ASCII15%ASCII20% JPEG, GIF, PNG, TIFF13%GIS18% Binary10%JPEG, GIF, PNG, TIFF17% NetCDF9%NetCDF15% GIS6%Binary13% Don't know3%KML, KMZ9% KML, KMZ3%Other preferred format3% CEOS2%OGC Web services3% Other format2%CEOS2% OGC Web services1% Number of respondents3842Number of respondents3842 Product Quality

15. Impact=0.4 =Significant Difference vs. 2008 Product Quality Improved significantly this year

16. Usage What instrument data did you get? (select any that apply) AIRS Altimetry (QuikScat, JASON, etc.) AMSR-E ASTER CERES (Terra, Aqua, TRMM) DORIS GLAS (ICESat) GNSS MISR MODIS (Land) MODIS (Atmosphere) MODIS (Ocean) MOPITT OMI PR/TMI/VIRS (TRMM) SAR (ERS, JERS, RADARSAT, PALSAR) SLR/LLR TES VLBI Other (Please specify) What instrument data did you get? (select any that apply) AIRS Altimetry (QuikScat, JASON, etc.) AMSR-E ASTER CERES (Terra, Aqua, TRMM) DORIS GLAS (ICESat) GNSS MISR MODIS (Land) MODIS (Atmosphere) MODIS (Ocean) MOPITT OMI PR/TMI/VIRS (TRMM) SAR (ERS, JERS, RADARSAT, PALSAR) SLR/LLR TES VLBI Other (Please specify) 16 What types of data did you get? (select any that apply) Socioeconomic data Satellite data In-situ measurements (field campaigns, land validation, lightning, etc.) Model data Other (Please specify) What types of data did you get? (select any that apply) Socioeconomic data Satellite data In-situ measurements (field campaigns, land validation, lightning, etc.) Model data Other (Please specify) 3601 Continue to Usage 2 162009 EOSDIS Survey Flow - CLB 3373 228 3601

17. Usage (cont.) How many people are using or will use the data you received? 1 2-4 5 or more How many people are using or will use the data you received? 1 2-4 5 or more Thinking about your most recent … Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) Yes (Please specify which tool or tools you used to work with the data.) No, I couldn’t find what I needed (please specify what you were looking for) No, I couldn’t understand how to use it (please specify what you were trying to use) No, didn’t need (Need to add in 2010 survey) Thinking about your most recent … Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) Yes (Please specify which tool or tools you used to work with the data.) No, I couldn’t find what I needed (please specify what you were looking for) No, I couldn’t understand how to use it (please specify what you were trying to use) No, didn’t need (Need to add in 2010 survey) Are you generally finding what you want in terms of type, format, time series, etc.? Yes No (Please specify and/or comment on what you want but are not finding.) Are you generally finding what you want in terms of type, format, time series, etc.? Yes No (Please specify and/or comment on what you want but are not finding.) Did you look for or get documentation? 17 No: Skip to Customer Service Yes: Continue to Documentation 1049 2793 3601 172009 EOSDIS Survey Flow - CLB 3601 From Did not search/order/../download From Searched but did not order 74 167 3842 Y: 3368 N: 233 Y: 3368 N: 233

18. Documentation What documentation did you use or were you looking for? Instrument specifications Science algorithm Product format Tools Science applications Data product description Production code Other What documentation did you use or were you looking for? Instrument specifications Science algorithm Product format Tools Science applications Data product description Production code Other Was the documentation Delivered with the data Available online Not found (Skip to Customer Service) Was the documentation Delivered with the data Available online Not found (Skip to Customer Service) Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Overall quality of the document (i.e., technical level, organization, clarity) Extent to which the data documentation helped you use the data Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… Overall quality of the document (i.e., technical level, organization, clarity) Extent to which the data documentation helped you use the data Have you requested assistance from ’s user services office during the past year? No: skip to ACSI Yes: Continue to Customer Service 2793 1080 2554 2762 239 1049 From did not look for Documentation 182009 EOSDIS Survey Flow - CLB 3842

19. What documentation did you use or were you looking for? Data product description 45% Product format 16% Science algorithm 14% Instrument specifications 7% Tools 6% Science applications 6% Production code 1% Impact=1.0 CSI for those whose documentation was not found is 69 vs. those who got it delivered with the data (78) or online (78). =Significant Difference vs. 2008 Was the documentation… Delivered with the data (18% vs. 16% in ‘08) Available online (73% vs. 70% in ‘08) Not found (9% vs. 14% in ‘08) Product Documentation

20. Customer Service Think about the user services staff you interacted with when you requested assistance from user services. On the same scale from 1 to 10 where 1 means “Poor” and 10 means “Excellent,” how would you rate the user services staff on… Professionalism Technical knowledge Accuracy of information provided Helpfulness in selecting/finding data or products Helpfulness in correcting a problem Timeliness of response Think about the user services staff you interacted with when you requested assistance from user services. On the same scale from 1 to 10 where 1 means “Poor” and 10 means “Excellent,” how would you rate the user services staff on… Professionalism Technical knowledge Accuracy of information provided Helpfulness in selecting/finding data or products Helpfulness in correcting a problem Timeliness of response Was it By phone By E-mail Both by phone and e-mail Was it By phone By E-mail Both by phone and e-mail Continue to ACSI 1080 202009 EOSDIS Survey Flow - CLB

21. Did you request assistance from the Data Center’s user services staff during the past year? No=72%. Of those who said yes, 84% used email, 3% used the phone, and 12% used both phone and e-mail. 89% (91% in 2008) were able to get help on first request. These respondents continue to have a significantly higher CSI (81) than those who did not (65). Impact=1.6 =Significant Difference vs. 2008 Customer Support

22. Outcomes 3 ACSI standard Using a 10-point scale on which 1 means “Very Dissatisfied” and 10 means “Very Satisfied,” how satisfied are you with the data products and services provided by ? Using a 10-point scale, on which “1” means “Not at all likely” and “10” means “Very likely,” how likely are you to use the services provided by in the future? Using a 10-point scale on where “1” means “Not at all likely” and “10” means “Very likely,” how likely are you to recommend to a colleague? Now, imagine an ideal provider of scientific data products and services. How close does come to that ideal organization you just imagined? Please use a 10- point scale on which 1 means “Not at all close to the ideal,” and 10 means “Very close to the ideal.” Using a 10-point scale on which 1 now means “Falls short of your expectations” and 10 means “Exceeds your expectations,” to what extent have the data products and services provided by fallen short of or exceeded your expectations 3842 Continue to Reported a problem 2762 1080 222009 EOSDIS Survey Flow - CLB Skipped Customer Service

23. Closing Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment) Your survey responses have been received. NASA appreciates your input and will use this feedback to better serve its customers. Your survey responses have been received. NASA appreciates your input and will use this feedback to better serve its customers. You have reached the end of the survey. Please click on the "Finish" button below to send your responses to CFI Group's secure database. 23 Reported a problem Using a 10-point scale on which “1” means “handled very poorly” and “10” means “handled very well”, please rate how well the problem was handled. Have you ever contacted to report a problem? No Yes Were you able to get the help you needed on your first request for assistance? Yes No Were you able to get the help you needed on your first request for assistance? Yes No 2965 3842 877 3842 232009 EOSDIS Survey Flow - CLB

24. 2006 ATTRIBUTES Ideal How close does [DAAC] come to the ideal organization? Overall satisfaction How satisfied are you with the data products and services provided by [DAAC]? Expectations To what extent have the data products and services provided by [DAAC] fallen short of or met your expectations? ACSI 72 7871 74 76 82 73 78 2005 2007 75 8073 2008 77 8174 75 71 79 73 75 2004 (+/-) 0.9 (+/-) 0.7 (+/-) 0.5 (+/-) 0.6 (+/-) 0.5 N=1016N=126 3 N=2857N=2291N=2601 2009 77 8173 75 (+/-) 0.4 N=3842 Customer Satisfaction Results

25. For those answering the survey over multiple years, scores have seen some movement No significant differences were see between 2008 and 2009 for those who have answered the survey over the last four years. Customers over Multiple Years

26. For those answering the survey in 2007, 2008 and 2009, Product Documentation saw a statistically significant score increase from 2008 to 2009. Customers over Three Years

27. For those answering the survey in 2008 and 2009, there are many statistically significant score increases. Customers over Two Years

28. These areas have remained consistent year over year. Top Improvement Priority Product Search (75) Product Selection and Order (76) Product Documentation (77) Areas of Opportunity

29.  NASA EOSDIS has made significant improvements versus last year in multiple areas (Product Documentation, Quality and Customer Support)  Product Selection/Order saw a small but significant decrease this year  Product Search, Selection and Order continue to be the top opportunities for improvement  Documentation continues the trend with a higher impact this year and also continues to be a top opportunity  Customer Support continues to be high impact for those who require it. Imperative to maintain the strong level of service. Summary Ensure that those who are providing customer support realize how important it is and how it affects satisfaction.

30. Comments Verbatim comments are supplied in separate word documents. In what format(s) were your data products provided to you? (select any that apply) Other (please specify and/or comment) Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) Yes (Please specify which tool or tools you used to work with the data.) No, I couldn’t find what I needed (please specify what you were looking for) No, I couldn’t understand how to use it (please specify what you were trying to use) Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment)