2011 ACSI Survey Summary HDF/HDF-EOS Workshop Riverdale, MD April 18, 2012 Project Background Measurement timetable Finalized questionnaire August 1, 2011 Data collection via web September 12, 2011 – October.
Download ReportTranscript 2011 ACSI Survey Summary HDF/HDF-EOS Workshop Riverdale, MD April 18, 2012 Project Background Measurement timetable Finalized questionnaire August 1, 2011 Data collection via web September 12, 2011 – October.
2011 ACSI Survey Summary HDF/HDF-EOS Workshop Riverdale, MD April 18, 2012 Project Background Measurement timetable Finalized questionnaire August 1, 2011 Data collection via web September 12, 2011 – October 18, 2011 Sending invitations spanned the first two weeks. Sending reminders spanned the last two weeks. The survey was in the field for a longer time this year for resending invitations. Topline results October 26, 2011 Results briefing November 29, 2011 Project Background Those who answered for more than one data center: Two: 103 Three: 14 Data collection Respondents • 3,996 responses were received • 3,996 responses were used for modeling Four: 2 Data Center Description Original ASDC–LaRC ASF SAR DAAC CDDIS GES DISC GHRC LP DAAC MODAPS LAADS NSIDC DAAC OBPG/Ocean Color ORNL DAAC/FLUXNET PO.DAAC-JPL SEDAC Total 2350 1371 1302 1551 678 25503 6939 5487 4893 3988 1361 2728 58151 NASA Survey Responses Emailed a Cleaned Survey Invitation 2350 2349 1370 1364 1275 1271 1544 1533 674 670 25490 25475 6839 6805 5487 5468 4893 4891 3976 3966 1352 1348 2728 2724 57978 57864 Bounce Backs Responded Response Rate 135 108 468 357 81 1477 482 619 721 197 103 148 4896 194 172 95 97 69 1849 484 398 200 229 85 124 3996 9% 14% 12% 8% 12% 8% 8% 8% 5% 6% 7% 5% 8% E-mail addresses from lists associated with some of the data centers were included to reach the large number of users who may have accessed data via anonymous ftp. NASA EOSDIS Benchmarks Strong performance continues … ACSI (Overall) Q2 2011 76 65 Federal Government (Overall) 2010 NASA EOSDIS - Aggregate 2011 77 News & Information Sites (Public Sector) 2011 75 30 40 50 60 70 ACSI (Overall) is updated on a quarterly basis, with specific industries/sectors measured annually. Federal Government (Overall) is updated on an annual basis and data collection is done in Q3. Quarterly scores are based on a calendar timeframe: Q1- Jan through March; Q2 – April through June; Q3 – July through Sept.; Q4 – Oct. through Dec. 80 NASA EOSDIS Customer satisfaction remains steady ACSI N=1016 N=1263 N=2857 N=2291 2004 2005 2006 2007 75 78 74 (+/-) 0.9 (+/-) 0.7 79 N=2601 N=3842 N=4390 N=3996 2008 2009 2010 2011 75 77 77 77 77 (+/-) 0.5 (+/-) 0.6 (+/-) 0.5 (+/-) 0.4 (+/-) 0.4 (+/-) 0.4 82 78 80 81 81 81 81 73 73 71 73 74 73 74 74 71 76 72 73 75 75 75 75 Overall satisfaction How satisfied are you with the data products and services provided by [DAAC]? Expectations To what extent have data products and services provided by [DAAC] fallen short of or exceeded expectations? Ideal How close does [DAAC] come to the ideal organization? NASA EOSDIS Model Product Search/Selection/Documentation most critical Customer Support Product Documentation 86 1.7 76 0.9 Product Selection and Order 87 77 1.1 75 Product Search 0.9 77 Recommend 3.8 Customer Satisfaction Index 89 Future Use 3.2 78 Product Quality 0.4 81 Sample Size: 3996 Delivery 0.4 Scores The performance of each component on a 0 to 100 scale. Component scores are made up of the weighted average of the corresponding survey questions. Impacts The change in target variable that results from a five point change in a component score. For example, a 5-point gain in Product Search would yield a 0.9-point improvement in Satisfaction. User background and interests questions Have you searched, ordered, downloaded data? Search questions 2011 EOSDIS Survey Overview no Did you look for or get documentation ? Delivery questions no Documentation questions Did not search Rate search Did not order Rate delivery Format questions Order questions Rate format Rate order Usage questions • Blue boxes designate general survey areas • White boxes indicate rating questions • Embedded skips are shown with arrows Have you reported a problem? Rate problem resolution Rate documentation Have you requested assistance from customer services? Did you get help 1st time? no ACSI standard 3 questions Customer Service questions ACSI outcomes 2 questions Rate customer service Thank you! no User background and interests questions 2011 EOSDIS Survey Overview 3996 Have you searched, ordered, downloaded data? Search questions no 3673 Delivery questions Did you look for or get documentation ? no 2954 Documentation questions Did not search Rate search Did not order Rate delivery Format questions Order questions Rate format Rate order Usage questions • Blue boxes designate general survey areas • White boxes indicate rating questions • Embedded skips are shown with arrows Have you reported a problem? Rate problem resolution Rate documentation Have you requested assistance from customer services? Did you get help 1st time? no ACSI standard 3 questions Customer Service questions ACSI outcomes 2 questions Rate customer service Thank you! no NASA EOSDIS 2008 – 2011 Scores hold steady; no change more than one point 77 77 77 77 Customer Satisfaction Index 86 86 85 84 81 80 81 81 Customer Support Delivery (+/-) 0.4 (+/-) 0.9 (+/-) 0.5 Product Quality 78 77 77 74 (+/-) 0.6 Product Selection and Order 77 77 76 77 (+/-) 0.5 Product Documentation 76 76 77 75 (+/-) 0.5 75 76 75 75 (+/-) 0.5 Product Search 2011 =Significant Difference vs. 2010 2010 2009 2008 Product Quality One-point gain from last year 78 77 Product Quality 77 74 78 77 Ease of using the data product in the delivered format 77 74 2011 =Significant Difference vs. 2010 2010 2009 2008 Impact=0.4 Product Quality Preferences somewhat in line with what provided GeoTIFF is most preferred format, while HDF-EOS/HDF is format in which products were provided the most. Only 8% of products provided in GIS although nearly one-quarter prefer that format. In 2010, 57% said products were provided in HDF-EOS and HDF and 42% said they were their preferred method. Format data products were provided HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services GIS KML, KMZ CEOS Don´t know Other format Number of Respondents ~Multiple responses allowed 53% 13% 9% 17% 41% 15% 1% 8% 5% 2% 4% 2% 3,673 Format preferred~ HDF-EOS/HDF NetCDF Binary ASCII GeoTIFF JPEG, GIF, PNG, TIFF OGC Web services GIS KML, KMZ CEOS OPeNDAP Other preferred format Number of Respondents 40% 20% 12% 24% 53% 18% 4% 23% 13% 2% 2% 3% 3,673 HDF-EOS/HDF Format Tools used when data was provided in HDF format Many of the respondents (687) selected ‘Other’ and listed alternate tool names or described custom approaches. Of these respondents 69 selected 'other‘ exclusively. Tools used with HDF Number % 867 818 493 509 512 506 163 73 123 144 438 109 42 96 303 1961 44% 42% 25% 26% 26% 26% 8% 4% 6% 22% 22% 6% 2% 5% 15% ENVI ArcGIS ERDAS IDL MATLAB MODIS Reprojection Tool SeaDAS Geomatica® Global Mapper IDRISI HDFView HEG NCL GrADS Other (Please specify) Number of HDF-EOS/HDF respondents ~Multiple responses allowed 2011 EOSDIS Survey Flow Overview CLB Experience with HDF Mostly high ratings but some “Ease of Use” problems HDF Users Experience Ratings 700 Ease of Use Quality of Product Usability of Data 600 Number of Respondents 500 Over 60% of the respondents rated all three areas as 8, 9 or 10.. 400 300 200 100 0 1 2 3 4 5 6 7 Ratings (10 = Excellent) 2011 EOSDIS Survey Flow Overview CLB 8 9 10 HDF User Comments Comments are both positive and negative • Survey respondents provided ~ 90 comments about their experience with HDF format, for example pertaining to – Search method “I found all of the HDF-4 files I needed easily, and in small sizes too which was a plus.” – Order processing “A mosaicking option for all data sets would be nice” – Preferences “Please no more HDF4 with irritating custom extensions” – What they are not finding “I need data in ASCII format . . . data from HDF is complicated” – Looking for documentation “Format Conversion (HDF to netcdf).” – Over half were voluntary comments or suggestions “ . . . size and complexity (HDF-format) of the data files . . . can be ameliorated with web services . . . “ • Verbatim comments are available for analysis 2011 EOSDIS Survey Flow Overview CLB Summary Satisfaction with NASA EOSDIS has held at 77 for four years. NASA continues to meet data users needs. HDF-EOS/HDF is a well supported format • Not all users are comfortable or satisfied with HDF • Comments received provide insight into users effective use and/or problems • Verbatim comments are supplied in separate word documents. Comments Verbatim comments are supplied in separate word documents. In what format(s) were your data products provided to you? (select any that apply) •Other (please specify and/or comment) Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) •Yes (Please specify which tool or tools you used to work with the data.) •No, I couldn’t find what I needed (please specify what you were looking for) •No, I couldn’t understand how to use it (please specify what you were trying to use) Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment)