1 Review of Government Agency Surveys and Interviews Terry Weaver Joy Gatewood-Fulton

2 The Accessibility Forum: Interviews Interviews with Federal Agencies regarding the Procurement Challenges of Section 508: acquiring accessible Electronic and Information Technology.

3 Section Interviews with Government Procurement (1 of 2) Nine agencies interviewed: from Depts. of HHS, Commerce, Treasury, and Defense. Purpose: to gather best practices in procuring accessible E&IT and to determine if agencies need further guidance in implementing Section 508.

4 Section Interviews with Government Procurement (2 of 2) Interviews were Confidential, with data presented in an aggregate form only. Range –Section 508 applicability to a purchase –VPATs –What 508 information do they require from vendors –Comments –Other

5 Implementing Section Question One How do you determine whether or not Section 508 applies to a particular procurement? –44% Section 508 applies to all E&IT procurements –56% We have tools and policies to advise of appropriate determinations and when it is not applicable

6 Implementing Section Question Two How do you document the results of your determination (whether or not Section 508 was applicable)? –22% - No formal documentation process –33% - An in-house tool leads users to the appropriate form or checklist that is used to document the determination and decision, including any exemption –56% - Other Other includes: Reports, Templates, Procurement Requests, VPATs,or other vendor supplied information

7 Implementing Section Question Three How do you determine which parts of the 508 standard apply to a purchase? –44% - The requiring official determines which applies –56% - Other: Guidance from the organization's Section 508 committee Often it's a matter of interpretation and is ultimately the Contracting Officer's call The standard is applied equally across the board The in-house tool guides the user. Weighted standards are even possible (one provision can be more heavily weighted than another can) We rely on the vendor to provide us that information

8 Implementing Section Question Four What process or procedure do you follow to determine whether or not vendor products, deliverables, and services meet the requirements of Section 508? –33% - Vendors are required to self certify that products meet Section 508 –22% - Market research which may include VPAT's –44% - A combination of both

9 Implementing Section Question Five What Information do you require of Offerors / Vendors? –33% - Self-Certification Language –33% - VPAT/template –22% - Vendor must complete our template –11% - Vendor must offer information about how their products meet Section 508 standards somewhere on their website, in brochures or handouts, in a standard template or VPAT format

10 Implementing Section Question Six How often do you use the Voluntary Product Accessibility Template (VPAT) in your assessment of a vendors products, deliverables, and services? –44% - Never –22% - Always –33% - Sometimes, when available

11 Implementing Section Question Six-A Is the VPAT the primary source of information? –44% - N/A - never use –44% - Yes –11% - No

12 Implementing Section Question Seven What other sources of information do you use? –44% - Market research –33% - N/A –11% - We also may test the product –11% - Our own template

13 Implementing Section Question Eight If the VPAT is used, we would like to follow up with some questions on how well it is working for you. What do you like about it? –44% - N/A –33% - Everything - easy to read and all the information is on one page –22% - Nothing

14 Implementing Section Question Nine (1 of 2) What do you dislike about it?(VPAT) –44% - N/A –22% - Nothing –56% - Other

15 Implementing Section Question Nine (2 of 2) Comments included: –Too complex –Ambiguous –Vendor simply states "meets" to every criteria - we ask the vendor to customize a VPAT that states how their product or service meets the criteria –Language is too technical for most requiring officials –People get confused and don't know what parts apply –Vendors often complete the VPATs incorrectly

16 Implementing Section Question Ten The next few questions deal with your opinion of the quality of information provided by the VPAT. Have you had to go back to the vendor with questions about the information provided? –33% - Yes –22% - No –44% - N/A

17 Implementing Section Question Eleven How many times? (Did you have to go back to the vendor regarding the VPAT) –11% - Dont Know –22% - Many –67% - N/A

18 Implementing Section Question Twelve The next few questions deal with your opinion of the quality of information provided by the VPAT. How difficult was the VPAT to use? (On a scale 1 to 5 where 1 is not difficult at all and 5 is extremely difficult) –33% - Not Difficult –11% - Difficult –56% - N/A

19 Implementing Section Question Thirteen The next few questions deal with your opinion of the quality of information provided by the VPAT. Did the VPAT save you time? –33% - Yes –11% - Not –56% - N/A

20 Implementing Section Question Fourteen How much confidence do you have in the accuracy of the information provided by the VPAT? (On a scale of 1 to 5 where 1 is little or no confidence and 5 is full confidence) –56% - N/A –22% - Full Confidence –11% - Fairly Confident –11% - No opinion

21 Implementing Section Question Fifteen Do you perform any technical evaluations of the products, deliverables, and services themselves to determine 508 compliance? –56% - No –44% - Yes

22 Implementing Section Question Sixteen After a product is delivered, are its 508 compliance claims verified? –78% - No –11% - Yes –11% - Dont Know

23 Implementing Section Question Seventeen Does your agency conduct 508 training? –100% - Yes

24 Implementing Section Question Eighteen To who is the training provided (who is the principal audience)? –56% - All employees –11% - Contract Force –11% - Program Officers –11% - ROs, COs, COTRs, Legal Staff –11% - ROs, COs, COTRs

25 Implementing Section Question Nineteen (1 of 2) Do you think there is any area where additional training is required? –44% - No –56% - Yes Comments: –Yes, eventually we would like to spread out to the program offices. –Yes, more should be done for general employees since anyone can be an RO at any time. May explore doing this as part of the new employee orientation.

26 Implementing Section Question Nineteen (2 of 2) More Comments –Make this training ongoing and part of a regular awareness programs. –Many ROs need training tailored to the products they are responsible to buy for the agency. The current training tends to handle the subject too broadly without enough hands-on examples. Small and disadvantaged businesses as well as value added and reseller vendors dont understand 508. –Future surveys will help us determine this.

27 Implementing Section Question Twenty What could raise your level of confidence in what youre buying meets 508 requirements? Another way to ask this question might be What information or other resources are lacking or would be helpful to you in determining whether or not an offeror can meet the requirements of Section 508? 44% - Some sort of logo, icon or other labeling program to confirm that the product is compliant would help. Like the UL laboratories icon on electrical products. If the products were certified as Section 508 compliant by an impartial 3rd party testing lab.

28 Implementing Section Question Twenty-one Do you have any examples of particularly good vendor responses regarding 508 requirements? (Could you provide us with a copy of anything that have been or can be sanitized and would represent good examples)? –33% - No –22% - Dell –11% - We have an database of customized VPATs, but they were proprietary to each purchase –11% - Oracle –11% - Acrobat and Dell have good VPATs –11% - Ricoh, Canon, and Sharp

29 Implementing Section Question Twenty-two Have you made any purchases where the award decision has hinged on 508 requirements? If yes, what kinds of products and services were purchased? –78% - No –22% - Dont Know –11% - Yes We recently needed to buy thousands of printers and one vendor said that they could not provide printers that were compliant. We approached another vendor who provided printers that are compliant. We have had similar experiences with copying machines.

30 Implementing Section Question Twenty-three What is your biggest challenge in doing a 508 compliant purchase? (What would make your job easier in dealing with Section 508)? –22% - Lack of dedicated resources (money and staff) to do justice to the 508 requirements –22% - Simply having a bigger pool of 508 compliant products –22% - What would be really helpful would be a seal of approval from an agency or independent body as to how well a product or service met Section 508 requirements. The example of the Green Star Energy logo used by EPA comes to mind.

31 Agencies Best Practices Pages (1 of 2) DOE: NOAA: ARNET/FAR:

32 Agencies Best Practices Pages (2 of 2) GSA - the definitive guide and buy accessible portal Usability.gov The recent recipient of a Hammer award

33 The Accessibility Forum: Interviews Interviews with Federal Agencies regarding the Procurement Challenges of Section 508: acquiring accessible Electronic and Information Technology Interviews conducted over August - September 2002 Interviews conducted by Forum staff and GSA Staff