1. Sally Waselik Joanne Vara 2005 Search Plan for IEEE Web Sites August 2005

2. 2005 Search Plan for IEEE Web Sites 1. Identify and Evaluate Technology Aug 2. Compare Features, Performance and UsabilityAug 3. Evaluation and Comparison of Software Sept (features, functionality, cost) 4. Make Recommendations Sept/Oct To Web Steering Committee, ExCom, ITSC 5. Post Evaluation Implementation PlanSept/Oct 6. Ongoing Usability Testing2006 7. Ongoing Performance Tuning2006 8. Ongoing Evaluation of Testing2006

3. 2005 Search Plan for IEEE Web Sites 1. Identify and Evaluate Technology l Verity Options l Current technology – Ultraseek minimum l Upgraded Version of Ultraseek l Verity K2 l Google Options l Google Free Domain Specific l Google Appliance

4. 2005 Search Plan for IEEE Web Sites 2. Compare Features, Performance and Usability Perform (In-House) Test l Identify user set = 175 potential participants l Representative of user base l ExComm, Web Steering Committee, Web Advisory Group, ITSC, OU’s, etc l I dentify set of queries for test l Pre-selected from historical data (actual searches) l Pre-Selected strategic search terms from WSC l User-Selected l Create user worksheet and ranking criteria l Installation, configuration and set up l Index and match all sets so that test is equal l Set up test procedure l Invite participants and conduct test l Tally results

5. 2005 Search Plan for IEEE Web Sites 2. Compare Features, Performance and Usability Evaluate Approach l Fair evaluation of high level and deep search terms l Define parameters of ieee.org index – Determine sites that we want to index l Currently – over 100 sites are indexed l Determine how deep we want to index – Xplore (full text/abstracts), societies, standards working groups, agendas, minutes l Determine what we want to eliminate from index - if we serve up too many results to users it diminishes the value of the results l Determine what portal/interface/page we want to send users to based on their search requests. l Determine if new portal pages should have priority in ranking l Determine how we present results to user l Logical ranking l Provide logical suggestions and options as alternatives l Consistent interface l Group or categorize advanced search - i.e., http://search2.computer.org/advanced http://search2.computer.org/advanced

6. 2005 Search Plan for IEEE Web Sites 2. Compare Features, Performance and Usability Evaluate Approach What level of depth should ieee.org search engine reach ? Associated Portals and Extranets/Password Protected Sites Intranets – staff-web-ieee.org Staff-web-ieee.org www.ieee.org What is the user expecting when they search at this level? Logged in areas = Xplore, Jobsite, myIeee Action based utilities = Shop, Join, Renew others Do we diminish results by going too deep? societiesvolunteers

7. 2005 Search Plan for IEEE Web Sites 2. Compare Features, Performance and Usability Timeline for In House Testing l Google Preparation l Google representatives came to IEEEAug 1 l Concurrence from Google to extend evaluation Aug 2 l Google provided Framework for PilotAug 3 l Google InstallationAug 4 l Set Up Google ConfigurationAug 5 l Google Indexing completedAug 18 l Create User worksheet and ranking criteriaAug 8-18 l Define user test groups Aug 8-18 l Define Search Terms Aug 8-18 l Run Tests Aug 19-26 l Roll up Results Aug/Sept l Evaluate Results Sept l Define options/make recommendationsSept/Oct l Socialize options/recommendation/Next StepsSept/Oct

8. 2005 Search Plan for IEEE Web Sites 3. Final Evaluation and Comparison of Software Final Evaluation Based on Four Criteria Test Results Features - tuning capabilities, use of ranking algorithms Functionality - best results, adaptability to our environment, maintenance and resource issues Cost – Included in new initiative for 2006 implementation.

9. 2005 Search Plan for IEEE Web Sites Post Evaluation 4. Recommendations to Web Steering Committee, ExCom and ITSCSept/Oct 5. Post evaluation Implementation PlanSept/Oct 6. Implementation of new software 2006 7. Ongoing Usability Testing2006 8. Ongoing Performance Tuning2006 9. Ongoing Evaluation of Testing2006