1. Making online federal agency information more accessible
Discussion document
December 14, 2006
Google Confidential

2. What we’re talking about
Intranet: Not an agency’s intranet or internal information
Site: Nor search within an agency’s public site
Web: Making content on an agency’s public site accessible to web search engine users
WEB
SITE
INTRANET
Google Confidential
Google Confidential

3. The opportunity
But online government information is not always easy to find
Most U.S. internet users seek government information online
WEB Searchable
DARK WEB Not searchable
73% of U.S. adult population is online
70% reach NIH sites through search engines (vs. 4% directly)
Up to 40% of content on some agency sites is uncrawlable, thus invisible to most search engine users
Source: Pew Internet & American Life Project (2006), ComScore Research, July 2006
Google Confidential
Google Confidential

4. Citizen users expect to find everything
The long tail of federal government information
Obscurity of query
Number of queries
irs refund
fda drug approval
veteran record deceased
ftc hearing sherman act 1992
tree faller nps yosemite
Google Confidential
Google Confidential

5. The Solution: Sitemap protocol
Enables a web publisher to proactively manage search engine crawling
“The launch of Sitemaps is significant because it allows for a single, easy way for websites to provide content and metadata to search engines”
—Tim Mayer, Senior Director of Product Management, Yahoo Search
“We are 100% behind this protocol - this kind of collaboration will help improve the search experience for all of our customers”
—Ken Moss, General Manager, Live Search
Sitemap protocol developed by Google in June 2005 and released under Creative Commons License
Adopted as an industry standard in November 2006:
Google Confidential
Google Confidential

6. XML sitemap XML: a comprehensive list of URLs in XML
Tagged with each URL’s location, last modification, change frequency and priority
XML: a comprehensive list of URLs in XML
Google Confidential
Google Confidential

7. Our approach and progress
Since May 2006
Direct outreach
Public speaking: GSA web managers event
Progress:
Opportunity through
collaboration
Potential
Indexed Content
Contacted
~40
Implementing ~5
Getting started
~15
No progress
~20
Current
trajectory
Time
Google Confidential
Google Confidential

8. Summary
Citizen users seek government information through web search engines
A significant segment of public government information is invisible to search engine users
The Sitemap protocol offers an easy solution:
No direct cost: Generally only a time investment
Non-proprietary: Based on open standard
No security risk: Implementation occurs on public side of site
Public content only: Limited to content on public site
Google Confidential
Google Confidential