1. 1 MyLifeBits: Attempting to realize the Memex Vision Gordon Bell February 2003 http://research.microsoft.com/barc/MediaPresence/MyLifeBits.aspx With Jim Gemmell & Roger Lueder

2. 2 Outline … MyLifeBits Background…fulfilling the Memex vision Background…fulfilling the Memex vision Cyberizing everything Cyberizing everything File to database transition File to database transition Use…beyond search Use…beyond search Long-term agenda and outlook Long-term agenda and outlook

3. 3 Memex Posited by Vannevar Bush in “As We May Think” The Atlantic Monthly, July 1945 “A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility” Supports: Annotations, links between documents, and “trails” through the documents “yet if the user inserted 5000 pages of material a day it would take him hundreds of years to fill the repository, so that he can be profligate and enter material freely”

4. 4 Sketch of memex

5. Bush’s camera on the head

6. 6 Capturing what you see

7. 7 Memory Overload As hard drives get bigger and cheaper, we're storing way too much. By Jim Lewis There's a famous allegory about a map of the world that grows in detail until every point in reality has its counterpoint on paper; the twist being that such a map is at once ideally accurate and entirely useless, since it's the same size as the thing it's meant to represent. There's a famous allegory about a map of the world that grows in detail until every point in reality has its counterpoint on paper; the twist being that such a map is at once ideally accurate and entirely useless, since it's the same size as the thing it's meant to represent.

8. 8 "The PC is going to be the place where you store the information and really the center of control“ Billg 1/7/2001 MyLifeBits is a project to “cyberize” everything! What? Recall of all articles, books, CDs, photos, video, communication (e.g. mail, phone), web What? Recall of all articles, books, CDs, photos, video, communication (e.g. mail, phone), web Why? …“because we can” Why? …“because we can” Office: communicate, store, & work Office: communicate, store, & work Home & Media Center: ambiance &entertainment Home & Media Center: ambiance &entertainment Immortality for progeny. Memory aids Immortality for progeny. Memory aids Goal: to understand the 1 TByte PC c2006: need, utility, cost, feasibility and tools. Goal: to understand the 1 TByte PC c2006: need, utility, cost, feasibility and tools.

9. 9 Gordon: Researcher, consumer, computer system tester, nerd wanna-be, and average man Melissa: middle manager Patrick: Consultant Nicholas: Analyst Sondra: Office manager Knowledge worker scenarios

10. 10 The guinea pig Gordon Bell is digitizing his life Gordon Bell is digitizing his life Has now scanned virtually all: Has now scanned virtually all: Books written (and read when possible) Books written (and read when possible) Personal documents (correspondence including memos and email, bills, legal documents, papers written, …) Personal documents (correspondence including memos and email, bills, legal documents, papers written, …) Photos Photos Posters, paintings, photo of things (artifacts, …medals, plaques) Posters, paintings, photo of things (artifacts, …medals, plaques) Home movies and videos Home movies and videos CD collection CD collection And, of course, all PC files And, of course, all PC files Now recording: phone, radio, TV (movies), web pages… conversations? Now recording: phone, radio, TV (movies), web pages… conversations? Paperless throughout 2002. 12” scanned, 12’ discarded. Paperless throughout 2002. 12” scanned, 12’ discarded. Only 30 GB!!! Only 30 GB!!!

11. 11 Capture and encoding

12. 12 I mean everything

13. 13 Input: tools, time, and cost Scanners: HP Digital Sender, flat beds with ADF, 2-HP photo, faxing. (Duplex, color, feed-thru, etc.) Scanners: HP Digital Sender, flat beds with ADF, 2-HP photo, faxing. (Duplex, color, feed-thru, etc.) A good commercial scanner costs 2K-10K A good commercial scanner costs 2K-10K Photos: $1 or 0.5-5 min. Large posters: ~ 1-5 hr. Artifacts: ~ 10 min. including photo Photos: $1 or 0.5-5 min. Large posters: ~ 1-5 hr. Artifacts: ~ 10 min. including photo Scanning to TIF, PDF: <1 min/page or.10/page Scanning to TIF, PDF: <1 min/page or.10/page OCR: for MODI or PDF: ~3-5 pages/min (old data) OCR: for MODI or PDF: ~3-5 pages/min (old data) OCR: to recreate an editable “original” 10 min/page! OCR: to recreate an editable “original” 10 min/page! OCR (Volume paper files): 400 pages/hr. 7 ppm. OCR (Volume paper files): 400 pages/hr. 7 ppm. Books: scanned at CMU ($10 - 100/book) in 1997 Books: scanned at CMU ($10 - 100/book) in 1997 Videos: tbd Videos: tbd

14. 14 Music 6.9 GB 1.8K files 180 CDs Working 2.3 GB 432 folders 2.9K files Archive 5.1 GB 477 folders 18.7 K files Video 2.6 GB 10 hours Low res My Books 98 MB 27.1K files & 42K.msg 17.7 GB (by size) Files (by number).xls.jpg.doc/html.pdf.ppt/ppt albums.tif CyberAll Nov.1, 2001 Mail.7 GB 43K msgs.doc/html.jpg.gif.xls.pdf.ppt.tif.gif

15. 15 gbell wag: 67 yr, 25Kday life

16. 16 gbell wag: 67 yr, 25Kday life

17. 17 MyLifeBits organization: time and space Timeline/ Context (space) Personal (some $s) GB Co. (angel, etc.) Professional ACM, etc., … @Microsoft.com, New co’s. Archival (time) Working

18. 18 MyLifeBits: Some Lives(t) Personal Personal Parents, children, grandkids Parents, children, grandkids CGB himself CGB himself Close friends Close friends GB $s GB $s Personal incl. several legal structures Personal incl. several legal structures Investments & boards Investments & boards Past companies/organiz’ns Past companies/organiz’ns DEC DEC Carnegie-Mellon U. Carnegie-Mellon U. DEC, NSF, Encore, Ardent, GB_consulting, DEC, NSF, Encore, Ardent, GB_consulting, CGB@ Microsoft CGB@ Microsoft MLB MLB Clusters Clusters Telepresence Telepresence WWW presence WWW presence Computer History Museum Computer History Museum BOD member BOD member Fund-raising Fund-raising CyberMuseum CyberMuseum Startups Startups Bell-Mason Director Bell-Mason Director Diamond & Vanguard Brds. Diamond & Vanguard Brds.

19. 19 MyLifeBits is: Memex and more (audio and video) Memex and more (audio and video) Universal store for all personal stuff Universal store for all personal stuff Guiding principles for the system: Guiding principles for the system: 1. Full text search & collections (> than hierarchy) 2. Visualizations for search, display, insight 3. Annotations and links add value and essential Increase search ability and value of information. Increase search ability and value of information. So make many kinds and them easy to create! So make many kinds and them easy to create! Stories are the ultimate annotation Stories are the ultimate annotation 4. Keep the links when you author: “transclusion”

20. 20 MLB database: size and content? Database features are essential: Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication. Database features are essential: Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication. Folders &Files were the starting point >> database into sets aka “collections” that are identical to the folder structure Folders &Files were the starting point >> database into sets aka “collections” that are identical to the folder structure Outlook (msgs, attachments, calendar, contacts) Outlook (msgs, attachments, calendar, contacts) Web trails including voice message annotation Web trails including voice message annotation Journal (Outlook), trails: every document use & transaction Journal (Outlook), trails: every document use & transaction What about? What about? Money (transactions, payees, etc.)…is their lifelog/trail Money (transactions, payees, etc.)…is their lifelog/trail Streets and trips to cross-index to all docs Streets and trips to cross-index to all docs Attributes for photos for retrieval? Location, time, settings Attributes for photos for retrieval? Location, time, settings Presentations as a report or trail. Each slide an object! Presentations as a report or trail. Each slide an object!

21. 21

22. 22 Media center 2

23. 23 CD VCR Cassette Plasma Panel DVD Media Center Computer Set top KbdMse Wfr Spkr IR Cable/ Satellite Ethernet SVHS-wide 5.1 digital 5 speakers stereo Video* 5.1 digital comp. stereo Video* Cables/links Speaker 5+1 Plasma 2 or 3 Cable/Enet 2 IR 8 Stereo 4 5.1 digital 2 Comp./S-video 3 Plasma panel 1 Power 10 Kbd/mse 2 Monitor II (opt.) 4 Camera 2 Total 42 – 46 Things 18+remotes *Video = composite or S-video Camera Mic Receiver L egacy R edundant

24. 24 Photos

25. 25 Caneel Bay Vacation Jan. 1998 Gordon, Gwen, Brig, Pam, Fiona, Bob, Laura and Kolbe

26. 26 Searching: the most useful app? Challenge: What questions for useful results? Challenge: What questions for useful results? Lots of ways to look at what you retrieve Lots of ways to look at what you retrieve Need for breaking the returns into segments Need for breaking the returns into segments Searching for an indexer and search engine: index service, Enfish, dtSearch Searching for an indexer and search engine: index service, Enfish, dtSearch Stuff I’ve Seen MSR’s index & search… evolving in the right direction. Stuff I’ve Seen MSR’s index & search… evolving in the right direction. Productizing would remove the pressure for Longhorn Productizing would remove the pressure for Longhorn

27. 27

28. 28

29. 29

30. 30 Detail view

31. 31 Resource explorer Ancestor (collections), annotations, descendant & preview panes turned on

32. 32 Interface to xls

33. 33

34. 34 Statistics of use

35. 35 Synchronized timelines with histogram guide

36. 36 Visualization Browsing & searching. “Get me what I want|need!” Browsing & searching. “Get me what I want|need!” Help the user find things among possible items versus Help the user find things among possible items versus Waiting for an ideal system that can find “what I want” Waiting for an ideal system that can find “what I want” Publication: Conventional & web, presentations, etc. Publication: Conventional & web, presentations, etc. Helps understand the nature of the content e.g. histogram of objects in time Helps understand the nature of the content e.g. histogram of objects in time Context: Links to help understand the relationship between objects. Provides more search handles. Context: Links to help understand the relationship between objects. Provides more search handles. Information density: what is it? What is its relationship to others? Information density: what is it? What is its relationship to others? Content important. Flash and form, less useful. Content important. Flash and form, less useful.

37. 37 Value of media depends on annotations “Its just bits until it is annotated” “Its just bits until it is annotated”

38. 38 System annotations provide base level of value Date 7/7/2000 Date 7/7/2000

39. 39 Tracking usage – even better Date 7/7/2000. Opened 30 times, emailed to 10 people (its valued by the user!) Date 7/7/2000. Opened 30 times, emailed to 10 people (its valued by the user!)

40. 40 Get the user to say a little something is a big jump Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC dim sum intern farewell Lunch” Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC dim sum intern farewell Lunch”

41. 41 Getting the user to tell a story is the ultimate in media value A story is a “layout” in time and space A story is a “layout” in time and space Most valuable content (by selection, and by being well annotated) Most valuable content (by selection, and by being well annotated) Stories must include links to any media they use (for future navigation/search – “transclusion”). Stories must include links to any media they use (for future navigation/search – “transclusion”). Cf: MovieMaker; Creative Memories PhotoAlbums Cf: MovieMaker; Creative Memories PhotoAlbums Dapeng was an intern at BARC for the summer of 2000 We took him to lunch at our favorite Dim Sum place to say farewell At table L-R: Dapeng, Gordon, Tom, Jim, Don, Vicky, Patrick, Jim

42. 42 Value of media depends on annotations Auto-annotate whenever possible e.g. GPS cameras Auto-annotate whenever possible e.g. GPS cameras Make manual annotation as easy as possible. XP photo capture, voice, photos with voice, etc Make manual annotation as easy as possible. XP photo capture, voice, photos with voice, etc Support gang annotation Support gang annotation Make stories easy Make stories easy “Its just bits until it is annotated”

43. 43

44. 44 MyLifeBits use scenarios 1. Acquire everything! (I mean everything!_ 2. Professional personal use at work! 3. Home/personal: Provide ambiance & entertainment using Home Media Center 4. Enhancing content through photo and video albums Events, places, trips, people, time intervals ---------- Database land and authoring -------- 5. How I spend my time or an interval of time. Recall a “trail“… What was I thinking about? 6. Endless need for authoring & reporting tools ► ISBQ: Interactive Story By Query ► A Person (auto- or -biography web hosted time line ► Personal/web/org. hosted collections & catalogs

45. 45 The Agenda for the Tbyte(s), Lifetime, PC: The killer app after office and mail. 1. Guarantee that data will live forever! “dear appy” problem 2. Cheap, easy, and data-rich (e.g. time, place) capture: GPS and time everywhere Paper capture has to be as easy as discard (scanner/shredder) E-book…e-magazines & journals need to have critical mass! Telephony and audio capture with indexing Media Center compatible for entertainment (photos, video, TV, radio) 3. One? dbase for all books, conversations, mail, web pages …vs. long-term use of hierarchical files. Is dbase intuitive? 4. Annotations/meta-information add every-increasing value Ease of annotation because it aids search and becomes the content Content analysis (critical for photo & video!) 5. Information control: privacy, security, expunge/deniability,… 6. New “killer apps”: alzheimer, immortality, surrogate memory? 7. Any GUI to improve use (e.g. time to learn, use, retention)

46. 46 The End

47. 47 The “dear appy” problem Dear Appy, How committed are you? Please come back to me, Lost and forgotten data Who’s responsible? Who’s responsible? media media platform, file, and databases platform, file, and databases evolving standards and formats evolving standards and formats evolving and/or disappearing apps evolving and/or disappearing apps

48. 48 Digitizing our lives Right now, it is affordable to buy 100 GB/year Right now, it is affordable to buy 100 GB/year In 5 years 1TB/year is afforadable! In 5 years 1TB/year is afforadable! It’s hard to fill a terabyte/year just by keeping what you see or hear, but you can: It’s hard to fill a terabyte/year just by keeping what you see or hear, but you can: Look at 9800 pictures a day (300 KB JPEGs) Look at 9800 pictures a day (300 KB JPEGs) Read 2900 documents a day (1MB files) Read 2900 documents a day (1MB files) Listening to audio or view compressed video 24 hours/day (it takes more than 256 kb/s to fill a TB in a year) Listening to audio or view compressed video 24 hours/day (it takes more than 256 kb/s to fill a TB in a year) Watch 1.5 Mb/s video 4 hours each day. Watch 1.5 Mb/s video 4 hours each day. As Bush said, we can “be profligate and enter material freely” As Bush said, we can “be profligate and enter material freely”