Guardian Angel Technologies: Providing Right Information to the Right People Raj Reddy Carnegie Mellon University June 2, 2014

Evolution of AI Dreams mergence o In 1950s and 60s, Computer Scientists Speculated on the Likely Emergence of Computers That Are Intelligent In 1970s and 80s, Speculations Changed Somewhat to The Creation of Superhuman Capabilities Rather than building autonomous intelligences, it was seen that creating systems around man-machine symbiosis could lead to superhuman capabilities. Development of Cognition Amplifiers that Lead to Superhuman Capabilities is now the Focus of Interest With the emergence sensor rich mobile devices, we can continuously monitor activities of the user, analyze and discover task being performed, and assist the user to do the task faster, with fewer errors, and less cognitive load.

Back to the Future: Talk from 20 years ago Towards a Lifelong Coach: Missing Science Raj Reddy Carnegie Mellon University Keynote at ARPA Knowledge-based Planning Initiative Tucson, Arizona, February 21-24, 1994 Tucson, Arizona, February 21-24, 1994 Modified 2006 and 2011

Flash Forward Video LSI Logic 1992

Attributes of a Lifelong Coach P P rocess, Index, Store and Retrieve relevant experience Data Capture, Remember and Learn from experience Share knowledge with a community of coaches Learn from own experience and experience of others Exhibit goal directed behavior Tolerate error and ambiguity Operate in real-time

A Lifelong Coach must See, Hear and Interact: A Society of Agents with Specialized Functions Kate: Good Morning! Kate here. MP: Good Morning, Kate. It’s April 1, 1997 and the time is 6:30AM. Kate News Selection reports new trade agreements between Japan and France affects Paris Spring Collection plus 24 other items of interest. Care to browse? Kate: Umm. Not now. Mail only, please. FP: Mail serve here. 43 items including 26 unopened for three days, 12 unopened for two days and (interruption: Kate: cancel list) Ummm. Alright! Alright, I’ve been busy. FP: Canceling. Were there further instructions? Kate: Yes, uh, did my Snap Catalog arrive? FP: I understood “catalog”. Snap. Checking.

A Lifelong Coach must Be Self Aware: including the hardware, software, and networking environment Kate: Yes, uh, did my Snap catalog arrive? FP:I understood "catalog". "Snap" -- checking. Kate: Alright! MP:Care to view large screen? MP:Did that mean "yes"? Kate: Yes. Switch to large screen.

A Lifelong Coach must Be Aware of Emotions Kate: Yes, uh, did my Snap catalog arrive? FP:I understood "catalog". "Snap" -- checking. Kate: Alright! MP:Enthusiasm detected. Care to ….

A Lifelong Coach must Know what it does not know Kate: Just do it. I'll fight that one later. MP:Searching. Did not understand last command. Kate: Do it. Do it. Do it. MP:Understood.

A Lifelong Coach must Learn from Experience MP:Incoming Vid-Phone call. George Smith from Snap Headquarters, priority one. Do you wish to preview Mr. Smith? Kate: Darn! MP:You've used that before. Do you care to define it?

A Lifelong Coach must Remember and Prompt at the Right Time Kate: Um...Picture Perfect, uh, Fashion Source and Alter Image. MP:You asked to be reminded. Search on Alter Image databank is very expensive and likely to exceed your monthly online budget. Cost is $485 per thousand-image scan.

A Lifelong Coach must Provide the Right Information: in Right Language at the Right Time in the Right level of Detail MP:James Dean image not available. Copyright Paramount Studios. Last usage granted at $15,000 to Levi Strauss for in-house promotion, one time only. Kate: Darn! MP:Alternate suggestions? Kate: Uh, yeah. Let's see...uh, image search, uh, circa 1959, uh, blue jeans, male, sex appeal. MP:Searching Nexus.. Access other databanks?

A Lifelong Coach must Offer Right Advice in the Right Context MP:As originator of the conference call, do you wish to choose resolution on live video? Kate:Whatever. MP: Default settings confirmed. You asked to be reminded, full video from Paris is expensive at $3.55 per minute. A call of less than a minute will exceed your budget of... Kate: Got it. Um, top resolution on George, face only on Genevieve. MP:Understood. Conference call open.

The Creators of a Lifelong Coach must also solve Several Thorny Policy Issues Privacy Privacy Individual Individual Other participants Other participants Legal Legal Can be subpoena-ed Can be subpoena-ed Security Security Information falling into wrong hands Information falling into wrong hands

Guardian Angel Technologies: Providing Right Information to the Right People Raj Reddy Carnegie Mellon University June 2, 2014

16 A Grand Challenge For Computer Science Get The Right Information To The Right People At The Right Time In The Right Language In The Right Medium of Text or Multimedia With The Right Level Of Detail (Granularity) Digital Bill of Rights, Jaime Carbonell 1994

Universal Human Rights Declaration of United Nations 1 Right to Equality2 Freedom from Discrimination 3 Right to Life, Liberty & Security4 Freedom from Slavery 5 Freedom from Torture 6 Right to Recognition as Person 7 Right to Equality before Law8 Right to a Competent Tribunal 9 Freedom from Arbitrary Arrest 10 Right to Fair Public Hearing 11 Innocent until Proven Guilty12 Right to Privacy and Home 13 Right to Free Movement 14 Right to Asylum 15 Right to a Nationality 16 Right to Marriage and Family 17 Right to Own Property18 Freedom of Religion 19 Freedom of Opinion 20 Right of Peaceful Assembly 21 Right to Free Elections22 Right to Social Security 23 Right to Join Trade Unions24 Right to Rest and Leisure 25 Adequate Living Standard26 Right to Education 27 Right to Cultural Life 28 Right to a Social Order 29 Free and Full Development30 Freedom from Interference

18 UN World Summit on The Information Society (WSIS) Human Rights for a Digital Society Unrestricted Access to Information And Communications Unrestricted Access to Information And Communications Right to Be Connected at High Speed And Without Borders Right to Be Connected at High Speed And Without Borders Right to Online Freedom of Expression Right to Online Freedom of Expression Right to Security and Data Protection Right to Security and Data Protection Right to Privacy of Personal Communications Right to Privacy of Personal Communications Right to Freedom of Association Online Right to Freedom of Association Online

19 The Digital Bill of Rights Get The Right Information To The Right People At The Right Time In The Right Language In The Right Medium of Text or Multimedia With The Right Level Of Detail (Granularity)

20 Not Much Progress Towards Digital Bill of Rights Since 1994 What do we mean by Right Information? What do we Mean by Right People? What do we Mean by Right Time? Some Progress in Summarization Some Progress in Translation to Different Languages Some Progress in Text to Speech technologies

21 The Right Information Right Information is All Information Impacting Life, Liberty and Happiness of Every Individual Information About Safety, Security and Wellbeing Natural Emergencies: Earthquake, Severe Weather, Flood Hurricanes, Tsunami, Tornado, Snow Storm, Blizzard, Heat Wave Man-made Emergencies: Hazmat, Nuclear, Biological Information About Disruption of Basic Necessities of Life such as Water, Electricity, Food, Health and Transportation Information About Daily Problems Traffic Jams, Strikes, School Closings, Stock Markets, etc

22 Missing Science of The Right Information Right Information is Relevant Information Most information is not of interest. Scarcity Of Human Attention implies we need to develop technologies for personalization of information. Right Information is also notification and alerts including possible solutions to the problem Given a Known Problem such as an Earthquake Identify each affected individual on the planet create plans for various contingencies use push technology for timely communication, using a database of personal profile of interests, including the 5 billion people who don’t read English and the 1 billion people who can’t read any language.

23 To The Right People Right People is Anyone and Everyone in the World Who May Be Impacted by Daily Events Right People is Anyone and Everyone in the World Who May Be Impacted by Daily Events A human being in the path of a typhoon could be notified of the possibility and severity or Man-made disasters such as war and terrorist attacks Global Need One or More Dynamic Global Data Bases of All Persons on the Planet, Providing Information such as Devices Owned and Accessible Contact Information ( , Phone Number etc.) Location Info for Emergency Notification Next of Kin Information Only accessible to the Individual and surrogates protecting privacy and security

24 At The Right Time Right Time is Just-In-Time, Not Too Soon And Not Too Late Too soon, we forget Too late and we are in trouble Usually the person does not know that a Tornado is about to hit What is needed is Information Push rather than Conventional Search Today Mobile phones broadcast Alerts in affected areas Time is Money! On Wall Street, Milliseconds is Millions of Dollars On Wall Street, Milliseconds is Millions of Dollars Reaction Time Translates to Lives Saved In India, EMRI 108 (911) Saves 80% of Lives by Accident Victims by Getting to The Scene in 10 Minutes

25 In The Right Language Right Language is Each Person’s Preferred Mode of Communication Right Language is Each Person’s Preferred Mode of Communication Messages in English (and other languages) to be Translated into Any Language and spoken out using Text To Speech Messages in English (and other languages) to be Translated into Any Language and spoken out using Text To Speech In spite of all the progress in Language Translation, we are not there yet! In spite of all the progress in Language Translation, we are not there yet! Acceptable Translation from Google in 80+ languages Still Errorful Hybrid Systems Combining Knowledge with Data Hybrid Systems Combining Knowledge with Data A Little Knowledge goes a long way! e.g. Deep Blue A Little Knowledge goes a long way! e.g. Deep Blue Speech to Speech Translation is a Little Worse Speech to Speech Translation is a Little Worse Tolerable Translation Compounds Errors of Translation and Speech Recognition

26 In The Right Medium Using Text or Speech or Video Right Medium is Right Medium is Technology Solutions that cater to Language Divide: 4 billion people don’t read English, Chinese, or EU languages A Text Message in English is Not Always Appropriate A Text Message in English is Not Always Appropriate Literacy Divide: 1 Billion People can’t read any language disabilities such as dyslexic, blind, deaf etc. Limitations such as inability to read or disabilities such as dyslexic, blind, deaf etc. require the use of appropriate media To Reach Every Person on the Planet, the Same Message may Need to be Sent in ALL the Different Media to All the Different Devices Or preferred mode as indicated by user with RSVP? Broadcast to Unicast on Mobile devices? Broadcast to Unicast on Mobile devices?

27 With The Right Level Of Detail Right Level of Detail Requires Systems that, Given a Text Corpus, can Summarize the Text as Precisely and Quickly as a Human Expert Do the Same for Speech, Music, Images, Video, Art, and Movies Far From The Goal of Effective Summarization Systems that are Context Aware Produces Different Summaries for Experts And Novices

28 Personal Guardian Angels (PGAs) to the Rescue To Get The Right Information to the Right People, We Assume that Every Person on The Planet has a with an Embedded Every Person on The Planet (all 7 Billion?) has a Smart Phone with an Embedded Personal Guardian Angel that Performs Future-aware Computation A Guardian Angel is a Virtual Avatar that Performs Future-aware Computation and whisper in your ear that a tornado is heading your way A Social Network Of Guardian Angels (Facebook Of Guardian Angels?) is Able to Learn and Predict What Events Are Likely To Impact their Owner by sharing anonymized knowledge using publish/subscribe mechanisms of social networking

29 Getting the Right Information to Every Person on the Planet Is A Big Idea If Every Man, Women and Child on the Planet is Entitled To Get The Relevant Information of Interest How do we filter out all the rest of data glut? How do we determine what the right information might be for every person on the planet? Do we create a data base of everyone on the planet? How can we know what is of interest to each of them? How do we communicate with them? Only those with smart phones? Just the information available in English?

30 Structure of Personal Guardian Angels A Personal Guardian Angel is a Virtual Avatar (like a shadow) for Each Person on the Planet A Personal Guardian Angel is a Virtual Avatar (like a shadow) for Each Person on the Planet Always-On, Always-Present and Always Working Enduring (life-long) Autonomic and Nonintrusive Always-Learning Ubiquitous: Device Independent (cloud resident or local ) Monitor, analyze and learn from experience; Monitor, analyze and learn from experience; Learn From Own Experience And Experience Of Others Share knowledge with a community of Guardian Angels Learn From Own Experience And Experience Of Others Share knowledge with a community of Guardian Angels Automated discovery of data and information sources Automated discovery of data and information sources

31 Functionality of Personal Guardian Angels A Personal Guardian Angel A Personal Guardian Angel must continuously derive future course of actions from local sensory observations and context ; must continuously derive future course of actions from local sensory observations and context ; A Facebook of Guardian Angels will be able know all disruptive events that may affect your life A Facebook of Guardian Angels will be able know all disruptive events that may affect your life earthquakes and hurricanes or fuel or food scarcity earthquakes and hurricanes or fuel or food scarcity A Guardian Angel can also assist you in solving day to day problem solving A Guardian Angel can also assist you in solving day to day problem solving find an alternative route in case of accident find an alternative route in case of accident warn you about kids being late from school warn you about kids being late from school help you in selecting the right college, help you in selecting the right college, detect and correct fraudulent use of your credit card and detect and correct fraudulent use of your credit card and even help you to find a compatible partner even help you to find a compatible partner

32 Personal Guardian Angels at Work Every Guardian Angel is always on, continuously monitoring the wellbeing of the owner and accumulating huge amount of episodic knowledge This Data, Suitably Anonymized, Can Be Used to Learn Appropriate Responses For Every Possible Situation by learning preferences by observing user choices, learning by task similarity and user similarity, learning by error correction and simply learning thru clarification dialog ( does that mean yes? Would you care to define it?). There will be enough data to swamp even the most demanding Big Data Advocate.

33 Computational Resources Access to Unlimited Computation, Memory and Bandwidth By 2020, Everyone on The Planet Has Access to a Smart Phone with Global Connectivity A Smart Phone is Expected Cost $20 to $50 Every Man, Woman And Child will have Access to 16GB+ of Space on The Cloud from Facebook, Google And Microsoft Enough To Host All Personal Information In Your Own Secure Privacy Protected Area Accessible Only To You And Your Guardian Angel And Ensure Your Digital Bill Of Rights Everyone on The Planet Has Access to Unlimited Computation Unlimited Memory and Unlimited Bandwidth Use Mobile And Cloud Computing For Prediction and Notification Use Mobile And Cloud Computing For Prediction and Notification

34 Big Elephant in The Room is The Cost Free Sensor Intensive Smart Phone to Every Person on the Planet: Current cost of $200/Unit Estimated 2020 Cost per Unit of $50 Assuming 8B population One per Family of 4? One per Person? Expected Global Cost $100 to $400 Billion 0.1% to 0.4% of the Gross World Product Funding from Government and Other Stakeholders Smart Phone Manufacturers Telecom Service Providers IT Product and Service Companies Increase in Economic Activity and Productivity Expected Payback Time: Less than 1 year

35 …and Privacy A Facebook of Guardian Angels Are Able To Share Anonymized Knowledge Using Publish/Subscribe Mechanisms Of Social Networking, and Enabling Each Guardian Angel to Learn and Predict What Events Are Likely To Impact their Protégé A Guardian Angel Knows the Location Information of The Protégé at All Times so as to Provide an Alert and/or a Detailed Notification. If Needed, It Must Be Able to Turn-on The Smart Phone. No Privacy Breach Since Each Guardian Angel only Shares Information Already Known to the Service Provider

36 The Global Brain of HG Wells? HG Wells Predicted the Appearance of “Synthetic Super-minds...” within “The Species Homo Sapiens” He Envisioned that These Super-human Intelligences Represent “An Ultimate Unification … of Knowledge and Memory” of All Humanity A Social Network Of Guardian Angels May be the Ultimate Instantiation of The Global Brain of HG Wells. The Guardian Angels monitor, analyze and learn from own experience and experience of others and share anonymized knowledge using publish- subscribe mechanisms of social networking

37 The Grand Challenge Of 21st Century In Conclusion Creation of Guardian Angel Technologies for providing the right information to every man, woman and child on the planet is likely to be a disruptive technology in the IT Eco System. While the cost appears to be prohibitive, on a per capita basis it represents less than 0.1% of GWP. A Social Network Of Guardian Angels that can anticipate and share potential disaster scenarios can save 50% to 80% of all accidental deaths in the world and save millions of lives and hundreds of billions of dollars of property damage every year It is certainly the most influential research project that MSR could undertake.