1. V2 – An Emerging Universal Remote Console Standard Gottfried Zimmermann University of Wisconsin-Madison, USA Technical Editor, V2 International Representative, V2

2. V2 - An Emerging Universal Remote Console Standard 2 Note for screen reader users These slides contain graphics. Please refer to the notes section of a slide to find long descriptions of the graphics it contains.

3. V2 - An Emerging Universal Remote Console Standard 3 Recent Technological Advancements  Moore’s Law  Steady increase in speed and power, decrease in size and prize  Mobile devices have computing power of big irons 15 years ago  Ubiquitous Computing  Computers everywhere  The “disappearing” computer  Wired and wireless connection technologies  Remote operation of networked devices and services  “Artificial Intelligence”  Speech Recognition  Natural Language Understanding  Intelligent user agents

4. V2 - An Emerging Universal Remote Console Standard 4 But There Are Challenges…  Existing user interfaces are not intuitive and easy to understand for many users  For remote control, networked devices have to provide separate user interface specifications for every controller platform  Intelligent user agents may be the remedy for tomorrow, but not mature enough for today

5. V2 - An Emerging Universal Remote Console Standard 5 Isn’t it time to think about more universal ways to interact with the devices and services around us?

11. V2 - An Emerging Universal Remote Console Standard 11 … and at the same time facilitate more natural ways of interaction with tomorrow’s technology

18. V2 - An Emerging Universal Remote Console Standard 18 People with disabilities wouldn’t be left out. In fact, everybody would benefit.

19. V2 - An Emerging Universal Remote Console Standard 19

20. V2 - An Emerging Universal Remote Console Standard 20

21. V2 - An Emerging Universal Remote Console Standard 21 Overview  What is the V2 Standard?  Universal Remote Console  Design goals  Technical concepts  Examples  Who is building on V2?  Members  Standardization roadmap  Prototypes  Demo (screenshots)  if time allows  Conclusion and where to get more information

22. V2 - An Emerging Universal Remote Console Standard 22 Universal Remote Console  a device or software application, such as a Personal Digital Assistant (PDA) a cell phone etc.  … through which a user accesses and controls  … any electronic & IT product, such as a TV, a washing machine, a thermostat, an alarm clock a telephone, a copy machine an elevator, a ticket vending machine etc.

23. V2 - An Emerging Universal Remote Console Standard 23 Idea: Use any of these controllers… (“URCs”)

24. V2 - An Emerging Universal Remote Console Standard 24 … to access any of these products (“Targets”)

25. V2 - An Emerging Universal Remote Console Standard 25 Design Goals for the URC Framework  “Universal” Remote  Bi-Directional Communication  Middleware-Neutrality  Device & Modality Independence  URC-Specific Custom User Interfaces  Direct Access to Target Functions  Support for I18n  Reuse Existing Standards and Technologies  Easy Target Maintenance  User Interface Extensions Beyond What’s Built Into the Target  Extensibility for Emerging Technologies  A11y? – implicit

26. V2 - An Emerging Universal Remote Console Standard 26 Goal: “Universal” Remote  One URC can control all Target products  Targets: devices and services  Anywhere  at home  at work  at school  in public places  But: Dedicated remotes for specific Targets possible  e.g. TV  “Let the user decide”

27. V2 - An Emerging Universal Remote Console Standard 27 Goal: Bi-Directional Communication  Bi-directional communication between URC and Target  Universal Remote Console  Implicit synchronization of the Target’s status information  Push notifications onto URC

28. V2 - An Emerging Universal Remote Console Standard 28 Concept: Composition ofUser Interface User Interface Generator User Interface Socket User Interface (on the Universal Remote Console) + = Resources LabelHelp … Access key … Keyword …… User Interface Implementation Description +

29. V2 - An Emerging Universal Remote Console Standard 29  Socket: machine-operable access and control point for a Target (or a functional unit of a Target)  One Target has one or more sockets  Socket elements  variables  statics  commands  notifications  Socket elements are implicitly synchronized between URC and Target  Note: Actual socket implementation (with “living values”) is platform-specific Concept: User Interface Socket

30. V2 - An Emerging Universal Remote Console Standard 30  Describes the socket elements in a platform-neutral way, including  Named elements  Types (subset of XML Schema Definition types)  Dependencies (XPath)  The socket description informs a URC about the functionality of a socket in terms of its socket elements  Socket and Socket Description is everything that’s needed to access a Target through a “bare-bone UI” Concept: User Interface Socket Description (SD)

31. V2 - An Emerging Universal Remote Console Standard 31 Example: SocketDescription 570

32. V2 - An Emerging Universal Remote Console Standard 32 Goal: Middleware-Neutrality  Implementation of standard should be possible on a variety of middleware platforms for discovery and control  Binding to platform should not affect user interface  Bridging between different middleware platforms

33. V2 - An Emerging Universal Remote Console Standard 33 Concept: Target-URC Network (TUN)  TUN is middleware that connects the socket on the Target with its copy on the URC  TUN provides services for discovery, control & eventing  Possible TUNs  Universal Plug and Play (UPnP)  Jini/Java  Web service technologies (UDDI, WSDL, SOAP, etc.)  ZigBee?  XForms data model?  Others?  Note: Implementation guidelines necessary to ensure interoperability within TUN  Gateways can bridge between different TUNs

34. V2 - An Emerging Universal Remote Console Standard 34 Layered Model for URC and Target Universal Remote Console Target

35. V2 - An Emerging Universal Remote Console Standard 35 Concept: Target Description (TD)  Description of a target for discovery purposes  Identifier  Location description  Description of sockets Socket identifier Pointer to socket description Pointer to UIIDs  Directory of available resources  Other (Dublin Core) properties…  One TD per Target  TD is middleware-neutral  Can be used across multiple TUNs

36. V2 - An Emerging Universal Remote Console Standard 36 Example:Target Description <target about="http://www.myCompany.com/DigitalThermometer" xmlns="http://www.incits.org/incits392-2005"...> 0123456 http://www.incits.org/incits392-2005 socket/socketdescription.xml 41.11.22.00

37. V2 - An Emerging Universal Remote Console Standard 37 Goal: Device & Modality Independence  Target can be fully controlled by any URC, without user programming or downloading  PDAs  Cell phones  Wrist watches  Computers  URC devices  AT devices  Single-source authoring of remote user interface  Modality independence of user interface  Separation of content and presentation  Multi-channel delivery of user interface

38. V2 - An Emerging Universal Remote Console Standard 38 Concept: User Interface Implementation Description (UIID)  “View” component for a socket  UIID provides additional information how to build a “usable” user interface  Structure  What user interface components (e.g. “widgets”) to use  How to map components to socket elements (“binding”)  One socket may have multiple UIIDs for different purposes (i.e. multiple views)  UIIDs can be proprietary  BUT: One UIID must be written in standard format

39. V2 - An Emerging Universal Remote Console Standard 39 Concept: Presentation Template (PT)  Subtype of UIID, defined by V2  Modality-independent user interface description  Defines UI components and structure for socket  Elements (“Interactors”) similar to XForms:   &   &   “Core Presentation Template”: reflects full functionality of a socket  Mandatory for every socket

40. V2 - An Emerging Universal Remote Console Standard 40 Example:Presentation Template <pret name="http://www.mycorp.com/thermometer/corepret" id="pret" xmlns="http://www.incits.org/incits391-2005">

41. V2 - An Emerging Universal Remote Console Standard 41 Goal: URC-Specific Custom User Interfaces  Ability to implement URC-specific custom user interfaces  (in addition to modality-independent user interfaces)  take advantage of capabilities of URC platform e.g. special UI for iPAQ  branding through user interface  can be important for differentiation when functionality is mostly the same across vendors

42. V2 - An Emerging Universal Remote Console Standard 42  UIID may be implemented in any technology:  Presentation Template (defined by URC standard)  Java/Swing  Flash  SVG  Etc.  Note: All UIIDs are based on the same socket Concept: UIID Format Open

43. V2 - An Emerging Universal Remote Console Standard 43 Goal: Direct Access to Target Functions  Concept: Direct access to all socket elements  at any time  without user navigation (“browsing”)  URC knows about dependencies  Concept: UIID may bind to multiple sockets (of multiple Targets)  user doesn’t need to know what Target to pick for a specific function  more usable user interface if task-oriented

44. V2 - An Emerging Universal Remote Console Standard 44 Goal: Support for I18n  Easy to localize user interface  even after Target installation  Localization work may be out-sourced by Target manufacturers to 3 rd parties  Even graphics may be locale-specific

45. V2 - An Emerging Universal Remote Console Standard 45 Concept: Atomic Resources & Resource Sheets  Atomic Resource definition:  object within a user interface that is used as an atomic entity in the construction of a concrete user interface  An Atomic Resource can be specific to  Language/locale  Runtime environment  Audience  etc.  Atomic Resources are grouped and described in Resource Sheets  typically stored as files on the Target (or on Resource Services)  typically one Resource Sheet per language/locale

46. V2 - An Emerging Universal Remote Console Standard 46 Concept: Atomic Resources & Resource Sheets (cont’d)  V2 standard defines an Atomic Resource Description containing:  Identifier for Atomic Resource  Type (Text, Image, etc.)  Format (MIME type)  Use context Element reference (to a socket element, PreT element or TD element) Value reference Operation reference Role (label, help, access key, keyword, etc.) Language context  Other (Dublin Core) properties…

47. V2 - An Emerging Universal Remote Console Standard 47 Example:Resource Sheet <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#"> temperature en MyCorp, Inc. Text

48. V2 - An Emerging Universal Remote Console Standard 48 Concept: Resource Directory  Resource Sheets are listed in a Resource Directory  Resource Directory is part of the Target Description  A Resource Directory contains links to any Target Resources (and external Resources)  Tree structure with leaves providing “scents” up to the root

49. V2 - An Emerging Universal Remote Console Standard 49 Example:Resource Directory <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#"> rsheet-en.rdf Text en

50. V2 - An Emerging Universal Remote Console Standard 50 Goal: Reuse Existing Standards and Technologies  No need to re-invent the wheel when already existing  People are already familiar with existing technologies  Tools and code libraries already existing

51. V2 - An Emerging Universal Remote Console Standard 51 Reused Standards/Technologies in V2 Standards  XML & XML Schema Definition  XPath & XPointer  XForms form controls  RDF & Dublin Core

52. V2 - An Emerging Universal Remote Console Standard 52 Goal: Easy Target Maintenance  Target may be a “dumb” device  e.g. thermostat  In general, Targets will last longer than URCs  Target features can be added even after purchase  if either the Target or the URC has access to the Internet  No URC update or central registry necessary

53. V2 - An Emerging Universal Remote Console Standard 53 Concept: “Dumb” Target and “Smart” URC  Target Resources  static  independent from middleware platform  may be on ROM  can be served by simple HTTP server  URC  picks appropriate resources  generates user interface by “mix-and-match”  URC could be composed of gateway and ultra-thin client

54. V2 - An Emerging Universal Remote Console Standard 54 Target Components

55. V2 - An Emerging Universal Remote Console Standard 55 URC Components

56. V2 - An Emerging Universal Remote Console Standard 56 Goal: User Interface Extensions Beyond What’s Built Into the Target  Target manufacturer, URC manufacturer and 3 rd parties may provide extensions to the Target’s user interface  skins  icons  sounds  etc.  Simplification  hide complex functions  use same UI for a class of Targets  Any special adaptation for  a particular URC  a particular audience  etc.

57. V2 - An Emerging Universal Remote Console Standard 57 Concept: Supplemental Resources

58. V2 - An Emerging Universal Remote Console Standard 58 Goal: Extensibility for Emerging Technologies  Natural Language Interaction  Intelligent user agents  Semantic Web concepts applicable  Emerging security & privacy mechanisms  Future-proof  Buy products today, and use them with simple URCs  In the future, you may use intelligent user agents to control them

59. V2 - An Emerging Universal Remote Console Standard 59 Concept: Open URC Framework  Use of XML, XML Schema Definition and RDF  Atomic Resources are extensible  Resource Services are versatile  Model-based user interface generation  Semantic information available  More semantic information can be added to socket description  as Supplemental Atomic Resources  or as Supplemental UIIDs  or in any other way facilitated by dedicated Resource Services  Security & Privacy services can be added  as part of TUN  as independent services

61. V2 - An Emerging Universal Remote Console Standard 61 Toward an ANSI standard  V2 Technical Committee  … within the InterNational Committee for Information Technology Standards (INCITS)  … under the American National Standards Institute (ANSI)  V2 comprises members from industry, user groups, academia and government

62. V2 - An Emerging Universal Remote Console Standard 62 Members of V2 … and others …joining…

63. V2 - An Emerging Universal Remote Console Standard 63 5 Draft ANSI Standard Documents  “Protocol to Facilitate Operation of Information and Electronic Products through Remote and Alternative Interfaces and Intelligent Agents: …”  INCITS 389-2005: Universal Remote Console  INCITS 390-2005: User Interface Socket Description  INCITS 391-2005: Presentation Template  INCITS 392-2005: Target Properties Sheet  INCITS 393-2005: Resource Descriptions  2 nd public review starting soon

64. V2 - An Emerging Universal Remote Console Standard 64 Implementation Guidelines for TUNs  V2-UPnP Implementation Guidelines in development  Looking for a home for publication  V2-UPnP Modes:  RemoteUI mode  Legacy mode  Other implementation guidelines will be developed as needed

65. V2 - An Emerging Universal Remote Console Standard 65 Toward an ISO/IEC Standard  Submitted as “New Work Item Proposal” to ISO/IEC JTC1 SC35 on User Interfaces  Project has been approved by JTC1  Assigned to SC35 WG8 “User Interfaces for Remote Interactions”  Goal: Release as international standard  International participation  Continuity  Improvements

66. V2 - An Emerging Universal Remote Console Standard 66 Members of JTC1 SC35 P-Members:  Austria  Canada  China  Czech Republic  Denmark  Finland  Germany  Greece  Italy  Japan  Korea  Sweden  United States O-Members:  Ireland  Netherlands  Poland  Russian Federation

67. V2 - An Emerging Universal Remote Console Standard 67 V2 Prototype Implementations  University of Wisconsin-Madison, USA  Java / Web-based Simulation Environment, www.myurc.comwww.myurc.com  Java / UPnP (SDK in development)  Java / Jini (old)  Georgia Institute of Technology, Atlanta, USA  C#/C++ / UPnP  National Institute of Standards and Technology (NIST), USA  Java / Jini  Marquette University, Wisconsin  Just starting

68. V2 - An Emerging Universal Remote Console Standard 68 Demo (screen shots) Web-based Prototype from University of Wisconsin www.myurc.com/V2SimEnv www.myurc.com/V2SimEnv

69. V2 - An Emerging Universal Remote Console Standard 69

70. V2 - An Emerging Universal Remote Console Standard 70

71. V2 - An Emerging Universal Remote Console Standard 71

72. V2 - An Emerging Universal Remote Console Standard 72

73. V2 - An Emerging Universal Remote Console Standard 73

74. V2 - An Emerging Universal Remote Console Standard 74

75. V2 - An Emerging Universal Remote Console Standard 75 Conclusion  The V2 standard is close to 2 nd public review under ANSI  Watch www.incits.org/tc_home/v2.htmwww.incits.org/tc_home/v2.htm  Contact zimmer@trace.wisc.edu if you want to be added to URC- SIG mailing listzimmer@trace.wisc.edu  ISO/IEC JTC1 SC35 is developing the V2 standard on an international level  White paper available at www.myurc.comwww.myurc.com  Prototype and SDK developments under way  If this standard doesn’t succeed, proprietary technologies may prevail, trying to monopolize the market

76. V2 - An Emerging Universal Remote Console Standard 76 Thank you for your Attention! Work supported by… U.S. Department of Education (NIDRR) National Science Foundation through the National Computational Science Alliance (NCSA) Microsoft Sun Microsystems