1. Co-funded by the European Union Semantic CMS Community Knowledge Interaction and Presentation Copyright IKS Consortium 1 DFKI GmbH. September, 2011

2. www.iks-project.eu Page: Overview  Introduction  Semantic Web: Interaction with Content  Standards (RDF, RDFa, JSON-LD, …)  Semantic Interaction Patterns  What are patterns?  List of patterns  IKS Interaction with Knowledge  The Stack: Interaction & Presentation  VIE  Interaction Patterns & Widgets

3. www.iks-project.eu Page: Overview  Introduction  Semantic Web: Interaction with Content  Standards (RDF, RDFa, JSON-LD, …)  Semantic Interaction Patterns  What are patterns?  List of patterns  IKS Interaction with Knowledge  The Stack: Interaction & Presentation  VIE  Interaction Patterns & Widgets

4. www.iks-project.eu Page: Semantic Interaction: What is it?  Imagine you want want to plan a vacation in NY

5. www.iks-project.eu Page: Semantic Interaction: What is it?  Imagine you want to plan a vacation in NY  You found the right package on a touristic portal

6. www.iks-project.eu Page: Semantic Interaction: What is it?  Imagine you want to plan a vacation in NY  You found the right package on a touristic portal…  …and want to access additional information

7. www.iks-project.eu Page: Semantic Interaction: What is it?  Imagine you want to plan a vacation in NY  You found the right package on a touristic portal…  …and want to access additional information  Usually you have to grab the information from many sources manually (i.e. looking in many web pages)

8. www.iks-project.eu Page: Semantic Interaction: What is it?  Imagine you want to plan a vacation in NY  You found the right package on a touristic portal…  …and want to access additional information  Usually you have to grab the information manually (i.e. looking in many web pages)  With semantics you can access this information automatically (e.g., Terkait) https://github.com/neogermi/terkait

9. www.iks-project.eu Page: Semantic Web: in general.  Motivation  Definition  Web of Data vs WWW

10. www.iks-project.eu Page: Semantic Web: the potential.  Web of Data as a CMS  Web Communities as CMS Editors

11. www.iks-project.eu Page: Integrate Data with the Web  WWW

12. www.iks-project.eu Page: Integrate Data with the Web  WWW  DATA Silos

13. www.iks-project.eu Page: Integrate Data with the Web  WWW  DATA Silos Web of Data

14. www.iks-project.eu Page: The Entire Process  Exporting data (more datasets)  Grab information from your content (i.e., recognize the „entities“)  Merging your data  Merge it from different data  Conbine with different datasets/content  Use data to interact with (e.g., configure) web services  Publishing Semantics/Content/interaction  Enrich your content with dinamically generated, interactive information

15. www.iks-project.eu Page: The Entire Process  Exporting data (more datasets)  Grab information from your content (i.e., recognize the „entities“)  Merging your data  Merge it from different data  Conbine with different datasets/content  Use data to interact with (e.g., configure) web services  Publishing Semantics/Content/interaction  Enrich your content with dinamically generated, interactive information

16. www.iks-project.eu Page: The Entire Process  Exporting data (more datasets)  Grab information from your content (i.e., recognize the „entities“)  Merging your data  Merge it from different data  Conbine with different datasets/content  Use data to interact with (e.g., configure) web services  Publishing Semantics/Content/interaction  Enrich your content with dinamically generated, interactive information

17. www.iks-project.eu Page: The Entire Process  Exporting data (more datasets)  Grab information from your content (i.e., recognize the „entities“)  Merging your data  Merge it from different data  Conbine with different datasets/content  Use data to interact with (e.g., configure) web services  Publishing Semantics/Content/interaction  Enrich your content with dinamically generated, interactive information

18. www.iks-project.eu Page: Standard Resources  Web standards and accessibility resources  World Wide Web Consortium (W3C)  Web Standards Project  Web Accessibility in Mind  Tutorials, references, statistics, and forums  W3Schools  JavaScript Kit  The jQuery Project  HTML5 Demos  HTML5 Reset  Web Safe Font Tester  Web Style Guide  960 Grid System  HTML/XHTML code validators  W3C Markup Validator  W3C Link Checker  W3C Log Validator  WDG HTML Validator  CSS code validators  W3C CSS Validator  Browser compatibility verification  Browershots  Browser Cam

19. www.iks-project.eu Page: Standard Resourses: Presentation  HTML5  Css3  XAML  SMIL  XUL

20. www.iks-project.eu Page: Standard Resourses: Interaction  Java  Jscript libraries  Jquery  Backbone js  Popcorn js…  Php/perl/

21. www.iks-project.eu Page: Standard Resourses: Annotation  RDF/OWL  RDFa  Microdata  Microformats  XML  JSON

22. www.iks-project.eu Page: SKOS Technologies Perspective OWL2 APML iCal EMMA CCXML VoiceXML SSML MPEG7 SMIL XAML XUL AJAX SOAP WSDL Microdata Microformats RDFa HTML5 CSS3 RDFa API FOAF JavaScript JQuery SCXML RESTful

23. www.iks-project.eu Page: Publish RDF: RDF + HTML  Plugins for  HTML + RDFa  HTML + Microformats  HTML + Microdata  … SubjectPredObject

24. www.iks-project.eu Page: Overview  Introduction  Semantic Web: Interaction with Content  Standards (RDF, RDFa, JSON-LD, …)  Semantic Interaction Patterns  What are patterns?  List of patterns  IKS Interaction with Knowledge  The Stack: Interaction & Presentation  VIE  Interaction Patterns & Widgets

25. www.iks-project.eu Page: AQCI Lifecycle

26. www.iks-project.eu Page: AQCI Lifecycle  Author: “A user of the system can author (create) semantically enhanced content.”  Query: “A user can query for semantically enhanced content.”  Consume: “A user can consume semantically enhanced content.”  Interact: “A user can interact with the content at the level of his domain knowledge.”

27. www.iks-project.eu Page: Interaction Patterns: Definition  Design pattern provide proven solutions to frequent design problems in a generative and human-readable format.  Interaction patterns (special case of design patterns) describes  recurring actions  a user performs  when interacting with a computer  to achieve a certain goal of a task.  These actions are  implicit if they arise from the discourse context - the previous actions of the user - and are called  explicit if they are directly triggered by the user (e.g., pushing a button). In the context of IKS, we focus the interaction patterns on interaction with content and moreover the interaction with the (ontological) knowledge that corresponds to this content element.  An interaction pattern consists of four parts that describe:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

28. www.iks-project.eu Page: Interaction Patterns

29. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

30. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

31. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

32. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

33. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

34. www.iks-project.eu Page: Interaction Patterns  Interaction Pattern:  describes recurring actions  a user performs when interacting with a computer  to achieve a certain goal of a task.  Actions are  Implicit: arise from the discourse context (e.g., the previous actions of the user)  Explicit: triggered by the user (e.g., pushing a button).  An interaction pattern consists of four parts:  the problem  the pattern (i.e., the solution of the problem)  use cases for the pattern  how the pattern applies for the use cases

35. www.iks-project.eu Page: Interaction Patterns: Basics Following the AQCI Lifecycle:  “Create”: An author/user of the system can create semantically enhanced content.  “Query”: An author/user can query for semantically enhanced content.  “Consume”: An author/user can consume semantically enhanced content.  “Interact”: An author/user can interact with the content at the level of his/her domain knowledge. DIMENTIONS Content Knowledge

36. www.iks-project.eu Page: Interaction Patterns: IKS List  Manual annotation of content with knowledge Manual annotation of content with knowledge  Semi-Automatic annotation of content with knowledge Semi-Automatic annotation of content with knowledge  Fully-Automatic annotation of content with knowledge Fully-Automatic annotation of content with knowledge  Rectifying knowledge annotation Rectifying knowledge annotation  Linking of entities Linking of entities  Semi-automatic Extraction of Knowledge from Content repository Semi-automatic Extraction of Knowledge from Content repository  Automatic Extraction of Knowledge from Content repository Knowledge Content  Automatic generation of templates Automatic generation of templates  Autocompletion support for content Autocompletion support for content  Recommendation of content links (Duplicate: Query) Recommendation of content links (Duplicate: Query)  Faceted search Faceted search  Disambiguation of query Disambiguation of query  Using annotated content items for searching Using annotated content items for searching  Providing personalized/contextual search result filters Providing personalized/contextual search result filters  Recommendation of content links (Duplicate: Create) Recommendation of content links (Duplicate: Create)  Search and Suggest Service: Annotations for textual content Search and Suggest Service: Annotations for textual content Create Query

37. www.iks-project.eu Page: Interaction Patterns: IKS List  Contextual rendering of content Contextual rendering of content  Highlighting of content of interest Highlighting of content of interest  Receive a stream of information/content of interest Receive a stream of information/content of interest  Browsing through entity hierarchy Browsing through entity hierarchy  Zoom in/out different data hierarchy levels along a tree Zoom in/out different data hierarchy levels along a tree  Visalizing content dependencies Visalizing content dependencies  Comparing content, based on the knowledge Comparing content, based on the knowledge  Recommendation of content links (Duplicate: Query) Recommendation of content links (Duplicate: Query)  Contextual activity pane Contextual activity pane  Contextual export interaction Contextual export interaction  Semantic Drag&Drop interaction Semantic Drag&Drop interaction  Registering and receiving updates of content changes Registering and receiving updates of content changes  Managing audits according to business rules Managing audits according to business rules  Assure a level of consistency of content according to business rules Assure a level of consistency of content according to business rules  Filtering content based on knowledge rules Filtering content based on knowledge rules  Automatic clustering of content/knowledge Automatic clustering of content/knowledge  Automatic summarization of content/knowledge Automatic summarization of content/knowledge  Exporting knowledge Exporting knowledge Consume Query

38. www.iks-project.eu Page: Overview  Introduction  Semantic Web: Interaction with Content  Standards (RDF, RDFa, JSON-LD, …)  Semantic Interaction Patterns  What are patterns?  List of patterns  IKS Interaction with Knowledge  The Stack: Interaction & Presentation  VIE  Interaction Patterns & Widgets

39. www.iks-project.eu Page: The Stack: Interaction & Presentation Functionality categories of the IKS stack’s layer of interaction and presentation:  Author: “A user of the system can author (create) semantically enhanced content.”  Query: “A user can query for semantically enhanced content.”  Consume: “A user can consume semantically enhanced content.”  Interact: “A user can interact with the content at the level of his domain knowledge.”

40. www.iks-project.eu Page: Interaction with Content in IKS Common representation of content on JavaScript level  If Content expressed with RDFa the content model can be easily extracted into JavaScript. Using Backbone.js:  supplies structure to JavaScript-heavy applications by  providing models with key-value binding and custom events,  collections with a rich API of enumerable functions,  views with declarative event handling,  connects it all to existing applications over a RESTful JSON interface.  With Backbone, the content extracted from the RDFa-annotated HTML page is easily manageable via JavaScript.

41. www.iks-project.eu Page: VIE  Pedigree:  Name: Vienna IKS Editable  Functionality: makes the content of web pages editable through annotations. Supports semantic-web developers in  Retrieval of semantic data  Storing semantic data  Accessing semantic web services (e.g., Apache Stanbol Enhancer)  Semantic markup (e.g., RDFa or Microdata)  Coordinates:  Basic concepts: http://wiki.iks-project.eu/index.php/VIEhttp://wiki.iks-project.eu/index.php/VIE  Development: http://github.com/IKS/VIEhttp://github.com/IKS/VIE

42. www.iks-project.eu Page: It‘s about abstraction VIE - UI Widgets „VIE-W“ VIE - UI Widgets „VIE-W“ VIE „Edit your content w. Semantics“ VIE VIE-2 „Edit your Semantics“ VIE-2 (Semantic) Services (e.g., Stanbol Enhancer, - EntityHub, Zemanta,...) (Semantic) Services (e.g., Stanbol Enhancer, - EntityHub, Zemanta,...) (Semantic) Databases (e.g., DBPedia, Geonames,...) (Semantic) Databases (e.g., DBPedia, Geonames,...)

43. www.iks-project.eu Page: VIE: UI Widgets UI Widgets On top of VIE we gathered a bunch of UI widgets in a library that help to simplifying embedding VIEs power into a webpage more directly.

44. www.iks-project.eu Page: VIE Widgets  VIE-Widgets are a sort of jQuery UI Widgets in order to:  achive maximum portability  accelerating lerning curve Widgets

45. www.iks-project.eu Page: UI Widget Example Description: The VIE Image Search widget search for images using semantic annotated content as parameter for the search. Once included in an HTML page, the developer can easily query and retrieve images from the photo service Flickr, based on the type-specific properties of the current entity in focus, e.g., if the current entity is a city it makes sense to start a geographic query, whereas, for persons, it would make sense to query for the name of that person. (http://neogermi.github.com/VIEwidgets/widgets/image_search/index.html)http://neogermi.github.com/VIEwidgets/widgets/image_search/index.html

46. www.iks-project.eu Page: Copyright IKS Consortium 46 W3C: Interaction and Semantics  Interaction  HTML5  JavaScript  JQuery  Semantic Web  OWL  RDF  RDFa  JSON July 5, 2011

47. www.iks-project.eu Page: Copyright IKS Consortium 47 W3C: Interaction and Semantics  Interaction  HTML5  JavaScript  JQuery  Semantic Web  OWL  RDF  RDFa  JSON VIE July 5, 2011

48. www.iks-project.eu Page: Copyright IKS Consortium 48 IKS: Coordinates  Stanbol  VIE  VIE2  Interaction Patterns  WIDGETS W3C | Technology and Society Domain | Semantic Web Activity | RDF Web Applications Working Group (RDFWA) July 5, 2011

49. www.iks-project.eu Page: References  http://linkeddata-specs.info/ http://linkeddata-specs.info/  http://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData http://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData  http://www.bioontology.org/wiki/images/6/6a/Triple_Stores.pdf http://www.bioontology.org/wiki/images/6/6a/Triple_Stores.pdf  https://github.com/IKS https://github.com/IKS  https://github.com/IKS/VIE https://github.com/IKS/VIE  https://github.com/neogermi/VIEwidgets https://github.com/neogermi/VIEwidgets  http://schema.org http://schema.org  http://wiki.iks-project.eu/index.php/Semantic_Editor http://wiki.iks-project.eu/index.php/Semantic_Editor  http://www.w3.org/TR/xhtml-rdfa-primer/ http://www.w3.org/TR/xhtml-rdfa-primer/  http://www.w3.org/2006/07/SWD/ http://www.w3.org/2006/07/SWD/  http://www.w3.org/TR/rdfa-syntax/ http://www.w3.org/TR/rdfa-syntax/  http://www.w3.org/TR/xhtml-rdfa-primer/ http://www.w3.org/TR/xhtml-rdfa-primer/  http://www.w3.org/2010/02/rdfa/ http://www.w3.org/2010/02/rdfa/  http://www.w3.org/TR/2011/WD-rdfa-api-20110419/ http://www.w3.org/TR/2011/WD-rdfa-api-20110419/  http://documentcloud.github.com/backbone/ http://documentcloud.github.com/backbone/  http://www.w3.org/2001/11/13-RDF-Query-Rules/ http://www.w3.org/2001/11/13-RDF-Query-Rules/  http://json-ld.org http://json-ld.org  http://www.interactive-knowledge.org/sites/www.interactive- knowledge.org/files/Tutorial_Knowledge_Interaction_Presentation.pptx http://www.interactive-knowledge.org/sites/www.interactive- knowledge.org/files/Tutorial_Knowledge_Interaction_Presentation.pptx