 Copyright 2008 Digital Enterprise Research Institute. All rights reserved. Semantic on the Social Semantic Desktop Simon Scerri, Siegfried Handschuh, Stefan Decker ESWC 2008 June 2008 Tenerife

2 Outline...  Introduction –Semantic Desktop – –Semantic  Social Semantic Desktop  Semantic –Semantic Annotation – Ad-hoc Workflows –Workflow Example  Semanta – Your Personal Assistant  Future Work & Conclusion

3 Semantic Web technology on the Personal Desktop Objects on the desktop become Resources with a URI Semantic Web Technologies improve Data Integration and Retrieval Semantic Desktop

4 Sharing resources within Network of Semantic Desktops Social aspect of SSD depends highly on Communication Communication channels need to support Semantic Knowledge Social Semantic Desktop

5 Most popular means of Electronic Communication –Asynchronous Communication –Flexible, dynamic nature is also a Virtual Workplace –Collaborative Environment –Knowledge creation, management and sharing Problems – Tracking – Classification – Retrieval – Overload

6 Eases Overload – Classification, Retrieval, Tracking Enhances Data Representation + Unification on and between SSD’s Annotation i. Thread metadata - Sequence, Social, Temporal Metadata ii. Content metadata - Intents and Expectations of written dialogue Semantic ! ?

7 Task Assign Speech Act Theory Multiple Intents and Expectations of an ’s content Speech Act Model: [Action, Object, Subject] Example “...Please make sure you have the document ready!..” Subject Recipient Speech Acts Object TaskInformation Resource Event Noun Activity Data Feedback RecipientSender Both Action Role Informative CompletiveRequestive Decline Assign Deliver Request InitiativeContinuative Imperative Negotiative Abort Suggest Propose

8 Ad-Hoc Workflows Conversations consist of concurrent, implicit, well- formed Ad-Hoc Workflows Example: »Request Meeting »Negotiate Different Time »Commit to the Meeting »Invite additional People Speech Act = Start/Continuation of a Workflow Workflow Artefacts – Shared concepts e.g. Events, Tasks, People, Projects… Artefacts created within need to be exported and shared between different SSD’s

9 Modelling Ad-hoc Workflows [Includes Participant] [Acknowledge] [Request] [Approve] Manage Activity [Propose] [Assign] [Amend] Participant* [Request] [Acknowledge] [Inc. Participant] [Approve] [Deliver Data] [Acknowledge] [Deliver Feedback] [Acknowledge] Suggest Activity Deliver Data Propose Activity Abort Activity Assign Activity Manage Activity Request Activity Request Data [Suggest Activity] [Deliver Data] [Propose Activity] [Abort Activity] [Assign Activity] [Request] [Includes Participant] [Activity] [Data] † [Acknowledge] Manage Activity † Manage Activity † Manage Activity [Decline ] Deliver Data Decline Data Request Activity Assign Activity Deliver Feedback † † † † † † [Includes Participant] [Includes Initiator] † [Inc. Participant] [Includes Initiator] Initiator [Decline ] † Decline Activity [Includes Initiator] Collect Feedback Manage Activity INITIATE TERMINATE Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items Speech Act: Action-Object-Subject Speech Act Model is represented within the sMail Ontology The ‘Action’ of a Speech Act, and possible roles ‘Object’ and ‘Subject’ of the Speech Act action Text Analytics Speech act recognition & annotation Ontology-based Information Extraction using GATE A Declarative model classifies text into speech acts according to linguistic characteristics The model is mapped over to JAPE Pattern/Action Rules Intuitive Annotation Wizard for semi- automatic annotation Intelligent support for handling action items

10 [Suggest Activity] [Deliver Data] [Propose Activity] [Abort Activity] [Assign Activity] [Request] INITIATE How about some dinner tomorrow after work? Workflow Example Initiator

11 [Data] INITIATE [Request] [Activity] Workflow Example How about some dinner tomorrow after work? Initiator

12 [Activity] Send Request Activity [Request] INITIATE Workflow Example How about some dinner tomorrow after work? Initiator

13 [Activity] Send Request Activity [Request] INITIATE Workflow Example How about some dinner tomorrow after work? Initiator

14 [Activity] [Amend] Participant [Approve] Send Request Activity [Decline] [Other] † [Ignore] [Request] INITIATE I would love to! Workflow Example How about some dinner tomorrow after work? Initiator

15 [Activity] [Approve] Send Request Activity † [Request] INITIATE [Includes Initiator] [Inc. Participant] I would love to! Workflow Example How about some dinner tomorrow after work? InitiatorParticipant

16 [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE [Includes Initiator] I would love to! Manage Activity Workflow Example How about some dinner tomorrow after work? ParticipantInitiator

17 [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE [Includes Initiator] I would love to! Manage Activity Workflow Example How about some dinner tomorrow after work? ParticipantInitiator

18 [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE I would love to! Manage Activity Send Assign Activity Workflow Example How about some dinner tomorrow after work? ParticipantInitiator

19 [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE I would love to! Manage Activity [Acknowledge] † [Other] [Ignore] Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

20 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] I would love to! Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

21 [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] I would love to! Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

22 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] Manage Activity I would love to! Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

23 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] Manage Activity I would love to! Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

24 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] Manage Activity I would love to! Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

25 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] Manage Activity I would love to! TERMINATE Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

26 [Includes Initiator] [Inc. Participant] [Includes Initiator] [Inc. Participant] [Activity] [Approve] Send Request Activity † [Request] INITIATE Manage Activity † [Acknowledge] Manage Activity I would love to! TERMINATE Workflow Example How about some dinner tomorrow after work? ParticipantInitiator Send Assign Activity

27 I would love to! Workflow Example How about some dinner tomorrow after work?

28 I would love to! sMail Ontology NRL NIENRL NMOPIMONCO Knowledge Integration Within the Personal Semantic Desktop Workflow Example How about some dinner tomorrow after work?

29 I would love to! Knowledge Integration Within the Personal Semantic Desktop Data Unification Among the Social Semantic Desktops Workflow Example How about some dinner tomorrow after work? sMail Ontology NRL NIENRL NMOPIMONCO

30 Applications  Tracking  Classification  Retrieval  Personal Information Management   Desktop Knowledge Integration Semanta

31 Semanta Flagging Semi-automatic Annotation

32 Support for Action Items (Annotations) Semanta Exporting Artefacts

33 Future Work Extending domain of application to other Electronic Communication Media (e.g. Instant Messaging) Extending Features – e.g. Social Relationships Combining our technology with related and relevant work (e.g. GTD - Getting Things Done) Evaluation of Semanta’s User Interface

34 The Social aspect of SSD depends on Communication ’s flexibility is also the source of Overload Semantic –Knowledge Integration Within the Personal Semantic Desktop –Data Unification Among the Social Semantic Desktops Conclusion sMail Ontology NRL NIENRL NMOPIMONCO