1. 3D Electronic Institutions eMarkets Research Group Helmut Berger

2. 2 Outline The VisionThe Vision ObjectivesObjectives Electronic InstitutionsElectronic Institutions 3D Virtual Worlds3D Virtual Worlds A Framework for 3D Electronic InstitutionsA Framework for 3D Electronic Institutions ImplementationImplementation Conclusion & Future WorkConclusion & Future Work

3. 3 The Vision

4. 4 Objectives New Metaphor: 3D Electronic InstitutionsNew Metaphor: 3D Electronic Institutions Establish TrustEstablish Trust Interact within a Social EnvironmentInteract within a Social Environment Participants areParticipants are –Human Users –Autonomous Software Agents

5. 5 Objectives Reliability and SecurityReliability and Security Immersive 3D ExperienceImmersive 3D Experience Identify an Application DomainIdentify an Application Domain Information-rich EnvironmentInformation-rich Environment LearningLearning

6. 6 Trust 1.Trust is the assured reliance on the character, ability, strength, or truth of someone or something. 1.Trust is the assured reliance on the character, ability, strength, or truth of someone or something. 2.One in which confidence is placed. (Merriam Webster‘s Collegiate Dictionary)

7. 7 Trust One of the most important social conceptsOne of the most important social concepts Helps humans to cope with their social environmentHelps humans to cope with their social environment Is present in all human interactionIs present in all human interaction Implicit trust assessment underlies every traditional bargainImplicit trust assessment underlies every traditional bargain Drawback in electronic markets: face-to-face interaction is missingDrawback in electronic markets: face-to-face interaction is missing Increased risk level related to online marketsIncreased risk level related to online markets

8. 8 Social Environments Social Interaction is the key featureSocial Interaction is the key feature ConversationsConversations Virtual Worlds are spaces were people meetVirtual Worlds are spaces were people meet Strong relation to TrustStrong relation to Trust “Whom do you know?“

9. 9 Electronic Institutions A Framework and Methodology developed by a Research Group led by Prof. Carles Sierra at the Artificial Intelligence Research Institute (IIIA), Barcelona, SpainA Framework and Methodology developed by a Research Group led by Prof. Carles Sierra at the Artificial Intelligence Research Institute (IIIA), Barcelona, Spain Electronic Institutions are „regulated“ Virtual Environments which allow Agents to interact autonomouslyElectronic Institutions are „regulated“ Virtual Environments which allow Agents to interact autonomously

10. 10 Electronic Institutions Natural extension of the social concept of institutions as regulatory systems shaping human interactionsNatural extension of the social concept of institutions as regulatory systems shaping human interactions Software systems inhabited by autonomous entities = software agentsSoftware systems inhabited by autonomous entities = software agents Agents interact according to predefined conventions on language and protocolAgents interact according to predefined conventions on language and protocol Conventions guarantee that certain norms of behaviour are enforcedConventions guarantee that certain norms of behaviour are enforced

11. 11 Electronic Institutions ConventionsConventions –on Language = Dialogical Framework –on Activities = Performative Structure –on Behaviour = Norms

12. 12 Dialogical Framework Language (Ontology)Language (Ontology) Illocutionary ParticlesIllocutionary Particles –Theory of Speech Acts –Particles are inform, failure or request Organizational Structure of the Agent SocietyOrganizational Structure of the Agent Society RolesRoles –External Vs. Internal –Incompatibilities –Relationships

13. 13 Dialogical Framework Ontology (start_double_auction: -> boolean) (offer:int*float -> boolean) (demand:int*float -> boolean) (want_to_buy:String -> boolean) (want_to_sell:String -> boolean) (close_meeting: -> boolean) (close_wait: -> boolean) Role Hierarchy

14. 14 Performative Structure Defines the Dialogues in which Agents might participateDefines the Dialogues in which Agents might participate Each Activity an Agent performs is associated with a Dialogue among the group of agents involved in this ActivityEach Activity an Agent performs is associated with a Dialogue among the group of agents involved in this Activity Dialogues are referred to as ScenesDialogues are referred to as Scenes

15. 15 Performative Structure Fixes the Protocol (possible dialogues) which might be enacted in each SceneFixes the Protocol (possible dialogues) which might be enacted in each Scene Defines the applicable Sub-Language of a particular SceneDefines the applicable Sub-Language of a particular Scene Minimum and Maximum number of participantsMinimum and Maximum number of participants

16. 16 Performative Structure Agents transit is regulated by special scenes referred to as TransitionsAgents transit is regulated by special scenes referred to as Transitions Transitions are used to re-route agentsTransitions are used to re-route agents Agents are synchronized via different types of transitionsAgents are synchronized via different types of transitions

17. 17 Performative Structure: An Example

18. 18 Performative Structure Behind the ScenesBehind the Scenes –A Finite-State Machine (FSM) regulates the Flow of Agents within particular scenes –The FSM contains special states defining possible Enter and Exit points for agents –Arcs between states are labelled with illocutionary patterns –Uttered illocutions change the state of the FSM

19. 19 FSM: Behind the Scenes Example Illocution: (W0 W1(inform (?s seller) (?t tradeMgr) offer(?qty ?price)) (?qty > 0 ))

20. 20 Norms Norms impose restrictions on actions agents are allowed to perform in scenesNorms impose restrictions on actions agents are allowed to perform in scenes Agents acquire Commitments / Obligations within an Electronic Institution while talkingAgents acquire Commitments / Obligations within an Electronic Institution while talking Commitments / Obligations restrict future actions of the agentCommitments / Obligations restrict future actions of the agent

21. 21 3D Virtual Worlds

22. 22 3D Virtual Worlds Space is designed and arranged according to human everyday experiencesSpace is designed and arranged according to human everyday experiences Space and objects in space produce an immersive environmentSpace and objects in space produce an immersive environment Construct a virtual representation of a particular domainConstruct a virtual representation of a particular domain

23. 23 3D Virtual Worlds A convenient and alternative interface for user interactionA convenient and alternative interface for user interaction Social experience: already through the simple presence of othersSocial experience: already through the simple presence of others Versatile ways of InteractionVersatile ways of Interaction

24. 24 3D Virtual Worlds Humans live in well-structured environments designed according to well-known MetaphorsHumans live in well-structured environments designed according to well-known Metaphors –Rooms, Buildings, Streets, Districts,... Virtual Worlds might be designed according to similar metaphorsVirtual Worlds might be designed according to similar metaphors Avatars are the embodied representation of human users within the Virtual WorldAvatars are the embodied representation of human users within the Virtual World

25. 25 Design Considerations Space is used to model impressionsSpace is used to model impressions –objects in space convey implicit meaning (buildings) –social power is expressed by “height” Proximity might indicate things of a similar type or groupProximity might indicate things of a similar type or group –eg. avatars located close to an object talking about that object (social context) Space and Proximity relate strongly to a person’s cultural backgroundSpace and Proximity relate strongly to a person’s cultural background

26. 26 Design Considerations Navigation: „Don‘t get lost!“Navigation: „Don‘t get lost!“ Visual DistanceVisual Distance –the distance to an object influences the level of detail Audibility DistanceAudibility Distance ViewpointViewpoint –„What is an Avatar looking at?“ Appearance of AvatarsAppearance of Avatars

27. 27 2D vs. 3D? Consider: „Besides the benefits obtained by adding an additional dimension for visualization purposes, this new degree of freedom might introduce new difficulties. Not every application domain has a suitable and usable representation in a 3D.“

28. 28 No, 2D and 3D! 2D elements support the usability of 3D representations2D elements support the usability of 3D representations Combine the AdvantagesCombine the Advantages Selection techniquesSelection techniques

29. 29 3D Electronic Institutions retain the features and advantages of the original metaphorsretain the features and advantages of the original metaphors the essence is „opening“ Electronic Institutions to human usersthe essence is „opening“ Electronic Institutions to human users explore the relationship between humans and software agents in a 3D Virtual Spaceexplore the relationship between humans and software agents in a 3D Virtual Space 3D Electronic Institutions combine the two metaphors of Electronic Institutions and 3D Virtual Worlds into one single metaphor

30. 30 3D Electronic Institutions A contradiction?

31. 31 3D Electronic Institutions Framework consists of three layersFramework consists of three layers Social User Interface Layer (top)Social User Interface Layer (top) –3D Virtual World Communication Layer (middle)Communication Layer (middle) –Connection and Communication Electronic Institution Layer (bottom)Electronic Institution Layer (bottom) –The actual Institution execution

32. 32 3D Electronic Institutions

33. 33 Electronic Institutions Layer Electronic Institutions are designed and specified with ISLANDERElectronic Institutions are designed and specified with ISLANDER AMELIAMELI –Runtime Environment for the Multi-Agent System –Electronic Institution Specification acts as input –Security and reliability –Opens a „communication“ channel for each agent

34. 34 Electronic Institutions Layer Agent-mediated Environment: The GovernorAgent-mediated Environment: The Governor Agents are connected to the infrastructure via GovernorsAgents are connected to the infrastructure via Governors “safe-guarding” the institutions and check whether a particular message is allowed to be said at the current stage or not“safe-guarding” the institutions and check whether a particular message is allowed to be said at the current stage or not

35. 35 3D Electronic Institution

36. 36 Communication Layer Consists of two partsConsists of two parts –Adobe Atmosphere Community Server –Causal Connection Server Adobe Atmosphere Community Server handles multiple users; i.e. the Virtual World is shared among usersAdobe Atmosphere Community Server handles multiple users; i.e. the Virtual World is shared among users Causal Connection Server builds on topCausal Connection Server builds on top

37. 37 Communication Layer Causal Connection: „whenever a change is made in the representation of the system, the system itself changes to maintain a consistent state and vice versa“Causal Connection: „whenever a change is made in the representation of the system, the system itself changes to maintain a consistent state and vice versa“ Reflective systems: „the representation of the system is part of the system itself“Reflective systems: „the representation of the system is part of the system itself“

38. 38 Communication Layer Each instantiated Virtual World (i.e. the human user) communicates via the Communication Layer with its associated GovernorEach instantiated Virtual World (i.e. the human user) communicates via the Communication Layer with its associated Governor Actions performed by users are passed in terms of messages from the user interface to the communication layerActions performed by users are passed in terms of messages from the user interface to the communication layer

39. 39 Communication Layer The Causal Connection Server captures these messages and postpones the actual executionThe Causal Connection Server captures these messages and postpones the actual execution Messages are sent to AMELI in order to determine their „validity“ and checks whether a particular message goes in line with the Electronic Institution rules or notMessages are sent to AMELI in order to determine their „validity“ and checks whether a particular message goes in line with the Electronic Institution rules or not A positive validation results in executing the requested actionA positive validation results in executing the requested action

40. 40 3D Electronic Institution

41. 41 Social User Interface Layer 3D User Interface3D User Interface Adobe Atmosphere PlayerAdobe Atmosphere Player –Free download from Adobe Website –Player is embedded in HTML Page –Multimedia (Sound, Movies, Live Streams...) –A combination of JavaScript and Java is used to communicate with „outer world“ One Player per UserOne Player per User

42. 42 Social User Interface Objectives Immersive and fancy ;-)Immersive and fancy ;-) UsableUsable –consistent, convenient, efficient, clearly designed –not overcrowded, not distracting Social interactionSocial interaction –visual (e.g. via gestures) –facial expressions –sound (even speech) –textual

43. 43 Social User Interface Objectives AwarenessAwareness –knowledge about my own location and the location of other participants –being aware of different types of participants (human users or software agents) –distinguish between internal and external participants (avatar visualization code)

44. 44 Social User Interface Objectives BelievabilityBelievability –Real vs. Surreal visualization; for instance, do we need gravity or collisions –„Can a visualization be too real?“ –Spatial layout of the Virtual World: Euclidian approach vs. „teleportation“

45. 45 Social Virtual World

46. 46 Implementation 3D Electronic Institution Specification3D Electronic Institution Specification –Developed an XML schema which enhances the Electronic Institution Specification Specification TransformerSpecification Transformer –Automatically converts the specification of an Electronic Institutions into a 3D Electronic Institution Specification

47. 47 Annotation Editor Design Tool for System EngineersDesign Tool for System Engineers Input: Electronic Institution SpecificationInput: Electronic Institution Specification Output: 3D Virtual WorldOutput: 3D Virtual World A basic conversion is done automaticallyA basic conversion is done automatically Additional features are incorporated by the System Engineer (eg., furniture)Additional features are incorporated by the System Engineer (eg., furniture)

48. 48 System Architecture

49. 49 Technologies Electronic Institutions LayerElectronic Institutions Layer –IDE-eli (Integrated Development Environment for Electronic Institutions) –Multi-Agent System is implemented in Java –It utilizes JADE and materialized in AMELI –Institution Specification is an XML file specified with ISLANDER

50. 50 Technologies Communication LayerCommunication Layer –Linux –Apache Web Server –Specifications are stored in XML –Adobe Atmosphere Community Server –Open Source Database (mySQL) –Java

51. 51 Technologies Social User Interface LayerSocial User Interface Layer –Adobe Atmosphere Editor (commercial) –Adobe Atmosphere Player (free) –Player is embedded in HTML –JavaScript –Java

52. 52 Conclusion New Metaphor: 3D Electronic InstitutionsNew Metaphor: 3D Electronic Institutions 3D Virtual Worlds & Electronic Institutions Human-centred ViewHuman-centred View TrustTrust Social InteractionSocial Interaction Application Domains: e-Commerce, E-Government, E-Learning...Application Domains: e-Commerce, E-Government, E-Learning...

53. 53 Next Steps Fully implement the Causal Connection Server at the Communication LayerFully implement the Causal Connection Server at the Communication Layer Conduct a Field TrialConduct a Field Trial –test acceptance, usability, performance LearningLearning –Relationship between Agent and Principal Make the Environment Information-RichMake the Environment Information-Rich –e.g., incorporate information bots

54. 54 Discussion