Brendan Neville and Jeremy Pitt Imperial College London A Computational Framework for Social Agents in Agent Mediated E-Commerce Brendan Neville and Jeremy Pitt Imperial College London

Abstract Malicious and incompetent agents are a potential hazard to open distributed ecommerce applications. Agent’s computational framework integration of socio-cognitive and economic theories. Artificial market place scenario. So why are we researching social agents in e-commerce, well its because malicious and incompetent agents are a potential hazard to open distributed e-commerce applications. We believe that a computational framework which formalises social theories of trust, reputation, recommendation and learning from direct experience will enable agents to against the actions of these malicious or incompetent agents. Our framework integrates these social elements with an agent’s economic reasoning resulting in an agent whose behaviour in commercial transactions is influenced by its social interactions, whilst being motivated and constrained even somewhat rationalised by economic considerations. The framework is presented in the context of an artificial market place scenario which is part of a simulation environment currently under development.

Approach Our approach to developing this framework focuses on four main steps. This is an adaptation of the synthetic method first formalised by Steels in 1995 We begin by formalising appropriate theories within the social sciences into computational structures and algorithms, these economic and social theories determine the agents behaviour in the multi agent system. By simulating a system composed of such agents and observing the outcome, we aim to tailor the formalisms to achieve the desired performance. This performance evaluation being based on the fairness and efficiency of the resulting communities. This fairness and efficiency will be quantified using practices established by welfare economics.

Agenda.. Our Simulation Scenario Agent Economic and Socio-Cognitive formalisms Preliminary Results Conclusions and Further work To start I will Define a retail scenario which will act as our artificial system’s virtual market place. Then I’m going to very briefly discuss the economic and social models of the agents participating in our multi agent system. If we have time I’ll Present some very preliminary simulation results, demonstrating the agents economic behaviour. And Finally I’ll conclude with a quick summary, and an outline of our future goals.

The Retail Scenario - overview Buying and selling of goods or services E.g. Media Products or Information Retrieval Two Groups: Producers create and sell goods and services Consumers purchase and consume goods and services Producers aim to maximise Profit Consumer aim to maximise Utility The scenario we have chosen for our simulation work is an example of a manufacturing retail marketplace. This market place facilitates the buying and selling of goods and services. In this scenario there are two groups: Producers create the goods or services and sell them. Consumers purchase and consume the producers goods. Producer agents aim to maximise the profit of their owners Whilst consumer agents aim to maximise the utility gained by their owners.

The Retail Scenario - msg sequence chart Consumer Owner Consumer Agent Consumer Agent n x Producer Agents Request Service Query Price Inform Accept and Pay or Reject Provide Service Provide Service

The Producer Agent -economic model Producer cost function To maximise Profit the agent must optimally set its price. Price setting Derivative Follower Reputation, Peer price and Consumer willingness to pay. Total Cost Function Increasing returns In this scenario our specification for producer agents is simply composed of an economic model, an assigned level of competence and a character type which determines its inclination towards malicious behaviour. The producers are parameterised with a cost function that reflects its increasing returns to scale, as the quantity produced increases which is followed by diminishing returns to scale. The graph shows an example total cost curve. As mentioned earlier a producer agent’s goal is the maximisation of its owner’s profit. To achieve this the agent must optimally set the price per unit resource sold. If it sets its price to low, demand for its resource will be so high a that the cost of supplying another unit will be higher than the money received for it. Too high a price and it doesn’t sell enough units to maximise its profits. Price Setting – In the paper I outline that the producers would employ a simple gradient following algorithm to set its price. However preliminary results show that it too often fails to find the global maximum profit. The answer to this appears to be a more complex strategy. Utilising three factors, Its reputation and that of it’s peer producers its peer producers prices and the consumers willingness to pay. Decreasing returns

The Producer Agent - behaviour determinants Two main determinants: 1. Character type determines whether the producer attempts to supply the consumers order. 2. Competence defined as a probability of successful service provision given the producer attempts to provide. Some producers more competent than others In our simulation work we model a producer who is either incompetent or malicious or both by using two variables. The first is character type, this determines if the agent is one who actively deceives, by not fulfilling orders which have been paid for. The second variable is a probability that the producer will successful supply the consumer given that it tries. Now moving on to the Consumer agent.

The Consumer Agent - framework [Recommendations] [Experiences] Trust Belief Outcome Utilities Opportunity to Trust Reputation Direct Experience [Credibility(Recommendation)] [Credibility(Experience)] Decision to Trust Confidence(Direct Experience) Confidence(Reputation) Peer makes a Recommendation Action of Trusting [Experiences] of peers as recommenders Trust in Recommenders Decision to Trust Recommenders This figure presents the main social and economic components of the framework. And more importantly the relationships between them. All the formulas used in the framework are detailed in the paper, I’m not going to present all of them today. Instead I’m going to cut the detail out of the middle of the framework.

The Consumer Agent - framework Opportunity to Trust Action of Trusting Outcome Utilities Decision to Trust Trust Belief Starting in the top left hand corner, the agent is faced with an opportunity and need to trust another agent about a generic task. In our retail scenario this would be selecting a producer to trust with payment prior to receiving the service. The first step for the agent is the calculation of the possible outcome utilities. [Recommendations] [Experiences] Peer makes a Recommendation

The Consumer Agent - economic model Discrete Outcome Utilities Estimate the utility that could be gained or lost by trusting the producer: U(Success) U(Failure) Diminishing Marginal Utility Each additional unit of resource consumed is valued less than the previous. More of a resource is always better than less. These Discrete Outcome Utilities estimate the utility that could be gained or lost as a result of trusting the producer. The basis on which the consumer agent calculates the expected utility of a successful outcome is the economic theory of diminishing marginal utility. This states that as more of a resource is consumed, the less utility an additional unit will provide. While more is always better than less.

The Consumer Agent - economic model To maximise U(Success) the consumer must order S1 units, where S=S1 solves: 100 200 300 400 500 600 4 8 12 16 20 24 28 Resource Units Ordered (S) Marginal Utility of Unit To calculate the Utilities This theory is reflected by the function dU/dS. Given the price per unit resource, the consumer can maximise its utility by solving marginal utility function equal to the price per unit. If it consumes more than this the utility gained by consuming the additional unit will be less than what it has paid for it. Consuming less would be sub-optimal. To estimate the utility gained from consuming S1 units of resource the agent integrates the marginal utility function between 0 and S1 and subtracts the price paid. In the event that the consumer is deceived and thus never receives the goods or services it ordered, it will have lost the money paid for them i.e. the quantity ordered multiplied by the price per unit. Price S1

The Consumer Agent - framework Opportunity to Trust Action of Trusting Outcome Utilities Decision to Trust Trust Belief Confidence(Reputation) Confidence(Direct Experience) Reputation Direct Experience [Credibility(Recommendation)] [Credibility(Experience)] Specifically our representation of trust borrows from theories by Castelfranchi and Falcone , they define trust not only as a truster’s evaluation of the trustee but also as the decision to and the action of trusting. [Recommendations] [Experiences] Decision to Trust Recommenders [Experiences] of peers as recommenders Trust in Recommenders Peer makes a Recommendation

The Consumer Agent - socio-cognitive model Trust Belief (DoTA,C,τ) Is the degree to which Agent A trusts Agent C to successfully complete a task τ, it is a subjective probability. (1 - DoTA,C,τ) Agent A`s belief that Agent C will fail to complete a task τ. Trust is the resultant belief of one agent about another based on its direct experiences of that other agent and the testimony of peers. Our computational representation of an agent’s trust belief is based on the formal model of Castelfranchi and Falcone. The agent’s trust belief is the degree to which Agent A trusts Agent C to successfully complete a task τ, it is represented by a subjective probability. Conversely, (1 - DoTA,C,τ) Agent A`s belief that Agent C will fail to complete a task τ. The trust belief is the result of the Agents direct experiences of that other agent and the testimony of peers.

The Consumer Agent - socio-cognitive model Trust belief a function of: Its belief about the Reputation of the trustee, The agents Direct Experience of the trustee, and the agents confidence in the respective accuracies of those beliefs. The Trust belief is now defined as a function of Its belief about the Reputation of the trustee, Its Direct Experience of the trustee, and the agents confidence in the respective accuracies of those beliefs. The dynamic of this equation is that: Agents confident that they have a large amount of recent experience of agent C about task tau are less reliant on peer testimony to determine the trustworthiness of agent C. However agents with little to no recent experience will rely on peer testimony to determine the trustworthiness of agent C.

The Consumer Agent - economic meets socio-cognitive Decision to Trust The agent will decide not to trust an agent if the Expected Utility of trusting that agent is less than or equal to zero. If the agent has a choice of more than one agent to trust it will decide to trust the one with the highest positive Expected Utility. In making the decision to trust The outcome utilities act as the economic influence And the social factor is the agent’s trust belief. The agent will decide not to trust an agent if the Expected Utility of trusting that agent is less than or equal to zero. If the agent has a choice of more than one agent to trust it will decide to trust the one with the highest positive Expected Utility.

The Consumer Agent - framework Opportunity to Trust Action of Trusting [Experiences] of peers as recommenders Outcome Utilities Decision to Trust Trust Belief Confidence(Reputation) Confidence(Direct Experience) Reputation Direct Experience [Credibility(Recommendation)] [Credibility(Experience)] If the agent decides to trust the producer this leads to the action of trusting, in our retail scenario this is sending an order and making payment. Or assigning a positive credibility to a recommendation. This creates experiences for the agent thus closing the feedback loop. [Recommendations] [Experiences] Decision to Trust Recommenders Trust in Recommenders Peer makes a Recommendation

Preliminary Results Producer Perspective Monopoly With Competition Price (over last 25 t) t t Profit (over last 25 t) t t

Preliminary Results Consumer Perspective Monopoly With Competition Money Spent (over last 100 t) t t Net Utility (over last 100 t) t t

Previous Results - direct experience vs. incompetence Control

Previous Results - direct experience vs. incompetence

In Summary Scenario for investigating the effect of the framework on the performance of an e-commerce retail market. Framework incorporates elements from Sociology Economics To: ensure rational behaviour discourage malicious behaviour. Scenario for investigating the effect of the framework on the performance of an e-commerce retail market. Framework incorporates elements from Sociology Economics So that it can ensure rational behaviour discourage malicious behaviour.

Further Work Simulation The end The aim of future endeavors will be to show through simulation that social order can be created and supported by the introduction of the socio-cognitive framework in support of the agent’s economic reasoning. We aim to demonstrate that the framework benefits benevolent members of the agent mediated market place and hence the human society in which they are embedded. Our simulation package will be used to provide experimental verification for the framework in a number of scenarios, by simulating heterogeneous groups of producer and consumer agents.