Tags and Image Scoring for Robust Cooperation By Nathan Griffiths Presented at AAMAS 2008

Introduction Cooperation in many multi-agent systems is based on reciprocity. In our study of Prisoner’s Dilemma, for example, the decision to cooperate or defect is often based on how the other agent has behaved in the past. Modern multi-agent systems may be so large that any two selected agents have never interacted before. They have no history to decide if to cooperate or not. Several approaches exist that encourage cooperation based on indicators other than mutual history. The concept of a ‘cheating’ agent that accepts donations (cooperation) but doesn’t donate in return (defects), even when the system rules says it should, is a major part of the paper.

Existing Approaches Riolo, Cohen and Axelrod (RCA): – Uses tags that let agents know how similar they are to other agents. Similarity between agents makes them better candidates for cooperation. Hales and Edmonds (HE): – Uses RCA’s approach within a peer-to-peer (P2P) network, with a couple of additions that allow agents to adopt the tag of another more successful agent. Matlock and Sen (MS): – Add the ability to cooperate with other agents that are outside of the current tag group. Nowak and Sigmund (NS): – Each agent has an image score that is visible to all other agents. This score increases or decreases as the agent cooperates with other agents.

The P2P Environment P2P network made up of many agents compared to the number of interactions. This means that agents involved in most interactions have never dealt with each other before. Each agent has a fixed number of connections to neighbors in the network. The agent has no way to change (or re- wire) these connections. Some percentage of the agents in the network will be cheaters. The ‘donation’ scenario is used where cooperation is seen as a donation. When an agent donates to another agent it incurs a cost of 0.1 while the recipient gains a benefit of 1.0. The benefit always exceeds the cost.

Interested Audience Any single agent doesn’t know the trustworthiness of any other agent because: – The probability is that they have not had previous interactions. – As previously stated, an image score is visible to other agents and goes up or down depending on the level of cooperation of an agent. Unfortunately, a cheater will lie about it’s image score.

Tolerance An agents tolerance level affects its willingness to cooperate. A high tolerance means an agent is more likely to cooperate, while a low tolerance makes it less likely to cooperate. An agent can change its tolerance level at run time based on the interactions it has with other agents. In this study, rather than deciding if a specific agent is thought to be cooperative, it is decided if the entire neighborhood is thought to be cooperative. This is based on the observations of interactions in the neighborhood that were made by an interested audience.

Experiment Setup The experiment basically follows the RCA approach with parameter settings as follows: – Network size (N) = 100 to 5000 nodes – Set of neighbors (n) = 49 – Tag values (t) = 0 to 1 – Tolerance Values (T) = to 1 – Tag Mutation probability (M t ) = 0.01 – Tolerance Mutation probability (M T ) = 0.01 – Interval Based Reproduction (P) = 3 to 20 interactions – Length of simulations = 500 to 5000 generations When cheaters are introduced, cooperation disappears as shown in Figures 1 and 2. The author used the PeerSim P2P Simulator for the experiments (

Results Figures 1 through 6 show results for the simulations. This approach shows significant improvement over the RCA approach, even in the base case of 0% cheaters.

Miscellaneous Issues Neighbors observe the interactions between other neighbors with some probability p o. The bottom of page 578 states that probabilities of 1 for all neighbors means that an iteration will be observed by all n neighbors. The first paragraph in section 4 describes the creation of the next generation as: – “after a certain number of interactions an agent compares itself to another selected at random. If the other agent is more successful then its details are copied, otherwise no change is made”. Whether all agents go through this process at the same time or on an individual time basis is unclear. Cheating is done by accepting donations from others without ever donating. I don’t believe cheating is done by altering a tag since there is no apparent advantage is doing so. There is some mention made of cheaters lying about their image score. The footnote on page 577 does say that a negative tolerance adjustment can be made, but it does appear to be a very small value.