1. A presentation by Dr. Robin Upton (2005-06-20). Available for download at www.altruists.org/ae12 Attribution – NonCommercial - ShareAlike www.altruists.org Altruistic Economics AE 12 : Meta-Evaluations Introduction to Meta-Evaluations Recommended Pre-requisite: AE10: Keeping Score v1.0

2. Introduction to Meta-Evaluations. Processed Data on WWW … format Altruistic Economics data for presentation to end users. … process an unmanageable mass into a small amount of human-readable data; WWW Top 10... www.Altruists.org This site is a monument to the Carp corp. Translations available from BlurflFish.Astalavista,net 0111010101100010111 0101011010110101010 1011010101010101101  #0’s=32 #1’s=26 Meta-evaluations … … are evaluations of evaluations;

3. AE Meta-Evaluations End users have a range of meta-evaluations available … Mass of Interrelated Data Items One Standard: End Users Processed Data on WWW Meta-evaluations process a unique data set in different ways Competing Alternatives... Others WWW

4. “Plain Vanilla” Meta-Evaluation. The AE pilot project will require some basic meta-evaluations A good AE data-standard will allow easy writing of meta-evaluations AE Meta-Evaluations Plain Vanilla Mass of Interrelated Data Items One Standard: AE Transactions Pilot Project Members... Alternatives to arise later Real Life End Users Others

5. “Plain Vanilla” Meta-Evaluations 1. Liars We will look at 3 illustrative cases … 2. Cheats (who try to exploit the system in other ways) 3. Free-Riders (who perpetually receive but never give back) 1. Liars (who intentionally misreport data) “Plain Vanilla”Later systems Principlessimple & illustrativemore comprehensive Aims help users detect problems automatically prevent problems

6. 1. Liars Misleading Predictions  Liars habitually misrepresent reality, to others and/or themselves We refer to the transaction records to re-interpret predictive evaluations. www.altruists.org/404 If self-evaluation [ AE10 ] is to be effective, liars must be fairly easily detected

7. Misleading Predictions Calibration Predictive Evaluation Post-hoc Re-Evaluation Time Self- Evaluation  Use records to compare predictions and post-hoc re-evaluations … Consider the 3 people shown on the right:, & : www.altruists.org/407 Past records suggest their post-hoc evaluations may be in reverse order to their predictions! ? ? ? Previous transaction record

8. Calibration Misleading Re-Evaluations Calibration increases the accuracy of estimates by adjusting them in the light of experience. Add a constant, C, to the predictive evaluations. ? Predictive Evaluation Post-hoc Re-Evaluation Time Self- Evaluation 0 1 2 If… average prediction = -0.2 h average re-evaluation = +2.3 h then C = 2.5 h We calibrate predictions by adding C… …so we expect a prediction of 0.5 h will be re-evaluated to 3 h. Calibrated Predictive Evaluation

9. Misleading Re-Evaluations 2. Cheats but fraudulent evaluations are possible, as only users can know what things are worth to them. www.altruists.org/403 … simple comparison reveals the unappreciative (whether or not they are disingenuous) Reported Subjective Value Self- Evaluation  Re-evaluation [ AE10 ] deters fraudulent transactions… However …

10. 2. Cheats. Integrity of Citation Sets It may be hard to automatically differentiate …  Cheats deliberately break the established protocols & standards. … so we flag up any broken standard. Accidental Breakage hardware/software failure Deliberate Breakage efforts to manipulate the system from

11. Integrity of Citation Sets Missing Citations A Record’s citation set should contain a pointer to everyone who cites it. Eval-1  Birbal Birbal /Eval-1 Aysha /Eval-1 Aysha should always keep the transaction’s citation set up to date. Citation Set Eval-1  A_Transaction Aysha Every record citation ( ) should have a corresponding entry ( ) in the citation set. Aysha evaluates her own transaction, So the citation set should mention this Birbal makes an evaluation of Aysha’s transaction So the citation set should mention this

12. Chris /Eval-1 Missing Citations. Detecting Missing Citations Transaction owners may cheat by deliberately omitting some citations... Eval-1 Birbal A_Transaction Aysha Citation Set Eval-1  Chris Birbal /Eval-1 Protocol requires that Aysha include this citation, but if Chris. makes a negative evaluation… Chris /Eval-1 she may be tempted not to.

13. A_Transaction Detecting Missing Citations. Searching for Cheats If Aysha omits Chris’ citation, asking Chris will reveal Aysha as a cheat… Aysha Eval-1 Birbal Citation Set Eval-1  Chris Birbal /Eval-1 But, how can a third party know to contact Chris ?

14. Searching for Cheats 3. Free Riders Friend Citation Set A_Transaction Aysha Dipti cross-checks the citation set given her by Aysha with Dipti Can I see the citation set? Are you citing Aysha /A_Transaction ? Eval-1  ChrisFriend Are you citing Aysha /A_Transaction ? OK, here it is.. Yes! Are you citing Aysha /A_Transaction ? an independent search for transaction evaluations.

15. 3. Free-Riders. Free-Riding Groups  Free riding is the policy of taking as much as possible without giving anything in return. If Birbal has benefited a lot from other people, but not done anything to help anyone else … Birbal, this will be clear from transaction records. Help Given

16. Free-Riding Groups. Perspectives on Free-Riders Groups can free ride in the same way that individuals can. ChrisBirbal Birbal & Chris have both given and received help from others, so they are not free riders by the previous definition. Considered as a group, they have benefited from others without helping anyone else. But…

17. Perspectives on Free-Riders. Evaluating Help Received From Aysha’s perspective, Chris is a free rider, but from Birbal’s he is not. Opinions may differ about who is a free rider… … so, everyone evaluates others from their own perspective Chris Birbal Aysha

18. Evaluating Help Received Indirect Help Each person asks the question: Birbal Aysha “What has this person done for me?” Cares about Helped Chris Direct reciprocity is not needed in an altruistic environment… since help for friends is easily translated into an equivalent amount of help for self. Chris’ indirect help of Aysha Chris’ direct help of Birbal & Aysha’s care for Birbal

19. Indirect Help. Evaluating Relationships Birbal Aysha Cares about Helped Chris Indirect help flows from those directly helped,. Dipti Cares about Helped Indirectly backwards along care relationships, in proportion to sympathy.

20. Evaluating Relationships. Relationship Credit Help Given & Help Received are helpful meta-evaluations when identifying free-riders “How much have I helped this person ?” (directly & indirectly) Other Me

21. . Evolution of Meta-Evaluations Relationship Credit It is helpful to graph successive interactions: Each successive interaction has a different sympathy This changing pattern of sympathy was referred to in AE2 & AE5 as “Swinging of relationship credit” My Time Friend’s Time My sympathy for my friend My friend’s sympathy for me Direct Indirect Win-Win quadrant    Each relationship starts at (0,0)

22. Evolution of Meta-Evaluations User behaviour & meta-evaluations co-evolve at a speed proportionate to dissatisfaction… Increasing diversity should make exploitation more and more difficult. Summary of Meta-Evaluations Users Update Meta-evaluations Unscrupulous users discover. a successful exploit Improved Meta- Evaluations Plain Vanilla  Dissatisfied Users

23. Summary of Meta-Evaluations Recommended Further Reading: AE13: Modelling Relationships http://www.altruists.org/ae13 A simple but flexible data standard will assist their development Many families of meta-evolutions are likely to evolve over time Meta-evaluations are essential in rendering the mass of records intelligible to users Users may use several different meta-evolutions simultaneously