1. prof. dr. Lambert Schomaker Heterogeneous-Information Integration Kunstmatige Intelligentie / RuG KI2 – 8

2. 2 Heterogeneous-information integration  aka –multi-sensor fusion –multi-expert combination –multi-agent collaboration  The improved use of multiple information sources which are of different unit and scale

3. 3 Heterogeneous-information integration  Examples: –terrorist & weapon classification –friend or foe –forensic evidence collection –finding oil sources –pattern classification by multiple experts –audio-visual speech recognition

4. 4 … different units …  Celsius  microgram  Volt  Ampere  Lumen  probability  pseudo-probability  integer count

5. 5 … different scale …  ratio scale  interval scale  ordinal scale (1 st 2 nd 3 rd 4 th 5 th 6 th … )  nominal scale –yes/no –green red purple –good bad ugly –true/false 0.0AB AB

6. 6 Architecture, example real world Expert 1, NN Expert 2, Rule-based Measurement i Expert 3, Bayesian agent k agent lagent m Measurement j COMBINE DECISION

7. 7 How to combine heterogeneous information?  trained parameter-estimation methods  context-free methods

8. 8 Trained, parametric combination methods  Use a trainable function approximator: –mean field (linear, weights) –multi-layer perceptron (NN) –polynomial –Bayes!  cumbersome: train individual components, train the combination  if a new module or expert is added, the system must be completely retrained!  independent training sets are needed for the single functions and for the combination function

9. 9 Context-free combination methods  majority voting  plurality voting  product rule  sum rule  rank combination schemes

10. 10 Voting  A candidate c i is a person, object or proposal, and C is the set of all possible candidates, and C e is the set of candidates taking place in a particular election  A voter is a function v j : C e  R, in words, each candidate partaking in the election obtains a real- valued confidence of v j in c i

11. 11 Election  An election is a tuple (C e,V e ) where C e  C and V e  V, such that  v j  Ve v j : C e  R yielding |Ve| orderings of the candidates, in R

12. 12 Voting system criteria  Condorcet winner: will win from all candidates if elections were held in a pairwise fashion. A Condorcet loser could exist too  Consistency: if c i is a winner for voters Vk and for voters Vm, then c i should also be the winner if the election is based on {Vk  Vm}

13. 13 More voting-system criteria  Monotonicity: if votes become available, this should not affect the existing valuation (humans often react non-monotonously in a sequential voting procedure). Also, voting procedures which eliminate candidates one by one are non monotonous.  Pareto optimality: the voting system choses c x over c y if all voters choose c x over c y

14. 14 Example: majority vote in unreliable but independent experts

15. 15 Special case: Borda rank combination  Each of N voters ranks M candidates  The assumption is that an optimal ranking exists  Individual voters utilize an unknown evaluation function v j : C e  R where j=[1,N], e=[1,M]  Evaluations are sorted, such that the ‘best’ evaluation ranks 1, etc. up to M, ‘worst’

16. 16 Example: Evaluation scores 0-100 BeerVoter AVoter BVoter C Heineken453099.1 Grolsch423170. Hertog Jan 30.21231.2 Duvel10.4540.8 Koninck804090.9

17. 17 Example: Ranks BeerVoter AVoter BVoter C Heineken2 nd 3 rd 1 st Grolsch3 rd 2 nd 3 rd Hertog Jan 4 th 5 th Duvel5 th 4 th Koninck1 st 2 nd

18. 18 Example: Ranks BeerVoter AVoter BVoter CCombined Heineken231? Grolsch323? Hertog Jan445? Duvel554? Koninck112?

19. 19 How to combine rankings?  Several models are possible –standard Borda: take the average (best guess) –also: –median rank (disregard outlying ranks) –mode of ranks (plurality of ranks) –min of ranks (optimistic) –max of ranks (pessimistic)

20. 20 standard Borda: mean rank BeerVoter AVoter BVoter CCombined Heineken2312  2 nd Grolsch3232.67  3 rd Hertog Jan4454.33  4 th Duvel5544.67  5 th Koninck1121.33  1 st

21. 21 modal rank BeerVoter AVoter BVoter CCombined Heineken2312 Grolsch3233 Hertog Jan 4454 Duvel5545 Koninck1121

22. 22 min rank BeerVoter AVoter BVoter CCombined Heineken2311 Grolsch3232 Hertog Jan 4454 Duvel5544 Koninck1121

23. 23 min rank BeerVoter AVoter BVoter CCombined Heineken2311 Grolsch3232 Hertog Jan 4454 Duvel5544 Koninck1121 How to solve ties?

24. 24 max rank BeerVoter AVoter BVoter CCombined Heineken2313 Grolsch3233 Hertog Jan 4455 Duvel5545 Koninck1122

25. 25 How to solve ties in the combined Borda ranking?  Random choice of candidates  If the validity of the voters’ judgment is known: take the rank of the best voter  But: then we digress towards knowledge- based and probabilistic schemes

26. 26 Example non-stochastic tie solving: Voter C is known to be superior to A, B BeerVoter AVoter BVoter CCombined Heineken2311 Grolsch3232 Hertog Jan 4454545 Duvel5544444 Koninck1121

27. 27 How to choose for a combination method?  mean? mode? median? min? max?  Empirical tests are needed, mostly  The type of question to be answered is important  Example: “sportsperson of the year contest”

28. 28 How to choose for a combination method?  The type of question to be answered is important  Example: “sportsperson of the year contest”  Not the average rank over N sports for M sportspersons but the minimum rank (best played sport) is indicative