1. CSH, May 23, 20051 Deconstructing the Law of Effect C. R. Gallistel Rutgers Center for Cognitive Science

2. CSH, May 23, 20052 Matching Is An innate strategy foraging strategy (or, possibly, an innate principle of choice = melioration) The apportionment of foraging time (and effort) in accord with the incomes obtained from the foraging locations:  T 1 /T 2 =I 1 /I 2, where T i = time invested in Location i & I i = income obtained from Location i  Equivalently: T 1 /(T 1 +T 2 ) = R 1 /(R 1 +R 2 ) & T 2 /(T 1 +T 2 ) = R 2 /(R 1 +R 2 ) [“Herrnstein fractions”] An evolutionarily stable strategy

3. CSH, May 23, 20053 Income vs Return Income (I i ) = (reward from Loc i)/(unit time spent foraging, without regard to where time is invested, that is, summed over i) Return (R i ) = (reward from Loc i)/(unit time spent foraging at a given Loc i), i.e., per unit time invested in a location Matching equates returns; T 1 /T 2 =I 1 /I 2 ---> I 1 /T 1 = T 2 /I 2

4. CSH, May 23, 20054 Model 1: Law of Effect Matching driven by inequality of returns Subject compares returns and reapportions investments to favor location yielding greater return If income is only loosely connected to investment, then this decreases return on favored side and increases it on disfavored side Eventually, a dynamic equilibrium is attained (the equation of returns) No change-detector necessary Hill climbing = slow!

5. CSH, May 23, 20055 Model 2: Innate Strategy Pure Feed Forward First proposed in a general way by Gene Heyman (“matching is unconditioned behavior”) Leaving rate parameters the inverse of the proportions among the incomes A separate process detects changes in income Income estimator works on small samples taken from the period between the time when a change point is (retrospectively) estimated to have occurred and the time at which the change is detected

6. CSH, May 23, 20056 Old Rat Results

7. CSH, May 23, 20057 Appearance of Matching in the Experimentally Naïve Mouse Mouse shuttles back and forth between two feeding hoppers Hopper entries monitored by IR beams

8. CSH, May 23, 20058 Coupling between investment and income Slightly coupled: traditional concurrent VIs with indefinite hold for harvesting Income completely independent of investment (pellets accumulate at random rates; the mouse harvests the accumulation whenever it comes Complete coupling: the schedule clock only for a hopper only runs when the mouse has its head in that hopper

9. CSH, May 23, 20059 Tracking Matching Compute difference in “Herrnstein fractions” after each feeding  R 1 /(R 1 +R 2 )- R 2 /(R 1 +R 2 )=(R 1 - R 2 )/ (R 1 +R 2 ) = Feeding Imbalance; possible values -1 & 1, depending on whether feeding was at 1 or 2  T 1 /(T 1 +T 2 )-T 2 /(T 1 +T 2 )=(T 1 - T 2 )/(T 1 +T 2 ) = Investment Preference; ranges -1 to 1, depending on how mouse spent its time between feedings  Cumulate differences: slope of cumulative record gives average values; when slopes of the 2 cumulative records are the same mouse is matching

10. CSH, May 23, 200510 Condition 1 Relative income jointly determined by programmed ratio and sampling behavior Naïve mouse samples rarely Hence initial ratio of incomes may be 1:1 no matter what the programmed ratio Mouse is always matching Changes in relative income and changes in relative investment coincide (even though income only very loosely connected to investment)

11. CSH, May 23, 200511 All Mice in Cond 1 They are, in general, as close to matching at the beginning of training as they are at the end of it

12. CSH, May 23, 200512 Condition 2 Income imbalances now constant from beginning (as intended) Mice match from the beginning

13. CSH, May 23, 200513 Condition 3 When income is directly tied to investment, the only place in behavioral parameter space for matching is exclusive devotion to one or the other option Thus, on our model, preference should be (almost) all-or-none, but unstable

14. CSH, May 23, 200514 A Closer Look The session in which the major reversal occurred Nothing in the returns justifies the switch! The change in income does justify the switch This change is caused by the switch, but, in our model, the animal doesn’t know that, because it takes no account of the impact of its behavior on what it observes

15. CSH, May 23, 200515 Terminal Economic Weirdness Animals are oblivious to the impact of their own behavior on the rewards they receive (pace Skinner & the entire economics profession)

16. CSH, May 23, 200516 Matching & the Hunter-Gatherer World Hunting & gathering, unlike farming, do not bring food into existence, they merely exploit what is there Intelligent sampling of a changeable environment is essential in hunter-gathering Matching maximizes income when income is environment-limited rather than investment-limited Matching samples the world intelligently