1. A Ratio. A Ratio. My Kingdom for a Ratio
James Shanteau
Kansas State University

2. My Problem*
My problem is that I have been persecuted by a math-ematical formula. For years, this formula has followed me around, has intruded in my most private analyses, and has assaulted me from the pages of our most public journals.
This formulation assumes a variety of disguises, being sometimes expressed openly, other times not, but never changing so much as to be unrecognizable. The persis-tence with which this formulation plagues me is far more than a random accident. There is a design behind it, some pattern governing its appearances. Either there is some-thing unusual about the formulation or else I am suffering from delusions of persecution.
(*With apologies to George A. Miller)

3. My Problem*
My problem is that I have been persecuted by a math-ematical formula. For years, this formula has followed me around, has intruded in my most private analyses, and has assaulted me from the pages of our most public journals.
This formula assumes a variety of disguises, being sometimes expressed openly, other times not, but never changing so much as to be unrecognizable. The persis-tence with which this formula plagues me is far more than a random accident. There is a design behind it, some pattern governing its appearances. Either there is some-thing unusual about the formula or else I am suffering from delusions of persecution.
(*With apologies to George A. Miller)

4. Examples • Speed-Accuracy Tradeoffs • Intelligence Quotient
• Size-Distance Constancy
• Performance = Motivation x Ability
• Signal-Noise Ratio
• Cost-Benefit Analysis
• Input-Output Analyses
• Price-Quality Relation
• F-Ratio & Chi-Square

5. Examples • Speed-Accuracy Tradeoffs • Intelligence Quotient
• Size-Distance Constancy
• Performance = Motivation x Ability
• Signal-Noise Ratio
• Cost-Benefit Analysis
• Input-Output Analyses
• Price-Quality Relation
• F-Ratio & Chi-Square
* Bayes Theorem!!

6. Commonalities
In these (& many more) cases, the common element is that the formula takes the form of a Ratio Score.

7. Commonalities
In these (& many more) cases, the common element is that the formula takes the form of a Ratio Score.
Note: Log of a Ratio = Difference Score, so Change Scores, Before-After Comparisons, & other Delta Scores can also can be seen as Ratio Scores.

8. Commonalities
In these (& many more) cases, the common element is that the formula takes the form of a Ratio Score.
Note: Log of a Ratio = Difference Score, so Change Scores, Before-After Comparisons, & other Delta Scores can also can be seen as Ratio Scores.
Another commonality is that ratios represent tradeoffs between incompatible goals, ie, ratios are used to describe tradeoffs.

9. Three Examples
Three personal examples illustrate advantage of ratio (tradeoff) scores over simple (absolute) scores:

10. Three Examples
Three personal examples illustrate advantage of ratio (tradeoff) scores over simple (absolute) scores:
Replacement of Absolute Thresholds by Signal Detection Theory (SDT)

11. Three Examples
Three personal examples illustrate advantage of ratio (tradeoff) scores over simple (absolute) scores:
Replacement of Absolute Thresholds by Signal Detection Theory (SDT)
(2) Switch from Learning Outcomes to Change Scores = Amount Gained

12. Three Examples
Three personal examples illustrate advantage of ratio (tradeoff) scores over simple (absolute) scores:
Replacement of Absolute Thresholds by Signal Detection Theory (SDT)
(2) Switch from Learning Outcomes to Change Scores = Amount Gained
(3) Change from % correct scoring for evaluat-ing experts to CWS Performance Measure

13. Signal Detection Theory
In traditional psychophysics, eg, thresholds were measured as absolute values or hits.
However, this approach failed to take false alarms into account.

14. Signal Detection Theory
In traditional psychophysics, eg, thresholds were measured as absolute values or hits.
However, this approach failed to take false alarms into account.
SDT incorporates both, in that
d’ = f(Hits) / f(FA) = Discrimination
This changes the concept of thresholds from a fixed value to a relative score.

15. Input-Output Analyses
Traditional measures of learning look at outcome scores, eg, on standardized tests
Such scores are used for promotions (or not) for teachers & changes to pedagogy
But better students will out-perform weaker students, regardless of teacher or pedagogy

16. Input-Output Analyses
Traditional measures of learning looked just at outcome scores, eg, on standardized tests
Such scores are used for promotions (or not) for teachers & changes to pedagogy
But better students will out-perform weaker students, regardless of teacher or pedagogy
Need to look instead at Performance Gains by Input-Output Analyses
Note: Nominee to Head Department of Educa-tion did not understand this difference

17. CWS
Evaluation of Expertise typically based on % correct scores compared to “gold standards”
Problems: (1) Experts work in settings where correct answers don’t exist, (2) Perfor-mance can improve beyond 100% correct, (3) Correct answers change over time.

18. CWS
Evaluation of Expertise typically based on % correct scores compared to “gold standards”
Problems: (1) Experts work in settings where correct answers don’t exist, (2) Perfor-mance can improve beyond 100% correct, (3) Correct answers change over time.
CWS looks at properties of performance, rather than agreement with “gold standard” answers
CWS = Discrimination/Consistency

19. Normalization vs Tradeoffs
Ratios can result when scores are normalized,
eg, test scores expressed as percentiles
IQ scores normalized for age of children
IQ = (Mental Age / Chronological Age) x 100
Note: Adult IQ scores reflect group norms
My concern is with tradeoff ratios, ie, where decisions are made, not normalization

20. Implicit vs Explicit Tradeoffs
Implicit tradeoffs = environmental constraints
eg, Price-Quality Tradeoffs occur because it is normally impossible to get high quality at low prices

21. Implicit vs Explicit Tradeoffs
Implicit tradeoffs = environmental constraints
eg, Price-Quality Tradeoffs occur because it is normally impossible to get high quality at low prices
Explicit tradeoffs = human constraints, eg, Speed-Accuracy Tradeoffs occur when decision are made about how much accuracy to give up to gain higher speed

22. Generalized Formulation
Generalized Ratio = f (good stuff / bad stuff), or = f (good stuff / total stuff), where
total stuff = good stuff + bad stuff
Either way, ratios reflect tradeoffs

23. Generalized CWS Ratio
Cochran-Weiss-Shanteau (CWS) describes performance as function of
Discrimination & Consistency, where
CWS = Discrimination / Inconsistency
Note: reverse scoring of denominator necessary so that higher scores are better

24. Generalized CWS Ratio
Cochran-Weiss-Shanteau (CWS) describes performance as function of
Discrimination & Consistency, where
CWS = Discrimination / Inconsistency
Note: reverse scoring of denominator necessary so that higher scores are better
Generalized CWS = S Good / S Bad
This allows for other benefits & costs, eg, time & money

25. Conclusions
Making tradeoffs is key step in almost all interesting decisions, esp. by Experts.
Such tradeoffs captured by ratios.

26. Conclusions
Making tradeoffs is key step in almost all interesting decisions, esp. by Experts.
Such tradeoffs captured by ratios.
When tradeoffs ignored (or overlooked), important components of problem missing.
As a rule-of-thumb, fields advance when tradeoff relations recognized, eg, SDT.

27. Final Comments & Questions?

28. Ratios in Statistics Most statistical tests are based on ratios, eg,
• F-Ratios (& t-tests)
• Chi-Square
• Bayes Likelihood Ratio

29. Ratios in Statistics Most statistical tests are based on ratios, eg,
• F-Ratios (& t-tests)
• Chi-Square
• Bayes Likelihood Ratio
Generally, these are
• Variance ratios, where
Variance Accounted For / Total Variance
However, statistical ratios are not of concern for analysis of cognitive tradeoffs

30. Dominance (Pareto) Frontier
For a set of options with negative correlations between attributes (eg, due to price-quality) the best options defined by dominance, ie, they dominant weak options.

31. Dominance (Pareto) Frontier
For a set of options with negative correlations between attributes (eg, due to price-quality) the best options defined by dominance, ie, they dominant weak options.
Thus, there are environmental limits to trade-offs that are possible, eg, between speed and fuel efficiency.

32. James Shanteau Kansas State University