Research for Management Policy Shyam Sunder, Yale University Yale-Great Lakes Research Conference Great Lakes Institute of Management Chennai, India, December 26, 2010

An Overview Social science and social engineering Robustness of findings to their discovery Field data: Causal direction, correlation of hypotheticals Experiments and bench testing Social science as instrument of policy Accounting policy beyond social sciences Challenges for policy research, historical experience Closed or open systems? 9/10/2015Sunder, Research for Accounting Policy 2

9/10/2015Sunder, Research for Accounting Policy 3 Social sciences as instruments of social engineering Social science as the dominant model of business research since mid-20 th century Although engineering originated well before the origins of natural sciences, advances in natural sciences have enabled us to engineer great many artifacts we consider indispensable today Management, too, originated well before policy research; but it does not seem unreasonable to think that we can also take advantage of learning from social sciences to make better social policy

9/10/2015Sunder, Research for Accounting Policy 4 Limits of Parallels This parallel can take us only so far Natural sciences search for, and identify, laws of nature valid across time and space Validity, replicability, and predictability of these laws confers prestige on sciences But there is another form of scholarship on our campuses--humanities that regard the behavior of sentient beings as infinitely variable; each of character in Iliad, Macbeth, or Ramayana is unique They pursue eternal truths, but admit no laws

9/10/2015Sunder, Research for Accounting Policy 5 What are social sciences? “Social” recognizes that the subject of study are sentient beings (with free will that humanists recognize), not marbles or atoms without will (that scientists study) “Science” seeks the prestige associated with the search for eternal laws Neither sciences nor humanities allow much room for laws of human behavior we seek in social sciences Free will and laws of behavior do not sit well together; we want but can’t have it both ways

9/10/2015Sunder, Research for Accounting Policy 6 Laws of Social Sciences To serve as a basis for social policy, the “laws” of social sciences must have stability (be robust to their own discovery) Since humans learn and adapt, social science findings can alter behavior in ways that tend to invalidate the findings Findings which are robust to such adaptation can be called “laws” of social sciences, and may serve as the basis for social policy

9/10/2015Sunder, Research for Accounting Policy 7 How Robust Are Our Findings? Independent of the method of research we use, robustness of findings (to their own discovery) is a pre-requisite for their use as basis for policy Like unclaimed dollar bills on side-walk, many findings (e.g., small firm and Monday effects) disappear upon being reported There are other findings (e.g., determination of price by intersection of demand and supply) are robust in this sense (not merely statistically) So the first pre-requisite for usefulness of any research findings for policy is this stability Most such laws are properties of institutions, not behavior [1]

9/10/2015Sunder, Research for Accounting Policy 8 Causal Link Policy makers want to know if the manipulation of the policy variable under consideration has a directional (causal) link to the desired objective. Correlation does nothing for them. Yet, the problem of establishing a causal link between two variables on the basis of field data remains largely unsolved due to endogeneity Experimental methods have been presented as an alternative to address this problem, but they, too, have limitations of their own Consider both approaches briefly

9/10/2015Sunder, Research for Accounting Policy 9 Problems of Inference from Field Data: Causal Direction Labeling of correlation as cause is more of a rule than an exception in accounting research journals When correctly labeled as correlation, what can the policy maker do with the finding? To claim that the finding is “consistent” with Hypothesis X fails to point out that –It is also consistent with innumerable other hypotheses not mentioned in the report, and –No hypothesis has been rejected (violating the essence of Fisher-Neyman-Pearson framework)

Accepting or Failing to Reject the Null Hypothesis It is important to note the philosophical difference between accepting the null hypothesis and simply failing to reject it. The "fail to reject" terminology highlights the fact that the null hypothesis is assumed to be true from the start of the test; if there is a lack of evidence against it, it simply continues to be assumed true. The phrase "accept the null hypothesis" may suggest it has been proved simply because it has not been disproved, a logical fallacy known as the argument from ignorance. Unless a test with particularly high power is used, the idea of "accepting" the null hypothesis may be dangerous.argument from ignorancepower

“Absence of evidence is not evidence of absence” When a researcher writes the qualified statement "we found no statistically significant difference," which is then misquoted by others as "they found that there was no difference." Actually, statistics cannot be used to prove that there is exactly zero difference between two populations. Failing to find evidence that there is a difference does not constitute evidence that there is no difference. This principle is sometimes described by the maxim "Absence of evidence is not evidence of absence."

Statistical Significance Attempts to educate researchers on how to avoid pitfalls of using statistical significance have had little success. In the papers "Significance Tests Harm Progress in Forecasting," and "Statistical Significance Tests are Unnecessary Even When Properly Done,“ Armstrong makes the case that even when done properly, statistical significance tests are of no value. A number of attempts failed to find empirical evidence supporting the use of significance tests. Tests of statistical significance are harmful to the development of scientific knowledge because they distract researchers from the use of proper methods. Armstrong suggests authors should avoid tests of statistical significance; instead, they should report on effect sizes, confidence intervals, replications/extensions, and meta-analyses. J. Scott Armstrong

Choosing the null: Ex ante, or after looking at the data? Some statisticians have commented that pure "significance testing" has what is actually a rather strange goal of detecting the existence of a "real" difference between two populations. In practice a difference can almost always be found given a large enough sample. The typically more relevant goal of science is a determination of causal effect size. The amount and nature of the difference, in other words, is what should be studied. Hypothesis testing is controversial when the alternative hypothesis is suspected to be true at the outset of the experiment, making the null hypothesis the reverse of what the experimenter actually believes; it is put forward as a straw man only to allow the data to contradict it. Many statisticians have pointed out that rejecting the null hypothesis says nothing or very little about the likelihood that the null is true.

9/10/2015Sunder, Research for Accounting Policy 14 Problems of Inference from Field Data: Hypotheticals and Efficiency Our attempt to produce policy-relevant research often take the following form: Info. Sys. 1  Price system 1 Info. Sys. 2  Price system 2 Suppose information system 1 and price function 1 are the status quo and the policy maker wants to know the consequences of changing the information system from 1 to 2

9/10/2015Sunder, Research for Accounting Policy 15 Problems of Inference from Field Data: Hypotheticals and Efficiency We can gather data on accounting numbers and prices under status quo and estimate their statistical relationship, R(1) In many situations, we can also calculate (or reasonably estimate) what the accounting numbers would have been under the policy alternative (the hypothetical) If we could observe prices that would be generated under the policy alternative, we could also estimate the statistical relationship R(2) Does a comparison of R(1) and R(2) help the policy makers? If stronger R(2) implies preference for the policy alternative, it is trivially simple to push R(2) to the upper limit by simply using prices for accounting

9/10/2015Sunder, Research for Accounting Policy 16 Problems of Inference from Field Data: Hypotheticals and Efficiency Of course, we are rarely so lucky as to be able to observe P(2) An oft-used practice is to estimate the statistical relationship between the hypothetical I(2) and actually observed P(1), and then compare this R(2)* with R(1) and suggest that a stronger R(2)* implies the alternative to be preferred policy Info. Sys. 1  Price system 1 Info. Sys. 2  Price system 1

9/10/2015Sunder, Research for Accounting Policy 17 Logic of Inference Reporting System 1 Reporting System 2Price System 2 Price System 1

9/10/2015Sunder, Research for Accounting Policy 18 Logic of Inference Reporting System 1 Reporting System 2Price System 2 Price System 1

9/10/2015Sunder, Research for Accounting Policy 19 Logic of Inference and Policy This type of inference from field data does not help the policy makers We like them to give us a nod to acknowledge our work, and perhaps even support it But the logical foundations of such inference, and its implications for policy remain to be worked out

9/10/2015Sunder, Research for Accounting Policy 20 What about Lab Experiments? Lab methods allow us to address the causality problem with greater confidence But they also raise new challenges for use of findings in making of accounting policy Accounting is highly institutionalized (complex interactions, expectations), like engineering Experimental methods were developed for social sciences where a single simple example can support a general proposition about existence, or otherwise Simpler experiments suffice for basic disciplines such as economics, psychology and physics, but not for accounting policy or bridge design Time scale problem: choosing points vs. functions

9/10/2015Sunder, Research for Accounting Policy 21 Bench Testing of Policy Bench testing of an accounting policy alternative calls for far greater complexity in design of the lab experiment than in case of testing economic theory More complex decisions, larger choice space, need more time (order of magnitude) Compare choice of a point on a function with choice of a function

9/10/2015Sunder, Research for Accounting Policy 22 Instruments of social sciences Analysis of data gathered from the field Analysis of controlled experiments in the lab or the field Abstract mathematical analysis Historical analysis Introspection If accounting research is to contribute to policy, we shall have to use all the tools at our disposal as and when necessary Prior commitment to one or the other tool set is likely to be self-defeating

9/10/2015Sunder, Research for Accounting Policy 23 Way forward: management policy beyond social sciences? Management policy includes considerations that go beyond the scope of social sciences I do not believe that a good system of management is feasible without broadly accepted social norms of personal responsibility in the business and managerial community Our own (academic) community could not function without such norms We should not expect development and validation of decision models to be sufficient to create order in the world of management Social and mathematical sciences can only do so much. The rest is up to the management community

9/10/2015Sunder, Research for Accounting Policy 24 The Challenge of Policy for Research If research does not (at least ultimately) lead to better understanding, practice or policy, it will be ignored The methods we have do not always take us in that direction Stick to these methods (because that is what get published) Or, move beyond the methods if they cannot enlighten us Does research affect management policy for better? History suggests that improving management and governance is not an easy task

9/10/2015Sunder, Research for Accounting Policy 25

9/10/2015Sunder, Research for Accounting Policy 26

9/10/2015Sunder, Research for Accounting Policy 27 Whither Management: Windows™ or Open Systems Comfort vs. choice Uniformity and stagnation vs. dynamic change Predictability vs. some disorder High prices or the advantages of technological progress Management as an eco-system or a machine (garden or a building) Huxley or Hayek Nanny or personal responsibility Where does the research community stand? There are many life times of research agendas here if we are willing to consider them

9/10/2015Sunder, Research for Accounting Policy 28 References Gode, D. K. and Shyam Sunder “Allocative Efficiency of Markets with Zero Intelligence Traders: Market as a Partial Substitute for Individual Rationality.” The Journal of Political Economy 101, no. 1 (February 1993): Sunder, Shyam. “Determinants of Economic Interaction: Behavior or Structure.” Journal of Economic Interaction and Coordination 1, no. 1 (May 2006): Sunder, Shyam. “What Have We Learned from Experimental Finance?” In Developments on Experimental Economics: New Approaches to Solving Real-world Problems edited by Sobei H. Oda, Lecture Notes in Economics and Mathematical Systems 590. Berlin: Springer, William T. Baxter (Professor Emeritus, LSE), made many of these arguments over half-a-century ago (“Recommendations on Accounting Theory” in Baxter and Davidson, Studies in Accounting Theory, 1st edition). Sunder, Shyam “Was Accounting a Root Cause of the Global Financial Crisis?” Plenary Address to the Annual Meeting of the International Corporate Governance Network, Toronto, June 7.

Thank You!