Multinomial Processing Tree Models

Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer, Assignment 1 solution(s). Math for next week. Assignment 2 problem statement.

Uses of MPT Models Data-analysis tool capable of disentangling and measuring separate contributions of different cognitive processes. –Provides a means for separately measuring latent processes that are confounded in observable data. Framework for developing and testing quantitative theories.

Purview of MPTs Categorical data: each observation falls into one and only one of a finite set of categories. –Example 1: Correct or incorrect. –Example 2: Reaction time < 100, 100 ≥ RT < 200, 200 ≤ RT.

Multinomial Distribution Consider a die with sides: 1, 1, 1, 2, 2, 3. If we roll the die 10 times, what is the probability we get five 1’s, three 2’s, and two 3’s?

Multinomial Distribution In general, Inverse goal is to determine the p i ’s given the n i ’s.

Statistical vs Explanatory Multinomial models (log-linear, logit) are statistical. –The parameters are used to explore main effects and interactions. MPT models are explanatory. –The parameters of the MPT model represent the underlying psychological processes.

MPT Models Root Process State 2 Process State 1 Process State 3 Process State 4 Response 1 Response 2 Response 1

MPT Models The probability of each process state change is represented by a parameter. The parameters range from 0 to 1. The parameters are independent. There are other restrictions…

MPT Models The response probabilities are given by polynomials, e.g., P(E 1 )=c·r. The parameter estimators (and other important properties) are easy to find.

Estimators A parameter is a descriptive measure in a model –E.g., a is a parameter the describes how quickly the line increases in y=ax+b. An estimator is a function on the data that gives a parameter estimate, usually denoted with a hat, â. –E.g., given some data, get an estimate of how quickly the linear trend increases. Estimators are usually picked to minimize the discrepancy between the predictions and the data.

Parameters and Flexibility As a rule of thumb, a model should have fewer parameters than degrees of freedom in the data. This model is saturated. CondDataModel p1 p2 p3 C1 C2 C3

Parameters and Flexibility In general, the more parameters, the more flexible the model. y=ax+by=b

But… 1 parameter2 parameters

Further… It is not always easy to count parameters.

Parameters and Flexibility A more restricted a model is –usually simpler. –usually easier to interpret. –usually more falsifiable. Possible Events Observed Events Model 1 Model 2

Nested Models Models 2 and 3 are submodels (nested in) Model 1. Models 2 are 3 are NOT nested. The main benefit of nested models is that it makes it relatively easy to compare the goodness-of-fit of the two models.

Parameters and Flexibility What is important is the flexibility of the model relative to –the data. –competing models.

Identifiability The parameters in a model are identifiable if there is a unique set of parameters that give rise to the model predictions. Identifiability is especially desirable if the parameter values are to be interpreted. vs

Batchelder and Modeling “The assumption is … clearly an approximation, but one that greatly simplifies the analysis of the model and still allows the model to reflect the main processing stages of the task” (p. 59).

Batchelder and Modeling “…there is usually a large number of parameters used to account for a small number of categories, leaving few, if any, degrees of freedom for testing the model’s fit…However, it is the measurement of the cognitive processes in the form of parameter estimates, and not the data-fitting capacity, that characterizes the usefulness of MPT models” (p ).

Batchelder and Modeling “… there will often be psychologically uniterpretable MPT models that nevertheless fit a given set of data well. Thus, the process of developing a valid model requires that one fit a number of data sets in the same paradigm and that the resulting parameter estimates be interpretable in terms of the underlying processing assumptions” (p. 76).

Batchelder and Modeling “Each MPT model is at best an approximation to a complete process description of categorical data, and the task of the modeler is to select the most important processes and capture them in a valid way” (p. 75).

Batchelder and Modeling “… the process of developing a valid model requires that one fit a number of data sets in the same paradigm and that the resulting parameter estimates be interpretable in terms of the underlying processing assumptions” (p. 76).

Batchelder and Modeling “…an even more crucial test of a model’s validity is to show that the model performs well under basic experimental manipulations. If the model’s parameters behave in a psychologically interpretable fashion, then the model gains credence as a valid measurement tool” (p. 82).

Batchelder and Modeling “…an acceptable MPT model must not only be able to fit data but its parameters must be globally identifiable, must be psychologically interpretable, and must pass appropriate validation experiments” (p. 78).

Batchelder and Modeling “…there are many models for categorical data that are not in the MPT class. If one of these models accounts for data in a a particular paradigm, then, technically one can infer that the MPT class is falsified in that paradigm. Of course, it may be possible to design an MPT model that closely mimics or approximates the successful fits of the non-MPT model; thus, it may be difficult to argue that the MPT framework is falsifiable in practice” (p. 78).