Rutgers Intelligent Transportation Systems (RITS) Laboratory Department of Civil & Environmental Engineering Modeling Traveler Behavior via Day-to-Day Learning Dynamics: Impacts of Habitual Behavior Paper No: 10-2607 Ozlem Yanmaz-Tuzel, M.Sc. And Kaan Ozbay, Ph.D. Rutgers, the State University of New Jersey LITERATURE REVIEW ABSTRACT This paper focuses on the development of learning-based behavioral mechanisms for updating route and departure time choices when a major and brand new facility is added to the existing transportation system by creating new route choices that did not exist previously. To model this complex user behavior based on empirical observations; this study applies the Bayesian-SLA framework recently developed by the authors. In this approach, Bayesian-SLA framework systematically accounts for commuters’ belief, perceptions and habitual tendencies about the transportation system, and represents these dynamics as random variables. The developed learning model is calibrated and validated using real traffic and travel time data from New Jersey Turnpike (NJTPK) toll road to investigate the impacts of Interchange 15X installation on the day-to-day departure time and route choice behavior of NJTPK travelers. The estimation confirm strong effect of habitual behavior on traveler choice, consistent with the preliminary traffic volume analysis findings. The proposed Bayesian-SLA model can successfully capture the significant learning dynamics, demonstrating the possibility of developing learning models as a viable approach to represent travel behavior. METHODOLOGY In the proposed framework, an agent-based learning system via Bayesian-SLA is designed which can learn the best possible actions and model travelers’ day-to-day travel choices in a non-stationary stochastic environment. Each traveler maintains a choice probability profile for the available alternatives, and updates his/her probability profile based on previous travel choices, exhibiting a tendency to search for satisfying choices rather than the best behavior; inertia and bounded rationality. The estimated learning parameters reflect travelers’ perception about the system and their response to the experienced traffic conditions. Posterior Distribution of the Learning Parameters In this paper, learning parameters (a,b) are estimated via Bayesian Inference approach. Given the likelihood of the observations and the prior information regarding parameters (a, b), joint posterior distribution of the learning parameters are estimated. Unlike maximum likelihood analysis, the aim of a Bayesian analysis is not to provide so-called point estimates of the model parameters; the result of the analysis is the posterior probability distribution itself. With this approach, it is possible to introduce user heterogeneity into the estimation process, and to investigate distribution of the learning parameters among different users. The proposed likelihood function of the observations is estimated via following equation: INTRODUCTION Modeling and understanding the relationship between individuals’ travel perception and learning process and the day-to-day traffic flows remain an important challenge for transportation researchers. This paper focuses on the development of learning-based behavioral mechanisms for updating learned route and departure time choices in the presence of new route inclusions to the transportation system considering the impacts of habitual behavior on travelers’ choice mechanisms. This study applies the Bayesian-SLA framework. In this approach, Bayesian-SLA framework systematically accounts for commuters’ belief, perceptions and habitual tendencies about the transportation system, and represents these dynamics as random variables. The developed learning model is calibrated and validated using real traffic and travel time data from New Jersey Turnpike (NJTPK) toll road to investigate the impacts of Interchange 15X construction on the day-to-day departure time and route choice behavior of NJTPK travelers. EMPIRICAL SETTING The developed Bayesian-SLA framework is implemented to investigate the impacts of the addition of 15X Interchange on December 2005 on the day-to-day departure time and destination choice behavior of NJTPK travelers. After nearly three years of construction, NJ Turnpike Authority (NJTA) opened the $250 million Interchange 15X on the Eastern Spur (just south of Interchange 16E) on December 1, 2005. The new interchange serves the new Secaucus Junction rail transfer station. The NJTA contributed an additional $84 million to develop the $450 million adjacent Allied Junction, which will have 3.5 million square feet of combined commercial and residential development, as well as up to 2,600 new parking spaces when the development is completed. Upon full development, Interchange 15X is expected to handle 40,000 vehicles per day. The proposed Bayesian-SLA model is applied to the road sections from Interchanges 11 (Garden State Parkway) and 14 (Jersey City – Holland Tunnel) to Interchanges 15X (Secaucus Junction) and 16E (Lincoln Tunnel). Before opening of Interchange 15X, the only alternative traveling to Lincoln Tunnel area was to travel through Interchange 16E. However, since December 2005, Interchange 15X became a viable alternative for these travelers.

Rutgers Intelligent Transportation Systems (RITS) Laboratory Department of Civil & Environmental Engineering Modeling Traveler Behavior via Day-to-Day Learning Dynamics: Impacts of Habitual Behavior Paper No: 10-2607 Ozlem Yanmaz-Tuzel, M.Sc. And Kaan Ozbay, Ph.D. Rutgers, the State University of New Jersey The prior distribution of the learning parameters (a, b) can be represented by p(a,b). In this paper, Dirichlet (multivariate generalization of the beta distribution), multivariate Normal and Uniform distributions are tested as joint prior distributions of p(a,b): The posterior distribution of the learning parameters given the observations, p(a,b|D), can be calculated using Bayes’ theorem: Posterior distribution of the learning parameters is a very complex multidimensional function which requires integrating. Thus, to obtain the mean and variance of the parameters (a, b) Metropolis-Hastings (M-H) algorithm is used. The M-H algorithm is a rejection sampling algorithm used to generate a sequence of samples from a probability distribution that is difficult to sample from directly. To ensure MCMC convergence, Heidelberger and Welch first test diagnostic was employed. This diagnostic compares the observed sequence of MCMC samples to a hypothetical stationary distribution, using the Cramer-von-Mises statistic. The test iteratively discards the first 10% of the chain until the null hypothesis is not rejected (i.e. the chain is stationary), or until 50% of the original chain remains Estimation Results The estimation process resulted in Beta distribution for the posterior distribution of each parameter. Since beta distribution always lies within [0, 1], the constraints on the learning parameters will be satisfied at all times. Mean values for the parameters (a, b) are (0.029, 0.0029), and standard deviations are (0.011, 0.00093), respectively. Mean values of the reward and penalty parameters estimated for NJTPK users are different than the ones in other disciplines. The low learning parameters indicate that even though the travel time from Interchanges 11 and 14 to Interchange 15X is lower compared with the travel time to Interchange 16E, high percentage of travelers put higher utility to Interchange 16E continue to use this choice. These results confirm the strong effect of habitual behavior on traveler choice, consistent with the preliminary analysis findings. CONCLUSIONS This research focuses on modeling learning based behavioral mechanisms for updating route and departure time choices in light of new facility additions to the existing transportation system. The proposed model extends the existing SLA theory by using it in a Bayesian framework and bounded rationality (BR), while considering the impacts of habitual behavior: Day-to-day learning behavior is modeled based on Bayesian-SLA theory, where each user updates his/her choice based on the rewards/punishments received due to selected actions in previous days. A linear reward-penalty reinforcement scheme is considered to represent day-to-day behavior of NJTPK users as a response to construction of a new Interchange. In order to account for travelers’ resistance to switch routes, concept of habitual behavior (inertia) is included in the proposed model, such that the travelers switch to the new route only if it has significantly less cost. Finally, learning parameters were modeled as probability distributions rather than deterministic values, and Bayesian posterior distributions are estimated. The empirical results obtained from the real transportation network confirm the strong effect of habitual behavior on traveler choice. The proposed Bayesian-SLA model can successfully capture the significant learning dynamics, demonstrating the possibility of developing a psychological framework (i.e., learning models) as a viable approach to represent dynamic travel behavior. A possible extension of the proposed methodology is to investigate how individuals’ tolerance level, and learning parameters change over time as the users gain more experience with the transportation system. Calibration and Validation of the Learning Parameters Calibration and validation are important processes in the development and application of day-to-day DTA models. These processes are developed to ensure that the models accurately replicate the observed traffic condition and driver behavior. Model calibration is a process whereby the values of model parameters are adjusted so as to match the simulated model outputs with observations from the study site. It is usually formulated as an optimization problem to determine the best set of model parameter values in order to minimize the discrepancies between the observed and simulated values. The calibration process is then to modify the values of the model parameters θ, so to find the best set of values which minimizes F. The proposed objective function F minimizes the difference between observed and simulated volumes: After determining the optimal set of parameters from the calibration process, a validation process is performed in order to determine whether the simulation model replicates the real system. Mean standard errors (MSE) are calculated for each day the validation process: