1. University of Bologna & UnipolSai Assicurazioni
The markovchain R package
Giorgio A. Spedicato
University of Bologna & UnipolSai Assicurazioni
Abstract
Representing DTMCs
Simulation, estimation and inference
Non-homogeneous DTMCs
The markovchain s4 method wraps plot functions from igraph and DiagrammeR packages
Discrete Time Markov Chains (DTMCs) represent a notable class of stochastic processes. Whilst their basic theory is rather simple, they are extremely effective to model categorical data sequences. Notable applications can be found in linguistic, information theory, life sciences, economics and sociology .
The markovchain package aims to fill a gap within CRAN ecosystem providing an unified infrastructure to easily manage DTMC and to perform statistical and probabilistic analyses on Markov chains stochastic processes.
The markovchain package contains S4 classes and methods that allow to create and manage homogeneous and non - homogeneous DTMC straightforwardly.
This means that DTMCs can be created, represented and exported using an approach natural and intuitive for any R programmer.
At the same time, it provides functions to perform structural analysis of transition matrices (that means classification of matrices and states, analysis of the stationary distribution, periodicity, etc...). In addition, it provides methods to estimate transition matrix from data and to perform some inference (confidence intervals calculations), statistical tests on the order and stationarity, etc... Finally embryonic procedures for Bayesian modeling of DTMCs, continuous time Markov Chains and higher order ones has been started to be provided.
Given a markovchain object, simulating a stochastic sequence given a transition matrix is straightforward
Non-homogeneous DTMCs help when transition probabilities across states structurally change during time. They are represented by an implicity ordered list of markovchain objects.
plot(mcWeather,main="Weather transition matrix")
plot(mcWeather, package = "DiagrammeR", label = "Weather transition matrix")
dailyWeathers<-rmarkovchain(n=365,object=mcWeather, t0=“sunny”)
dailyWeathers[1:7]
"sunny" "sunny" "sunny" "sunny" "nice" "rainy" "rainy"
mcC=as(matrix(data=c(0.1,.9,.5,.5),byrow=2,nrow=2),"markovchain")
mcA=as(matrix(data=c(0.4,.6,.1,.9),byrow=2,nrow=2),"markovchain")
mcB=as(matrix(data=c(0.2,.8,.2,.8),byrow=2,nrow=2),"markovchain")
>
myMcList=new("markovchainList",markovchains=list(mcA,mcB,mcC))
myMcList
list of Markov chain(s)
Markovchain 1
Unnamed Markov chain
A 2 - dimensional discrete Markov Chain with following states:
s1, s2
The transition matrix (by rows) is defined as follows:
s1 s2 s s
Markovchain 2 s s
Markovchain 3 s s
The markovchainFit function allows to estimate the underlying transition matrix for a given character sequence. Asymptotic standard error and MLE confidence intervals are given by default.
Objective
mleFit<-markovchainFit(data=dailyWeathers)
mleFit
$estimate
nice rainy sunny
nice
rainy
sunny
$standardError
nice rainy sunny
nice
rainy
sunny
$confidenceInterval
$confidenceInterval$confidenceLevel
[1] 0.95
$confidenceInterval$lowerEndpointMatrix
nice
rainy
sunny
$confidenceInterval$upperEndpointMatrix
nice
rainy
sunny
$logLikelihood
[1]
Develop an easy-to-use software package that permits to perform most common statistical analyses on Markov Chains.
Structural analysis of DTMCs
The package allows to perform structural analysis of DTMCs, thanks to an algorithm by (Feres, 2007) ported from Matlab.
Creating and operating with DTMCs
require(matlab)
mathematicaMatr <- zeros(5)
mathematicaMatr[1,] <- c(0, 1/3, 0, 2/3, 0)
mathematicaMatr[2,c(1,5)] <- 0.5
mathematicaMatr[c(3,4),c(3,4)] <- 0.5
mathematicaMatr[5,5] <- 1
mathematicaMc <- new("markovchain", transitionMatrix = mathematicaMatr,
name = "Mathematica MC")
names(mathematicaMc)<-LETTERS[1:5]
#the summary method provides an overview of structural characteristics of DTMCs
summary(mathematicaMc)
Mathematica MC Markov chain that is composed by:
Closed classes:
C D E
Recurrent classes:
{C,D},{E}
Transient classes:
{A,B}
The Markov chain is not irreducible
The absorbing states are: E
Creating DTMCs S4 classes is extremely easy
Function for simulate from and fit markovchainList are available as well
##create a DTMC object
library(markovchain)
#defining a Transition Matrix
weatherStates <- c("rainy", "nice", "sunny")
weatherMatrix <- matrix(data = c(0.50,0.25, 0.25,0.5, 0.0,
0.5, 0.25, 0.25, 0.5), byrow = TRUE, nrow = 3,dimnames = list(weatherStates, weatherStates))
#create the DTMC (long and short way)
#long way
mcWeather <- new("markovchain", states = weatherStates, byrow = TRUE,
transitionMatrix = weatherMatrix,name = "Weather")
mcWeather2<-as(mcWeather, "markovchain")
name(mcWeather2)<-"Weather Mc"
mcWeather2
Weather Mc
A 3 - dimensional discrete Markov Chain defined by the following states:
rainy, nice, sunny
The transition matrix (by rows) is defined as follows:
rainy nice sunny
rainy
nice
sunny
mcListSims<-rmarkovchain(n=1000,object=myMcList,what="matrix")
head(mcListSims,3)
[,1] [,2] [,3]
[1,] "s2" "s2" "s1"
[2,] "s2" "s1" "s2"
[3,] "s2" "s2" "s1"
myMcListFit<-markovchainListFit(data=mcListSims)
myMcListFit$estimate
list of Markov chain(s)
Markovchain 1
Unnamed Markov chain
A 2 - dimensional discrete Markov Chain with following states:
s1, s2
The transition matrix (by rows) is defined as follows:
s s2 s s
Markovchain 2 s s
Various functions have been defined at this purpose.
#checking accessibility
is.accessible(mathematicaMc,from = "A","E")
TRUE
is.accessible(mathematicaMc,from = "C","E")
FALSE
#canonic form
myCanonicForm<-canonicForm(mathematicaMc)
Mathematica MC
A 5 - dimensional discrete Markov Chain defined by the following states:
C, D, E, A, B
The transition matrix (by rows) is defined as follows:
C D E A B C D E A B
#periodicity
period(mathematicaMc)
Warning message: In period(mathematicaMc) : The matrix is not irreducible
period(as(matrix(c(0,1,1,0),nrow = 2),"markovchain"))
Algebraic and logical operations can be easily performed
Algebraic and logical operations can be naturally performed:
#equality
mcWeather == mcWeather2
TRUE
#exponentiation
mcWeather^2
Weather Mc^2
A 3 - dimensional discrete Markov Chain defined by the following states:
rainy, nice, sunny
The transition matrix (by rows) is defined as follows:
rainy nice sunny
rainy
nice
sunny
Acknowledgements
In addition to maximum likelihood, estimates can be obtained by bootstrap and bayesian approaches.
Special thanks are given to: the Google Summer of Code Project ( sessions) and the many users that continuously return feedbacks, suggestions and bugs checks.
Steady state distribution(s) can be easily found as well
#finding the steady state
steadyStates(mcWeather)
rainy nice sunny
Contact information
For further information please contact Giorgio A. Spedicato
This poster is available upon request.