Page 0 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Saurabh Prasad Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Estimating Kolmogorov Entropy from Acoustic Attractors from a Recognition Perspective

Page 1 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Estimating the correlation integral from a time series Correlation Integral of an attractor’s trajectory : Correlation sum of a system’s attractor is a measure quantifying the average number of neighbors in a neighborhood of radius along the trajectory. where represents the i’th point on the trajectory, is a valid norm and is the Heaviside’s unit step function (serving as a count function here) At a given embedding dimension (m > [2*D+1]), we have: ~ Fractal Dimension of the attractor

Page 2 of 14 Dynamical Invariants of an Attractor and potential applications for speech data order-q Renyi entropy and K2-Entropy Divide the state space into disjoint boxes If the evolution of the state space that generated the observable is sampled at Numerically, the Kolmogorov entropy can be estimated as the second order Renyi entropy (K 2 ) Represents the joint probability that lies in box i 1 lies in box i 2 and so on.

Page 3 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Second Order Kolmogorov Entropy Estimation of speech data Speech data, sampled at 22.5 KHz – Sustained Phones (/aa/, /ae/, /eh/, /sh/, /z/, /f/, /m/, /n/) Output – Second order Kolmogorov Entropy We wish to analyze: – The presence or absence of chaos in any time series. – Their discrimination characteristics across attractors from different sound units (for classification)

Page 4 of 14 Dynamical Invariants of an Attractor and potential applications for speech data The analysis setup Currently, this analysis includes estimates of K2 for different embedding dimensions Variation in entropy estimates with the neighborhood radius, epsilon was studied Variation in entropy estimates with SNR of the signal was studied Currently, the analysis was performed on 3 vowels, 2 nasals and 2 fricatives Results show that vowels and nasals have a much smaller entropy, as compared to fricatives K 2 consistently decreases with embedding dimension for vowels and nasals, while for fricatives, it consistently increases

Page 5 of 14 Dynamical Invariants of an Attractor and potential applications for speech data The analysis setup (in progress / coming soon)… Data size (length of the time series): –This is crucial for our purpose, since we wish to extract information from short time series (sample data from utterances). Speaker variation: – We wish to study variations in the Kolmogorov entropy of phone or word level attractors across different speakers. across different phones/words across different broad phone classes

Page 6 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Correlation Entropy vs. Embedding Dimension Various Epsilons

Page 7 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Correlation Entropy vs. Embedding Dimension Various Epsilons

Page 8 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Correlation Entropy vs. Embedding Dimension Various Epsilons

Page 9 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Correlation Entropy vs. Embedding Dimension Various SNRs

Page 10 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Correlation Entropy vs. Embedding Dimension Various Data Lengths

Page 11 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Measuring Discrimination Information in K 2 based features Kullback-Leibler (KL) divergence: Provides an information theoretic distance measure between two statistical models Average Discriminating Information between class i and class j: Likelihood: i vs. jLikelihood: j vs. i For Normal Densities:

Page 12 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Measuring Discrimination Information in K 2 based features Statistics of entropy estimates over several frames, for various phones

Page 13 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Measuring Discrimination Information in K 2 based features KL-Divergence Measure between K 2 features from various phonemes for two speakers

Page 14 of 14 Dynamical Invariants of an Attractor and potential applications for speech data Plans Finish studying the use of K 2 entropy as a feature characterizing phone-level attractors – We will be performing a similar analysis on Lyapunov Exponents and Correlation Dimension estimates Measure speaker dependence in this invariant Use this setup on a meaningful recognition task Noise robustness, parameter tweaking, integrating these features to MFCCs Statistical Modeling…