Structure creates a chance IK 2013: Wicked Problems, Complexity and Wisdom, Günne, March 2013 Structure creates a chance D.Sc. (Habil.) D. Volchenkov (Bielefeld University) dima427@yahoo.com The City, or the game of structure and chance. Mathematical analysis of complex structures and wicked problems
“NOW & HERE “ vs. “THEN & THERE” Searching activity in humans can be characterized by formation of a neighbourhood: “NOW & HERE “ Environmental structure does not matter; A characteristic scale (space/time) exists; “THEN & THERE” Movements are determined by the environmental structure (strong interactions); No characteristic time scale (space/time); 20 m 30 sec “NOW & HERE “ “THEN & THERE”
“NOW & HERE “ vs. “THEN & THERE” Searching activity in humans can be characterized by formation of a neighbourhood: “NOW & HERE “ Environmental structure does not matter; A characteristic scale (space/time) exists; “THEN & THERE” Movements are determined by the environmental structure (strong interactions); No characteristic time scale (space/time); 20 m 30 sec “NOW & HERE “ “THEN & THERE”
Structure = adjacency adjacency …. adjacency “NOW & HERE “ A graph G =(V,E), where V is the set of identical elements called vertices (or nodes) and E VV is a collection of pairs of elements from V called edges.
Is it possible to geometrize a structure? Complexity: No direct ordering of nodes/ entities; can contain information about processes evolving at different spatio-temporal scales; many “semantic levels”. Lack of global geometric structure! (binary relations between places, instead of geometry) “NOW & HERE “ How can we introduce distances and angles of our everyday intuition developed in Euclidean space?
Structure = adjacency adjacency …. adjacency “NOW & HERE “ A walk is a sequence of graph vertices and graph edges such that the graph vertices and graph edges are adjacent.
Structure = adjacency adjacency …. adjacency “NOW & HERE “ Exploration has no characteristic time scale, and therefore… …a walk is not necessary of the nearest neighbor type: we can move along adjacent edges as long as we like.
Representation of Graphs by Matrices The major advantage of using matrices is that calculations of various graph characteristics can be performed by means of the well known operations of linear algebra. Aij =1 if i ~ j , and Aij =0 if i ~ j. A=
Structure = adjacency adjacency …. adjacency “NOW & HERE “ Aij =1 if i ~ j , and Aij =0 if i ~ j. Then the number of walks of length n between i and j is given by Anij
Any data interpretation/classification/judgment is always based on introduction of some equivalence relation on the set of walks over the database: “NOW & HERE “ Rx: walks of the given length n starting at the same node x are equivalent Ry: walks of the given length n ending at the same node y are equivalent Rx Ry : walks of the given length n between the nodes x and y are equivalent
Equivalence partition of walks => random walk Given an equivalence relation on the set of walks and a function such that we can always normalize it to be a probability function: all “equivalent” walks are equiprobable. Partition of walks into equivalence classes The utility function for each equivalence class A random walk transition operator between eq. classes Set of all n-walks
We proceed in three steps: Step 0: Given an equivalence relation between paths, any transition can be characterized by a probability to belong to an equivalence class. Different equivalence relations Different equivalence classes Different probabilities Step 1: “Probabilistic graph theory” Nodes of a graph, subgraphs (sets of nodes) of the graph, the whole graph are described by probability distributions & characteristic times w.r.t. different Markov chains; Step 2: “Geometrization of Data Manifolds” Establish geometric relations between those probability distributions whenever possible; 1. Coarse-graining/reduction of networks & databases → data analysis; sensitivity to assorted data variations; 2. Transport optimization(Monge-Kontorovich type problems) → distances between distributions;
An example of equivalence relation: Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent Equiprobable walks: the nearest neighbor random walks Stochastic normalization
An example of equivalence relation: Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent Equiprobable walks: the nearest neighbor random walks Stochastic normalization Probability of a n-walk
An example of equivalence relation: Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent Equiprobable walks: Stochastic normalization Probability of a n-walk … “Structure learning”
An example of equivalence relation: Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent Equiprobable walks: Stochastic normalization Probability of a n-walk … ≠ “Structure learning” Stochastic normalization
What is a neighbourhood? Who are my neighbours? … 1.Neighbours are next to me… 2.Neighbours are 2 steps apart from me… n.Neighbours are n steps apart from me …
What is a neighbourhood? Who are my neighbours? … 1.Neighbours are next to me… 2.Neighbours are 2 steps apart from me… n.Neighbours are n steps apart from me … My neighbours are those, which I can visit with equal probability (w.r.t. a chosen equivalence of paths)…
An example of equivalence relation: Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent … Equiprobable walks: Stochastic matrices:
A variety of random walks at different scales Step 0 A variety of random walks at different scales An example of equivalence relation: Rx: walks of the given length n starting at the same node x are equivalent Left eigenvectors (m=1) Centrality measures: … Equiprobable walks: Stochastic matrices: The “stationary distribution” of the nearest neighbor RW
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices
Random walks of different scales Time is introduced as powers of transition matrices Still far from stationary distribution! Stationary distribution is already reached! Defect insensitive. Low centrality (defect) repelling.
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW Maximal entropy RW J. K. Ochab, Z. Burda
Random walks for different equivalence relations Nearest neighbor RW “Maximal entropy” RW J. K. Ochab, Z. Burda
Step 1: “Probabilistic graph theory” As soon as we define an equivalence relation … Graph Subgraph (a subset of nodes) Node Time scale Tr T The probability that the RW stays at the initial node in 1 step. “Wave functions” (Slater determinants) of transients (traversing nodes and subgraphs within the characteristic scales) return the probability amplitudes whose modulus squared represent the probability density over the subgraphs. Probabilistic graph invariants = the t-steps recurrence probabilities quantifying the chance to return in t steps. … | det T | The probability that the RW revisits the initial node in N steps. Return times to the subgraphs within transients = 1/Pr{ … } Centrality measures (stationary distributions) Return times to a node Random target time Mixing times over subgraphs (times until the Markov chain is "close" to the steady state distribution)
As soon as we get probability distributions… Step 2: “Geometrization of Data Manifolds” As soon as we get probability distributions… Given T, L ≡ 1- T , the linear operators acting on distributions. The Green function is the natural way to find the relation between two distributions within the diffusion process Drazin’s generalized inverse:
As soon as we get probability distributions… Step 2: “Geometrization of Data Manifolds” As soon as we get probability distributions… Given T, L ≡ 1- T , the linear operators acting on distributions. The Green function is the natural way to find the relation between two distributions within the diffusion process Drazin’s generalized inverse: Given two distributions x,y over the set of nodes, we can define a scalar product, The (squared) norm of a vector and the angle The Euclidean distance:
Transport problems of the Monge-Kontorovich type Step 2: “Geometrization of Data Manifolds” As soon as we get probability distributions… Given T, L ≡ 1- T , the linear operators acting on distributions. The Green function is the natural way to find the relation between two distributions within the diffusion process Drazin’s generalized inverse: Given two distributions x,y over the set of nodes, we can define a scalar product, Transport problems of the Monge-Kontorovich type The (squared) norm of a vector and an angle The Euclidean distance: “First-passage transportation” from x to y x y W(x→y) W(y→x) ≠
Transport problems of the Monge-Kontorovich type Step 2: “Geometrization of Data Manifolds” As soon as we get probability distributions… Given T, L ≡ 1- T , the linear operators acting on distributions. The Green function is the natural way to find the relation between two distributions within the diffusion process Drazin’s generalized inverse: Given two distributions x,y over the set of nodes, we can define a scalar product, Transport problems of the Monge-Kontorovich type The (squared) norm of a vector and an angle The Euclidean distance: (Mean) first-passage time Commute time Electric potential Effective resistance distance Tax assessment land price in cities Musical diatonic scale degree … Musical tonality scale
Example 1: Nearest-neighbor random walks on undirected graphs y1
Example 1: Nearest-neighbor random walks on undirected graphs y1 The spectral representation of the (mean) first passage time, the expected number of steps required to reach the node i for the first time starting from a node randomly chosen among all nodes of the graph accordingly to the stationary distribution π. The commute time, the expected number of steps required for a random walker starting at i ∈ V to visit j ∈ V and then to return back to i,
Around The City of Big Apple Federal Hall Public places City CORE Times Square SoHo City CORE 10 steps 100 East Village steps 500 (Mean) first-passage times in the city graph of Manhattan steps 1,000 steps Bowery East Harlem City Decay steps 5,000 steps 10,000 SLUM
Example 2: Electric Resistance Networks, Resistance distance An electrical network is considered as an interconnection of resistors: The currents are described by the Kirchhoff circuit law:
Example 2: Electric Resistance Networks, Resistance distance An electrical network is considered as an interconnection of resistors: The currents are described by the Kirchhoff circuit law: Given an electric current from a to b of amount 1 A, the effective resistance of a network is the potential difference between a and b, The effective resistance allows for the spectral representation:
Impedance networks: The two-point impedance and LC resonances
Geodesics paths of language evolution Levenshtein’s distance (Edit distance): is a measure of the similarity between two strings: the number of deletions, insertions, or substitutions required to transform one string into another. MILCH K = MILK The normalized edit distance between the orthographic realizations of two words can be interpreted as the probability of mismatch between two characters picked from the words at random.
The four well-separated monophyletic spines represent the four biggest traditional IE language groups: Romance & Celtic, Germanic, Balto-Slavic, and Indo-Iranian; The Greek, Romance, Celtic, and Germanic languages form a class characterized by approximately the same azimuth angle (belong to one plane); The Indo-Iranian, Balto-Slavic, Armenian, and Albanian languages form another class, with respect to the zenith angle.
The systematic sound correspondences between the Swadesh’s words across the different languages perfectly coincides with the well-known centum-satem isogloss of the IE family (reflecting the IE numeral ‘100’), related to the evolution in the phonetically unstable palatovelar order.
The normal probability plots fitting the distances r of language points from the ‘center of mass’ to univariate normality. The data points were ranked and then plotted against their expected values under normality, so that departures from linearity signify departures from normality.
The univariate normal distribution is closely related to the time evolution of a mass-density function under homogeneous diffusion in one dimension in which the mean value μ is interpreted as the coordinate of a point where all mass was initially concentrated, and variance σ2 ∝ t grows linearly with time. The values of variance σ2 give a statistically consistent estimate of age for each language group. the last Celtic migration (to the Balkans and Asia Minor) (300 BC), the division of the Roman Empire (500 AD), the migration of German tribes to the Danube River (100 AD), the establishment of the Avars Khaganate (590 AD) overspreading Slavic people who did the bulk of the fighting across Europe. Anchor events:
From the time–variance ratio we can retrieve the probable dates for: The break-up of the Proto-Indo-Iranian continuum. The migration from the early Andronovo archaeological horizon (Bryant, 2001). by 2,400 BC The end of common Balto-Slavic history before 1,400 BC The archaeological dating of Trziniec-Komarov culture The separation of Indo-Arians from Indo-Iranians. Probably, as a result of Aryan migration across India to Ceylon, as early as in 483BC (Mcleod, 2002) before 400 BC The division of Persian polity into a number of Iranian tribes, after the end of Greco-Persian wars (Green, 1996). before 400 BC
Proto-Indo-Europeans? The Kurgan scenario postulating the IE origin among the people of “Kurgan culture”(early 4th millennium BC) in the Pontic steppe (Gimbutas,1982) . Einkorn wheat The Anatolian hypothesis suggests the origin in the Neolithic Anatolia and associates the expansion with the Neolithic agricultural revolution in the 8th and 6th millennia BC (Renfrew,1987). The graphical test to check three-variate normality of the distribution of the distances of the five proto-languages from a statistically determined central point is presented by extending the notion of the normal probability plot. The χ-square distribution is used to test for goodness of fit of the observed distribution: the departures from three-variant normality are indicated by departures from linearity. The use of the previously determined time–variance ratio then dates the initial break-up of the Proto-Indo-Europeans back to 7,400 BC pointing at the early Neolithic date.
In search of Polynesian origins The components probe for a sample of 50 AU languages immediately uncovers the both Formosan (F) and Malayo-Polynesian (MP) branches of the entire language family. Headhunters
An interaction sphere had existed encompassing the whole region By 550 AD …pretty well before 600 –1200 AD while descendants from Melanesia settled in the distant apices of the Polynesian triangle as evidenced by archaeological records (Kirch, 2000; Anderson and Sinoto,2002; Hurlesetal.,2003).
Mystery of the Tower of Babel Nonliterate languages evolve EXPONENTIALLY FAST without extensive contacts with the remaining population. Isolation does not preserve a nonliterate language! Languages spoken in the islands of East Polynesia and of the Atayal language groups seem to evolve without extensive contacts with Melanesian populations, perhaps because of a rapid movement of the ancestors of the Polynesians from South-East Asia as suggested by the ‘express train’ model (Diamond, 1988) consistent with the multiple evidences on comparatively reduced genetic variations among human groups in Remote. Headhunters
Traps and landmarks Exploitation Exploration Recurrence time First-passage time: Landmarks, “guiding structures”: firstly reached , seldom revisited Exploitation Traps, “confusing environments”: can take long to reach, but often revisited Exploration
Musical Dice Game (*) The relations between notes in (*) are rather described in terms of probabilities and expected numbers of random steps than by physical time. Thus the actual length N of a composition is formally put N → ∞, or as long as you keep rolling the dice.
F. Liszt Consolation-No1 Bach_Prelude_BWV999 R. Wagner, Das Rheingold (Entrance of the Gods) V.A. Mozart, Eine-Kleine-Nachtmusik
A “guiding structure”: Tonality scales in Western music Increase of harmonic interval/ first –passage time Recurrence time First-passage time The recurrence time vs. the first passage time over 804 compositions of 29 Western composers.
Network geometry at different scales First-passage time Scale of RW … The node belongs to a network “core”, consolidating with other central nodes Recurrence times The node belongs to a “cluster”, loosely connected with the rest of the network.
Ricci flows and photo resolution
Possible analogy with Ricci flows “Densification” of the network of “positive curvature” “Contraction” of a “probabilistic manifold” First-passage time Scale of RW … Recurrence times A “collapse” of the network of “negative curvature”
References D.V., Ph. Blanchard, “Introduction to Random Walks on Graphs and Databases”, © Springer Series in Synergetics , Vol. 10, Berlin / Heidelberg , ISBN 978-3-642-19591-4 (2011). D.V., Ph. Blanchard, Mathematical Analysis of Urban Spatial Networks, © Springer Series Understanding Complex Systems, Berlin / Heidelberg. ISBN 978-3-540-87828-5, 181 pages (2009).