1. Peter Grzybek http://www-gewi.uni-graz.at/quanta Austrian Research Fund Project #15485  Von der Ökonomie der Sprache zur Selbst-Regulation kultureller Systeme  Korpuslinguistik vs. Textanalyse  Exakte Literaturwissenschaft: Zur Prosa Karel Čapeks  Was tun die Wörter im Vers miteinander? Zur Poesie A.S. Puškins

2. Peter Grzybek http://www-gewi.uni-graz.at/quanta Austrian Research Fund Project #15485 Korpus-Linguistik vs. Text-Analyse

3. Analysis of Letter Frequencies Methodological Problems in Former Studies 1.Insufficient Data Distinction (graphemic and phonematic/phonetic data) 2.Insufficient Control of Data Homogeneity (text / text segments / text mixtures (corpora) 3.Frequency Models: Continuous vs. Discrete (a) theoretical entropy, repeat rate (b)  p i = 1 4.Goodness of Fit Graphics vs. tests, R² vs.  ²

4. Analysis of Letter Frequencies Methodological Decisions 1.Data Distinction Graphemic data 2.Control of Data Homogeneity Text vs. text segments vs. text cumulations vs. text mixtures (corpus) 3.Discrete Frequency Models Test of relevant models 4. Goodness of Fit  ² test  C =  ² / N (C < 0.02 = * ; C < 0.01 = **)

5. Analysis of Letter Frequencies Slavic Alphabets inventory size minimal25Slovene maximal46Slovak medium32/33Russian (е / ё)

6. Analysis of Letter Frequencies Russian

7. Zipf (Zeta) distribution Basic assumption: r x f r = c  f r = c / r

8. Zipf-Mandelbrot distribution Basic assumption:  f r = c / (r + b) a

9. Zipf and Zipf-Mandelbrot Distributions: Goodness of Fit (38 Russian samples)

10. Geometric Distribution and Good Distribution

11. n = inventory size, x = class 2 parameters: K and M Negative Hypergeometric Distribution Analysis of Russian Letter Frequencies: Corpus: 37 Texts (ca. 8.5 mio. letters)

12. Analysis of Russian Letter Frequencies Comparison of Texts, Text Segments, Text Cumulations, Text Mixtures, and Complete Corpus Constancy of goodness of fit (C)Constancy of Parameters (K, M) Negative Hypergeometric Distribution

13. Analysis of Slovene Letter Frequencies Corpus: ca. 130.000 letters Goodness of fit (C= 0.0094) Negative Hypergeometric Distribution

14. Analysis of Slovene Letter Frequencies Comparison of Texts, Text Segments, Text Cumulations, Text Mixtures, and Complete Corpus Constancy of goodness of fit (C)Constancy of Parameters (K, M) Negative Hypergeometric Distribution

15. Analysis of Slovene Letter and Phoneme Frequencies: Corpus: ca. 130.000 Slovene LettersSlowene Phonemes

16. First Tentative Results of Slowak Letter Frequencies Tasks: 1.Interpretation of Parameters: „foreign letters Q-W-X“ influence inventory size 2.Exploration of Data Basis: Texts, Text Segments, Text cumulations, text mixtures

17. The Question of Data Homogeneity

18. “[…] the magnitude of words tends, on the whole, to stand in an inverse (not necessarily proportionate) relationship to the number of occurrences” Zipf (1935: 25) Four major problems in research

19. What is the direction of dependence: Does frequency depend on length or vice versa? What is the unit of measurement: Is word length measured in letters, phonemes, syllables, morphemes,...? What is frequency: Absolute occurrence or the rank of words, or of word forms? What is the text basis: Corpus data, frequency dictionaries,..., individual texts?

20. Assuming that word length is a variable of frequency Measuring word length in the number of syllables per word Analyzing the absolute occurrence of words the influence of the text basis shall be tested: Individual texts vs. text cumulations vs. corpus data DATA HOMOGENEITY

21. Intertextual Inhomogeneity vs. Intratexual Inhomogeneity Combination (“mixture”) of different texts A ‘text’ in itself does not consist of homogeneous elements Different Languages  Different Different Authors Different Text Types complete novel, composed of chapters complete book of a novel, consisting of several chapters individual chapters dialogical vs. narrative sequences within a text

23. Russian Anna Karenina (ch. 1) x frequency y length 1 2 3 4 5 6 7 8 9 10 13 19 20 37 2.92 2.14 2.05 1.50 1.33 1.50 1.67 1.00 3.03 2.04 1.70 1.53 1.43 1.36 1.31 1.27 1.24 1.22 1.17 1.12 1.11 1.06 a = 2.0261, b = 0.9660 R² = 0.88, N = 397

24. TextLanguage N R² a b Anna Karenina (I,1)Russian397 0.882,030,97 Evgenij Onegin (I)Russian1871 0.961,700,79 Na badnjakCroatian2450 0.931,950,51 Zářivé hlubinyCzech1363 0.941,760,59 Hiša M.P. (I) Slovenian1147 0.841,800,40 Zakliata pannaSlovak926 0.881,480,69 Hänsel und GretelGerman803 0.871,160,51 Fairy Tale by MóraHungarian234 0.961,570,84 Di lembung kuring Sundanese431 0.911,860,51 Burung api Indonesian1393 0.922,440,26 Portrait of a Lady (I)English1104 0.891,230,83 0.84  R²  0.96

25. The course of the theoretical curves

26. The relationship between parameters a and b

27. The relationship between text length (N) and parameter a

28. Obvious data inhomogeneity 1. Texts from different languages, authors, and various text types 2. Violation of the ceteris paribus condition Ergo: The data in this mixture are not adequate for testing the hypothesis at stake

29. Lev N. Tolstoj: Anna Karenina Chap. I,1 vs. I (34 chapters) N (Types) Cab AK (I, 1)3970.972.030.97 AK (I)86610.862,600.27

30. Henry James: Portrait of a Lady Chap. 1 vs. novel (52 chapters) N (Types) Cab I11040.891.230.83 I-52107270.581,840.27

31. N (Types) Cab narrative19130.911.930.54 dialogues6730.961.610.84 Ks.Š. Gjalski: Na badnjak Narrative vs. dialogical sequences

32. Evgenij Onegin Text cumulation (I – VIII) ChapterN Types M Tokens abR2R2 I I+II I-III I-IV I-V I-VI I-VII text (I-VIII) 1871 2918 3951 4851 5737 6509 7476 8329 3209 5546 8359 10936 13376 15978 19061 22482 1,70 1,84 1,92 1,97 1,95 1,97 2,03 2,05 0,79 0,69 0,57 0,53 0,48 0,52 0,43 0,40 0.96 0.88 0.92 0.94 0.86 0.88 Results of fitting y = ax^-b + 1 to the cumulative text of Evgenij Onegin

33. Evgenij Onegin – text cumulation (chap. I – VIII) Fitting y = ax^-b R² = 0.92 Dependence of parameter b on parameter a

34. Evgenij Onegin Text cumulation (I – VIII) Dependence of a on Text Length (N): a = 0.6493N 0.1286 (R² = 0.96 )

35. Summary & Results (I) Data corroborate hypothesis: There is a specific interrelation of parameters: a = f (N)b = g(a) b = h(N) f, g, h  functions of the same type

36. Summary & Results (II) 1.Homogeneous texts do not interfere with linguistic laws, inhomogeneous texts can distort the textual reality. 2.Text mixtures can evoke phenomena which do not exist as such in individual texts 3.Short texts do not allow a property to take appropriate shape; long texts (and corpora) contain mixed generating regimes superposing different layers, what may lead to “ artificial ” phenomena. 4.With an increase of text size the resulting curve of the frequency-length relationship is shifted upwards; this is caused by the fact that the number of words occurring only once increase up to a certain text length. If this assumption is correct, then b converges to zero, yielding the limit y = a.

37. F I N I S