GLOCALL 2015 Globalization and Localization in Computer-Assisted Language Learning The future of Vocabprofiling Tom Cobb Université du Québec à Montréal

GLOCALL 2015 Globalization and Localization in Computer- Assisted Language Learning The future & meaning of Vocabprofiling Tom Cobb Université du Québec à Montréal

Backgrounder Vocabprofiling is an activity performed on my website Lextutor Which is probably the thing I am best known for + the reason you have asked me here So the framing question of my presentation is this: 3

Is Lextutor a bunch of software, or a coherent theory of language acquisition? 4

A clue. 5

Take 2 on a framing issue with a bit of “localization” Korean learners are interested in vocabulary lists? Don’t try to stop this interest Instead give them the right lists and a good way to use them 6

First some scene-setting stats So A LOT of people use this 7

Korea always in daily Top 10 user countries 8

And elsewhere in GloCall land... 9

10

At a rate of… 11

What do these folks look at ? Basically every routine gets significant use 12

13

What do these folks look at ? But a few routines get the lion’s share 14 VOCABPROFILE group VOCAB TESTS CONCORDANCE group

What do these folks look at ? But a few routines get the lion’s share 15 VOCABPROFILE group VOCAB TESTS CONCORDANCE group

What do these folks look at ? But a few routines get the lion’s share 16 VOCABPROFILE group VOCAB TESTS CONCORDANCE group

So what is Vocabprofiling? 17 (A.k.a. VP-ing) => a procedure for matching learner to text through word frequency

What is word frequency? What is concordancing? 18

19

Word frequency in a general corpus = Best predictor of word knowledge 20

Which eventually gives us… 21 Etc.

…From which we can determine The frequency of the words the learners know vs. the frequency of the words in a given text 22

1. The frequency of the words the learners know 23

2. The frequency of the words in a given text 24 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours.

So when we know the profile of the learner, + the profile of the text, + that 95% of words must be known for minimal comprehension …we can make some fairly reliable and very useful predictions 25

Using this simple multiplication formula Learner profile K1 - 70% x K2 - 50% x K3 - 40% x K4 - 20% x Text profile K1 - 80% = K2 - 10% = K3 - 5% = K4 - 5% = % 26 Predicted % of words comprehended K1 - 56% K2 - 5% K3 - 2% K4 - 1% % With <95% of words known, even just to level of passive meaning recognition, comprehension will be minimal

Have you ever seen something like this in you learners reading materials? 27

Case 1: Can this learner read this text ? CHANGE THIS: START WITH IMPOSSIBLE CASE 28 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours.

Yes The learner knows all the words sampled at 1k- 2k-3k The text is 95% 1k-3k words, so he probably knows all those The learner knows > ½ the words 4k-5k The text is 5% of 4k-5k words The learner will know > half of these = 3% The learner knows 95+3=98% of words in the text 29

Case 2: Can this learner read this text ? 30 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours.

31 This learner will know most of the k1-k2 words (90%), but half or less k3 and up But 10% of words in this text are k3 and up! This learner will know about 93% of the words

Here is this text with 93% of words known lextutor.ca/cloze/vp/ 32

So what can this learner do with this text? First let us nuance the 95% empirical finding 33

From empirical research, we know that… 34 With <90% of words known, comprehension is difficult/impossible With 90-95% known, comprehension is possible with resources Dictionary, discussion, content lecturer, seminar discussion… With >95% known, (1) independent reading is feasible (2) between 95-98%, fluency will improve

So, again, can this learner read this text ? 35 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours. YES for a with-resources intensive reading task NO for a reading examination NO if the goal is fluency development

Case 3: Can this learner read this text ? 36 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours.

The learner knows 70% of K1 words The text is 74% k1 words He will know about.7 x.7= 50% of k1 words He knows a handful after k1 The text is 27% after k1 At most he has 20% of these So he will know.27x.2= 5.4% of 2k-5k words So he will know < 60% of words in the text 37

Here is this text with 68% of words known… lextutor.ca/cloze/vp/ 38

What happens if you give this text to this learner? 39 There are strict time limits on the detention of persons without charge. An arrested person may not be detained without charge for more than 24 hours, unless a serious arrestable offence has been committed. If a serious arrestable offence has been committed a superintendent can extend the period to 36 hours to secure or preserve evidence by continued questioning. When a serious arrestable offence has been committed and the suspect needs to be held in custody beyond the 36 hour period, the police must bring the suspect before a magistrate to extend the time limit to a maximum of 60 hours.

Have you ever seen something like this in you learners readers? 40 This is an, um… undesigned reading experience With what objective?

Don’t believe this works? Test this idea with your learners Here are 7 handy pre- profiled texts, with 95% cut-offs at different k- levels (pic is link) 41

So with Testing+VP we can find level-appropriate and task-appropriate texts Only find? Where will we find interesting texts for beginnners? With VP we can also create such texts 42

43

44

Original Edited-to-a-profile 45

Plus this neat little feature 46

Ongoing VP work BNC lists supplemented by BNC-Coca – Brit and US English Frequency joined by Greco-Latin / Anglo-Saxon indicators – A.k.a. Multi- and single syllable words Incorporation of Multi-word units – “a lot” should be one k-1 unit, – not one k-1 + one k-3 Differentiation of homoforms – River bank and money bank should be two words at two k- levels AND FINALLY Move to Mobile 47

48

49 The final vocab lists we need 1,000 List3,000 List ? somethingcourse… …something_of_a bank _1 …… of_coursebank _2

General move to phrases in Lextutor 50

So the Future of Vocabprofiling? Ever more sophisticated analysis, without leaving teacher behind Cut some words in two (bank) And some words into phrases (a lot) Allow teachers to build collective library of texts on Lextutor, by topic x and 95% k-level y Adapt VP as Web search engine to find texts of keywords x, y, z and VP k1=x, k2=y, k3=z with Google API 51

Need more? > – Horst & Cobb chapter on Vping in… 52

So we have seen some useful VP tips and tricks from Lextutor But is it more? Thesis: Text computing + language acquisition are tightly connected in 2015 ALL LEXTUTOR ROUTINES ARE INSTANCES OF ‘DATA DRIVEN LANGUAGE LEARNING’ (DD-LL) 53

DD-LL in the broader scheme 54

DD-LL in the broader scheme 55

DD-LL in the broader scheme 56

Input as data Large supplies of ‘Comprehensible input’ in real life are not simple to come by While waiting, ‘Comprehensible computer input’ can do some of the job + create readiness We saw VP predict text readability Computing can also make language comprehensible in several other ways 57

Concordance 1. For word meaning If you can’t infer a meaning in one context, maybe in another 58

Concordance 2. To expose collocations 59

Concordance 3. For what does not exist in a language 60

Concordance 4. As data-linked writing tool 61

62

TTS To (1) slow speech down and (2) allow repetition Generated straight from text without pre-recording 63

Code-link texts to external resource Many generated by algos straight from text without recording 64

So if ‘comprehensible input’ is a theory of language acquisition Then Data-Driven Language-Learning is a theory about how to assure a reliable supply of this – Esp where NSs are in short supply And our best guess yet as to a principled use of the computer in language learning 65

But wait! Does DD-LL ‘work’? Ongoing meta-analysis of a wide range of DDLL application with the great Alex Boulton 66

205 recent DDL studies investigated 56 studies selected for analysis (selected for pre-post or exptl-control quantitative design) 67

149 studies thrown out for… Small n-size Unreported or uncalculable Std Deviations Plagiarism of other studies/ students’ PhDs Math errors Leaving these in the final line-up 68

69

Many of the 56 From GloCALL zone Nam2010; Yoon & Jo,

A wide range of RQ’s 71 Does DDL help with grammar in academic writing? Does DDL help with synonym use in academic writing? Does DDL help with preposition choice in academic writing? Does DDL help with writing in academic writing? Does DDL help with word collocations in academic writing? Does DDL help with word connotations in academic writing? Does DDL help with phraseology in academic writing? Does DDL help with spelling in academic writing? Does "corpus-based collocation instruction" help with collocations? Does DDL help with lexical knowledge (definitions and translations)? Does DDL help with interpreting? Does DDL help with lexical knowledge (definitions and translations)? Does DDL help with interpreting? Does DDL help with lexical knowledge (definitions and translations)? Does DDL help with interpreting? etc…

Boiled down to a common measure: e.s. (= comparing means in light of overall standard deviation) 72

So, over 56 good studies, pre vs. post means, or control vs. experimental means are an average 1.46 Std Devs apart 73

So e.s.=1.46 would be something like this (about a 15% difference with typical SDs) Group A (Control or Pre) M = 71 SD = 8 Group B (Experimental or Post) M = 82.5 SD = 6 74 e.s. = 11.5 / 7 = 1.5

So DD-LL ‘works’ for many learning objectives The only problem is, usually in some amazingly adapted version of concordancing Which can be found where? Not exactly the delight of commercial app-developers Obviously I see Lextutor as providing accessible DDL software Through a sustained program of continuous development A particular example  75

76

77

The End Thank You Questions now, or via 78 Tom Cobb Université du Québec à Montréal