Total cognitive care for development of perfect baby brains Włodzisław Duch Dept. of Informatics, Nicolaus Copernicus University, Poland Dept. of Computer Science, School of Computer Engineering, Nanyang Technological University, Singapore Google: Duch

PlanPlan Background: cognitive infant developmentBackground: cognitive infant development Few science factsFew science facts Total cognitive infant careTotal cognitive infant care Toys that make you smartToys that make you smart Techno remarksTechno remarks Plans & strategy of developmentPlans & strategy of development Endless possibilities...Endless possibilities...

Developmental problems Brain is the most complex organ and frequently not functioning well. About 5-10% of all children have a developmental disability that causes a delay in their speech and language development. If serious hearing problems are not found in the first six months of life, and the baby given the appropriate treatment, that child will grow up with significant damage to his or her language abilities. Children with normal cognitive development whose hearing losses are identified before six months can develop language at the same or a similar rate to a hearing child. Identification of congenital hearing loss in USA is at 2½ years of age! The British Dyslexia Association estimates that 10% of children have some degree of dyslexia, while about 4% will be affected severely (an average of one in every class).

Cognitive development Brain: the ultimate engineering problem! How to improve it? Although genes set a limit for potential individual development this limit is never reached; genetic manipultion is dangerous. Early development is critical for unfolding brain’s potential. Challenge: prevent abnormalities and boost normal development. Since genes specify only roughly where neurons should go (ex: eye to visual cortex, across the whole brain) too many neurons are created, and those unused die (apoptosis). Maximum number of neurons: 1-2 month before birth, although infant’s brain is only ¼ of the final size. General principle: growing up is specializing = narrowing potential possibilities. How to keep more possibilities open?

Example: motor development Duke University, NC, 2002: velcro-covered mittens help 2-3 month infants develop more quickly. Infants are no yet able to grasp objects, but velcro mittens help them to catch and then touch toys; this speeds up motor coordination and perceptual development. Babies who had experience with the mittens outperformed the babies who didn't in a number of ways. Long-term consequences of this procedure are investigated by psychologists. News shown by CNN and elsewhere: in this area it is easy to get a free press... Lynn-Flynn effect: IQ grows everywhere in the world, 24 points in USA since 1918, 27 points in UK. Toys and nutrition help to develop better brains?

Infant communication Brain is capable of learning, but poor sensory analysis and motor coordination makes it difficult to communicate with infants. High-amplitude suction-methods are used with newborns and infants to find how much interest they have in different stimuli. Sign-language for communication with infants (US patent): using simple signs with babies to bridge the gap between understanding and speech development Recommendations of developmental psychologists: pre-natal stimulation: BabyPlus™ “enhanced heartbeat”, playing music to babies in the womb. Playing tapes with foreign language lullabies etc. All this is passive learning, much less effective than interactive learning in which the infant is actively engaged. Can one learn walking just by observation? Learning to perceive, speaking and drawing inferences should be active, guided by well-structured inputs and positive feedback.

Speech perception facts Speech perception is based on syllables, combinations of elementary phonemes; 2-month old recognize syllables. There are about 6800 languages, but only 800 phonemes. The ability to hear phonetic contrasts of all possible human languages is in-born, 6 month old are good in any language. This ability declines sharply 10 month after birth, only the ability to distinguish contrast in languages that are spoken to the child is preserved – one of the earliest specializations. At 5 month children are able to categorize a vowel in a speaker-independent way. Learning many languages correlates positively with IQ Tonal languages, prevalent in Asia, are phonetically rich, but even ‘simple’ /ra/-/la/ contrast is impossible for Japanese, /vi/-/bi/ for Spanish, /s/-/th/ for most non-English native speakers.

PhonemesPhonemes Vowels spoken by 97 speakers, seen in the first 2 formants only. Mapping to the auditory cortex looks similar. Categorical perception means that what we hear jumps from a to e with no gradation between. ma – ma diff?

Speech problems Up to 4 million primary and secondary school students in USA have difficulty distinguishing between phonemes, particularly between consonants like b, d and p; is it pee or bee, pun or bun? “Fast ForWord”® software by Scientific Learning helps to develop learning skills for 4-7 year olds. It is used in the Chicago public school system. Private clinicians provide Fast ForWord training (>$2,500). It shows remarkable success with kids who suffer from central auditory processing disorder that leads to speech understanding problems and retards reading. Big market: Learning Company, the producer of Reader Rabbit, was acquired by Mattel for $3.5 billion. Can we eliminate hearing and speech problems improving perceptual and cognitive skills of babies at the same time?

Active stimulator of brain’s speech centers, especially infant and children Application no: , date: , granted Creator and the sole rights holder: Duch Wlodzislaw, Torun, PL Active stimulator of brain’s speech centers, especially infant and children, with input device, analog-digital converter, memory containing phonemic patterns, comparator of phonemes received through inputs and retrieved from the memory, logical decision unit, digital-analog converter, and the output device for sound. The input device connects to the analog-digital converter, this converter is connected to one input of the comparator, another input of the comparator connects to the memory containing phonemic patterns, and the comparator’s output is connected to a logical decision unit; the output from this unit connects with the sound producing device through the digital-analog converter.

US abstract Hannah R Cohen (US) 1999 patent: A computer toy for infants that promotes normal speech development by facilitating the infant's experimentation with babbles and other elementary sounds. Additionally, the toy provides an enriched environment for language learning by prompting the infant with a repertoire of verbal sounds including phonemes, syllables, and simple spoken words. The toy includes a microphone input device for detecting vocalizations by an infant and an audiovisual output device for providing feedback to the infant including the immediate playback of the infant's own vocalizations, and a control means for transforming detected vocalizations into instructions for use by the audiovisual output device. Philips “Magic Mirror” is based on this principle.

Philips “Magic Mirror” Some companies are going in this direction, making interactive toys, for example:Some companies are going in this direction, making interactive toys, for example: “Magic Mirror” introduced in 2003 Records and replays baby’s voice, triggers first speech sounds and has mirror, pushing buttons produces some sounds, encourages experimentation. It is not aimed at enhancing specific perceptual skills.

Total cognitive care Observe the baby and interpret his/her behavior by monitoring vocalization, sucking response, movements, GSR etc. Challenge the baby to solve perceptual/abstract problems. Use natural audio-visual-tactile stimulation. Reward behaviors that correlate with differences in stimuli. For example, if a series of sounds is played: “la la la la ra ra ra ra”, reaction to la-ra change shows that the baby has noticed an important phonetic contrast. PerCog devices for enhancement and therapy of perceptual and cognitive skills use intelligent AI control to model what the baby has already learned, what and how often should be presented, observing, diagnosing, correcting and teaching infants. Hypothesis: active learning should gently pressure baby’s brain to develop perceptual/cognitive skills in the desired direction.

Toys for speech development Goal: enhance phonematic hearing. Prepare a database of phonemes and syllables, present pairs that are similar, and if the infant notices phonetic contrasts, apply positive stimulation. Start with basic contrasts, end with subtle. The infant is not able to repeat speech sounds: changes in the larynx, highly sophisticated control over vocal cords, are slower than neural development. Hypothesis: 1) Early (~10 month) specialization in correct discrimination of mother tongue phonetic contrast may be eliminated, giving the baby ability to learn any language she/he chooses. 2) Sharper discrimination between basic phonemes should prevent some hearing and speech problems.

Toys for musical ear Goal: enhance musical hearing, develop perfect pitch. Only one person in has perfect pitch; adult professional musicians try to learn it but it is very hard. Prepare a database of musical sounds of different instruments at different pitch, present pairs of sound samples with few seconds of silence in between, reward for signaling that sounds were of the same pitch although tones may differ. Start with pure frequencies, move to more complex sounds and accords of several sounds. Hypothesis: memory for absolute sound pitch should develop, increasing the ability to hear subtle musical structures.

Toys for abstract thinking Goal: enhance the ability to think! G. Marcus et al, “Rule learning by seven-month-old infants”, Science 1999, Vol. 283, pp. 77 – 80.Infants habituated for 2 minutes with sentences like ga ti ga, li na li, of the ABA structure, recognize that wo fe wo has correct grammatical structure but wo wo fe does not. G. Marcus et al, “Rule learning by seven-month-old infants”, Science 1999, Vol. 283, pp. 77 – 80. Challenge the infant using sounds and color lights with structures of increasing complexity, reward for noticing differences. Hypothesis: solving problems of this kind has strong influence on development of the cortex, in particular frontal and temporal areas, and should increase working memory span, that correlates well with general IQ.

More PerCog devices Goal: enhance perceptual discrimination in all modalities, improve categorical perception, increase working memory span, long-term perceptual memory, encourage abstractions and faster reactions. Responses to stimuli, behavioral interaction patterns should allow for early diagnosis of developmental problems.

Imagine... A world in which people could communicate without problems and even adults learn languages easily.A world in which people could communicate without problems and even adults learn languages easily. A world in which developmental abnormalities related to speech and reading would largely vanish.A world in which developmental abnormalities related to speech and reading would largely vanish. A world much more rich in subtle sensory experiences.A world much more rich in subtle sensory experiences. A world in which the potential of each child would always fully develop, and natural curiosity will grow...A world in which the potential of each child would always fully develop, and natural curiosity will grow... We just have to start quite early, adding structure to the development of the brain – in a natural, effortless way.We just have to start quite early, adding structure to the development of the brain – in a natural, effortless way. This is possible with PerCog devices, such as cognitive toys!This is possible with PerCog devices, such as cognitive toys! Guess what is the market for such devices?Guess what is the market for such devices?

So far Polish patent application was restricted to phoneme perception, the present proposal is much wider and differs in important aspects. We have made some research assuming that the main feedback from the infant will be via vocalization: phonemes from 28 languages have been collected, software to present them, recognize change from silence/speech and vowel/consonant change in baby’s vocalization has been used; this is done by microphone and speaker connected to PC sound card. Sucking (HAS), head turning preference, visual fixation and other procedures used by psychologists for infants < 6 month are better. Some work on hardware implementation has been done, but it still requires a PC to analyze the data; hardware working prototype due to technical problems.

RoadmapRoadmap First personalized medicine device, not related to molecular level: adapts to the individual needs/abilities. Patent application: “Devices for enhancement and therapy of perceptual and cognitive skills of infants and children” for all kinds of PerCog devices and total cognitive care of infants. Ready for quick commercialization and more R&D. Largest market for speech devices is in the Far East, in the USA more prevention/diagnosis/treatment of speech/reading problems. Construction of PerCog devices in form of electronic toys or crib environments, initially with a modest number of phonemes covering basic phonetic contrasts, is not difficult. Computer games for babies and children may be created using the same principle.

Research opportunities neurolinguistics, speech/hearing/language pathologist, education experts, signal processing,, data mining...Interdisciplinary group including experts in developmental psychology, phonetics, neurolinguistics, speech/hearing/language pathologist, education experts, signal processing, electronics (sensors + IC), toy design, software development, data mining... Creation of larger phonetic databases: most important phonetic contrasts for major languages, syllables and words which are hard to distinguish, like van/ban, thick/sick, extending it to cover subtle differences, and finally full spectrum of speech sounds. Improvement of the feedback recognition systems: wireless sucking monitoring devices, analysis of infant babbling for sounds that infants may use for signaling, observation of movements. Follow-up studies to investigate the effect of using cognitive speech toys on phonetic contrast perception (a few month), language learning (long-term studies) and reduction of speech and reading abnormalities (long-term).Follow-up studies to investigate the effect of using cognitive speech toys on phonetic contrast perception (a few month), language learning (long-term studies) and reduction of speech and reading abnormalities (long-term).

Brave new world? New generation of children may grow up to be: much more intelligent than we are think much faster than we do be much more musical be free of speech and hearing problems be able to learn any language without accent see things we hardly notice... Popular book: The Scientist in the Crib, Alison Gopnik, Andrew N. Meltzoff, Patricia K. Kuhl (William Morrow & Company 1999.