Korea Maritime and Ocean University NLP Jung Tae LEE

Slides:

Advertisements

Similar presentations

Change-Point Detection Techniques for Piecewise Locally Stationary Time Series Michael Last National Institute of Statistical Sciences Talk for Midyear.

Advertisements

Artificial Neural Networks (1)

PARTITIONAL CLUSTERING

Phonic Phases linked to Letters and Sounds. Working within Phase 1.  Explores and experiments with sounds and words  Distinguishes between sounds in.

Summer 2011 Monday, 8/1. As you’re working on your paper Make sure to state your thesis and the structure of your argument in the very first paragraph.

Support Vector Machines

Acoustic Model Adaptation Based On Pronunciation Variability Analysis For Non-Native Speech Recognition Yoo Rhee Oh, Jae Sam Yoon, and Hong Kook Kim Dept.

Machine Learning Lecture 4 Multilayer Perceptrons G53MLE | Machine Learning | Dr Guoping Qiu1.

Ch. 4: Radial Basis Functions Stephen Marsland, Machine Learning: An Algorithmic Perspective. CRC 2009 based on slides from many Internet sources Longin.

Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.

Clustering short time series gene expression data Jason Ernst, Gerard J. Nau and Ziv Bar-Joseph BIOINFORMATICS, vol

Connectionist models. Connectionist Models Motivated by Brain rather than Mind –A large number of very simple processing elements –A large number of weighted.

Connectionist Modeling Some material taken from cspeech.ucd.ie/~connectionism and Rich & Knight, 1991.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks / Fall 2004 Shreekanth Mandayam ECE Department Rowan University.

True music must repeat the thought and inspirations of the people and the time. My people are children and my time is today.

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks I PROF. DR. YUSUF OYSAL.

Clustering Ram Akella Lecture 6 February 23, & 280I University of California Berkeley Silicon Valley Center/SC.

Lecture 09 Clustering-based Learning

Spelling is a tool for writing Virginia Outred and Jane Denny (CSO) From David Hornsby lecture

Radial Basis Function (RBF) Networks

Clustering Unsupervised learning Generating “classes”

This module provides training on how to give and score the new DIBELS measure called First Sound Fluency. CLICK.

Evaluating Performance for Data Mining Techniques

Methods in Medical Image Analysis Statistics of Pattern Recognition: Classification and Clustering Some content provided by Milos Hauskrecht, University.

The Tutorial of Principal Component Analysis, Hierarchical Clustering, and Multidimensional Scaling Wenshan Wang.

IE 585 Introduction to Neural Networks. 2 Modeling Continuum Unarticulated Wisdom Articulated Qualitative Models Theoretic (First Principles) Models Empirical.

Neural Networks Chapter 6 Joost N. Kok Universiteit Leiden.

Pronunciation done by Khawla Abdulla latifa Humaid pronunciation done by Khawla Abdulla latifa Humaid

Fourth Grade Reading Night Teaching the Five Components of Reading.

Korea Maritime and Ocean University NLP Jung Tae LEE

Chapter 7 Neural Networks in Data Mining Automatic Model Building (Machine Learning) Artificial Intelligence.

Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.

1 Pattern Classification X. 2 Content General Method K Nearest Neighbors Decision Trees Nerual Networks.

Recognition of spoken and spelled proper names Reporter : CHEN, TZAN HWEI Author :Michael Meyer, Hermann Hild.

American Speechsounds How to Use the Program. AmericanSpeechsounds Why use American Speechsounds? Practice the problem sounds of American English Learn.

CLUSTERING. Overview Definition of Clustering Existing clustering methods Clustering examples.

Graph-based Text Classification: Learn from Your Neighbors Ralitsa Angelova ， Gerhard Weikum : Max Planck Institute for Informatics Stuhlsatzenhausweg.

Introduction to Neural Networks and Example Applications in HCI Nick Gentile.

Data Science and Big Data Analytics Chap 4: Advanced Analytical Theory and Methods: Clustering Charles Tappert Seidenberg School of CSIS, Pace University.

BCS547 Neural Decoding.

BIRCH: Balanced Iterative Reducing and Clustering Using Hierarchies A hierarchical clustering method. It introduces two concepts : Clustering feature Clustering.

DATA MINING WITH CLUSTERING AND CLASSIFICATION Spring 2007, SJSU Benjamin Lam.

Neural Network Basics Anns are analytical systems that address problems whose solutions have not been explicitly formulated Structure in which multiple.

CS 8751 ML & KDDData Clustering1 Clustering Unsupervised learning Generating “classes” Distance/similarity measures Agglomerative methods Divisive methods.

381 Self Organization Map Learning without Examples.

Over-Trained Network Node Removal and Neurotransmitter-Inspired Artificial Neural Networks By: Kyle Wray.

Phonics Welcome. Please help yourself to refreshments.

Chapter 6 Neural Network.

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

The Language of Thought : Part II Joe Lau Philosophy HKU.

Clustering Approaches Ka-Lok Ng Department of Bioinformatics Asia University.

How Spelling Supports Reading Based on the article “Why Spelling Supports Reading And Why It Is More Regular and Predictable Than You May Think” By Louisa.

Big Data Infrastructure Week 9: Data Mining (4/4) This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States.

Teaching pronunciation

Parametric calibration of speed–density relationships in mesoscopic traffic simulator with data mining Adviser: Yu-Chiang Li Speaker: Gung-Shian Lin Date:2009/10/20.

Self-Organizing Network Model (SOM) Session 11

Lesson Plan: Phonemic awareness

Basic machine learning background with Python scikit-learn

Parametric calibration of speed–density relationships in mesoscopic traffic simulator with data mining Adviser: Yu-Chiang Li Speaker: Gung-Shian Lin Date:2009/10/20.

4.3 Feedforward Net. Applications

of the Artificial Neural Networks.

10.02-Procedure for Developing & Designing a Presentation

An Improved Neural Network Algorithm for Classifying the Transmission Line Faults Slavko Vasilic Dr Mladen Kezunovic Texas A&M University.

Text Categorization Berlin Chen 2003 Reference:

BIRCH: Balanced Iterative Reducing and Clustering Using Hierarchies

Today’s Lecture Project notes Recurrent nets Unsupervised learning

Learning to Read and Write

Today’s Lecture Project notes Recurrent nets Unsupervised learning

Presentation transcript:

Korea Maritime and Ocean University NLP Jung Tae LEE

` 1. Window Size Two reason of choose the window with seven letter  First, significant amount of the information needed to correctly pronounce a letters is contributed by the nearby letters.  Secondly, limited by computational resources to exploring small networks => the limited size of the window also meant that some important nonlocal information about pronunciation and stress could not be properly taken into account by our model.

` Mutual information provided by neighboring letters and the correct pronunciation of the center letter as a function of distance from the center letter.

` 2. Changes in the network Changed network performence  Dictionary : common English word layer repeat 11input groups & 80hidden units 7input groups & 80hidden units 25 passes7% higher > 55 passes97.5%95% The number of input groups was varied from seven to eleven.

Changed network performence  Dictionary : common English word Adding an extra layer of hidden units also improved the performace. layer repeat Two layers of 80hidden units 7input groups & 120hidden units 55 passes97% 1passes87%85% Network with two layers of hidden units was better at generalization but about the same in absolute performance.

` 3. Analysis of the Hidden Units Graphical representation of activation of the hidden units  Levels of activation in the layer of hidden units for a variety of words Phoneme, /E/ was produced by output. The input string is shown at the left with center letter emphasized. The area of the white square is proportional to the activity level. Chief_ speak_ negro nity_ least believe equa arty_ see_ appy_ each nily_ only_

` Hierarchical clustering of hidden units for letter to sound correspondences.

`  A hierarchical clustering technique was used to arrange the letter-to- sound vectors in groups based on a Euclidean metric in the 80- demensional space of hidden units. Hierarchical clustering of hidden units for letter to sound correspondences.  Shown figure, was striking : - the most important distinction was the complete separation of consonants and vowels. For the vowels : - the next most important variable was the letter. For the consonants : - clustered according to a mixed strategy that was based more on the similarity of their sounds.

`  The same clustering procedure was repeated for three networks starting from different random starting states. Hierarchical clustering of hidden units for letter to sound correspondences. - The patterns of weights were completely different. - But, the clustering analysis revealed the same hierarchies. With some differences in the details, for all three networks.

` 4. Conclusions NETtalk is and illustration in miniature of many aspects of learning. 1. Network start out without ”innate” knowledge in the form of input and output => network could have been traind on any language with the same set of letters and phonemes. 2. Network acquired its competence through practice, went through several distinct stages, and reached a significant level of performance 3. Network is distribute the information without single unit or link 4. The network was fault tolerant and degraded gracefully with increasing damage. => but, network recovered from damage much more quickly than it took to learn initially

Conclusions  NETtalk is too simple to serve as a good model for the acquisition of reading skills in humans - ex) when children learn to talk, after reprsentation for word and their meaning, they learn to read.  This approach would have to be generalized to account for prosodic features in continuous text.  Human level of performance would require the integration of information form several words at once

Korea Maritime and Ocean University NLP Jung Tae LEE