Motion Capture and Analysis of Tongue During Speech by Jared Kiraly

Purpose Collect articulatory movement of tongue, lips, jaw, soft palate. Use this information to develop a realistic display of movement data, for biofeedback purposes.

AG500 The machine used to collect the data is the AG500 Signal from 6 transmitter coils is recorded by the 12 sensors After collection, data is translated into X, Y, Z position data, as well as tilt and yaw angles Data is stored in binary double format

Model Deformation Vertices on a model are hand registered to correspond to sensor positions on tongue These vertices are used as control points for radial basis function deformation Radial basis function deformation moves points on a model towards a control vertex Points farther from control vertex are affected less

Visualizations of Moving Tongue General Features –Smoke trails for control points –Ability to rotate/translate model while animating –Option of viewing teeth/palate –Coordinates displayed with animation –Can rotate and translate model to view from all angles Use with “Reference Tongue” –Side by side –Overlapping –Overlapping with one model transparent Ideas tried but not used –Cross section Not deemed useful –Boolean operations Too slow

Graphing Data In addition to the visualization of the tongue, a program has been created to graph the captured data Noise, bad data can be found more easily this way

Issues/Future Work Points on model have to be hand picked, and are hard- coded –Other data readings cannot be read in dynamically Animation sometimes too slow Motion appears jerky due to noise in data, even after the data has been filtered –Improved transmitter coils are being installed Data collected from subjects does not yield good results If sensors are placed too close together, they can interfere with one another