Cognitive Systems Foresight Language and Speech

Cognitive Systems Foresight Language and Speech How does the human system organise itself, as a neuro-biological system, to integrate top-down and bottom-up information during language interpretation and production? How does this relate to the organisation of current natural language processing systems? Lack of knowledge of working of neuro-biological speech & language processing, as compared with (say) human-visual system (because of inaccessibility of this region in brain). To what extent is it going to be easy to use fMRI techniques to analyse speech processing in the brain due to high noise levels with these techniques Also fMRI lacks temporal dynamics, perhaps MEG would be more useful for capturing this aspect of speech (esp with regard to switching linear dynamical systems models). Gains to be had by combining various modalities. There is a need to define the necessary experimental resources and ways of developing and deploying the relevant technology for the community.

Cognitive Systems Foresight Language and Speech How far are the characteristics of human processors determined by the statistical properties of the speech input and how do these relate to current statistical techniques used in automatic speech recognition? Modelling of semantics – how do we formalise the representation of conversational styles (i.e. dynamic aspects of conversational styles) The use of Partially Observable Markov Decision Processes for this task (in conjunction with reinforcement learning). Hindered by lack of data… Use of game theory with multiple agents for use in dialogue systems

Cognitive Systems Foresight Language and Speech What problems must we solve to develop human-computer interfaces which demonstrate human levels of robustness and flexibility? Engineering systems do not necessarily need to be inspired by neuro- biological systems. But our current systems are starting to reach their performance limits, so we do need to look towards cognitive sciences for inspiration. We need to understand what information is available in speech signals which current systems (HMMs etc) are not making use of.

Cognitive Systems Foresight Language and Speech Will the study of language in its interactive context lead to new approaches to basic language processing, and to the design of dialogue systems? Yes – we need our systems to start processing continuous streams, recognising that there is both a listener and a speaker. We have a good understanding of representation of both low and high-level information, but not the intermediate level. However the real improvement in computer vision happened when we stopped trying to understand this level and let machine learning techniques deal with it. Does “intermediate” mean the same thing in machine learning and neuroscience? Key question - do we really need to understand this intermediate representation (both in systems and in the brain)? Emphasis should be on understanding speech processing in the brain to highlight where our current statistical models differ from this processing, and then use this knowledge to advance the state of the art (as opposed to taking direct inspiration from biology when developing our systems).