The Language of Thought : Part II Joe Lau Philosophy HKU

Issues Arguments for LOT. These are arguments based on inference to the best explanation. No reason to accept LOT if there are better alternative theories. Connectionism ?? –

What is connectionism? A theory that explains mental processes in terms of parallel computations by neuron-like units. – “Parallel Distributed Processing” – “Artificial Neural Networks”

A neural network

A neuron

Connections

Model

Operations of a neural network Units have input / output connections. amount of activation connection weight The activation received from each connection is the the amount of activation along that connection multiplied by the connection weight of the connection. The amount of output activation is a function of the total activation received. – Example : threshold function

Learning A network can be trained to perform an input/output task by modifying the connection weights. The weights encode the knowledge acquired in the learning process.

Examples Churchland’s sonar application Sejnowski and Rosenberg (1987) NetTalk – Task : pronouncing written text – Input : text, output : phonetic codes sent to speech synthesizer – 3 layer net with 80 hidden units – Learning : from babbling to intelligible speech

Attractions of neural networks Biological plausibility Pattern recognition / classification Graceful degradation – Damage – Incomplete input Implicit learning without task-specific programming

Implementation or alternative? How is connectionism related to LOT? Implementational connectionism – Shows how LOT can be neurally implemented Radical connectionism – Neural networks provide an alternative to LOT – No representations in the brain with combinatorial syntax or semantics

Two isses Are connectionist networks really biologically plausible? – Connections and that be both excitatory and inhibitory. – Some training rules not biologically realistic. Are connectionist networks incompatible with LOT? – What kind of representations appear in connectionist networks?

Localist representations Representations are identical to individual units, not groups of units. P&Q Q P

Problem with localism How can networks with purely localist representations explain systematicity and productivity?

Distributed representations Distributed representations : representations constituted by groups of units or patterns of activations over a group of units Two types of distributed representations

Example #1 : Structured Green grass Green snow White grass White snow

Another example

Example : Non-structured Green grass Green snow White grass White snow

Comments Many connectionist nets make use of structured representations as input and output representations. Planning and reasoning often requires changing part of the content of a representation. Not clear how this can be done using only unstructured distributed representations.

Argument from direct manipulation “Left, left and right. No, it should be left, right, and then right.” LLRLRR

Terrain of debate Connectionism Localist Distributed Structured Non-structured Not viable Compatible with LOT Alternative to LOT ?

Reminder Possibility of hybrid architectures : make use of both LOT and unstructured distributed representations. Knowledge representation very different in classical architectures. – Connection weights encode knowledge but it is often impossible to say which weights are responsible for which piece of knowledge.

A complex network

What word is this? FOOD