1. 1 Pattern and Speech Recognition Pattern Recognition John Beech School of Psychology PS1000

2. 2 Pattern Recognition The term “pattern recognition” can refer to being able to recognise 2-D patterns, in particular alphanumerical characters. But “pattern recognition” is also understood to be the study of how we recognise objects in our environment. The term “object recognition” is more specific and is just about recognising 3-D objects. We appear to be able to be very flexible, e.g. handwriting takes many different forms and still can be read (usually). See the figure on the next slide.

3. 3 Examples of different letter As.

4. 4 Pattern Recognition We are still better than computers as pattern recognisers. E.g. at Carnegie-Mellon a program called “Gimpy” has an 850-word dictionary and it chooses a word from here, and mangles the image by letter warping, putting in distractions (e.g. blobs) in foreground and background. The web user has to recognise the word, which humans can do easily – even a child – but computer programs find this very difficult. As we find it so easy to pattern recognise, often psychologist resort to very brief presentations or they use reaction time measures.

5. 5 Pattern Recognition For this topic we are going to cover the following areas: 1.Template theory 2.Feature theory and its evidence 3.Global vs local processing 4.Structural theory

6. 6 1. Template theory Template theory suggests a copy of each pattern in long term memory (LTM) for all known patterns. The closest match to a template produces recognition. These templates are wholistic and unanalysed. But there are difficulties…

7. 7 1. Template theory The problems with templates The template, when the comparison is made, has to be in the same position and at the same angle AND be the same size. So there is the problem that a system using templates would be continuously adjusting templates to match up – and this is supposing that the correct template was chosen in the first place. Another difficulty is that there is such a great variety of potential patterns, even if we were to confine ourselves just to recognising handwriting. Thus stimuli can be very complex, so a template system might be unworkable.

8. 8 1. Template theory Although one might dismiss template theory, there could be situations in which it is viable. There is evidence that sensory information is preserved briefly in a way that could be viewed as unanalysed templates. These could be subsequently analysed during the process of pattern recognition.

9. 9 1. Template theory Phillips (1974) gave subjects patterns (8 x 8 matrices) such as this: It was on for 1 second and then went off. Straight afterwards another matrix would appear that was either identical or similar in pattern and Ss had to say if it were the same or different. Half the time this 2 nd matrix was in the same location and in the other half of trials it had moved by exactly the width of one cell. Phillips also varied the time between the two matrices. Moved? StillMove Same Diff

10. 10 1. Template theory The results on the right are what Phillips found. In the “still” condition where the pictures were in the same location accuracy is very good at the shortest interstimulus interval (20 msec). But this declines rapidly. So a sensory store was being used but information decayed from it very rapidly. When we look at the “move” condition, where the second matrix was in a different position, the sensory store couldn’t be used so interstimulus interval has no effect on accuracy. This might be seen as evidence for some kind of rapid template matching in the 20 msec “still” condition.

11. 11 2. Feature theory Template theory might give a good account of how a sensory image is stored. But such an image is only fleeting. So this may not ultimately have much of a role in the processing of the image. To achieve this it is more likely that some kind of description of the image is used. In order to do this the parts of the images have to be described, which is where feature theory comes in. Gibson (1969) proposed that when children process patterns they discover features that are used to differentiate one pattern from the other. Consider M and N. A child initially is likely to find these two confusing, so the slanting line on the M that slants upwards is a distinctive feature that can be used to differentiate the two.

12. 12 2. Feature theory Some of the evidence for feature theory: Simple patterns can be broken into units or features Hubel & Weisel (1962) observed single cell firing in cat’s visual cortex. Small bar of light rotated (7 deg) affected firing. Angle of bar analysed.

13. 13 2. Feature theory Some of the evidence for feature theory: Their experimental work suggested simple cells with excitatory and inhibitory regions (the white and grey regions below), so the cell fires if there is input (light) in its receptive field. So it is responding to spatial features. Then there are complex cells as shown below. Complex cells for edges at a particular orientation. They would also operate in a larger region. The hypercomplex cells could respond, e.g. to 2 edges at right angles in an even larger region.

14. 14 2. Feature theory Stabilised image on retina by Pritchard (1960). Image projected to retina disappears systematically. Loss of features.

15. 15 2. Feature theory Neisser (1964): Searching for a letter in a matrix of letters shows influence of features when searching for the letter (e.g. Q) in these two lists: FTHLI LMVXZ ETVQH INHET ODBCU DOSGB RPBOU QSGUR

16. 16 2. Feature theory The pandemonium model (Selfridge, 1959) This relies on the extraction of features. Level 1: feature demons. Level 2: cognitive demons. Level 3: the decision demon. Predicts confusion between letters with similar visual features.

17. 17 2. Feature theory: The pandemonium model (Selfridge, 1959) Feature demonsCognitive demonsDecision demon Vertical lines?Letter A? Horizontal lines?Letter B? Oblique lines?Letter C? Right angles?Letter D? Acute angles?Letter E? Discontinuous curves?Letter F? Continuous curves?etc

18. 18 2. Feature theory: An evaluation In relation to template theory, feature theory is more efficient as there would probably be much fewer features than templates. Stimulus equivalence: this is an advantage for feature theory - a character could have many different forms and still be recognised by feature theory. However, the figure shows effect of context on pattern recognition. Middle letter the same in features but read differently. Feature theory does not acknowledge that context important. 3-D stimuli more difficult to identify from features. One needs more than just features. Need to know the relationships between features.

19. 19 2. Feature theory: An evaluation Featural analysis needs time. But if need to process features of individual letters, how is this done so fast? For instance, we read at 100-400 words/min (average 300), little time for analysis. One estimate is that normal reading in this manner would need 5,000 feature detections a second. We can read sentences quite well, even if some of the letters are missing. In this case we use our knowledge of grammar, of words and so on to help us to fill in the gaps. E.g. “W_ f_ll _n g_ps _n s_nt_nc_s w_ll” Similarly, when shown even a simple pattern (e.g. a letter), one has to know the spatial relationships between the features. Is it possible to recognise an overall shape before examining its features?

20. 20 3. Global v. local processing Psychologists (and philosophers before them) have been interested in the concept of processing local aspects of stimuli vs. the global impression of the stimulus. The global aspect suggests that one is processing something more than the sum of the parts. This was close to the ideas of the Gestalt psychologists. One way of illustrating this is by looking at the following: AAAAAAA EEEE AAAAAAA EE EE AA EE EE AAAAAAA EEEEEEEE AAAAAAA EEEEEEEE AA EE EE AAAAAAA EE EE AAAAAAA EE EE

21. 21 3. Global v. local processing Navon (1977) examined this issue as follows: He produced large upper case letters made up of smaller letters (see fig. of global letter H comprised of local letter Ss) People heard ‘H’ or ‘S’ faintly and had to choose whether they’d heard H or S. Also shown picture. So, the task was to say if they’d heard an S or an H sound (which was very faint). Found: If shown global H (composed of local letter “Ss”) and the sound was actually H, Ss faster than if global H shown and the sound was an S. Local letters had no effect. In fact most failed to notice the local letters. In this case showed that the whole can be seen before the parts.

22. 22 3. Global v. local processing In another experiment, Navon (1977) gave his subjects two separate tasks. Task 1 They had to decide if they had been given a global H or a global S. Result They were not affected by the local letters. Task 2 They had to decide if they’d been given local H or local S. Result They were affected by the global letters. So if given local Hs they found it more difficult if these were within a global S rather than a global H. Navon concluded that it seems one invariably processes overall shape.

23. 23 4. Structural descriptions A problem for featural theory is that we construct descriptions of patterns in the absence of any features. This picture is a good example of that: Here we create the triangle even though there are actually no edges of the triangle shown. It seems likely that we use structural descriptions. These consist of propositions, or statements, describing the nature of the relationships between elements in a picture or pattern. For instance, for the letter L it might be: ‘it has a horizontal line and a vertical line. They are joined so that the left-most part of the horizontal is connected to the lowest point of the vertical line’.

24. 24 4. Structural descriptions These descriptions are about the important aspects or the structure of the picture. For instance, length of lines is not important. Featural theory does not make any discrimination between features on the grounds of importance. Structural descriptions are important for 3-D structures. Early work by Winston (1975) involved producing an analysis of pictures of blocks. This analysed for L, T and K shapes as well as for arrows and forks. The program worked out concave and convex edges as well as surfaces and likely shapes (see next Figure).

25. 25 4. Structural descriptions: Winston 1975

26. 26 4. Structural descriptions Biederman’s component model He believed that structural descriptions wouldn’t describe the lines and curves in an object. Instead, it would describe simple volumes, such as cones, cubes, cylinders, etc. So Biederman thought in terms of a limited number of components that could describe most objects. Different arrangements can produce different objects. E.g the mug and pail in the picture. Biederman (1985) suggested that we need only 35 geons, as he called them, to represent the world of objects as we know them. Thus structural descriptions need only be concerned with the relations between a limited set of components.

27. 27 4. Structural descriptions Biederman’s component model In his experiment he removed 65% of the contours, such as from the objects to the right. But in the one on the left he removed from the middle of the segments, while for the one on the right he removed from the vertices (where the segments joined). They were then presented briefly (100 msec) and performance was 70% v. 50% for the left and right objects (averaging over similar types of objects). So people find relational information very useful.

28. 28 4. Structural descriptions Conclusion for structural descriptions Structural theories are necessary because they extend feature theories by showing the relationships between features. And to be able to work we need a quite powerful descriptive language to account for our impressive abilities at pattern recognition.

29. 29 Conclusion of pattern recognition 1. Template theory suggests that patterns are matched to internal templates. But unworkable with complex stimuli. 2. Feature theory is more efficient than template theory and there are several strands of evidence suggesting the use of featural analysis: e.g. Hubel & Weisel (1962), Pritchard (1960), Neisser (1964). The Pandemonium model shows how featural analysis might operate. However, featural analysis has problems as it is insufficient by itself to account for many things, such as the effect of context and the relationships between features. 3. Global vs. local processing – it appears that global processing is more important than featural processing (e.g. Navon 1977). People have a preference for overall shape over individual features. 4. Structural theory involves propositional descriptions of the important aspects of the structure of a picture. Structural features have more to offer than featural theory by showing the relationship between features. But the challenge for structural theory is to account for the impressive abilities of humans over computers in the sphere of pattern recognition.