1. Discussion Class 3
Stemming Algorithms

2. Discussion Classes Format: Question
Ask a member of the class to answer
Provide opportunity for others to comment
When answering:
Give your name. Make sure that the TA hears it.
Stand up
Speak clearly so that all the class can hear

3. Question 1: Conflation methods
(a) Define the terms: stem, suffix, prefix, conflation, morpheme
(b) Define the terms in the following diagram:
Conflation methods
Manual Automatic (stemmers)
Affix Successor Table n-gram
removal variety lookup
Longest Simple
match removal

4. Question 2: CATALOG System
Search Term: users
Term Occurrences
1. user 15
2. users 1
3. used 3
4. using 2
Which term (0 = none, CR = all):
(a) The CATALOG stemmer differs in a fundamental way from other tools that we have seen in this course. What is it?
(b) What impact does this have on measurements of precision and recall?

5. Question 3: Successor variety methods
Test word: FINDABLE
Corpus: ABLE, APE, DAB, DABBLE, FIN, FIND, FINDABLE, FINDER, FOUND, FINISH, FIXABLE
Prefix Successor Variety Letters F FI
FIN
FIND
FINDA
FINDAB
FINDABL
FINDABLE
(a) Fill in this table

6. Question 3 (continued): Successor variety methods
(a) Segment FINDABLE using the complete word segmentation method.
(b) Segment FINDABLE using the peak and plateau method.

7. Question 4: n-gram methods
(a) Explain the following notation:
statistics => st ta at ti is st ti ic cs
unique diagrams =>at cs ic is st ta ti
statistical => st ta at ti is st ti ic ca al
unique diagrams => al at ca ic is st ta ti
(b) Calculate the similarity using Dice's coefficient:
S = 2C
A + B
A is the number of unique diagrams in the first term
B is the number of unique diagrams in the second term
C is the number of shared unique diagrams
(c) Explain the statement that it is a bit confusing to call this a "stemming method".

8. Question 5: Porter's algorithm
(a) What is an iterative, longest match stemmer?
(b) How is longest match achieved in the Porter algorithm?

9. Question 6: Porter's algorithm
Conditions Suffix Replacement Examples
(m > 0) eed ee feed -> feed
agreed -> agree
(*v*) ed null plastered -> plaster
bled -> bled
(*v*) ing null motoring -> motor
sing -> sing
(a) Explain this table
(b) How does this table apply to: "exceeding", "ringed"?

10. Question 7: Evaluation
(a) In Web search engines, the tendency is not to use stemming. Why? (There are at least three answers.)
(b) Does your answer to part (a) mean that stemming is no longer useful?