1. Information Retrieval (7)
Prof. Dragomir R. Radev

2. 11. Lexical semantics and wordnet
IR Winter 2010 …
11. Lexical semantics and wordnet

3. Lexical Networks Used to represent relationships between words
Example: WordNet - created by George Miller’s team at Princeton
Based on synsets (synonyms, interchangeable words) and lexical matrices

4. Lexical matrix

5. Synsets Disambiguation Synonyms {board, plank} {board, committee}
substitution
weak substitution
synonyms must be of the same part of speech

6. $ ./wn board -hypen
Synonyms/Hypernyms (Ordered by Frequency) of noun board
9 senses of board
Sense 1
board
=> committee, commission
=> administrative unit
=> unit, social unit
=> organization, organisation
=> social group
=> group, grouping
Sense 2
=> sheet, flat solid
=> artifact, artefact
=> object, physical object
=> entity, something
Sense 3
board, plank
=> lumber, timber
=> building material

7. Sense 4
display panel, display board, board
=> display
=> electronic device
=> device
=> instrumentality, instrumentation
=> artifact, artefact
=> object, physical object
=> entity, something
Sense 5
board, gameboard
=> surface
Sense 6
board, table
=> fare
=> food, nutrient
=> substance, matter

8. Sense 7
control panel, instrument panel, control board, board, panel
=> electrical device
=> device
=> instrumentality, instrumentation
=> artifact, artefact
=> object, physical object
=> entity, something
Sense 8
circuit board, circuit card, board, card
=> printed circuit
=> computer circuit
=> circuit, electrical circuit, electric circuit
Sense 9
dining table, board
=> table
=> furniture, piece of furniture, article of furniture
=> furnishings

9. Antonymy “x” vs. “not-x” “rich” vs. “poor”?
{rise, ascend} vs. {fall, descend}

10. Other relations
Meronymy: X is a meronym of Y when native speakers of English accept sentences similar to “X is a part of Y”, “X is a member of Y”.
Hyponymy: {tree} is a hyponym of {plant}.
Hierarchical structure based on hyponymy (and hypernymy).

11. Other features of WordNet
Index of familiarity
Polysemy

12. Familiarity and polysemy
board used as a noun is familiar (polysemy count = 9)
bird used as a noun is common (polysemy count = 5)
cat used as a noun is common (polysemy count = 7)
house used as a noun is familiar (polysemy count = 11)
information used as a noun is common (polysemy count = 5)
retrieval used as a noun is uncommon (polysemy count = 3)
serendipity used as a noun is very rare (polysemy count = 1)

13. Compound nouns advisory board appeals board backboard backgammon board
baseboard
basketball backboard
big board
billboard
binder's board
binder board
blackboard
board game
board measure
board meeting
board member
board of appeals
board of directors
board of education
board of regents
board of trustees

14. Overview of senses
1. board -- (a committee having supervisory powers; "the board has seven members")
2. board -- (a flat piece of material designed for a special purpose; "he nailed boards across the windows")
3. board, plank -- (a stout length of sawn timber; made in a wide variety of sizes and used for many purposes)
4. display panel, display board, board -- (a board on which information can be displayed to public view)
5. board, gameboard -- (a flat portable surface (usually rectangular) designed for board games; "he got out the board and set up the pieces")
6. board, table -- (food or meals in general; "she sets a fine table"; "room and board")
7. control panel, instrument panel, control board, board, panel -- (an insulated panel containing switches and dials and meters for controlling electrical devices; "he checked the instrument panel"; "suddenly the board lit up like a Christmas tree")
8. circuit board, circuit card, board, card -- (a printed circuit that can be inserted into expansion slots in a computer to increase the computer's capabilities)
9. dining table, board -- (a table at which meals are served; "he helped her clear the dining table"; "a feast was spread upon the board")

15. Top-level concepts {act, action, activity} {animal, fauna} {artifact}
{attribute, property}
{body, corpus}
{cognition, knowledge}
{communication}
{event, happening}
{feeling, emotion}
{food}
{group, collection}
{location, place}
{motive}
{natural object}
{natural phenomenon}
{person, human being}
{plant, flora}
{possession}
{process}
{quantity, amount}
{relation}
{shape}
{state, condition}
{substance}
{time}

16. WordNet parameters wn reason -hypen - hypernyms
wn reason -synsn - synsets
wn reason -simsn - synonyms
wn reason -over overview of senses
wn reason -famln - familiarity/polysemy
wn reason -grepn - compound nouns

17. 12. Latent semantic indexing Singular value decomposition
IR Winter 2010 …
12. Latent semantic indexing
Singular value decomposition

18. Problems with lexical semantics
Polysemy (sim < cos)
Bar, bank, jaguar, hot
Synonymy (sim > cos)
Building/edifice, Large/big, Spicy/hot
Relatedness
Doctor/patient/nurse/treatment
Sparse matrix
Need: dimensionality reduction

19. Techniques for dimensionality reduction
Based on matrix decomposition (goal: preserve clusters, explain away variance)
A quick review of matrices
Vectors
Matrices
Matrix multiplication

20. Eigenvectors and eigenvalues
An eigenvector is an implicit “direction” for a matrix where v (eigenvector) is non-zero, though λ (eigenvalue) can be any complex number in principle
Computing eigenvalues:

21. Eigenvectors and eigenvalues
Example:
Det (A-lI) = (-1-l)*(-l)-3*2=0
Then: l+l2-6=0; l1=2; l2=-3
For l1=2:
Solutions: x1=x2

22. Matrix decomposition
If S is a square matrix, it can be decomposed into ULU-1
where
U = matrix of eigenvectors
L = diagonal matrix of eigenvalues
SU = UL
U-1SU = L
S = ULU-1

23. Example

24. Example Eigenvalues are 3, 2, 0
x is an arbitrary vector, yet Sx depends
on the eigenvalues and eigenvectors

25. SVD: Singular Value Decomposition
A=USVT
U is the matrix of orthogonal eigenvectors of AAT
V is the matrix of orthogonal eigenvectors of ATA
The components of S are the eigenvalues of ATA
This decomposition exists for all matrices, dense or sparse
If A has 5 columns and 3 rows, then U will be 5x5 and V will be 3x3
In Matlab, use [U,S,V] = svd (A)

26. Term matrix normalization
D1 D2 D3 D4 D5
D D D D D5

27. Example (Berry and Browne)
T1: baby
T2: child
T3: guide
T4: health
T5: home
T6: infant
T7: proofing
T8: safety
T9: toddler
D1: infant & toddler first aid
D2: babies & children’s room (for your home)
D3: child safety at home
D4: your baby’s health and safety: from infant to toddler
D5: baby proofing basics
D6: your guide to easy rust proofing
D7: beanie babies collector’s guide

28. Document term matrix

29. Decomposition
u =
v =

30. Decomposition Spread on the v1 axis s = 1.5849 0 0 0 0 0 0

31. Rank-4 approximation s4 = 1.5849 0 0 0 0 0 0 0 1.2721 0 0 0 0 0

32. Rank-4 approximation u*s4*v'

33. Rank-4 approximation u*s4 -1.1056 -0.1203 0.0207 -0.5558 0 0 0

34. Rank-4 approximation s4*v'

35. Rank-2 approximation s2 = 1.5849 0 0 0 0 0 0 0 1.2721 0 0 0 0 0

36. Rank-2 approximation u*s2*v'

37. Rank-2 approximation u*s2 -1.1056 -0.1203 0 0 0 0 0

38. Rank-2 approximation s2*v'

39. Documents to concepts and terms to concepts
A(:,1)'*u*s
>> A(:,1)'*u*s4
>> A(:,1)'*u*s2
>> A(:,2)'*u*s2
>> A(:,3)'*u*s2

40. Documents to concepts and terms to concepts
>> A(:,4)'*u*s2
>> A(:,5)'*u*s2
>> A(:,6)'*u*s2
>> A(:,7)'*u*s2

41. Cont’d >> (s2*v'*A(1,:)')' -1.7523 -0.1530 0 0 0 0 0 0 0
>> (s2*v'*A(2,:)')'
>> (s2*v'*A(3,:)')'
>> (s2*v'*A(4,:)')'
>> (s2*v'*A(5,:)')'

42. Cont’d >> (s2*v'*A(6,:)')' -0.4730 0.6078 0 0 0 0 0 0 0
>> (s2*v'*A(7,:)')'
>> (s2*v'*A(8,:)')'
>> (s2*v'*A(9,:)')‘

43. Properties A is a document to term matrix. What is A*A’, what is A’*A?
A'*A

44. Latent semantic indexing (LSI)
Dimensionality reduction = identification of hidden (latent) concepts
Query matching in latent space

45. Useful pointers http://lsa.colorado.edu

46. Readings
MRS18
MRS17, MRS19
MRS20