1. Hubs and Authorities & Learning: Perceptrons
Artificial Intelligence
CMSC 25000
February 3, 2004

2. Roadmap Problem: Methods: Challenge I: Beyond literal matching
Matching Topics and Documents
Methods:
Classic: Vector Space Model
Challenge I: Beyond literal matching
Expansion Strategies
Challenge II: Authoritative source
Hubs & Authorities
Page Rank

3. Authoritative Sources
Based on vector space alone, what would you expect to get searching for “search engine”?
Would you expect to get Google?

4. Issue Text isn’t always best indicator of content Example:
“search engine”
Text search -> review of search engines
Term doesn’t appear on search engine pages
Term probably appears on many pages that point to many search engines

5. Hubs & Authorities Not all sites are created equal
Finding “better” sites
Question: What defines a good site?
Authoritative
Not just content, but connections!
One that many other sites think is good
Site that is pointed to by many other sites
Authority

6. Conferring Authority Authorities rarely link to each other Hubs:
Competition
Hubs:
Relevant sites point to prominent sites on topic
Often not prominent themselves
Professional or amateur
Good Hubs Good Authorities

7. Computing HITS Finding Hubs and Authorities Two steps: Sampling:
Find potential authorities
Weight-propagation:
Iteratively estimate best hubs and authorities

8. Sampling Identify potential hubs and authorities
Connected subsections of web
Select root set with standard text query
Construct base set:
All nodes pointed to by root set
All nodes that point to root set
Drop within-domain links
pages

9. Weight-propagation Weights: Updating: Converges
Authority weight:
Hub weight:
All weights are relative
Updating:
Converges
Pages with high x: good authorities; y: good hubs

10. Google’s PageRank Identifies authorities
Important pages are those pointed to by many other pages
Better pointers, higher rank
Ranks search results
t:page pointing to A; C(t): number of outbound links
d:damping measure
Actual ranking on logarithmic scale
Iterate

11. Contrasts Internal links Outbound links explicitly penalized
Large sites carry more weight
If well-designed
H&A ignores site-internals
Outbound links explicitly penalized
Lots of tweaks….

12. Web Search Search by content Search by structure Vector space model
Word-based representation
“Aboutness” and “Surprise”
Enhancing matches
Simple learning model
Search by structure
Authorities identified by link structure of web
Hubs confer authority

13. Nearest Neighbor Summary

14. Nearest Neighbor: Issues
Prediction can be expensive if many features
Affected by classification, feature noise
One entry can change prediction
Definition of distance metric
How to combine different features
Different types, ranges of values
Sensitive to feature selection

15. Nearest Neighbor: Analysis
Issue:
What features should we use?
E.g. Credit rating: Many possible features
Tax bracket, debt burden, retirement savings, etc..
Nearest neighbor uses ALL
Irrelevant feature(s) could mislead
Fundamental problem with nearest neighbor

16. Nearest Neighbor: Advantages
Fast training:
Just record feature vector - output value set
Can model wide variety of functions
Complex decision boundaries
Weak inductive bias
Very generally applicable

17. Summary: Nearest Neighbor
Training: record input vectors + output value
Prediction: closest training instance to new data
Efficient implementations
Pros: fast training, very general, little bias
Cons: distance metric (scaling), sensitivity to noise & extraneous features

18. Learning: Perceptrons
Artificial Intelligence
CMSC 25000
February 3, 2003

19. Agenda Neural Networks: Perceptrons: Single layer networks Conclusions
Biological analogy
Perceptrons: Single layer networks
Perceptron training
Perceptron convergence theorem
Perceptron limitations
Conclusions

20. Neurons: The Concept Dendrites Axon Nucleus Cell Body
Neurons: Receive inputs from other neurons (via synapses)
When input exceeds threshold, “fires”
Sends output along axon to other neurons
Brain: 10^11 neurons, 10^16 synapses

21. Artificial Neural Nets
Simulated Neuron:
Node connected to other nodes via links
Links = axon+synapse+link
Links associated with weight (like synapse)
Multiplied by output of node
Node combines input via activation function
E.g. sum of weighted inputs passed thru threshold
Simpler than real neuronal processes

22. Artificial Neural Net w x w
Sum Threshold + x w x

23. Perceptrons Single neuron-like element Binary inputs Binary outputs
Weighted sum of inputs > threshold

24. Perceptron Structure y compensates for threshold w0 wn w1 w3 w2 x0=1 xn
x0 w0
compensates for threshold

25. Perceptron Convergence Procedure
Straight-forward training procedure
Learns linearly separable functions
Until perceptron yields correct output for all
If the perceptron is correct, do nothing
If the percepton is wrong,
If it incorrectly says “yes”,
Subtract input vector from weight vector
Otherwise, add input vector to weight vector

26. Perceptron Convergence Example
LOGICAL-OR:
Sample x1 x2 x3 Desired Output
Initial: w=(0 0 0);After S2, w=w+s2=(0 1 1)
Pass2: S1:w=w-s1=(0 1 0);S3:w=w+s3=(1 1 1)
Pass3: S1:w=w-s1=(1 1 0)

27. Perceptron Convergence Theorem
If there exists a vector W s.t.
Perceptron training will find it
Assume for all +ive examples x
||w||^2 increases by at most ||x||^2, in each iteration
||w+x||^2 <= ||w||^2+||x||^2 <=k ||x||^2
v.w/||w|| > <= Converges in k <= O steps

28. Perceptron Learning Perceptrons learn linear decision boundaries E.g. x1 x2 x2 +
But not + x1
xor
X1 X2
w1x1 + w2x2 < 0
w1x1 + w2x2 > 0 => implies w1 > 0
w1x1 + w2x2 >0 => but should be false
w1x1 + w2x2 > 0 => implies w2 > 0

29. Perceptron Example Digit recognition Assume display= 8 lightable bars
Inputs – on/off + threshold
65 steps to recognize “8”

30. Perceptron Summary Motivated by neuron activation
Simple training procedure
Guaranteed to converge
IF linearly separable