1. Evolutionary Search Artificial Intelligence CSPP 56553 January 28, 2004

2. Agenda Motivation: –Evolving a solution Genetic Algorithms –Modeling search as evolution Mutation Crossover Survival of the fittest Survival of the most diverse Conclusions

3. Genetic Algorithms Applications Search parameter space for optimal assignment –Not guaranteed to find optimal, but can approach Classic optimization problems: –E.g. Traveling Salesman Problem Program design (“Genetic Programming”) Aircraft carrier landings

4. Genetic Algorithms Procedure Create an initial population (1 chromosome) Mutate 1+ genes in 1+ chromosomes –Produce one offspring for each chromosome Mate 1+ pairs of chromosomes with crossover Add mutated & offspring chromosomes to pop Create new population –Best + randomly selected (biased by fitness)

5. Fitness Natural selection: Most fit survive Fitness= Probability of survival to next gen Question: How do we measure fitness? –“Standard method”: Relate fitness to quality :0-1; :1-9: Chromosome Quality Fitness 1 4 3 1 1 2 1 43214321 0.4 0.3 0.2 0.1

6. Crossover Genetic design: –Identify sets of features: 2 genes: flour+sugar;1-9 Population: How many chromosomes? –1 initial, 4 max Mutation: How frequent? –1 gene randomly selected, randomly mutated Crossover: Allowed? Yes, select random mates; cross at middle Duplicates? No Survival: Standard method

7. Basic Cookie GA+Crossover Results Results are for 1000 random trials –Initial state: 1 1-1, quality 1 chromosome On average, reaches max quality (9) in 14 generations Conclusion: –Faster with crossover: combine good in each gene –Key: Global max achievable by maximizing each dimension independently - reduce dimensionality

8. Solving the Moat Problem Problem: –No single step mutation can reach optimal values using standard fitness (quality=0 => probability=0) Solution A: –Crossover can combine fit parents in EACH gene However, still slow: 155 generations on average 123454321 2 3 4 5 4 3 2 1 00000002 00000003 00787004 00898005 00787004 00000003 00000002 23454321

9. Questions How can we avoid the 0 quality problem? How can we avoid local maxima?

10. Rethinking Fitness Goal: Explicit bias to best – Remove implicit biases based on quality scale Solution: Rank method –Ignore actual quality values except for ranking Step 1: Rank candidates by quality Step 2: Probability of selecting ith candidate, given that i-1 candidate not selected, is constant p. –Step 2b: Last candidate is selected if no other has been Step 3: Select candidates using the probabilities

11. Rank Method Chromosome Quality Rank Std. Fitness Rank Fitness 1 4 1 3 1 2 5 2 7 5 4 3 2 1 0 1234512345 0.4 0.3 0.2 0.1 0.0 0.667 0.222 0.074 0.025 0.012 Results: Average over 1000 random runs on Moat problem - 75 Generations (vs 155 for standard method) No 0 probability entries: Based on rank not absolute quality

12. Diversity Diversity: –Degree to which chromosomes exhibit different genes –Rank & Standard methods look only at quality –Need diversity: escape local min, variety for crossover –“As good to be different as to be fit”

13. Rank-Space Method Combines diversity and quality in fitness Diversity measure: –Sum of inverse squared distances in genes Diversity rank: Avoids inadvertent bias Rank-space: –Sort on sum of diversity AND quality ranks –Best: lower left: high diversity & quality

14. Rank-Space Method Chromosome Q D D Rank Q Rank Comb Rank R-S Fitness 1 4 3 1 1 2 1 7 5 4 3 2 1 0 1534215342 1234512345 0.667 0.025 0.222 0.012 0.074 Diversity rank breaks ties After select others, sum distances to both Results: Average (Moat) 15 generations 0.04 0.25 0.059 0.062 0.05 1425314253 W.r.t. highest ranked 5-1

15. Genetic Algorithms Evolution mechanisms as search technique –Produce offspring with variation Mutation, Crossover –Select “fittest” to continue to next generation Fitness: Probability of survival –Standard: Quality values only –Rank: Quality rank only –Rank-space: Rank of sum of quality & diversity ranks Large population can be robust to local max

16. Machine Learning: Nearest Neighbor & Information Retrieval Search Artificial Intelligence CSPP 56553 January 28, 2004

17. Agenda Machine learning: Introduction Nearest neighbor techniques –Applications: Robotic motion, Credit rating –Information retrieval search Efficient implementations: –k-d trees, parallelism Extensions: K-nearest neighbor Limitations: –Distance, dimensions, & irrelevant attributes

18. Machine Learning Learning: Acquiring a function, based on past inputs and values, from new inputs to values. Learn concepts, classifications, values –Identify regularities in data

19. Machine Learning Examples Pronunciation: –Spelling of word => sounds Speech recognition: –Acoustic signals => sentences Robot arm manipulation: –Target => torques Credit rating: –Financial data => loan qualification

20. Machine Learning Characterization Distinctions: –Are output values known for any inputs? Supervised vs unsupervised learning –Supervised: training consists of inputs + true output value »E.g. letters+pronunciation –Unsupervised: training consists only of inputs »E.g. letters only Course studies supervised methods

21. Machine Learning Characterization Distinctions: –Are output values discrete or continuous? Discrete: “Classification” –E.g. Qualified/Unqualified for a loan application Continuous: “Regression” –E.g. Torques for robot arm motion Characteristic of task

22. Machine Learning Characterization Distinctions: –What form of function is learned? Also called “inductive bias” Graphically, decision boundary E.g. Single, linear separator –Rectangular boundaries - ID trees –Vornoi spaces…etc… + + + - - -

23. Machine Learning Functions Problem: Can the representation effectively model the class to be learned? Motivates selection of learning algorithm ++ - - - For this function, Linear discriminant is GREAT! Rectangular boundaries (e.g. ID trees) TERRIBLE! Pick the right representation!

24. Machine Learning Features Inputs: –E.g.words, acoustic measurements, financial data –Vectors of features: E.g. word: letters –‘cat’: L1=c; L2 = a; L3 = t Financial data: F1= # late payments/yr : Integer F2 = Ratio of income to expense: Real

25. Machine Learning Features Question: –Which features should be used? –How should they relate to each other? Issue 1: How do we define relation in feature space if features have different scales? –Solution: Scaling/normalization Issue 2: Which ones are important? –If differ in irrelevant feature, should ignore

26. Complexity & Generalization Goal: Predict values accurately on new inputs Problem: –Train on sample data –Can make arbitrarily complex model to fit –BUT, will probably perform badly on NEW data Strategy: –Limit complexity of model (e.g. degree of equ’n) –Split training and validation sets Hold out data to check for overfitting

27. Nearest Neighbor Memory- or case- based learning Supervised method: Training –Record labeled instances and feature-value vectors For each new, unlabeled instance –Identify “nearest” labeled instance –Assign same label Consistency heuristic: Assume that a property is the same as that of the nearest reference case.

28. Nearest Neighbor Example Problem: Robot arm motion –Difficult to model analytically Kinematic equations –Relate joint angles and manipulator positions Dynamics equations –Relate motor torques to joint angles –Difficult to achieve good results modeling robotic arms or human arm Many factors & measurements

29. Nearest Neighbor Example Solution: –Move robot arm around –Record parameters and trajectory segment Table: torques, positions,velocities, squared velocities, velocity products, accelerations –To follow a new path: Break into segments Find closest segments in table Get those torques (interpolate as necessary)

30. Nearest Neighbor Example Issue: Big table –First time with new trajectory “Closest” isn’t close Table is sparse - few entries Solution: Practice –As attempt trajectory, fill in more of table After few attempts, very close

31. Roadmap Problem: –Matching Topics and Documents Methods: –Classic: Vector Space Model Challenge I: Beyond literal matching –Expansion Strategies Challenge II: Authoritative source –Page Rank –Hubs & Authorities

32. Matching Topics and Documents Two main perspectives: –Pre-defined, fixed, finite topics: “Text Classification” –Arbitrary topics, typically defined by statement of information need (aka query) “Information Retrieval”

33. Three Steps to IR ● Three phases: – Indexing: Build collection of document representations – Query construction: ● Convert query text to vector – Retrieval: ● Compute similarity between query and doc representation ● Return closest match

34. Matching Topics and Documents Documents are “about” some topic(s) Question: Evidence of “aboutness”? –Words !! Possibly also meta-data in documents –Tags, etc Model encodes how words capture topic –E.g. “Bag of words” model, Boolean matching –What information is captured? –How is similarity computed?

35. Models for Retrieval and Classification Plethora of models are used Here: –Vector Space Model

36. Vector Space Information Retrieval Task: –Document collection –Query specifies information need: free text –Relevance judgments: 0/1 for all docs Word evidence: Bag of words –No ordering information

37. Vector Space Model Computer Tv Program Two documents: computer program, tv program Query: computer program : matches 1 st doc: exact: distance=2 vs 0 educational program: matches both equally: distance=1

38. Vector Space Model Represent documents and queries as –Vectors of term-based features Features: tied to occurrence of terms in collection –E.g. Solution 1: Binary features: t=1 if present, 0 otherwise –Similiarity: number of terms in common Dot product

39. Question What’s wrong with this?

40. Vector Space Model II Problem: Not all terms equally interesting –E.g. the vs dog vs Levow Solution: Replace binary term features with weights –Document collection: term-by-document matrix –View as vector in multidimensional space Nearby vectors are related –Normalize for vector length

41. Vector Similarity Computation Similarity = Dot product Normalization: –Normalize weights in advance –Normalize post-hoc

42. Term Weighting “Aboutness” –To what degree is this term what document is about? –Within document measure –Term frequency (tf): # occurrences of t in doc j “Specificity” –How surprised are you to see this term? –Collection frequency –Inverse document frequency (idf):

43. Term Selection & Formation Selection: –Some terms are truly useless Too frequent, no content –E.g. the, a, and,… –Stop words: ignore such terms altogether Creation: –Too many surface forms for same concepts E.g. inflections of words: verb conjugations, plural –Stem terms: treat all forms as same underlying

44. Key Issue All approaches operate on term matching –If a synonym, rather than original term, is used, approach fails Develop more robust techniques –Match “concept” rather than term Expansion approaches –Add in related terms to enhance matching Mapping techniques –Associate terms to concepts »Aspect models, stemming

45. Expansion Techniques Can apply to query or document Thesaurus expansion –Use linguistic resource – thesaurus, WordNet – to add synonyms/related terms Feedback expansion –Add terms that “should have appeared” User interaction –Direct or relevance feedback Automatic pseudo relevance feedback

46. Query Refinement Typical queries very short, ambiguous –Cat: animal/Unix command –Add more terms to disambiguate, improve Relevance feedback –Retrieve with original queries –Present results Ask user to tag relevant/non-relevant –“push” toward relevant vectors, away from nr –β+γ=1 (0.75,0.25); r: rel docs, s: non-rel docs –“Roccio” expansion formula

47. Compression Techniques Reduce surface term variation to concepts Stemming –Map inflectional variants to root E.g. see, sees, seen, saw -> see Crucial for highly inflected languages – Czech, Arabic Aspect models –Matrix representations typically very sparse –Reduce dimensionality to small # key aspects Mapping contextually similar terms together Latent semantic analysis

48. Authoritative Sources Based on vector space alone, what would you expect to get searching for “search engine”? –Would you expect to get Google?

49. Issue Text isn’t always best indicator of content Example: “search engine” –Text search -> review of search engines Term doesn’t appear on search engine pages Term probably appears on many pages that point to many search engines

50. Hubs & Authorities Not all sites are created equal –Finding “better” sites Question: What defines a good site? –Authoritative –Not just content, but connections! One that many other sites think is good Site that is pointed to by many other sites –Authority

51. Conferring Authority Authorities rarely link to each other –Competition Hubs: –Relevant sites point to prominent sites on topic Often not prominent themselves Professional or amateur Good Hubs Good Authorities

52. Computing HITS Finding Hubs and Authorities Two steps: –Sampling: Find potential authorities –Weight-propagation: Iteratively estimate best hubs and authorities

53. Sampling Identify potential hubs and authorities –Connected subsections of web Select root set with standard text query Construct base set: –All nodes pointed to by root set –All nodes that point to root set Drop within-domain links –1000-5000 pages

54. Weight-propagation Weights: –Authority weight: –Hub weight: All weights are relative Updating: Converges Pages with high x: good authorities; y: good hubs

55. Google’s PageRank Identifies authorities –Important pages are those pointed to by many other pages Better pointers, higher rank –Ranks search results –t:page pointing to A; C(t): number of outbound links d:damping measure –Actual ranking on logarithmic scale –Iterate

56. Contrasts Internal links –Large sites carry more weight If well-designed –H&A ignores site-internals Outbound links explicitly penalized Lots of tweaks….

57. Web Search Search by content –Vector space model Word-based representation “Aboutness” and “Surprise” Enhancing matches Simple learning model Search by structure –Authorities identified by link structure of web Hubs confer authority

58. Nearest Neighbor Example II Credit Rating: –Classifier: Good / Poor –Features: L = # late payments/yr; R = Income/Expenses Name L R G/P A0 1.2G B25 0.4P C5 0.7 G D 20 0.8 P E 30 0.85 P F11 1.2 G G7 1.15 G H15 0.8 P

59. Nearest Neighbor Example II Name L R G/P A0 1.2G B25 0.4P C5 0.7 G D 20 0.8 P E 30 0.85 P F11 1.2 G G7 1.15 G H15 0.8 P L R 3020 10 1 A B C D E F G H

60. Nearest Neighbor Example II L 3020 10 1 A B C D E F G H R Name L R G/P I6 1.15 J22 0.45 K 15 1.2 G IP J ?? K Distance Measure: Sqrt ((L1-L2)^2 + [sqrt(10)*(R1-R2)]^2)) - Scaled distance

61. Efficient Implementations Classification cost: –Find nearest neighbor: O(n) Compute distance between unknown and all instances Compare distances –Problematic for large data sets Alternative: –Use binary search to reduce to O(log n)

62. Efficient Implementation: K-D Trees Divide instances into sets based on features –Binary branching: E.g. > value –2^d leaves with d split path = n d= O(log n) –To split cases into sets, If there is one element in the set, stop Otherwise pick a feature to split on –Find average position of two middle objects on that dimension »Split remaining objects based on average position »Recursively split subsets

63. K-D Trees: Classification R > 0.825?L > 17.5?L > 9 ? No Yes R > 0.6?R > 0.75?R > 1.025 ?R > 1.175 ? No YesNo Yes No PoorGood Yes No Yes GoodPoor NoYes Good No Poor Yes Good

64. Efficient Implementation: Parallel Hardware Classification cost: –# distance computations Const time if O(n) processors –Cost of finding closest Compute pairwise minimum, successively O(log n) time

65. Nearest Neighbor: Issues Prediction can be expensive if many features Affected by classification, feature noise –One entry can change prediction Definition of distance metric –How to combine different features Different types, ranges of values Sensitive to feature selection

66. Nearest Neighbor Analysis Problem: –Ambiguous labeling, Training Noise Solution: –K-nearest neighbors Not just single nearest instance Compare to K nearest neighbors –Label according to majority of K What should K be? –Often 3, can train as well

67. Nearest Neighbor: Analysis Issue: –What is a good distance metric? –How should features be combined? Strategy: –(Typically weighted) Euclidean distance –Feature scaling: Normalization Good starting point: –(Feature - Feature_mean)/Feature_standard_deviation –Rescales all values - Centered on 0 with std_dev 1

68. Nearest Neighbor: Analysis Issue: –What features should we use? E.g. Credit rating: Many possible features –Tax bracket, debt burden, retirement savings, etc.. –Nearest neighbor uses ALL –Irrelevant feature(s) could mislead Fundamental problem with nearest neighbor

69. Nearest Neighbor: Advantages Fast training: –Just record feature vector - output value set Can model wide variety of functions –Complex decision boundaries –Weak inductive bias Very generally applicable

70. Summary Machine learning: –Acquire function from input features to value Based on prior training instances –Supervised vs Unsupervised learning Classification and Regression –Inductive bias: Representation of function to learn Complexity, Generalization, & Validation

71. Summary: Nearest Neighbor Nearest neighbor: –Training: record input vectors + output value –Prediction: closest training instance to new data Efficient implementations Pros: fast training, very general, little bias Cons: distance metric (scaling), sensitivity to noise & extraneous features