Algorithms

Homework None – Lectures & Homework Solutions: – Video: 046JFall-2005/VideoLectures/detail/embed03.htm 046JFall-2005/VideoLectures/detail/embed03.htm Next Week: Mid-Term – Open everything: book / laptop / notes / network

Old Business

CFGs: Context Free Grammars (Ch8)

Ambiguity

The Chomsky Hierarchy – Type 0 > Type 1 > Type 2 > Type 3 – Recursively Enumerable > CS > CF > Regular Examples – Type 3: Regular (Finite State): Grep & Regular Expressions Right-Branching: A  a A Left-Branching: B  B b – Type 2: Context-Free (CF): Center-Embedding: C  …  x C y Parenthesis Grammars:  ( ) w w R – Type 1: Context-Sensitive (CS): w w – Type 0: Recursively Enumerable – Beyond Type 0: Halting Problem

New Business

Hash Tables n steps Linear Time Constant Time n 2 time n time

Profiling

O(n) << O(n 2 ) Time n = |words|

Excel nslowfast 1, , , ,

Group Problems by Time Bounds log(n) – Binary Search, Fibonacci n – tr, cat, cut, uniq, egrep n log n – Sort n 2 – Edit Distance ( n 3 – Matrix Multiplication n – Context-Free Parsing 2 n – Factoring – Satisfiability

Summary (for Mid-Term) Symbolic Processing v. Stat Packages – Practical Symbolic: LISP, Wolfram Alpha Stats: Newton’s Method, R – Fundamental Stuff you can’t do with stat packages LISP: Recursion, Eval, Symbolic Differentiation Lambda Calculus – Small is Beautiful (beyond reason) – Church’s Thesis & Computability Theory

Summary (continued) Unix for Poets – Pipes: tr | sort | uniq –c Python & NLTK – Unix for Poets (without Unix) – Objects – Polymorphism – Equivalence Classes – Partial Orders Algorithms – Group problems by time & space bounds

Introduction to Programming Traditional (Start with Definitions) Constants: 1 Variables: x Objects: – lists, strings, arrays, matrices Expressions: x == 1 Statements: Side Effects – x = 1 (assignment) Conditionals: – If (x<=1) return 1; Iteration: for loops Functions Recursion Streams Non-Traditional (Start with Examples) Recursion def fact(x): if(x <= 1): return 1 else: return x * fact(x-1) Streams: – Unix Pipes Briefly mentioned – Everything else

Examples Factorial Fibonacci Counting Words in a Text

Python def fact(x): if(x <= 1): return 1 else: return x * fact(x-1) def fact2(x): result=1 for i in range(x): result *=(i+1); return result Recursion Iteration

Out-takes

Cosine Distance

Inner Products

Inner Product in Python def bag_of_words(words): D = {} for w in words: if(D.has_key(w)): D[w] = D[w] + 1; else: D[w]=1; return D def inner_product(D1,D2): sum = 0.0 for key in D1: if key in D2: sum += D1[key] * D2[key] return sum