1. Construction of Index: (Page 197) Objective: Given a document, find the number of occurrences of each word in the document. Example: Computer Science students know computers and computer languages. Keywords: computer, computers, science, students, know, and, languages.

2. Linear time algorithm: Let T be the text, |T| the length of T. We can find the occurrences of each word in T in O(|T|) time.

3. Constructing an automaton: onk scienc tupmoc l na egaugna edutn sr e s w d s t e

4. Remarks: There is a final state for each word. There is a counter on each final state storing the number of occurrences that the final state is reached. While reading, the algorithm creates new states for the new word. For words having met before, we just go through the corresponding states. When the final state is read, add 1 to the counter.

5. Extended Boolean Model: Disadvantages of “Boolean Model” : No term weight is used Counterexample: query q=K x AND K y. Documents containing just one term, e,g, K x is considered as irrelevant as another document containing none of these terms. No term weight is used The size of the output might be too large or too small

6. Extended Boolean Model: The Extended Boolean model was introduced in 1983 by Salton, Fox, and Wu[703] The idea is to make use of term weight as vector space model. Strategy: Combine Boolean query with vector space model. Why not just use Vector Space Model? Advantages: It is easy for user to provide query.

7. Extended Boolean Model: Each document is represented by a vector (similar to vector space model.) Remember the formula. Query is in terms of Boolean formula. How to rank the documents?

8. Fig. Extended Boolean logic considering the space composed of two terms k x and k y only. k y k x

9. Extended Boolean Model: For query q=K x or K y, (0,0) is the point we try to avoid. Thus, we can use to rank the documents The bigger the better.

10. Extended Boolean Model: For query q=K x and K y, (1,1) is the most desirable point. We use to rank the documents. The bigger the better.

11. Extend the idea to m terms q or =k 1  p k 2  p …  p K m q and =k 1  p k 2  p …  p k m

12. Properties: The p norm as defined above enjoys a couple of interesting properties as follows. First, when p=1 it can be verified that Second, when p=  it can be verified that Sim(q or,d j )=max(x i ) Sim(q and,d j )=min(x i )

13. Example: For instance, consider the query q=(k 1  k 2 )  k 3. The similarity sim(q,d j ) between a document d j and this query is then computed as Any boolean can be expressed as a numeral formula.

14. Exercise: 1. Give the numeral formula for extended Boolean model of the query q=(k1 or k2 or k3)and (not k4 or k5). (assume that there are 5 terms in total.) 2. Assume that the document is represented by the vector (0.8, 0.1, 0.0, 0.0, 1.0). What is sim(q, d) for extended Boolean model? Also try to do more exercise for other Boolean formulas.