Data Mining CS 341, Spring 2007 Final Project: presentation & report & codes

2 Judgment File for Testing Data is Ready to Download project/test04-complete-rel.txt

3 What’s Uue on Monday (May 7 th ) Your output files for the 25 testing queries Your output files for the 25 testing queries –by 12:00am Monday (two electronic files via ) 15-minute PowerPoint presentation 15-minute PowerPoint presentation –In class A written (typed) report A written (typed) report –In class Codes (a hard copy of the programs you wrote for this project) Codes (a hard copy of the programs you wrote for this project) –In class

4 sameple_output.txt 76 Q0 NYT Q0 NYT Q0 APW Q0 APW Q0 APW Q0 NYT Q0 APW Q0 APW Q0 APW Q0 NYT Q0 APW

5 A Written Report on the Final Project Problem Statement Problem Statement Problem Analysis Problem Analysis Techniques/Approaches (what, why and how) Techniques/Approaches (what, why and how) Implementation Implementation Experimental Results and Analysis Experimental Results and Analysis Lessons Learned Lessons Learned

6 Good Luck!

7

8 Final project Goal: Apply available data mining techniques to solve real world problem. Goal: Apply available data mining techniques to solve real world problem. Requirement: Requirement: –Apply two techniques/algorithm and implement at least one algorithm. (Find existing codes online or team up with your classmates) Problem: Relevant sentence retrieval Problem: Relevant sentence retrieval –Retrieve the set of relevant sentences given a query and a collection of documents

9 An Example Query: India and Pakistan Nuclear Tests India and Pakistan Nuclear Tests Description: Description: »On May 11 and 13, 1998 India conducted five nuclear tests; Pakistan responded by detonating six nuclear tests on May 28 and 30th. This nuclear testing was condemned by the international community. Narrative : Narrative : »Relevant documents mention the nuclear testing conducted in May 1998 by both India and Pakistan. Historical information about the antagonism and rivalry between the two countries is not relevant. Mention of the furor created around the world by these detonations is relevant.

10 An Example Document XIE XIE XIE XIE XIE XIE XIE XIE France, Canada Condemn Pakistani Nuclear Tests France, Canada Condemn Pakistani Nuclear Tests XIE XIE NEW YORK, May 28 (Xinhua) -- More countries have come out to condemn Pakistan's nuclear tests. NEW YORK, May 28 (Xinhua) -- More countries have come out to condemn Pakistan's nuclear tests.

11 An Example Document XIE XIE The French Foreign Ministry issued a communique Thursday to deplore and condemn the nuclear tests conducted by Pakistan on the same day. The French Foreign Ministry issued a communique Thursday to deplore and condemn the nuclear tests conducted by Pakistan on the same day. XIE XIE France calls on both India and Pakistan not to conduct any more nuclear tests but to sign the Comprehensive Test Ban Treaty and join talks on the banning of production of fissile materials that can be used to produce nuclear arms, said the communique. France calls on both India and Pakistan not to conduct any more nuclear tests but to sign the Comprehensive Test Ban Treaty and join talks on the banning of production of fissile materials that can be used to produce nuclear arms, said the communique.

12 An Example Document XIE XIE Canadian Foreign Affairs Minister Lloyd Axworthy said in a statement released Thursday, "We continue to urge Pakistan and India to renounce their nuclear weapons programs and to sign the Nuclear Non-Proliferation Treaty and the Comprehensive Test Ban Treaty." Canadian Foreign Affairs Minister Lloyd Axworthy said in a statement released Thursday, "We continue to urge Pakistan and India to renounce their nuclear weapons programs and to sign the Nuclear Non-Proliferation Treaty and the Comprehensive Test Ban Treaty." XIE XIE He also announced a series of sanctions against Pakistan, which he said are consistent with those imposed on India after its nuclear tests. He also announced a series of sanctions against Pakistan, which he said are consistent with those imposed on India after its nuclear tests.

13 Training and Testing Training set Training set –25 queries –Collections of documents –Relevance judgment of each sentence Testing set Testing set –Another 25 queries –Collections of documents

14 Questions to Be Considered: How do you represent queries and sentences? How do you represent queries and sentences? What features may be considered for the task? What features may be considered for the task? What data mining techniques can be used and how? What data mining techniques can be used and how? How do you evaluate the performance of your system? How do you evaluate the performance of your system?

15 Evaluation: Precision and Recall Precision and Recall The F measure The F measure –F = 2*Precision*Recall/(Precision + Recall) Average Precision for query Average Precision for query –Calculate by averaging precision as recall increases. Mean Average Precision Mean Average Precision

16 Data Mining Techniques Classification Classification Clustering Clustering Association Rules Association Rules

17 Fuzzy c-means clustering Each point has a degree of belonging to clusters, rather than belonging completely to just one cluster Each point has a degree of belonging to clusters, rather than belonging completely to just one cluster For each point x we have a coefficient giving the degree of being in the kth cluster uk(x) For each point x we have a coefficient giving the degree of being in the kth cluster uk(x)

18 Fuzzy c-means clustering The centroid of a cluster is the mean of all points, weighted by their degree of belonging to the cluster: The centroid of a cluster is the mean of all points, weighted by their degree of belonging to the cluster: The degree of belonging is related to the inverse of the distance to the cluster The degree of belonging is related to the inverse of the distance to the cluster The coefficients are normalized with a real parameter m > 1 so that their sum is 1. The coefficients are normalized with a real parameter m > 1 so that their sum is 1.

19 Fuzzy c-means clustering Choose a number of clusters. Choose a number of clusters. Assign randomly to each point coefficients for being in the clusters. Assign randomly to each point coefficients for being in the clusters. Repeat until the algorithm has converged (that is, the coefficients' change between two iterations is no more than ε, the given sensitivity threshold) : Repeat until the algorithm has converged (that is, the coefficients' change between two iterations is no more than ε, the given sensitivity threshold) : –Compute the centroid for each cluster, using the formula above. –For each point, compute its coefficients of being in the clusters, using the formula above.

20 Next Class (Wednesday April 11 th ) A 10-minute presentation on your project design, including A 10-minute presentation on your project design, including –(1) Problem statement(s) –(2) Your technical approaches (I & II) –(3) A work plan –(4) Anticipated results –(5) Evaluation metrics