1. WEB MONITORING E6125 Web enHanced Information Management Presentation on Design of Web Monitoring applications. By Satyajeet Shaligram Columbia University MS CS Fall-08.

2. 1.Introduction 2.The web monitoring task 3.Architectural components 4.How it all works? 5.Applying machine learning 6.Difference generation 7.User notification 8.Some examples 9.Future improvements 10.Questions? WEB MONITORING You are here!!

3. What’s the task? “There could be countries out there giving away shit for free!!”

4. It’s different! RSS feeds..!! Search Engines..!! 1.User driven. 2.Involves Analytics. 3.Supports a wide range of tasks. 4.Customization! Understanding what web monitoring deals with..

5. How do I design one?? 1.Client Server model. 2.Scalable! Scalable! Scalable!! 3.Efficient & Accurate. 4.User friendly and non-obtrusive.

6. 1.Introduction 2.The web monitoring task 3.Architectural components 4.How it all works? 5.Applying machine learning 6.Difference generation 7.User notification 8.Some examples 9.Future improvements 10.Questions? WEB MONITORING You are here!!

7. Show me the design!! Change Detector - Whizbang! Labs.

8. Detailed Design! WebCQ – developed at Georgia Tech

9. 1.Introduction 2.The web monitoring task 3.Architectural components 4.How it all works? 5.Applying machine learning 6.Difference generation 7.User notification 8.Some examples 9.Future improvements 10.Questions? WEB MONITORING You are here!!

10. Machine Learning! 1.Taxonomy driven crawling 2.Text classification 3.Entity detection 4.Identifying interesting changes Building smarter systems to do the job!!

11. Difference Generation Developing algorithms with linear running time. Sub dividing the problem into many parts…

12. Content based chunking

13. Content based chunking… Smart techniques like Rabin’s fingerprinting algorithm make this possible.

14. 1.Introduction 2.The web monitoring task 3.Architectural components 4.How it all works? 5.Applying machine learning 6.Difference generation 7.User notification 8.Some examples 9.Future improvements 10.Questions? WEB MONITORING You are here!!

15. Notification Displaying relevant changes!

16. Notification…

17. Future trends… 1.Improving feeds and updates currently provided… E.g. Facebook. 2.Integrating with existing applications such as browsers 3.Developing more intuitive user interfaces to web monitors

18. Questions!? “Sometimes questions are more important than answers.” - Nancy Willard.

19. Thank you! …and have a nice day!!