1. Introduction to Web ScienceSlide 1 of 51 http://west.uni-koblenz.de What turns an area into a science?  Why is it „Web Science“ and not „Web practice“ what we try to learn/teach/research here?

2. Web Science & Technologies University of Koblenz ▪ Landau, Germany Web Observatory Steffen Staab

3. Introduction to Web ScienceSlide 3 of 51 http://west.uni-koblenz.de Doing Science on the Web - Methods  Observations  Web Data Snapshot –Google Cache –Your own crawl Diachronous –Internet Archive

4. Introduction to Web ScienceSlide 4 of 51 http://west.uni-koblenz.de Wayback machine

5. Introduction to Web ScienceSlide 5 of 51 http://west.uni-koblenz.de Web Graph Slides by Maren van Stehen Full slide deck and free book on „Graph theory and Complex Networks“ available from http://www.distributed-systems.net/gtcn

6. Introduction to Web ScienceSlide 6 of 51 http://west.uni-koblenz.de

7. Introduction to Web ScienceSlide 7 of 51 http://west.uni-koblenz.de

8. Introduction to Web ScienceSlide 8 of 51 http://west.uni-koblenz.de

9. Introduction to Web ScienceSlide 9 of 51 http://west.uni-koblenz.de

10. Introduction to Web ScienceSlide 10 of 51 http://west.uni-koblenz.de

11. Introduction to Web ScienceSlide 11 of 51 http://west.uni-koblenz.de

12. Introduction to Web ScienceSlide 12 of 51 http://west.uni-koblenz.de

13. Introduction to Web ScienceSlide 13 of 51 http://west.uni-koblenz.de

14. Introduction to Web ScienceSlide 14 of 51 http://west.uni-koblenz.de

15. Introduction to Web ScienceSlide 15 of 51 http://west.uni-koblenz.de

16. Introduction to Web ScienceSlide 16 of 51 http://west.uni-koblenz.de

17. Introduction to Web ScienceSlide 17 of 51 http://west.uni-koblenz.de

18. Introduction to Web ScienceSlide 18 of 51 http://west.uni-koblenz.de

19. Introduction to Web ScienceSlide 19 of 51 http://west.uni-koblenz.de

20. Introduction to Web ScienceSlide 20 of 51 http://west.uni-koblenz.de

21. Introduction to Web ScienceSlide 21 of 51 http://west.uni-koblenz.de

22. Introduction to Web ScienceSlide 22 of 51 http://west.uni-koblenz.de

23. Introduction to Web ScienceSlide 23 of 51 http://west.uni-koblenz.de

24. Introduction to Web ScienceSlide 24 of 51 http://west.uni-koblenz.de

25. Introduction to Web ScienceSlide 25 of 51 http://west.uni-koblenz.de

26. Introduction to Web ScienceSlide 26 of 51 http://west.uni-koblenz.de

27. Introduction to Web ScienceSlide 27 of 51 http://west.uni-koblenz.de

28. Introduction to Web ScienceSlide 28 of 51 http://west.uni-koblenz.de

29. Introduction to Web ScienceSlide 29 of 51 http://west.uni-koblenz.de

30. Introduction to Web ScienceSlide 30 of 51 http://west.uni-koblenz.de

31. Introduction to Web ScienceSlide 31 of 51 http://west.uni-koblenz.de

32. Introduction to Web ScienceSlide 32 of 51 http://west.uni-koblenz.de

33. Introduction to Web ScienceSlide 33 of 51 http://west.uni-koblenz.de

34. Introduction to Web ScienceSlide 34 of 51 http://west.uni-koblenz.de

35. Introduction to Web ScienceSlide 35 of 51 http://west.uni-koblenz.de Issues with crawling Issues:  duplicate pages (available under different URLs)  deep web pages  temporarily available pages  closed / semi-public pages  Restricted content  no robot  Republishing Even Facebook-shares may lead to written warnings („Abmahnung“) with fees

36. Introduction to Web ScienceSlide 36 of 51 http://west.uni-koblenz.de OTHER OBSERVATION EFFORTS

37. Introduction to Web ScienceSlide 37 of 51 http://west.uni-koblenz.de

38. Introduction to Web ScienceSlide 38 of 51 http://west.uni-koblenz.de

39. Introduction to Web ScienceSlide 39 of 51 http://west.uni-koblenz.de Linked Open Data Cloud

40. Introduction to Web ScienceSlide 40 of 51 http://west.uni-koblenz.de

41. Web Science & Technologies University of Koblenz ▪ Landau, Germany Observing Users Steffen Staab

42. Introduction to Web ScienceSlide 42 of 51 http://west.uni-koblenz.de Doing Science on the Web - Methods  Observations  Web Data Snapshot –Google Cache –Your own crawl Diachronous –Internet Archive  User Data Web Site Operator

43. Introduction to Web ScienceSlide 43 of 51 http://west.uni-koblenz.de Server Logs Apache Common Log Format  Example: 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326  127.0.0.1 (%h)  IP address of the client (remote host) which made the request to the server (may be proxy!).  - (%l) identity of client not available  frank (%u) userid of the person requesting the document as determined by HTTP authentication

44. Introduction to Web ScienceSlide 44 of 51 http://west.uni-koblenz.de Server Logs Apache Common Log Format  Example: 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326  [10/Oct/2000:13:55:36 -0700] (%t) The time that the request was received.  "GET /apache_pb.gif HTTP/1.0" (\"%r\“) The request line from the client  200 (%>s) status code that the server sends back to the client.  2326 (%b) size of the object returned to the client 

45. Introduction to Web ScienceSlide 45 of 51 http://west.uni-koblenz.de Server Logs Apache Combined Log Format  Example: 127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)“ The additional fields are:  "http://www.example.com/start.html" (\"%{Referer}i\“) The "Referer" (sic) HTTP request header.  "Mozilla/4.08 [en] (Win98; I ;Nav)" (\"%{User-agent}i\“) The User-Agent HTTP request header. Identifying information that the client browser reports about itself.

46. Introduction to Web ScienceSlide 46 of 51 http://west.uni-koblenz.de AOL Search Query Log

47. Introduction to Web ScienceSlide 47 of 51 http://west.uni-koblenz.de

48. Introduction to Web ScienceSlide 48 of 51 http://west.uni-koblenz.de AOL Query Log Mirrors

49. Introduction to Web ScienceSlide 49 of 51 http://west.uni-koblenz.de Doing Science on the Web - Methods  Observations  Web Data Snapshot –Google Cache –Your own crawl Diachronous –Internet Archive  User Data Web Site Operator User Experiments (cf. Indexing quality)

50. Introduction to Web ScienceSlide 50 of 51 http://west.uni-koblenz.de Current Research Challenge  Web Observatory  Analogy to Virtual Observatories  What should it include?

51. Web Science & Technologies University of Koblenz ▪ Landau, Germany Predicting Behaviour Steffen Staab

52. Introduction to Web ScienceSlide 52 of 51 http://west.uni-koblenz.de Example problems of predicting people behavior  Politics  Rules and laws  Social welfare  Economics  Buying behavior  Unemployment  Social science (in a more narrow sense)  birthrates