Automatic Data Collection: Server Logs As with all methods, have to ask: What are the goals for your system? –What constitutes success, or good quality.

Slides:



Advertisements
Similar presentations
Web Usage Mining Web Usage Mining (Clickstream Analysis) Mark Levene (Follow the links to learn more!)
Advertisements

Statistics Review and Design Implications [TEMPLATE]
® Microsoft Office 2010 Browser and Basics.
TCP/IP Protocol Suite 1 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 22 World Wide Web and HTTP.
Digital Marketing Analytics v10. Introduction  Name / job role  What company are you with  How much experience do you have using Webtrends  Create.
Back to Table of Contents
Chapter 12: Web Usage Mining - An introduction
Introduction to Web Analytics Web analytics is the measurement, collection, analysis and reporting of internet data for purposes of understanding and optimizing.
Web Metrics October 26, 2006 Steven Schwartz President, PowerWebResults.com Southeastern Massachusetts E-Commerce Network University of Massachusetts –
SESSION 9 THE INTERNET AND THE NEW INFORMATION NEW INFORMATIONTECHNOLOGYINFRASTRUCTURE.
Application Layer  We will learn about protocols by examining popular application-level protocols  HTTP  FTP  SMTP / POP3 / IMAP  Focus on client-server.
1 The World Wide Web. 2  Web Fundamentals  Pages are defined by the Hypertext Markup Language (HTML) and contain text, graphics, audio, video and software.
Browser and Basics Tutorial 1. Learn about Web browser software and Web pages The Web is a collection of files that reside on computers, called.
Insight on Google Analytics Features - Suresh. K.
Using Website Statistics To Your Advantage 19 January 2010 Webinar Copyright Danemco LLC.
COMPUTER TERMS PART 1. COOKIE A cookie is a small amount of data generated by a website and saved by your web browser. Its purpose is to remember information.
Internet Basics.
Translating Google Analytics into Marketing Metrics
Creating a Web Site Back to Table of Contents. Creating a Web Site Conceiving a Web Site Planning a Web Site 2 Creating a Web Site Section 9-1 Section.
Evaluating Web Server Log Analysis Tools David Strom SD’98 2/13/98.
Alexander Hartmann.  Free service offered by Google that generates detailed statistics about the visitors to a website. A premium version is also available.
CIT 256 SEO and Web Commerce Dr. Beryl Hoffman. After you create a website Buy a domain name and rent web server space or go for a free one if you don’t.
WEB ANALYTICS Prof Sunil Wattal. Business questions How are people finding your website? What pages are the customers most interested in? Is your website.
Prof. Vishnuprasad Nagadevara Indian Institute of Management Bangalore
Web server and web browser It’s a take and give policy in between client and server through HTTP(Hyper Text Transport Protocol) Server takes a request.
1.Understand the decision-making process of consumer purchasing online. 2.Describe how companies are building one-to-one relationships with customers.
FALL 2012 DSCI5240 Graduate Presentation By Xxxxxxx.
Fall 2006 Davison/LinCSE 197/BIS 197: Search Engine Strategies 6-1 Module II Overview PLANNING: Things to Know BEFORE You Start… Why SEM? Goal Analysis.
Open Source Server Side Scripting ECA 236 Open Source Server Side Scripting Cookies & Sessions.
Chapter 16 The World Wide Web Chapter Goals ( ) Compare and contrast the Internet and the World Wide Web Describe general Web processing.
Lecturer: Ghadah Aldehim
14 Publishing a Web Site Section 14.1 Identify the technical needs of a Web server Evaluate Web hosts Compare and contrast internal and external Web hosting.
27.1 Chapter 27 WWW and HTTP Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Jump to first page Tracking users Analyzing how people use your site by Dylan Tweney
XHTML Introductory1 Linking and Publishing Basic Web Pages Chapter 3.
5 Chapter Five Web Servers. 5 Chapter Objectives Learn about the Microsoft Personal Web Server Software Learn how to improve Web site performance Learn.
Using audience metrics to grow revenue January 2010.
Web Metrics 1. Overview Introduction What ARE “web metrics”? Why Use Them? Server Logs Other Data Sources Wrap-up 2.
Copyright © 2002 Pearson Education, Inc. Slide 8-1.
Chapter 4 Online Consumer Behavior. Buyer Decision Making Process 4-2.
Google Confidential and Proprietary 1 Google University Google Analytics and Website Optimiser Dyana Najdi, Customer Analytics Manager, EMEA Lee Hunter,
Chapter 8 Cookies And Security JavaScript, Third Edition.
Lecture 8 – Cookies & Sessions SFDV3011 – Advanced Web Development 1.
Sustainability: Web Site Statistics Marieke Napier UKOLN University of Bath Bath, BA2 7AY UKOLN is supported by: URL
Cookies Web Browser and Server use HTTP protocol to communicate and HTTP is a stateless protocol. But for a commercial website it is required to maintain.
Log files presented to : Sir Adnan presented by: SHAH RUKH.
Chapter 12: Web Usage Mining - An introduction Chapter written by Bamshad Mobasher Many slides are from a tutorial given by B. Berendt, B. Mobasher, M.
EVALUATE YOUR SITE’S PERFORMANCE. Web site statistics Affiliate Sales Figures.
Web Metrics Terminology & Measurement. Visit A visit is a Web user with a unique address entering a Web site at some page for the first time that day.
1 UNIT 13 The World Wide Web Lecturer: Kholood Baselm.
1 Web Servers (Chapter 21 – Pages( ) Outline 21.1 Introduction 21.2 HTTP Request Types 21.3 System Architecture.
27.1 Chapter 27 WWW and HTTP Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Glossary of Terms Sessions - (old name: Visits) Users - (old name: Unique Visitors) Pageviews Pages/Session Avg. Session Duration Bounce Rate %New Sessions.
Website Design, Development and Maintenance ONLY TAKE DOWN NOTES ON INDICATED SLIDES.
ITM © Port,Kazman 1 ITM 352 Cookies. ITM © Port,Kazman 2 Problem… r How do you identify a particular user when they visit your site (or any.
Project 5: Customizing User Content Essentials for Design JavaScript Level Two Michael Brooks.
Microsoft Office 2008 for Mac – Illustrated Unit D: Getting Started with Safari.
Introduction Web analysis includes the study of users’ behavior on the web Traffic analysis – Usage analysis Behavior at particular website or across.
Session 11: Cookies, Sessions ans Security iNET Academy Open Source Web Development.
27.1 Chapter 27 WWW and HTTP Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Web Analytics and Reporting Michal Neuwirth Product Manager – Kentico Software.
Introduction. Internet Worldwide collection of computers and computer networks that link people to businesses, governmental agencies, educational institutions,
Chapter 1: Internet Marketing Foundations. Chapter Objectives Describe how computers and servers communicate to enable people to interact with webpages.
1 UNIT 13 The World Wide Web. Introduction 2 Agenda The World Wide Web Search Engines Video Streaming 3.
1 UNIT 13 The World Wide Web. Introduction 2 The World Wide Web: ▫ Commonly referred to as WWW or the Web. ▫ Is a service on the Internet. It consists.
Web Analytics Fundamentals Presented by Tejaswi, Chandrika, Sunil.
CS 115: COMPUTING FOR THE SOCIO-TECHNO WEB TECHNOLOGIES FOR PRIVATE (AND NOT-SO-PRIVATE) COMMUNICATIONS.
The need for persistence Consider these examples  Counting the number of “hits” on a website  i.e. how many times does a client load your web page source.
What is Cookie? Cookie is small information stored in text file on user’s hard drive by web server. This information is later used by web browser to retrieve.
Presentation transcript:

Automatic Data Collection: Server Logs

As with all methods, have to ask: What are the goals for your system? –What constitutes success, or good quality service? –How can you conceptualize and operationalize quality? What information can you get using this method? How will this info help you evaluate performance?

Sources of data about visits and visitors Provided by users –Registration, and whatever demographics and preferences are asked about Captured by system –Server log files –Cookies

Benefits of monitoring data Can yield lots of data for relatively low investment Unobtrusive; “outcroppings” Numbers communicate well Numbers are useful for comparisons –“hits are up 20% over this time last year”

However: what do the data mean?

One example of simple stats Compare January (with DWR photos) to March (DWR photos removed)

Common measures “According to Forrester Research, many companies still use hits as the primary measurement of website success, followed by page views and session length.”

Hit “The retrieval of any item, like a page or a graphic, from a Web server. For example, when a visitor calls up a Web page with four graphics, that's five hits, one for the page and four for the graphics. For this reason, hits often aren't a good indication of Web traffic. See page view.”

Measuring success “Companies sometimes make the mistake of buying elaborate software packages that analyze data a million ways, and then neglect to look at the most basic, day-to-day measurements of how a site is doing in its primary function…. For an e-commerce site, those basic measurements are conversion rate—that is, the ratio of buyers to visitors—and average order size. For sites that make money via advertising banners… the number of ad banners viewed; other sites can measure traffic from return visitors versus traffic from new visitors. Remember one of the most basic elements of delivering a good customer experience: making sure that pages load quickly, even when the site is barraged with traffic.”

Server logs contents Time IP Address Server Action Object Result code and size Browser version and platform Referring URL

Server log contents Time | IP Address | Server | Action | Object | Result code and size | Browser / version and platform | Referring URL 01:50: ICICWEB1 GET /images/pdq.gif Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+98) 01:50: ICICWEB1 GET /images/banner1.gif Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+98) 01:50: ICICWEB1 GET /images/news.gif Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+98)

What you can get from server logs tmlhttp:// tml ndex.phphttp:// ndex.php

Some issues in using log data Differentiating users from machines or proxies –Cookies and registration Relating IP addresses, user locations, user characteristics identifying sessions –Cookies; assumptions about nature of sessions Measuring hits –cached pages? Interpreting results relative to your goals

One source recommends: – Who is visiting your site –unique visitor identification so you know whether a visitor is returning to your site. –The path visitors take through your pages -- “visitor trails” –knowing each page a visitor viewed and the order, you can identify trends in how visitors navigation through your pages. –what element (link, icon) a visitor clicked on each page to go to the next page. – How much time visitors spend on each page –They say: “A pattern of lengthy viewing time on a page might lead you to deduce the page is very interesting or very confusing.” –But…How do you know what (else) the user is doing?

Recommendations, cont. – Where visitors are leaving your site –The last page a visitor viewed before leaving your site might be a logical place to end the visit, or it might be a place where the visitor bailed out. –The success of users’ experiences at your site –Purchases transacted, downloads completed, and information viewed are concrete indicators of tasks accomplished. From Tec-Ed, Inc., "Assessing Web Site Usability from Server Log Files" on Tec-Ed., Inc. Web site

Another example promises statistics about: Web server activity –number of visitors, the number of unique IPs, bandwidth used, number of hits they received, broken down by Time Increment, Day of the Week, and Hour of the Day Type of data visitors access on your site –Web pages viewed, files downloaded, directories accessed, images accessed during a time period. Broken down by Page Views, Browsing Sequences, Downloaded Files, Accessed Directories, Accessed Images. Referrer information –Referring Domains and Referring URLs. (Referrers are sites with links to your site. )

Promises, cont. Search engine performance –the search engines which referred visitors to the site, the phrases and keywords visitors searched for broken down by Top Search Engines, Keywords, and Each Search Engine. Visitors' geographic region –Displays a Most Active Countries graph and a table showing which Countries your visitors come from. Browsers and platforms visitors used Errors visitors encountered at the site

Promises, cont. Advanced visitor filters –Visitors who accessed specific pages or files. –Visitors who came from specific referring URLs. –Day of Week (Example: see what happened on a specific day); Hour of Day. – Visitors whose first visit is a specific page. –Visitors' countries or regions. –Visitors who make purchases on your web site: see information on visitors who actually buy something from your web site. Source:

cookies Simulate continuous connection, session Identify user Store info about user, preferences, past activity

Cookies “the server nytimes.com wishes to set a cookie that will be sent to any server in the domain nytimes.comnytimes.com The name and value of the cookie are nytime-s … The cookie will persist until Tues April 8 14:25: ”

Set-Cookie: NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME; secure NAME=VALUE : a sequence of characters. The only required attribute. expires=DATE : valid life time of that cookie. Once reached, cookie no longer stored or given out. domain=DOMAIN_NAME : When searching the cookie list for valid cookies, domain attributes of the cookie are compared with domain name of host from which URL will be fetched. Default is the host name of the server which generated the cookie response. path=PATH; the subset of URLs in a domain for which the cookie is valid. If not specified, is assumed to be the same as the document described by the header which contains the cookie. Secure: Cookie will only be transmitted if the communications channel with the host is a secure one.

Other methods Analyses of queries on site search engines s: –Customer queries and requests for more information –Customer complaints Suggestion boxes

Analyses Frequencies Cross tabulations –Page visited by IP address Correlations –Beware of assumptions about causality Graphics Exponential distributions