NHS DIRECT MULTI-CHANNEL SERVICE Dr Shirley Large and Kate Arnold

NHS DIRECT MULTICHANNEL SERVICE

Telephone service – over 650,000 calls per month NHS Direct Online – 1 million visits per month Self-help guide in Thomson Local directory – 18 million homes NHS Direct Interactive – 5% of digital satellite viewers Evaluation – effort focussed on getting services up and running, evaluation relatively new – developing evaluation strategy and this is one example of our work – web log analysis

Web log analysis: how users interact with NHS Direct Online Server web logs provide a direct record of the health information searching behaviour of visitors to the website, thus providing a huge resource of rich information ready to be excavated and analysed. However, despite the potential promise of this rich data source current approaches to web log analysis identified in the literature describe a number of limitations. The traditional approach to analyse web log is to undertake summary analysis of log files such as the total number of pages viewed rather than how, or which particular aspects of a health topic is viewed. This study adopts a new approach to viewing and analysing the web logs which reduces some of the previously associated limitations. One health topic area is analysed in-depth to understand the public interest in the topic and to understand which particular aspects of the topic area are viewed. Understanding the navigational paths through a particular topic area has beneficial implications for the system design and the information architecture of the website.

  Objectives Develop an approach to web log analysis to enable the NHS Direct New Media web logs to be analysed in a meaningful and informative way Establish an approach to extracting, truncating and analysing the data from NHS Direct New Media web logs Develop an awareness of the navigation and health information seeking behaviour of users accessing MRSA information web pages on the NHS Direct New Media website.

Death certificates showing resistant and non-resistant MRSA 1993-2003 the number of death certificates mentioning Staphylococcus aureus (S.aureus) infection increased each year from 1993 to 2003 in England and Wales. The rise in the number of death certificates specifying MRSA accounted for the increased number. Most of the deaths involving S.aureus or MRSA were in the older age groups (http://www.statistics.gov.uk).

1 Day’s server activity Even exploring just one day’s activity from three servers for the website the size of each server comprised large datasets as outlined below:   Server 1 = 40.2 MB Server 2 = 41.0 MB Server 3 = 40.3 MB

Global Metrics 32,728 visits to the website on 4th October 2004 MRSA was the 15th most commonly viewed topic on 4th October 2004. 242 visits to the MRSA topic

Of the 557 MRSA pages viewed the introductory page was viewed most frequently (n=260; 46.68%), followed by the treatment page (n=73; 13.11%), symptoms (n=62; 11.13%), prevention (n=57; 10.23%), causes (n=54; 9.69%) and the link to an external health information page (n=40; 7.18%). The print or PDF page was viewed 11 times (1.97%), this indicates that all pages of MRSA content was viewed, saved or printed on these 11 occasions.

Greatest user volume was between 7pm and 10pm Greatest user volume was between 7pm and 10pm. The second peak in page viewing activity was from 5pm to 6pm.

→ Causes → Treatment → Prevention → Symptoms 2 5 page path ways Visitors Introduction → Causes → Treatment → Prevention → Symptoms 2 → Causes → Treatment → Symptoms → External link 1 → Causes → Symptoms → Treatment → External link → Causes → Introduction → Prevention → Introduction → Symptoms → Introduction → Treatment → Treatment Introduction → Prevention → Causes → Symptoms → Treatment 1 Introduction → Treatment → Prevention → Symptoms → External link Introduction → Treatment → Symptoms → Causes → Introduction Symptoms → Treatment → Treatment → Treatment → Treatment Symptoms → External link → Introduction → Prevention → Treatment Total 11

→ Introduction → Causes → Treatment ? Symptoms → 6 page pathways Visitors Introduction → Cause → Treatment → Prevention → Symptoms → External link 3 Introduction → Introduction → Causes → Treatment ? Symptoms → 1 Introduction → Cause → Treatment → Symptoms → Prevention → 1 External link Introduction → Treatment → Prevention → Cause → Symptoms → 1 External link Introduction → Causes → Introduction → Introduction → Causes → Causes 1 Introduction → Causes → Symptoms → Treatment → External link → Prevention 1 Symptoms → Treatment → Introduction → Cause → External link → External link 1 Symptoms → Treatment → Treatment → Symptoms → Symptoms → Treatment External link → Introduction → Prevention → Causes → Symptoms → Treatment Introduction → Prevention → Causes → Symptoms → Treatment → 1 External link External link → Symptoms → Prevention → Treatment → Causes → Introduction 1 Total 13

Highest: 1st and 2nd page 3.45 mins 15th and 16th page 4.38 mins Mean time on the website 6 mins 54 secs compared to WebTrends 7 mins 23 secs

Findings The main findings of this study show that MRSA was the 15th most viewed health topic on 4th October 2004, which reflects the public and health care concern. The navigational path through the sections of the MRSA topic did not mimic the classic textbook presentation; introduction, causes, symptoms, treatment and prevention. Although in a few instances there appeared to be good coverage of the topic, page views did not follow any discernibly logical pattern. More research is required to understand; if the same for different topics, if the way information is viewed is a phenomenon of Internet health information searching or if reveals a problem with layout and navigational links on the webpage. The PDF or print page was viewed nine times during the day, which indicates that all the topic information was accessed, possibly to be viewed at the visitors leisure or with friends/family/colleagues when offline.

Conclusions Understanding the interest in and the navigational pathways through the health topics on the website has implications for both the system design and the information architecture of the website and consequently may improve the efficiency and effectiveness of the website. Analysing and exploring the patterns in the visitor’s behaviour when searching for health information on the website will contribute of the understanding of the quality and delivery of information services. Web log analysis as just one component of a larger assessment of the activity for evaluating NHS Direct New Media services. Other complimentary forms of user analysis should be considered for evaluating the use of the Internet health information such as an online questionnaire, face-to-face interviews, direct observation of health information seeking behaviour or focus groups. These methods will allow further exploration of some of the questions highlighted in this study. Understanding the interest in and the navigational pathways through the health topics on the website has implications for both the system design and the information architecture of the website and consequently may improve the efficiency and effectiveness of the website. The findings of this study show interesting health information seeking behaviour and identify questions about user navigation. Analysing and exploring the patterns in the visitor’s behaviour when searching for health information on the website will contribute of the understanding of the quality and delivery of information services. Therefore, the findings of this study potentially have a practical application in respect of providing information for an impending redesign of the NHS Direct website. However, it is important to interpret and consider web log analysis as just one component of a larger assessment of the activity for evaluating NHS Direct New Media services. Other complimentary forms of user analysis should be considered for evaluating the use of the Internet health information such as an online questionnaire, face-to-face interviews, direct observation of health information seeking behaviour or focus groups. These methods will allow further exploration of some of the questions highlighted in this study.   An important aspect of planning and maintaining a Website is to identify who the likely users are and to review their needs. This is particularly important when the content of the website provides an important public health service. Understanding the interest in and the navigational pathways through the health topics on the website has implications for both the system design and the information architecture of the website and consequently may improve the efficiency and effectiveness of the website. The findings of this study show interesting health information seeking behaviour and identify questions about user navigation. Analysing and exploring the patterns in the visitor’s behaviour when searching for health information on the website will contribute of the understanding of the quality and delivery of information services. Therefore, the findings of this study potentially have a practical application in respect of providing information for an impending redesign of the NHS Direct website. However, it is important to interpret and consider web log analysis as just one component of a larger assessment of the activity for evaluating NHS Direct New Media services. Other complimentary forms of user analysis should be considered for evaluating the use of the Internet health information such as an online questionnaire, face-to-face interviews, direct observation of health information seeking behaviour or focus groups. These methods will allow further exploration of some of the questions highlighted in this study.