Measurement, Data and Information for Residential Aged Care

Overview Section One: An introduction to measurement, data and information Measurement, data and information defined Why measure? Some background to measurement practices

Measurement, Data and Information Defined to ascertain the extent, dimensions, quality of something Data: known or available facts or figures Information: knowledge communicated or received concerning some fact or circumstance

Turning Data Into Useful Information and Knowledge

Why Measure? Benefits derived from collecting, analysing and reporting data and information are: Provides the home with the ability to demonstrate its current level of performance to all stakeholders and against stakeholder needs Enables the home to gather information about the needs of stakeholders in an ongoing way Provides information for strategic and improvement planning Enables the home to monitor how well its plans are being implemented

The Link between Data and Information, Self-Assessment and Continuous Improvement As a regular and cyclical process, rigorous measurement and analysis of results… Enables the staff and management of the home to understand their current performance for example through results of resident satisfaction, resident outcomes, clinical indicators, audit results, other survey results Ensures staff and management of the home have up to date knowledge of their current level of achievement against the Accreditation Standards As part of self-assessment or routine performance review assists the home to identify areas where improvements could be made

The Link between Data and Information, Self-Assessment and Continuous Improvement Is an input into planning processes enabling decisions abut improvement, priorities, goals and resourcing Also informs the home’s management of the results achieved through improvement projects previously implemented Ensures that staff and management of the home have up to date evidence available to support their accreditation application and evidence provided during a support contact

Data and Information As Part of Self Assessment

Background to Measurement Practices The CSIRO found the following problems with collection and use of information. This package aims to touch on these problems. Little relationship existed between the strategic intents of the organisation and what was actually measured and reported Information usually focused only on financial performance Information did not support assessment and management of performance Information was largely operational Ambiguity about what was being measured Lack of capability to provide the appropriate analysis Lack of understanding of variation and statistics

Overview Section Two: Deciding what to measure Levels of measurement Start the thinking about measurement with what the home strives to achieve Process drive achievement – understanding processes leads to other appropriate measures Measurement and the expected outcomes of the Accreditation Standards Where measurement associated with improvements fits in

Levels of Measurement It is important to be clear about what measures are important and for what reason. For instance: How the the home performs overall in its major areas of care and service Enable staff to manage a particular job or process such as care planning or the catering services To track the success of a quality improvement project

Levels of Measurement The following steps simplify the approach to measurement: Define what is important to the residents, other stakeholders and the home Identify what the residential aged care home wants to achieve Look at the major processes of the home and how they are undertaken Decide what information will assist staff and managers to assess performance at the various levels

Measurement – What the Home Strives to Achieve It is important to begin with a review of what overall achievements the home aims for. Information about critical areas for achievement could be found through: Review of the home’s strategic plans and major objectives Review of the mission and vision statements Workshops and discussion between the managers and the board

Resident Focus Resident satisfaction and the satisfaction of other stakeholders are generally important high level measures for a home. Good management practice also focuses on the needs, expectations and satisfaction of other key stakeholders such as: Family members, carers and/or resident representatives Staff Visiting professionals The owners of the home Local and professional communities Other agencies

Understanding Processes, Outputs and Outcomes Results are what we aim for… Systems and processes help us achieve them!

When a Problem Arises

Measurement from a Process Perspective There are three sorts of measures that may be relevant to the residential aged care home’s major processes: Measurement of outcome Measurement of output Measurement within the process itself

Measurement of Outcome Outcome measures provide answers to the following probing questions… “You did this…and so…?” “And the result of this effort was…?” “Did this benefit the residents?…the staff?” They provide information about the ultimate results achieved.

Measurement Around Output Output measures are a step removed from the measures of outcome and can sometimes be seen as predictors of outcome. That is if results of outputs measures are positive the results of the measurement of outcome are also likely to be positive.

Measurement Inside the Processes Examples of commonly used in-process measures include: Staff compliance rates (to procedure and/or protocols) for example: - compliance to residents’ needs assessment procedures - compliance to medication procedures - compliance to cooking procedures and times - compliance to occupational health and safety procedures such as hand washing and universal precautions

Measurement and the Expected Outcomes of the Accreditation Standards The issue of effectiveness is the link between processes and outcome measures. Where the home has robust information around outcomes, effectiveness may be relatively easy to assess. Without this data, the home will need other ways of demonstrating that care and services are effective. R

Where Measurement Associated With Improvement Activities Fits In Measurement can be used at various levels and to suit various purposes: Routine high level measures of performance for management and staff Routine measurement to enable the management of processes Targeted measurement for quality improvement projects

Where Measurement Associated With Improvement Activities Fits In Each set of measures suits a particular purpose and as such: The various sets of measures should be considered differently and appropriately A comprehensive measurement system would most likely include measures to enable review of each of the three aspects of management described above It is important to distinguish between the measures being used

Measurement and Quality Improvement The development of appropriate measures and target for each quality improvement project enables the home to readily assess whether its improvement efforts are successful and, if measured over a defined period, will also show whether improvements have been sustained. Without such measurement and analysis it is difficult for a home to demonstrate the effectiveness of its improvements and the benefits achieved for residents and others.

Overview Section Three: Considerations in data collection Aspects of effective data management Aspects of effective data collection Sampling Types of data and data collection tools Planning for routine data collection

Aspects of Effective Data Management The appropriate collection of data, covered in this section, is one of the important aspects of a data management system that ensures that the data and information does in fact provide value to the users. Effective data management includes the presentation, analysis and interpretation of the data and information and these are covered in the next section

Aspects of Effective Data Collection Major aspects of data collection include: Clarity about the relevance and importance of the measure The definition of how the measure will be made or taken Clarity about who will collect the measure, where and when Definition of how the results will be reported – in what format and to whom Clarity about who has responsibility to respond to the results of the measurement

The Relevance and Importance of the Measure If the purpose of any measure is not clear, or it is obvious that the results are difficult to understand or act on…why measure it? Try asking: What major area of care and/or service delivery does this measure relate to? Is it clear how it relates? Will it effectively tell us about the area of care and/or service delivery we are interested in? Are there other measures that would provide more accurate information about the area under study? Are there additional measures that would provide more complete information about the area under study?

Definition About How the Measure Will be Made or Taken The definition could include details of: Any tools required for the measurement The appropriate level of detail required in the measurement Instructions about timing Any associated information that should be collected at the same time How to record the measurement How to store the measurement once recorded

Who Will Perform the Measurement? Some measurements could be taken easily by any staff member and others may require specialised skills and knowledge. The detail can include: The designated staff responsible for collecting the measurement Where they will source the measurement Detail around the choice of subject Detail around timing and sampling

Reporting of the Measurement Clarity about reporting could include details of: How often the results are to be reported Who they are reported to such as the department manager, the quality committee, the director of nursing, the management committee, or other forums In what format they will be reported for instance as: - raw data - averages, ranges - in tables - in run charts - other formats as required

Identifying who has Responsibility to Respond to the Results The person responsible will vary according to the particular measures being evaluated. For instance: The Chief Executive Officer, Director of Nursing, Care Manager or other senior member of staff are likely to be responsible for review of, and response to, the residential aged care home’s outcome measures and other high level data Service managers such as unit managers, catering managers and a support services manager may be responsible for the review of, and response to, process measures used to monitor their individual activities

Sampling There are several methods of sampling, four of which are described: Incidental Sampling The person conducting the measurement chooses whoever they wish. It may result in bias. Though easy, it is not recommended. Stratified Sampling Involves identifying sub-groups within the population and choosing an appropriate number from each category.

Sampling Random Sampling Random sampling minimises the introduction of bias and provides a sound basis for generalising the results found to the whole population. It is considered one of the best types of sampling. Systematic Sampling Systematic sampling involves choosing a subject at regular intervals from the whole population of subjects for instance every fifth or tenth subject from a list. It is not truly random but approaches this.

What is a Good Approach to Sampling? To decide the best overall approach for any particular study, audit or other assessment consider: Are there particular sub-groups that are important to this measurement? If so, note this. Attempt to choose the subjects (residents, staff, files) as randomly as possible, choosing numbers from any relevant sub-groups proportionally. If sub-groups are not important in a particular study, attempt to choose files, residents or other subjects at random.

Sample Size Attempt to review approximately 10-20% of cases for any audit, review or assessment. Be flexible in the approach. For instance if the audit or review shows unusual findings or mixed results, you may want to sample more subjects to develop a clearer view and understanding of the findings. When sampling from a sub-group attempt wherever possible to have a minimum number (say five) in each individual sub-group. An inclusive approach where all subjects are in the study, is entirely appropriate where the home has considered the importance of the assessment and plans the appropriate resources for the data collection.

Types of Data and Data Collection Tools Quantitative Data Can be counted. The results can be described and analysed numerically. Examples of data collection tools to gather quantitative data include audits, check sheets, tick sheets, count sheets. Qualitative Data Is generally concerned with individuals and their opinions, thoughts, feelings, experiences and other feedback. Examples of qualitative data collection tools are surveys, questionnaires, one to one interviews and focus groups.

Audits An audit can be defined as: “a planned, independent, and documented assessment to determine whether agreed-upon requirements are being met.” Arter, D.R, 94 Some considerations before developing any audit: Be clear about the objectives of the audit Ensure that this element of care and/or service is important and warrants its own auditing approach – never audit for the sake of auditing Be clear about exactly what you will do with the results

Collecting Data on Opinions Steps in developing a questionnaire Define what it is you want to ask about Draft the tool (such as the survey or questionnaire, interview process) Test the tool Refine the tool based on the findings from the test Conduct the process and review It is clear that this process follows the simple Plan-Do-Check-Act cycle common in any quality improvement exercise.

Planning for Routine Data Collection Most residential aged care homes collect a range of data using a range of tools. It is a useful exercise to list the major data collection activities and schedule them. Preparing such a schedule provides a number of advantages such as: Communication to staff, managers and others about the range of measurement activities Providing a timetable for staff and managers for their particular audits and activities Providing a platform for review of the measurement of activities

Reviewing the Data that is Collected and Reported Some useful questions to review measurement activities. For each audit, survey or or other data collection exercise ask: How does this measurement activity fit with the information needs of the residential aged care home? How important is it? How well does the data and information meet the needs of the user? What do the results show?

Overview Section Four: Presentation and analysis of data Aspects of effective data management Effective presentation and analysis of data Presentation and analysis of quantitative data Case Study and exercise

Aspects of Effective Data Management This section addresses the final cornerstone of an effective data management system – the way that data are presented and analysed. Turning data into useful information and knowledge.

Effective Presentation and Analysis of Data The choice of the format for presentation of data relies on two important considerations: the type of data and the audience. Quantitative Tables Bar charts & histograms Run charts Control charts Pie charts Pareto charts Scatter diagrams Qualitative Flowcharts Cause and effect diagrams

Effective Presentation and Analysis of Data Whichever format is chosen it is important that data and information are presented as clearly and accurately as possible with attention to standard considerations for data presentation such as: Clear title for the data and information set Clear dates and other identifying information The origin of the data clearly identified on the data and information set (for instance “this data was collected from the north wing of XYZ Nursing Home”) Details of who collected the data Consideration to what features are required to enable the user to appropriately analyse the information.

Presentation and Analysis of Quantitative Data Single data points for results (such as a solitary figure in a table in a monthly report) rarely provide meaningful information to the user. The ability to analyse results is strengthened when comparative information is also provided along with the current results. Whatever the presentation format chosen, the chart or table should be clean and uncluttered. The baseline against which to compare current results (say the median and range) can be calculated from the previous 12-18 months data if these results appear to have been relatively stable.

Presentation and Analysis of Quantitative Data If significant improvements are achieved over time, and sustained, then the baseline will need to be recalculated to accurately represent the current system and its results. A new standard has been set. If comparing your residential aged care home’s results with those of others ensure that the details of data collection and presentation are the same. This simple point lies at the root of much dissatisfaction with many data comparison programs. It may be more valuable to focus on refining and developing the home’s own internal measurement system to best meet its own needs before embarking on external comparative programs.

Analysis of Qualitative Data Important Note! Where results for a particular month are in line with what has been previously achieved, this indicates the home’s performance is stable. That is, with all else being equal and nothing changing, the results can be expected to continue. It is quite another question whether the results are acceptable. The use of an acceptable range or a target strengthens the ability to determine if results achieved are acceptable.

Presentation and Analysis of Qualitative Data Qualitative data includes such things as staff ideas, resident ideas and suggestions, and the process knowledge and understanding of staff and managers. Presentation formats that are useful for qualitative data include: Flow charts Cause and effect diagrams Note: whilst surveys and questionnaires collect qualitative data (such as opinions and perception) the analysis of survey and questionnaire results commonly involves giving some numeric number to the responses. As such, numerical results for survey responses are presented and analysed as for quantitative data.