Section 2: Longitudinal study samples From the CLOSER Learning Hub Module: Study Design

Defining the target population Geographic boundaries: identify a population within certain geographic limits (e.g. city, county, country) Targeting specific groups: identify a population that shares a common experience (e.g. birth year, disease, school year) Targeting the whole population: no limits, challenge is that population changes and sample must be dynamic The first thing study teams need to decide is who the study will focus on. Geographic boundaries Most studies select their sample from within certain geographic limits. This might be for practical or scientific reasons. The geographic limits could be very small, for instance a city or county, or very large, such as the whole of the UK. Targeting specific groups The first two examples are known as cohort studies and target specific groups or sections of the population. Cohort study samples share a common experience at a particular point in time. For example, a birth cohort follows children born within a specific period. Other cohorts follow groups of students in the same year at school, patients diagnosed with a certain disease at a particular point in time, or new recruits entering an organisation or industry in a given year. Targeting the whole population Some studies, like Understanding Society, target the UK population as a whole. One challenge this presents is the fact that the population is always changing. Studies that seek to represent the whole population must be ‘dynamic’ – that is, there needs to be a way in which new members can join the sample. Otherwise there is a risk that, over time, the sample will become increasingly different to the population it is meant to represent. Understanding Society creates a dynamic sample by including people who move into participating households. For example, if the child of a participating household goes away to university, his or her new flatmates will join the sample. Similarly, if a couple breaks up and forms two new households, both new households become part of the sample.

Sampling frames Sampling frame: a list of everyone in the target population of interest, from which a sample can be drawn Evaluating a potential sampling frame: Does it include people who are not in the target population? Is it missing people who are in the target population? In order to select a sample, researchers need a ‘sampling frame’. This is a list of everyone in the target population of interest, from which a sample can be drawn. The choice of sampling frame depends on who the study wants to sample and when they would like to first interview them. In some cases, no appropriate sampling frame exists. When assessing the sampling frame used for a study, it is important to consider how accurately the frame reflects the target population of interest. For example, does it include people who are not in the target population at all (and who need to be identified and weeded out)? Or is it missing people who are in the target population?

Example: Child Benefit Records Sampling frame for the Millennium Cohort Study In 2000-01, Child Benefit was universal – records were accurate reflection of all UK families with a child born in the study’s target year Today benefit is no longer universal, meaning records no longer useful sampling frame Example: Child Benefit Records Child Benefit Records were used as the sampling frame for the Millennium Cohort Study. At the time, Child Benefit was universal, which meant that the list of recipients in 2000-01 (when the study started) was an accurate reflection of all UK families with a child born in the study’s target year. However, Child Benefit Records are no longer a suitable sampling frame for birth cohort studies as the benefit is no longer universal. Changes made in 2013 mean that the records under-represent higher earners, who are no longer entitled to Child Benefit. If a study were to use the current Child Benefit Records as a sampling frame today, the sample would under-represent higher income households.

Sample size Likely achieved sample size: how many participants are likely to take part at any given sweep Response rate: the number of people who complete the survey divided by the number of people who were invited to take part Attrition: participants dropping out of the study, either permanently or temporarily Every time a new round of data collection is planned, longitudinal study teams need to consider what the likely achieved sample size will be – that is, how many participants are likely to take part. Cross-sectional study teams will identify the ideal achieved sample size, as well as the likely response rate – that is, the number of people who complete the survey divided by the number of people who were invited to take part. Study teams usually issue a sample that is larger than their ideal achieved sample size to take into account that response rates are almost never 100%. With longitudinal studies, these calculations are more complex. The study teams need to think about the sample over a longer time period, collecting data a number of times. An important consideration for longitudinal study teams is attrition – that is, participants dropping out of the study, either permanently or temporarily. Some attrition is unavoidable (for example, participants might die or leave the country). Other attrition is avoidable but challenging to overcome (for example, keeping in touch with participants who move or persuading reluctant participants to take part). The sample design for a longitudinal study will involve making judgements about the starting sample size needed to ensure that the study can withstand likely attrition levels over time.

Sample selection and representivity In many cases, the list of people in the sampling frame is too long and a subset must be selected Different methods used to ensure that any subset remains representative of wider population Opinions differ across disciplines on value of representative sample In the case of most longitudinal studies, the list provided by the sampling frame is too long and a smaller subsample needs to be selected. Study teams use various methods to make sure that this subsample is as representative of the target population as possible. These sampling methods have become more sophisticated over time as the science evolves. For example, the first three British birth cohorts selected their sample of births by choosing a specific week within the relevant year (1946 , 1958 and 1970). All births within those weeks were eligible to be included in the studies. There were several limitations to this approach. For instance, selecting births in a single week creates a sample where all participants are born within the same season. The sample is, in other words, potentially not representative of everyone born in that year – only of those born in that season. This makes it is impossible to use the data to explore issues like whether season of birth affects later outcomes, such as educational attainment. This is one of the reasons that the most recent birth cohort, the Millennium Cohort Study, selected its sample of births from across a whole school year. This allows researchers to be confident that the data collected can be used to make inferences about the wider population born at the turn of the century. However, it is important to be aware that there is a debate within epidemiology about whether the importance of having representative samples drawn from well-defined populations has been overrated. Instead, it is argued, some research questions are better addressed by sample designs that focus upon particular groups of interest rather by seeking to obtain a representative sample of the relevant population as a whole.

Boosted samples Some studies ‘boost’ numbers for minority groups so there are sufficient numbers for analysis Sample weighting: counting individuals as more/less than one case Weights can be applied to adjust results so they are representative As covered in the previous sections, most longitudinal study teams aim to select representative samples that reflect the composition of the target population. However, unless the starting sample is very large indeed, this means that there will be relatively small numbers of participants from minority groups. While the proportions of participants from minority groups might accurately reflect the make-up of the wider population, the small numbers can constrain the research that can be done using these groups. For example, imagine a particular group represents 2 per cent of the UK population as a whole. If a longitudinal study achieves 8,000 interviews in its first sweep of data collection, it will include around 160 participants from the minority group – too small for any detailed statistical analysis, especially if some of these participants drop out at subsequent sweeps. As a result, some studies now ‘boost’ the number of participants from particular. If a study contains a boosted number of participants from a particular group, survey weights should be applied to adjust the overall results so that they are representative of the population as a whole. Sample weighting involves some individuals counting as less than one case, while others may count for more.

Boosted samples: examples Understanding Society: boosted samples of Indian, Pakistani, Bangladeshi, Caribbean and African households, currently carrying out new immigrant and ethnic minority boost Millennium Cohort Study: boosted samples from Scotland, Wales and Northern Ireland, and deprived and BAME areas of England Examples of longitudinal studies that have boosted samples include: Understanding Society included boosted samples of Indian, Pakistani, Bangladeshi, Caribbean and African households from its outset in 2009. It did this by oversampling areas within the UK with a high density of ethnic minority groups and selecting households from those areas which had at least one adult from one of the ethnic minorities of interest. The study is currently carrying out a new immigrant and ethnic minority boost. The Millennium Cohort Study boosted the number of children from the smaller UK countries (Scotland, Wales and Northern Ireland), and from areas of England that were deprived or had high concentrations of black and Asian families.