Common quantitative methods

Chapter

9

Common quantitative methods Linda Shields and Roger Watson

KEY TERMS

LEARNING OUTCOMES

control experimental designs explanatory (independent) variable manipulation observational designs outcome (dependent) variable quasi-experimental designs randomisation

After reading this chapter, you should be able to: • describe the overall purpose of quantitative designs • distinguish the differences between experimental, quasi-experimental and non-experimental designs • list the criteria necessary for inferring cause-and-effect relationships • apply critical review criteria to evaluate the findings of selected quantitative studies.

Schneider_1374_Chapter 9_main.indd 161

7/25/2012 6:11:08 PM

NURSING AND MIDWIFERY RESEARCH 4E

INTRODUCTION This chapter provides an overview of the meaning, purpose and issues related to quantitative research designs, and presents the common approaches used to answer a variety of nursing and midwifery questions. Related issues such as sampling, data collection, assessment of measurement instruments and data analysis are discussed in Chapters 10–13. The focus is on providing research consumers with the information to evaluate quantitative studies critically. Quantitative research refers to studies where the variables of interest are measurable and the results are quantifiable and coded as numerical data. Figure 9.1 (Bryman & Cramer 2005) shows one way of looking at quantitative research and shows how it can be divided into two branches: survey (also called observational and correlational) and experimental. Both branches include a range of designs but both are capable of testing theory by asking research questions through the specific identification of independent and dependent variables, the formulation of hypotheses and their testing using statistical methods. However, these two branches are not equally capable of discerning cause and effect between variables and there is, essentially, a hierarchy in the ability to do this as illustrated in Table 9.1. There are three major categories of designs along the quantitative continuum: observational, quasi-experimental and experimental. Each category includes a range of designs. Choice of a design relates to the research question or hypothesis, the amount of control a researcher can have and study feasibility. Much of what we know as quantitative research methods have been designed by the discipline of epidemiology. Interestingly, one of the first epidemiologists was Florence Nightingale. Statistical tests which she developed to produce evidence about outcomes for wounded and sick soldiers at the Crimea were recognised at the International Congress of Statistics in 1860 (McDonald 2001). Modern epidemiology (from Greek, meaning ‘the study of people’) is defined by the Australasian Epidemiological Association (2010) as the study of diseases in populations. Epidemiology has three main aims: to describe disease patterns in human populations; to identify the causes of diseases (also known as aetiology); and to provide data essential for the management, evaluation and planning of services for the prevention, control and treatment of disease. There are many excellent books on epidemiology written expressly for nurses and midwives; a recent example is Epidemiology for Advanced Nursing Practice by Kiran Macha and John McDonough, published in 2011 by Joyce and Bartlett Learning in the USA. Any midwife or nurse undertaking quantitative research is well advised to learn the basics of epidemiology.

CONCEPTS UNDERPINNING QUANTITATIVE RESEARCH The empirical paradigm that informs quantitative research designs emphasises ‘objective’ observation, accuracy and control — elements originating from scientific investigations in laboratory settings. Medicine in particular, but also other disciplines such as nursing and midwifery, have followed this tradition when undertaking quantitative research. Figure 9.2

illustrates the focus on variables and relationships for each type of quantitative design. The term ‘design’ implies the organisation of research components into a coherent and systematic plan, and represents the major distinctive approach chosen for the specific purpose of answering an explicit research question. The question or aims and objectives influence and guide the choice of design. To make an informed choice about which design will best answer the research question, and for

162


7/25/2012 6:11:08 PM

9 • Common quantitative methods Theory Hypothesis Operationalisation of concepts Selection of respondents or participants Survey/correlational design

Experimental design

Conduct interviews or administer questionnaires

Create experimental and control groups Carry out observations and/or administer tests or questionnaires Collect data Analyse data Findings

Figure 9.1 The research process (From Bryman A and Cramer D 2005. Quantitative data analysis with SPSS 12 and 13: a guide for social scientists. Taylor & Francis, London.)

TABLE 9.1 Continuum of quantitative research designs DESIGN INCREASING CONTROL AND ABILITY TO ASSIGN CAUSALITY Observational

SUB-TYPES Descriptive Correlational Cross-sectional Retrospective Case-control Cohort Longitudinal

FEATURES Describes variables Examines relationships between variables Examines variables and relationships at one time point Re-traces participants with an outcome backwards to a possible exposure Cases matched to controls Follows participants from exposure to outcome Repeated measurements of participants over time

Quasi-experimental

Time-series Non-equivalent control group

Manipulation (intervention) Manipulation + Control

Experimental

After-only experimental Randomised controlled trial Cluster-randomised controlled trial

No measurement prior to intervention Control & Manipulation + Random assignment to groups Groups of participants (not individuals) are randomised 163


7/25/2012 6:11:08 PM

NURSING AND MIDWIFERY RESEARCH 4E Study design

Variables and relationship of interest

Descriptive

Variable 1

Variable 2

Identifies variable/s; not able to test relationships

less control over these elements, particularly three major concepts: 1) control 2) randomisation 3) manipulation. The strongest designs contain elements of all three of the above aspects.

Control

Correlational

Variable 1

Variable 2

Direct relationship between identified variables; direction un-tested Cohort/interventional

Variable 1

Variable 2

Direct ‘cause-and-effect’ relationship tested

Figure 9.2 Examining variables in quantitative designs

a consumer to understand the implications and the use of research, a clear understanding of designs is important. On this basis alone, clinicians can begin to determine whether a paper describing a research study could be of value in informing their practice (Greenhalgh 2010). One of the most valuable places to find up-to-date information about quantitative research is the website of the Cochrane Collaboration: http://www.cochrane.org/ Important elements in quantitative research include objectivity in the conceptualisation of the problem, operational definition, accuracy, feasibility, control of the experimental environment, hypothesis testing, replication, internal validity and external validity. Selection of a particular design gives a researcher more or

Control is the presence of constants in a study, including controlling for extraneous variables, using comparison groups and implementing an explicit study protocol so that all participants are involved in the same way. In experimental research the comparison group is the control group (that receives the usual care or treatment), rather than the innovative experimental group under investigation. In some longitudinal or crossover research designs, participants act as their own controls. The strength and rigour of a quantitative design relates to controlling the effects of any extraneous variables that may cause bias (threats to internal validity, such as selection, history and maturation) and influence the study findings. These variables can be either antecedent or intervening. An antecedent (preceding) variable occurs before the study commences but may affect the outcome variable of interest and influence findings (e.g. age, gender, socioeconomic status or pre-existing health status could affect outcomes such as recovery time and ability to integrate healthcare behaviours). Similarly, an intervening (mediating) variable is not part of the study design, but occurs during the course of the study and may influence the outcome variable (e.g. a change in the model of clinical care during a longitudinal study). Threats to ‘internal validity’ are discussed further in Chapters 11 and 12.

Randomisation Randomisation (random assignment) to study groups is designed to ensure that groups or participants are similar on the variables of interest, so that differences in the outcome variable can be attributed to the intervention. If possible, participants are assigned to either the experimental or control group on a purely

164


7/25/2012 6:11:09 PM

9 • Common quantitative methods

BOX 9.1 Excerpts from experimental studies of random assignment to study groups ‘This is a randomised controlled trial. 120 sets of computer-generated random numbers were used, and patients who fitted the criteria were randomised to the study or control group.’ (Wong et al. 2010 p 270) ‘Using random stratified sampling from the total population of hospital staff …’ (Shields et al. 2011 p 156) ‘… participants were randomly allocated to one of the three groups: … Randomisation was performed in blocks separately for each hospital, using labelled cards in numbered closed envelopes prepared by a statistician not involved in the study.’ (Goedendorp et al. 2010 p 1124) ‘Eligible participants were asked whether they had ever had a class in which they practiced CPR (yes/no) and then assigned by coin flip to receive or not receive a cell telephone for manikin resuscitation.’ (Merchant et al. 2010 p 539)

RESEARCH IN BRIEF • Random assignment is the random allocation of participants (or whatever one is trying to investigate) to the intervention or control group of an experimental study. • Random sampling is a process of selecting a representative group of participants from the population of interest. • Randomisation need not be complicated. For small numbers, series of coin tosses (heads into one group, tails into another) or pulling names out of a hat (1 — into control group, 2 — into experimental group, 3 — into control group, and so on) are valid ways of random allocation. • An important point for randomisation techniques: if at all possible, the people who are doing the research should not know how the randomisation falls, or who goes into what groups. ‘Blinding’ or ‘concealment’ to the researchers makes for a much more rigorous study, as lack of these means that the researchers might inadvertently influence the responses that the participants give. If this is not possible, it is important that non-blinding or non-concealment is noted in the limitations section of the published study.

Manipulation

random, or chance, basis. Each participant therefore has an equal and known probability of being assigned to any group. If randomisation is not possible, the design is quasi-experimental. Random assignment may be done individually or by groups, with a variety of approaches available — from a very simple coin toss or ‘draw from a hat’, to more sophisticated techniques such as a table of random numbers, or computerised random number generation (see Box 9.1). Randomisation assumes that important intervening variables are then equally distributed between the groups. Studies should therefore report how groups actually compared on important variables prior to any intervention (see ‘Research in brief ’).

Manipulation is relevant only in interventional (experimental or quasi-experimental) studies, not observational designs. A researcher manipulates the causal or ‘explanatory (independent) variable’ by introducing a ‘treatment’ or ‘intervention’. The intervention (treatment) group receives manipulation of the explanatory variable, while the control group does not. The intervention might be a treatment, a teaching plan or a medication. The effect of this manipulation is measured to determine the effect of the intervention. Studies using true experimental designs in clinical settings are called randomised controlled trials (RCTs), reflecting the high level of control imposed and the importance of randomisation in gaining evidence for causation (discussed later in this chapter). A study protocol or procedure ensures that all participants in the intervention group receive similar treatment, and assists the reader in understanding the nature of the experimental treatment. 165


7/25/2012 6:11:09 PM


The three broad categories of observational, quasi-experimental and experimental designs are discussed below, after the section on ‘blinding’.

Blinding Many studies are described as ‘blind’, or ‘double blind’, a technique used most commonly in clinical trials of a drug or some sort of intervention. Blinding means that some pertinent information is hidden or withheld from those conducting the research and, oftentimes, from the participants as well. In this way, there can be no influence, or ‘contamination’ from those applying the research, or the treatment, or by the participant creating a ‘placebo effect’. A crude (and imaginary) example is a study of an ointment for wound healing. The ointment containing the relevant medication is being contrasted with another one which does not contain the medication (a placebo). If the researchers and their assistants, or the nurses working with the patients, know which ointment is being applied to the wound, they may influence the results by, perhaps, allowing the medicated ointment to stay on longer, or cleaning the wound better for application of the medicated ointment rather than the nonmedicated one. By blinding the ointment so no-one knows which one is being applied, and then contrasting the results, a true indication of the effectiveness of the ointment will be found. Of course, it would be good if the patients on whom the ointment is being tested don’t know either. This level of blinding, where the people doing the applying and those receiving the application, do not know which one is being used, is called ‘double blinding’. To give a real example (rather than the imaginary one I have just given), this process is commonly used in drug trials. Only by ensuring that no-one knows which patients receive the drug, and which ones received the placebo, can true results, free of contamination, be obtained. A Chinese study (Huang et al. 2011) of the use of a herbal medicine, CCH1, for constipation was carried out in three long-term care units. A powder containing the medicine or a placebo (contained no medication) was given to patients randomly assigned to either the control or experimental (intervention) group. No-one knew who was receiving which type of

powder — not the researchers, nor the treating nurses and doctors, nor the patients themselves. Only once the trial was finished, and the data collection complete, did the statistician and computer programmer, who had set up the coding, open the code to know which patient had received which intervention, and the effects analysed. The herbal medicine was found to be an effective treatment for constipation. RESEARCH IN BRIEF A randomised controlled trial of an intervention of home visits by nurses on 12-year-old children of economically disadvantaged women (n = 613) were compared with a control group (Kitzman et al. 2010). The families were similar in both experimental and control group, but only the experimental group had the home visits. The characteristics under examination included use of cigarettes, alcohol and marijuana in the 30 days before interview, a range of educational tests in reading and mathematics and reports of behaviour. The authors found that the children having the home visits were less likely to smoke, drink alcohol or use drugs, and were less likely to have psychological disorders. When the mothers were visited by nurses, the children scored higher academically than those in the control group, but there was no difference in the incidence of behavioural problems. The authors used appropriate statistical tests to examine the equivalence of the groups for categorical variables.

OBSERVATIONAL DESIGNS Observational (non-experimental) designs are used when a researcher wishes to construct a picture of a phenomenon or explore events, people or situations as they naturally occur in the environment. The aim is therefore to observe and identify variables of interest and explore relationships between those variables. These designs are used when: • there is little information or research about the topic of interest • the phenomena of interest are not amenable to experimental designs (e.g. when ethical considerations do not allow a ‘treatment’ to be denied to a control group).

166


7/25/2012 6:11:09 PM


Observational designs work from a clear, concise problem statement based within an appropriate theoretical framework. Although a researcher does not actively manipulate an intervention variable in this design, the concepts of control and rigour are still important and should be evident to the reader. Demographic data are collected to describe the sample of participants, such as age, gender, income level, ethnicity, occupation and educational level. Reporting this information assists a reader in considering the generalisability of the study and the potential application of the findings to their own practice setting. The common types of observational designs are discussed below, noting their advantages and disadvantages, and using examples from published clinical research to illustrate features of specific designs. RESEARCH IN BRIEF In a study of attitudes to working with children and their parents in a tertiary children’s hospital, staff (nurses, doctors, allied health and ancillary staff) were randomly selected from the total population of a hospital (Shields et al. 2011). They were asked to complete a questionnaire with two questions from which scores were derived for working with children and working with their parents, and the two were compared across the groups. Staff gave a significantly more positive score for working with children than working with their parents. This was the first study of its kind to examine such aspects of family-centred care, and the randomised sample facilitated the generalisability of the findings. Further work on this study will include subset analyses to see if different occupational groups give different scores, if the length of time working in paediatrics has an influence, and if those participants who are parents themselves give a different score for working with children and parents, than those who do not have children.

Descriptive/exploratory studies Descriptive studies use a variety of approaches to measure the variable/s of interest, including observation or questioning of participants via a questionnaire or interview, or health service data sets, for example. Direct observation of study participants from a quantitative perspective

involves only limited or no interaction with participants. Observers can use a pre-identified classification system to categorise observations or document free-text ‘field notes’ (see Chapter 12 for further discussion of participant observation). A survey is another common type of observational design (using either a questionnaire or interview) that enables collection of information about the characteristics of particular individuals, groups, institutions or situations, or about the frequency of a phenomenon’s occurrence, particularly when little is known within the positivist paradigm. Variables of interest commonly investigate participants’ knowledge, beliefs or attitudes about a particular topic or concept. The terms ‘exploratory’, ‘descriptive’ and ‘survey’ are used to describe this study design. Note that the study purpose is only to relate one variable to another. Relationships between variables or the direction of an effect cannot be tested. There are both advantages and disadvantages to consider when undertaking survey research (see Table 9.2). Importantly, descriptive studies cannot make comparisons between groups or determine causality. A study of lesbian, gay, transgender or bisexual (LGBT) parents seeking healthcare for their children included a survey of attitudes to homosexuality held by nursing and medical students (Chapman et al. 2011). While there is

TABLE 9.2 Advantages and disadvantages of survey studies ADVANTAGES

DISADVANTAGES

A lot of information can be obtained from a large population in an economical manner.

Large-scale surveys can be time-consuming and costly.

Accurate if sample is representative of the population of interest.

Information tends to be brief and superficial, with breadth rather than depth of data the outcome. Requires expertise in a variety of research areas (e.g. sampling techniques, questionnaire construction, interviewing, data analysis) to produce a reliable and valid study. 167


7/25/2012 6:11:09 PM


a large literature around differing sexualities, few have examined the use of health services by LGBT parents. Students become health practitioners, and so this study shed light on if and where education should be provided to ensure that once the students graduated, they were able to communicate with LGBT parents in an open and sensitive manner.

Correlational studies An extension of descriptive research is to explore the relationships between variables that provide a deeper insight into the phenomenon of interest. Correlational studies enable examination of the relationship between pairs of variables, as well as a comparison between groups. A researcher uses this design to quantify the strength of the relationship between variables (i.e. as one variable changes, does a related change occur in the other variable?). For example, Shields et al. (2006, 2010) investigated the correlation between breastfeeding up to age 3 months and obesity at 14 and 21 years of age. Using a longitudinal data set of over 7000 people, logistic regression was used to determine models which would demonstrate possible correlations. At age 21, whether or not people were breastfed had no effect on whether or not they were overweight or obese, but at age 14, there was a trend that indicated a possible effect. However, maternal education level and BMI were found to be far more influential on longterm outcomes for overweight and obesity. See Chapter 14 for explanation of correlation data analysis. Correlation designs cannot, however, test a ‘cause-and-effect’ relationship (i.e. whether a change in one variable causes a measurable change in another variable). When reading a correlational study, identify the variables and the relationship being tested, and consider whether the implied relationship is consistent with the conceptual framework and research question being asked. A common misuse of a correlational design is a researcher’s attempt to conclude that a causal relationship exists between the study variables. Despite this limitation, the design is useful — clinical practice applicability relates to the study question and feasibility. The advantages and disadvantages of correlational studies are listed in Table 9.3.

TABLE 9.3 Advantages and disadvantages of correlational studies ADVANTAGES

DISADVANTAGES

Increased flexibility to investigate relationships among variables.

Unable to determine a causal relationship between variables because of the lack of manipulation, control and randomisation.

Efficient and effective method of collecting a large amount of data about an issue of interest.

No random sampling possible with preexisting groups — the ability to generalise is therefore decreased.

Provides a framework for exploring the relationship between variables that are not able to be manipulated.

Unable to manipulate the variables of interest.

A foundation for potential, future interventional studies.

Cross-sectional studies Surveys that measure data at one point in time (i.e. data are collected on only one occasion with the same participants rather than on the same participants at several points in time) are called cross-sectional studies. These studies are categorised as either ‘descriptive’ or ‘analytic’ (Schoenbach 1999). Importantly, cross-sectional analytic (CSA) studies use inferential statistics to infer a relationship between two or more variables. To examine the prevalence of obstructive sleep apnoea (OSA) and resulting quality of life (QoL) in cardiac patients in India, 50 congestive heart failure patients were matched with 50 healthy controls and, using several validated questionnaires, a cross-sectional study was carried out (Patidar et al. 2011). Of the cardiac patients, 18% had OSA, as did 8% of the control group. Excessive daytime sleepiness was significantly associated with OSA in the cardiac patients (P = 0.02), while clinical severity and duration of illness were not. Increased body mass index and neck circumference significantly increased the risk of OSA; QoL was poor, and was worse in those cardiac patients with OSA. This design was appropriate to measure the variables of interest in a broad sample at one

168


7/25/2012 6:11:09 PM


point in time. It used a convenience sample, so there was a potential for selection bias, and the questionnaires used were self-report, so potentially were not objective. The use of inferential statistics was appropriate and demonstrated links between certain physical factors and OSA in this sample. A limitation of this design is that the outcome of interest and the causal factor are measured simultaneously. The lack of time as a factor therefore provides only weak evidence of causality. In contrast, a longitudinal study which followed a group of people for a period of time during a series of studies would provide more strength of causality (see later in this section).

Retrospective studies The aim of a retrospective study is to link present outcomes to some past events. The design is also called ‘ex-post facto’ (favoured by social scientists), or comparative, and is similar to a case-control study (see below). In a study of records kept by child health nurses at an Aboriginal community in Australia, the growth of children of 13 families was retrospectively tracked over three generations (Alsop-Shields 1997). There were statistically significant increases in weights across the generations at birth, one year and five years of age, indicating that the children’s health had improved over the 45 years of the records. In a Taiwanese study of peripherally inserted central catheters (PICCs) (Ting-Kai 2011), failures of the devices were evaluated by comparing two periods of retrospective case studies, before and after implementing changes in nursing practice for PICCs. Wound oozing, infection, phlebitis, occlusion and leaking rates improved significantly between the two periods, providing evidence that the changes in nursing care were effective. Such a study would provide more rigorous results if done as a RCT, but the investigators reported that such a trial was not possible for them and that this study provided them with a good basis from which they could design further studies.

Case-control studies A case-control study is an epidemiological approach which examines participants on the basis of a study outcome (clinical characteristic, condition or disease) that is or is not present

(e.g. lung cancer, heart disease). The study direction is retrospective, from the study outcome backwards to the cause or the exposure. Individuals with the outcome of interest are the ‘cases’. Cases are matched with ‘controls’, a sample of people who come from the same ‘study base’ (similar to the cases for characteristics such as age, gender or clinical procedure), but do not have the outcome of interest. There are commonly multiple controls matched to each case (2–3 : 1). Detailed past histories, particularly in relation to possible risk factors (exposure), are examined from both groups to see if the cases have been more exposed to the suspected causative factor variable than the controls. RESEARCH IN BRIEF A case control study in the USA (McHugh et al. 2011) was designed to examine patient and hospital characteristics associated with severe poor glycaemic control, an indication of severe complications of diabetes. All patients (cases) (n = 261) with poor glycaemic control admitted to California acute care hospitals from 2005 to 2006, and 261 controls, were matched using data for age, sex, major diagnostic category and severity of illness. The statistical test used was adjusted odds ratio (OR) for experiencing poor glycaemic control. Mortality (16% vs 9%, p = 0.01) and total costs ($26 125 vs $18 233, p = 0.026) were higher among cases with poor glycaemic control. When the type of hospital was tested, levels of nurse staffing was not significantly influential in teaching hospitals (OR: 0.98, 95% CI: 0.88– 1.11), but in non-teaching hospitals each additional nursing hour per adjusted patient day significantly reduced the odds of poor glycaemic control by 16% (OR: 0.84, 95% CI: 0.73–0.96). The authors ascribed these results about registered nurse hours in teaching hospitals to possible increased complexities of patients’ conditions (though this should have been accounted for by the matching of control and experimental participants), but the authors tell us that while their matching procedures were as rigorous as possible, there were several clinical factors that could not be matched, a common problem with this sort of study. Despite this, case control studies are valuable ways of comparing different interventions or experiments. 169


7/25/2012 6:11:09 PM


Selection bias may exist if the cases are not from a well-defined study base (in time and place). A critical reader therefore needs to cautiously evaluate the conclusions drawn in relation to measurement error. Advantages are similar to those of a correlational design, but a higher level of control is possible. Disadvantages include an inability to draw a causal linkage between the two variables (an alternative hypothesis may cause the relationship), and finding naturally occurring groups of participants who are similar in all respects except for their exposure to the variable of interest is very difficult.

Cohort studies A cohort study is an epidemiological approach where the direction is from the exposure to the outcome, or cause to presumed effect. Commonly, a researcher studies the development of a particular health outcome or disease state. Participants are selected from a population known to be free of the health outcome under study, and then classified according to whether they have one or more explanatory variables hypothetically related to the outcome. These participants (referred to as the cohort) are then studied over a time period ranging from days to months to years, to determine who develops the outcome of interest. The incidence of the study outcome is observed in relation to exposure to possible causes/risk factors (Grimes & Schulz 2002). Cohort studies can be cross sectional, or longitudinal. Cross-sectional studies examine cohorts at one point in time, or perhaps sequences of time points, while longitudinal cohort studies investigate cohorts over long time periods. In 2010 Sanada et al. used crosssectional cohorts of patients with pressure ulcers to test the efficacy and cost effectiveness of a system of pressure ulcer management by skilled nurses, over a three-week period. Cohort designs can be used to examine relationships both retrospectively and prospectively. Prospective (longitudinal) studies explore hypothesised causes, differences or relationships, and move forward in time to the presumed effect. A disadvantage of this design is the time period and high costs involved, as there may be a considerable time lag between time of

exposure and the subsequent study outcome. This may be partly overcome in an historical cohort study if some of the data of individuals exposed in the past are available for use.

Longitudinal studies In contrast to a cross-sectional design, longitudinal studies collect data from the same group of participants at different points in time (repeated measures). By collecting data from each participant at certain intervals, a longitudinal perspective of the outcome variable is possible. An example of a longitudinal cohort study is the classic Nurses’ Health Study, one of the largest cohort studies of risk factors for major chronic diseases in women. In 1976 over 120 000 married registered nurses aged 30–55 were initially enrolled in the study (Belanger et al. 1978) and the cohort continues to be followed (Baer 2011), examining a wide range of issues including diet, menopause and breast cancer. At the time of writing, contemporary articles continue to be published from follow-up assessments; over 1200 articles have been published from this data set from 1978–2010 (see the Nurses’ Health Study website in ‘Additional resources’ at the end of this chapter). Similarly, the Framingham heart study examined the effect of blood pressure, cholesterol levels, smoking, exercise and other variables on the development of coronary artery disease in a cohort of healthy men, at specified intervals over a period of years (e.g. Margolis et al. 1974; Ho et al. 2010). There are three types of longitudinal design, each with their advantages and disadvantages: 1) longitudinal trend studies 2) longitudinal cohort studies 3) longitudinal panel studies and each of these will be considered in turn. Longitudinal trend studies

This type of design is good for looking at general trends in populations; however, it is not suitable for following individuals. Trend studies are used, for example, by newspapers in the run up to a general election and are carried out by taking regular samples from the population to find out what likely voting patterns are going to be. Therefore, trend studies use the same population

170


7/25/2012 6:11:09 PM


but different samples from it over time. Trend studies are unique in longitudinal designs in being immune to attrition, or dropout, from the study as they make no effort to re-sample the same people (although they may do this incidentally) and sampling at each stage simply proceeds until sufficient participants have responded. Longitudinal cohort studies

This type of design uses a sample—called a cohort—and then runs over time at regular intervals by sampling from that cohort. For example, the cohort may be all the students graduating in one year in nursing from a large university nursing program. A university may be interested in following up this cohort over the years after graduation and they can do this by sending out questionnaires to students in the group. The sampling can be done at random from the cohort or the whole cohort may be sampled. However, over time, attrition will take place due to people becoming impossible to trace and some people may not respond at one time point but may reappear the next. Therefore, in this design, there is no effort to sample exactly the same people, although many of the same people will respond; rather it is information about the whole group that is being sought through those sampled at each stage. Clearly, this design is more vulnerable to attrition than the trend design. Longitudinal panel studies

This type of design is similar to the cohort design as it follows a defined group of people; however, unlike the cohort design, a panel study is designed to follow individuals over time and makes every effort to re-recruit and, thereby, re-sample precisely the same people at each stage in the study. This is a very powerful way of studying individual change and individual differences over time but provides less information about a whole group than a cohort study and little information about the population. However, panel studies are very vulnerable to attrition as, once someone is lost to the study, they rarely reappear and, while statistical methods exist to compensate, their data are essentially lost to the study. The advantages and disadvantages of a longitudinal design are listed in Table 9.4.

TABLE 9.4 Advantages and disadvantages of longitudinal studies ADVANTAGES

DISADVANTAGES

Each participant is followed separately and therefore acts as her or his own control.

Long duration of data collection — costly in terms of time, effort and resources.

Both relationships and differences can be explored between variables.

Threats to internal validity include ‘testing’ and ‘mortality’ (loss to follow-up), and the influence of confounding variables.

Changes in the variables of interest are assessed over time; early trends in data can be investigated at a subsequent measurement.

Social desirability bias is possible (participants respond in a way they believe is congruent with the researchers’ expectations) — ‘Hawthorne effect’.

RESEARCH IN BRIEF The Mater University Study of Pregnancy (MUSP) (http://www.socialscience.uq.edu.au/ musp) has for nearly 25 years studied a cohort of about 8000 children, and has published over 200 papers about the growth and development of children. Shields et al. (2006, 2010) used the MUSP data set to examine the effect of breastfeeding on children at aged 14 and 21 years respectively. From a total of 7776 children originally recruited at the beginning of the study, in the 14 year study, data were available for 3698 children and 2553 at 21 years. Such drop off is a characteristic of longitudinal studies with a long timeframe, and statistical analysis of the loss to follow-up is an inherent part of subsequent studies. In the 14-year-old children, breastfeeding for longer than 6 months was protective of obesity (OR 0.6, 95% CI 0.4, 0.96), but not of overweight. However, when confounding variables were considered the effect size diminished and lost statistical significance (OR 0.8, 95% CI 0.5, 1.3). In short, breastfeeding for less than 6 months had no effect on either obesity or overweight, though a trend was found for increased prevalence of overweight at 14 years with shorter periods of breastfeeding. By age 21, there was no influence on either overweight or obesity from breastfeeding. 171


7/25/2012 6:11:09 PM


Point to ponder When assessing the appropriateness of a cross-sectional study compared to a longitudinal study, first examine the purpose of the study in relation to the design. There are many longitudinal data sets available for study around the world, and the custodians of these are keen for other investigators to use them. Consider approaching authors if you are interested in doing this sort of work. It is an interesting approach to quantitative research that can yield not only worthwhile results, but also publications in high ranking journals. The following is a list of such datasets: Mater University Study of Pregnancy (MUSP): (http://www.socialscience.uq. edu.au/musp) Avon Longitudinal Study of Parents and Children (ALSPAC): http://www.bristol. ac.uk/alspac/ Raine Study: http://www.rainestudy.org.au/ Peel Child Health Study: http://www. peelchildhealthstudy.com.au/ The Dutch Famine Birth Cohort Study: http://www.dutchfamine.nl/index_files/ study.htm

Causality in observational designs As noted in this section, observational designs are often limited in their ability to determine ‘causality’ — where one variable influences another in a cause-and-effect relationship. Historically, only experimental research has been able to support the concept of causality. There are, however, many instances in clinical research where experimental studies cannot be conducted because of ethical or practical reasons. To overcome this limitation, several statistical analytical techniques are available to explain relationships among variables and establish causal links. These ‘causal modelling’ analyses (also termed ‘causal analysis’, ‘path analysis’, ‘Linear Structural Relation Analysis [LISREL]’ and ‘structural equation modelling [SEM]’) are introduced in Chapter 13 and discussed further in other specialty statistics texts (see ‘Additional resources’ at the end of this chapter).

QUASI-EXPERIMENTAL DESIGNS Quasi-experiments are designs where a researcher manipulates an experimental treatment (intervention; explanatory variable) but some characteristic of a ‘true’ experiment is lacking — either control or randomisation. Both designs test cause-and-effect relationships, but the lack of control and/or randomisation in quasiexperimental designs threatens the study’s internal validity and weakens any causal inference. Many different quasi-experimental designs exist; the common types are discussed here (see Figure 9.3). Suppose a researcher is interested in the effects of a new cardiac education program on the physical and psycho-social outcome of patients newly diagnosed with acute coronary syndrome. A researcher could randomly assign participants to either the group receiving the new program or the group receiving the usual program, but for any number of practical reasons, that design is not possible. One difficulty is when patients on the same clinical unit have to receive both experimental and control interventions, without any overlap. The potential for participant ‘contamination’ needs to be considered by the researcher — how to stop participants receiving or adopting the ‘wrong’ intervention. The choices are to abandon the experiment, or conduct a quasi-experimental study. With the latter, a researcher can find a similar clinical unit that has not introduced the new educational program and study the newly diagnosed cardiac patients who are admitted as the comparison (control) group.

RESEARCH IN BRIEF ‘Study contamination’ occurs when participants allocated to a specific study group receive the alternative intervention. It can also occur when participants in the experimental group become aware of the effects of the intervention from an external source; perhaps people in the two groups compare effects and so each group may express outcomes that are not due to the intervention under study.

172


7/25/2012 6:11:10 PM

9 • Common quantitative methods A Non-equivalent control group design

B

Experimental group

pre-test

Control group

pre-test

experimental treatment

post-test post-test

After-only non-equivalent control group design Experimental group


post-test

Control group C

post-test

One group pre-test post-test design Experimental group

D

pre-test


post-test

Time series design Experimental group

pre-test

pre-test

Control group

pre-test

pre-test


post-test

post-test

post-test

post-test

Figure 9.3 Comparison of quasi-experimental designs

There is no random assignment of patients by the researcher in this situation; therefore a quasi-experimental design. There is, however, a range of potential threats to validity when interpreting the results of quasi-experimental studies (see Table 9.5).

TABLE 9.5 Advantages and disadvantages of quasiexperimental studies ADVANTAGES

DISADVANTAGES

Practical, feasible and able to be generalised.

Need to rule out any plausible alternative explanations for the findings — control by design or statistical analysis.

May be the only design to evaluate some hypotheses, particularly in clinical settings.

Unable to make clear cause-and-effect inferences.

Non-equivalent control group studies This design is the quasi-experimental version of a ‘true experimental’ design, except participants are unable to be randomised to study groups because of practical or feasibility issues (Figure 9.3, A). Despite the lack of randomisation, this design is commonly used in clinical research as it is relatively robust to threats to internal validity. Collection of data at ‘pre-test’ (measurement prior to the intervention) allows comparison of the two groups before the intervention is introduced, and any differences between groups can also be controlled during data analysis. This feature strengthens the influence of the intervention, and minimises any effects of extraneous variables.

More adaptable to the real-world practice setting than controlled experimental designs.

173


7/25/2012 6:11:10 PM


RESEARCH IN BRIEF A quasi-experimental design was used to evaluate three nursing interventions — occlusive wrap, chemical mattress and regulation of delivery room temperature— singly and in combination on thermoregulation in six groups of low birth weight infants (Lewis et al. 2011). 133 infants under 1500 grams were included, and the interventions tested on different groups of infants over 3 years. A control group of 295 infants on which retrospective chart data were available over an earlier 3-year period was used. For each of the interventions, the percentage with a normal temperature in NICU in each intervention group was larger than the percentage of the control group, but it was not a significant difference. Each intervention — occlusive wrap alone, occlusive wrap in addition to chemical mattress and occlusive wrap in addition to chemical mattress and increased delivery room temperature — had a positive influence on thermoregulation.

After-only non-equivalent control group studies Suppose a clinical researcher did not measure specific characteristics of participants before the introduction of a new intervention, but later decided that it would be useful to have data demonstrating the effect of the program. Perhaps a health service manager asked for these data to determine whether the extra cost of a new teaching program was justified. The study that could be conducted would be the after-only non-equivalent control group design, shown in Figure 9.3, B. This design minimises ‘testing effects’ (completing a pre-test can affect the post-test scores), but assumes that the two groups are equivalent before the intervention is introduced.

Points to ponder Check that two non-randomly assigned groups are comparable on important characteristics at the beginning of the study. Examine the concept of ‘loss to follow-up’ and how it can influence statistical results: http://www.consort-statement.org/ resources/glossary/e---l/loss-to-follow-up/

Under the next heading there is an example of a study of an intervention of a teaching leaflet about pressure ulcers for older people (Hartigan et al. 2011). Measuring the participants’ knowledge after the education leaflet would not tell us whether their knowledge levels differed before they received the leaflet, or whether the education motivated individuals to learn more about their health problem. Therefore a researcher’s conclusion that the teaching program improved physical status and psycho-social outcome would be subject to the alternative conclusion that the results were an effect of pre-existing knowledge (selection effect) in combination with greater learning in those more motivated (selection–maturation interaction). See Chapters 11 and 12 for further description of these threats to study validity.

One group pre-test–posttest studies This design is used when only one group is available for studying the effects of an intervention (Figure 9.3, C). The lack of randomisation and a control group limit the internal validity and generalisability of any findings. A one group pre-test–post-test design was used in an Irish study about older people and pressure ulcers (Hartigan et al. 2011). In Ireland, a quasi-experimental study of a patient education leaflet about pressure ulcers for older people was conducted (Hartigan et al. 2011). Uncontrolled, it used a pre- and post-test to assess 75 participants’ level of knowledge about preventative strategies for pressure ulcers. A tool to test knowledge was developed and then applied to both pre- and post-test groups. Post-intervention, the participants’ knowledge increased about: what a pressure ulcer was (68% pre-test — 91% post-intervention); causes (77%–89%); prevention when sitting in a chair (79%–91%) and a figure (no numbers or percentages were given for knowledge about signs and symptoms) showed a marked improvement of knowledge post-intervention, and knowledge about common sites was high and not changed by the leaflet. While the statistics in this study were very simple percentages only, it is still valid. More complicated statistics may have yielded more definitive information about the changes, but for

174


7/25/2012 6:11:10 PM


the purposes of a study such as this, simple results can be just as informative as the most complicated statistical analyses. In fact, they are often more informative as they are easy to follow and understand, and are therefore more readily critiqued and eventually applied in practice. Attempting to include a control group in this environment would have led to study contamination, but limited generalisability. There was minimal description of the training program. The measuring instrument was previously developed by the researcher; acceptable reliability was reported but validity was not discussed. The sample was small and in one site only; these limitations were noted by the author.

Before–after design Sometimes researchers use a ‘before–after’ design to test the effects of an introduced intervention when participants are different individuals before the intervention than after the intervention. The Irish study (Hartigan et al. 2011) again provides an example.

1

Tutorial Trigger Given the information in the ‘before– after design’ section, describe the reasons why an RCT design was not possible in the Irish study (Hartigan et al. 2011).

Time-series studies Another approach when only one group is available is to study that group over a longer period, using a time-series design (illustrated in Figure 9.3, D). To rule out alternative explanations for the findings of a one-group pre-test–post-test design, the variable/s of interest are measured over a longer period while introducing the intervention at some time during the data collection period. Investigators from Chile (Scarella et al. 2011) used three time points to study the effects of a new classification system on caesarean section rates, over a 21-month period. At the first time point (3 months after implementation), the rate had reduced from 36.8% to 26.5%, and again in the second period (6 months; RR 0.71 CI

0.63–0.81). After the intervention was stopped, the caesarean rate increased again to 31.8% (RR 1.19 CI 1.09–1.32), but it was still a decrease of 5% from the first, basal period (RR 0.86 CI 0.76–0.97). The authors concluded that their new system was an effective and safe strategy to reduce the rate of caesarean section. They suggest that a strength of the study is that it was designed in steps, with the first period allowing them to identify the main groups contributing to the overall caesarean rate, creating ‘groups of interest’ that represent more than 50% of the caesarean sections in their maternity ward. However, they suggest that the Hawthorne effect (behaviour change because of knowing one is part of a study) may have influenced the findings, they had no control group with which to compare and, to quote the authors, a major limitation pertinent to time period studies was that ‘an interrupted time series methodology has the potential bias that it does not control all variables, including the fact that seasonal variations can influence the results’ (Scarella et al. 2011 p 139). Even with the absence of a control group, the number of data collection points minimises threats to validity such as ‘history’ effects. However, ‘testing’ is a threat to validity with this design because measures are repeated so many times. Why alternative explanations for the findings in a quasi-experimental study are not plausible should be discussed by the researcher. In some cases, clinical or practical knowledge of the problem and patient population can suggest that a particular explanation is not plausible. Nonetheless it is important to replicate studies to support any causal inferences developed through quasi-experimental designs.

EXPERIMENTAL DESIGNS An experiment is a scientific investigation that involves observation and data collection according to explicit criteria and protocol. Experimental designs have all three identifying properties: randomisation, control and manipulation. These designs are used to test ‘cause-and-effect relationships’ between an intervention (treatment) and an outcome, and minimise or control any alternative explanations (threats to validity) for the study findings (Thompson 2004). To infer causality requires: 175


7/25/2012 6:11:10 PM


• the explanatory (causal) and outcome (effect) variables must be associated with each other • the ‘cause’ to precede the ‘effect’ • that the relationship is not able to be explained by another (extraneous) variable/s. Experiments can be classified by setting. While ‘field’ experiments such as an RCT and ‘laboratory’ experiments share the same design characteristics and properties, they are conducted in fundamentally different environments. Laboratory experiments take place in an artificial setting created specifically for the purpose of research, enabling almost complete control by a researcher. Field experiments, however, occur in a real, existing social setting such as a hospital unit, clinic or community where the phenomenon of interest actually occurs. Conducting an experimental study can be challenging, as all relevant variables need to be identified and then controlled, manipulated or measured. However, they are the only way that nurses and midwives can generate research relevant to their knowledge and evidence requirements. Unlike the physical sciences, nursing and midwifery researchers continue to identify important complex concepts and relationships that are the province of our disciplines. Nurses and midwives are ideally placed to use the broadest range of research methods available, pertinent to the topics they wish to examine. Nursing and midwifery, with their emphasis on interactions with human beings, are often the best disciplines from which thoroughly conceptualised and rigorously prepared human factor evidence and knowledge can be generated. This puts nurses and midwives in a particularly privileged position, as they are best able to determine the appropriate design of a study to answer certain questions relating to the human factors which are an inherent part of the two disciplines. While most experiments in nursing and midwifery are field experiments and control is such an important element, it is common that a researcher cannot control contamination. Conversely, studies conducted in the laboratory are by nature ‘artificial’, and therefore not ‘real-world’. Although laboratory experiments have stronger internal validity than fieldwork studies, their weakness is with external validity. When reading research reports, it is important to consider the setting of the experiment and what impact this may have on the study findings.

There are several different experimental designs, each based on the classic ‘true experiment’ presented in Figure 9.4, A. Participants are randomly assigned to the control or experimental group(s), with the treatment given only to those in the experimental group. The outcome variable is measured in both groups as a pre-test and post-test (before and after the intervention is introduced). When reviewing experimental (or quasiexperimental) studies, the prime focus is on assessing the validity of the findings. Did the intervention (explanatory variable) cause the desired effect on the outcome variable? Validity of the findings and conclusion depend on how well the researcher has controlled other variables that may also explain any change in the outcome variables. Although random assignment and control in the classic experimental design minimise the effects of many threats to internal validity, it is not perfect in practice as some threats are difficult to control. In particular, ‘mortality’ effects (a threat to internal validity) may be a problem for studies with long follow-up periods, as participants tend to drop out because of the burden over an extended period of time (called ‘loss to follow-up’ [CONSORT 2011]). There may be important differences between participants who withdraw and those who complete the study; these differences may explain the study findings, not the intervention. Guidelines for reporting experimental studies have been developed by the CONSORT (consolidated standards of reporting trials) group for reporting these types of threats (Moher et al. 2001; see ‘Additional resources’ at the end of this chapter), and it is very important to examine and discuss any loss to follow-up in studies as these may influence the study’s findings (Hennekens et al. 1987).

2

Tutorial Trigger

The group assignment for your research subject is to critique an assigned quantitative study. To proceed, you must first decide on the design to be used: you think it is an ex-post facto design; the others in the group think it is an experimental design because it has several explicit hypotheses. How would you convince them that you are correct?

176


7/25/2012 6:11:11 PM

9 • Common quantitative methods A True or classic experiment

Random assignment

Experimental group

pre-test

Control group

pre-test


post-test

post-test

B Solomon four-group design

Random assignment

Experimental group

pre-test

Control group

pre-test

Control group


post-test


Control group C

post-test

post-test

post-test

After-only experimental design Experimental group


post-test

Random assignment Control group

post-test

Figure 9.4 Comparison of experimental designs

RANDOMISED CONTROLLED TRIALS A randomised controlled trial (RCT) is the clinical equivalent of a true experiment, and is the ‘gold standard’ for testing cause-and-effect relationships in clinical research (Thompson 2004). As illustrated in Figure 9.4, A, participants are randomly assigned to the experimental and control groups, so that any pre-intervention differences (antecedent variables) are measured and/or controlled. These pre-test measures or observations provide a baseline score for verifying similar characteristics between control and treatment groups, and therefore the effect of the intervention. The intervention is then introduced to the ‘treatment’ group and the outcome variable is again measured to see whether it has changed. The

control group gets no experimental treatment but is also measured for comparison with the experimental group. The difference between the two groups at post-test reflects whether a causal link actually exists between the explanatory and outcome variables. In a study about breastfeeding rates in a health region in the UK (Jolly et al. 2011), 2724 women gave birth following antenatal care in 66 clinics, so 33 clinics (1,267 women) were randomised to the intervention (a service given by peer support workers [PSW] in both anteand postnatal periods) and 33 clinics (1457 women) to usual care. Of those, 848 women consented to additional follow-up by questionnaire at 6 months. Data about breastfeeding (any and exclusive) at 10–14 days, 6 weeks and 6 months were obtained from routine records. Loss to follow-up was 6% at 177


7/25/2012 6:11:11 PM


10–14 days and 22% at 6 months. There was no difference between intervention and usual care in any breastfeeding at 10–14 days (OR 1.07, 95% CI 0.87–1.31, p = 0.54), and the proportion of women reporting any breastfeeding in the intervention group at 6 weeks was 62.7% and 64.5% in the usual care group (OR 0.93, 95% CI 0.64–1.35); and at 6 months was 34.3% and 38.9% respectively (OR 1.06, 95% CI 0.71– 1.58). The authors concluded that this peer support service did not improve breastfeeding rates up to 6 months in this population. However, the loss to follow-up of almost a quarter of the original sample may have influenced the findings — if all had been included at 6 months, would the results have been different?

Solomon four-group design A more complex experimental design used to minimise the effect of repeated ‘testing’ (a threat to internal validity) is the Solomon four-group design. This design has two groups identical to those used in the classic experimental design, but two further groups, an experimental after–only group and a control after–only group are also included. As Figure 9.4, B, illustrates, all four groups have randomly assigned participants. Addition of the last two groups helps to rule out ‘testing’ threats that the pre-test groups may experience. This design enables evaluation of the effect of the pre-test on the post-test. A limitation, however, is the need for a larger sample size when compared to a two-group design. In contemporary research, this design is not common in nursing or midwifery. Evidence-based practice tip Solomon four-group designs have come under methodological scrutiny. A systematic review of studies about behaviour using this process found that no studies could be included due to significant methodological quality issues (McCambridge et al. 2011). The statistics for this design are complex and so more study is needed into their feasibility and application.

After-only design Another alternative design is the after-only design (or post-test-only control group), shown

in Figure 9.4, C. This design is composed of two randomly assigned groups, but neither group is measured at pre-test. The process of randomly assigning participants to groups is assumed to be sufficient to ensure a lack of bias in determining whether the treatment created significant differences between the two groups. This design is particularly useful when testing effects are expected to be a major problem and the number of available participants is too limited to use a Solomon four-group design.

Point to ponder Experimental studies are not the most commonly used designs in nursing or midwifery research. Experimentation assumes that all of the relevant variables involved in a phenomenon have been identified and are measurable. For many areas of nursing and midwifery this is not the case, and descriptive or exploratory studies need to be completed before experimental interventions can be applied to a clinical issue.

Cluster-randomised controlled trials A cluster-randomised controlled trial (c-RCT) uses coherent groups or clusters of individuals, such as nursing homes (Pettersson et al. 2011), classes of students (e.g. Chen & Chung 2011), midwifery clinics (e.g. Jolly et al. 2011) or hospitals (Shields et al. 2007) as the random allocation unit, not individuals. All individuals in the group allocated to the intervention then receive the experimental treatment, eliminating the potential for study contamination. These studies are, however, expensive to conduct as the ‘cluster’ is the unit of analysis and any variance within the group works against any intervention effect (Christie et al. 2009) (see ‘Additional resources’ at the end of this chapter). Chapter 15 discusses the critical review process, which is directed towards evaluating the appropriateness of the study design in relation to factors such as the research problem, theoretical framework, hypothesis, methods and data analysis and interpretation. The overall purpose of reviewing studies is to assess the validity of findings and to determine whether these findings are worth incorporating into your professional

178


7/25/2012 6:11:11 PM


BOX 9.2 Criteria for evaluating quantitative designs a) What design was used in the study (observational, quasi-experimental or experimental)? b) Which specific type of the above design was used in the study, and was it appropriate? c) Was the design appropriate for the research problem and data collection methods? d) Was the problem examining a cause-andeffect relationship? e) Were the common threats to the validity of findings for this design addressed? f) Was the design suited to the study setting? g) Were the limitations of the design adequately discussed? h) Were there other limitations related to the design that you identified but they were not discussed? i) For observational designs, did the report go beyond the parameters of this design, and infer cause-and-effect relationships between variables? j) For experimental designs, how were randomisation, control and manipulation applied? k) What were the plausible alternative explanations for findings, and were they discussed and discounted? l) Are the findings able to be generalised to other practice settings and the larger population of interest?

practice and/or as the evidence base for current institutional clinical practices. Criteria for the review of quantitative designs relate to how well any potential biases in the methods have been addressed (see Box 9.2). The most important question to ask as you read experimental studies is, ‘What else could have happened to explain the findings?’ This question of potential alternative explanations will be addressed by a well-written report, which will systematically review potential threats

to the validity of the findings. You must then decide if the author’s explanations are clear and logical.

RESEARCH IN BRIEF The study above about a peer support intervention to improve breastfeeding rates used cluster randomisation (Jolly et al. 2011). This design was used for practical reasons in preference to an RCT, but equally it helped to avoid contamination across study groups within individual clinics.

Cluster randomisation could be used to study a model of care that is in use across many institutions. Family-centred care is widely used in children’s hospitals, but is untested (Shields et al. 2007). An RCT using an intensive family-centred care model involving the whole hospital could be conducted, with one group of hospitals receiving the family-centred care intervention, while the other group used their normal everyday practice. Each hospital would constitute a cluster. However, there is probably only one country where this could be successfully carried out. The United Kingdom has 33 children’s hospitals all run along similar lines under the National Health Service, so that would remove variables that might influence the outcomes such as different ways of running the hospital. Also they are geographically far enough apart to prevent contamination between the clusters. However, this would be extremely expensive to conduct, and so other ways to test the model called ‘family-centred care’ have to be devised.

SUMMARY Quantitative research methods enable a researcher to manipulate numerical data to answer specific research questions and draw inferences from their findings. Observational (non-experimental) designs are used to construct a picture or description of events as they naturally occur, but cannot establish cause-andeffect relationships between variables. Quasiexperimental designs are frequently used in clinical research to explore cause-and-effect 179


7/25/2012 6:11:11 PM


relationships, as there are times when experimental designs which can test this relationship are impractical or unethical to conduct within that setting. True experiments are characterised by control of extraneous variables, manipulation of the explanatory

variable and to randomly assign participants to study groups. The following four chapters discuss, respectively, sampling, data collection, measuring instrument assessment and data analysis issues for quantitative research.

KEY POINTS • Three main strengths of quantitative approaches are the objectivity, precision and control afforded through design, sampling strategies and analytical tests. • Experimental and other quantitative designs allow nurses and midwives to justify in a scientific manner the outcomes of their actions and provide the basis for effective high quality and evidence-based clinical practice. • Extraneous and confounding variables represent a major influence on the interpretation of any quantitative study as they limit the validity of the study and generalisability of the findings. This is particularly evident with observational designs. • When reviewing experimental studies, check that the author has addressed all of the relevant issues, such as evidence of a representative sample, and minimal and unbiased loss to follow-up.

Learning activities 1. Quantitative research uses numbers, counts and statistical tests. True False 2. A study using questionnaires with numerical scores to examine feelings, perceptions and social effects is a qualitative study. True False 3. Administration of an intervention to one group of participants and not another is an experimental study. True False 4. Bias is introduced when factors extraneous to the study influence the findings. True False

5. Non-experimental correlation studies are used frequently in nursing and midwifery research so that the findings from this design can be generalised to larger populations. True False 6. A time series design has two groups identical to the true experimental design plus an experimental after-group and a control after-group. True False 7. A true experiment design includes three properties — randomisation, control and manipulation. True False

180


7/25/2012 6:11:11 PM


8. Control is implemented in the design of a study by manipulation of the outcome variable. True False 9. A randomised controlled trial is the most powerful type of design for examining cause-and-effect relationships. True False 10. When data are collected multiple times before and after the introduction of the intervention, the study is a non-equivalent control group design. True False 11. In quasi-experimental designs, one of the characteristics of a true experimental design is lacking. True False 12. Loss to follow-up is one of the potential disadvantages of longitudinal studies. True False

13. Identify whether the following studies are experimental or quasi-experimental. Use the abbreviations E for experimental and Q for quasi-experimental. a) 100 elderly people with leg ulcers are randomly assigned into an experimental wound support group and a regular support group. Before the program and at the end of the 6-month program, wound healing patterns are compared between the two groups. b) Parents on two separate neonatal units are given a stress score questionnaire to complete at the end of their infant’s first hospital day and on the day of discharge. The infants on one unit receive care directed by a nurse case manager, and those on the other unit receive care from the usual rotation of nurses. Parent stress scores are compared. c) Student midwives are randomly assigned to two groups. One group receives an experimental independent study program and the other receives the usual classroom instruction. Both groups receive the same post-test to evaluate learning. d) A study was conducted to compare the effectiveness of shortened fasting times for adults pre-anaesthetic on postoperative recovery. Participants were randomly assigned into groups and post-operative nausea and vomiting were measured in the first day post-operatively.

Additional resources

Critical Appraisal Skills Program (CASP), UK National Health Service Critical Appraisal Tools — randomised controlled trials; cohort studies; case control studies. Online. Available: http:// www.phru.nhs.uk/casp Dutch Famine Birth Cohort Study. Online. Available: http://www.dutchfamine.nl/index_files/study.htm Fletcher R H, Fletcher S W 2005 Clinical Epidemiology: the Essentials, 4th edn. Lippincott Williams & Wilkins, Baltimore, USA

Avon Longitudinal Study of Parents and Children (ALSPAC) Online. Available: http:// www.bristol.ac.uk/alspac/ Cochrane Collaboration 2011. Online. Available: http://www.cochrane.org/ Cook T D, Campbell D T 1979 Quasi-experimentation: Design and Analysis Issues for Field Settings. Rand-McNally, Chicago, USA

181


7/25/2012 6:11:11 PM


Hanson B P 2006 Designing, conducting and reporting clinical research: a step by step approach. Injury, International Journal of the Care of the Injured 37:583–94 Macha K, McDonough J 2011 Epidemiology for Advanced Nursing Practice. Joyce and Bartlett Learning, New York, USA Mater University Study of Pregnancy 2011 Mater University Study of Pregnancy (MUSP). Online. Available: http://www.socialscience.uq.edu.au/musp Medical Research Council (United Kingdom) 2008 Complex interventions guidance. Online. Available: http://www.mrc.ac.uk/Utilities/Documentrecord/ index.htm?d=MRC004871 Moher D, Schulz K F, Altman D G, for the CONSORT Group 2001 The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357:1191–4 Munro B H 2005 Statistical Methods for Health Care Research, 5th edn. Lippincott Williams & Wilkins, Philadelphia, USA Nurses’ Health Study. Online. Available: http:// www.channing.harvard.edu/nhs/index.html Peel Child Health Study. Online. Available: http:// www.peelchildhealthstudy.com.au/ Polivka B J, Nickel J T 1992 Case-control design: an appropriate strategy for nursing research. Nursing Research 41:250–3 Raine Study. Online. Available: http://www.rainestudy. org.au/ Weinert C, Burman M 1996 Nurturing longitudinal samples. West Journal of Nursing Research 18:360–4

References Alsop-Shields L 1997 The growth of children for two generations at an Australian Aboriginal Community. Journal of Pediatric Nursing 11(6):402–8 Australasian Epidemiological Association 2010 What is epidemiology? Online. Available: http:// www.aea.asn.au [accessed 4 June 2012] Baer H J, Glynn R J, Hu F B, et al 2011 Risk Factors for mortality in the Nurses’ Health Study: A Competing Risks Analysis. American Journal of Epidemiology 173(3):319–29 doi:10.1093/aje/ kwq368 Belanger C F, Hennekens C H, Rosner B, Speizer F E 1978 The nurses’ health study. American Journal of Nursing 78:1039–40 Bryman A, Cramer D 2005 Quantitative Data Analysis with SPSS 12 and 13: A Guide for Social Scientists. Routledge, London, UK Chapman R, Watkins R, Zappia T, Nicol P, Shields L 2011 Nursing and medical students’ attitudes, knowledge and beliefs regarding lesbian, gay,

bisexual and transgender parents seeking health care for their children. Journal of Clinical Nursing doi: 10.1111/j.1365-2702.2011.03892.x Chen H S, Chung C H 2011 The learning effectiveness of nursing students using online testing as an assistant tool: A cluster randomised controlled trial. Nurse Education Today. Early view, doi:10.1016/j. nedt.2011.03.004 Christie J, O’Halloran P, Stevenson M 2009 Planning a cluster randomised controlled trial: methodological issues. Nursing Research 58(2):128–34 CONSORT 2011. Loss to follow-up. CONSORT: Transparent Reporting of Trials. Online. Available: http://www.consort-statement.org/resources/ glossary/e---l/loss-to-follow-up/ [accessed 20 November 2011] Goedendorp M, Peters M E W J, Gielissen M F M, et al 2010 Is increasing physical activity necessary to diminish fatigue during cancer treatment? Comparing cognitive behavior therapy and a brief nursing intervention with usual care in a multicenter randomised controlled trial. The Oncologist 15(10):1122–32 Greenhalgh T 2010 How to Read a Paper, 4th edn. Wiley Blackwell, Oxford, UK Grimes D A, Schulz K F 2002 Cohort studies: marching towards outcomes. Lancet 359:341–5 Hartigan I, Murphy S, Hickey M 2011 Older adults’ knowledge of pressure ulcer prevention: a prospective quasi-experimental study. International Journal of Older People Nursing doi: 10.1111/j.1748-3743.2011.00274.x Hennekens C H, Buring J E, Mayens S L (ed) 1987 Epidemiology in Medicine. Lippincott, Williams and Wilkins, Philadelphia, UK Ho J E, Lee D S, Gona P, et al 2010 Abstract 15840: Clinical predictors of heart failure with preserved vs reduced ejection fraction. Data from the Framingham Heart Study Circulation 122: A15840 Huang C H, Su Y C, Li T C, et al 2011 Treatment of constipation in long-term care with Chinese herbal formula: a randomised, double blind placebocontrolled trial. Journal of Alternative and Complementary Medicine 17(7):639–46 doi:10.1089/acm.2010.0150 Jolly K, Ingram L, Freemantle N, et al 2011 Effect of a peer support service on breast-feeding continuation in the UK: a randomised controlled trial. Midwifery. Early view, doi:10.1016/j. midw.2011.08.005 Kitzman H J, Olds D L, Cole R E, et al 2010 Enduring effects of prenatal and infancy home visiting by nurses on children: follow-up of a randomised trial among children at age 12 years. Archives of Pediatric and Adolescent Medicine 164(5):412–18

182


7/25/2012 6:11:11 PM


Lewis D A, Sanders L P, Brockopp D Y 2011 The effect of three nursing interventions on thermoregulation in low birth weight infants. Neonatal Network: the Journal of Neonatal Nursing 30(3):160–4 Margolis J R, Gillum R F, Feinleib M, Brasch R C, Fabsitz R R 1974 Community surveillance for coronary heart disease: the Framingham Cardiovascular Disease Survey: methods and preliminary results. American Journal of Epidemiology 100: 425–36 Mater — University Study of Pregnancy 2011 Mater — University Study of Pregnancy (MUSP). Online. Available: http://www.socialscience.uq.edu.au/musp [accessed 7 November 2011] McCambridge J, Butor-Bhavsar K, Witton J, Elbourne D 2011 Can research assessments themselves cause bias in behaviour change trials? A systematic review of evidence from Solomon 4-group studies. PLoS One 6(10): e26223 doi:10.1371/journal.pone.0025223 McDonald L 2001 Florence Nightingale and the early origins of evidence-based nursing. Evidence Based Nursing 4:68–69 doi:10.1136/ebn.4.3.68 McHugh M D, Shang J, Sloane D M, Aiken L H 2011 Risk factors for hospital-acquired ‘poor glycemic control’: a case–control study. International Journal of Quality in Health Care 23(1):44–51. doi:10.1093/intqhc/mzq067 Merchant R M, Abella B S, Abots E J, et al 2010 Cell phone cardiopulmonary resuscitation: audio instructions when needed by lay rescuers: a randomised, controlled trial. Annals of Emergency Medicine 55(6):538–43.e1 Moher D, Schulz K F, Altman D G, for the CONSORT Group 2001 The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357:1191–4 Patidar A B, Andrews G R, Seth S 2011 Prevalence of obstructive sleep apnoea, associated risk factors, and quality of life among Indian congestive heart failure patients: a cross-sectional survey. Journal of Cardiovascular Nursing 26(6):452–9 Pettersson E, Vernby Å, Mölstad S, Lundborg C S 2011 Can a multifaceted educational intervention targeting both nurses and physicians change the prescribing of antibiotics to nursing home residents? A cluster randomised controlled trial. Journal of Antimicrobial Chemotherapy 66(11):2659–66 doi:10.1093/jac/dkr312

Sanada H, Nakagami G, Mizokami Y, et al 2010 Evaluating the effect of the new incentive system for high-risk pressure ulcer patients on wound healing and cost-effectiveness: A cohort study. International Journal of Nursing Studies 47(3): 279–86 doi:10.1016/j.ijnurstu.2009.08.001 Scarella A, Chamy V, Sepulveda M, Belizan J 2011 Medical audit using the Ten Group Classification System and its impact on the cesarean section rate. European Journal of Obstetrics & Gynecology and Reproductive Biology 154(2):136–40 doi:10.1016/ j.ejogrb.2010.09.005 Shields L, Mamun A, O’Callaghan M, Najman J, Williams G, Bor W 2010 Breastfeeding and obesity at 21 years: a cohort study. Journal of Clinical Nursing. 19:1612–17 doi:10.1111/j.13652702.2009.03015.x Shields L, Mamun A, Pereira S, O’Nions P, Chaney G 2011 Measuring family centred care: working with children and their parents in a tertiary hospital. The International Journal of Person Centered Medicine 1(1):155–60 Shields L, O’Callaghan M, Williams G, Najman J, Bor W 2006 Breastfeeding and obesity at 14 years: a cohort study. Journal of Paediatrics and Child Health 42(5):289–96 Shields L, Pratt J, Davis L M, Hunter J 2007 Familycentred care for children in hospital. Cochrane Database of Systematic Reviews 2007, Issue 1. Art. No: CD004811. DOI: 10.1002/14651858. CD004811.pub2 Schoenbach V J 1999 Analytic Study Designs. Online. Available: http://www.epidemiolog.net/evolving/ AnalyticStudyDesigns.pdf [accessed 15 February 2012] Thompson C 2004 Fortuitous phenomena: on complexity, pragmatic randomised controlled trials, and knowledge for evidence-based practice. Worldviews on Evidence-Based Nursing 1:9–17 Ting-Kai L, Chi-Ming L, Cheng-Jeng, C, Yueng-Ling L, Chia-Chin L 2011 A retrospective study on the long-term placement of peripherally inserted central catheters and the importance of nursing care and education. Cancer Nursing 34(1):E25–E30. doi: 10.1097/NCC.0b013e3181f1ad6f Wong F K Y, Chow S K Y, Chan T M F 2010 Evaluation of a nurse-led disease management programme for chronic kidney disease: A randomised controlled trial. International Journal of Nursing Studies 47(3):268–78

For further content associated with this chapter visit http://evolve.elsevier.com/AU/Schneider/research/

183


7/25/2012 6:11:11 PM