Crosssectional survey methods to assess

0 downloads 0 Views 213KB Size Report
a modified prior birth history method in which a representative sample of ... increase in mortality studies in humanitarian emergencies, though, we have found no ... to be logistically complicated and expensive (Tabutin, 1984). In addition ... Retrospective assessment of cause of death is often based on the responses of surviv.
doi:10.1111/j.0361-3666.2008.01085.x

Cross-sectional survey methods to assess retrospectively mortality in humanitarian emergencies K. Lisa Cairns, Bradley A. Woodruff, Mark Myatt, Linda Bartlett, Howard Goldberg and Les Roberts1

Since the rates and causes of mortality are critical indicators of the overall health of a population, it is important to evaluate mortality even where no complete vital statistics reporting exists. Such settings include humanitarian emergencies. Experience in cross-sectional survey methods to assess retrospectively crude, age-specific, and maternal mortality in stable settings has been gained over the past 40 years, and methods appropriate to humanitarian emergencies have been developed. In humanitarian emergencies, crude and age-specific mortality can be gauged using methods based on the enumeration of individuals resident in randomly selected households— frequently referred to as a household census. Under-five mortality can also be assessed through a modified prior birth history method in which a representative sample of reproductive-aged women are questioned about dates of child births and deaths. Maternal mortality can be appraised via the initial identification of maternal deaths in the study population and a subsequent investi­ gation to determine the cause of each death. Keywords: cross-sectional surveys, humanitarian emergencies, mortality

Introduction The rates and causes of mortality are critical indicators of the overall health of a popu­ lation. Whereas crude mortality rates (CMRs) reflect the overall mortality experience of the population, specific mortality indices within particularly vulnerable population subgroups, such as children less than five years of age, may be earlier indicators of a decline in the overall health of a population. Information on the causes of mortality can help target limited resources at the most important public health problems and advocate for more resources for these conditions. Data on mortality rates collected over time can monitor the impact of events, such as war or the outbreak of disease, or the effect of specific interventions, such as childhood immunisation, on the popula­ tion. Finally, mortality surveys may serve to document atrocities committed during emergency situations (Spiegel and Salama, 2000).   Because of the importance of such data, it is essential to assess mortality rates in settings without complete vital statistics reporting or death surveillance. In response to this need, demographers have done extensive work on methods to estimate mor­ tality rates in such settings; this work is reported in the older demographic literature (United Nations, 1983; 1984; Vallin, Pollard and Heligman, 1984; Hill and David, 1988). Non-demographers have used retrospective surveys to document the mortality Disasters, 2009, 33(4): 503−521. No claim to original US government works. Journal compilation © 2009 Overseas Development Institute. Published by Blackwell Publishing, 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA

504 

K. Lisa Cairns et al.

experience of populations in humanitarian emergencies (Glass et al., 1980; Boss, Toole and Yip, 1994; Goma Epidemiology Group, 1995; Roberts and Despines, 1999; Spiegel and Salama, 2000; Salama et al., 2001; Bartlett et al., 2002; Grein et al., 2003; Depoortere et al., 2004; Bartlett et al., 2005).   Humanitarian emergencies represent exceptional circumstances in which populations may be displaced, family members lose contact, record-keeping and data-gathering mechanisms cease to function, and mortality causes and rates shift dramatically over short periods. These exceptional circumstances may result in an emergency-affected, displaced population that differs in demographic profile from the stable population from which it was drawn and the host population that may receive it. Since humani­ tarian emergencies may differ so substantially from the relatively stable settings for which many retrospective approaches to assessing mortality rates were developed, the extent to which these approaches can be applied in humanitarian emergencies may be limited. Nonetheless, much of this experience remains relevant to measuring mortality rates in humanitarian emergencies.   An excellent technical discussion of the interpretation and use of mortality data in humanitarian emergencies using current methodologies was published by the Humanitarian Practice Network in 2005 (Checchi and Roberts, 2005). Despite the increase in mortality studies in humanitarian emergencies, though, we have found no broad overview in the medical literature of methodologies to estimate retrospec­ tively mortality rates in settings without vital registration or any discussion of the applicability of these methodologies to humanitarian emergencies. Such an overview would serve to orient those interested in this subject area and enable them to con­ sider how existing methods might be adapted to evaluate mortality rates in different circumstances. Furthermore, notwithstanding discussions of difficulties and biases surrounding retrospective mortality surveys (Woodruff, 2002; Woodruff and Kaiser, 2004), some of the lessons of the past have not been as thoroughly integrated into these studies as they deserve. This paper reviews basic methods that may be employed in humanitarian emergencies to determine crude, under-five and maternal mortality rates in cross-sectional surveys and provides references to more detailed descriptions of these methods.

Background Retrospective assessment of mortality through cross-sectional surveys may have two components: determining the mortality rate and determining major causes of death. Although it is occasionally possible to include the entire population in an appraisal of mortality, in most situations the population is too large. Measuring mortality rates and ascertaining major causes of death then requires an assessment of a representa­ tive sample of the population. There are four basic types of approach to measuring mortality rates in cross-sectional surveys: • The most basic method, and the first method widely applied in population-based surveys to gauge retrospectively mortality rates and causes, consists of ‘single-round’

Cross-sectional survey methods

surveys. In such surveys, surviving members of households, interviewed at a single point of time, report the aggregate number of household members and the aggre­ gate number of deaths among household members that occurred during some period in the past. However, comparisons of data derived from such methods with those obtained from prospective studies show that this approach appears to result in underestimation of mortality rates with greater omission of deaths in certain age groups, reported variously as children aged 5–15 years (Tabutin, 1984) and young children (Hill and David, 1988). Typically, the deaths most likely to be excluded are those of infants, particularly infants who survive for an extremely short time. Differential reporting by gender has also been reported (Blacker, 1984). Furthermore, deaths are frequently incorrectly included or excluded from the time frame under consideration. The cumulative effect of these errors may result in underestimat­ ing mortality rates by 30–40 per cent (Tabutin, 1984), minimising this method’s usefulness in any setting, including that of humanitarian emergencies. • Realisation of these shortcomings led in the 1970s and 1980s to the development of more complex approaches to estimating mortality rates. The second method, the ‘multi-round’ approach, uses consecutive interviews of the same individuals or individuals in the same households conducted at regular intervals to detect changes in household composition over time. This approach, described in some detail by Tabutin (1984), is particularly useful in identifying new arrivals in the household (for example, births) or departures from the household (for instance, through relocation or death). Because it relies to a greater extent on the enumera­ tor’s observations and less on the recall of the respondent, this approach is more robust than that of the single-round survey. Comparison of results from singleround and multi-round surveys conducted in the same population demonstrated that multi-round surveys led to higher detection of vital events, thus helping to elucidate some of the deficiencies of single-round retrospective surveys. However, while theoretically appealing, in practice multi-round surveys have been found to be logistically complicated and expensive (Tabutin, 1984). In addition, this method may frequently be unsuited to humanitarian emergencies because the mobil­ ity of populations, dissolution of households and evolving security concerns in these settings make the identification of previously interviewed respondents dif­ ficult. Moreover, survey results are needed urgently to address emergency health conditions; they cannot wait for multiple survey rounds. • The third approach involves the use of survivorship methods to assess mortality rates. Here, simple questions are asked of surviving relatives. For example, the number of childhood deaths may be estimated by asking women for the total number of live-born children they have ever had, and the number of these still alive. The number of adult deaths may be estimated by asking individuals about the survivor­ ship of siblings, spouses or parents according to the question to be investigated. Frequently, various assumptions and models are used to transform the responses to these questions into mortality rates (Hill, 1984). Transformation of the proportion of children still alive into survivorship estimates depends on the application of a set of multipliers to the proportion of children surviving for each five-year age

505

506 

K. Lisa Cairns et al.

group of women. This methodology, originally devised by Brass (1968), has been modified and expanded since it was originally published (Preston, 1996). One child survivorship method has been adapted for use in humanitarian emergencies (Myatt, Taylor and Robinson, 2002) and is described below. Nonetheless, many adult survivorship methods are of limited use in humanitarian emergencies for two key reasons. First, population movement lessens the likelihood that respond­ ents are aware of the status of family members with whom they no longer live. Second, questions on survivorship among adults often generate mortality rates for a period preceding the present by many years; in humanitarian emergencies such estimates are not sufficiently timely to be of use (Hill, 1984). • A fourth approach to the measurement of mortality rates is an adaptation of the basic single-round method; however, instead of asking respondents for aggregate numbers of household members and deaths, interviewers ask household respond­ ents to list each member of their households. Both the numerator and denominator of the estimated mortality rate are then based on an enumeration of individuals resident in a household, a process frequently referred to as a household census. This has emerged recently as the approach most frequently employed in humanitarian emergencies. The past period of interest is much shorter than with survivorship methods, thus providing an estimate of recent mortality rate. This approach encom­ passes several methods that are described below.   A distinction that is often made in describing methods to assess retrospectively mortality rates is that of ‘direct’ versus ‘indirect’. Survivorship methods are indirect because they rely on assumptions or models to convert collected data to estimates of mortality rates. In contrast, the other methods are direct because they allow calcu­ lation of mortality rates directly from the data collected without relying on any such assumptions (Rutenberg and Sullivan, 1991).   Retrospective assessment of cause of death is often based on the responses of surviv­ ing family members to questions contained in verbal autopsy algorithms. Because an extensive literature on verbal autopsy methodology has been published elsewhere (Kalter et al., 1990; Chandramohan et al., 1994; Marsh et al., 1995; WHO, 1995; Anker 1997; Maude and Ross, 1997; Chandramohan et al., 1998a; 1998b; WHO, The Johns Hopkins School of Hygiene and Public Health and The London School of Hygiene and Tropical Medicine, 1999; Kahn et al., 2000; Quigley et al., 2000; Chandramohan, Setel and Quigley, 2001), this methodology will not be dealt with in detail here. However, it is important to note that the sensitivity and specificity of verbal autopsies in determining specific causes of death depend on the entity studied; some causes of death have relatively pathognomonic signs or symptoms while others do not. Moreover, the positive predictive value of a specific cause of mortality as measured by verbal autopsy also depends on the prevalence of this cause of mortality in the population. Finally, verbal autopsies can frequently indicate the general cause of death, such as pneumonia, but more detailed information, such as bacteriologic aetiology of pneumonia, cannot be ascertained. Such information must be derived from special studies or laboratory-supported surveillance.

Cross-sectional survey methods

Data collection methods Measuring crude and age-specific mortality rates in the general population: household census methods The crude mortality rate is the rate of death among everyone in a specific population. Regardless of the method of data collection, CMRs are calculated from a numerator (the number of deaths), a denominator (the size of the population within which these deaths occurred), and a time element. If mortality data are collected in a survey, the numerator comes from the number of deaths reported from households included in the survey sample. The denominator is the number of persons living in these house­ holds. Other mortality rates, such as age-, sex-, or cause-specific mortality rates, are calculated by including only deaths in the numerator that occur in the specific group of interest and by including only persons in the denominator that fall into the same group. To compare mortality rates in different populations during different time periods, rates are expressed as the number of deaths occurring in a standard size of population per standard time interval. Regardless of the standard population and standard time interval used, the rate of death is the same. For example, a mor­ tality rate of 14 per 1,000 population per year is the same rate as 0.38 deaths per 10,000 per day. The denominator for mortality rates is often expressed in person-time units, thus incorporating the concept that risk of death can only be accurately assessed by considering both the size of the population studied and the time period in which the population’s risk of death is evaluated.

The recall period Measuring mortality rates retrospectively requires the precise definition of a past time period, called the ‘recall’ period. Respondents in selected households report deaths during the recall period among members of that household. The time interval of the mortality rate is the length of the recall period, and the population denominator for the mortality rate is the survey sample itself.   Figure 1 demonstrates how movements into and out of a household, such as births, deaths and migration, might relate to the recall period. Some persons are members of the household at the beginning of the recall period, while others enter it after the beginning of the recall period. Some members leave the household before the end of the recall period. During periods of social upheaval or population displacement, many households may lose or add members during a recall period.   Survey respondents must be able to conceive of and remember easily the recall period, especially its beginning point; the end point of the recall period is generally the date of interview. In cultures that do not live by the calendar, the beginning of the recall period should be a memorable historical event or a major holiday or fes­ tival. In cultures where the dates of major household events, such as the death of a household member, can be remembered precisely, the beginning of the recall period can be a relatively arbitrary point, such as six months prior to the day of the inter­ view. Nevertheless, when calculating the mortality rate the exact length of the recall

507

508 

K. Lisa Cairns et al.

Figure 1 Measuring mortality: general principles

Note: HH = Household Source: Authors’ elaboration.

period must be known. Therefore, persons analysing survey data must know both the date of the event marking the beginning of the recall period and either the date of the interview for each household or the average date of data collection for the survey.   When measuring mortality rates in a cross-sectional survey, the number of persons needed in the survey sample to achieve a given level of precision around the point estimate of the mortality rate depends on, among other factors, the length of the recall period. The longer the recall period, the more person-time units are included in the denominator with the same survey sample size. However, in humanitarian emergen­ cies, where mortality rates may be changing rapidly and public health professionals require a relatively recent estimate of mortality rates, a shorter recall period, and hence a larger sample size of households, may be preferable.

Data collection to estimate crude mortality rates The acute phase of a humanitarian emergency is often defined as that period in which the CMR is twice that of the baseline period. In the absence of an estimate of the baseline CMR, a CMR of 1/10,0000/day or above can be used as the definition of the acute phase (MSF, 1997; The Sphere Project, 2004). The CMR is selected because it may be easier to measure in emergency settings than age-specific or cause-specific mortality rates. The threshold of 1/10,000/day reflects experience in high-fertility countries in Sub-Saharan Africa. However, this figure predates the HIV/AIDS (Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome) pan­ demic; populations with high HIV/AIDS prevalence may have higher baseline crude

Cross-sectional survey methods

mortality than those without. It is important to bear in mind that the CMR is influ­ enced by the underlying age structure of the population, that is, a younger population may have a lower CMR because, in general, children, especially those older than five years of age, have a lower rate of mortality than that of elderly persons. As a result, assuming age-specific mortality rates to be equal across two populations, an older population would have a higher CMR than a younger population.   In emergency settings, several methods have been used to gather the necessary data from household respondents to calculate CMR among households selected for a sur­ vey sample. Some surveys (Roberts, 2000) have focused on only the total number of people who live in the household and the total number of deaths since the begin­ ning of the recall period. This method requires only a few questions and only a few minutes at each household. As mentioned previously, though, validation studies per­ formed in stable settings indicate that surveys that ask such summary questions generally underestimate mortality rates. Although we are unaware of comparable studies performed in emergency environments, it seems prudent to assume that sum­ mary questions could also lead to underestimates of mortality rates in such settings. Those working with survivorship methods have thought that careful enumeration of all individuals in the group of interest may decrease the number of omissions (Rutenberg and Sullivan, 1991). Furthermore, mothers should be the preferred in­ formants when asking about child mortality to decrease the likelihood that deaths among young children be differentially omitted (David, Bisharat and Hill, 1990).   Several methods have been employed to overcome this tendency to underestimate the number of deaths. One such method, sometimes called the past household census method, asks household respondents to list each person who lived in the household at the beginning of the recall period. Respondents are then asked about the current status of each of these household members. Current status can be classified as cur­ rently alive and living in the household, currently alive and living elsewhere, dead, or unknown. Respondents must also report persons who entered the household during the recall period, either by birth or by moving into the household. The numerator of the mortality rate is then the total number of household members in the sample who are reported as dead. The basic denominator is the total number of household members alive at the beginning of the recall period. However, those who left the household, either through death or by moving out, and those who entered the household, either through birth or by moving in, did not contribute an entire recall period at risk of death. If one assumes that such household departures and entrances were evenly spread throughout the recall period, each of these departing and entering household members contributed, on average, one-half of a recall period of exposure to mortality. The population denominator is then adjusted by subtracting one-half of a recall period for each household member who is no longer a member of the household at the time of the survey and adding one-half of a recall period for each person entering the household during the recall period.   Another method frequently used has sometimes been called the current household census method. Respondents are asked to list each current household member, that is, household members at the time of the survey. Respondents are then asked to list

509

510 

K. Lisa Cairns et al.

persons who died or left the household during the recall period and to identify those current household members who entered the household during the recall period. The basic population denominator of current household members is then adjusted by subtracting one-half of a recall period for persons who entered the household and adding one-half of a recall period for persons who left the household during the recall period.   A third method has recently been proposed for use in emergency settings (SMART Working Group, 2006). This method makes a simplifying assumption frequently employed in other demographic procedures to measure mortality rates: the average population during the time period of interest (the denominator of the mortality rate) is the average of the population at the beginning of the period and the end of the period. This is called the mid-interval population. In this method, a complete house­ hold census is conducted both at the time of the survey (the end of the recall period) and at the beginning of the recall period. The population denominator is then the average of the population at the beginning and the end of the recall period. Such double enumeration, although taking more time than a single household census, provides internal validation of the data from each household and may most reliably capture all individuals who resided in the household during the recall period when household splitting or dissolution are common. Nonetheless, interviewers must be especially careful not to miss deaths among newborns and infants, especially those born after the beginning of the recall period who died before the date of the survey. These children would not be counted in household censuses done at the beginning or end of the recall period.   One must keep in mind that none of these methods has been validated against a more accurate process for counting deaths, such as a death registration system with good reporting. The various arguments for and against specific data collection methods given in this and other papers are based on extrapolation of concerns regarding other methods or on strictly theoretical considerations, rather than on the results of studies.

Data collection to estimate other mortality rates Since the household census methods record all deaths in a household during the recall period, group-specific mortality rates can also be calculated if the appropriate data are collected. For example, if the age and sex of deceased and surviving house­ hold members are determined with reasonable accuracy, age- or sex-specific mortal­ ity rates can be calculated. If some estimate of the cause of each death is determined, cause-specific mortality rates can also be ascertained. However, for relatively rare causes, such as pregnancy-related death, a very large sample size is necessary to achieve a statistically precise estimate of the associated rate. Unless the survey has a sufficient sample size, even estimates pertaining to more common events such as deaths of chil­ dren aged less than five years may lack adequate precision.   If the date of each death can be gauged accurately, deaths rates for sub-intervals of the recall period can be calculated. Such calculations may be used to ‘monitor’ trends in mortality rates during the recall period. One should bear in mind, though, that data from child survivorship methods suggest that respondents tend to ‘heap’ births

Cross-sectional survey methods

and deaths, for example around the ages of six and 12 months, or dates (Hill and David, 1988). In addition, people may tend to recall traumatic events, such as the death of a family member, as having occurred more recently than they actually did (Prohaska, Brown and Belli, 1998). This would lead to a spuriously low mortality rate at the beginning of the recall period and a spuriously high rate at the end.   The household census methods suffer from several potential biases, including those associated with sampling, which are inherent in all surveys, and those connected specifically with data collection on mortality. Sampling biases occur when a nonrepresentative sample is selected. In emergencies, this may happen because of security constraints preventing survey teams from collecting data from certain locations or subpopulation. Such locations or subpopulations may have, because of increased insecurity, considerably higher mortality than the rest of the population. Another sampling bias is survivor bias, which arises because households in which all members have died cannot be selected for the survey sample or a surviving member cannot be interviewed. In populations with high mortality or within which mortality is highly clustered in households, such a bias may lead to substantial underestimation of the mortality rate.   Non-sampling biases in data collection may include intentional distortion by respondents. This may occur in emergency settings because survey respondents wish­ ing to exaggerate their plight may overstate the number of deaths in their households. Conversely, respondents in households receiving food rations calculated according to the number of household members may fail to report deaths for fear of having their quota decreased. Furthermore, recall bias may happen because traumatised populations may not remember deaths of household members during the acute emer­ gency. Persons who develop post-traumatic stress disorder may have incomplete memory of traumatic events shortly after the ordeal (Southwick et al., 1997). As a result, survey data collected during the acute humanitarian emergency may under­ estimate the recent mortality rate. In contrast, because traumatic events are often remembered as having taken place more recently than they actually did, if the begin­ ning of the recall period is not clear, deaths occurring before the recall period may be reported as having happened during the recall period. This would lead to an overestimate of the mortality rate during the recall period. To some extent, the potential for these latter biases can be minimised by the employment of a double enumeration, the use of an appropriate household respondent, and careful selection, training and supervision of fieldworkers. Moreover, questionnaire questions asking directly if anyone in the household has died since the beginning of the recall period may be more subject to intentional distortion. Utilisation of a household census method and asking only the current status of specific household members may avoid this bias to some extent. Indices of mortality in children aged less than five years

Definition and importance The age-specific mortality rate for children aged less than five years is calculated by dividing the number of deaths in children aged less than five years (numerator)

511

512 

K. Lisa Cairns et al.

by the number of children less than five years in the population at mid-year (denomi­ nator) (Srinivasan, 1998). As mentioned above, the mortality rate can be calculated easily if the age of living and deceased persons in selected households is collected. There are two additional indices of mortality in children aged less than five years that are commonly used in development situations—although regularly referred to as ‘rates’, these measures are actually probabilities. The ‘infant mortality rate’ (IMR) is the probability of dying between birth and exactly one year of age expressed in terms of per 1,000 live births. The ‘under-five mortality rate’ is the probability of dying between birth and exactly five years of age expressed in terms of per 1,000 live births (UNICEF, 2007). The IMR can be interpreted as the proportion of live-born children who die before their first birthday. Similarly, the under-five mortality rate can be interpreted as the proportion of live-born infants who die before their fifth birthday. Consequently, these measures express the cumulative risk of death before the first and fifth birthday, respectively (David, Bisharat and Hill, 1990).   Historically, under-five mortality rates have been of particular interest to public health workers. Young children are frequently the target of specific interventions (for example, Integrated Management of Childhood Illness), and mortality rates in young children are less influenced than the CMR by the age structure of the population. Furthermore, the age-specific mortality rate in children less than five years of age can be a particularly helpful index of mortality in humanitarian emergencies because young children are more vulnerable to death than older children or adolescents and there­ fore have higher mortality rates. Moreover, in some emergencies, the mortality rate for children less than five years of age may rise before the CMR, making this an earlier indicator of worsening health status (Davis, 1996).

Measuring under-five mortality rates As we have seen, it is possible to calculate the under-five mortality rate using the house­ hold census approach, although sample sizes may frequently be too small to permit precise estimates. Another family of approaches to estimating under-five mortality rates is based on the ‘indirect’ method developed by Brass (1964) in the 1960s in which women in defined age groups respond to pre-set questions on the number of children ever born alive and the number surviving. In these approaches, no information is collected on dates of births or deaths; instead, models of fertility and mortality are used to distribute these events in time and to estimate the probabilities of surviving to certain ages. Such methods are not useful in humanitarian emergencies because they depend on the simplifying assumptions, unlikely to be true in such contexts, that mortality rates have been stable over time and that the survival of children is independent of that of their mothers. Furthermore, they cannot provide reliable estimates of mor­ tality rates in the two or three years preceding the survey (United Nations, 1990).   Another approach that has been employed for many years to estimate under-five mortality rates in development contexts is the previous birth history (PBH) method. This method collects the dates of births and deaths of children from a sample of women of reproductive age and uses the data to construct life-tables for the most recent

Cross-sectional survey methods

five-year period. However, this method still gives estimates of mortality rates cen­ tred around a date preceding the survey by approximately two-and-a-half years (Hill and David, 1988; David, Bisharat and Hill, 1990)—too distant to be useful in many emergencies.   The PBH method was modified to give more recent estimates of under-five mortality rates in humanitarian emergencies (Myatt, Taylor and Robinson, 2002). In the amended version, only deaths occurring during a more recent period are considered.   Unlike some of the other methods discussed for measuring mortality rates, this one is standardised to a high level of detail, including the actual phrasing of the questions used. Data collected using these questions produce information on the number of children surviving at the beginning of the recall period (the population denominator at risk) and the number of deaths during the recall period (the numera­ tor). A step-by-step guide to collecting and analysing data using this approach can be found in Myatt, Taylor and Robinson (2002).   The PBH approach minimises underestimation of deaths and manipulation by respondents by not asking directly about deaths; information about deaths is obtained by enquiring about the general status of each child. In addition, respondents (mothers) have a single relationship to the decedents (their children), unlike other methods in which any available adult household member may report on any household deaths. Because the number of variables is small, data can be easily tallied by hand within each cluster of a cluster survey. This makes data analysis simple enough to do without a computer. As with household census methods, survivor bias may lead to under­ estimating the mortality rate. If the mother is missing or dead, no deaths can be reported for her offspring who have a much greater than average chance of dying precisely because the mother is absent (Koenig et al., 1988; WHO, 1996). This will be particularly important in situations of high maternal and under-five mortality (Myatt, Taylor and Robinson, 2002).   There have been few attempts to validate the completeness of reporting of early childhood deaths collected via the PBH approach. However, this method is the standard one currently used in most large population-based surveys that measure infant and child mortality around the world. In theory, it obtains nearly complete reporting of deaths and births. In reality, completeness of reporting depends largely on the training and quality of those engaging in data collection, the wording of ques­ tions, and cultural factors that can lead to under-reporting of the deaths of children, particularly those dying very shortly after birth. Measuring maternal mortality Traditionally, a maternal death has been defined as any death from a pregnancy or delivery-related cause either during the pregnancy or within 42 days of termination of pregnancy. More recently, this definition has been frequently modified to include deaths from any cause related to pregnancy, or its management, within one year of pregnancy outcome, irrespective of the duration or site of the pregnancy. Just as there

513

514 

K. Lisa Cairns et al.

are commonly used measurements of child mortality, the World Health Organization (WHO) notes three distinct and frequently used measures of maternal mortality: the maternal mortality ratio; the maternal mortality rate; and the lifetime risk of maternal death. The WHO definitions of these indices follow: • Maternal mortality ratio: the number of maternal deaths during a given time period per 100,000 live births during the same time period. This measures the risk of death once a woman has become pregnant. • Maternal mortality rate: the number of maternal deaths in a given period per 100,000 women of reproductive age during the same time period. This measure reflects how frequently women are exposed to mortality risk because of pregnancy. • Lifetime risk of maternal death: this measure takes into account both the probability of becoming pregnant and the probability of dying during pregnancy. It can be approximated as [1-(1-maternal mortality rate)35], where the exponent 35 represents the average number of reproductive years per woman.2   Some publications also cite a fourth key indicator: the proportion of deaths among women aged 15–49 due to maternal causes. This measure reflects both mortality risk from pregnancy and the frequency of pregnancy (ORC Macro, 2005a).   Measuring maternal mortality is even more challenging than measuring crude and early childhood mortality in humanitarian emergencies and post-conflict settings. Even where measures of maternal mortality are high, maternal deaths remain rare events. However, measuring maternal mortality allows maternal complications to be prioritised among other causes of death and, by implication, the need for maternal services to be prioritised relative to that for other aspects of health care. Furthermore, depending on the survey tools used, the specific causes and risk factors for maternal death can be identified, thus providing information to guide health services.   Three basic approaches exist to determine maternal mortality in settings with inadequate death records. The first relies on incorporating questions into a compre­ hensive national population census (ORC Macro, 2005b). However, this approach is not realistic in most humanitarian emergencies or post-conflict settings and will not be discussed further here. A second approach uses various survivorship methods to determine maternal mortality; by far the best known of these are the direct and indirect sisterhood methods (Graham, Brass and Snow, 1989; WHO, 1997; ORC Macro, 2005a). These methods are also less appropriate for humanitarian emergen­ cies for two reasons: they produce an average estimate of maternal mortality for many years in the past that may or may not be relevant to the current emergency situation; and they require sophisticated data analysis techniques with which many personnel working in humanitarian emergencies may not be familiar. The third approach is the reproductive age mortality survey (RAMOS) (ORC Macro, 2005c). Unlike the sisterhood methods, RAMOS gathers data only on women of reproductive age who are included in the survey sample. Women of reproductive age may be defined as any age group that is appropriate to the population, taking into account cultural con­ siderations. Although the age range 15–49 years is frequently used, if births among

Cross-sectional survey methods

either very young or older women are common the investigators may want to seek to expand the age range accordingly. An important component of a RAMOS is iden­ tification of the causes of maternal death.

RAMOS The RAMOS is conducted in two stages: identification of all deaths among women of reproductive age in the study population through, for example, review of burial sites, health records, census data, or household surveys, followed by further inves­ tigation to determine the cause of each death using, for instance, information from health facilities, death certificates, or verbal autopsy interviews with families of the deceased women (WHO, 1995). Because of the relative rarity of maternal deaths, data must be gathered from a very large number of households to obtain a reasonably precise estimate of maternal mortality. This requires many resources and can be logistically difficult, particularly in humanitarian emergencies. For example, in one assessment of Afghan refugees in Pakistan, deaths occurring during a 19-month period in more than 16,000 households were detected during a census. Of the 1,197 deaths, 66 were of women of reproductive age, and of these, 27 were judged to be due to maternal causes (Bartlett et al., 2002). In another such assessment, a large household survey that included 90,816 household members detected 357 deaths of women of reproductive age, of which 154 were due to maternal causes (Bartlett et al., 2005).   Data collected directly in a RAMOS allow direct estimation of proportional mortality among women of reproductive age due to reproductive causes. With infor­ mation on births and population size, the maternal mortality ratio, maternal mor­ tality rate and lifetime risk of maternal death may also be calculated (ORC Macro, 2005c). This technique has been used in two recent measurements of maternal mortality (Bartlett et al., 2002; 2005). Additional measurement methods and tools for maternal mortality have been developed in the past several years, which will merit assessing their feasibility as to usefulness in emergency settings (Graham et al., 2008).

Discussion In almost all situations, deaths can best be counted through prospective data collec­ tion that provides more recent estimates of mortality rates and does not rely on recall for completeness or accuracy. In addition, prospective death surveillance allows greater flexibility in that mortality rates for any time period can be calculated, and trends in mortality rates can be followed. Ongoing community involvement may also result in more complete mortality reporting and sensitise the community to leading causes of death. In displaced populations in relatively stable camps, death registration by community-based reporters may provide the most accurate and timely estimates of mortality (Spiegel et al., 2001). To produce an accurate point estimate of mortality, though, prospective death reporting necessitates relatively complete report­ ing of deaths and a good estimate of the population denominator. These requirements are often absent in acute humanitarian emergencies.

515

516 

K. Lisa Cairns et al.

  Although relatively recent mortality rates can be assessed retrospectively using the methods described above, surveys are unable to determine current mortality rates. Furthermore, once the survey has been completed, it is not possible to follow trends in mortality rates over time unless follow-up surveys are conducted or prospective surveillance is established. Cross-sectional surveys may also depend on relatively small sample sizes to generate mortality rates. Estimates of mortality rates derived from these studies may lack precision, particularly if the mortality rate is low. Indeed, in some situations it may not be possible to obtain a sufficiently precise estimate of the mortality rate that is useful because the sample size necessary to do so is too large to be feasible.   The different methods discussed above require varying amounts of skill and time in terms of fieldwork and data analysis. In general, methods that offer the greatest flexibility also need the greatest level of skill in terms of questionnaire preparation, interviewer training and data analysis. Surveys using the PBH and sisterhood mortality methods follow a more algorithmic approach and thus require less skill in question­ naire preparation and data collection at the expense of some flexibility. Regardless, careful fieldwork is critical to minimise data errors and omissions. Omissions of deaths have been traced to fieldworkers leaving out questions (Blacker, 1984). Fieldworkers also may be tempted to minimise work by shifting the dates of births or deaths (Potter, 1977). In a study conducted by Taylor et al. (1993) in what was Zaire, investigators were concerned that interviews with mothers did not capture all of the children who had been born to these mothers and subsequently died. The investigators hired university-educated interviewers, restricted the focus and length of the questionnaire and conducted more rigorous training and supervision. In re-interviews, additional infant births and deaths not captured in the initial interview were discovered; the authors attributed this to the changes in survey implementation that they had made. One should keep in mind that there is no substitute for thorough planning and good implementation of studies, including proper selection and adequate training of field staff and supervisors.   Although mortality data have been collected in surveys in which measuring mor­ tality was the only survey objective, it may be more efficient and timely to combine the measurement of mortality with the measurement of other health and nutrition outcomes into one survey during emergency situations. However, such attempts may produce biased mortality rates. For example, collecting mortality data in a survey meant to assess nutritional status in children less than five years of age may generate a biased estimate if only households currently containing eligible children are included in the survey sample. Such a sample would obviously not include households in which all children aged less than five years had died, thus leading to an underestimate of both CMR and under-five mortality (Myatt, Taylor and Robinson, 2002). Avoiding this potential sampling bias is important, but may not in itself justify the additional time and expense of carrying out a separate survey just to measure mortality rates if other health and nutrition outcomes are also important. Instead, survey coordi­ nators should make every effort to ensure that all households (or individuals) in the

Cross-sectional survey methods

population of interest have roughly equal probabilities of being included in the survey sample to minimise the potential for selection bias in mortality estimates. Selecting a random sample of all households in the population would produce a representative sample of the entire population for mortality measurement and a repre­ sentative sample of children less than five years of age for nutrition assessment if all eligible children in selected households were included in the nutrition assessment.   In some settings, there may be information supporting, at least in part, the credibil­ ity of cross-sectional mortality data. Where surveillance systems are newly established or grave counting is possible, the recently recalled mortality rate should be consistent with the current mortality rate if conditions have not changed substantially. The authors have personally experienced three situations in which it was possible to vali­ date, albeit incompletely, data obtained through mortality surveys done using one of the household census methods. In Goma, Zaire, in 1994, surveys in each of the three camps employed a household census method to estimate the CMR among Rwandan refugees during their first month in the country. In addition, the CMR was calculated by dividing the number of people buried in mass graves by the esti­ mated population of the three camps (Goma Epidemiology Group, 1995). The overall CMR measured by cross-sectional surveys was virtually identical to CMR calculated from the body collection system when the most probable population denominator was employed. A similar comparison was done in two villages in Sierra Leone where the village chief had recorded all deaths in a register. All of the deaths reported by survey respondents in those villages were later confirmed in the registers, dimin­ ishing the investigators’ concern that people were fabricating reports of deaths. The third situation was in the Democratic Republic of the Congo. A survey carried out during the civil war (2001) found that the areas with the highest infant mortality, as calculated from survey results, had fewer children of less than two years of age than expected in a normal population (Roberts, 2001). Thus, the age structure of the population agreed with the estimated IMR.   As mentioned above, regardless of the increased recent experience with these var­ ious methods of measuring mortality rates in cross-sectional surveys in emergencies, there are no good studies providing quantifiable measures of the validity of any of these methods. Such studies are badly needed as these methods are being applied more frequently and their results are having an increasing effect on humanitarian policy and funding. A recent paper describes many of the issues of uncertainty and makes recommendations for studies to explore them (Working Group for Mortality Estimation in Emergencies, 2007).

Conclusion A variety of methods have been developed to measure mortality retrospectively in settings without prospective death surveillance. However, many of the methods require underlying assumptions or produce an estimated mortality rate for time periods in the relatively distant past, making them less suitable for use in humanitarian emergencies.

517

518 

K. Lisa Cairns et al.

Careful fieldwork, appropriate selection of respondents, and awareness of the limi­ tations and potential sources of bias in those methods best suited for humanitarian emergencies will maximise the accuracy, and hence the usefulness, of the results from such surveys. In addition, it is important to remember that the validity of the results of mortality surveys also depends on all of the factors that must be considered when carrying out any survey, such as the sampling scheme used, careful training and supervision. Finally, studies are needed to validate the methods described against more accurate methods of counting deaths and to compare them to one another.

Correspondence K. Lisa Cairns, Global Immunization Division, National Immunization Program, Coordinating Center for Infectious Diseases, Centers for Disease Control and Pre­ vention, 1600 Clifton Road, Mail Stop E-05, Atlanta, GA 30333, United States. E-mail: [email protected].

Endnotes 1

2

K. Lisa Cairns, Medical Officer, Global Immunization Division, National Immunization Program, Coordinating Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States; Bradley A. Woodruff, Consultant in International Health and Nutrition, Beijing, China; Mark Myatt, Senior Research Fellow, University College London, London, United Kingdom; Linda Bartlett, Associate Scientist, Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States; Howard Goldberg, Associate Director for Global Health, Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Coordinating Center for Health Promotion, Centers for Disease Control and Prevention, Atlanta, GA, United States; Les Roberts, Associate Clinical Professor, Program on Forced Migration and Health, Mailman School of Public Health, Columbia University, New York, NY, United States. See http://www.reliefweb.int/library/documents/2003/who-saf-22oct.pdf.

References Anker, M. (1997) ‘The effect of misclassification error on reported cause-specific mortality fractions from verbal autopsy’. International Journal of Epidemiology. 26(5) pp. 1090–1096. Bartlett, L.A. et al. (2002) ‘Maternal mortality among Afghan refugees in Pakistan, 1999–2000’. The Lancet. 359(9307). pp. 643–649. Bartlett, L.A. et al. (2005) ‘Where giving birth is a forecast of death: maternal mortality in four districts of Afghanistan, 1999–2002’. The Lancet. 365(9462). pp. 864–870. Blacker, J.G.C. (1984) Experiences in the use of special mortality questions in multi-purpose surveys: the single round approach. Data bases for mortality measurement. United Nations, New York, NY. Boss, L.P., M.J. Toole and R. Yip (1994) ‘Assessments of mortality, morbidity, and nutritional status in Somalia during the 1991–1992 famine. Recommendations for standardization of methods’. Journal of the American Medical Association. 272(5). pp. 371–376.

Cross-sectional survey methods

Brass, W. (1964) Uses of census and survey data for estimation of vital rates. United Nations Economic Commission for Africa, Addis Ababa. Brass, W. (1968) The Demography of Tropical Africa. Princeton University Press, Princeton, NJ. Chandramohan, D. et al. (1994) ‘Verbal autopsies for adult deaths: Issues in their development and validation’. International Journal of Epidemiology. 23(2). pp. 213–222. Chandramohan, D. et al. (1998a) ‘Verbal autopsies for adult deaths: their development and validation in a multicentre study’. Tropical Medicine and International Health. 3(6). pp. 436–446. Chandramohan, D. et al. (1998b) ‘The validity of verbal autopsies for assessing the causes of institu­ tional maternal death’. Studies in Family Planning. 29. pp. 414–422. Chandramohan, D., P. Setel and M. Quigley (2001) ‘Effect of misclassification of causes of death in verbal autopsy: can it be adjusted?’ International Journal of Epidemiology. 30. pp. 509–514. Checchi, F. and L. Roberts (2005) Interpreting and using mortality data in humanitarian emergencies, a primer for nonepidemiologists. Overseas Development Institute (ODI), London. David, P.H., L. Bisharat and A.G. Hill (1990) Measuring childhood mortality: a guide for simple surveys. United Nations Children’s Fund (UNICEF), Amman. Davis, A.P. (1996) ‘Targeting the vulnerable in emergency situations: who is vulnerable?’ The Lancet. 348(9031). pp. 868–871. Depoortere, E. et al. (2004) ‘Violence and mortality in West Darfur, Sudan (2003–4): epidemiological evidence from four surveys’. The Lancet. 364(9442). pp. 1315–1320. Glass, R. et al. (1980) ‘Rapid assessment of health status and preventive-medicine needs of newly arrived Kampuchean refugees, Sa Kaeo, Thailand’. The Lancet. 1(8173). pp. 868–872. Goma Epidemiology Group (1995) ‘Public health impact of Rwandan refugee crisis: what happened in Goma, Zaire in July 1994?’ The Lancet. 345. pp. 339–344. Graham, W., W. Brass and R. Snow (1989) ‘Estimating maternal mortality: the sisterhood method’. Studies in Family Planning. 20(3). pp. 125–135. Graham W.J., S. Ahmed, C. Stanton, C.L. Abou-Zahr and O. Campbell (2008) ‘Measuring maternal mortality: an overview of opportunities and options for developing countries’. BMC Medicine. 6(12). http://www.biomedcentral.com/content/pdf/1741-7015-6-12.pdf. Grein, T. et al. (2003) ‘Mortality among displaced former UNITA members and their families in Angola: a retrospective cluster survey’. British Medical Journal. 327. pp. 650–654. Hill, A.G. and P.H. David (1988) ‘Monitoring changes in child mortality: new methods for use in developing countries’. Health Policy and Planning. 3(3). pp. 214–226. Hill, K. (1984) An evaluation of indirect methods for estimating mortality. Methodologies for the collection and analysis of mortality data. Proceedings of a Seminar at the International Union for the Scientific Study of Population (IUSSP), Dakar, Senegal, 7–10 July 1981. Kahn, K. et al. (2000) ‘Validation and application of verbal autopsies in a rural area of South Africa’. Tropical Medicine and International Health. 5(11). pp. 824–831. Kalter, H.D. et al. (1990) ‘Validation of postmortem interviews to ascertain selected causes of death in children’. International Journal of Epidemiology. 19(2) . pp. 380–386. Koenig, M.A. et al. (1988) ‘Maternal mortality in Matlab, Bangladesh: 1976–85’. Studies in Family Planning. 19(2). pp. 69–80. Marsh, D. et al. (1995) ‘Verbal autopsy in Karachi slums: comparing single and multiple causes of child deaths’. Health Policy and Planning. 10(4). pp. 395–403. Maude, G.H. and D.A. Ross (1997) ‘The effect of different sensitivity, specificity and cause-specific mortality fractions on the estimation of differences in cause-specific mortality rates in children from studies using verbal autopsies’. International Journal of Epidemiology. 26(5). pp. 1097–1106. MSF (Médecins sans Frontières) (1997) Refugee Health: An Approach to Emergency Situations. McMillan, London.

519

520 

K. Lisa Cairns et al.

Myatt, M.A., A. Taylor and W.C. Robinson (2002) ‘A method for estimating mortality rates using previous birth history’. Field Exchange. 17. pp. 13–15. ORC Macro (2005a) DHS Maternal Mortality Module. http://www.cpc.unc.edu/measure/publications/ html/ms-02-09-tool27.html (accessed 1 August 2007). ORC Macro (2005b) Measuring Maternal Mortality from a Census: Guidelines for Potential Users. http:// www.cpc.unc.edu/measure/publications/html/ms-02-09-tool11.html (accessed 1 August 2007). ORC Macro (2005c) Tool Name: Reproductive Age Mortality Survey (RAMOS). http://www.cpc.unc.edu/ measure/publications/html/ms-02-09-tool20.html (accessed 30 December 2005). Potter, J.E. (1977) ‘Problems in using birth-history analysis to estimate trends in fertility’. Population Studies. 31(2). pp. 335–364. Preston, S.H. (1996) ‘Population studies of mortality’. Population Studies. 50(3). pp. 525–536. Prohaska, V., N.R. Brown and R.F. Belli (1998) ‘Forward telescoping: the question matters’. Memory. 6(4). pp. 455–465. Quigley, M.A. et al. (2000) ‘Validity of data-derived algorithms for ascertaining causes of adult death in two African sites using verbal autopsy’. Tropical Medicine and International Health. 5(1). pp. 33–39. Roberts, L. (2000) Mortality in Eastern DRC: Results from five mortality surveys. International Rescue Committee, New York, NY. Roberts, L. (2001) Mortality in eastern Democratic Republic of Congo: Results from Eleven Mortality Surveys. International Rescue Committee, New York, NY. Roberts, L. and M. Despines (1999) ‘Mortality in the Democratic Republic of the Congo’. The Lancet. 353(9171). pp. 2249–2250. Rutenberg, N. and J. Sullivan (1991) Direct and indirect estimates of maternal mortality from the sisterhood method. The Demographic and Health Surveys World Conference, Washington, DC, 5–7 August. IRD/MACRO International Inc., Columbia, MD. Salama, P. et al. (2001) ‘Malnutrition, measles, mortality and the humanitarian response during a famine in Ethiopia’. Journal of the American Medical Association. 286(5). pp. 563–571. SMART Working Group (2006) Measuring Mortality, Nutritional Status and Food Security in Crisis Situations: SMART Methodology. United States Agency for International Development (USAID), Washington, DC. Southwick, S.M. et al. (1997) ‘Consistency of memory for combat-related traumatic events in vet­ erans of operation desert storm’. American Journal of Psychiatry. 154. pp. 173–177. Spiegel, P.B. and P. Salama (2000) ‘War and mortality in Kosovo, 1998–1999: an epidemiological testimony’. The Lancet. 355(9222). pp. 2204–2209. Spiegel, P.B. et al. (2001) ‘The accuracy of mortality reporting in displaced persons camps during the post-emergency phase’. Disasters. 25(2). pp. 172–180. Srinivasan, K. (1998) Basic Demographic Techniques and Applications. Sage Publications India Pvt Ltd., New Delhi. Tabutin, D. (1984) Comparison of single and multi-round surveys for measuring mortality in developing countries. Methodologies for the collection and analysis of mortality data. Proceedings of a Seminar at the IUSSP, Dakar, Senegal, 7–10 July 1981. Taylor, W.R. (1993) ‘Mortality and use of health services and surveys in rural Zaire’. International Journal of Epidemiology. 22 (Supplement 1). S15–S19. The Sphere Project (2004) Humanitarian Charter and Minimum Standards in Disaster Response. The Sphere Project, Geneva. UNICEF (United Nations Children’s Fund) (2007) Definitions: basic indicators. http://www.unicef.org/ infobycountry/stats_popup1.html (accessed 17 October 2008). United Nations (1983) Manual X: Indirect techniques for demographic estimation. United Nations, New York, NY.

Cross-sectional survey methods

United Nations (1984) Data bases for mortality measurement. United Nations, New York, NY. United Nations (1990) Step-by-step guide to the estimation of child mortality. United Nations, New York, NY. Vallin, J., J.H. Pollard and L.E. Heligman (1984) Methodologies for the collection and analysis of mortality data. IUSSP, Dakar. WHO (World Health Organization) (1995) Verbal autopsies in maternal deaths. WHO, Geneva. WHO (1996) Mother–baby package: implementing safe motherhood in countries. WHO, Geneva. WHO (1997) The sisterhood method for estimating maternal mortality: guidance for potential users. WHO, Geneva. WHO, The Johns Hopkins School of Hygiene and Public Health and The London School of Hygiene and Tropical Medicine (1999) A standard verbal autopsy method for investigating causes of death in infants and children. WHO, Geneva. Woodruff, B.A. (2002) ‘Measuring mortality in cross-sectional surveys: a commentary’. Field Exchange. 17. p. 16. Woodruff, B.A. and R. Kaiser (2004) ‘Violence and mortality in West Darfur’. The Lancet. 364(9442). pp. 1290–1291. Working Group for Mortality Estimation in Emergencies (2007) ‘Wanted: studies on mortality esti­ mation methods for humanitarian emergencies, suggestions for future research’. Emerging Themes in Epidemiology. 4. p. 9.

521