Integrating Quantitative and Qualitative Methods to Assess the Impact ...

10.1177/109019802237025 Lindsay / Program Evaluation Health Education & Behavior (October 2002)

Integrating Quantitative and Qualitative Methods to Assess the Impact of Child Survival Programs in Developing Countries: The Case of a Program Evaluation in Ceara, Northeast Brazil Ana Cristina Lindsay, DDS, MPH, DrPH

In evaluating public health programs, the tradition has been to design quantitative approaches, relying on epidemiological and statistical techniques to determine if and to what extent a program has an effect on a predetermined targeted population. More recently, however, qualitative methods such as rapid ethnographic assessments and focus groups have been implemented more frequently. This article describes an outcome evaluation of a community health workers program that integrated quantitative and qualitative methods to assess the impact of child survival interventions in reducing infant mortality and inadequate weight gain in children among municipalities in the state of Ceara, Northeast Brazil. By using multiple methods that combine quantitative and qualitative components, researchers can broaden their understanding of complex public health issues and direct use of data for decision making.

During the past decade, evaluation of public health programs has received considerable attention. The approach of the new millennium has forced many programs and governments to demonstrate the impacts of their health and nutrition interventions on child survival and development and monitor the achievement of “Healthy People 2000” goals.

Ana Cristina Lindsay, Department of Nutrition, Harvard School of Public Health, Boston. Address reprint requests to Ana Cristina Lindsay, Public Health Nutrition, Department of Nutrition, Harvard School of Public Health, 665 Huntington Avenue, Boston, MA 02115; phone: (617) 432-0983; fax: (617) 432-2435; e-mail: [email protected]. The evaluation research presented in this article was supported in part by the School of Public Health of Ceara (Northeast Project/World Bank); the National Council for Research and Development (CNPq), Brazil; and UNICEF/Fortaleza, Ceara Office. Additional funding for Dr. Lindsay was provided by the International Nutrition Foundation for Developing Countries (INFDC), the Margaret McNamara Fellowship for Women From Developing Countries, the David Rockefeller Center for Latin American Studies, and the Department of Maternal and Child Health of the Harvard School of Public Health. The Program Evaluation Fellowship of the Harvard Children’s Initiative, Harvard University, supported the development of this article. The author thanks the other research team members involved in design and implementation of the three evaluation studies of the Ceara’s Health Workers’Program (P.A.C.S.), Dr. Alberto Ascherio, Dr. Karen Peterson, and Dr. Jane Gardner of the Harvard School of Public Health; Dr. Ennio Cuffino of UNICEF/Ceara; Dr. Francisca Andrade, Mrs. Maria Ines Amaral, and Mrs. Maria Vilane Craveiro of the Ceara State Secretariat of Health. I am also grateful to Elizabeth Peterson for her review and editing of this manuscript. Health Education & Behavior, Vol. 29 (5): 570-584 (October 2002) DOI: 10.1177/109019802237025 © 2002 by SOPHE

570

Lindsay / Program Evaluation

571

In evaluating programs, quantitative or qualitative paradigms may be followed. Given that the ultimate goal of most child health programs is to achieve a health impact, defined as a reduction in mortality and morbidity and an improvement in health status, projects are required to measure their progress in terms of quantifiable goals.1-3 Therefore, in evaluating health and nutrition programs, researchers have relied on quantitative approaches, counting on epidemiological and statistical techniques to determine if and to what extent a program has an effect on a predetermined target population. Different designs including randomized experiments and observational studies (cohort, case control, cross-sectional, and ecological studies) are used within the quantitative paradigm to collect numerical data on a sample of the population.1-3 Furthermore, in developing countries, major funders of public health research have emphasized “epidemiological research.” In this context, international agencies such as the World Health Organization, UNICEF, and USAID and their respective selected programs in tropical disease research, GOBI-FFF (growth monitoring, oral rehydration, breast-feeding, immunization, family planning, female literacy, and food supplementation), and the Combating Childhood Communicable Diseases Program have emphasized the use of case control studies.1-3 These studies dichotomize exposures; outcome measurements; and knowledge, attitude, and practice (KAP) surveys, all of which lack qualitative components. More recently, however, qualitative methods involving more rapid data collection techniques, such as rapid ethnographic assessment and focus groups have been implemented more frequently in public health evaluation and research.4-10 The inherent strengths and weaknesses of quantitative and qualitative research approaches complement each other. Many researchers have suggested that combining both methods will be advantageous for public health research.11-16 Quantitative methods that evaluate the outcomes of child survival programs may not provide the information that program planners and decision makers need to design interventions, to modify programs, and to develop policies that consider the sociocultural environment in which their programs take place.9,15,17 Although quantitative methods may illuminate which programs perform successfully and which interventions are associated with the observed outcomes, Stange and Zyzanzki18 suggest that it may be equally important to know what determines a program’s success and how those who manage and participate in it think and behave. To answer these questions, one must understand how individuals interpret the world around them and how they construct meanings, generate motives, and develop relationships. In this case, the evaluation of child health programs would benefit from the use of open-ended interviews; focus groups; observations; and other qualitative research techniques designed to understand community and individual knowledge, beliefs, preferences, and behaviors, as well as the economic and sociodemographic characteristics necessary for planning, implementing, monitoring, and evaluating public health interventions. In this evaluation, qualitative methods were used to strengthen our understanding of the quantitative results, including gaining a broad understanding of the causes and circumstances of infant deaths, generating local explanatory mechanisms for infant mortality, and identifying potential factors associated with infant survival amenable to interventions. This article illustrates how quantitative and qualitative methods were integrated to evaluate the outcomes of a child survival program in Ceara, Northeast Brazil. The aims of the article are to (1) demonstrate how quantitative techniques (e.g., ecological analysis and cross-sectional interviews) and qualitative techniques (e.g., in-depth interviews or

572

Health Education & Behavior (October 2002)

verbal autopsy) were used in the study and (2) show how the combination of these two methods yielded information necessary for the design and improvement of programs and policies related to child health in Ceara.

BACKGROUND In 1995, an outcome evaluation of Ceara’s Health Workers’ Program (P.A.C.S.)/child survival program was undertaken in collaboration with the State Secretariat of Health (SESA), UNICEF, and the Harvard School of Public Health. Since early program implementation, SESA routinely collected data for monitoring purposes. However, no systematic evaluation had been conducted. Realizing the need to document the impact of the program, SESA and UNICEF requested assistance from a group of researchers from the Harvard School of Public Health to design an impact evaluation. The goals for the evaluation were as follows: (1) to achieve a broad diagnosis of the determinants of infant mortality and prevalence of inadequate weight gain that interplay at the community level and (2) to describe circumstances of postneonatal deaths (1-11 months of age) and health-careseeking behaviors of mothers to identify factors amenable to intervention. Brief History of the P.A.C.S./Child Survival Program In 1987, responding to one of the most severe droughts in Ceara’s history, the Ceara SESA recruited a number of community members to promote health-benefiting behaviors during the disaster. The program known then as “Frente da Seca” created jobs for approximately 7,300, mostly female, community health workers and 235 half-time nurse supervisors.19 The 1987 drought represented an important opportunity “to work at the grassroots level with communities to improve their health and living conditions.”19 In 1988, after the success of the emergency initiative, the state government officially created P.A.C.S. to improve the ability of community members to meet their own health needs.19 The P.A.C.S. program is now implemented on a municipal basis throughout the state of Ceara, covering both urban and rural areas. When the evaluation began, the program operated in 170 of the 184 municipalities. The program now covers all 184 municipalities in the state and employs approximately 7,953 community health workers in the interior and 1,000 in the capital city of Fortaleza. Every month, each of P.A.C.S.’s community health workers (CHWs) visits the homes of 50 to 250 assigned families to provide health and nutrition education. CHWs weigh infants and children and collect information on health indicators. Pregnant women are referred to Health Units for prenatal care, as are sick children in need of treatment. CHWs report information monthly to a supervisor, who aggregates the data at the municipality level.20 To assess the impact of specific child survival interventions on infant mortality rates and the prevalence of undernutrition, three investigations were undertaken that involved using quantitative and qualitative methods. The first three investigations used an ecological design to examine the determinants of variations in infant mortality rates,21 diarrheaspecific infant mortality rates,22 and prevalence of inadequate weight gain in children 0 to 23 months of age23 among municipalities in the state of Ceara during the 30-month period of January 1994 to June 1996. The objectives of the ecological studies were to (1) address the relative contribution of infant growth, breast-feeding, water supply and sanitation, poverty, female illiteracy, and utilization of prenatal and child care to differences in infant mortality rates among municipalities in Ceara, Northeast Brazil and (2) examine the rela-


573

tive contributions of behavioral, health services, and socioeconomic variables to variations in prevalence of inadequate weight gain in infants and young children among municipalities in Ceara. The third investigation was a verbal autopsy study,24 combining quantitative and qualitative research methods. This study sought to identify and examine 1. 2. 3. 4.

factors associated with preventable postneonatal deaths, circumstances of infant deaths, timing and place of infant deaths, and mothers’ health-care-seeking behaviors during their infants’ fatal illnesses.

METHOD Ecological Studies21-23 One hundred and forty municipalities that reported complete information every month from January 1994 to June 1996 were included in the analyses. Data before January 1994 were incomplete for several municipalities. Data for these studies were obtained from the 1991 census for the state of Ceara and P.A.C.S. Table 1 presents the variables included in the infant mortality, diarrhea-specific mortality, and inadequate weight gain studies. More detailed definitions of P.A.C.S. and census variables are provided elsewhere.21-23 Data Analysis Analyses were conducted using simple and multiple linear regression (in the infant mortality and diarrhea-specific mortality studies weighted by number of live births). The statistical significance of individual covariates was assessed using the Wald test. Residual diagnostics and influence statistics were used to confirm that the regression assumptions were satisfied. All p values were two-tailed and were considered significant when less than .05. For the infant mortality study only, we used the regression coefficient of the final model to calculate ecological relative risks to compare our results with those obtained in cohort and case-control studies of the same risk factors. All data management and analyses were carried out using SAS/PC version 6.12. Verbal Autopsy Study24 The field research was conducted by the principal investigator (Dr. Ana Cristina Lindsay) and the senior researcher (Dr. Alberto Ascherio). The study population comprised all mothers of infants who had died during the previous 12 months (June 1995-May 1996) in 11 municipalities in the state of Ceara, Northeast Brazil. The 11 municipalities are distributed across the state’s 14 health departments, 6 of which are located along the state’s coastal area, 4 in the “sertao” (dry area), and 1 in the metropolitan area. A map of the state of Ceara with the 11 municipalities is provided elsewhere.24 As part of the P.A.C.S. program, all infant deaths and names and addresses of mothers are reported monthly to a supervisor. To select the municipalities to be included in the survey, we first ranked all the 184 municipalities of the state according to their infant mortality rates and then chose the highest 6 and the lowest 5 municipalities among those that had

574


Table 1.

b

Univariate Statistics for All Variables Included in Analysis of Ecological Study of Determinants of Variations in Infant Mortality Rates, Diarrhea-Specific Mortality Rates, and Prevalence of Inadequate Weight Gain in Children Among 140 Municipalities in the State of Ceara, Northeast Brazil, 1994-1996 M

P.A.C.S. Data (1994-1996) Dependent variable 60.0 Infant mortality ratea Diarrhea-specific infant mortality rate 14.5 % of infants with inadequate weight gain 28.1 % of children with inadequate weight gain 34.9 Behavioral variable % of infants exclusively breast-fed in the first 4 months of life 41.7 Health services variable % of infants participating in growth monitoring 78.3 % of children participating in growth monitoring 77.8 % of infants with immunization up-to-date 78.9 % of pregnant women with prenatal care up-to-date 71.3 Census data (1991) Socioeconomic variable % of households with inadequate water supply 78.6 % of households with inadequate sanitation 98.2 Female illiteracy rate (women 15-44 years) 43.0 % of households with low income 74.9 Per capita gross municipality product (GMP) 362.5 % urbanization 43.8

SD

Minimum

Maximum

20.8 8.8 10.1 9.3

4.6 0 8.9 10.8

143.7 43.7 61.6 61.9

9.3

23.3

69.3

12.4

42.7

99.6

12.9

42.3

99.6

12.3

38.6

96.5

13.1

27.7

91.6

13.7

26.8

99.8

5.6 7.1 9.5

50.4 20.8 44.5

100.0 62.3 93.0

676.1 18.1

4.0 11.4

601.7 100.0

NOTE: P.A.C.S. = Community Health Workers’ Program. a. Rate per 1000 live births.

reported at least 15 deaths during the previous 2 years. Contacts with the supervisors in the selected municipalities were made through the P.A.C.S. headquarters. Visits to the selected municipalities took place during the months of June, July, and August of 1996. Families of the deceased infants were located, explained the purpose of the study, and asked to participate. All families who were located agreed to participate in the study. The data were collected in a uniform manner during individualized interviews conducted in the household with mothers by two of us (Lindsay, Ascherio) accompanied by the P.A.C.S. CHW or supervisor responsible for the area. In 9 (7%) of the 127 households, the mother was not available, and we interviewed the closest relative (i.e., eight grandmothers and one aunt). A total of 127 interviews were conducted with mothers who had lost their infants during the period of June 1995 to May 1996 in 11 municipalities in the state of Ceara. Data


575

for the verbal autopsy study were collected using both quantitative and qualitative research techniques. For the collection of quantitative data, a questionnaire including both closed- and open-ended questions on causes and circumstances of death, on use of indigenous remedies, and on all the contacts with health care providers that occurred during the infant’s terminal illness was used. The qualitative data were collected through indepth interviews using a semistructured format conducted with mothers or child caretakers. Respondents were asked to tell us about the illness that led to the infant’s final illness, and the timing and place of death. Prompts were used if respondents did not spontaneously cover these topics. All interviews were conducted in Portuguese by one of the investigators, a public-health-trained Brazilian woman (Dr. Ana Cristina Lindsay), taking into account lay concepts and local terminology. Data Analysis Quantitative. Analysis of the quantitative data was conducted in two phases. First, we calculated descriptive statistics to characterize demographic and socioeconomic characteristics, use of health services, perceived severity of illness, main cause of death, and mother’s satisfaction with health care for the whole sample and stratified by municipalities with high infant mortality rates (high IMR, Group 1) and low infant mortality (low IMR, Group 2). In the second phase of our analysis, we used the mothers’ report of the first care providers, which were either traditional or lay providers (i.e., traditional healer or lay pharmacist/drug vendor) or biomedical providers (i.e., those who are part of the official public health care system including doctors, nurses, and P.A.C.S. CHWs). The chisquare test was used to identify significant factors (p < .05) independently associated with mothers’ choice of care provider during their infant’s final illness. Qualitative. All verbal autopsy interviews were translated and transcribed from Portuguese to English by one of the investigators (Dr. Ana Lindsay). The transcripts were analyzed inductively to identify recurring themes, from which we generated a list of categories of themes based on the most frequent answers cited by respondents. The qualitative data were coded into three main categories of problems: delay in seeking medical care; delay in receiving medical care; and timely, but ineffective care. The analysis and interpretation of facts were grounded on mothers’ accounts and based on the judgment of one of the investigators (Dr. Ana Lindsay). More detailed information about data management and analyses are provided elsewhere.24

RESULTS Ecological Studies Infant Mortality and Diarrhea-Specific Mortality To select a final predictive model, we fit models containing all the significant predictors from the reduced regression models for proximate health services and socioeconomic determinants. In Tables 2 and 3, we present the proximate health services and socioeconomic models for the determinants of infant mortality rates and diarrhea-specific mortality rates. The final selected model for infant mortality included percentage of

576

Table 2.

Regression Model of the Determinants of Variations in Infant Mortality Rates Among 140 Municipalities in the State of Ceara, Northeast Brazil, 1994-1996 % Exclusively Breast-Feeding β

SE (β)

–.38*** –.38*** –.41*** –.39*** –.32**

(.14) (.14) (.14) (.14) (.15)

Model 1 2 3 4 5

% Households With Inadequate Water Supply

1 2 3 4 5

% Prenatal Care Up-to-Date β –.47**** –.51**** –.49**** –.50**** –.49****

Female Illiteracy Rate

% Households With Low Income

SE (β)

β

SE (β)

β

SE (β)

(.11) (.11) (.11) (.11) (.11)

1.16**** 1.09**** 1.17**** 1.09**** 1.07****

(.25) (.25) (.25) (.25) (.25)

–.56*** –.61*** –.57*** –.71**** –.61****

(.20) (.20) (.20) (.20) (.20)

% Urbanization

Per Capita Gross Municipality Product (GMP)

β

SE (β)

β

SE (β)

β

SE (β)

F

R2

p Value

(.09) (.11) (.09)

(.002)

(.11)

.12 .18* .11

–.003

.17

.20*

(.11)

.21*

(.11)

–.003

(.002)

14.3 14.6 16.8 20.6 13.0

39.3 39.7 38.6 37.9 40.9

.0001 .0001 .0001 .0001 .0001

*p ≤ .05. **p < .01. ***p < .001. ****p < .0001.


577

exclusively breast-fed infants, up-to-date prenatal care, low household income, female illiteracy rate, inadequate water supply, urbanization, and per capita gross municipality product (GMP) (Table 2). It explained approximately 41% of the variance in municipalitylevel infant mortality rates. From this model, we calculated ecological relative risk, which was interpreted as the relative risks comparing a child born in a community with a 100% prevalence of the determinant of interest with a child born in a community with a 0% prevalence. These relative risks (RRs) were 0.62 (p = .02) for exclusively breast-feeding, 0.43 (p = .0001) for prenatal care up-to-date, 2.24 (p = .0001) for female illiteracy rate, 1.25 (p = .06) for inadequate water supply, and 1.23 (p = .05) for urbanization (not shown). On the other hand, approximately 34% of the variance in municipality-level diarrhea-specific infant mortality rates was explained by the prevalence of infants exclusively breast-feeding, percentage of infants with adequate weight gain, percentage of pregnant women with prenatal care up-to-date, female illiteracy rate, and inadequate water supply and urbanization (Table 3). Prevalence of Inadequate Weight Gain During the 30-month study period, the average municipality-level prevalence of inadequate weight gain in infants and young children was 28.1% and 34.9%, respectively. The results from the final predictive model for the prevalence of inadequate weight gain in infants and young children are shown in Table 4. Findings showed that participation in growth monitoring and urbanization were inversely and significantly associated with the prevalence of inadequate weight gains in infants, accounting for 38.3% of the variance. Female illiteracy rate, participation in growth monitoring, and degree of urbanization were all positively associated with prevalence of inadequate weight gains in children. Together, these factors explained 25.6% of the variation (Table 4). Verbal Autopsy (VA) Study As shown in Table 5, 59.9% (n = 76/127) of the infants died at home. Most of these children had been admitted to a hospital (n = 27/76) or examined at least once in a health center during the disease episode that caused death (n = 33/76). Among children admitted to a hospital (i.e., 27), 14 died within 3 days of having been medically discharged, and 10 died after leaving the hospital without medical authorization. Sixty-four percent (28/44) of the in-hospital deaths occurred 48 or more hours after admission. Results from the qualitative methods (i.e., in-depth interviews with mothers of deceased infants) illustrated the importance of carefully examining the behavioral aspects of maternal health care seeking during their infant’s illnesses. The analysis of the in-depth interviews revealed that one-third of the deaths occurred in a hospital and twothirds at home (Table 5). Almost all the infants who died at home, however, had been examined one or more times by a doctor, and 36% of them had been hospitalized during the disease episode that resulted in death. For most (85%) of these children, the causes of death were diarrhea or acute respiratory infection, and it is likely that death could have been averted if appropriate treatment had been initiated promptly (not shown). Three major groups of factors that alone or in combination appeared to contribute to most deaths were delays in seeking medical care on behalf of the parents, medical interventions reported as ineffective by mothers, and delays in providing medical care to children who arrived at the hospital too late in the day to be scheduled for consultation (Table 5).

578

Table 3.

Regression Model of the Determinants of Variations in Diarrhea-Specific Infant Mortality Rates Among 140 Municipalities in the State of Ceara, Northeast Brazil, 1994-1996 % Infants Exclusively Breast-Feeding β

SE (β)

–.27*** –.23*** –.23*** –.22** –.21** –.19** –.21**

(.06) (.07) (.07) (.07) (.07) (.07) (.06)

Model 1 2 3 4 5 6 7

Households With Inadequate Water Supply β 1 2 3 4 5 6 7 *p ≤ .05. **p < .01. ***p < .001.

SE (β)

.05 (.04) .05 (.04) .19*** (.05) .18*** (.04)

% Infants With Adequate Weight Gain β –.15** –.13** –.12 –.13* –.12 –.15* –.17**

SE (β)

β

SE (β)

(.06) (.06) (.06) (.06) (.06) (.06) (.06)

–.14*** –.11* –.11* –.11* –.11* –.12*

(.05) (.05) (.05) (.05) (.05) (.04)

% Urbanization β

SE (β)

.21*** (.04) .20*** (.04)

% Pregnant Women With Prenatal Care Up-to-Date

Female Illiteracy Rate% β

.12 .04 .03 .22* .23*

SE (β)

(.08) (.10) (.10) (.11) (.11)

Per Capita Gross Municipality Product (GMP) β

SE (β)

–.0008 (.001) –.001 (.0009)

F

R2

p Value

11.9 11.6 9.3 7.7 6.5 10.2 11.4

14.8 20.4 21.6 22.3 22.7 35.1 33.9

.0001 .0001 .0001 .0001 .0001 .0001 .0001

Table 4.

Regression Model of the Determinants of Variation in the Prevalence of Inadequate Weight Gain in Infants (0-11 months) and in Children (12-23 months) Among 140 Municipalities in the State of Ceara, Northeast Brazil, 1994-1996 % Participating Growth Monitoring

Model Infants 1 2 3 4 5 6 Children 1 2 3

β

Female Illiteracy Rate β

SE (β)

SE (β)

–.41**** –.43**** –.42**** –.43**** –.45**** –.47****

(.06) (.05) (.06) (.05) (.05) (.05)

.31*** (.09)

–.27**** –.29**** –.29****

(.05) (.05) (.05)

.30*** (.09) .19* (.11) .22* (.11)

.15

*p < .10. **p < .05. ***p < .01. ****p < .001.

(.12)

% Households With Low Income β

SE (β)

.22**** .19** .25**** .12

(.07) (.09) (.07) (.09)

Per Capita Gross Municipality Product (GMP) % Urbanization β

SE (β)

–.001 (.001) –.001 (.001)

–.002* (.001) –.002 (.001)

β

F

R2

p Value

–.11** (.05) –.15**** (.04)

37.2 41.4 26.9 27.2 22.1 42.5

35.2 36.7 37.3 37.5 39.6 38.3

.0001 .0001 .0001 .0001 .0001 .0001

–.08* –.08*

15.3 12.5 15.6

25.2 27.1 25.6

.0001 .0001 .0001

SE (β)

(.04) (.04)

579

580


Table 5.

Categories and Subcategories of Health Care–Related Themes in Maternal Reports of Circumstances Associated With Infants’ Deathsa and Classification of Postneonatal Deaths According to Timing and Place, Ceara, Northeast Brazil (N = 127)

Health care–related themes in maternal reports of circumstances associated with infants’ death Delay in seeking health care Sought care from traditional/lay provider sourcesb Tried to treat infant at home Transportationc Maternal popular beliefs Lack or poor recognition of disease severity Lack of confidence in medical care available Lack of child care Delay in receiving medical care Arrived on time for consultation but could not get appointment Infant needed treatment that was not available locally (transferred) Arrived after consultation hours, did not get emergency care Ineffective medical care Poor communication between health care professional and mother Consulted several times with health provider during fatal illness, never admitted to hospital Unavailability of needed care Timing and place of deaths At home After medical discharge of the same disease episode After medical discharge, but not of the same disease episode After leaving health care facility without medical authorization Did not receive ambulatory care during disease episode because mother did not want to take infant to a health center or hospital Received ambulatory care, but was not referred to the hospital In the hospital Forty-eight hours after admittance Less than 48 hours after admittance On way to the hospital

%

N

70.3 50.0 39.1 25.8 25.8 15.6 7.8 6.3 13.3 6.3 6.3 0.8 43.0 39.8

90 64 50 33 33 20 10 8 17 8 8 1 55 51

7.8 7.1

10 9

59.9 11.0 2.4 7.9

76 14 3 10

12.6 15.7 33.8 22.0 12.6 5.5

29 20 43 28 15 8

a. It was likely that infants’ circumstances of deaths were associated with a number of factors that were interrelated. Factors were recorded independently, and more than one main category or subcategory of factors could be assigned as related to infants’ circumstances of death. b. Traditional healer and local pharmacy. c. Unavailable or too costly.

DISCUSSION In this outcome evaluation of a CHW program, we integrated quantitative and qualitative methods to assess the impact of child survival interventions in reducing infant mortality including diarrhea-specific infant mortality and inadequate weight gain in children among municipalities in the state of Ceara, Northeast Brazil. The results of the infant mortality study suggested that promotion of exclusive breast-feeding and increased prenatal care utilization, as well as investments in female education, might have substantial


581

positive effects in further reducing infant mortality rates in the state of Ceara,21 whereas the findings from the diarrhea-specific mortality study suggested that community-based promotion of exclusive breast-feeding in the first 4 months of life and caregiving behaviors that prevent weight faltering, including weaning practices and feeding during and following diarrhea episodes, may further reduce municipality-level diarrhea-specific mortality. On the other hand, the results of the nutrition study suggest that efforts to reduce the average municipality female illiteracy rate of 43%, in combination with participation in growth monitoring, may prove to be effective in reducing municipality-level prevalence of inadequate weight gain in infants and children in Ceara.23 The findings from the verbal autopsy study suggested that government efforts to further reduce infant mortality in Ceara should focus on health education interventions that address quality of home care, recognition of signs of severity and danger, the importance of seeking timely medical care, and improving the quality of care provided at community health centers and hospitals.24 Measures to ensure prompt access to medical consultation to young children brought to health centers or hospitals with potentially life-threatening conditions, such as diarrhea and pneumonia, are needed. Health education that ensures that mothers understand the need for continuing treatment after discharge, are able to return to the medical care facility if the child does not recover, and have access to prescribed medicine is also likely to improve infants’ chance of survival. Further benefits could be obtained by using CHWs, now integrated into the Family Medicine Program (PSF) health teams, to provide health education, to supervise home care, to refer mothers to health centers, and to facilitate their access to hospitals. More detailed information on the findings, limitations, and strengths of the verbal autopsy study can be found elsewhere.24 Many researchers have indicated the importance of integrating quantitative and qualitative methodologies to research complex and multidimensional public health problems such as infant mortality.11-16 In this evaluation, two reasons compelled us to combine methods. First, a triangulation approach, combining quantitative and qualitative research methods to study the same problem, provides a powerful means for analysis and interpretation of data in that both common and unique variances can be explained.11,12,16,25,26 Information generated from multiple approaches (triangulation) guarantees diversity of findings and perspectives. In this evaluation, the quantitative method used (i.e., ecological analysis and cross-sectional survey) yielded information about infant mortality rates (including diarrhea-specific mortality) and prevalence of inadequate weight gain, as well as the factors associated with variations in these rates among municipalities. In contrast, the use of an “anthropological approach” yielded information about health decision–making processes and an understanding of the disease or illness experience within a specific cultural context. Qualitative methods were used as a follow-up to the conclusions from the ecological data analysis (i.e., infant mortality, diarrhea-specific infant mortality, and inadequate weight gain) and supplemented the quantitative research by exploring complex phenomena in more depth and enhancing the understanding of the quantitative results. For example, the quantitative methods showed a significant association between female literacy level and infant mortality rates. This finding, although important, does not provide information into the potential mechanisms through which mother’s educational status influences child survival. In contrast, the findings from the qualitative methods showed that one of the mechanisms through which mother’s education influences an infant’s chance of survival is the choice of treatment of life-threatening illnesses. These findings combined suggest the importance of maternal education to child care practices and may be used to design interventions that improve child health care practices that can

582


ultimately reduce the mortality from both diarrhea and acute respiratory infections. For example, health education messages should appeal to caretakers’ concerns about specific symptoms associated with illnesses such as persistence, duration, vomiting, fever, and blood in stools. They also should teach mothers to recognize signs of illness that can be used as indicators of severity, so that mothers may, for example, correctly administer oral rehydration therapy for dehydration or bring children with respiratory distress for medical treatment. Furthermore, mass media could be used to communicate messages related to popular beliefs and to communicate effective messages to caretakers about signs of danger and severity of common infections, and what caretakers should do when a severe illness episode occurs. Further benefits could be obtained by using CHWs to provide health education, to supervise home care, to refer mothers to health centers, and to facilitate their access to hospitals. Second, combining methods helps researchers to obtain a broad diagnosis that is of direct use for decision making. Previous research has suggested that a combination of methods may generate information that could be more directly used for decision making in the public health sector.17 Increasingly, research is needed to obtain information for decision makers in the public health sector, and the type of research needed includes both quantitative and qualitative components.9,11,12,16 As Kroeger17 has stated, research that uses qualitative methods tends to yield information about health decision–making processes as well as an understanding of the disease or illness experience within a cultural context. The integration of the two methods provided different data and information, answering questions from multiple perspectives, thus enriching our knowledge of health and providing a more comprehensive understanding of complex public health issues and programs. In conclusion, the two methodologies yielded different and valuable information to address the problem of infant mortality and child undernutrition in different ways. The results generated by the quantitative methodology (i.e., ecological studies) have allowed government health officials and decision makers to establish the relative importance of selected factors and set priorities according to appropriateness and impact levels, taking into consideration not only the strength of the association between selected risk factors and infant mortality rates and prevalence of inadequate weight gain but also the prevalence of the condition being considered in their municipalities. On the other hand, the results from the qualitative research have provided baseline information for program managers and health officials to develop interventions aimed at changing risky behaviors or conferring protection. Previous studies have documented the importance of cultural information about the constituent populations served by specific community-based programs. Many problems in primary health care programs are due to a lack of knowledge about local health practices, a lack of sensitivity to economic and cultural factors, and other elements that require in-depth knowledge about cultural practices and local ecological constraints. Knowledge about behavioral relationships and prevailing cultural and economic exigencies (and other contextual conditions) may serve as a basis for justification and design of behavioral intervention.4 Specifically, the findings from the qualitative research have guided program planning and have been especially critical for the design of health education interventions involving health care providers and child caretakers. By using multiple methods that integrate quantitative and qualitative components, researchers can broaden their understanding of complex public health issues and provide hypotheses and explanatory mechanisms to account for observed problems. Although quantitative methods are important to document the contribution of risk factors to dis-


583

ease, poststudy qualitative research is crucial if findings are to result in changes in practices. This study adds to research that integrates quantitative and qualitative approaches to tackle diverse, complex public health problems. Implications for Research and Practice Historically, a rivalry between advocates of qualitative and quantitative methods has precluded the recognition of the benefits and the extensive applications of both methods (i.e., a multimethod approach) in the same study. By using multiple methods that integrate quantitative and qualitative components, researchers can broaden their understanding of complex public health issues and provide hypotheses and explanatory mechanisms to account for observed problems. For example, although quantitative methods are important to document the contribution of risk factors to disease, poststudy qualitative research is crucial if findings are to result in changes in practices. This article presents an example of a comprehensive evaluation of a child survival program in Ceara, Northeast Brazil, that used an integrative approach that combined quantitative (e.g., surveys and ecological analysis) and qualitative methods (e.g., in-depth interviews and verbal autopsies) to understand the determinants and circumstances of infant mortality and inadequate weight gain among municipalities. The integration of the two methods provided different data and information, answering questions from multiple perspectives, thus enriching our knowledge of health and providing a more comprehensive understanding of complex public health issues and programs. The results generated by the quantitative methodology (i.e., ecological studies) have allowed government health officials and decision makers to establish the relative importance of selected factors and set priorities according to appropriateness and impact levels, taking into consideration not only the strength of the association between selected risk factors and infant mortality rates and prevalence of inadequate weight gain but also the prevalence of the condition being considered in their municipalities. On the other hand, the results from the qualitative research have provided baseline information for program managers and health officials to develop interventions aimed at changing risky behaviors or conferring protection.

References 1. Chen LC, Hill AG, Murray CJL, Garenne M: A critical analysis of the design, results and implications of the mortality and use of health services surveys. Int J Epidemiol 22, no. 5(suppl. 1):S73-S80, 1993. 2. Ewbank DC: A strategy for evaluating the mortality impact of child survival programmes in Africa: Combating childhood communicable diseases project. Int J Epidemiol 22, no. 5(suppl. 1):S8-S14, 1993. 3. Murray CJL, Chen LC: Monitoring child survival programmes in Africa: The Africa child survival initiative—Introduction. Int J of Epidemiol 22, no. 5(suppl. 1):S1-S7, 1993. 4. Bentley ME, Pelto GH, Straus WL, Schuman DA, Adegbola C, de La Pena E, Oni GA, Brown KH, Huffman SL: Rapid ethnographic assessment: Applications in a diarrhea management program. Soc Sci Med 27(1):107-116, 1988. 5. Smith GS: Development of rapid epidemiologic assessment methods to evaluate health status and delivery of health services. Int J Epidemiol 18, no. 4(suppl. 2):S2-S15, 1989. 6. Van Norren B, Ties Boerma J, Sempebwa EK: Simplifying the evaluation of primary health care programmes. Soc Sci Med 28(10):1091-1097, 1989.

584


7. Scrimshaw SCM, Hurtado E: Rapid Assessment Procedures for Nutrition and Primary Health Care: Anthropological approaches to improving programme effectiveness. Tokyo, the United Nations, 1991. 8. Thompson JC: Program evaluation within a health promotion framework. Can J Public Health 83(suppl. 1):S67-S71, 1992. 9. Yach D: The use and value of qualitative methods in health research in developing countries. Soc Sci Med 35(4):603-612, 1992. 10. Anker M, Guidotti RJ, Orzeszyna S, Sapirie SA, Thuriax MC: Rapid evaluation methods (REM) of health services performance: Methodological observations. Bull World Health Organ 71(1):15-21, 1993. 11. De Vries H, Weijts W, Dijkstra M, Kok G: The utilization of qualitative and quantitative data for health education program planning, implementation, and evaluation: A spiral approach. Health Educ Q 19(1):101-115, 1992. 12. Hugentobler MK, Israel BA, Schurman SJ: An action research approach to workplace health: Integrating methods. Health Educ Q 19(1):55-76, 1992. 13. Buchanan DR: An uneasy alliance: Combining qualitative and quantitative research methods. Health Educ Q 19(1):117-135, 1992. 14. Helitzer-Allen DL, Kendall C: Explaining differences between qualitative and quantitative data: A study of chemoprophylaxis during pregnancy. Health Educ Q 19(1):41-54, 1992. 15. Pope C, Mays N: Reaching the parts that other methods cannot reach: An introduction to qualitative methods in health and health services research. Br Med J 311:42-45, 1995. 16. Barbour RS: The case for combining qualitative and quantitative approaches in health services research. J Health Services Research & Policy 4(1):39-43, 1999. 17. Kroeger A: Health interviews in developing countries: A review of the methods and results. Int J Epidemiol 12(4):465-483, 1983. 18. Stange KC, Zyzanzki SJ: Integrating qualitative and quantitative research methods. Fam Med 21:448-451, 1989. 19. Minayo MC, D’Elia JC, Svitone E: Programa de Agentes de Saude do Ceara: Estudo de Caso. Fortaleza, Brazil, UNICEF, 1990. 20. Secretaria Estadual da Saude. Sistema de Informacao do Programa Agentes Comunitarios de Saude (P.A.C.S.): Instrutivo para o preenchimento de Fichas e Documentos. Fortaleza, Brazil, Outubro, 1992. 21. Terra de Souza AC, Svitonne E, Peterson KE, Gardner J, do Amaral MIV, Ascherio A: Variations in infant mortality rates among municipalities in the state of Ceara, Northeast Brazil: An ecological analysis. Int J Epidemiol 28:267-275, 1999. 22. Terra de Souza AC, Svitonne E, Peterson KE, Gardner J, do Amaral MIV: Underlying and proximate determinants of diarrhea-specific infant mortality rates among municipalities in the state of Ceara, Northeast Brazil: An ecological study. J Biosoc Sci 33:227-244, 2001. 23. Terra de Souza AC, Peterson KE, Cuffino E, Gardner J, Craveiro MVA, Ascherio A: Risk factors for inadequate weight gain in children 0-23 months among municipalities in the state of Ceara, Northeast Brazil: The role of health services and socio-economic variables. Bull World Health Organ 77(11):895-905, 1999. 24. Terra de Souza AC, Peterson KE, Gardner J, Andrade MF, Ascherio A: The circumstances of post-neonatal deaths in Ceara, Northeast Brazil: Mothers’health care-seeking behaviors during their infants’ fatal illnesses. Soc Sci Med 51:1675-1693, 2000. 25. Bauman LJ, Adair Greenberg E: The use of ethnographic interviewing to inform questionnaire construction. Health Educ Q 19(1):9-23, 1992. 26. Sells SP, Smith TE, Sprenkle DH: Integrating qualitative and quantitative methods: A research model. Fam Process 34:199-218, 1995.