Global Health Action
æ
ORIGINAL ARTICLE
The path to impact of operational research on tuberculosis control policies and practices in Indonesia Ari Probandari1*, Bagoes Widjanarko2, Yodi Mahendradhata3, Hary Sanjoto1, Ancila Cerisha1, Saverina Nungky1, Pandu Riono4, Sumanto Simon5, Muhammad Noor Farid4, Sardikin Giriputra6, Artawan Eka Putra7, Erlina Burhan6, Chatarina U. Wahyuni8, Dyah Mustikawati9, Christina Widianingrum9, Edine W. Tiemersma10, Bachti Alisjahbana11 on behalf of the Tuberculosis Operational Research Group (TORG) 1
Department of Public Health, Faculty of Medicine, Sebelas Maret University, Surakarta, Indonesia; Department of Health Promotion, Faculty of Public Health, Universitas Diponegoro, Semarang, Indonesia; 3 Centre for Health Policy and Management, Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia; 4Department of Biostatistics, Faculty of Public Health, Universitas Indonesia, Jakarta, Indonesia; 5 Department of Clinical Pathology, Faculty of Medicine, Universitas Atmajaya, Jakarta, Indonesia; 6 Department of Pulmonology and Respiratory Medicine, Faculty of Medicine, Universitas Indonesia, Jakarta, Indonesia; 7Department of Public Health, Faculty of Medicine, Universitas Udayana, Denpasar, Indonesia; 8Department of Epidemiology, Faculty of Public Health, Universitas Airlangga, Surabaya, Indonesia; 9Sub-directorate of Tuberculosis, Ministry of Health, Jakarta, Indonesia; 10 KNCV Tuberculosis Foundation, The Hague, The Netherlands; 11Department of Internal Medicine, Faculty of Medicine, Universitas Padjajaran, Bandung, Indonesia 2
Background: Operational research is currently one of the pillars of the global strategy to control tuberculosis. Indonesia initiated capacity building for operational research on tuberculosis over the last decade. Although publication of the research in peer-reviewed journals is an important indicator for measuring the success of this endeavor, the influence of operational research on policy and practices is considered even more important. However, little is known about the process by which operational research influences tuberculosis control policy and practices. Objective: We aimed to investigate the influence of operational research on tuberculosis control policy and practice in Indonesia between 2004 and 2014. Design: Using a qualitative study design, we conducted in-depth interviews of 50 researchers and 30 policy makers/program managers and performed document reviews. Transcripts of these interviews were evaluated while applying content analysis. Results: Operational research contributed to tuberculosis control policy and practice improvements, including development of new policies, introduction of new practices, and reinforcement of current program policies and practices. However, most of these developments had limited sustainability. The path from the dissemination of research results and recommendations to policy and practice changes was long and complex. The skills, interests, and political power of researchers and policy makers, as well as health system response, could influence the process. Conclusions: Operational research contributed to improving tuberculosis control policy and practices. A systematic approach to improve the sustainability of the impact of operational research should be explored. Keywords: qualitative study; knowledge translation; research; operational research; tuberculosis; Indonesia Responsible Editor: Diana Gil-Gonzalez, University of Alicante, Spain.
*Correspondence to: Ari Probandari, Department of Public Health, Faculty of Medicine, Universitas Sebelas Maret, Jalan Ir. Sutami 36A, Surakarta 57126, Indonesia, Email:
[email protected] Received: 23 September 2015; Revised: 3 January 2016; Accepted: 27 January 2016; Published: 25 February 2016
Global Health Action 2016. # 2016 Ari Probandari et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
1
(page number not for citation purpose)
Ari Probandari et al.
Introduction Operational research (OR) can be defined as ‘the search for knowledge on interventions, strategies, or tools that can enhance the quality, effectiveness, or coverage of programs in which the research is being done’ (1). For the past 10 years, OR has been recognized as an important pillar of the control strategy for tuberculosis (TB) (2, 3). Indonesia, a country with a high TB burden, has made efforts to promote the implementation of OR in the TB control program. The Tuberculosis Operational Research Group (TORG), formed by the National Tuberculosis Control Program (NTP) in collaboration with partners (e.g. KNCV Tuberculosis Foundation, the United States Agency for International Development [USAID], the Global Fund to fight AIDS, TB and Malaria), has been conducting OR capacity-building trainings in Indonesia since 2004. Details on the TORG’s OR capacity-building activities and outputs have been published previously (4). Dissemination of OR findings in peer-reviewed journals is only a single indicator of the success of OR; however, the ability of OR to contribute to improving policy and practices is even more important (5). This study aimed to explore how OR influenced TB control policies and practices in Indonesia between 2004 and 2014.
Materials and methods Research settings Indonesia is a middle-income archipelago in Southeast Asia consisting of 34 provinces. Since 2001, the government has applied a decentralized policy system. For TB control, this decentralization implies the sharing of responsibilities among the central, provincial, and district governments. The district government conducts basic management of TB control, including distribution of anti-TB drugs and laboratory reagents, training, supervision, monitoring, and evaluation of health facilities. The provincial government trains, supervises, monitors, and evaluates the districts. The central government is responsible for the development and supervision of standards related to TB, as well as the provision of quality TB drugs and laboratory consumables. The TB control program is integrated into the health care system, including primary health centers, lung clinics, and hospitals (2). Since medical schools and/or public health schools commonly exist in the capital of each province, the TORG selected provincial OR groups for capacity-building purposes. Each OR group consisted of two to three researchers from academic institutions (‘academic-based researchers’) and one to two TB program staff (‘program-based researchers’). The process of capacity building has been described previously (4). Research design This was a qualitative study conducted between March 2014 and December 2014. Data were collected by thorough in-depth interviews and document reviews.
2 (page number not for citation purpose)
Informants and sampling By February 2014, 33 provincial OR groups had participated in the TORG proposal development workshop. The newest province, which was established in late 2012, was not involved in the TORG when this study was conducted. Only 31 groups conducted a field study. Two OR groups experienced teamwork problems and did not continue with the field study. Four OR groups were still in the data analysis phase and had not completed the project. Therefore, we included 27 OR groups in this study (Fig. 1). We obtained a list of researchers who had participated in any of the TORG proposal development workshops held between 2004 and 2014 from the NTP. We selected the principal researcher and/or another co-researcher from each OR group. Relevant policy makers or program managers (i.e. hospital managers, heads of Communicable Disease Control (CDC) at the provincial health offices, district level TB program staff, and NTP focal points) were selected by snowball sampling based on information from the researchers. Data collection and analysis We developed the guidelines of in-depth interviews according to the framework of Hanney et al. (6). The interview guidelines extracted information regarding the following:
. Results and recommendations of the OR . Follow-up actions after results were disseminated 33 OR groups registered in the National TB Program Unit MoH databases, which participated TORG courses
31 OR groups conducted and completed field studies
27 OR groups conducted results dissemination activities
2 OR groups did not conduct field studies
4 OR groups had not reported their results at the time of our study
Invitations issued to at leastt 1 researcher and 1 policy maker of each OR group for interviews
Interviews with a total of 70 informants mants antss (28 researchers from academic institutions, 22 researchers from health institutions, and 30 policymakers/program managers) related to 25 OR groups were included in the data analysis
2 policy makers/program managers related to 2 OR groups declined the invitation for interviews
Fig. 1. Selection of operational research projects and informants in this study. OR: operational research; MoH: Ministry of Health; TB: tuberculosis; TORG: Tuberculosis Operational Research Group.
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Impact of operational research on TB control in Indonesia
. Influence of OR upon policy and/or practice changes, as well as means of support and impediments We collected OR project reports and harvested information on the recommendations from the study projects. This information was used for triangulation of data from interviews to assess the implementation of the OR project recommendations. We excluded the OR projects of two provinces from the analysis since we were unable to obtain information from the policy makers’ perspective. Ultimately, we included interview data from 50 researchers and 30 policy makers in the analysis. The researchers consisted of 28 academics, 17 provincial health office staff members, 2 district health office staff members, and 3 hospital staff members. Policy makers included 4 national TB programmers, 21 provincial CDC/TB programmers, 3 district CDC/TB programmers, and 2 TB health service coordinators. Trustworthiness To improve accuracy (7), this study applied triangulation of data sources (interviews and document reviews) and peer debriefing strategies to TORG members. AP and HS performed most of the interviews, whereas SN and AC were responsible for writing verbatim transcripts within 48 h. AP and SN checked the accuracy of verbatim transcripts before analysis. The preliminary results of the analysis were discussed among all the authors. Research ethics This study received ethical clearance from the Committee of Research Ethics of the Faculty of Medicine of Universitas Sebelas Maret, Indonesia. The interviewers provided information about the study via emails and phone calls before conducting the interviews, and the informants provided verbal informed consent before the interviews. The interviews were taped, and informed consent was recorded. Transcripts of the interviews were anonymized, as were analyses and data interpretation/ presentation. Data analysis Content analysis (8, 9) was used for data interpretation. The authors read the verbatim transcripts several times in order to explore the response codes, which contained manifest meanings (i.e. obvious content, or meanings that are self-evident) or latent meaning (i.e. the indirect underlying message of the text). Codes sharing a commonality were grouped into a single category. A theme represented the link between categories. We used Open Code 4.02 software (10) to facilitate analyses, the results of which are illustrated in Tables 1 and 2.
jects were oriented to the improvement of TB program implementation at the provincial, district, or health service center level, while three OR projects targeted national level TB program policy improvement. The analysis revealed a ‘path to impact’ theme that consisted of three categories: the impact, path, and keys. The impact described issues pertaining to the contribution of OR projects to relevant TB policies and practices. The path revealed the processes from the end of the OR field study to the success/failure of OR projects in contributing to TB policies and practices. The keys represented relevant factors that supported or hindered the implementation of the OR projects’ recommendations. The impact Spectrum of impact Descriptions of OR impact are presented in Table 3. Our analysis found that most OR projects (22 of 25) contributed to TB policy and practices in a spectrum of domains and scopes (Table 4). The spectrum showed that OR project contributions to TB policy and practices could be divided into two domains: 1) the impact of the OR project on the development of new TB program policies or practices and 2) actual evidence that the OR project improved or reinforced existing policies or practices. The impact of the OR project was observed on the national, provincial, district, and health facility center levels. Thirteen OR projects related to the first domain of impact. For example, the OR project in Bali Province led to the development of reward systems for private practitioners’ contributions to TB suspect referrals. Additionally, the OR project on TBHIV collaboration at a teaching hospital in West Java Province triggered the development of standard operating procedures for TB HIV collaboration at the hospital level. Nine OR projects involved strengthening current TB control policies and practices upon evaluating the strengths and weaknesses of the activities of TB control programs. Some OR projects produced recommendations that the TB control program had already implemented, although the studies were not yet completed. For instance, the OR project on pediatric TB was initiated concurrently with the development of the national pediatric TB guideline. However, the guideline was finalized while the OR project was still ongoing. The OR duration and the dynamics of the TB control program activities were important factors, as expressed in the following quotation:
Results The 25 OR projects included in the analysis varied in topics and study designs (Table 3). Twenty-two OR proCitation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
The multidrug-resistant TB program was very dynamic. Because our OR project had a long duration, the recommendations from our OR could not keep up with the continuously shifting dynamics of the program. (Academic-based researcher, Jakarta Province)
3
(page number not for citation purpose)
Ari Probandari et al.
4
(page number not for citation purpose)
Table 1. Example of data analysis Topic
Meaning unit
Code
How the findings of the study results were disseminated
We disseminated the results by inviting all heads of district health offices Result dissemination in Banda Aceh Province.
Follow-up actions after results of
There was an order [to implement the OR project recommendation] from
OR were disseminated
Sub-category
Category
The milestones The path
Policy development
the provincial health office, i.e. a letter circulated to private clinics and hospitals. We presented the OR results and recommendations in meetings, both
Advocacy meetings
national and district level. We were invited to the provincial Monev [monitoring evaluation] meeting in Time of policy document Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
2010 to discuss strategies of accommodating the OR results that were disseminated in 2008. The draft [of the policy document] was completed
Time
development
in 2011. I think it [the implementation of the OR recommendation] took
One year of
Sustainability
approximately 1 year in the study district.
implementation;
of impact
The impact
unsustainable action Influence of the ORs on TB service, program practice, and/or policy
We have ideas on what we should do. The OR results strengthened them Strengthening current [the ideas of the policy makers].
Factors perceived as important to the There was a change in leadership at the health office [in the district]. process of knowledge translation of I foresee that innovations [from the OR results] will not be sustained.
Spectrum
policy/practice
of impact
Changes of policy makers
Policy makers
the ORs I am a bit frustrated . . . Yes, really frustrated. It [the OR recommendations] Power of program-based should be supported. I am just a staff member, I can only grumble.
researcher
We could do these . . . because we have national and provincial policy
Support of policy
support. OR: operational research; TB: tuberculosis.
Researchers Health systems
The keys
Theme The path to impact
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Table 2. Summary of analysis: codes, sub-categories, categories, and theme The path to impact
Theme Category
The path
The impact
The keys
Sustainability Sub-category Code
Milestones
Time
Spectrum of impact
Time of policy
Domain of impact
advocacy;
document
Innovation
dissemination
development
New policy/
practice Evidence-
Preliminary
seminar Policy-brief
Time of advocacy
documents
Preliminary policy
Improvement of
document
existing policy/
development
practice
Letter of commitment
Scope of impact
Unsustained
Researchers
actions
Policy makers
Lack of confidence
Budget
group
in policy makers’
Financial support
Cohesiveness of OR
ability to deploy
Logistic matter
innovation Enthusiasm of
Support of policy and regulation
Collaboration of OR
One year of
implementation Several months of
group Communication skills
implementation
of researcher
policy makers
Relevant guideline
Intensive contacts with
Appreciation of
Changes of
policy maker
policy maker
Four years of
implementation
Replication to
Confidence of
targeted policy
Confidence of
maker
Perceived priority of policy maker
Follow-up
Formal advocacy
researcher
of OR
Change of job position
recommendations
Letter of recommendation
meetings
5
(page number not for citation purpose)
Citation of OR
of program-based
results
researcher Researcher’s skills in
preparing contingency Researcher’s advocacy
Enhanced policy
document Hospital regulation
Memorandum of understanding
skills
Perceived feasibility
by policy makers
Changes of policy maker
Impact of operational research on TB control in Indonesia
researcher to approach
other districts
program field staff
Political power of
researcher to persuade Power/authority of program-based
response
informed policy
Health system
of impact
No
Title of study (year of study)
Study design
Province
Topic: Community-based TB case findings 1. The effect of health promotion to An experimental South informal community groups in
study
Sulawesi
detecting TB suspects at Gowa
Recommendations*
To involve informal community champions to find
Implementation of recommendations**
presumptive TB cases and to refer to primary health
community cadres for every TB case that was
centers.
identified by the cadres.
District (2007)
Provision of financial incentives for health
One year continuing implementation of the intervention in the study district.
2.
3.
The involvement of the Acehnese
An experimental Aceh
To involve the Acehnese community champions in the A pilot project of the involvement of the community
local community champion in
study
education of the community on TB and presumptive TB champions in TB education and TB presumptive
TB control (2009) The role of religious leaders in
An experimental East Nusa
case identification. Training of religious leaders in educating the
increasing pulmonary TB case
study
Tenggara
4.
community about TB.
notification (2010)
Provision of TB information materials.
Involving the traditional market
An experimental Southeast
The provincial health office should involve the
community in identification of
study
traditional market community in identification of
Sulawesi
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
suspected TB cases (2010)
case identification in a district for 1 year. Provision of TB educational materials.
None.
suspected TB cases.
Topic: Treatment-seeking behavior of people with symptoms of TB 5.
Treatment-seeking behavior of
A qualitative
TB patients (2005)
study
Yogyakarta
The national TB control program should collaborate
Stepwise training for hospitals on standardized
with hospitals and private medical practitioners.
TB case management including recording/
TB education in the community to increase access
to DOTS health facility. Active TB case findings within community.
reporting the cases to the district health office.
The involvement of private medical practitioners in TB control in a district.
Provision of TB education materials by the health promotion division of the Provincial Health Office.
Training for primary health care staff on TB case detection.
The implementation of active TB case discovery in the city of Yogyakarta.
6.
Treatment-seeking behavior of community and TB patients (2007)
A crosssectional study
Lampung
Training of midwives and nurses in presumptive TB case identification.
Training of cadres on presumptive TB case identification in a sub-district (2007).
North
Improvement of lab facilities, including the lab waste
None.
Sumatera Papua
system. Education for patients and drug administration observers.
Topics: TB services at primary health care centers 7.
TB control program performance in
8.
primary health centers (2007) sectional study Treatment compliance of TB patients A casecontrol treated under the DOTS strategy in primary health centers (2008)
A cross-
study
Development of IEC materials in the Papuan language.
The production of IEC materials/media in the Papuan language (2011). The use of the materials for education of TB/HIV patients (2011).
Ari Probandari et al.
6
(page number not for citation purpose)
Table 3. Characteristics of operational research projects included in the present analysis
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Table 3 (Continued ) No
Title of study (year of study)
Study design
Province
9.
The effectiveness of contact tracing
An experimental Bengkulu
in increasing TB suspect identification (20092010)
study
Recommendations*
To conduct contact tracing, as an additional
strategy to passive case discovery of TB. A cost-effectiveness evaluation of contact tracing strategy.
Implementation of recommendations**
Implementation of contact tracing as an additional strategy to passive case discovery of TB (2010).
A letter was circulated from the provincial health
Development of district level policy to support
office to district health offices in Bengkulu
contact tracing.
Province and primary health centers regarding the recommendation to conduct contact tracing (2010).
10. TB suspect identification and TB case notification by enhancing the satellite public health center
The development of networks between satellite
health centers and main health centers. Provision of logistics.
East
Dissemination of the DOTS strategy to hospitals.
Benchmarking visit to provincial health office and
Kalimantan
Hospitals should prepare for human resources
a hospital of West Sumatra, on the implementation
responsible for the implementation of DOTS
of DOTS strategy.
An experimental Central study
Maluku
Training of trainers on the development of networks between satellite health centers and main health centers (2014).
network on Haruku Island (2010) 11. Knowledge, attitude, and
A cross-
commitment of primary health center sectional study staff on TB program (2011)
strategy at hospitals.
The development of SOPs for DOTS strategy implementation in hospitals.
Routine coordination.
12. Implementation of DOTS strategy in hospitals (2005)
A mixed methods study
Central Java
To develop collaboration between specialist and other Guidelines and SOPs of DOTS strategy medical staff at hospital. implementation in hospital.
13. Implementation of directly observed
A qualitative
South
The hospital should appoint specific (senior) TB staff to None.
study
Kalimantan
monitor the implementation of the DOTS strategy.
A casecontrol
Bali
Private practitioners should be involved in TB control,
treatment short-course strategy in hospitals (2007) 14. The contribution of private medical
7
(page number not for citation purpose)
practitioners to presumptive TB case identification and referral (20072008)
Training in the DOTS strategy. study
mainly referral of TB suspects to primary health
A pilot study examining the involvement of private medical practices and non-governmental
centers.
organizations in TB control (in 2009).
A simplified TB recording and reporting system for private medical practices. A reward system from the Indonesian Medical Association for private medical practices who refer TB patients to primary health centers.
Impact of operational research on TB control in Indonesia
Topic: Publicprivate mix for TB control
No
Title of study (year of study)
15. Readiness of DOTS strategy implementation at hospitals (2009)
Study design A qualitative
Province Banten
Recommendations*
study
To increase the commitment of hospitals in the
Implementation of recommendations**
implementation of the DOTS strategy, e.g. through the production of decision letters, job descriptions,
Production of letters of decision to establish a DOTS strategy implementation team at one of three hospitals in the study.
and SOP and infrastructure support.
Routine supervision by district health office.
Coordination with district health office to intensify referral of cases from hospitals to primary health centers.
16. The implementation of the ISTC in hospitals (2009)
A cross-
DKI Jakarta
To prepare human resources, supplies, and
instruments for sputum tests at hospitals. To improve joint supervision between the provincial
on the DOTS strategy in hospitals (2010). Dissemination of the DOTS strategy by writing a
health office, hospital association,
letter to all private hospitals in the province
sectional study
and professional organizations.
To develop a memorandum of understanding
(2010).
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
between the provincial health office, hospital
Dissemination of the DOTS strategy among staff in hospitals.
Inclusion of DOTS strategy implementation in
Dissemination of the DOTS strategy in hospitals during three monthly monitoring evaluation
association, and professional organizations on ISTC implementation.
The provincial health office conducted trainings
meetings for all district TB program staff.
The inclusion of the DOTS strategy in the national hospital accreditation (2012).
national hospital accreditation. 17. Association between treatment observer characteristics and
A casecontrol
West Java
study
defaulting from TB treatment in
Hasan Sadikin Hospital, Bandung (2009)
The presence of TB drug observers during patients’
Education to TB patients and drug observers
visits to the hospital so that the provider can
approximately once a week during the busiest
educate TB drug observers.
day at the outpatient unit at Hasan Sadikin
Improvement of the pathway of TB services in
hospital. The education was conducted by
hospitals by educating TB drug observers.
residents of the pulmonology department. The Department of Pulmonology initiated the education without any changes to hospital SOPs regarding TB services.
18. Development of a network between private laboratory and private medical practices in implementing
A qualitative
West
study
Sumatra
the DOTS strategy (2010)
Diagnostic tests for TB in private labs should use
Training in private labs with province budget.
sputum tests.
A network of private labs and a system of
The dissemination of information on proper suspect
reporting from private labs to Dinas Kesehatan
criteria, sputum specimen collection, and sputum
was established and implemented.
tests. Private labs should report TB cases every month or every 3 months.
Ari Probandari et al.
8
(page number not for citation purpose)
Table 3 (Continued )
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Table 3 (Continued ) No
Title of study (year of study)
19. The effectiveness of TB education through SMS on treatment compliance among TB patients in
Study design
Province
An experimental Central Java
Recommendations*
study
Implementation of recommendations**
The use of SMS technology to provide education to
The adoption of an SMS-based educational
TB patients during treatment.
system by TB patients in a district.
hospitals (2011) Topic: TB lab quality 20. Low quality of sputum specimens for A crossCentral Java TB diagnosis and its factors (2005) sectional survey 21. Quality of sputum tests in public health centers (2010)
A cross-
Jambi
Training for providers on TB sputum specimen collection.
Education for patients on how to collect sputum.
Collaboration between TB program staff and
sectional study
Trainings for laboratory staff (20062014)
Continuation of the collaborative work between
laboratory staff at primary health centers.
TB program staff and laboratory staff at primary
Monitoring of the recommended collaborative work
health centers for some months in 2011.
by heads of primary health centers.
Improvement of lab infrastructure.
Supportive supervision by district health office. Assistance of provincial health office to district TB staff in managerial skills.
Advocacy to head of district health office in Jambi Province.
22. The effectiveness of training on
study
Tenggara
Continue to conduct training of the laboratory staff at the primary health centers.
laboratory staff (2011)
Training for lab staff at primary health centers.
Logistics and equipment provision.
Laboratory staff workload reduction.
The MDR-TB program plan to decentralize
Topic: TB in children, MDR-TB, TB-HIV 23. Delay in treatment among MDR-TB patients (20112012)
A cross-
DKI Jakarta
sectional study
Decentralized treatment from hospital to primary health care level.
MDR-TB treatment at primary health care
Incentives for MDR-TB patients to support the negative impact of loss of income during the
centers (2013).
treatment.
9
(page number not for citation purpose)
24. Assessment of TB pediatric scoring chart (20112012)
An experimental DKI Jakarta study
Training and technical assistance to general
Training of medical doctors in primary health centers on pediatric-TB-scoring chart (2013).
Inclusion of pediatric TB cases on the agenda
practitioners at primary health centers on a pediatric TB diagnostic scoring system.
of routine monitoring and evaluation meetings (20132014).
Continuing technical assistance on the use of scoring charts for pediatric TB diagnosis (2014).
Impact of operational research on TB control in Indonesia
TB microscopy diagnosis among
An experimental West Nusa
No
Title of study (year of study)
25. The improvement of TB-HIV collaboration in Hasan Sadikin Hospital, Bandung (2011)
Study design Action research
Province West Java
Recommendations*
Training for hospital staff on TB-HIV collaboration.
Supervision of TB-HIV collaboration. Improving the reporting system.
Implementation of recommendations**
The sub-directorate TB MoH planned and implemented the following: (1) Supervision and technical assistance to provinces with under-achievement of TB-HIV program target indicators (2) Workshops on recording, reporting, and strengthening anti-TB-HIV collaboration efforts (3) Routine meetings of the national TB-HIV team These three activities were not perceived to
directly impact the operational research. Intensified communication between the TB and
A modified model of integrated TB-HIV services
HIV units at the hospital (20132014). at the hospital (20132014). Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
A TB-HIV team at the hospital level, which is responsible for the management of services for TB-HIV patients (2014).
Improved reporting system of the TB and HIV units (20132014).
Improved indicators of the TB-HIV collaboration program (20132014).
*Information was collected from the OR study reports; ** Information was collected from the interviews with policy makers/program managers and researchers. DOTS: directly observed treatment short-course; HIV: human immunodeficiency virus; IEC: information, education, and counseling; ISTC: International Standard for Tuberculosis Care; MDR-TB: multidrug-resistant TB; MoH: Ministry of Health; SMS: short message service; SOP: standard operating procedure; TB: tuberculosis.
Ari Probandari et al.
10
(page number not for citation purpose)
Table 3 (Continued )
Impact of operational research on TB control in Indonesia
Table 4. Spectrum of OR impact in TB control program policies or practices Domain Strengthening current TB control program policies or practices Scopes National
New TB control program policies or practices
Decentralized treatment of MDR-TB at primary health
NA
services
Inclusion of TB service in the hospital accreditation assessment
Province
District
Health
Intensified trainings on pediatric TB diagnosis scoring system
Reward system for private practitioners’ contributions to TB control
The implementation of pediatric-TB-scoring charts at
Modified reporting form of TB suspect
TB services of primary health centers
identification by private practitioners
Validated MDR-TB data
Intensified trainings on laboratory
Intensified trainings on DOTS strategy to hospital staff
Intensified trainings on laboratory
Intensified trainings on DOTS strategy to hospital staff Production of TB education materials
A modified model of integrated TB-HIV services
Short message servicebased TB education
TB education materials in local language
facility
Innovative approaches to TB case finding Collaborative work between the TB program staff and lab staff at primary health centers
Standard operating procedure on collaborative TB-HIV services
NA: not applicable.
Sustainability of impact The recommendations of most OR projects included in our study resulted in changes to policies or practices that lasted for various lengths of time. For example, the recommendation from the OR project in Jambi Province that the work of the TB program staff and laboratory staff be integrated at the primary care level was only maintained for less than a year after the results dissemination seminar. The recommendation of involving community members in active TB case detection in Aceh Province was maintained for approximately 1 year. The recommendation to award accreditation points to private practitioners involved in TB case detection in Bali Province was maintained for a longer period. OR recommendations were less sustainable when they concerned innovations of policies or practices instead of recommendations to strengthen existing policies and practices. The path The milestones The overall process of translating knowledge obtained through OR into influence on TB program policy and practice is presented in Fig. 2. Milestones include the following: 1) preliminary advocacy; 2) the dissemination seminar; 3) development of policy documents; 4) advocacy meetings; and 5) new policies and practices, or reinforcement of current ones. Some OR groups informally shared preliminary findings before the official dissemination seminar to raise
interest among relevant policy makers. Some OR groups also created policy-brief documents. Approximately 1 year after commencing field work, all OR groups conducted an official dissemination seminar for stakeholders from multiple disciplines who may have had the capacity to implement, or catalyze implementation of, the study recommendations. There were three types of policy makers invited: health facility level program managers (i.e. heads of hospitals, heads of lung clinics, and TB service providers); provincial or district level managers (i.e. heads of health offices, heads of CDC units, TB program staff, provincial/district health planning and budgeting bureau staff, and members of the provincial house of representatives); and national level managers (i.e. national TB program staff). During the dissemination seminars, the OR group typically presented the results and recommendations of the study; discussions with stakeholders then followed. In general, the policy makers accepted the OR results and recommendations, since these were regarded as relevant, practical, and having the potential to improve TB program performance.
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
I think the results [of OR] are useful. Other [district] TB program coordinators could accept them [the results and recommendations]. Yes, these [the recommendations] are feasible . . . (TB program officer, Bengkulu Province)
11
(page number not for citation purpose)
Ari Probandari et al.
Staffing changes of policymakers/program managers; Lack of political Lack of communication power of and cohesiveness within researchers the OR group Preliminary advocacy
No followup actions
The dissemination seminar
Persuasiveness and negotiation skill of the researchers
Political power of researchers Staffing changes of policy makers/program managers No follow up actions
Development of policy documents
Advocacy meetings
The OR recommendations matched the existing TB program development plan
Persuasiveness and negotiation skill of the researchers Political power of researchers
Reinforcing current policy and practices
New policy or practices Efforts of policymakers/ program managers to locate funding Sustainable
Staffing changes of policymakers/program managers Lack of a national policy support Lack of TB program logistics Not sustainable
Fig. 2. The process from operational research dissemination to impact, including support mechanisms and impediments. OR: operational research; TB: tuberculosis.
It was common for policy makers to develop preliminary policy documents (e.g. a letter of commitment to support the OR recommendations) as a follow-up to the dissemination seminar. However, the preliminary policy documents did not guarantee that the policy makers would deploy the recommendations of the OR projects (Fig. 2). Only a few OR projects led to the development of enhanced policy documents, such as a signed memorandum of understanding between institutions or a change in hospital regulations. Usually, development of the enhanced policy documents was preceded by a series of advocacy meetings. For some projects, researchers or/and policy makers conducted additional advocacy activities to obtain supporting policy papers and funding, thereby ensuring the application of the OR recommendations. In general, the policy makers/program managers requested that the researchers conduct additional presentations about the OR
12 (page number not for citation purpose)
results and recommendations in a formal meeting (such as the provincial TB surveillance meeting). Otherwise, the policy makers/program managers promoted the recommendations of the OR projects to all relevant facilities under their authority. Some researchers highlighted evidence from the OR project in their presentations at various forums. Time The time between the dissemination seminars and the deployment of the recommendations ranged from months to years (Table 3). Complex actions related to policy (e.g. reward systems and national accreditation) took longer than those related to practices (e.g. training, modified recording/reporting forms, and education of TB patients). The keys Researchers, policy makers/program managers, and health system personnel form the keys to translating knowledge
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Impact of operational research on TB control in Indonesia
from OR projects into changes to TB program policy or practices. The roles of these individuals in OR contributions and achievement of milestones in TB policy or practices are presented in Fig. 2. The researchers Good collaboration and cohesion existed in the OR groups that impacted TB program policy or practices. In contrast, a group member leaving for a new job could impede follow-up on the OR recommendations (Fig. 2). One academic-based researcher mentioned, ‘We could not follow up [on the recommendations] intensively because of the move of the program-based researcher to another position’ (Banten Province). OR projects that led to program or policy changes tended to have program-based researchers with sufficient power and confidence to approach and persuade the policy makers and/or program managers. Conversely, program-based researchers who had no relevant authority felt less capable of persuading TB program policy makers. Communication and advocacy skills, as well as contingency planning, were also key to the successful influence of an OR project on policy and practice (Fig. 2) as exemplified in the following quotation: We coordinated and tried to work with the local Indonesian Medical Association. We had conducted activities [to involve private medical practitioners] together [with the Indonesian Medical Association]. However, the results did not satisfy our expectations, as most private practitioners did not attend the meeting. Therefore, we changed our strategy the following year. (Program-based researcher, Bali Province)
The policy makers A combination of enthusiasm and political power on the part of the policy makers was shown to be an important facilitator of achieving impact (Fig. 2). Those policy makers/program managers who took a personal interest in the OR findings and recommendations encouraged their peers to pursue the implementation of the recommendations. Such advocacy was successful if the policy maker/program manager had sufficient power to push for the deployment of the OR project recommendations. Changes of high level policy makers/program managers (e.g. head of district or head of provincial/district health office) or middle level policy makers/program managers (e.g. TB program staff at provincial or district levels, or heads of primary health centers) were often mentioned as obstacles to implementing recommendations for policy or practice changes. After a change of policy maker, it was more difficult for the researchers to ensure that the OR recommendations were implemented. Hence, such changes were also barriers to the sustainability of the impact of the OR.
Health system response Availability of a relevant national policy document or guideline was another key factor regarding implementation of innovative recommendations from the OR projects (Fig. 2). Some of these recommendations were not supported by existing regulations. For instance, the recommendation to apply enhanced case detection involving community members (such as religious leaders, women, or the traditional market community) could not be implemented in the TB control programs of some provinces due to the lack of a relevant national policy on an active case detection approach, which is required to formally promote such strategy. At the time that these recommendations were launched, the NTP still prioritized the passive case detection approach; the district/province level policy makers were reluctant to deploy innovations that were not supported by any existing higher level policy: When we disseminated our study results to other districts, they [the district TB program staff] were questioning how they could implement the recommendations [of community-based active case finding] . . . Until now, the national TB control guideline [has] only [included] the passive case detection approach . . . (Program-based researcher, Aceh Province)
Availability of financial support was another key to facilitating the impact of OR projects (Fig. 2). Some study recommendations could not be deployed due to the lack of funding. However, a few policy makers were able to resolve financial barriers. In such cases, the enthusiasm of policy makers for the OR results motivated them to locate alternative sources of funding. The preparedness of the Ministry of Health to respond to logistical provisions related to enabling application of OR project recommendations was another challenge. For example, the recommendation to implement active TB case detection by so-called community champions in Aceh was followed, and the numbers of TB cases reported by primary health centers increased. This consequently required additional drugs and laboratory supplies, something the logistics planners had not anticipated. Finally, staffing changes hampered the sustainability of the OR-recommended actions that involved TB program field staff. For example, the impact of the integrative work of the staff from the TB program and the laboratory in Jambi Province was diluted mostly due to the turnover of the TB program staff at the primary health centers.
Discussion Our study provides empirical evidence regarding the impact of OR projects on changes of policies and practices; similar impacts have been described previously for other areas of focus (11, 12). The OR projects produced
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
13
(page number not for citation purpose)
Ari Probandari et al.
a range of influences on TB programs (Table 4), which was expected given the wide spectrum of possible OR projects (13). Our findings showed that the OR projects contributed chiefly to the improvement of TB program practices at the provincial and district levels. OR can vary because of the heterogeneous settings of Indonesian provinces. Therefore, the TORG implemented a local context approach when devising the research questions (4), especially as OR is a form of research investigating problems in the health program up close. Hence, OR potentially avoids the problem of unnecessary wasteful research (14). The mechanisms of the influences of OR projects on TB control policies and practices are complex and nonlinear (Fig. 2), as has been argued by others (15, 16). Our study showed that the TORG’s approach of combining academics and health staff into a single OR group helped the OR projects to achieve greater impact. As stated by Grimshaw et al. (17), the inclusion of health staff as program-based researchers has helped the OR projects to contextualize the research problems, increase the applicability of the recommendations, and facilitate the communication of research findings to the relevant policy makers and program managers. However, careful selection of the health staff is important, as those with excellent communication and advocacy skills as well as sufficient influence are critical assets. Our study shows that the time elapsed between the OR projects’ duration and TB program development is a challenge. In particular, shorter study durations are needed for the development of new policies and practices based on innovative ideas emanating from OR projects. In our study, some policy makers or program managers perceived that the OR recommendations only came after the introduction of new policies or practices. Although this was true for a number of the studies included here, such studies still yielded evidence useful to policy makers and program managers regarding whether their policy changes were in fact improving the program. Prompt communication of the OR results to policy makers and managers is essential (12). Khotari et al. (18) also suggested that interaction between producers (i.e. researchers) and users (i.e. program managers) increases the users’ understanding of the research and enhances the likelihood that the research findings will be valued. The recommendations from some OR projects in our study could not be maintained. This was sometimes due to the irrelevance of the recommendations to existing policies; at the time study results were disseminated, there would already be a new and better policy. Therefore, process dynamics and policy relevance should be taken into account when the researchers plan for OR; as Ioannidis argued, fine-tuning between research and existing policy should be considered (19). Our study found that the implementation of OR project recommendations may require a broader health system response, such as the availability of overarching
14 (page number not for citation purpose)
policies and additional resources. Others have also shown that financial and human resource constraints impede the use of research evidence in decision-making (20). Again, prompt communication between researchers and policy makers/program managers should be encouraged to improve the preparedness of the health system to implement OR project recommendations (6). Our study evaluated OR projects facilitated by the TORG in the context of decentralized health systems; this limits the generalization of our findings. Nevertheless, we believe that our study provides evidence for the impact of the OR projects on policies and practices. Moreover, the lessons shared in this paper could be of use to ongoing initiatives involving OR capacity building in other countries.
Conclusions and recommendations Our study concluded that OR contributed to the development of new policies, introduction of new practices, and strengthening of current TB control policies and practices in Indonesia. However, the sustainability of these changes was often limited. The process of translating OR outcomes was complex. Even though OR findings are useful for TB control program improvement, the deployment of recommendations can be influenced by other factors. The skills and political power of researchers, interests and political power of policy makers, and health system response could all influence the process. Therefore, we recommend the following. First, the OR initiative should be intensified and promoted. The curriculum of OR capacity building should include skills of knowledge translation and communication for advocacy. Second, the OR group should include health staff with sufficient power to achieve changes in the TB control program. Researchers with sufficient influence will contribute heavily to translating knowledge into policy. Third, the timing of advocacy actions should be carefully planned in order to increase the possibility of sustainable deployment of the recommendations. Fourth, short communication lines between the OR groups and policy makers or program managers during and after executing the OR projects ought to be established to ensure that the results and recommendations of the research are properly implemented. Finally, ‘OR on OR’ studies are required, particularly to explore potential mechanisms to sustain the impact of OR on policies and practices.
Authors’ contributions AP contributed to the conception of the study, data collection, analysis, interpretation as well as drafting and revising the manuscript. BW, EWT and BA contributed to the conception of the study, data interpretation and revising the manuscript. HS, AC and SN contributed to data collection, analysis, interpretation and drafting the manuscript. YM, PR, SS, MHF, SG, AEP, EB, CUW,
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
Impact of operational research on TB control in Indonesia
DM, and CW contributed to interpretation of data for the manuscript, and gave critical comments to the revision of the manuscript. All authors gave final approval to the version to be published and agree to be accountable for all aspects of the work.
Acknowledgements This research was part of the study Impact of Operational Research to Tuberculosis Program Policy and Practice in Indonesia (Project No. 01.137/KNCV/III/2014). The authors thank the TB subdirectorate of the Ministry of Health, Republic of Indonesia, for endorsement and inputs for the study. Last but not least, we greatly appreciate all the subjects who participated in the study. This study was made possible by generous support from USAID. The contents are the responsibility of TB CARE I and do not necessarily reflect the views of USAID or the United States government.
Conflict of interest and funding The project received funding from USAID TBCARE I through KNCV Tuberculosis Foundation (Project No. 01.137/KNCV/III/2014). AP received financial support for publication from Global Fund. AP, BW, YM, PR, SS, MNF, SG, AEP, EB, CUW, and BA are members of TORG. EWT is a consultant for KNCV Tuberculosis Foundation, who provides technical assistance to TORG.
References 1. Zachariah R, Harries AD, Ishikawa N, Rieder HL, Bissel K, Laserson K, et al. Operational research in low-income countries: what, why, and how. Lancet Infect Des 2009; 9: 71117. 2. Ministry of Health, Republic of Indonesia. National strategy for tuberculosis control 20152019. Jakarta: Ministry of Health Republic of Indonesia; 2015. 3. World Health Organization. The end TB strategy: global strategy and targets for tuberculosis prevention, care, and control after 2015. Available from: http://www.who.int/tb/post2015_TBstrategy. pdf [cited 31 December 2015]. 4. Mahendradhata Y, Probandari A, Widjanarko B, Riono P, Mustikawati D, Tiemersma EW, et al. Embedding operational research into national disease control programme: lessons from 10 years of experience in Indonesia. Glob Health Action 2014; 7: 25412. doi: http://dx.doi.org/10.3402/gha.v7.25412
5. Kumar AM, Satyanarayana S, Wilson N, Zachariah R, Harries AD. Operational research capacity building in Asia: innovations, successes and challenges of a training course. Public Health Action 2013; 3: 1868. 6. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilization of health research in policy-making: concepts, examples, and methods of assessment. Health Res Policy Syst 2003; 1: 2. Available from: http://www.health-policy-systems. com/content/1/1/2 [cited 31 December 2015]. 7. Patton MQ. Qualitative research & evaluation methods. 3th ed. Thousand Oaks, CA: Sage Publications; 2002, pp. 5447. 8. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004; 24: 10512. 9. Elo S, Kynga¨s H. The qualitative content analysis process. J Adv Nurs 2008; 62: 10715. 10. ICT Services and System Development and Division of Epidemiology and Global Health. OpenCode 4.02. Umea˚: Umea˚ University; 2013. 11. Harries AD, Rusen ID, Reid T, Detjen AK, Berger SD, Bissel K, et al. The Union and Me´decins Sans Frontieres approach to operational research. Int J Tuberc Lung Dis 2011; 15: 14454. 12. Zachariah R, Ford N, Maher D, Bissell K, Van den Bergh R, Van den Boogaard W, et al. Is operational research delivering the goods? The journey to success in low-income countries. Lancet Infect Dis 2012; 12: 41521. 13. Lienhardt C, Cobelens FG. Operational research for improved tuberculosis control: the scope, the needs and the way forward. Int J Tuberc Lung Dis 2011; 15: 613. 14. Ioannidis JP. Clinical trials: what a waste. BMJ 2014; 349: g7089. doi: http://dx.doi.org/10.1136/bmj.g7089 15. Bissell K, Lee K, Freeman R. Analysing policy transfer: perspectives for operational research. Int J Tuberc Lung Dis 2011; 15: 11408. 16. Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med 2011; 104: 5019. 17. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci 2012; 7: 50. 18. Khotari A, Birch S, Charles C. ‘‘Interaction’’ and research utilisation in health policies and programs: does it work? Health Policy 2005; 71: 11725. 19. Ioannidis JP. How to make published research true. PLoS Med 2014; 11: e1001747. doi: http://dx.doi.org/10.1371/journal.pmed. 1001747 20. Elliott H, Popay J. How are policy makers using evidence? Models of research utilisation and local NHS policy making. J Epidemiol Community Health 2000; 54: 4618.
Citation: Glob Health Action 2016, 9: 29866 - http://dx.doi.org/10.3402/gha.v9.29866
15
(page number not for citation purpose)