principles - Wiley Online Library

Commentary

doi: 10.1111/1753-6405.12411

The NHMRC Road Map ‘benchmark’ principles: a formal evaluation process is needed to improve their application Michelle S. Fitts,1 Caryn West,1,2 Jan Robertson,1 Kim Robertson,1 Nicholas Roberts,1 Bronwyn Honorato,1 Alan R. Clough1 1. Community-based Health Promotion and Prevention Studies Group, Australian Institute of Tropical Health and Medicine, James Cook University, Queensland 2. School of Nursing, Midwifery & Nutrition, James Cook University, Queensland

T

he appropriate conduct of health research in Indigenous communities is an ongoing topic of discussion, underpinned by ethical, epistemological and methodological issues.1,2 Data collected, analysed and disseminated from research in Indigenous communities has typically met the needs of the researcher first,3 with Indigenous communities often playing a passive role in the research process.

understood, respected and navigated in order to conduct research successfully and appropriately with Indigenous Australian communities. The authors are involved in a large, NHMRC-funded (APP1042532) evaluation study aiming to describe long-term impacts on important health, economic and social outcomes of complex interventions restricting the supply of alcohol in Queensland’s discrete Indigenous communities.10 First implemented by the Queensland Government more than a decade ago (2002–03), Alcohol Management Plans (AMPs) aimed to reduce alcohol misuse, violence and injury.11,12 Evidence indicates that this policy has had some favourable impacts, but not in all communities.13,14

Australia’s National Health and Medical Research Council’s (NHMRC) ‘Road Maps I and II: Strategic Framework for Improving Aboriginal and Torres Strait Islander Health through Research’ provide researcher guidelines to ensure the needs and concerns of Indigenous Australian (Aboriginal and Torres Strait Islander) communities are met.4,5 The guidelines’ six principles relate to ethical research with the overall objective of making research relevant to the peoples and communities involved (see Table 1).4,5 Concerns have been raised in the public health literature that there is a lack of knowledge about converting priorities and guidelines into ethical research practices while maintaining strong, robust and beneficial research objectives for researchers and for the populations being studied.6,7 While there is some emerging commentary reflecting on methods used – including local level consultation processes, community engagement, capacity building and appropriate feedback – and how these contribute towards connecting researchers and study subjects,8,9 most research continues to focus on reporting study outcomes. Few researchers communicate the processes used to do their research, the challenges experienced and the unique historical, cultural, political and contextual factors that must be

2015 vol. 39 no. 4

This commentary summarises the reflections of the research team during four stages in this project: concept, implementation, data collection and feedback of findings. Team meeting minutes, field trip notes and financial records were used to map the research team’s activities against key principles underpinning the NHMRC Road Maps I and II. As a way of assessing the activities and practices against specific principles and including them in Table 1, consensus between the team was used to verify if the project had complied with a relevant principle at each stage. The study will be conducted in several clusters of the 15 discrete Indigenous communities and the nearby towns affected or influenced by AMPs across Queensland. The activities described in this commentary relate to the first cluster of communities (one regional town and two discrete communities – one about 12 kilometres from the regional town and the other 91 kilometres away by road). These activities and their links with the Road Map principles are described here and summarised in Table 1. We suggest that this kind of

reflective process for researchers, perhaps augmented by more formal, independent evaluation, will lead to more comprehensive compliance with Road Map principles.

Intensive consultation with community leaders regarding project concept The NHMRC states: “Community consultation needs to occur earlier in the research process … whether they are to be involved in an active or advisory capacity need to be identified so that the ability of people to participate in the research is clear.”5 The initial steps and negotiations to conceptualise and develop the research plan were conducted over a 12- month period. An important consultation step occurred in October 2011, at the annual conference of the Local Government Association of Queensland (LGAQ), the peak body for local government in Queensland. The conference had established the Indigenous Leaders Forum (ILF) of Indigenous community mayors and their councils’ executive staff.15 The focus of the ILF is to identify key challenges faced by Indigenous communities and to consider ways to engage effectively with state and federal governments to address the challenges. As one community leader stated: “... Indigenous mayors and councillors live first-hand with the outcomes of every state and federal government initiative that affects our communities. But we know what is needed and what will work and what won’t work because we are listening to our communities.” The LGAQ conference passed an ILF resolution, supported by all 73 Queensland local government councils, calling for a review of AMPs. In March 2012, during the Queensland state election campaign, both major parties promised to review AMPs. The now incumbent Premier committed during the election campaign for a review of AMPs to commence by the end of 2012.16 In May 2012, authors AC and JR were invited to the second assembly of the ILF. AC presented the draft study protocol10 for a formal evaluation of the impacts of AMPs in Cape York. ILF support for the study was unanimous. The ILF also expressed a strong desire to determine their own measures to limit alcohol-related harms and to ensure that past alcohol misuse and associated community disruption did not recur, while acknowledging the need for independent research to inform these decisions (complying

Australian and New Zealand Journal of Public Health © 2015 Public Health Association of Australia

305

Commentary

with Principles 2, 3 and 4, see Table 1). The idea that the project data for each community could inform submissions with the Department of Aboriginal and Torres Strait Islander and Multicultural Affairs (DATSIMA) was welcomed.17

Staged implementation of the project

to: i) suit the manner in which the community members were recalling the changes seen within their community connected with the AMP (complying with Principle 3, Table 1); and ii) ensure all pertinent information relevant to communities and councils in considering their approach to AMPs would be captured (complying with Principles 2 and 4, Table 1).

After ethics approval was received, project team members travelled from Cairns, Queensland, to the communities and met with the mayors, council members and other stakeholders. During these project implementation trips, a guide for semistructured interviews was tested and adjusted

Communication with key contacts including men’s and women’s groups, youth support services, local elders and community members has been ongoing and is vital to the maintenance of the good relationships established. The research team sustained good relationships within communities by

maintaining contact via email, telephone, attendance at important community events and the sharing of photographs from previous trips.

‘Whole of life’ views of health The original intention of the AMPs was to address the harm alcohol brings to physical health. Previous published evaluations have relied on quantitative data regarding physical injuries, incidents of person-to-person violence and hospital admissions to gauge the success of this policy.10,17 This evaluation, however, explores the changes the AMP policy has had on the broader social, emotional and

Table 1: Project activities mapped against the NHMRC ‘Road Map’ principles. NHMRC ‘Road Map’ Principle 1. Health being conceived of as “… not just the physical wellbeing of the body but a whole of life view, which embraces the life, death, life concept.” 2. Community involvement in the development, conduct and communication of the research.

3. Communication of research plans, progress and results.

4. Ethical research aiming to be of practical value to Aboriginal and Torres Strait Islander peoples and their service providers. 5. Research support including the enhanced development of skills, knowledge and capacity in the Aboriginal and Torres Strait Islander research workforce. 6. A focus on identifying ‘positive models’ or examples of success.

Project Concept Existing evaluations or understanding of what changes the AMP has had are relevant to physical health outcomes.

Implement Project No evidence available to support this principle.

Semi-structure interview tested and adjusted to suit community members during initial trips in May 2013.

Attendance at annual conference of the LGAQ in 2011 and 2012. Resolution passed calling for independent review of the AMPs.

Initial community trips to re-engage after funding application successful and ethics approval completed.

Early community consultation in the research process ensured information collected would help to inform the impending government review process regarding communities’ alcohol management strategies.

Feedback of findings No evidence available to support this principle.

Multiple sources included: service providers, stakeholders, councils and community members.

Considered data collection should record the broader AMP-related implications for community members and their local community. Presentation of study protocol at the conference. Project objectives and outcomes overwhelming supported by LGAQ members.

Data from community surveys would be sent to the corresponding council and service providers to assist in submissions to DATSIMA regarding amending AMP controls and conditions. No evidence available to support this principle.

Project Stage Data Collection Data collection captured information on the effects of the AMP on social, emotional and cultural health in addition to physical health.

Gradual implementation of the project afforded time to gain deeper appreciation of the regions’ history, spirituality, social structures and economic activities. Tailoring of the data collection tools ensured data would be ethically sound and the outcomes relevant to the community.

Use of a flexible study design with a participatory methodology. Survey recommended by community.

Employment of a local worker could have assisted the event.

Employment of local worker may have assisted to capture more young females during survey collection. Team practices in place to recruit local workers in subsequent communities. Community trips were every one to two months. Ongoing contact with services and stakeholders via telephone or email.

Community level feedback disseminated through double-sided one page flyers in November 2013.

Meetings with key stakeholders including Community Committees, traditional elders, and service providers to discuss progress in September 2013.

Dissemination activities could have reached larger community audience. Strategic plan recommended for future community clusters.

Most of project team had existing community relationships.

Feedback dissemination processes ensured equality across both stakeholder and community level.

Data collection methods encompassed ‘doing no harm, protection’ of participants, maintenance of trust and clear accountability’.

Project Officer provided the research team with a wealth of tacit knowledge and historical contexts of her family’s community.

Project officer’s research skills and capacity developed and enhanced.

Project Officer delivered feedback to key, respected community members.

No evidence available to support this principle.

Data collection tools (survey) allowed general community members to have access to preliminary analysis of the data during the course of the project.

Extensive consultation helped achieve to address an issue of concern for the health of the community population. Feedback of survey data gave community members access to similar study-related results provided to organisations and bureaucracies. Project data was used by one community to inform their DATSIMA application to amend their existing AMP.

306


2015 vol. 39 no. 4

Commentary

cultural aspects of health for local community members and their whole community (complying with Principle 1, Table 1).

Capacity building The research team was fortunate to recruit an Indigenous project officer with family links to one of the discrete communities (author KR). Through her involvement in all phases of the project, the project officer acquired a broad range of new research skills (complying with Principle 5, Table 1). Dissemination of the findings to recognised and respected key leaders was considered by the project officer as an honour and a highlight of her involvement in this work. This project also represented a significant opportunity to build knowledge and skills in the Community Justice Groups that were charged with the responsibility for consulting about the AMPs more than 10 years ago, but were not provided full support to fulfil their legislated role or to monitor the effects of their decisions.

Ensuring safety and protection of participants The majority of team members have preexisting professional and familial relationships and networks with Queensland’s Indigenous communities, while also bringing a range of professional and research backgrounds that include health research and practice, epidemiology, criminology, psychology, Indigenous social policy, anthropology and community development. The lead investigator (AC) has more than 25 years’ experience working in communities dedicated to improving Indigenous health. Although there were professional ties in the communities, the team was generally perceived as an independent group that was separate to council, community stakeholders and government. This meant community people were, for the most part, comfortable in sharing their opinions, experience and knowledge of the AMPs. Some community members expressed concerns over the potential disclosure of information about their own or others’ illegal activities. Examples of study feedback resources from previous projects with the community were used to demonstrate how findings would be disseminated in the community and to external audiences, with no individual participants identified (complying with Principle 3, Table 1).

2015 vol. 39 no. 4

Flexible but rigorous study design employing participatory methods A common criticism that arose during qualitative interviews with stakeholders was that there had been limited consultation with the communities during the development and imposition of the AMP policies prior to 2002. One retired community leader, when asked how best to provide the information we had begun to collect back to the community, suggested the idea of a quantitative survey to capture a wider community voice. The survey provided an additional opportunity to disseminate the findings and themes from the qualitative interviews to the wider community, since a thematic analysis informed the development of the question topics. The survey provided the basis for verifying the qualitative interview information from key stakeholders. Surveys were prefaced with an explanation that the tool was both a findings dissemination and data collection instrument. Collecting a large number of community members’ views through surveys allowed the research team to produce feedback that was directly meaningful to the communities (complying with Principle 2, Table 1). In one of the field trip notes: “…[mayor] wants to hear opinions of community, as many of them agreed to keep AMPs but he wonders if some have just said that to go along with the majority.”

location for broader feedback would be a community festival. The feedback flyers were circulated at key service sites in the nearby regional centre before the festival – providing feedback to the same organisations and sites as were involved in data collection is essential. The festival was expected to be large that year as it coincided with an anniversary of considerable historical significance; however, the gathering was smaller than anticipated. Again, employing local research workers may have assisted to engage the community at the event. Author AC was asked by the local council from one of the communities for additional findings. The findings were to be used to inform the council’s application to DATSIMA to amend particular conditions and controls of their existing AMP (complying with Principle 6, Table 1). Providing the information collected in the community directly to the council added the strength of objectively acquired evidence to the council’s application for their community planning.

A formal evaluation process is needed In summary, the team’s research practices and activities addressed the majority of the NHMRC Road Map principles because of the following key activities: • extensive community consultation with elected Councils, local government leaders and other key organisations

The research team recruited a sample reflective of the community population across broad age groups and gender. In hindsight, greater use of local workers who live in the community, particularly the employment of younger females, would have assisted the recruitment process in younger age groups.

• employment of an Indigenous project officer

Giving the information back promptly and in a manner appropriate for consumers

• inclusion of community ideas for data collection and feedback

A total of 85 semi-structured interviews adequately represented stakeholder diversity. After preliminary analysis, the qualitative findings were disseminated at key meetings of traditional leaders and service providers. Presenting the information in these forums added strength to the analysis of the qualitative information as the key themes were verified. Surveys were completed by 95 community members. Key findings were presented in a plain English two-page flyer. Some community members suggested an ideal

• a research team with professional links and diverse employment background • practical linkages between project stages • short periods between community trips maintaining enthusiasm for the research

• team meetings to discuss project decisions and debrief after community trips. We have examples of activities and practices conducted by the team that could have met the Road Map principles at project stages where no information is stated in Table 1. However, as team consensus could not be reached regarding their use, the information was not included. Importantly, reflection on our research activities demonstrates that a research project can preserve a robust methodology while facilitating knowledge dissemination and exchange together with community


307

Commentary

Acknowledgements

capacity building. This process assisted to inform community-level priorities and interests. Furthermore, it allowed us to streamline the research implementation in other communities in the study, including the dissemination of results, and to make other improvements to study practices, such as employment of local workers. Ultimately, having an integrated and co-ordinated project methodology was essential to ensuring the fulfilment of the Road Map principles. While this process identified several key findings of how to implement research guided by the NHMRC principles, these remain our own reflections. A reflective process is a critical part of any research project, especially those requiring cultural safety. However, to enhance transparency for similar projects in future, a more formal and objective evaluation framework should be designed and built into project implementation processes from the beginning, with an independent researcher or person with connections to the community conducting the evaluation. Project budgets with adequate resources would be required to meet this need. In collaboration with the research team input, feedback from other stakeholders including the community is needed to ensure the assessment is both critical and comprehensive. Taken together, this will lead to more transparent and measured evaluations of projects against the NHMRC principles in the future.

308

The authors gratefully acknowledge the support of study participants, study staff and partner organisations. The study was funded by the National Health and Medical Research Council of Australia (NHMRC, Project Grant #APP1042532), with additional support from the APHCRI-funded Centre for Research Excellence for the Prevention of Chronic Conditions in Rural and Remote High Risk Populations at James Cook University and University of Adelaide. The research is a project of the Australian Primary Health Care Research Institute, which is supported by a grant from the Commonwealth of Australia Department of Health. The information and opinions contained in this paper do not necessarily reflect the views or policy of the Australian Primary Health Care Research Institute or the Australian Government Department of Health. Alan Clough holds a Research Fellowship from the NHMRC. Funding bodies had no role in the study design, the collection, analysis or interpretation of data, the writing of the manuscript or the decision to submit the manuscript for publication.

References 1. Bessarab D, Ng’andu B. Yarning about yarning as a legitimate method in indigenous research. Int J Crit Indig Stud. 2010;3(1):37–51. 2. Smith LT. Decolonizing Methodologies: Research and Indigenous Peoples. London (UK): Zed Books; 1999. 3. Rigney L. Internationalization of an Indigenous anticolonial cultural critique of research methodologies: A guide to Indigenist research methodology and its principles. Wicazo Sa Rev. 1999;14(2):109-21. 4. National Health and Medical Research Council. The NHMRC Road Map: A Strategic Framework for Improving Aboriginal and Torres Strait Islander Health through Research. Canberra (AUST): NHMRC; 2003. 5. National Health and Medical Research Council. The NHMRC Road Map II: A Strategic Framework for Improving Aboriginal and Torres Strait Islander Health through Research. Canberra (AUST): NHMRC; 2010.

6. Donovan RJ, Spark R. Towards guidelines for survey research in remote Aboriginal communities. Aust N Z J Public Health. 1997;21(1):89-95. 7. Thompson SC, Taylor KP. Are we really committed to making a difference Reflections on Indigenous health research and dissemination. Aust N Z J Public Health. 2009;33(5):403-4. 8. Hing N, Breen H, Gordon A. Respecting cultural values: Conducting a gambling survey in an Australian Indigenous community. Aust N Z J Public Health. 2010;34 (6):547-53. 9. Jamieson LM, Paradies YC, Eades S, Chong A, MapleBrown L, Morris, P, et al. Ten principles relevant to health research among Indigenous Australian populations. Med J Aust. 2012;197(1):16. 10. Clough AR, Fitts MS, Robertson JA, Shakeshaft A, Miller A, Doran CM, et al. Study Protocol - Alcohol Management Plans (AMPs) in remote indigenous communities in Queensland: Their impacts on injury, violence, health and social indicators and their cost-effectiveness. BMC Public Health. 2014;14:15. 11. Queensland Government. Meeting Challenges Making Choices. Brisbane (AUST): The Government; 2005. 12. Fitzgerald T. Cape York Justice Study. Brisbane (AUST): State Government of Queensland; 2001. 13. Margolis S, Ypinazar V, Muller R. The impact of supply reduction through alcohol management plans on serious injury in remote Indigenous communities in remote Australia: A ten-year analysis using data from the Royal Flying Doctor Service. Alcohol Alcohol. 2008;43:104–10. 14. Margolis SA, Ypinazar V, Muller R, Clough A. Increasing alcohol restrictions and rates of serious injury in four remote Australian Indigenous communities. Med J Aust. 2011;194(10):503–6. 15. Local Governement Association Queensland. Unprecedented Turnout for Special Indigenous Leaders Forum [Internet]. Brisbane (AUST): LGAQ; 2011 [cited 2014 Feb 13]. Available from: http://lgaq.asn.au/ news/-/asset_publisher/pG32/content/unprecedentedturnout-for-special-indigenous-leaders-forum 16. Liberal National Party. LNP CANDO Action: Responsible Alcohol Management - A Community Approach [Internet]. Brisbane (AUST): LNP; 2012 [cited 2014 Feb 13]. Available from: http://lnp.org.au/wp-content/uploads/ attachments/AlcoholManagement.pdf 17. Department of Aboriginal and Torres Strait Islander and Multicultural Affairs. Review of Alcohol Management Plans [Internet]. Brisbane (AUST): DATSIMA; 2012 [cited 2014 Feb 13]. Available from: http://www.datsima. qld.gov.au/resources/atsis/government/programsinitiatives/alcohol-reforms/amp-review-process.pdf

Correspondence to: Ms Michelle S. Fitts, Community-based Health Promotion and Prevention Studies Group, Australian Institute of Tropical Health and Medicine, James Cook University, PO Box 6811, Cairns, Queensland 4870; e-mail: [email protected]


2015 vol. 39 no. 4