Sayers, M., Coutts, M., Goldfeld, S., Oberklaid, F ...

4 downloads 0 Views 698KB Size Report
Sayers, M., Coutts, M., Goldfeld, S., Oberklaid, F., Brinkman, S., & Silburn, S. (2007). Building Better Communities for Children: Community Implementation and ...
Sayers, M., Coutts, M., Goldfeld, S., Oberklaid, F., Brinkman, S., & Silburn, S. (2007) Building Better Communities for Children: Community Implementation and Evaluation of the Australian Early Development Index, Early Education and Development, Vol. 18, No. 3: pages 519–534 Building Better Communities for Children: Community implementation and evaluation of the Australian Early Development Index

Abstract Since 2004 the Australian Early Development Index (AEDI) has been completed in 54 Australian communities over seven states and territories on more than 30,000

children.

A

concurrent

systematic

evaluation

of

community

implementation and use of the AEDI was undertaken that included both a process and impact component. The purpose of this paper is to discuss the evaluation findings between 2004 and 2006. Results have shown there are a number of barriers and facilitators for communities in implementing the AEDI, disseminating results and planning for community actions.

The evaluation

findings indicate AEDI implementation helps raise awareness of the importance of early childhood development, assists communities to work more collaboratively and to strategically plan actions to work towards improving outcomes for children.

The successful Canadian development and implementation of the Early Development Instrument (EDI) (Janus & Offord, 2000; Hertzman, 2002; Hertzman, 2004; Mort, 2004) as a population measure of children’s development was a critical catalyst to its uptake in Australia. Of particular value has been the convincing demonstration of the community benefits from

Implementation and Evaluation of the AEDI 2 implementing the EDI as well as the more obvious benefits that arise from having small area level data on children’s development at school entry. A Canadian evaluation study found a number of key benefits in communities that had implemented the EDI in British Columbia (Mort, 2004). It was noted that the implementation process facilitated the forging of new relationships between stakeholders with an interest in the development of young children within communities. All school districts that had implemented the EDI were unanimous in attributing the EDI process and results to assisting the development of new, or consolidating of existing inter-sectoral early childhood coalitions. Teachers reported feeling valued by the process and felt it gave them credibility in the profession and community. The results were being utilised within communities to assist in establishing resource allocations, district-planning processes and in discussions with community agencies. Given the successful implementation of the EDI in Canada and the timely requirement for population level child development data in Australia, the Centre for Community Child Health in Melbourne, in partnership with the Telethon Institute for Child Health in Perth, undertook a project to adapt the Early Development Instrument (EDI) for Australia, trial the Australian version (the Australian Early Development Index-AEDI) in up to 60 communities around Australia over three years, and fully evaluate the implementation of the AEDI in communities. The AEDI is an adapted version the EDI and is a population measure of young children's development from a teacher-completed checklist on children in the first year of school. There is no uniform school starting age across the Australia as each state and territory government oversees their respective educational systems. Across eight states and territories there are five different minimum school starting ages (MCEETYA, 2006). Additionally there are varying levels of preschool or kindergarten participation. Therefore the AEDI is

Implementation and Evaluation of the AEDI 3 completed in the first full-time year of schooling in primary school. The AEDI is administered by teachers after they have had a chance to observe each child’s development (usually after the child has been at school for six months). Between 2004 and 2006 the average age of AEDI completion was 5.95 years, however average ages ranged 5.6 years in New South Wales to 6.4 years in Queensland. The AEDI measures five domains of children’s development: physical health and well-being; social competence; emotional maturity; language and cognitive skills; and, communication skills and general knowledge Goldfeld et al. (this volume) describes the processes used to adapt the Canadian EDI for Australia and Brinkman et al. (this volume) reports on the construct and concurrent validity of the AEDI. This paper outlines the community implementation of the AEDI in Australia and the results of the three year process and impact evaluation of the community implementation of the AEDI. Community Implementation of the AEDI in Australia In developing the national implementation of the AEDI there were a number of key factors to consider, including the vast geographic size of the country and the different state and territory government structures and educational systems. To address these issues a number of measures were taken to help facilitate community implementation including the establishment of an AEDI National Support Centre (a central contact hub designed to support communities), the development of a paper-less, web-based data entry system and a range of guides and materials to support communities to implement the AEDI (Goldfeld et al., 2006). A key component of the implementation was to support and empower local leaders or champions to engage their own communities in the AEDI, rather than be reliant on the National Support Centre staff to actively be involved in community briefings or engagement.

Implementation and Evaluation of the AEDI 4 This model differs from British Columbia, Canada, for example, where the Human Early Learning Partnership staff are actively involved in making presentations in communities both before the EDI implementation and after the results are available (Mort, 2004). While there are a number of strategic benefits to this approach, particularly the capacity to discuss the results with a high level of authority and “draw a crowd”, adopting such a model across Australia would be unsustainable because of the large geographic distances. Therefore a key priority of the AEDI National Support Centre was to support the leaders remotely by providing them with a range of guides, tools and resources. A series of guides were developed to assist communities to successfully implement the AEDI, understand the results and initiate action as a result of the AEDI findings. The key guides developed were, the Community Selection Guide, the Community Preparation and Implementation Guide and the Community Dissemination and Action Guide. The Community Selection Guide provided information about the support and resources provided by the AEDI National Support Centre, and information about the AEDI implementation process to assist communities to determine their capacity to implement the AEDI. The Community Preparation and Implementation Guide was developed to assist the Local AEDI Coordinator prepare for the AEDI implementation by providing detailed information, tools and resources to support them throughout the implementation such as PowerPoint presentations used to engage schools and the wider community in the AEDI, useful letters for schools, and planning templates. The Community Dissemination and Action Guide was developed to assist the Local AEDI Coordinator and other community members understand the AEDI Community Profile (data and maps), and provide ideas and resources on how to successfully disseminate the AEDI results within the community and develop an action plan. These three guides were revised annually to take into account the feedback captured in the evaluation of the project. There was also a series of Fact Sheets developed for the project including: Frequently Asked

Implementation and Evaluation of the AEDI 5 Questions; Understanding the Results; Community Benefits; and, the AEDI Domains Fact Sheet (http://www.rch.org.au/australianedi/com.cfm?doc_id=8775#AEDI_Fact_Sheets). The AEDI results were provided to communities in the form of an AEDI Community Profile. Age based cut offs (4,5 and 6 year old cut-offs) were developed for analysing the AEDI data to take into account the differing school starting ages across the country. The Community Profile contains background information about the participating schools and children, and the AEDI results, in both table and map format. The maps were created in MapInfo version 7.0 (and CDATA2001: Australian Bureau of Statistics Census data). To help in understanding the role that other socio-economic and demographic factors may play on child development, the Community Profile also included geographic maps that display the distribution of socio-economic, demographic, health and education factors obtained from a range of national or state data sources. The socio-demographic maps use the same suburb or area boundaries as the AEDI maps. All maps were posted on the AEDI website once they were released to communities (http://www.rch.org.au/australianedi/results.cfm?doc_id=9180). The methodology for providing AEDI results to the community and the wider public in map format was based on the experience in British Columbia where this had been an effective process for communicating the results widely and presenting the results in communities (Hertzman et al., 2002; Mort, 2004; Hertzman, 2004, Kershaw et.al., 2005) The AEDI data and maps can assist in identifying variations in child development across different parts of the community including differing average scores and proportions of children developmentally vulnerable in the geographical areas of the community and where the strengths and vulnerabilities lie across the domains of child development. By providing the AEDI maps together with socio demographic maps it enabled communities to consider the influence of socio-economic and community factors on child development and how well the community is supporting young children and their families by investigating the community

Implementation and Evaluation of the AEDI 6 resources available to families with young children in neighbourhoods where children do well and where they do not. Table 1 provides a summary of the AEDI community implementation model. (Insert Table 1 here) Between 2004 and 2006 there were three Expression of Interest rounds (where communities could self-nominate to participate in the AEDI). A key requirement for communities to implement the AEDI was that they could confirm the participation of more than 80% of schools in their community (both government and non-government schools) and they could raise funding to provide for the teacher backfill (i.e. funding required to compensate schools for the time taken by teachers to complete the AEDI checklists). Communities were also required to nominate a Local AEDI Coordinators to assist in the implementation process. Over the three years of the project, these coordinators came from a wide array of disciplines and backgrounds (e.g. community development workers, health promotion professionals, school principals). Between 2004 and 2006, 77 communities applied to implement the AEDI, and 54 communities actually completed the AEDI. A summary of the school and community implementation in the AEDI trial is shown in Table 2. (Insert Table 2 here) Evaluation of the AEDI Alongside the national implementation of the AEDI has been a comprehensive evaluation strategy undertaken by the Centre for Community Child Health. The broad purposes of the evaluation included: monitoring the implementation of the project; investigating the effectiveness and utility of the AEDI as a community-planning tool to support children’s health, development and wellbeing; ascertaining any barriers the project encountered and suggesting possible solutions; documenting exemplary or innovative practice

Implementation and Evaluation of the AEDI 7 related to the AEDI that could be transferred to other sites; and making recommendations regarding the further support and implementation of the AEDI. The evaluation incorporated both a process and an impact evaluation. Methods A combination of qualitative and quantitative methods were used to collect data from communities who implemented the AEDI, including surveys and interviews. A key goal of the evaluation was to capture the learnings from communities progressively so the implementation model could be adapted to meet the needs of communities. This methodology was extremely useful as the annual evaluation findings enabled a continuous improvement regime, particularly in refining all tools and resources provided to communities, reporting and mapping process and the development of the quality assurance protocols. The evaluation cycle and how it was linked into the project development process is illustrated in Figure 1. (Insert Figure 1 here) Evaluation data were gathered from communities at multiple time points, from when they applied to join the project (Expression of Interest and Selection Application Forms), just prior to implementing the AEDI (Pre-implementation Survey), after implementation (Teacher Evaluation Survey), three months after they received their results (Post-implementation Survey) and one year post-implementation of the AEDI. Additionally a number of semistructured interviews were conducted with communities three to four months after they received their results. The respondents to the evaluation were varied, from Local AEDI Coordinators, teachers who completed the AEDI checklists and school principals. A summary of the evaluation/ measurement tools and the participants involved in the evaluation is provided in Table 3. (Insert Table 3 here)

Implementation and Evaluation of the AEDI 8 The response rates for the evaluation tools were Expression of Interest (100%, n=521) and Selection Application Form (96% n=74/77), Pre-implementation survey (61% over three years, n=47/77), Teacher Evaluation Survey (70% over three years, n=1,308/1,868), Postimplementation survey (52% over first two years, n=14/27 communities who completed the AEDI in 2004 and 2005). There was insufficient time in the evaluation to collect the Postimplementation Survey from the 2006 communities. Additionally there were 12 community visits made to the 2004 and 2005 communities (44% n=12/27) and 51 schools that were either visited or interviewed over the phone over the three years. Thematic analyses were undertaken to identify common themes using both the survey and interview data. Evaluation Findings Pre-implementation of the AEDI. The pre-implementation stage covers the period from when a community decides to implement the AEDI up until they have the agreement of more than 80% of schools in their community to complete the AEDI. During this time the Local AEDI Coordinator must engage schools as well as other key stakeholders in the community in order to enable the AEDI data collection to go ahead and to obtain funding for teacher backfill (funding required to compensate schools for the time taken by teachers to complete the AEDI checklists) from within the community or from external bodies. From the information collected in the surveys and interviews, community respondents (Local AEDI Coordinators and school principals) suggested there were a number of enablers in the pre-implementation phase including both internal and external facilitating factors. Internal factors that facilitated uptake of the AEDI included: the existence of early childhood initiatives in the community that could foster the work of the AEDI; the availability a local leader; strong partnerships between services and sectors; established services for children; 1

In 2004 and 2005 there was a two-staged selection process for communities applying to implement the AEDI was adopted where communities were required to complete an Expression of Interest Form and a Selection Application Form. In 2006

Implementation and Evaluation of the AEDI 9 cooperation from schools; and an easily identifiable geographic location making identification of key players more straightforward. External facilitating factors included: the support provided by the AEDI National Support Centre including the guides, tools and resources; support from state-based champions of the AEDI and their influence in ‘selling’ the AEDI to schools; and encouragement for the project by other external bodies such as state government and regional education bodies. The two major barriers identified by communities in the pre-implementation phase were engaging schools in the project and accessing funding for teacher backfill. In engaging schools the majority of communities (91%, n=43/47) used multi-modal methods including telephone and e-mail contact, mailed information, individual face-to-face meetings and forums with principals. Some of the barriers reported by the Local AEDI Coordinators in engaging schools were: the competing assessment and testing requirements placed on schools; the added workload for teachers to complete the AEDI; effectively capturing principals’ attention; change in principals and the time required to re-engage the new principal; concerns about teachers being absent from the class in order to complete the AEDI; and the time required engaging schools in the AEDI project. However, despite the barriers encountered by the local leaders, with the exception of eight communities, the 54 communities who have implemented the AEDI were able to meet the requirement for more than 80% of schools to participate in the project. School participation rates for the eight communities below the 80% schools participation rate ranged from 65 to 79%. Whilst these communities completed the AEDI, in particular suburbs or neighbourhoods that had less than 80% schools participation, a disclaimer was placed on the results for these areas. In exploring the reasons why schools became involved in the project principals have reported the most common reason being to

communities were only required to complete one form, the Selection Application Form.

Implementation and Evaluation of the AEDI 10 obtain information about the needs of the children in their school and to obtain an evidence base to support the issues they had already noted anecdotally as impacting on their children. Local AEDI Coordinators reported difficulties in accessing funding for the teacher backfill. The inability of some communities to raise the required funds meant they were unable to participate in the project in the year they volunteered to join the project. However, on the whole the majority of communities that successfully implemented the AEDI were able to acquire the funding for the teacher remuneration. Examples of funding sources included state and federal government community capacity building funding, local businesses and inkind support by regional education authorities. Figure 2 provides a summary of the community implementation by year and illustrates that not all communities were able to implement the AEDI in the year they originally nominated to join the project. (Insert Figure 2 here) Over the three years, 77 communities applied to participate in the AEDI project, with 70% (n=54) of applicants completing the AEDI in their community. Of these, 41 (76%) communities completed in the year of application, and 13 (24%) completed in following years. One community took three years to complete the project (applying in 2004 and completing in 2006), and a further 12 communities took two years to complete the AEDI (applying in 2005 and completing in 2006). The evaluation results highlight that the necessity to raise funding for teacher backfill and engaging schools in the project provided barriers for some communities to be able to complete the AEDI. Implementation. The implementation period covers the time from when teachers are provided with the school-codes to access the web-based data entry system until all the AEDI data have been collected in the community. All schools in a community are required to

Implementation and Evaluation of the AEDI 11 complete the AEDI checklists over a one month period. Between 2004 and 2006, the majority of communities completed the AEDI in Term 3 of the school year (after the teachers have known the children for around six months). The capacity for teachers to enter data through the web based data entry system was a critical component in establishing the AEDI as a national tool. Utilising the web enabled rapid data uploading and cleaning and the ability for short turn around times for reporting back to communities. The web-based data entry system also validated the data as they were being entered by teachers, providing for an instant check of whether answers entered were in the valid range and that all items were completed. Goldfeld et al. (this volume) describes the development of the web-based data entry system. Teachers are provided with a comprehensive Guide for Teachers which provides technical instructions on how to use the web based data entry system and a guide on how to answer the specific questions in the AEDI checklist (based on the EDI Guide for Teachers: http://www.offordcentre.com/readiness/teachers.html). Additionally on the web-based data entry system there is on-line help to guide teachers on how to answer the AEDI questions. Other than the Guide for Teachers, no formal training was offered to schools and teachers. Teachers were surveyed at the end of the data entry period for their feedback on the ease of completion of the checklists, the use of the system and any benefits for teachers and the community. This survey also enabled for continuous improvements of the web-based data entry system as teachers were asked to provide feedback on any difficulties with the system. The key findings from this survey are in Table 4. (Insert Table 4 here) The results from this survey indicated that teachers did not have difficulties completing the AEDI on-line or answering the AEDI checklist questions and that they could see the benefits for their own work in the classroom and for the wider community in promoting a better understanding of children in their area.

Implementation and Evaluation of the AEDI 12 Post-implementation-Dissemination of the results. Once all the AEDI data are entered in a community, all the AEDI data were downloaded by the AEDI National Support Centre for cleaning, analysis and mapping. Due to the data validation processes built into the web-based data entry system there was very minimal missing data or cleaning required. Additionally, turnaround times from when teachers finished the data entry to when communities receive their results (AEDI Community Profile) was very short (2-4 months). After receiving the AEDI Community Profile communities reported disseminating the results to a wide range of groups. This included: school principals; early years strategy groups/initiatives; early years professionals; regional or district education bodies; community members; local and state government bodies; paediatric professional groups; maternal and child health professional groups; and the media. While some Local AEDI Coordinators withheld disseminating the AEDI Community Profile until they felt confident in understanding and contextualising the results, other coordinators quickly disseminated to all relevant organisations within the community. Many Local AEDI Coordinators reported holding community forums to disseminate the AEDI results and to bring various community members together to discuss how they might respond to the AEDI. Attendances at the forums varied and included: members of the community; representatives from health, education, local schools, childcare, pre-school, local government, community development and urban renewal programs; and other relevant professionals and/or organisations. Typically these forums were used to explore the results in greater depth, with a focus on understanding the local context of the results by examining the socio demographic data provided alongside the AEDI results and local community knowledge about the resources and services available to families and young children. Post-implementation-Community mobilisation and actions. From the evaluation there were four key outcomes noted by Local AEDI Coordinators and school principals in

Implementation and Evaluation of the AEDI 13 communities that have implemented the AEDI. These were: the implementation of the AEDI increased community awareness of the importance of early childhood development; the implementation assisted communities to build stronger relationships and work more collaboratively; the AEDI results were helpful in promoting strategic development and planning for the needs of children; and the AEDI results supported funding grant applications. 1. Raising awareness of the importance of early childhood development For the majority of communities that have completed the AEDI, the evaluation findings showed that the AEDI helped mobilise communities in terms of raising awareness of early childhood development. In interviews many of the Local AEDI Coordinators discussed the usefulness of the AEDI in raising the profile and importance of the early years, particularly to sectors that have a broader focus than the early years. The coordinators suggested the results provided valuable, evidence-based, nationally comparable data that accurately described the wellbeing of a community’s children. Many of the community respondents (school principals and Local AEDI Coordinators) found the AEDI results encouraged them to reflect on specific factors impacting on early child development within their community. Such factors included the transition period from early childhood settings to school, nutrition, physical activity, adequate clothing, emotional development and experiences in care before entering school. Local AEDI Coordinators also noted the benefits of having the AEDI results mapped as it gave them a visual representation that they were able to use as evidence to create awareness around supporting investment in the early years. They were able to use the results to initially bring various professionals together and to then plan a strategic response to the results presented by the AEDI.

Implementation and Evaluation of the AEDI 14 2. Building relationships and working collaboratively. One of the significant findings to date from the evaluation was the reflection, from an array of community professionals, that the AEDI was instrumental in forging stronger relationships between key organisations and sectors, particularly between schools and other early childhood professionals. The AEDI was considered the mechanism for creating “conversations” and strengthening relationships between schools, district education offices and early years professionals across a number of sectors. 3. Asset mapping, planning and strategic development. Communities were using the AEDI results together with other information such as national and local statistics, community feedback, and service provider feedback to examine how they were better able to address the issues raised. Some communities were in the process of mapping other community resources, services and infrastructure together with the AEDI results to assist in the planning process. The AEDI data prompted some service providers to review how existing services and programs were delivered and to think about the types of new initiatives that might be needed. Community respondents reported that the AEDI influenced the key strategic areas of policy, service delivery, levels of funding, service resource planning, how organisations worked together and interagency collaboration. Figure 3 showed the Local AEDI Coordinators perception that the AEDI would impact ‘to a great extent’ and ‘to some extent’ on six key strategic areas in the post-implementation phase. At least 70% of coordinators reported that each area would be impacted, with service and resource planning and interagency collaboration perceived to be the greatest areas of impact. (Insert Figure 3 here)

Implementation and Evaluation of the AEDI 15 4. Application for grants. Many agencies, such as schools, local government, community organisations, and early years groups used the AEDI data to support grant and funding applications. Examples included grants for: playgroups for Aboriginal families; community renewal funding; upgrading infrastructure; reading programs; and other targeted strategies. Overall the AEDI evaluation highlighted that professionals were keen to respond to the AEDI findings with action, however many communities found that responding takes considerable time and resources. Communities varied considerably in their response to the AEDI results, with some initiating a great deal of action and some yet to demonstrate any action. In reviewing the evaluation findings there are two common themes that appear to be important within a community in order to promote action: that at least one person within the community is a ‘champion’ of the AEDI; and the community can identify a process for moving forward after they have received their results. There also needs to be a high degree of understanding and support from executive levels of key organisations and departments. Discussion The key findings from the evaluation are that communities can successfully implement the AEDI, understand and disseminate the results. In communities that implemented the AEDI, the implementation process and results promoted community mobilisation around early childhood and facilitated strategic planning and action. Communities reported that having local leadership and AEDI champions is critical to the success of the AEDI implementation and dissemination, mobilisation and community action. Some communities have difficulty in implementing the AEDI because of lack of support from the schools sector and accessing funding for teacher backfill. Support from educational authorities and state governments assists communities to more easily implement the AEDI. The evaluation

Implementation and Evaluation of the AEDI 16 findings have also shown that communities want to respond to the AEDI with evidence-based community actions to improve the outcomes for children, however responding to the results takes considerable time and resources. The evaluation findings of the implementation of the AEDI in Australian communities support the earlier evaluation findings in British Columbia (Mort, 2004). As in British Columbia, the Australian evaluation findings illustrate that the community implementation process and AEDI results can facilitate the development of community partnerships and coalitions, raise awareness of the importance of early childhood development within the community, and assist communities to map and understand their assets and identify priorities for action to improve outcomes for children. The results from the evaluation also highlight the need for local champions and strong support from the education sector to successfully implement the AEDI. In planning for the AEDI implementation in Australian communities, many features of the original EDI’s implementation model were adopted. Additional features of the Australian model included teachers completing the AEDI on a web-based data entry system, establishing a national support centre, and the development of a series of guides and resources to assist communities implement the AEDI, understand the results and initiate action as a result of the AEDI findings. The evaluation findings confirm that the Australian approach is an appropriate model for implementing the AEDI across a geographically vast country. The evaluation met with two limitations. The first limitation was the response rate was lower than desired for the Pre-implementation Survey (61% over three years) and PostImplementation Survey (52% over two years). Whilst conducting semi-structured interviews with a sample of communities was most effective for obtaining very rich qualitative data (n=12 Local AEDI Coordinators, n=51 school principals), it was not possible due to cost constraints to conduct interviews with all communities. However, despite the response rates,

Implementation and Evaluation of the AEDI 17 both the interviews and the surveys elicited similar themes and thus support the validity of the findings. The second limitation was the short time frame in which to investigate community action arising from the AEDI. However, despite this limitation, the themes that arise from the communities indicate evidence of community mobilisation as a result of implementing a population measure of children’s development. Conclusion A concurrent comprehensive evaluation alongside the national implementation of the AEDI has been a critical component of the process. Communities have demonstrated they are keen to learn from the experiences of other communities and have valued the evaluation feedback that has been incorporated into all the guides and the website. A long-term longitudinal evaluation beyond the initial three years of the current evaluation will be required to assess how the AEDI has contributed to communities’ efforts to make sustainable change and actions to improve children’s outcomes over the longer term.

Acknowledgements The Australian Early Development Index: Building Better Communities for Children Project (AEDI Project) was funded by the Australian Government Department of Family and Community Services was conducted by the Centre for Community Child Health, Melbourne in partnership with the Telethon Institute for Child Health Research, Perth.

Implementation and Evaluation of the AEDI 18

References Brinkman, S., Silburn, S. Lawrence, D., Goldfeld, S., Sayers, M., & Oberklaid, F. (this volume). Construct and concurrent validity of the Australian Early Development Index. Early Education and Development. Goldfeld, S., Brinkman, S., Sayers., M., Liddelow, T., Daws, A., Coutts, M., Muller, W., Silburn, S., & Oberklaid, F. (2006). A national approach to measuring child development in Australia: Process challenges and policy implications. Paper prepared for the Measuring Early Child Development Conference, Vaudreuil, Canada. Goldfeld, S., Sayers, M., Oberklaid, F., Brinkman, S., & Silburn., S. (this volume). The process and policy challenges of adapting and implementing the Canadian Early Development Index in Australia, Early Education and Development. Hertzman, C. (2004). Making early childhood a priority, lessons from Vancouver. Canadian Centre for Policy Alternatives, BC Office, May. Hertzman, C., McLean, S., Kohen, D., Dunn, J., & Evans,. T. (2002). Early development in Vancouver: Report of the community asset mapping project (CAMP). Human Early Learning Partnership, Vancouver. Janus, M. (2006). Measuring community early child development. Canadian Association of Principals Journal, 14(3). Janus, M.. & Offord, D. (2000). Readiness to learn at school, Canadian Journal of Policy Research, 1(2). Kershaw, P., Irwin, L., Trafford, K., & Hertzman, C. (2005). The British Columbia atlas of child development. Human Early Learning Partnership and Western Geographical Press, Vol 40. MCEETYA (Ministerial Council on Education, Employment Training and Youth Affiars), (2006). Final repor:Cost/benefit analysis relating to the implementation of a common school starting age and associationed nomenclature by 1 January 2010. Vol 1. (http://www.mceetya.edu.au/verve/_resources/Volume_1_CSSA_Final_Report.pdf) Mort, J. (2004). The EDI (Early Development Instrument) impact study. Human Early Learning Partnership, Vancouver. (http://www.earlylearning.ubc.ca/documents/EDI%20Impact%20Study%20%20Oct%201%202004.pdf)

Implementation and Evaluation of the AEDI 19 Table 1. Community AEDI implementation model 2004-2006 Task

AEDI National Support Centre materials/support

Local AEDI Coordinator and community coalitions

Preparation phase

• Community Selection Guide

• Engage the community in the

• Community Preparation and

AEDI • Source funding for teacher remuneration • Engage district/regional education authorities • Engage and confirm the participation of schools (more than 80% in the community and 100% in rural areas)

Implementation Guide • Telephone and email support to Local Coordinator

Implementation phase

• AEDI Guide for Teachers

• Distribution of web-based data

• Administration of web-based data

entry access codes and implementation materials to schools • Tracking schools implementation

entry system • Telephone and e-mail support to Local AEDI Coordinator and teachers Data Analysis/ Mapping/ Reporting

• All data analysis and mapping

Post-implementation phase Dissemination and actioning results

• Community Dissemination and

• Understanding the results in

Action Guide • Posting all community AEDI maps and tables on the website www.australianedi.org.au • Telephone and email support to Local Coordinator • Annual training session for Local AEDI Coordinators

order to present/ disseminate the results to the community • Dissemination of results • Working with the community to take action as a result of the findings and develop and action plan

• AEDI Community Profile

• Liaison with the National

Support Centre about mapping boundaries

Implementation and Evaluation of the AEDI 20 Table 2. 2004-2006 AEDI participation Year

No of children

No of schools

No of Teachers

No of communities

2004 2005 2006

6,261 12,358 13,310

152 329 389

322 719 827

8 19 27

Total

31,929

870

1,868

54

Implementation and Evaluation of the AEDI 21 Table 3. AEDI Evaluation Responses Evaluation tools/methods

Respondents

2004 2005 2006 communities communities communities

TOTAL

Pre-implementation Expression of Interest

Local AEDI Coordinators

19

33

N/A

52

Selection Application

Local AEDI Coordinators

10

30

34

74

Pre-implementation Survey

Local AEDI Coordinators

10

17

20

47

Teachers who completed AEDI

194

402

712

1,308

Post-implementation Survey

Local AEDI Coordinators

3

11

N/A

14

Site Visits (communities)

Local AEDI Coordinators

6

6

N/A

12

Site Visits (schools)

School principals

10

15

N/A

25

Telephone interviews

School Principals

0

0

26

26

252

514

792

1,558

Implementation Teacher Evaluation Survey (after completion of the AEDI) Post-implementation

Total

Implementation and Evaluation of the AEDI 22 Table 4. Results from teacher survey 2004-2006 Feedback 2004 - 2006

(n= 1,308 teachers)

I found the web-based data entry system easy to use

1,255 (96%)

AEDI Checklists easy to complete for most or all children

1,217 (93%)

My involvement in this project will assist our community to better understand the health, development and wellbeing of children in our area

1,094 (84%)

The experience of completing the AEDI checklists will be beneficial to my work

817 (63%)

Completing the AEDI checklists was a good use of my time

807 (62%)

Implementation and Evaluation of the AEDI 23 Figure 1. Development of the AEDI Project

Evaluation Results

2004 Community Guides and Resources (Version 1)

2004

2004 Project Design

2005 Community Guides and Resources (Version 2)

2005

2005 Project Design

2006 Community Guides and Resources (Version 3)

2006

2006 Project Design

Community Guides and Resources

Implementation and Evaluation of the AEDI 24

2006

2005

2004

Figure 2. AEDI Expression of Interest rounds

Selected to complete the AEDI

Preparation phase

Completed data entry of the AEDI checklist 7 communities*

At least 80% school participation in 5 communities

10 communities

10 communities

Did not start data entry 3 communities

Less than 80% school participation in 2 communities

Selected to complete the AEDI

Preparation phase

Completed data entry of the AEDI checklist 19 communities

At least 80% school participation in 16 communities

33 communities

33 communities

Did not start data entry 14 communities

Less than 80% school participation in 3 communities

Completed data entry of the AEDI checklist 27 communities

At least 80% school participation in 24 communities

Did not start data entry 7 communities

Less than 80% school participation in 3 communities

Selected to complete the AEDI

Preparation phase

34 communities

34 communities

*Plus one community that piloted the web-based data entry system and all materials and guides

Implementation and Evaluation of the AEDI 25 Figure 3: Proportion of Local AEDI Coordinators that reported the AEDI will impact ‘to a great extent’ and ‘to some extent’ on six key strategic areas (n=14*) 100%

90%

80%

70%

Per cent

60%

50%

40%

30%

20%

10%

0% AEDI impact on policy

AEDI impact on service delivery

AEDI impact on determining levels of funding

* Source: Post Implementation Survey

AEDI impact on service and resource planning

AEDI impact on how organisations work together

AEDI impact on the promotion of interagency collaboration?