Clarifying program assessment categories under the Communities for Children Facilitating Partners requirement
Evidence-based, promising or emerging?
You are in an archived section of the AIFS website
At AIFS, we’ve been busy working our way through the Communities for Children Facilitating Partners alternative program assessments. Along the way we’ve received some common questions from programs and we’d like to address three of the most frequently asked questions around the program assessments:
- Why did my program’s rating change?
- What’s the difference between Promising and Evidence-Based Programs?
- Can I have my program accredited?
Why did my program’s rating change?
In 2015, there were two main ways for Communities for Children Facilitating Partners to meet the 30% requirement that came into effect in the same year. The first way involved delivering programs from the Evidence-Based Programs guidebook. The second way was to have programs assessed by AIFS as a Provisionally Approved Program or a Fully Approved Program.
In response to feedback from services, in 2016 we changed the terminology we were using. This change was intended to:
- better reflect the range of programs being submitted for assessment;
- help clarify the difference between the three levels of approval; and
- emphasise that there is a hierarchy of evidence underpinning the different categories.
For consistency, all programs assessed under the previous arrangements were made to align with the new assessment categories. Programs that were assessed as Fully Approved Programs in 2015 were subsequently renamed Promising Programs. Those that were assessed as Provisionally Approved Programs were now named Emerging Programs.
This change was understood by some providers as “downgrading” their program classification. In fact, for Promising Programs, the assessment criteria remained exactly the same as Fully Approved Programs. Therefore, there was no “downgrading” of programs as a result of this change. Instead, we now have a better-defined category that aligns with an evidence hierarchy. That is, Emerging Programs reflect new and innovative programs that are in the process of being tested, while Promising Programs were assessed by five criteria that represent minimum quality standards for a program.
Promising Programs are distinct from Evidence-Based Programs, as the latter are required to meet a number of more robust criteria (see below).
The criteria for Communities for Children Facilitating Partners program assessments is available from the CFCA. You can also read more about hierarchies of evidence or listen to a webinar about different types of evaluation.
What’s the difference between Promising and Evidence-Based Programs?
The distinction between Promising Programs and Evidence-Based Programs is the rigour of the evaluation design.
Promising Programs are evaluated using pre- and post-program outcomes measures, and with a minimum of 20 participants. These programs meet the minimum standards of good quality for the purposes of the Communities for Children Facilitating Partners program requirements.
Evidence-Based Programs have been evaluated using more scientific methods, such as randomised controlled trials. These methods provide increased levels of confidence that the program will work for the intended population and that the program is responsible for any positive changes seen after the completion of the program.
While delivering Evidence-Based Programs is desirable, the alternative program pathway recognises there may be good quality programs operating with strong program logics that match the needs of the intended target group/s, but the organisation doesn’t have the capacity or resources to conduct highly rigorous evaluations. It also acknowledges that the Evidence-Based Programs may not always be the best fit for a particular community.
The CFCA website contains stories of good evaluation practice:
- Establishing good evaluation practice: Building a better outcomes framework for families: A story from the Mallee
- Implementing evidence-based programs: Planning for the successful implementation of two evidence-based programs
Can I have my program accredited?
Sometimes the alternative program assessment process is referred to by organisations as “accreditation”. However, AIFS does not offer an accreditation service. The alternative program assessment process is only available to Communities for Children Facilitating Partners funded programs and assessment is restricted to programs meeting minimum quality standards for inclusion in the contractual requirement for Communities for Children Facilitating Partners.
We recognise that Communities for Children Facilitating Partners have sometimes found it difficult to incentivise Community Partners to engage in the assessment process and finding other ways of describing the process (such as accredited) has made this easier for Facilitating Partners. We encourage Communities for Children Facilitating Partners to explain why the five criteria are important in ensuring services are most likely to be improving outcomes for families and children. Undertaking the work to address the five criteria will also help services demonstrate to others (such as funding bodies) that their programs work, as well as why and how their programs work. Being able to demonstrate this clearly through program logics and outcomes measurement is an increasing expectation of most funding bodies.
Communities for Children Facilitating Partner Initiative is funded by the Australian Government Department of Social Services (DSS). The Facilitating Partners provide prevention and early intervention services to families and children aged 0–12 years in disadvantaged communities throughout Australia. The objectives of the program are focused on approaches that promote positive family functioning, safety and child development outcomes. Services support the wellbeing of children by building strong parenting skills and stronger and more sustainable families and communities. Read more about AIFS work to date on the initiative.
- Practitioners on evidence series
Practitioner profiles published in Family Matters, No. 99 – April 2017
- Evaluation and innovation in family support services
CFCA practitioner resource
- Planning for evaluation I: Basic principles
CFCA practitioner resource
- Implementation and adaptation of evidence-based programs
CFCA webinar – 23 February 2016