Frequently Asked Questions for Communities for Children Facilitating Partners

Evidence-based program guidebook

How were programs chosen?

Programs selected for the guidebook were chosen from a range of international databases and clearinghouses of evidence-based programs (EBP) and/or promising practice profiles. In order to ensure greater coverage of programs that have been developed and implemented in Australia, which international clearinghouses may not have assessed, a range of Australian best practice and promising practice guides and assessments were also searched.

The criteria for inclusion in the evidence-based program guidebook have been developed with these issues in mind. In addition, programs need to be easily replicable and training or manuals easily accessible in Australia.

Back to top

How do I know which program to choose?

Knowing which evidence-based program is “right” for your setting involves making judgements in relation to three factors:

  1. Program match: How well does the program fit with your purposes, your organisation, the target audience and the community in which it will be implemented? How well does the program match the level or complexity of risk factors or current problems among potential participants?
  2. Available resources: What expertise, personnel, financial support and time will be needed to implement the program? What training and/or qualifications are needed to run the program?
  3. Program quality: The program should have a solid evidence base to show that it is effective. The process that CFCA has used to choose the programs in the guidebook is sufficient to ensure that these programs are of good quality.

Listen to Elly Robinson discuss choosing the right program:

This recording is an excerpt from a CFCA webinar, Evidence, evidence-based programs and evaluation. Access the full recording, including a transcript and slides.

Read more about each of these areas in our guide to  Selecting an evidence‐based program.

Back to top

Can we adapt the programs listed in the guidebook to suit our potential participants and/or the resources we have available?

You need to contact the developer of the program to discuss any proposed adaptations to the program or if they know of any existing adaptations that may meet the needs of your target group.

Discussing any adaptations with program developers is important as there needs to be consideration of the effects any changes may have on the “fidelity” of the program, that is, staying true to the original design in order to replicate the program outcomes. What may make the program workable for your organisation or help attract participants may reduce or eliminate the effects of the program.

Adaptations that make the program a better cultural “fit” for potential participants, such as modifying the language used or replacing images of children and families to those that reflect the target audience, are less likely to have an effect on the program outcomes. Eliminating parts of the program or reducing the number of sessions are examples of adaptations that are more likely to reduce the impact of the program.

Listen to Elly Robinson discuss adaptation:

This recording is an excerpt from a CFCA webinar, Evidence, evidence-based programs and evaluation. Access the full recording, including a transcript and slides.

Read more about fidelity and adaptation (PDF 305 KB).

Back to top

Assessment of alternative programs

How do I submit an alternative program for approval by CFCA?

CfC FPs can submit alternative programs for consideration to be included in the 50% requirement. These programs need to meet the requirements of either a promising program or an emerging program to be included in the 50% requirement.

The process of submitting a program to be assessed by CFCA is as follows:

Step 1: Discuss your intention to submit the program with your Grant Agreement Manager.

Step 2: Read the assessment criteria and decide on the most appropriate approval category. You will be asked to select an approval category when submitting the program.

Step 3: Prepare any documentation that you will use to demonstrate that the program meets the relevant assessment criteria.

For promising programs, you can either:

Emerging program submissions should include program logic and an evaluation plan and timeline. A program logic template is available, if you would like to use it.

Step 4: Once you have all your documents ready, you can submit them by completing the submission webform and attaching any documents.

Step 5: CFCA researchers will assess the documentation against the criteria for the chosen approval category and provide feedback that indicates one of the following:

  • Meets the criteria for a promising program and can be included in the 50% requirement.
  • Meets the criteria for an emerging program and can be included in the 50% requirement.
  • Eligible to be included in the 50% requirement AND evidence-based program guidebook. To be included in the Guidebook, a program would have to meet additional criteria.
  • The program cannot at this stage be included in the 50% requirement. Programs that receive this outcome can re-submit provided that the assessment feedback is addressed.

If you are an existing or prospective Community Partner and you wish to submit your program for assessment, please discuss this with a relevant Facilitating Partner. Programs can be submitted by Facilitating Partners only.  

Back to top

How long before we will know if our program can be included?

The program will be considered by CFCA researchers for inclusion. Depending on the volume of submissions, it is estimated that the process will take between 4–6 weeks.

Back to top

What do you mean by a program being “documented”?

Program documentation should provide clear information on the fundamental elements of the program, creating a common understanding of the program for all involved. Documentation needs to be in a written format.

You may have one document that covers all five criteria or a number of documents. For example, you may have a separate document regarding the program evaluation and another that covers the other criteria.

Examples of documentation could include:

  • a full program manual;
  • a specific training manual;
  • practice guidelines;
  • session outlines;
  • activity worksheets; and/or
  • handouts, “homework” or other resources for use with clients.

Back to top

What do you mean by a theoretical and/or research background?

The broad theories and/or research that informed the development of the program need to be clearly outlined, and you need to indicate where the theories or research were sourced from (i.e., the reference). Theoretically based means:

  • The documentation details how the program is based on a theory, or explanatory framework, that describes the relationship between activities and outcomes. For example, attachment theory shows that forming a secure attachment to a primary caregiver leads to good outcomes for children. A program may include activities that help the formation of secure attachments, such as increasing a parent’s ability to recognise and effectively respond to a baby’s distress.
  • Theories should have been empirically tested: that is, a formal process of data collection has been undertaken that shows the expected outcomes. For example, researchers have studied attachment in a range of circumstances, and the results of the research support the theoretical principles of attachment.
  • Relevant theories must be ones that are generally accepted in the social science literature as having an evidence base.

Other examples of theoretical or research frameworks in the family support area are cognitive behavioural theories, ecological model and social learning approaches. The program (or practices that underpin the program) must demonstrably draw on commonly understood principles in these theories. Alternatively, the program can reflect a broader evidence base that informs effective responses to the needs of your target group.

Back to top

What is a program logic?

Program logic is a critical element in an evaluation because it sets out a graphic and easily understandable relationship between program activities and the intended outcomes of the program. Two words are important to consider in this definition:

  • Relationship: a logical link should be seen between each stage of the program logic model. There should be a clear relationship between the activities that are being undertaken, and the outcomes that are proposed.
  • Intended: the program logic model serves as a road map for the program. The model shows why the activities, if implemented as intended, should effectively reach the desired outcomes.

By referring to the model on a regular basis, you will have a deeper understanding about where things can go or have gone right or wrong, and the model can be adapted accordingly. It is also a good tool for educating and informing staff and other stakeholders about the program and the evaluation. Each part of your logic model needs to have enough detail so that it is apparent from examining the model what you need to do or to measure in order to demonstrate whether the intended changes occur.

Other terms that are commonly used for models that depict a similar causal pathway for programs are theory of change, program theory and logic models - any of these are acceptable.

Listen to Elly Robinson discuss program logic:

This recording is an excerpt from a CFCA webinar, Evidence, evidence-based programs and evaluation. Access the full recording, including a transcript and slides.

Read more about program logic models.

Back to top

What kind of evaluations will CFCA accept when assessing whether my program can be included in the 50% requirement?

Evaluations need to show, to the extent possible, that the implementation of the program has led to the outcomes that the program aimed to achieve. These outcomes need to be specifically outlined in a good quality program logic as described above.

Listen to Elly Robinson describe different types of evaluation:

This recording is an excerpt from a CFCA webinar, Evidence, evidence-based programs and evaluation. Access the full recording, including a transcript and slides.

Evaluations will usually assess one or more of the short- or medium-term outcomes in a program logic; for example, positive parenting practices have increased as a result of engagement with the program. It is not expected that an evaluation will show that a program has achieved all outcomes as outlined in a program logic, particularly long-term outcomes.

The type of evaluation undertaken is less important than the results of the evaluation. At a minimum, the evaluation must show a positive outcome of the program as indicated by pre- AND post-testing of participant outcomes.

Ideally, your evaluation approach would use pre- and post-testing as well as one of the following:

  • a comparison of the outcomes of those who received the program and those who didn’t (i.e., is engaging in the program better than not engaging in the program?); or
  • a comparison of two types of service interventions (i.e., which service type is better for these outcomes?).

Listen to Elly Robinson discuss the assessment of other programs:

This recording is an excerpt from a CFCA webinar, Evidence, evidence-based programs and evaluation. Access the full recording, including a transcript and slides.

If you have specific circumstances that you feel affect your ability to meet the evaluation requirement, please contact us.

Back to top

What help is available for evaluating my program?

The Industry List service is available to Communities for Children service providers. Members of the Industry List have expertise in program planning, implementation and evaluation and can assist with a range of evaluative activities, big or small.

The Industry List price guide is a good starting point if you are considering accessing help. Contact the expert panel team to discuss your requirements.

Information on the types of evaluation and how to undertake an evaluation can be found in the following CFCA resources:  

Back to top

I would describe my activity as an evidence-informed practice, not a program. I work holistically with families. How can I show this is evidence-based?

Structured programs that follow a sequence set out in a manual are only one way of articulating good practice based on research evidence. Of course, many practitioners delivering services outside of structured programs are doing good work. To be able to have confidence that activities are leading to the best outcomes for children and families, however, some minimum standards are necessary for such practices to be considered “evidence-based”. That is why in order for your activity to be considered evidence based, it must meet the five key criteria, which are considered the minimum standards of a quality program for the purposes of the 50% requirement.

Back to top

But I work with communities, not individuals or families …

The criteria for the promising program category has been adapted to include community development style activities. If you are considering submitting a community development program, please contact us to discuss the assessment requirements.

It should be noted that one-off community engagement events, such as Family Fun days, will not fit the criteria.

Back to top

Aren't you prioritising just one type of evidence?

Yes … and no! Different types of evidence show different things, and as a society, we trust some types of research and evidence more than others in particular circumstances. You wouldn’t want to take a new drug or try an experimental medical procedure unless you were confident that it wasn’t going to do you harm and, in fact, was likely to have a positive impact.

Similarly, the investment of community resources and funding of activities to support children, families and their communities is built on the assumption that tangible outcomes can be demonstrated.

Back to top

Families and Children Activity Industry List

How was the Industry List established?

Industry List members are research, practice and evaluation experts from a range of service delivery, research, training, academic and service support backgrounds. Members of the Industry List were selected through an open tender process, via which they established their expertise against several criteria related to program planning, program implementation and program evaluation and/or outcomes measurement.

Read more about the Industry List

Back to top

Who can use the Industry List service?

The Industry List was established to assist service providers funded under the Families and Children Activity. Organisations that work with families and children more broadly can also access help via the Industry List.

Back to top

What type of assistance is available?

Industry List members can help with large evaluation projects and with work that isn’t big or complicated. You may simply need someone for a day or two to help sort out or clarify documentation, provide advice on measurement tools or other smaller tasks.

See the Industry List price guide for estimated service prices.

Back to top

Do I have to use the Industry List for assistance?

You don't have to use the Industry List for assistance - some services have existing partnerships with universities and other providers, or for other reasons don't require support. It is not intended that the Industry List will replace or circumvent research and evaluation activities that are already being undertaken by the sector. However, bear in mind that Industry List members are considered to show a high level of proficiency in program planning, implementation and evaluation.

Back to top

What if I don’t know what I need help with?

CFCA can help. Our role is to work with service providers to help narrow down your requirements. That way, when you get quotes from Industry List members, you can be more confident you are getting the assistance that you need.

Back to top

How do I access the Industry List online database?

You must be registered to access the online database. Registered users will be able to view their profile information and project history, and will receive email alerts as their project progresses. FaC Activity service providers will be automatically registered when they make a Request for Services. Contact us if you wish to use the Industry List.

Back to top

Why is there such a focus on sharing the outcomes of work done with the Industry List?

The purpose of this project is to help to create a more robust, evidence-based and effective service system for children and families. Sharing what you learn helps ensure that all services can be built on the best available evidence of what works (and what doesn’t work) for children, young people and families. The information provided to others is intended to be general findings and outcomes, not specific details about the actual project if this is inappropriate.

Read about recently completed projects in the Expert Panel Focus on ... series.

Back to top

How do I access funding to pay for help from the Industry List?

Funding to source assistance from the Industry List was built into the new DSS Grant Agreements commencing 1 July 2014. FaC services can use that existing funding to directly source targeted assistance from their preferred Industry List member.

Back to top