Program planning and evaluation guide
Read the publication
Understanding the community in which you are working is the first step in evidence-based programming.
Community members – service users or people with lived experience – should be engaged in the design or delivery of services or programs, and consulted about their views on issues and services.
1.1 Identifying and prioritising issues
Exploring the issues and assets within a community ensures that policy, programs and services are relevant and effective.
Resources
- Needs Assessment
This Families and Children Expert Panel practice resource is for practitioners and policy makers who want to learn more about the needs assessment process or how to conduct a needs assessment. - Data sources in needs assessments
This resource is a guide for those who are planning a needs assessment and want to identify and assess possible data sources. - Responding to the coronavirus pandemic: Conducting a needs assessment in a time of rapid change
This short article provides guidance to services on conducting a needs assessment during the coronavirus pandemic.
1.2 Community engagement and consultation
When designing community services or programs, it's vital to consult the people for whom they're intended.
Resources
Community engagement: A key strategy for improving outcomes for Australian families
This CFCA paper seeks to clarify what community engagement involves, and the role it can play in improving outcomes for children and families.
Before addressing a social problem it is important to understand it. This includes: thinking through the causes and consequences of the problem; exploring the research evidence on that type of problem; and understanding who is most affected by the problem.
A public health approach may be a useful framework to guide this process.
2.1 The public health approach
A public health approach aims to prevent a social problem before it occurs. This approach identifies a problem and its causes, then identifies and evaluates solutions.
Resources in this section describe the public health approach and how it can be used to address specific social problems.
Resources
Defining the public health model for the child welfare services context
This CFCA resource sheet gives an overview of the public health model and how it applies to the Australian child welfare and child protection system.
2.2 Understanding and exploring social issues
Social issues are often complex and can have a range of causes and consequences. Understanding the causes of a social issue and the way it is experienced by different groups and individuals will enable you to identify and implement a more effective program or service.
Resources
- How to review the evidence
This short resource provides guidance and links to additional information to step you through a basic literature review. - Risk and protective factors for child abuse and neglect
This CFCA resource sheet gives an overview of the risk and protective factors for child abuse and neglect in families. - Sources of information on selected topics:
2.3 Meeting the needs of your target group
Most projects, programs or services are designed to meet the needs of a group of people who share certain characteristics.
This section includes resources that describe how to ensure your program meets the needs of your target group, as well as resources on how to work effectively with specific groups such as Aboriginal and Torres Strait Islander people.
Resources
- Aligning target group, activities and outcomes
This CFCA webinar explained how funders, managers and practitioners can ensure their program's target group, activities and outcomes are in alignment. - Community involvement in Indigenous program evaluation
This case study provides an overview of Ninti One’s approach to working with local communities to deliver the Stronger Communities for Children program. - Effective Indigenous community-managed programs
This CFCA paper gives a review of the literature on Indigenous community-managed programs and organisations, with a focus on what works in these initiatives. - How to review the evidence
This Expert Panel Project short resource provides guidance and links to additional information to step you through a basic literature review. - Working with Indigenous children, families and communities
These practice guidelines summarise lessons learned from services that are working effectively to reduce Indigenous disadvantage in Australia.
Effective programs and services are those that have been planned and designed using evidence and a clear program theory.
There are many different programs and strategies to address social issues, so it is important to consider what will be most effective in the context in which you are working.
3.1 Selecting a suitable approach
There are usually many different options for programs, services or projects to address a social problem. This section includes resources that can help you to identify and understand different approaches, different types of programs and activities, and the evidence behind them.
Resources
- What is theory of change?
This resource explains what theory of change is, why to develop one and how it can be used for program planning and evaluation. - What is community development?
An overview of community development for service providers and practitioners - Applying community capacity-building approaches to child welfare
This CFCA paper explores how insights from the field of community capacity-building can improve child welfare practice and policy in Australia. - Collective impact: Evidence and implications for practice
This CFCA paper explores the development of the collective impact framework and its ability to create population-level change on complex social issues. - Evidence Informed Practice in Intensive Family Support Programs: Are we there yet?
This webinar described how The Benevolent Society applies the Resilience Practice Framework to their intensive family support programs.
3.2 Program logic: Step by step
Central to successful program design, implementation and evaluation is being clear about how a program should work and what outcomes it is intended to produce. This is often done with a program logic.
Resources
- A guided tour through program logic
A guided tour through: Program logic models” is an instructional video, produced by AIFS Expert Panel Project. The video steps through how to develop a program logic model. It outlines common terminology and uses examples to explore practical considerations for child and family practitioners.
- How to develop a program logic
This step-by-step practice resource has been designed for anyone who is interested in developing a program logic. - Downloadable program logic checklist - [PDF, 195 KB]
This checklist is designed to be used once you have drafted a logic model, and your logic model is ready to review.
To achieve their intended outcomes, programs or services must be implemented in line with a plan. This is true whether you have designed your own program or you have selected an evidence-based program.
However sometimes programs need to be adapted to suit the context or needs of the participants. Knowing how to adapt a program and having a clear plan for its implementation will help ensure that it meets the needs of participants while still achieving its intended outcomes.
4.1 Delivering and adapting programs
Resources
- Key Concepts In Implementation
This video is part one of two instructional videos based on the publication Implementation in Action: A Guide to Implementing Evidence-Informed Programs and Practices. ‘Key concepts in implementation’ explains the theory that underpins implementation science.
- Implementation In Action
This video is part two of two instructional videos based on the publication Implementation in Action: A Guide to Implementing Evidence-Informed Programs and Practices. ‘Implementation in Action’ provides practical tools and checklists for undertaking program implementation using practical examples familiar to practitioners and program managers who work to deliver programs for families and children.
- Implementation in action: A guide to implementing evidence-informed programs and practices
The purpose of this guide is to help in the implementation of evidence-informed programs and practices in the child and family service sector. - Implementation and adaptation of evidence-based programs
This webinar addressed some of the challenges of implementing evidence-based programs within diverse community settings and client groups. - Responding to the coronavirus pandemic: Documentation tips for monitoring and evaluation
This short article is for anyone who is delivering, or planning to deliver, a program or service modified in response to the coronavirus pandemic. - Selecting an evidence-based program
This comprehensive how-to guide for practitioners is designed to assist Communities for Children Facilitating Partners with the process of selecting a program from the evidence-based program profiles.
In social services, evaluation is usually undertaken to find out whether a program or service was delivered the way it was planned, and to examine its effects on program participants.
This page will give you the basics about evaluating programs and services, from developing evaluation questions to interpreting data. You will also find guidance on selecting an evaluation approach that is suited to your program participants, objectives and resource requirements.
- 5.1 Introduction to evaluation
- 5.2 Ethical considerations for evaluation projects
- 5.3 Planning an evaluation
- 5.4 Approaches to evaluation
- 5.5 Selecting program outcomes to measure
- 5.6 Finding the right tool to measure program outcomes
- 5.7 Tailoring your evaluation to program participants
- 5.8 Using evaluation findings
5.1 Introduction to evaluation
Evaluation can be complex and confusing, and it can be hard to know where to start. Resources in this section explain what evaluation is, why it’s important and what to think about before you begin.
Resources
- What is evaluation?
This short resource defines program evaluation in the context of child and family support services. It was written for those who are new to evaluation and unfamiliar with evaluation terms. - The nuts and bolts of program evaluation
This webinar gives a guaranteed, easy-to-understand 'nuts and bolts' overview of evaluation. - Developing a culture of evaluation and research
This CFCA paper provides practical information on the structures, practices and actions that support a change toward a strong culture of evaluation and research. - Building a positive evaluation culture: Key considerations for managers in the families and children services sector
This resource has tips and key considerations for managers wanting to help build an organisational culture where evidence and evaluation are valued. - Planning an evaluation
This resource provides a step-by-step guide and templates for evaluation for beginners who need help with evaluation planning. - Tips for commissioning an external evaluation
This short article is for anyone who is planning to commission an evaluation from an external evaluator. - Working with evaluators: What do you need to know?
This webinar discussed ways social service providers and evaluators can build successful evaluation partnerships for the benefit of children, families. - Making the most out of evaluation
This Expert Panel Project resource presents nine principles to apply to your evaluation to maximise its use.
5.2 Ethical considerations for evaluation projects
It is essential that evaluations are conducted ethically and with minimal risk to participants. This section provides an overview of ethical standards in evaluation and explains the ethics approval process.
Resources
- Ethics in evaluation
This document outlines the ethical issues that should be considered in any evaluation and provides guidance on when an evaluation is likely to need a formal review by a Human Research Ethics Committee (HREC). - Demystifying ethical review
This CFCA resource sheet provides an overview of the ethical review process and how it applies to service providers evaluating their own programs. - Ethical considerations in research and evaluation with children and young people
This short article unpacks the main ethical considerations that should be incorporated into research or evaluation projects involving children and young people.
Examples of ethical evaluation guidelines
5.3 Planning an evaluation
You can start planning an evaluation by deciding on the:
- evaluation questions
- level of evidence you need to collect
- type of data (qualitative or quantitative) that will give the best evidence
The resources in this section will help you make those decisions.
Resources
- Evaluation design
This resource gives a quick overview of some of the main evaluation designs used for outcomes evaluations or impact evaluations. - Planning an evaluation
This resource provides a step-by-step guide and templates for evaluation for beginners who need help with evaluation planning. - Identifying evaluation questions
This short article is for people new to evaluation who are planning to conduct or commission an evaluation. - Demonstrating community-wide outcomes: Exploring the issues for child and family services
Investigates issues relating to evaluating whole-of-community initiatives - Collecting data from parents and children for the purpose of evaluation
This practice guideline is an outline of the challenges child and family services face when collecting data directly from parents and children for evaluation. - Using qualitative methods in program evaluation
This short article outlines some key considerations for using qualitative methods in program evaluation. - Responding to the coronavirus pandemic: Assessing rapid service changes
This short article discusses how you can assess the changes you have made to programs and services in response to the coronavirus pandemic. - Evaluation and Value for Money
This resource sheet provides a snapshot of evaluation and Value for Money as a process to assist organisations to plan, develop and improve their programs.
5.4 Approaches to evaluation
Evaluation approaches can help to guide the design and implementation of your evaluation and are particularly useful if there are specific ways you want to work with participants (e.g., in partnership) or develop and answer evaluation questions (e.g., through experimentation). The following resources provide information on selected approaches to evaluation.
Resources
- Empowerment evaluation: Benefitting participants
This CFCA practitioner resource is a snapshot of empowerment evaluation, a model that aims to create a sense of ownership to ensure program outcomes are beneficial for participants. - Participatory action research: Partnering with communities
This CFCA practitioner resource is a snapshot of participatory action research, an approach that views the participant as an equal partner with the researcher. - "Realist evaluation" in action
In this case study, Margaret Cargo and Lisa Warner discuss the "realist" approach used to evaluate the Aboriginal Parental Engagement Program (APEP). - Developmental evaluation
This CFCA practitioner resource gives an overview of developmental evaluation—what it is, why it was established, and what it can be used for.
5.5 Selecting program outcomes to measure
Program outcomes are what you anticipate will happen as a direct result of the program or service you are delivering. Most program evaluations will target short and medium-term outcomes.
This section provides guidance on choosing specific, realistic and measurable outcomes that can be used to demonstrate program effects.
Resources
- A guided tour through outcomes measurement
A guided tour through: Measuring outcomes is an instructional video, produced by AIFS Expert Panel Project, that steps the audience through how to measure outcomes. It outlines common terminology and uses examples to explore practical considerations for child and family practitioners when measuring outcomes. It includes a focus on standardised outcome measures. The video is designed to be accessible to practitioners with limited evaluation experience.
- Demonstrating community-wide outcomes: Exploring the issues for child and family services
This practice guideline investigates issues relating to evaluating whole-of-community initiatives. - Measuring outcomes in programs for Aboriginal and/or Torres Strait Islander families and communities
This webinar discussed ways to measure the outcomes of programs for Aboriginal and/or Torres Strait Islander families and communities.
5.6 Finding the right tool to measure program outcomes
Measuring program outcomes properly means using the right tool (“instrument”). You might choose to use a standardised survey, observation checklist or an interview guide – or a combination of all three.
This section will help you to choose an outcomes measurement tool, or tools, to suit your needs.
Resources
- A guided tour through outcomes measurement
A guided tour through: Measuring outcomes is an instructional video, produced by AIFS Expert Panel Project, that steps the audience through how to measure outcomes. It outlines common terminology and uses examples to explore practical considerations for child and family practitioners when measuring outcomes. It includes a focus on standardised outcome measures. The video is designed to be accessible to practitioners with limited evaluation experience.
- How to choose an outcomes measurement tool
This short article outlines how to choose an outcomes measurement tool, and provides links to established examples for use with children and families. - Communities for Children outcomes measurement matrix
A tool for programs delivered under the Communities for Children initiative - Building a better outcomes framework for families: A story from the Mallee
This webinar told the story of two organisations that worked in partnership to establish good practice in evaluating service delivery.
5.7 Tailoring your evaluation to program participants
It is critical to the success of your evaluation that you tailor it to the needs of the people attending your program or service. Factors such as literacy levels, age and cultural background will influence how you engage participants in the evaluation process, and the methods you use to collect information.
This section covers working with participant groups in evaluation and proposes solutions for overcoming challenges.
Resources
- Collecting data from parents and children for the purpose of evaluation
This practice guideline is an outline of the challenges child and family services face when collecting data directly from parents and children for evaluation. - Collaboration and co-design when evaluating intergenerational trauma projects
This short article outlines how co-design and collaboration is fundamental to the work of the Healing Foundation. - Measuring outcomes in programs for Aboriginal and/or Torres Strait Islander families and communities
This webinar discussed ways to measure the outcomes of programs for Aboriginal and/or Torres Strait Islander families and communities.
5.8 Using evaluation findings
Evaluation has the potential to improve our knowledge about how and why a program or service works or doesn’t work and it can be a powerful tool for continual improvement. The broader sector will also benefit from sharing your findings.
This section explores various means of research dissemination and provides guidance on ways to use evaluative findings for impact.
Resources
- Dissemination of evaluation findings
This CFCA practitioner resource provides information on how to disseminate evaluation findings to the broader service sector, including tips for writing and publishing an evaluation report. - Making the most out of evaluation
This Expert Panel Project resource presents nine principles to apply to your evaluation to maximise its use.
Share
