Program planning and evaluation guide

Program planning and evaluation guide

Introduction

The aim of this guide is to help community sector organisations plan, deliver and evaluate evidence-informed programs. Its practical advice and resources are divided into five steps:EXPP process flow

Program planning and evaluation guide

Step 1: Identify and prioritise issues for action

Understanding the community in which you are working is the first step in evidence-based programming.

Community members – service users or people with lived experience – should be engaged in the design or delivery of services or programs, and consulted about their views on issues and services.

This page explains why community engagement is important, and will help you to identify useful engagement strategies.

1.1 Identifying and prioritising issues

Exploring the issues and assets within a community ensures that policy, programs and services are relevant and effective.

Resources

Needs Assessment [Multi-page Families and Children Expert Panel practice resource]

Responding to the coronavirus pandemic: Conducting a needs assessment in a time of rapid change [Short article]

1.2 Community engagement and consultation

When designing community services or programs, it's vital to consult the people for whom they're intended.

Step 2: Understand the issue

Before addressing a social problem it is important to understand it. This includes: thinking through the causes and consequences of the problem; exploring the research evidence on that type of problem; and understanding who is most affected by the problem.

A public health approach may be a useful framework to guide this process.

This page contains resources that will help you to frame the way you understand and approach social problems. 

 

2.1 The public health approach

A public health approach aims to prevent a social problem before it occurs. This approach identifies a problem and its causes, then identifies and evaluates solutions.

Resources in this section describe the public health approach and how it can be used to address specific social problems.

Resources

A public health model for child welfare services [CFCA Resource Sheet]

 

2.2 Understanding and exploring social issues

Social issues are often complex and can have a range of causes and consequences. Understanding the causes of a social issue and the way it is experienced by different groups and individuals will enable you to identify and implement a more effective program or service.

Resources

Child abuse and neglect prevention [List of CFCA publications by topic]

Communities and neighbourhoods [List of CFCA publications by topic]

Culturally and linguistically diverse families [List of CFCA publications by topic]

Early intervention and prevention programs [List of CFCA publications by topic]

Family dispute resolution [List of CFCA publications by topic]

Parents [List of CFCA publications by topic]

Social inclusion [List of CFCA publications by topic]

Protecting Australia's Children: Research and Evaluation Register [Database of completed projects]

 

2.3 Risk and protective factors

Every social problem has risk factors and protective factors. These are things that make it more or less likely that an individual may be affected by an issue. They can also have an influence on the way an individual experiences the consequences of an issue.

2.4 Meeting the needs of your target group

Most projects, programs or services are designed to meet the needs of a group of people who share certain characteristics.

This section includes resources that describe how to ensure your program meets the needs of your target group, as well as resources on how to work effectively with specific groups such as Aboriginal and Torres Strait Islander people.

Step 3: Plan your program or service

Effective programs and services are those that have been planned and designed using evidence and a clear program theory.

There are many different programs and strategies to address social issues, so it is important to consider what will be most effective in the context in which you are working.

This page contains resources that will help you to understand different approaches to programs, services and projects, and how they can be applied on the ground. 

3.1 Selecting a suitable approach

There are usually many different options for programs, services or projects to address a social problem. This section includes resources that can help you to identify and understand different approaches, different types of programs and activities, and the evidence behind them.

Resources

Applying community capacity-building approaches to child welfare [Multi-page CFCA Paper]

Defining community development [CFCA Resource Sheet]

Guidance on selecting an evidence-based program [Designed for Communities for Children Facilitating Partners, but contains useful general advice]

Implementing an evidence-informed practice framework [Webinar recording, transcript and slides]

3.2 Program logic: Step by step

Central to successful program design, implementation and evaluation is being clear about how a program should work and what outcomes it is intended to produce. This is often done with a program logic.

Resources

PDF iconDownloadable program logic checklist [checklist for reviewing a completed logic model] 

How to develop a program logic [Step-by-step practice resource]

Instructional video (Opens in YouTube)

Still from the guided tour through program logic video

 

 

Step 4. Implement and adapt your program or service

To achieve their intended outcomes, programs or services must be implemented in line with a plan. This is true whether you have designed your own program or you have selected an evidence-based program.

However sometimes programs need to be adapted to suit the context or needs of the participants. Knowing how to adapt a program and having a clear plan for its implementation will help ensure that it meets the needs of participants while still achieving its intended outcomes.

Resources on this page advise on how to implement and adapt evidence-based programs.

4.1 Delivering programs

Resources

Implementation and adaptation of evidence-based programs [Webinar recording, transcript and slides]

Implementation in action: A guide to implementing evidence-informed programs and practices [Comprehensive how-to guide for practitioners, with tools and checklists]

Implementation of two evidence-based programs [Case study]

Selecting an evidence-based program [Designed for Communities for Children Facilitating Partners, but contains useful general advice]

4.2 Adapting programs

Evidence-based programs sometimes require adaptation to meet the needs of a specific target group or context, however this can have an effect on the outcomes.

Step 5: Evaluate your program or service

In social services, evaluation is usually undertaken to find out whether a program or service was delivered the way it was planned, and to examine its effects on program participants.

This page will give you the basics about evaluating programs and services, from developing evaluation questions to interpreting data. You will also find guidance on selecting an evaluation approach that is suited to your program participants, objectives and resource requirements.

Resources on this page explain what evaluation is, describe different types of evaluation, and give examples of how agencies can build a culture of research and evaluation.

5.1 Introduction to evaluation

Evaluation can be complex and confusing, and it can be hard to know where to start. Resources in this section explain what evaluation is and why it’s important.

Back to top

5.2 Types of evaluation

There are many different types of evaluation. Social services will most likely opt for a process or impact evaluation, or in some cases a combination of the two. This resource explains the difference between impact and process evaluation.

Resources

Evaluation and innovation in family support services [CFCA Practitioner Resource]

Instructional video (Opens in YouTube)

Still from the guided tour through outcomes measurement video

 

 

Back to top

5.3 Ethical considerations for evaluation projects

Some evaluation projects will need to undergo an ethics review process to ensure that evaluation is conducted with integrity and with minimal risk to participants. This section provides an overview of ethical standards in evaluation and explains the ethics approval process.

Back to top

5.4 Selecting program outcomes to measure

Program outcomes or objectives are what you anticipate will happen as a direct result of the program or service you are delivering. Most program evaluations will target short and medium-term outcomes.

This section provides guidance on choosing specific, realistic and measurable outcomes that can be used to demonstrate program effects.

Resources

The basics of evaluation [CFCA Practitioner Resource]

Demonstrating community-wide outcomes [Practice guidelines]

Detailed guidance for designing an outcomes evaluation [CFCA Practitioner Resource]

Measuring outcomes in programs for Aboriginals and Torres Strait Islanders [Webinar recording, transcript and slides]

Instructional video (Opens in YouTube)

Still from the guided tour through outcomes measurement video

 

 

Back to top

5.5 Tailoring your evaluation to program participants

It is critical to the success of your evaluation that you tailor it to the needs of the people attending your program or service. Factors such as literacy levels, age and cultural background will influence how you engage participants in the evaluation process, and the methods you use to collect information.

This section covers working with participant groups in evaluation and proposes solutions for overcoming challenges.

Back to top

5.6 Designing an evaluation

Evaluation design is rarely a linear process; there are many overlapping parts. You will be deciding on the:

  • evaluation questions
  • level of evidence you need to collect
  • evaluation approach
  • type of data (qualitative or quantitative) that will give the best evidence

The resources in this section will help you make those decisions.

Back to top

5.7 Finding the right tool to measure program outcomes

Measuring program outcomes properly means using the right tool (“instrument”). You might choose to use a standardised survey, observation checklist or an interview guide – or a combination of all three.

This section will help you to choose an outcomes measurement tool, or tools, to suit your needs.

Resources

Available outcomes measurement tools [Summary matrix and explanatory notes]

Detailed guidance for designing an outcomes evaluation [CFCA Practitioner Resource]

Developing an outcomes measurement framework: The Mallee [Webinar recording, transcript and slides]

How to choose an outcomes measurement tool [Short article]

Instructional video (Opens in YouTube)

Still from the guided tour through outcomes measurement video

 

 

Back to top

5.8 Analysing evaluation data

Once you have collected the data you need you can start analysing. Data analysis primarily involves organising data so that you can report against your intended program outcomes and make judgements about what the data means. This resource provides some general advice about analysing qualitative and quantitative data.

Resources

The basics of evaluation [CFCA Practitioner Resource]

Back to top

5.9 What to do after you complete the evaluation

Evaluation has the potential to improve our knowledge about how and why a program or service works or doesn’t work and it can be a powerful tool for continual improvement. The broader sector will also benefit from communicating or “disseminating” your findings. 

This section explores various means of research dissemination and provides examples of practitioners who consistently use evaluative findings and evidence to strengthen programs and practice.

Back to top