Program planning and evaluation guide

Program planning and evaluation guide

Program planning and evaluation guide – March 2018
""

Overview

The aim of this guide is to help community sector organisations plan, deliver and evaluate evidence-informed programs. Its practical advice and resources are divided into five steps:

Program planning and evaluation guide steps. 1 Identify and prioritise issues; 2 Analyse issues; 3 Plan program; 4 Implement program; 5 Evaluate program

step2step3section3.2step4step5

Read the publication

Step 1: Identify and prioritise issues for action

Graphic - Step 1: Identify and prioritise issues for action

Understanding the community in which you are working is the first step in evidence-based programming.

Community members – service users or people with lived experience – should be engaged in the design or delivery of services or programs, and consulted about their views on issues and services.

1.1 Identifying and prioritising issues

Exploring the issues and assets within a community ensures that policy, programs and services are relevant and effective.

Resources

1.2 Community engagement and consultation

When designing community services or programs, it's vital to consult the people for whom they're intended.

Resources

Community engagement: A key strategy for improving outcomes for Australian families
This CFCA paper seeks to clarify what community engagement involves, and the role it can play in improving outcomes for children and families.

Step 2: Understand the issue

Step 2: Understand the issue

Before addressing a social problem it is important to understand it. This includes: thinking through the causes and consequences of the problem; exploring the research evidence on that type of problem; and understanding who is most affected by the problem.

A public health approach may be a useful framework to guide this process.

2.1 The public health approach

A public health approach aims to prevent a social problem before it occurs. This approach identifies a problem and its causes, then identifies and evaluates solutions.

Resources in this section describe the public health approach and how it can be used to address specific social problems.

Resources

Defining the public health model for the child welfare services context
This CFCA resource sheet gives an overview of the public health model and how it applies to the Australian child welfare and child protection system.

2.2 Understanding and exploring social issues

Social issues are often complex and can have a range of causes and consequences. Understanding the causes of a social issue and the way it is experienced by different groups and individuals will enable you to identify and implement a more effective program or service.

Resources

2.3 Meeting the needs of your target group

Most projects, programs or services are designed to meet the needs of a group of people who share certain characteristics.

This section includes resources that describe how to ensure your program meets the needs of your target group, as well as resources on how to work effectively with specific groups such as Aboriginal and Torres Strait Islander people.

Resources

Step 3: Plan your program or service

Step 3: Plan your program or service

Effective programs and services are those that have been planned and designed using evidence and a clear program theory.

There are many different programs and strategies to address social issues, so it is important to consider what will be most effective in the context in which you are working.

3.1 Selecting a suitable approach

There are usually many different options for programs, services or projects to address a social problem. This section includes resources that can help you to identify and understand different approaches, different types of programs and activities, and the evidence behind them.

Resources

3.2 Program logic: Step by step

Central to successful program design, implementation and evaluation is being clear about how a program should work and what outcomes it is intended to produce. This is often done with a program logic.

Resources

  • A guided tour through program logic

 

Step 4. Implement and adapt your program or service

Step 4. Implement and adapt your program or service

To achieve their intended outcomes, programs or services must be implemented in line with a plan. This is true whether you have designed your own program or you have selected an evidence-based program.

However sometimes programs need to be adapted to suit the context or needs of the participants. Knowing how to adapt a program and having a clear plan for its implementation will help ensure that it meets the needs of participants while still achieving its intended outcomes.

4.1 Delivering and adapting programs

Resources

Step 5: Evaluate your program or service

Step 5: Evaluate your program or service

In social services, evaluation is usually undertaken to find out whether a program or service was delivered the way it was planned, and to examine its effects on program participants.

This page will give you the basics about evaluating programs and services, from developing evaluation questions to interpreting data. You will also find guidance on selecting an evaluation approach that is suited to your program participants, objectives and resource requirements.

5.1 Introduction to evaluation

Evaluation can be complex and confusing, and it can be hard to know where to start. Resources in this section explain what evaluation is, why it’s important and what to think about before you begin.

Resources

5.2 Ethical considerations for evaluation projects

It is essential that evaluations are conducted ethically and with minimal risk to participants. This section provides an overview of ethical standards in evaluation and explains the ethics approval process.

Resources

Examples of ethical evaluation guidelines

Australasian Evaluation Society [PDF]

National Health and Medical Research Council

5.3 Planning an evaluation

You can start planning an evaluation by deciding on the:

  • evaluation questions
  • level of evidence you need to collect
  • type of data (qualitative or quantitative) that will give the best evidence

The resources in this section will help you make those decisions.

Resources

5.4 Approaches to evaluation

Evaluation approaches can help to guide the design and implementation of your evaluation and are particularly useful if there are specific ways you want to work with participants (e.g., in partnership) or develop and answer evaluation questions (e.g., through experimentation). The following resources provide information on selected approaches to evaluation.

Resources

  • Empowerment evaluation: Benefitting participants 
    This CFCA practitioner resource is a snapshot of empowerment evaluation, a model that aims to create a sense of ownership to ensure program outcomes are beneficial for participants.
  • Participatory action research: Partnering with communities 
    This CFCA practitioner resource is a snapshot of participatory action research, an approach that views the participant as an equal partner with the researcher.
  • "Realist evaluation" in action
    In this case study, Margaret Cargo and Lisa Warner discuss the "realist" approach used to evaluate the Aboriginal Parental Engagement Program (APEP).
  • Developmental evaluation
    This CFCA practitioner resource gives an overview of developmental evaluation—what it is, why it was established, and what it can be used for.

5.5 Selecting program outcomes to measure

Program outcomes are what you anticipate will happen as a direct result of the program or service you are delivering. Most program evaluations will target short and medium-term outcomes.

This section provides guidance on choosing specific, realistic and measurable outcomes that can be used to demonstrate program effects.

Resources

  • A guided tour through outcomes measurement

5.6 Finding the right tool to measure program outcomes

Measuring program outcomes properly means using the right tool (“instrument”). You might choose to use a standardised survey, observation checklist or an interview guide – or a combination of all three.

This section will help you to choose an outcomes measurement tool, or tools, to suit your needs.

Resources

  • A guided tour through outcomes measurement

5.7 Tailoring your evaluation to program participants

It is critical to the success of your evaluation that you tailor it to the needs of the people attending your program or service. Factors such as literacy levels, age and cultural background will influence how you engage participants in the evaluation process, and the methods you use to collect information.

This section covers working with participant groups in evaluation and proposes solutions for overcoming challenges.

Resources

5.8 Using evaluation findings

Evaluation has the potential to improve our knowledge about how and why a program or service works or doesn’t work and it can be a powerful tool for continual improvement. The broader sector will also benefit from sharing your findings. 

This section explores various means of research dissemination and provides guidance on ways to use evaluative findings for impact.

Resources

  • Dissemination of evaluation findings 
    This CFCA practitioner resource provides information on how to disseminate evaluation findings to the broader service sector, including tips for writing and publishing an evaluation report.
  • Making the most out of evaluation
    This Expert Panel Project resource presents nine principles to apply to your evaluation to maximise its use.

Publication details

Program planning and evaluation guide
Published by the Australian Institute of Family Studies, March 2018

Publication meta