How to choose an outcomes measurement tool

How to choose an outcomes measurement tool

14 April 2016
How to choose an outcomes measurement tool

This article outlines how to choose an outcomes measurement tool, and provides links to established examples for use with children and families.

Deciding which tools to use to measure outcomes for your program relies on clearly identifying what you will measure (e.g. attributes, behaviours, feelings, skills) and then framing these concepts in a way that is measureable.

Step 1

Clearly state what the program is intending to change. These are the outcomes in your program logic.

Which outcomes will we measure?

For the purposes of the evidence-based program requirement, you would usually measure short or possibly medium term outcomes. These are the ones that are likely to be directly related to your program. Long-term outcomes are great to have, but are often more aspirational and more difficult to measure.

Step 2

Explore what tools are available to measure your outcome/s. You can use an existing tool, or you can develop your own tool. Figure 1 will help you decide which best suits your circumstances.

Figure 1: Instruments decision tree

Figure 1. Hierary of evaluation designs and data collection

Read text description of Figure 1

Ideally, you would use an existing tool. This is because existing tools have often been used and reported on by other evaluators and researchers, so they are more likely to be reliable (the tool measures the same thing at different times, e.g. a program that is run multiple times) and valid (the tool measures what it is supposed to be measuring).

How do I know if a tool already exists?

We have provided a list of such tools below, as well as other sources related to measuring child and family outcomes. Have a look through these first for a suitable tool.

As mentioned in the decision tree above, you may find a tool but feel that it needs to be adapted in some way. You need to be careful that adaptations don’t change the content of the items, as this may change what the tool is measuring. It may be better to either get some help to adapt the tool, or choose another evaluation method/tool.

You can also create your own outcomes measurement tools, although designing survey and other instruments to measure specific constructs can be a complex and time-consuming undertaking. Creating a tool that can reliably measure program outcomes requires a specific set of skills that may not be accessible within your organisation.

What are the tools that are already translated into SCORE?

As part of our work on the Expert Panel, AIFS translated a small number of measurement tools that were being commonly used in the families and children sector into SCORE.

The translated tools are:

  • Strengths and Difficulties Questionnaire (SDQ)
  • Kessler K-10
  • Parental Empowerment and Efficacy Measure (PEEM)
  • Edinburgh Postnatal Depression Scale (EPDS)
  • Child Neglect Index (CNI) 

  • Outcome Rating Scale (ORS)
  • Session Rating Scale (SRS)
  • Bringing up Great Kids
  • Personal Wellbeing Index – Adult (PWI-A)

Download the SCORE translation matrix [PDF, 740 KB].

Need help?

Help with choosing the right tool is available through the Industry List, including assistance with small pieces of work. Contact the Expert Panel team for more information by emailing fac-expert-panel@aifs.gov.au.

Sources of established tools for measuring child and family outcomes

Evaluating your intervention, from Children of Parents with a Mental Illness (COPMI)

The following pages list measures that are of particular relevance to families and children:

Instrument collections, from the American Evaluation Association

This page hosts a comprehensive list of links to sites and resources for evaluation tools and instruments recommended by members of the American Evaluation Association. Most are free to access, although some instruments may require registration or purchase. A relevant example from the site is the FRIENDS Compendium of Annotated Measurement Tools, which allows you to search for tools by child protection factor.

Personal Wellbeing Index, from the Australian Centre on Quality of Life

An Australian brief measure of personal wellbeing can be found at the Australian Centre on Quality of Life website. Parallel forms have been developed for use with adults, school-aged and pre-school children, and for people with intellectual or cognitive disability. The site includes information on the development and testing of the questionnaire, and instructions and guidance on its administration.

Protocols for the delivery of social and emotional wellbeing and mental health services in Indigenous communities: Guidelines for health workers, clinicians, consumers and carers

This publication contains items for assessing social and emotional wellbeing of Indigenous people, including practice tips and strategies for carrying out the assessment and helping the client elaborate on their responses. A sample of the document is available, however you will need to contact UQ to obtain the full version.

  

The feature image is by Luis Marina, CC BY 2.0.

Comments

Some good resource links here. Thank-you.
Gareth

Add a comment

Authors

Kathryn is a Senior Research Officer within the Practice Evidence and Engagement area at the Australian Institute of Family Studies.

Elly Robinson

Elly was formerly Executive Manager, Practice Evidence and Engagement at the Australian Institute of Family Studies.

Measuring outcomes

Need some help?

CFCA offers a free research and information helpdesk for child, family and community welfare practitioners, service providers, researchers and policy makers through the CFCA News.

Subscribe

CFCA News

Sign up to our email alert service for the latest news and updates

Subscribe