Kathryn is a Senior Research Officer within the Practice Evidence and Engagement area at the Australian Institute of Family Studies.

This article outlines some key considerations for using qualitative methods in program evaluation.
Qualitative research seeks to answer questions that stress how social experience is created and given meaning, beyond the scope of numbers and statistics (Rogers & Goodrick, 2010). Qualitative research can use a variety of methods such as interviews and observations – the method you choose should be based on what you are researching and the resources you have. However, being rigorous and transparent is the key to good qualitative research. That is, to be of high quality your research should be supported by a series of logical and justifiable steps.
To begin with, have a strong rationale for why you have chosen a qualitative approach to answer your research question/s, and identify suitable data collection methods (interviews, focus groups, observations, open-ended surveys etc.) and any key perspectives that should be captured. Consider what techniques or concepts will guide the data analysis and interpretation stage, and what quality checks you can put in place to justify your interpretations.
Overall, being clear about the methodological process will help to strengthen the credibility of your findings.
If you’re considering adopting a qualitative approach to evaluate the impact of a program1, be aware that using strictly qualitative methods may not generate the answers you need, and is generally recommended in only a handful of cases. For instance, if you have a small population size, there are no existing outcomes measures that can be used with your target group, or you are evaluating a pilot version of the program.
An alternative option is to incorporate qualitative methods into your evaluation design alongside quantitative methods, as part of a mixed-methods design. Evaluations that adopt a mixed-methods approach are well placed to establish any causal relationships between the program content and outcomes, and to tell us how and why these changes occurred. Mixed-methods can also be used to:
Regardless of the techniques you choose to gather information about your program, conducting a quality qualitative evaluation relies on having time and expertise. Collecting participant insights can be time consuming, especially if only one evaluator is involved in the project. There is also the potential to be left with a large volume of data that needs to be interpreted, synthesised and communicated. Having an expert in qualitative methods conduct or assist with the evaluation will help to ensure that these tasks are completed in a systematic and rigorous way. The evaluation write-up is a good opportunity to demonstrate that your evaluation is of high quality – but again, this requires a specific set of skills.
If you don’t have the desired skill set to undertake your research and can’t access help from within your organisation, a member from the Industry List can help with a range of tasks, from providing advice on your research design through to conducting a program evaluation. Contact the expert panel team at fac-expert-panel@aifs.gov.au for more information.
Rogers, P. & Goodrick, D. (2010). Qualitative data analysis. In Wholey, J., Hatry, H., & Newcomer, K., (Eds.), Handbook of practical program evaluation (3rd ed., pp. 429–53). San Francisco: Jossey-Bass.
1. For a discussion of the differences between impact and process evaluation, see CFCA Practitioner Resource Evidence-based practice and service-based evaluation.
The feature image is by Stef Lewandowski, CC BY-NC 2.0.
Kathryn is a Senior Research Officer within the Practice Evidence and Engagement area at the Australian Institute of Family Studies.
Kelly is a Senior Research Fellow and Manager of Project Evaluation and Qualitative Research at the Australian Institute of Family Studies.
An outline of the steps involved in planning an evaluation, and an overview of the key concepts relating to the broad stages of evaluation
A guide to the why, who, what, when, where and how of evaluation, particularly the vital tasks needed to ensure an evaluation goes smoothly
Information on how to disseminate evaluation findings to the broader service sector, including tips for writing and publishing an evaluation report
Provides practical information on the structures, practices and actions that support a change toward a strong culture of evaluation and research
Information about the types of evaluation methods that inform evidence-based practice, and tips for services on establishing a culture of evaluatio
CFCA offers a free research and information helpdesk for child, family and community welfare practitioners, service providers, researchers and policy makers through the CFCA News.
Sign up to our email alert service for the latest news and updates
The Australian Institute of Family Studies acknowledges the traditional country throughout Australia on which we gather, live, work and stand.
We acknowledge all traditional custodians, their Elders past, present and emerging, and we pay our respects to their continuing connection to their culture, community, land, sea and rivers.
Comments