Therapeutic residential care services in Australia

A description of current service characteristics
CFCA Paper No. 52 – August 2019

Appendix A: Methodology

An online survey methodology was used to collect information on service provision in a sample of services that self-nominated as providing therapeutic residential care. The purpose of this survey was to document the service and staffing characteristics, care needs, therapeutic frameworks, models and practices, and other key issues that characterise current Australian therapeutic residential care. The survey was commissioned by the CFCA information exchange at the Australian Institute of Family Studies.

Research ethics approval for this survey was received from the University of South Australia's Human Research and Ethics Committee and reviewed and noted by the Australian Institute of Family Studies' Ethics Committee.

The survey was promoted by the CFCA information exchange at the Australian Institute of Family Studies via their newsletter and website, and by the National Therapeutic Residential Care Alliance of Australia via their membership list and distribution pathways. The Australian Institute of Family Studies also promoted the survey directly to services that were identified from its existing network of services and other organisations working in the sector, such as peak bodies; and information about the survey was distributed by the author opportunistically at key industry events during the study period.

The survey questions were based on the index of reporting standards proposed by Lee and Barth (2011) with amendments, based on the outcome of a consultation with the National Therapeutic Residential Care Alliance reference group. The final survey questions were tested for comprehension and usability by the author and the Australian Institute of Family Studies. Data collection was between 5 July and 5 October 2018. Completed data were uploaded and securely transferred by a password-protected link to the primary author for analysis.

Each survey contained 35 questions (see Appendix B). This included a mix of questions targeting:

  • demographic and other descriptive information (8 questions)
  • characteristics and needs of young people being cared for (2 questions)
  • service characteristics such as location and size (4 questions)
  • staffing configuration, staff retention and training (11 questions)
  • therapeutic approaches, models and activities (6 questions)
  • open-ended prompting questions aimed at obtaining feedback about residential care and suggestions for further development (4 questions).

All survey data were reviewed with reference to predetermined inclusion criteria. Surveys were selected for inclusion if: a) all relevant demographic data and services information was completed; and b) survey responses were least 75% completed overall.

All responses were combined for each question and reported as a group. Data from individual services and/or jurisdictions were not reported separately.

The survey questions required a mix of quantitative and qualitative responses; and the scoring and reporting of responses obtained from participants varied accordingly. Box A.1 provides more information about how survey responses were scored; depending on the type of responses required. Throughout the survey, participants were asked to:

  1. provide demographic information
  2. provide numerical responses
  3. rank options from a list of response choices
  4. provide open-ended responses.

Box A.1: The scoring of survey responses

This survey used a mix of quantitative and qualitative survey questions. Scoring and reporting of survey responses depended on the type of responses obtained from participants. Throughout this report, survey responses are reported as group medians, group modal scores, simple weighted rankings or content themes, depending on the type of data being analysed.

Survey responses reported as median scores

Some questions asked participants to provide a discrete numerical response; for example, 'How many therapeutic residential care homes does your service run in this state/territory?'. Throughout this report, these discrete numerical responses were averaged to produce a group median score. A median score was considered more appropriate than a group mean score due to the presence of outliers in the data.

Survey responses reported as modal values

Some questions asked participants to provide a discrete response, in response to a forced-choice question; for example, 'Is your agency funded to provide therapeutic residential care?' (yes/no). Throughout this report, these responses were reported as group modal values. In this context, a modal value refers to the most common response.

Survey responses that were reported as rankings

Some survey questions required participants to rank options from a prescribed range of response choices; for example, 'What are the most common care arrangements for young people in the homes run by your service? - Please number each box in order of preference from 1 to 5: 1. Voluntary care orders; 2. Other person guardianships; 3. Temporary/assessment orders; 4. Young people with mixed care arrangements; 5. Guardianships to age 18.'

In some cases, participants were able to rank only the top five choices. Each response option was ranked according to a simple weighted group score (a response option that was ranked 1 by a participant received a higher score than a response option that was ranked 5, to produce total scores for each response option). The resultant ranked score for each of the possible response options was reported. Participants were also given the opportunity to provide additional information or to provide responses other than those listed; and this is also reported where available.

Survey responses that were reported as themes

Survey questions that required open-ended commentary were grouped by content themes; and broad content themes are reported in this paper. There were four broad questions that asked for this kind of qualitative information from participants.

Limitations

This survey has limitations due to the categorical or ranked nature of most responses. The decision to adopt questions based on the reporting framework of Lee and Barth (2011) offered the advantage of gathering information in a way that can be compared in a meaningful way with service providers internationally; but there were also limitations imposed by adopting categories that may be more applicable to services outside of Australia. On the whole, however, the responses to this survey, when considered in conjunction with participants' narrative responses, provide foundational information that can be used to further refine future survey questions and identify areas of further exploration.

While a number of participants began the survey, it was evident that only staff in leadership positions were able to provide the kind of information about recruitment, funding and theoretical approaches asked for in the study. This inevitably meant that the views of leadership dominated this research, and the views of workers 'on the floor' are not well-represented in this study. While cross-jurisdictional comparisons were not possible due to the small number of services in some jurisdictions, this should be considered in the future if feasible.

A significant consideration in interpreting the results of this survey is that the sample is not representative of all services offering therapeutic residential care in Australia. The results offer first insight information into current practice in Australian therapeutic residential care. Another significant consideration in interpreting the results of this survey is the extent to which participating services are indeed offering therapeutic care, as opposed to being described as a therapeutic service; and how a residential care service is defined by funders as providing therapeutic care (for a broader discussion related to the definition of therapeutic residential care in Australia, see McLean, 2018). To further develop this important form of service delivery, there is a need for researchers, practitioners and funders to invest in the development of an agreed methodology for defining and operationalising therapeutic service elements, and for monitoring outcomes in a way that is meaningful for service providers, case managers and funders.