Audio transcript: Evidence Informed Practice in Intensive Family Support Programs: Are we there yet?
Good afternoon, and welcome to the CFCA webinar: Evidence Informed Practice in Intensive Family Support Programs: Are we there yet?
My name is Ken Knight, and I'm Senior Communications Officer at the Child Family Community Australia Information Exchange.
I would like to begin by acknowledging the traditional owners of the lands on which we are meeting. I pay my respects to their elders past and present, and to the elders from other communities who may be participating today.
Throughout May CFCA has focused on Intensive home-based family support. We have released a range of resources on this topic that we hope will assist you in your work. If you haven't checked them out, please see our website - the url is on the slide in front of you.
You will be able to submit questions via the chat box during the webinar. There will be a limited amount of time in which Marie and Greg can respond to your questions, but we will try to get through as many as possible.
It is now my great pleasure to introduce today's speakers, Marie Iannos and Greg Antcliff.
Marie is a registered psychologist with 8 years experience in the field of child and adolescent mental health.
Marie is a research assistant at the Australian Centre for Child Protection, where her main role is to provide research support across the many ongoing projects in the Centre that have a clinical focus.
Greg is a registered psychologist and Director of Professional Practice with The Benevolent Society, Australia.
His work focuses on the implementation of evidence informed practice across diverse child and family programs in NSW and QLD.
In 2007 Greg was awarded a Churchill Fellowship to investigate programs that integrate early learning and care for vulnerable children and families in North America and Europe.
Greg has extensive experience in the field as a clinician, trainer and supervisor/manager of child and family programs in both government and non-government sectors.
Please give Marie and Greg a very warm, virtual welcome.
The aim of the project was to develop a coherent practice framework specifically designed for the Benevolent Society's Intensive Home-based Family Support and reunification programs, based on a resilience-led approach. The framework was designed for programs for families where children were at high risk of abuse or neglect, or where abuse and neglect has already been substantiated. And, although the focus of the programs varied, the core business was essentially to ensure that children were safe from maltreatment and receiving adequate care.
Where adequate care for this purpose was considered sufficiently satisfactory to reduce the risk of re-notifications or re-substantiations of abuse, close the case without court involvement; prevent removal of children into alternative care, or to facilitate family reunification.
So how did the resilience-led approach form the basis of the practice framework? The resilience-led approach firstly defines resilience as having strength in the face of adversity, a capacity to adapt and rebound from stressful life events, strengthened and more resourceful.
It focuses on maximizing the likelihood of better outcomes for children by building a protective network around them. This approach then, emphasizes nurturing the child's adaptive ability and capacity to benefit from the resources, which are available to them, and to make use of those resources to buffer the effects of adversity.
The resilience-led approach identifies five high-level child outcomes that support the development of resilient families, and these were applied across all of the Benevolent Society's child and family programs. These high level outcomes are improving safety, which involves physical, emotional and environmental safety; strengthening, secure and stable relationships; improving coping skills for both children and parents (and that's around emotional self-regulation specifically); increasing self-efficacy; and improving empathy.
The project team determined that while working to enhance children's self efficacy and empathy as useful, the core focus of these target programs was essentially to protect children and ensure their safety in care. Thus, three of the five resilience outcomes were chosen to form the basis of the practice framework, and these were: safety, which refers to the child being physically and emotionally safe and free from family violence and all forms of abuse, and where parents are able to manage the problems which are contributing to the abusive or neglectful circumstances within the family.
Secondly, we look at Secure and Stable Relationships. And this refers to where the child is experiencing stable, secure and dependable relationship across the family, school and community levels, and also is experiencing secure attachment with a primary care giver, and where the child and parent interactions are not coercive or harmful in any way.
And lastly, the outcome of coping skills, which refers to where children are receiving the appropriate support to develop the skills necessary to effectively process past abuse or neglect they may have experienced. And also to develop coping skills to deal with stress in a safe and adaptive way so as to foster resilience, and at the same time, this outcome also refers to the capacity for parents to develop their own skills in emotional self-regulation.
With those three broad outcomes as a starting point, the project team needed to establish the outcomes for practice. This required us to identify what the indicators of a safe and resilient family were. So we had to ask how would practitioners know that children and families were safe, strong, and doing well? The line of logic here was that if families receive the evidence informed practices that worked, these child and parent safety indicators would occur, thus strengthening family resilience and reducing the risk of abuse and neglect.
To determine these safety indicators, the team consulted the literature to determine what it looked like for children to be free from abuse and receiving adequate care in their families. So we consulted the family therapy and child protection literature to identify the main areas of family functioning - in terms of strengths and needs - which are assessed by practitioners who work with families at risk.
This process identified six domains that were then mapped under those six resilience outcomes. These domains included physical and emotional safety, environmental safety (which included stable housing, nutrition, health care and hygiene), plus two categories of parental capability. One, which related firstly to the parents capability to keep their children physically safe from harm. Secondly, the parent's ability to interact with their child - in a positive and nurturing manner -which fostered a secure attachment.
Then there was a domain of social support, which refers to the family's connectedness to the community and the family's social support networks. And lastly, a coping skills domain, which refers to the child having the opportunity and support to pass trauma and develop coping skills to deal with future adversity. This process resulted in the identification of 19 child and 30 parent safety indicators of resilient families. And this table shows 19 child safety indicators that describe what we would expect children to be experiencing when receiving adequate care. They include the child being free from all forms of abuse; living in a safe environment where their physical and - health care needs are being met; where they are receiving nurturing and consistent parenting; are engaged in their schooling; and have supportive social connections within their families, such as their siblings and also with their peers; and lastly, developed coping skill to foster resilience.
This table here shows some of the parent safety indicators that resulted. These describe what parents would be achieving with the support of family - workers, in order to strengthen family resilience. The line of logic here was that if parents were demonstrating these safety indicators, they would in turn, result in the child safety indicators we spoke about before. The parent indicators include, for example (we're not able to go through all of them but some of them include) parents being able to address those issues which are impacting on the family's safety- namely domestic violence, substance abuse, or mental health issues or intellectual disabilities; parents being able to meet their children's basic physical care needs such as stable housing, adequate supervision, health care and nutrition. And also meeting emotional care needs such as consistent nurturing and developmentally appropriate parenting.
For parents to develop the emotional regulation skills so that they are able to manage stress actively and deal with family conflict and crises appropriately, and develop non-coercive and non violent parenting strategies in times if stress. And for parents to understand the impact of trauma o their child's wellbeing, and lastly, to develop self care skills so that they are able to - the confidence to make the changes that are required to increase their families safety. For example, by being able to access the available community supports and resources around them.
The next step was to find out what the research evidence told us worked in practice to help families achieve these outcomes. So to do this, the research team conducted a literature review of the programs and interventions that had been applied to vulnerable families at risk of abuse and neglect, or to families where abuse or neglect had already occurred. We looked for those that had been rigorously evaluated and where possible, had been subject to randomized control trials. However it should just be noted here that not all of the programs and interventions that were subject to the literature review could be evaluated with a high level of rigor of a randomized control trial.
Nevertheless the literature search identified 13 programs of interventions in total. These came from a range of professional disciplines, including social work, psychology and nursing, and they included not just specialized programs developed for families in the child protection population, but also general parenting programs and family therapies from a variety of theoretical approaches which had been applied to high risk families with positive results. These include for example, cognitive behavioural' and attachment theory approaches.
Briefly, here is a list of the programs and interventions which came up in the literature review, and as you can see, they range widely from generalist parenting programs such as the Incredible Years and Triple P Parenting Program, to tailored programs for parents with substance abuse issues such as Parents Under Pressure, and specific programs developed -families at risk, such as The Homebuilders and Project SafeCare programs. The common thread through all of these programs was that they had demonstrated positive outcomes for children and families who received them - either through a reduction in abusive parenting practices, or the prevention of further abuse or neglect, or that there was a reduction in the re-notification or removals of children back into the child protection system, following participation in such programs.
While it was good to find the programs and interventions which the studies told us were effective, the next step was to find out what made them work, so we asked what are the specific practice components in each these programs which we could identify? It should be noted here that while a program can be conceptualized as a collection of practices performed within a particular theoretical structure or format, practices themselves are the specific skills, techniques and strategies that the practitioner uses within the program of treatment. And it was the practice components themselves that we needed to identify next in this process.
Two workshops with the Benevolent Society were held to examine each of these programs and interventions in detail, and break them down into their practices, which we called critical practice components. This process all up identified 36 critical practice components that were effective. The critical practice components were described in terms of worker activities, which would be provided to the family. So, for example, some of the critical practice components identified included developing safety plans with parents; facilitating a families access to concrete resources - including emergency food or accommodation; providing parent skill straining in basic - skills such as basic hygiene, nutrition, home safety and childcare; practitioners helping parents develop skills in conflict resolution, problem solving and stress management; teaching positive parenting and behaviour management strategies; or providing therapeutic interventions which are aimed at strengthening parent-child attachments; and lastly, another example is where workers may facilitate the family's connections to social support networks within their own community, and also within the school environment.
The next phase then involved examining these critical practice components and breaking them down even further, into their individual practice elements or practice activities. A practice activity is a specific trainable activity that comprises micro-skills, or also known as kernels. For example, a micro-skill may be teaching relaxation techniques, or teaching a parent how to use praise or 'time out' effectively. Each of these practice activities is then linked to practice skill guides which from the foundation of the practice framework.
So in summary, the development of this practice framework was a process of both a 'top down' an 'bottom up' approach to applying the research evidence we knew that worked, in a systematic way, and imbedding it into the resilience-led approach framework structure. The end point of the process resulted in the identification of the 44 practice skill guides and resources that underpinned the practice framework for the Benevolent Society's family support programs. The majority of the practice guides were developed collaboratively by the Parenting Research Centre, together with the Benevolent Society, and they were written under each of the five high level resilience outcomes. They were also adapted from practice wise evidence based service database. Three of the guides were developed by the Centre For Child Protection and these were around developing a sound safety plan, parent's skills training and also a guide around the application of motivational interview techniques for engaging families.
When supplemental information was required by practitioners, this framework also refers to the resources which were developed by the evidence- informed Australian parenting websites, in particular the Raising Children Network to which the parenting research centre contributed, and also the Children of Parents With Mental Illness website. The practice guides themselves are highly structured, goal focused, and they're written in a practical step-by-step instructional format that describes the micro skills involved in each practice.
The guides describe who the practice is for, and when and where the practices are to be conducted. They also provide, for some of them, a corresponding supplementary worksheet for workers, which they can refer to, and use with parents and children, if appropriate. This slide here shows an example of some of the practice guide titles that fit under each of those three resilience outcomes. They cover practices, which, for example, aim to foster more secure attachment between parent and children.
For instance, here we've got under Secure and Dependable Relationships, one of the practices is called Following Your Childs Lead, and what we find in a lot of our child protection families or more intensive court services, what parents tend to do is to follow when they should lead, and lead when they should follow. So this practice really helps parents to get into the 'world' of the child and to improve that security and dependability of the relationship.
So here is an example of a practice guide. This one is around assisting parents identifying children's emotional states.
So before me move on to discuss the implementation of the framework, its important first to clarify here, this practice framework is not a prescribed program, but rather an evidence-based - to use as a guide to support workers' interventions with families. It is intended for use by practitioners, after they have first completed a comprehensive family assessment of the families' strengths and needs and develop their case plan. Workers then use their own clinical judgment to determine which practices are required and at what phase of the intervention they should be applied. For example, a practitioner may first work on establishing safety and providing basic concrete resources with the family to establish stability, prior to working on issues around attachment. So, how has this framework been utilized by workers at the benevolent Society? Well next Greg will talk about how the implementation process proceeded.
Thanks Marie. I'm going to talk about the uptake of evidence informed practice and some of the barriers and gaps we have. What you've seen so far is the development that we have great evidence informed practices, but its not enough to have the product. And when we're talking evidence informed practice, I suppose the definition we're working from is that it's the best evidence combined with the knowledge and the experience of practitioners, but also the views and experiences of service users, and the context in which its going to be delivered.
I'll talk a little bit more about the context as we delve more into the implementation. What we also know is when we're working in intensive family support services that the work is challenging, its often difficult and multi-faceted, crisis-driven and (there are) multiple complexities of issues. At the same time, what practitioners have to deal with is an avalanche of evidence informed practice - and how do you know to select which interventions for which family, at which time, to get the required effect?
We also know that there's a significant gap between what we know from research that works and when we get into the world of practice. And I've cheekily put in there "mind the gap". But what we now know is that there's implementation that can bridge the gap between the world of research and the world of practice. There's been some research around the gap between when we know something works, and until its implemented into practice is around 17 years. Now, if we think about that, there are three cohorts of children under the age of five that are gone by the time something's been translated from the world of research to practice. But now what we have is implementation frameworks that help us to bridge that gap and link the world of research to practice.
We also know that often what is known is not what is adopted to help children and families. And that's often because there's barriers to systems or at organisational levels, and sometime practitioners' preferences too - even though they know something works, they prefer to use something else. We also know that investing just in training practitioners, or training alone, doesn't result in the uptake of evidence. It's about a five percent uptake of new skills developed by training alone. However, if you couple that with coaching and supportive systems and processes, you get about a 95% uptake for practice transformation.
So there's a gap. So there's no clear pathways to implementation and what we often know - and I've seen happen in many survives over the time - you implement something, and then within two years' time (because of staff turnover, etc.) it often disappears. So how do we ensure that when we implement something that it's going to be sustainable over time? So what we've got so far then is, we've got evidence on effectiveness. What that does is it helps us select what to implement and for whom. However, evidence on these outcomes, it doesn't help us to implement the program or practice.
I've implemented a program once and we invested heavily in a parenting program, and because it had the best evidence of effectiveness. We trained all the staff and we went "off you go, there it is" and two/three years down the track, it's been a bit of a waste of money because we've had high staff turn over and its not been used in the matter it was intended. So we know that there are critical barriers to implementing evidence informed practice. Some of those can be in our organization setting, or the context, sometimes it could be the capacity of the workforce to implement a particular evidence informed practice.
We need to know that the workforce has the skill to implement the practice that we're asking them to implement. We also need to address organisational policies and procedures, and another barrier is that narrow projects or practice standards and guidelines -or a procedure-orientated approach to introducing new evidence - can be really limited. I know once you've got a practice guide you think "well everybody know about it", but how often does it not translate into meaningful practice change on the ground?
So in order to implement the evidence informed practice, so far what we have is ...program or practices that we know are effective (in and of themselves) to achieve those five level outcomes. Now I want to focus the attention "well how do we effectively implement these practices?" and these are strategies to maintain and change behaviour of the practitioners to ensure that it's a hospitable organisational systems that are going to support it. And in order to do that there's a "who?" part there that we need expert implementation assistance to do it.
So I just wanted to show you a picture here of ... how once an evidence informed practice has been developed, close attention needs to be paid to the process of the implementation. And largely, we focus on training staff...training staff. But as you can see, implementation drivers (from this diagram) that there's many more things that we need to address in order for it to lead to improved performance, and to get the improved outcome for children and families.
On one side you've got your competency drivers, so its not just training its "what's the coaching you out in place to sustain the skill development of staff over time" and then "what implications does that have about who we select in our staff selection processes? On the other side, we need to identify what are the systems or the facilitative drivers or what data are we going to collect to inform our decision making? (so they're the organisational drivers). And that requires a technical type of leadership to achieve all those things. So, buy training staff, its not implementation, its only one small element of a larger process if we're going to have sustainable practice transformation over time.
What we've learnt from implementation science is that - and there's a science to how we implement that - that implementation occurs in stages. Here's a model outlined by Dean Fixson and his group, and they say to have effective implementation, it takes a two to four year process. Well when I first heard this I thought, "that's absolutely crazy"- we've got these practices and we'll be done and dusted in twelve months. But I sit here now and we're three years in, and we're just about to go live, so I now have a much greater respect for the science behind the implementation.
So The Benevolent Society has been working closely with a parenting research centre, and we're using an implementation framework called the Quality Implementation Framework. This has been a recent synthesis of 25 different implementation frameworks, and one that we're applying to roll out is the resilience practice framework. As you can see here, there are four stages - four phases - to implementation. Phase One is where you can have all the initial considerations regarding the host setting; Phase Two is when we start looking at our structures, and what structure we need to create for effective implementation; Phase Three is the ongoing structure once the implementation begins; and Phase Four is how we improve the applications
We're straddling between Phases One and Two at the moment with our implementation. I mean, we've done the self-assessment strategies...we needed to have a consistent practice framework. It fits across our organisation and aligns with our organisational strategy. And I'm going to focus on "Are we ready?" an undertaking readiness assessment and then looking at the structural features implementation as well.
So, when I'm talking about organisational readiness, it's a readiness for change... refers to organisational members resolved to implement a change, or "what's their commitment to a change and their shared belief in their capability to do it?" when organisational readiness is high, organisational members are more likely to initiate the change, exert greater effort, exhibit greater persistence, and display more cooperative behaviour. And failure to actually establish sufficient readiness in organisations can account for about a half of all unsuccessful, large-scale organisational efforts. So that's why we've put significant focus on "are we ready?"
We conducted an organisational readiness assessment of the staff about "well how do staff actually feel about implementing these evidence informed practices? And the readiness measures that we used were from Holt (2007), and it asked four questions: How appropriate do staff feel the practice of change is? What degree of management support is there? How confident does the staff feel to implement the practice change? (that's the change efficacy); and lastly, how they feel about the impact on them personally.
So, the result of this study went out to over 300 of our practitioners and largely, we got great results saying that we were ready. The majority of respondents were supportive of the changes and they believed that they be good for the organization. Most employees felt confident about their ability in terms of integrating the changes into their practice. They also indicated that they didn't think their jobs would be adversely impacted by the changes. Staff in three regions (one in a rural, regional and metro area) were unsure if senior management were supportive of the changes. Now this is quite relative, as the changes were quite high, so we still had 60-70% positive responses, and staff in one regional and metro area - they were the least positive about the changes.
So what this told us is that we needed to go back and do some readiness testing, or something, to combat that in those areas. So what we did then is...our readiness is high, its telling is "all systems go" to implement. We needed to create the structures for implementation, and what we've learnt is that too many times we've pushed out changes on to staff, rather than getting them involved in the decision-making. So we created three local implementation teams, in our three areas that we selected to go first. Two are in regional and rural contexts, and ones in the metropolitan context. And those regional implementation teams have their managers and senior practice staff on there. What we do there is we've got some expert implementation and support from The Parenting Research Centre, and we've been having fortnightly meetings to discuss all of those elements - 14 elements - of the quality implementation framework.
We take each element to ensure that we're ready - that we've addressed it - and then all the conditions necessary for us to implement. So what this has resulted in, is increased the readiness of each of these areas and identified things that we have to adapt for the context in which the evidence informed practice are going to be delivered in. one of the learnings that we've had on this is a realization that we needed t go back and do more training, that the assessment tool that we were using was not being implemented or used in the manner in which it was meant to. So we actually needed to go back and rewrite the assessment tool and adapt it. So now it's called The Resilience Assessment Tool and Resilience Outcomes Tool. So now there's a clear line of sight between the engagement assessment and analysis and planning interventions and reviewing outcomes.
We realized that we couldn't just train people in practices, because if they've had more skills in their bag, but we needed to realise the information you collect and analyse has implications for what interventions you use, for which families, and when. So that was a very important learning for us that slowed down the implementation process, however, staff now feel more ready, and we've just finished rolling out their pre-innovation training on their tool, and the practitioners have just been reporting back that they can now see the line of sight between what you collect information in, and then selection of practices.
So this is what I mean by linking assessment analyses to intervention and its just an example of our analysis tool. You'll see up there it says Secure and Stable Relationships -well that's one of the outcomes that we're trying to improve in, and its got room there to record strengths, needs, the conclusions from our analysis, and areas to work on together. The middle column there is a prompter for the practitioners - there the information domains that we collected information in - that's we're your going to be pulling your information on to populate the strengths/needs areas to work on. And over on the other side, are the practices that are known - in and of themselves - to improve and secure in unstable relationships.
For instance here, you've got a descriptive praise, attending to your child's needs, engaging an infant, etc. so that's how we're linking the line of sight to the assessment analyses and intervention. So where are we at in the next steps? We're going to continue to roll out the resilience assessment tool with a quality assurance wrapped around it to ensure that we know that people are using it in the manner its intended. We continue our detailed project planning to address all the areas of the elements of the implementation that we need to for each area. We're currently co-designing the training framework with the Parenting Research Centre, and because its a skill development base for our practitioners, it will be observed practice feedback...just like when we want parents to learn new skills, it requires first to see it, then to practice it, and then get feedback so you can improve on it.
After the training, because that in itself is not enough, we need to have a coaching framework that's wrapped around it, that's going to support staff to be able to reflect on where they're at in their skill development, and then improve upon that in applying all the 44 evidence informed practices. We're also identifying practice coaches in each area and he'll be providing that coaching to staff on a monthly basis. I think one of the other important things that we do within the Benevolent Society - we're developing a new case management, or computer system, and the assessment tool and all the interventions will front-end loaded and built in to the new system. So this will help us determine which of the interventions that we're using are more likely to lead to which outcomes, for which families. Which will be very exciting down the track, as of January next year, that we can really start collecting that data and using it for a quality improvement framework to really wonder about our practice and to improve upon it.
We're also designing an evaluation and an outcomes framework around this. How did we know that we would use the intervention framework effectively? Because if something's not implemented effectively, you're not going to get the outcomes you intended. And its very exciting, and nerve wrecking, that we go live in 2013. So we're going to have to watch this space about where did this land? Hopefully, that we're addressing all the necessary conditions for it to be very successful. So I did want to point out before we finish that there are some limitations to this approach.
The evidence informed practices that have been developed for this framework - they've not been tested or bundled together before. There's evidence of their efficacy in and of themselves, but packaging-up of these practices - to improve on those five outcomes - is yet untested. I suppose our outcomes and our process evaluation will actually shed light on that. The other thing is, we could have gone and selected one of those thirteen programs off the shelf that Marie talked about. Its very expensive to do that, and often prohibitive for not-for-profit organisations to do that, because of licensing fees, etc. but implementing a framework like we've developed, like this, I mean it is innovative in nature, but it means that it's been a long and hard and quite often slow process, so we're asking people to trust in the approach that's going to reap the rewards.
The last one that we're addressing there - that there has been no cultural adaptation of these evidence informed practices for Aboriginal clients or culturally and linguistically diverse families. We have tested the resilience practice framework and the notion of resilience with our Aboriginal clients and staff, and they've said that the notion of resilience resonates with them because they regard themselves as resilient people. However, the practices that we've now identified...we're going to have to go through a process with our internal Aboriginal network and also engage in some clients from culturally and linguistically diverse backgrounds to test whether these interventions are culturally appropriate.
So are we there yet? No, but we're well on the way to implementing evidence informed practice framework, with the help of the Australian Centre for Child Protection and The Parenting Research Centre, and of course with the staff at the Benevolent Society.
Thank you, Goodbye
END OF TRANSCRIPT
IMPORTANT INFORMATION - PLEASE READ
The transcript is provided for information purposes only and is provided on the basis that all persons accessing the transcript undertake responsibility for assessing the relevance and accuracy of its content. Before using the material contained in the transcript, the permission of the relevant presenter should be obtained.
The Commonwealth of Australia, represented by the Australian Institute of Family Studies (AIFS), is not responsible for, and makes no representations in relation to, the accuracy of this transcript. AIFS does not accept any liability to any person for the content (or the use of such content) included in the transcript. The transcript may include or summarise views, standards or recommendations of third parties. The inclusion of such material is not an endorsement by AIFS of that material; nor does it indicate a commitment by AIFS to any particular course of action.