Collective impact: Opportunities and challenges for implementation

Collective impact: Opportunities and challenges for implementation

Jessica Smart, Kerry Graham and Arlene Hand
30 November 2017

This webinar presented an overview of collective impact with a focus on leadership and governance, community engagement, and evaluation.

Collective impact: Opportunities and challenges for implementation

Audio transcript: Collective impact: Opportunities and challenges for implementation

Audio transcript (edited)

MS GOLDSWORTHY

Good afternoon everyone, and welcome to today's webinar, “Collective impact: Opportunities and challenges for implementation”. My name is Kat Goldsworthy and I'm a Senior Research Officer here at the Australian Institute of Family Studies. Today's webinar presentation will provide an overview of collective impact with a focus on leadership and governance, community engagement and evaluation.

Before I introduce our speakers, I'd just like to acknowledge the traditional custodians of the lands on which we are meeting. In Melbourne the traditional custodians are the Wurundjeri people of the Kulin nation. I pay my respects to their elders, past and present, and to the elders from other communities who may be participating today.

Firstly, though, some housekeeping details. One of the core functions of the CFCA Information Exchange is to share knowledge, so, I'd like to invite everyone to submit questions via the chat box at any time during the webinar. We will respond to your questions at the end of the presentation. We'd also like you to continue the conversation we begin here today. To facilitate this, we've set up a forum on our website where you can discuss the ideas and issues raised, submit additional questions for our presenters and access related resources. We will send you a link to the forum at the end of today's presentation.

As you leave the webinar, a short survey will open in a new window and we would appreciate your feedback. Please remember that this webinar is being recorded and the audio transcript and slides will be made available on our website and YouTube channel soon.

It's now my pleasure to introduce today's presenters, Jessica Smart, Kerry Graham and Arlene Hand. Jessica Smart is a Senior Research Officer in the Practice Evidence and Engagement Team at AIFS. She works across both the Child Family Community Australia Information Exchange and Expert Panel projects. Her previous work has been in the not-for-profit and health sectors, developing, delivering and evaluating community development and population health projects, including projects with young people, people with intellectual disability, and culturally and linguistically diverse groups.

Kerry Graham is a Founder and Director of Collaboration for Impact, Australia’s leading organisation for enabling people to tackling big, tough problems and create large scale impact through collaboration. In this role, Kerry provides support to and curates learning for communities, corporations, governments, philanthropy and non-profit organisations on how to drive large-scale social change in intractable social challenges. Kerry also lectures on collaborative practice with Centre for Social Impact at The University of New South Wales.

And, finally, Arlene Hand has worked in the community services sector for over 20 years, working in both not-for-profit organisations and local government. She has a passion for working on community and sector development initiatives with her early career starting in Youth Work. Over the years, Arlene transitioned to community development and project management roles in early years, youth and mental health sectors. Arlene joined Communicare, Communities for Children in 2015 to initiate and guide the Community Dimensions collective impact project in the early years.

Unfortunately, Arlene is unable to be here today, so, we'll be playing a pre-recording of Arlene's presentation for today's webinar. This means Arlene will be unavailable for the Q&A session at the end, but will be available to answer questions via the online forum on the CFCA website. Please join me now in giving our presenters a very warm, virtual welcome. I'll hand over to you, Jess.

MS SMART

Thanks, Kat. So, as you've heard from Kat, we've got three presenters today. I'm Jessica, I'll talk about the theory and evidence for collective impact, then we'll hear from Kerry Graham, who'll give us a national perspective on collective impact and, finally, you'll hear the recording of Arlene's presentation which is a case study of her collective impact project in Armadale district in Western Australia.

Okay, so, we'll start at the beginning, what is collective impact? It's a framework that's used to achieve population-level change on complex or wicked problems and that is problems like homelessness, family violence or obesity, problems with multiple and intersecting causes and problems where interventions to address them can have unknown and unpredictable outcomes.

The term, "Collective impact" was coined by Kanya & Kramer in an article in the Stanford Social Innovation Review in 2011, so only—what's that—six years ago. They presented collective impact as the opposite to isolated impact. So, actions taken by a single organisation to address a single issue and Kanya & Kramer argue that isolated impact has not been effective at addressing complex problems.

So, collective impact is designed to work in complexity, and, in a collective impact project, agencies, groups and individuals that have an interest in the particular issue come together and they use the collective impact framework to guide their collaboration. And stakeholders, usually include a mixture of non-government agencies, various departments and levels of government, private business, community members and particularly community members with lived experience of the issue, and, depending on what the issue is, people from other sectors. So, it might be people from the education, health or transport sectors.

And collective impact has been rapidly adopted in Australia, particularly in the last few years, and Kerry will talk about this a little bit more, and it's most frequently employed in a place-based setting. So, collective impact works through these stakeholders coming together and applying five conditions, and the five conditions, you see there, that's: a common agenda; continuous communication; the backbone; mutually reinforcing activities; and, a shared measurement system.

I won't explain these. If you're not familiar with them you can check out the Kanya & Kramer article, "Collective impact" which is in your resources. I do, briefly, want to mention the backbone, though. I'm going to talk a little bit today about a few things that I think are particularly promising about collective impact and one of these is the backbone. And, "The backbone" is the name that's given to the supportive infrastructure that sustains the coalition or collaborative of stakeholders. And this is an essential function that I think has been really overlooked and under-resourced in a lot of collaborative change efforts, and I think it's something really valuable that collective impact has brought to the field.

I recently heard someone describe collective impact as a skeleton that needs meat on its bones and I think that was a really apt description. So, I think these five conditions provide a skeleton or a framework to achieve collective impact, but by themselves, they're not enough. And there's been some criticism of these conditions, mostly to do with what is missing from them, rather than a criticism of the conditions themselves. But, in my opinion, and I've written about this in the collective impact paper that's available in your resources, the gaps in the framework are now mostly addressed by other resources or by evolutions of the framework, such as the Tamarack Institute's Collective Impact 3.0.

So, before I get to our three topics for today, I think it would be remiss of me to not talk about what the evidence is for collective impact, and I'm going to start by talking about what we don't have. So, we don't have conclusive, irrefutable, high-quality evidence that collective impact is effective, we don't have any systematic reviews or any meta-analyses that can tell us, without a doubt, that implementing a collective impact project is likely to have population-level impact.

There is some projects, particularly in the States and Canada that have reported positive results, but we don't yet have consistent high-quality evidence. And why don't we have that evidence? Well, not because collective impact is ineffective, necessarily, but simply because it's relatively new. So, it's only been six years since Kanya & Kramer published their article in the Stanford Social Innovation Review and collective impact is really still evolving and settling into something that can be evaluated, and we're still working out what collective impact actually means and how best to do it and what it looks like in Australia as opposed to the States or Canada.

And, as I mentioned, it's really new and even more so in Australia than in North America. Mini-projects here are still in the early stages of development. Kerry may talk more about this but I'd say there's very few collective impact projects in Australia that have progressed enough to be measuring outcomes.

So, what do we know? Well, we know that collective impact is in line with what we think is the most effective way to address complex problems. We know that complex problems need intersectoral and multi-level interventions and collective impact can provide a framework for that. We also know that there's been rigorously evaluated collaborative initiatives in public health that share many characteristics with collective impact. So, I'm talking about things like Communities That Care in The United States and we know that they've demonstrated effectiveness.

So, today, each of us—myself, Kerry and Arlene—are going to focus on three areas that provide opportunities and challenges for collective impact. When I first started reading about collective impact, I was actually pretty sceptical about it and I think mostly that was because of the way the early articles promoted what I understood to be a very top-down approach, and, as a community development practitioner, I just couldn't see how we could achieve large-scale social change without citizen participation.

But as I read more about it and as I spoke to people that were implementing it I changed my mind on this and I now think that it has significant potential not only to address complex problems, but also to improve the way that some things are done in the social services sector. In saying that, I think that a lot of work needs to go into implementation. I think that the potential of collective impact isn't a straightforward or an easy thing to realise, and that's why I chose these three topics to focus on today.

So, community engagement, leadership and governance, and evaluation. I think each of these areas are challenging for collective impact in that, if they're not done well, they limit the effectiveness of a collective impact project. But, on the other hand, if they are done well, they not only enhance the effectiveness of a collective impact project, but they can also have a positive impact on the way we do things in the social services sector more broadly.

So, I'll start with community engagement. This was one of the major initial criticisms of collective impact, and it was my biggest problem initially with collective impact as well, was that it didn't include the community in collaborative change efforts. To be clear, when I say, "the community", I mean the citizens, the residents of an area in which a collective impact project is taking place and, specifically, people with lived experience of the issue which the project is trying to address. I should say that, now, there's actually many collective impact resources on community engagement. It's quite a hot topic I think in the collective impact community.

So, we talk a lot about community engagement and how important it is and, certainly, there's a lot of ethical arguments for why community engagement is necessary, but there's also practical reasons to do community engagement. And these practical reasons mean that, without meaningful community engagement and participation, a collective impact project is less likely to be effective.

So, firstly, there's often significant differences between people working on a collective impact project and people who are the intended beneficiaries of the project. So, these might be difference in education, employment, income, cultural or socio-economic background and life experience. These differences mean that, unless we do community engagement, we're really relying on assumptions about what the community needs and what the community wants and how we should deliver these responses. Community members are experts on the local context and we really need that knowledge to design effective strategies and solutions.

Secondly, and this is discussed a lot in the collective impact literature, including community members increases the chance that a project will achieve transformational change. So, there's two reasons for this. Firstly, involving community members necessitates a shift and a rebalance in power dynamics and, secondly, if we only have services and government involved in a project, we're likely to get service-oriented solutions, and while it's important that we have effective services, they can really only alleviate the symptoms of a problem and they can't really address the causes of those problems.

So, community engagement is important, but it's not easy to do in practice and I think it's rarely done well, and this is true across the whole social services sector, so, I'm not just talking about collective impact projects here. But genuine community engagement really requires a paradigm shift, and I like the way that Harwood talks about this as a turning outwards so that the community becomes the focus of a change effort, rather than the agencies who are involved. And, finally, I think community engagement's most effective when it brings community members together with people who have power so the decision makers in a community, and I think that there's some collective impact sites that are doing that really well.

And that brings me, quite neatly to the next area I'm going to talk about which is leadership and governance. So, again, I think if these are done well, if leadership and governance is done well, I think it can really provide an opportunity for innovation in the social services sector. I want to talk about two things here and I call them opportunity challenges, because I think, like a lot of the other things I'm talking about today, they are both an opportunity and a challenge.

So, the first of these is the need to combine top-down and bottom-up leadership and to do this in a way that ensures all voices are heard and given weight. Governance structures need to be flexible enough to facilitate meaningful community engagement and participation but they also need to be strong enough to hold a really diverse group of stakeholders together through what can be an uncertain and unclear, and, at times, a really slow process. And linked to this is the challenge of combining different data sources, so combining research evidence and population level data with community knowledge.

The second opportunity challenge is the shift in leadership style that's required by collective impact, so, more so than kind of your traditional authority based leadership, I think leadership in collective impact is relational. Leaders have a facilitative role and they're responsible for the health of the collaborative as much as they're responsible for the outcomes of the collaborative, and this requires leaders to build and maintain trust, to be effective communicators and facilitators and to be able to do this with a really diverse group.

And I think this is a difficult, but a crucial role, not only because you need to keep stakeholders engaged and motivated, but also because you really need to build a culture that is non-competitive. So, a culture where you can recognise what's working and you can scale that up, and you can recognise, activities or interventions that aren't working and you can change or stop those. And for this to happen you really need to build and maintain high levels of trust and a safe-to-fail culture.

So finally, evaluation. I think there's no doubt that evaluation is one of the most challenging areas for collective impact projects, but I also think that the use of data-driven decision-making is also one of the most exciting things about collective impact. An evaluation, generally, I think is unloved in the social services sector. It's often seen as challenging and expensive, it's not prioritised, often practitioners or managers don't have evaluation skills and I think evaluation is quite often seen as an add-on to the essential business of service delivery.

While I generally disagree with that, I think that evaluation is particularly critical for collective impact and, for collective impact projects to be successful, I think they really need to embrace evaluation. And there's two forms of evaluation that I think are both critical. So, the first of those is ongoing data collection and evaluation for continuous learning. When you're working in complexity, actions and interventions can have unanticipated effects, so, it's important that they're monitored.

So, through monitoring, you can see what's working and scale it up and you can see what's not working, and you can change it. And this type of monitoring and evaluation I think can be done in a couple of different ways. So, you can use, developmental evaluation methodologies or you can use something more like an action research cycle. I think that so much of the power and potential of collective impact is in the way that it responds to complexity and is adaptive to changes in the context and evaluation for continuous learning really enables that.

Secondly is evaluation for impact. So, the shared measurement system is part of this story. It can determine whether or not a project is having an impact on population-level outcomes, but evaluation for impact, and I'm talking here about evaluating outcomes can also provide accountability to stakeholders, including community members and to funders, and it also contributes to the evidence base for collective impact.

So, I talked earlier about what evidence we do and we don't have for collective impact, if we're not doing rigorous outcome evaluation, not only do you not know the impact of a project, but you also can't build and share knowledge and contribute to that evidence base, about how collective impact works. Okay, so, that's it for me, I'll now hand over to Kerry. Please do remember to submit questions via the question box and Kerry and I will be able to answer them at the end of the presentation. So, over to you, Kerry.

MS GRAHAM

Thanks so much, Jess, really stimulating start to the webinar, and hello everyone. So, Jess has asked me to speak today about what are we seeing nationally in the collective impact movement? So, in order to be able to comment on that I just want to spend a moment saying what vantage point does CFI to see national patterns.

So, we're a very small non-profit that seeks—the only role we play is to enable people and collaborations to do the work of changing the system in their place or on the issue that they care about using collaboration. So, quite often we describe our role as walking alongside collaborative efforts for a minimum of seven years to help them build the capacity and the infrastructure needed to drive large-scale change.

So, what we've been seeing is that, since collective impact was published in 2011, there's been a fairly steady increase over time. Many initiatives that were using collaboration as a way to make change read the collective impact article and said, "Wow, they've described exactly what we're doing". But they learnt what things they needed to do better and that particularly relates to two things Jess has spoken strongly about, which is getting clear on the role of the backbone and its unique role and strengthening that. But also shared measurement, getting disciplined about measuring the shared outcomes that the collaboration is to achieve.

But many initiatives or many communities or issues read about collective impact and thought, "This is something that could help us make change, the type of change we've yearned for and not been able to achieve". So there's been a significant increase over time. Last year we plotted around 75 initiatives who were intentionally using collective impact or drawing on its conditions to inform their work. Now, this is a trend sort of figure—not an accurate figure—but the graph, the picture that you're looking at on-screen shows roughly where they are around the country.

So, the field has been evolving since 2011. The most common things that communities are mobilising around, or initiatives are mobilising around, is early childhood and particularly cradle to career. So, that might be across the life-course from birth to your first job but also, focusing on different cohorts across that life course with another group quite focused on outcomes for young people as they're leaving school.

The other place where collective impact has been taken up and contextualised is in around Indigenous advancement and there are a cohort of Indigenous-led reforms around the country that are informed by this way of working. So, if there's around 75 initiatives, there's really still only a small handful that have been going long enough to reach a certain level of maturity where I would say they're past agenda setting, the process required for many diverse stakeholders to agree—a shared agenda—and they've built the infrastructure required to implement and are now doing the hard work of implementing that agenda. The bulk or the rump of the field is still in building readiness and the foundations to work this way.

So, what are we seeing? Well, to share a little bit about what we're seeing, let me also share the perspectives we bring when we do look nationally to see what's going on. As Jess has so strongly framed, you would only reach the collective impact if you agreed that the problem or the challenge that you're trying to address is a complex one. You wouldn't convene diverse stakeholders across a system unless you were attempting to make change in a complex problem.

So, the second lens we bring is a lot of systems thinking. So, collaboration requires us to bring diverse stakeholders together and look at where can we intervene in the system that those stakeholders represent that will drive large-scale change. So, the great sort of tagline for this is service delivery alone will not solve the challenges that we're experiencing, we need other interventions to change the way the system operates and to get to root causes, change the conditions within which young children and young people are growing up in order to reach population-level change.

Now, while this field of collective impact is emerging, we do know that collaboration is considered one of the best approaches if you have a complex problem that requires system solutions as well as program solutions. So, with that lens, what we're seeing, in the three domains that we're talking about on this webinar is—returning to community engagement, we're seeing that everybody who is using collective impact as a way to frame their work—is talking about community engagement, but they're experiencing it in ways that are challenging. So, in many initiatives, we're seeing collaborators feel quite paralysed; they don't know, from their own practice or their own organisations how to do community engagement well and they certainly don't know how to do it at scale.

The other trend or pattern that we're seeing is that community engagement is often front-ended at the beginning of collective impact initiatives and is seen as a project that has a start date and a stop date that you do to gather community insights in order to form agenda setting. It's not seen as a foundational way of working, an ongoing practice about being in a relationship and dialogue with community and working, with community in a way that helps the citizens, residents, those that are affected by the challenge to, frame the opportunities and determine their own solutions.

So, there's still quite a bit to do around community engagement, when we sit and look at why; why is community engagement challenging for some initiatives? We really are unpacking questions of power. Are experts in social change, like policy makers and service providers, prepared to sharing power with the benefactors and citizens? And, how can we understand and unearth some of the assumptions that we might be holding, one of which we hear or try to surface quite regularly is, do those who have traditional power or traditional expertise, are they holding the assumption that if we hand power over to community they will, in fact, make the wrong decisions? And, once we name that assumption, how do we actually, test, is that real? Is that assumption correct? Our hunch is that if we start working around understanding the benefits of sharing power and really testing those assumptions around the skills and expertise of community, we'll be able to move past the paralysis and past the project-based approaches to community engagement.

Okay, so, turning to leadership and governance, if we accept the challenges we're grappling with are complex and we need to change the system, not just improve the programs that we deliver, then the principle is convene as diverse group of people as you can and bring their assets and strengths towards understanding the challenge and coming to some shared solutions. What we're seeing around leadership is that most commonly, cross-sector leadership tables are weighted by government and services providers, or, in fact, exclusively held by government and service providers.

There’s not a strong presence of other actors in the system from business, philanthropy, academia and there's an even lower presence of citizens and those with lived experience at the leadership level. What we see happen is that, when government and service providers get together, as Jess has already alluded to, there is an almost predetermined focus on service delivery as the primary way of making change, as the primary solution set. So, the types of characteristics we see is the dominance of program and technical thinking, people moving to agenda setting very early in the process, in the life of a collaboration, limited engagement with community, often to understand, "Are the services that we're offering to you now good enough? Do they meet your needs? And, if not, would you like to be involved in designing new services?" And, a very predictive planned linear approach to making change, when we know, in complexity, we need much more experimentation around our intervention sets.

And the last thing we're seeing is low levels of learning and adaptation when the leadership table is comprised predominantly of government and service providers. When we seek to look at the behaviours that are driving that, again, we think those behaviours are connected to, apologies, those behaviours are connected to power dynamics. Let me jump back through the slide.

So, our hypothesis that we test with collaborations, are - is this - that we prefer to work with people like us? When we engage diversity people, by definition, come with different understandings of the challenge and different opinions about the solutions and does that make us feel uncomfortable? Are we avoiding that? Are we avoiding sharing power? And a very strong narrative is, "We're not the ones that need to change, it's others out there, it's government that needs to change, it's families and that needs to change or it's the way businesses operate and employ people that need to change, it's never ourselves."

So, one of the things we observe is there are a plethora of theories of change on how to change behaviours in the lives of children and young people and their families and we need those—they're the evidence base, they're the practice that many of us turn to when we're seeking to make change. But there are very few theories of change or frameworks that help us understand, what are the behaviour changes we need in other parts of the system? How does government change structurally and change its behaviours to align to the shared goals? How do service providers structurally change and behaviourally change to achieve the shared goal? And, similarly, business, philanthropy, academia. And this, to me, is where the promise of collective impact lies and how exciting it is that there are 75 plus initiatives having a crack at this, because they will be emerging what are the theories of change that create structural change locally and behaviour change, at locally, so that everyone is aligned to the shared goals of achieving better outcomes for children, young people and their families.

So, if we don't have theories of change for how to chang, behaviours of government, service providers and other actors in the system, I think we end up doing something like this. We believe that bringing people together to collaborate will create some magic and, you know, we'll have more alignment and better outcomes, when actually what we need is quite strategic and disciplined collaborative processes and those that are really knuckling down and working out the practice of how to implement collective impact, I think, are generating those theories of change.

So, the last one is evaluation. Similar to Jess, I think the primary purpose of evaluation in collective impact initiatives, certainly the primary purpose in the beginning, in those early stages, is actually strategic learning. How do you get data and insights quickly back to the collaborators and those involved in the change process? How do people use data to be able to adapt the interventions based on changes in the context and the learnings they're experiencing. So, it is fundamentally a driver for cementing a new way of working. Our current way of working is very linear, very predictable, and we need to move to more adaptive, more experimental ways of working that are well informed by data so that we're making the best informed decisions we can.

What we're seeing is that, in the evaluation community in Australia, there are very few evaluators that either know how to work this way or are authorised to work this way. Evaluators are still very much doing formative and impact evaluation, They're not—either they don't hold roles or they don't have the skills and confidence to actually get into collaborations, run alongside them, help them understand, in real time, what data and insights do you need? And then go out and get them in a rigorous way that is timely to help the collaboration make better decisions.

So, we see this as a great developmental need if the field of collective impact is going to continue to evolve in Australia and deliver some of its promises. So, it's probably enough from me, over back to you, Jess.

MS GOLDSWORTHY

Thanks, Kerry, that was really great. We are just going to finish up by putting on the pre-recorded, presentation that Arlene did for us a few days ago, so, I'll leave it with Arlene.

(Recording played.)

MS HAND

So, thank you for inviting me to talk about how Community Dimensions has applied the collective impact framework.

I guess before I speak to the three key areas of community engagement, governance and evaluation, I just wanted to provide some background context to the project and what we're hoping to achieve. The work of Community Dimensions is actually concentrated in the three local government areas of Perth in Western Australia, that's Gosnells, Armadale and Serpentine Jarrahdale.

The project itself was actually born out of discussions the Communities for Children facilitating partner had with some other organisations in the area about how we could improve outcomes for children. It was actually a youth partnership project that had applied the collective impact framework, and started some early work in the area and the question was asked, "Could we use the same framework to improve the outcomes for children?"

The name itself, "Community Dimensions", was driven from a leadership group member who had seen her son play Lego Dimensions, where Batman might be working with Scooby-Doo and Homer Simpson might be driving Batman's car and they're all doing it to get up to the next level. And for us that really spoke to the idea of what collective impact was about, that we could use each other's resources, to change, the future direction.

Some of the key signposts for change for us in the community and why we wanted to implement collective impact was things like our children in care statistics, and our family and domestic violence statistics in particular, because the Armadale district had the highest number of incidents and reports compared to any other area in the Perth metro area. The percentage increase was continuing each year and no matter how much money or new programs were being put into the area, we didn't seem to be making any significant change and, again, our AEDC results, whilst they were improving, they still weren't great in comparison to other areas.

So, we've been on a bit of a journey of engagement, and our journey map sort of highlights some of the key things that we've done. But what we realised very early on is that it wasn't about the three or four people that had sat around the table at the start of the journey, talking about collective impact, that we really needed to be ensuring we were engaging the whole community around that vision for change. So, we started the process by asking, "Is collective impact something we want to use?" And we brought a number of service providers together back in March 2017 just to learn a little bit more about collective impact and share the idea about how, potentially, we could use it to improve some of these statistics around our children.

There was an overwhelming support from that and, so, that really took us on the first step of our journey. Since then we've implemented numerous initiatives, including our Say It Like It Is survey, growing conversations trees, which our playgroups did an area for us, and we held about five tables of 20. Now, those tables of 20 brought four groups consisting of five members each from your community members, so, your mums and your dads, your business and your industry, your not-for-profit services and your local government. , and, I guess, we had a really scientific method of engaging people post each of those initiative, just by asking them if they wanted to be involved, to pop their name on a popsicle stick and leave it in the jar at the end of the day. So, we started to collect—we've got now probably over 70 people who want to be part of the journey with us.

So, given that we had over 70 people that wanted to be part of the journey, we sort of needed to re-evaluate where our leadership and our governance structure was heading. We started in the very early days, where, obviously, Communicare had funding from myself as a project officer and we engaged the initial four organisations to form a leadership group and we asked who else should be at the table So, we really, at that stage, were just looking at services and we hadn't really involved community members.

By the time we'd gone through a series of surveys and tables of 20s, we obviously had over 70 people who wanted to be part of the process. So, by February 2017, we went back and said, "Look, is this something, now, we feel how we can involve all of those people that wanted to be involved?" So, we had, you know, potentially four groups. So, you know, people who could connect with community, people who could support us around some of our governance and sustainability, some people who could gather around the data and research and evaluation to help drive the project and people who were interested in communications.

Obviously, at the centre of that was our community aspiration and our values which, at that time, got articulated as, "Keeping children safe for a better future" and, in particular, keeping children safe from abuse and neglect. At the bottom of that, we also said that we would like to have some kind of strategic guidance, some system changers and system thinkers that could provide, sort of twice yearly support in terms of direction, and for research and connection back into the community, and what was happening for that local community, and potential policy change.

And when the community reviewed that as a potential governance structure there was a general agreement around it but it really felt that, again, our leadership group needed to be involved. So, the blue circle in the current governance structure speaks to that, so the alignment group is now drawn from the membership of each of the other four working groups. And we also had people who said, "I'm really interested in being involved, but I can't commit for a significant amount of time each week or each month, but I'm available to be on-call for particular things if needed". So, there was another group who became our friends of Community Dimensions. At this point in time, we really, just at the end of July, defined our focus area around keeping children safe from abuse and neglect, we are now looking to set up that strategic and system thinkers group just to support our leadership structure. We also needed to work out, how do we make some decisions? Who needs to be involved in those decisions? So, we established this community engagement and decision-making matrix. The, I guess, the essence of this is that the higher the level the decision, the more people that actually need to be engaged in that decision. So, whilst a high-level decision might sit with the alignment group, it's really important for the alignment group to be engaging all of the other groups in that decision. So, that includes the action group, the Community Dimension teams and the Executive of the alignment group as well.

Part of the process as well for us to provide some clarity around what each of the working groups were doing and the decisions that were being made, and what was being asked of them was to develop a project brief template. So, that, for us, really spoke to the task that was at hand for those action groups, and some of the milestones that we'd be looking to achieve.

So, I guess, I liken Community Dimensions to a bit of a hop-on, hop-off bus, when the project started, we started with a very broad aim, which multiple people could align to but, I guess, as we've narrowed the focus and we've defined the issue as keeping children safe from abuse and neglect, some people felt that that didn't - that wasn't something they could align to as well and, so, naturally, hopped off the bus. We've also had parents who've been engaged with us along the way who suddenly had their second child or they've gone back into the workforce, so, there's been that natural attrition and there's also, likewise with services and government, people moving in and out of organisations and restructuring of departments that's meant that we've, you know, had some changes there in terms of our governance and capacity for people to contribute.

We're obviously operating our structure in a very much a shoestring budget, with just one worker allocated to the project and it really drew in on the support from the community and service and governments, just to provide that infrastructure for us.

So, when looking at how we're evaluating our success, there's a few things that we have done and, to be honest, I can admit that we could probably do better. I think what we have done well is that we've tracked our output. So, we've looked at who we're actually engaging and if we're engaging the right people. Some of the things that we've also done along the way is really track how—what our members experience is with the collective. And those are some of the quotes that you'll see there that's been shared with us.

The other thing that we've sort of done well is look at the health of the leadership group, be that the original leadership group or how we're going now with our current alignment group, and one of things, I guess, that we've started to do is really add some learning focus items onto the agenda for each session, we're looking to learn a little bit more about collective impact and also reflect on what we might've learnt or take away from each of the meetings as well.

We've also looked to the FSG framework for evaluating the work of the collective by looking at the four key areas in terms of the context within which we're operating, the initiative and how we're doing ourselves and our learning culture, how we're influencing the system so that behaviour and policy level change and impact. And, obviously, in the first two years, we really only have defined the issue and we're really only gathering data around the context, both in terms of the community and the political context, and how well we're going in terms of building the capacity of the project and our learning culture as well.

I guess, that's the main thing for us, is really seeking to understand the lived experience. We've also developed a logic model to guide us in our project, but we really feel that, for us, the next stage for our project is more about, how do we look at providing some kind of sort of developmental evaluation and maybe action research that both contributes to the project and evaluates the relationships? And the learning culture that is part of what we're doing, so, this is, I guess, a big focus for us as we move into the new year.

So, hopefully, this has given you a good understanding of where Community Dimensions is at and I would just like to thank you for allowing us to share the journey we've been on to date.

WEBINAR CONCLUDED

IMPORTANT INFORMATION - PLEASE READ

The transcript is provided for information purposes only and is provided on the basis that all persons accessing the transcript undertake responsibility for assessing the relevance and accuracy of its content. Before using the material contained in the transcript, the permission of the relevant presenter should be obtained.

The Commonwealth of Australia, represented by the Australian Institute of Family Studies (AIFS), is not responsible for, and makes no representations in relation to, the accuracy of this transcript. AIFS does not accept any liability to any person for the content (or the use of such content) included in the transcript. The transcript may include or summarise views, standards or recommendations of third parties. The inclusion of such material is not an endorsement by AIFS of that material; nor does it indicate a commitment by AIFS to any particular course of action.

Slide outline: Collective impact: Opportunities and challenges for implementation

1. Collective Impact: Opportunities and challenges for implementation

Jessica Smart (AIFS)
Kerry Graham (Collaboration for Impact)
Arlene Hand (Communicare)

2. Overview

Jessica Smart

  • Theory and evidence for collective impact

Kerry Graham

  • National perspective on collective impact 

Arlene Hand

  • Case study of collective impact in Armadale, Western Australia

3. What is collective impact?

  • A framework used to achieve population-level change on complex or “wicked” problems
  • The term originated from Kania and Kramer in 2011
  • Contrasted with “isolated impact”
  • Collaboration between stakeholders on a defined issue
  • Most frequently employed in a place-based setting

4. Five conditions of collective impact

5. Evidence for collective impact

  • No conclusive, irrefutable, high quality evidence that collective impact is effective
  • Collective impact is new and evolving
  • Limited evaluation of Australian collective impact projects
  • What we do know:
    • Collective impact generally fits with what we think will be effective when addressing complex problems
    • Place-based, collaborative initiatives in health have been effective

7. Community engagement

  • One of the major initial criticisms of collective impact
  • Now many collective impact resources on community engagement
  • Community engagement is important
    • Community members are experts on local context and issues
    • Demographics of community professionals often different to people who change is intended to benefit
    • Transformational change
  • A challenge, not just for collective impact but for the whole community sector
  • Not always done well
  • Requires a paradigm shift
  • Community engagement most effective when it brings community members together with people who have power

8. Leadership and governance

  • Two key opportunity-challenges
  • First opportunity-challenge: combining top-down and bottom-up leadership
    • Combining research evidence, data and community knowledge
    • Flexible but strong governance structures
  • Second opportunity-challenge: shift in leadership style
    • Leaders as facilitators
    • Importance of relationships and “soft skills”
    • Developing a productive culture
    • Importance of trust

9. Evaluation

  • Evaluation is critical for collective impact
  • Evaluation for continuous learning
    • Monitoring actions and interventions to see what is effective
    • Action research cycle, developmental evaluation
  • Evaluation for impact
    • Have we contributed to population-level change?
    • Accountability to stakeholders, community, funders
    • Contribute to evidence base for collective impact

10. 
Collaboration for impact

Kerry Graham

11. Who is CFI?

Collaboration for Impact is Australia’s leading organisation for enabling people to tackle big, tough problems and create large-scale change through collaboration.

We believe in a society where everyone is able to thrive - where communities come together to solve their own challenges and seize opportunities.

As a learning organisation we

  • Build skills and capacity to address complex social problems
  • Offer a range of face-to-face and online learning events and programs
  • Develop capability to convene, design and lead collaborations
  • Use data, facilitate learning, and measure progress & impact

12. What are we seeing?

a) Intended impact

  • Early childhood
  • Young people
  • Cradle to career
  • Children, young people & families
  • Indigenous advancement

b) Maturity of the field

14. What are we seeing?

15. The lens we bring

16. Community engagement

Everyone is talking about community engagement…but

  • Shifting from seeing engagement with communities as transactional activity to embedding a practice of ongoing engagement where communities are defining their opportunities and framing their problems and designing their own solutions.

17. Community engagement

What is required?

  • Are often stuck in paralysis because they don’t know how to do it
  • Most are still engaging community in one off, ad hoc ways as an activity not a practice

Why?

  • Power? Retaining ‘expertise’ or reluctance to give up or handover power to communities?
  • Assumption that community will make the wrong decisions?

18. Leadership and governance

19. Evaluation

  • Primary purpose is strategic learning – to get data and insights quickly back to the collaborators
  • The collaborators and innovators needs to be able to adapt the interventions based on changes in the context and learning.
  • Very few evaluators in Australia who work this way

20. Community Dimensions

Keeping children safe for a better future

21. Community Dimensions Project

22. Sign Posts for Change

Key Sign Posts for Change

23. Community engagement

24. Leadership and governance

Community Dimensions Engagement and Decision Making Matrix

25. Leadership and governance

a) Sustaining the Governance Structure

b) Community Dimensions operates on a shoe-string budget with a 0.6FTE worker allocated and the rest of the backbone drawn from our “popsicle” people.

26. Evaluation

What we have done:

a) Tracked our outputs – who we are engaging and if we are engaging the right people

b) Members experience within the collective

"Great to get community - local people local ideas - with support and framework from 'lead' agency"

"Excellent collaborative process which I think will lead to positive community outcomes"

"Nice to have a child in the room to remind us why we are here"

c) Health of the Leadership Group

  • Leadership Group Survey 2016
  • Learning focus on meeting agenda

d) Developed and revised a logic model

e) Looked to the FSG framework

f) Currently exploring how we might support an evaluation that sits alongside the project that both informs and measures our progress – action research & developmental evaluation.

27. Questions?

  • Join the conversation & access key resources
  • Continue the conversation started here today and access related resources on the CFCA website:
  • www.aifs.gov.au/cfca/news-discussion

This webinar was held on 30 November 2017.

Collective impact is a promising approach to addressing complex problems and has been enthusiastically adopted in Australia. The evidence to support use of the collective impact framework is still emerging, so it is largely uncertain how effective it is or how it is best implemented. However, there are some key aspects that are frequently identified by collective impact sites as presenting both opportunities and challenges. These include:

  • Leadership and governance 
  • Community engagement 
  • Data collection, continuous learning and evaluation

This webinar presented an overview of collective impact, drawing upon recent literature to consider these three issues. It included an examination of how collective impact is currently being implemented across Australia, and provided a case study of a collective impact project in Armadale, Western Australia.

Related resources

Further reading


Featured image: © iStock/davidf

About the presenters

Jessica Smart

Jessica Smart is a Senior Research Officer at AIFS working across multiple projects with a focus on evaluation capacity building. Her previous work has been in the not-for-profit sector, developing, delivering and evaluating community development and population health projects. Recent projects have been in the area of sexual health and respectful relationships with people with intellectual disability, with a focus on engaging community members as partners in research, program delivery and evaluation. Jessica recently completed a Master of Public Health at the University of Melbourne.

Kerry Graham

Kerry Graham is a Founder & Director of Collaboration for Impact – Australia’s leading organisation for enabling people to tackling big, tough problems and create large scale impact through collaboration. In this role, Kerry provides support to and curates learning for communities, corporations, governments, philanthropy and non-profit organisations on how to drive large-scale social change in intractable social challenges. Kerry also lectures on collaborative practice with Centre for Social Impact at University of New South Wales.  Prior to this Kerry held executive roles within national non-profit organisations and advised governments on social policy. She holds qualifications in public policy, law, social work and community management.

Arlene Hand

Arlene Hand has worked in the community services sector for over 20 years, working in both not-for-profit organisations and local government. She has a passion for working on community and sector development initiatives with her early career starting in Youth Work. Over the years, Arlene transitioned to community development and project management roles in early years, youth and mental health sectors. Arlene joined Communicare, Communities for Children in 2015 to initiate and guide the Community Dimensions collective impact project in the early years.

Questions and comments

Thank you for a very interesting webinar
Alan