By using evaluation to assess what we do we can make better decisions and improve services for our clients. Evaluation works best when the whole organisation embraces it, the organisation acts on evaluation findings, and there is shared responsibility for evaluation. This can be called having an 'evaluation culture', and managers are essential to building a positive evaluation culture.

In this resource we share some tips and key considerations for managers who want to help build an organisational culture where evidence and evaluation are valued and can inform and improve practice. Although staff at any level of an organisation can be involved in evaluation, and be evaluation champions, managers often have a special role in supporting and facilitating a positive evaluation culture.

This resource is predominately for managers but may also be useful for others in an organisation who wish to promote an evaluation culture.

We have drawn on a recent research project to show what works for managers to effectively engage in building such an organisational culture.

The information presented in this resource draws from a recent research project undertaken at the Australian Institute of Family Studies (AIFS). That project investigated the ways in which managers and other types of evaluation champions can encourage the use of evaluation findings (both at an executive and service level), and the successful approaches and processes that build a positive evaluation culture.

Why managers and the organisational environment are important

Organisations that successfully use evaluation create a culture where evidence and evaluation are seen as important and staff at all levels are comfortable using evidence to improve their practice.

The reason we come to work today is to make people's lives better. We want to be confident we're doing that. We want to be confident we're doing that to the best we know today, and that if we're using evidence, we can keep improving not only what we're doing today but tomorrow. So I think having really strong communication about why we're doing evaluation is really, really important. (Program Manager)

Building this kind of organisational culture usually can't be achieved without some behaviour change and professional development. This often means senior leadership needs to set the tone and help to create the right environment. Less senior evaluation champions, such as evaluation officers and frontline practitioners, can also help to create a positive evaluation culture but these staff are most effective when they have managerial support.

Many of the managers and evaluation champions we spoke to referred to the importance of the 'authorising environment' in building an organisational culture where research evidence and evaluation were seen as valuable and a priority for the organisation. Without the right authorising environment, and particularly without the support of key decision makers in an organisation, it can be hard to achieve change. However, the right authorising environment can help build a positive evaluation culture by making available the necessary resources (e.g. time, funding and evaluation infrastructure) and encouraging staff to see their role in creating and sharing knowledge.

As an agency we're supporting staff to implement the things they learn about evaluation. So it's about us having structures that encourage, and also making sure that staff see the outcomes of it and not just that it goes off into the distance and people think it's been great, but there's no way of genuinely knowing whether it's a better or worse way of doing things. (Executive Manager)

What works in leading evaluation in your organisation

We talked to managers in the families and children services sector to find out more about what they have found most helpful or effective in building positive evaluation culture within their organisations. They helped us to understand 'what works' when leading evaluation in an organisation; that is, how to encourage the use of evidence and evaluation data, and how managers can support staff undertaking evaluation activities.

What works:

• high level endorsement
• a clear strategy for evaluation
• investment in building workforce capacity.

High level endorsement

Senior managers' high level endorsement of evaluation can help create an environment within an organisation where staff feel encouraged and capable of undertaking evaluation activities and acting on the data. This sets the tone for how staff perceive evaluation - both in terms of their attitude towards conducting evaluation work and their understanding of how evaluation can be used. It can also assist in embedding evaluation in your organisational processes.

So we're really lucky at our organisation to have a CEO who is very data positive and who demands high levels of evaluation activity, throughout the tender process to reporting, to everyday management of the program. So that's coming from the top, which is really helpful and enabling when you're trying to do activities at that kind of more team or individual level. (Evaluation Officer)

What works

Effectively endorse evaluation by demonstrating the value, purpose and usefulness of evaluation to the organisation as a whole, and not just as a way of meeting funding requirements.

For example, by communicating the usefulness of evaluation, you can demonstrate to staff how evaluation findings contribute to:

• the development of program resources
• enhanced understandings of client needs and experiences
• advocacy and providing clients with a 'voice'
• building the knowledge base.

Foster a culture of continual learning and improvement that empowers staff to undertake routine evaluation activities themselves and to see the value of this to their work and personal development.

It is important to identify your staff's strengths and provide positive reinforcement. Finding simple but effective ways to show your appreciation to staff can build a positive culture by helping them see the results of their work and how it matters. One example from our research came from an organisation where the evaluation team put the results of an annual survey of a particular program into a small book for staff as a means of recognising and valuing their work.

A clear strategy

Having a clear strategy for how your organisation will use evidence and carry out evaluations can:

• contribute to more effective and streamlined evaluations
• promote more evidence-based decision making in staff
• ensure that any evaluation and research is aligned with organisational values.

A clear strategic approach may include developing a whole-of-organisation strategy that embeds evaluation and research activities in organisational policies, frameworks and processes.

A clear strategic approach also helps set the tone for how staff perceive evaluation and research. It can influence the attitudes and motivation of staff because they can see how the collected data will be used to improve practice (beyond just being an administrative requirement). Embedding evaluation in organisational processes can also assist in evaluation becoming normal practice.

Of course everyone always knows you have to report back to funders but it's like a cultural shift now. It's not just about reporting back to funders, it's about informing us at the organisation, evaluating what we're doing, improving, sharing our knowledge. We want to move to that. (Program Manager)

What works

Embedding evaluation into work systems, processes and policies to make it part of regular practice is an important leadership strategy.

This can include:

• embedding program logics into broader organisational planning
• embedding evaluation into whole-of-organisation frameworks and policies
• streamlining monitoring and information management systems to support evaluation activities.

Provide clear oversight of all scheduled evaluations within your organisation (regardless of funding source) to help with managing organisational resources and maintaining accountability to funding bodies.

A useful strategy is committing to an overarching evaluation and research framework for the entire organisation. You can develop this through internal processes within your organisation and it doesn't necessarily require a lot of resources. It generally includes who has oversight and responsibility for research and evaluation activities, how the research evidence will be used, and a schedule of all evaluation activities within the organisation. This helps to keep everyone on track with individual program evaluations and ensures that the staff involved in an evaluation have the capacity to undertake all of the activities it involves.

By sticking to the whole-of-organisation strategy, you can more effectively champion evaluation and have better control over all evaluations within an organisation. This can help in managing evaluation budgets and ensures that evaluations of different programs or services contribute to organisational goals and learning. With this approach your organisation can also keep track of individual evaluations of programs tied to discrete funding sources, and you can also share the learnings from program-based evaluation across your organisation.

Promote the use of evaluation for continuous monitoring, learning and improvement - not just for evaluating the impact of service.

You can adjust client intake and assessment tools so that they work as pre-service measures or adjust clients' final sessions to include post-measures. For example, develop short questionnaires for clients to fill out before and after receiving the program or service. You could include aligning intake and exit tools with the Data Exchange System (DEX) categories in order to streamline evaluation processes for Commonwealth Government funded activities.

One of the things that I think that everyone is missing and what inspires staff culture is that services need to see themselves as evidence builders. So our staff are really kind of inspired, they see data, they use data, they are part of making data and evidence. And so they see how that evidence and every level of that evidence is used, say in policy submissions, to advocate around the need for specific client groups. (Chief Executive Officer)

Increase buy-in and understanding of evaluation among staff by involving them in co-planning processes.

For example, have staff contribute to the design of program logic models and data collection processes, and include them in discussions on setting strategic goals and whole-of-organisation strategies.

Investment in building workforce capacity

Committing to an investment in building workforce capacity is one of the most important ways an organisation can create a positive evaluation culture and make evaluation genuinely useful to an organisation. One way to do this is by providing staff with access to technical support and expertise as well as giving staff opportunities for hands-on learning. Another is to make sure your organisation internally shares information, resources and research in order to develop evaluation capacity. Building workforce capacity may also involve investing in appropriate monitoring and information management systems.

What works

I think you have to kind of understand different ways that people can engage with data. So not everyone's going to have like a thrill of joy when they hear the words 'statistically significant'. Like it doesn't mean anything. (Evaluation Officer)

Recognise the different levels of skill and evaluation capacity within an organisation and adjust expectations to appropriately support the learning needs of staff.

Use different strategies for staff at different levels of the organisation. Organisations with internal research and evaluation teams can run training at a basic level for staff with little or no previous experiences of evaluation, and more advanced training for staff with more experience. A good launchpad for staff with no prior knowledge of evaluation includes training in the 'why' and 'how' of developing program documentation (e.g. program logic models) and monitoring activities (e.g. routine collection of measures). Introducing staff to evaluation in empowering ways will help to demystify evaluation and research activities and make the purpose of those activities clear.

[Evaluation] is not just a one-step, two-step process - we can't remove the practitioners from it and we can't remove the participants from it. It's all about them at the end of the day and the families that we work with. It's about managers taking the staff on the learning journey and explaining why the research and their contribution is important and how this information is valuable. (Initiatives Manager)

Staff development may require both internal and external supports.

Internal actions to build workforce capacity can include ensuring that staff have access to appropriate resources such as time, funding, training or technical support as well as encouragement to undertake continual learning. External supports can also be important and include the types of training and learning opportunities that might not be available within an organisation. This may include tailored and general support from external consultants or other services such as the AIFS team.

As a way of developing support for staff and investing in their evaluation capacity, managers can engage in relationship building with external stakeholders (funding bodies and others). This includes building partnerships with universities and evaluation networks and employing contractors when required.

Resources and further reading