What is evaluation?

Evaluation is the systematic process of assessing what you do and how you do it to arrive at a judgement about the 'worth, merit or value' of something (Mertens & Wilson, 2013; Scriven, 2003-04). It involves being methodical in your attempt to better understand a program or service.

What do we mean by children?

In this guide, the term 'children' is used to describe those under the age of 12 years. This is consistent with the definition used by the Australian Institute of Health and Welfare.

However, when considering the guidance below, you should also consider the child's developmental stage and capacities. Children of the same age can vary significantly in their abilities and maturity. Evaluators must take this into account in their planning and in practices such as establishing ethical procedures, choosing evaluation methods and communicating with children.

Because children's ability to participate in research evaluation depends on a range of factors, we tend not to refer to specific ages throughout this guide.

Reasons to involve children in evaluation

Family support services are increasingly evaluating their programs and practices to better understand how they work and how to improve them. For people working with children and/or delivering programs that seek to benefit children, involving children in evaluation with purpose and care can give you valuable insights and perspectives you might otherwise miss. You cannot fully understand how such programs work and/or are experienced without hearing directly from children.

Children also have a right to participate in evaluation under the United Nations Convention on the Rights of the Child (UNCRC). Article 12 of the UNCRC states that children have a right to be listened to when decisions will affect them, and that those views should be taken seriously and be responded to (United Nations, 1989).

However, evaluation practices that involve children are under-used in child and family services (Knight & Kingston, 2021), with many service providers defaulting to adult-reported data for evaluation. This happens partly because the cost, skills and time needed to develop child-appropriate and effective evaluation techniques and processes can be prohibitive (Kelly, 2017).

Unconscious and unchallenged beliefs about children, including that they are vulnerable and incapable, can also lead adults to exclude children from participating in evaluation (Clark & Moss, 2011; Lansdown, 2005). Such beliefs are at odds with the UNCRC as well as contemporary research, which has shown that children are competent social actors who can offer unique perspectives about their experiences, create knowledge and 'co-construct meaning' (Garcia-Quiroga & Agoglia, 2020; Greig, Taylor & MacKay, 2007; Lansdown, 2005; Naughton, MacNaughton, Rolfe, & Siraj-Blatchford, 2010). When evaluators use deliberate engagement strategies to involve children in evaluation, children can also develop new skills and build their confidence and agency (Graham & Fitzgerald, 2010).

Ways of involving children in evaluation

There are more ways to involve children in evaluation than simply collecting data from them about their experiences or views of programs and services. Other ways children can contribute to or be involved in evaluation include:

• co-designing or testing consent protocols
• testing survey instruments
• interpreting data
• collaborating on research reports
• sharing in the key evaluation results.

In the sections that follow, we offer ideas for how to engage children at different points within an evaluation.

Key considerations for involving children in evaluation

While there are many good reasons to involve children in evaluation, there is no single approach or tool that you can adopt to do it meaningfully in every situation. You can, however, try to ensure you conduct an engaging and inclusive evaluation for children by factoring in these three things:

• the children's ages and development
• the needs of children with additional support requirements
• ethical considerations for safe and respectful child involvement.

We discuss these points in greater detail below. A summary of common data collection methods is provided in later sections of this guide.

Age and development

In general, the age and developmental capabilities of the children you are working with will determine your evaluation questions, the methods you use and how you use the data. However, in this guide we have not always specified the age or exact age ranges at which a child is able to participate in a specific evaluation activity. This is because a child's capacity to participate is not solely dependent on their numeric age but also on their developmental stage and individual circumstances.

Young children

When actively supported by adults to express their views, young children can meaningfully participate in evaluation (Lundy, McEvoy, & Byrne, 2011). However, it can be challenging to use traditional evaluation methods to collect data from young children (e.g. those in the lower years of primary school and younger) that 'accurately and authentically' reflect their needs and experiences (Pascal & Bertram, 2009). Answering questions requires capabilities that are still developing in young children. Cognitive ability, language, thought and memory, and thinking beyond their immediate experience is difficult for younger children (Lundy et al., 2011). Young children (including those in early primary school) are also often likely to want to please researchers and agree with what they say. They may also have difficulty concentrating and be easily distracted (Greig et al., 2007; Khanum & Trivedi, 2012).

Young children can use non-verbal skills, such as pointing, to identify people, objects and places, and they can understand past and present in the context of their daily routines (Greig et al., 2007). Creative research methods, such as the mosaic approach (see the methods section below), have been successfully used to collect data from children in early childhood settings.

Middle years

The evidence suggests that children as young as 8 years old have the cognitive and language skills to complete clearly written surveys (including 5-point response scales) that use age-appropriate language (Rebok et al., 2001; Rubie-Davies & Hattie, 2012). Many children in this age group can also concentrate on assigned tasks (Coyle, Russell, Shields, & Tanaka, 2007).

Starting at age 10, children generally begin to experience more awareness of self and their social relationships, and they seek 'acceptance, admiration and respect' (Ballonoff, Ballard, Hoyt, & Ozer, 2021). This means that although various data collection methods are suitable for this age group, evaluation approaches that facilitate belonging, autonomy and social connection can be particularly effective. Because of this, the research and evaluation literature often suggest using youth-led participatory action research (YPAR).

This resource explains how to use YPAR in evaluation.

Supporting children with additional needs

Some children may experience more challenges or barriers to participation in evaluation than others. This can be especially true for children with child protection and/or domestic and family violence histories, children with disabilities, children from culturally and linguistically diverse backgrounds and Aboriginal and Torres Strait Islander children.

Some barriers to children's participation relate to their capacity, interest or confidence to engage with evaluation or research (and the people undertaking them). However, adults can also create barriers to children's participation by making assumptions about their abilities or best interests; for example, by deciding that they are too vulnerable to participate without first seeking the child's view (Garcia-Quiroga & Agoglia, 2020).

Evaluators and service staff can support and empower children to contribute their unique perspective and skills. Children may be more inclined to engage with evaluation if methods are participatory, creative and fun. Giving children different options for having input into evaluation can also encourage participation. Children with disabilities, for example, can have multiple impairments and varying communication needs and preferences, and you may need to try a combination of methods and techniques to gather feedback (Seed, 2016). The section on data collection methods offers some ideas for how to incorporate choice, flexibility and fun into evaluation projects.

Essential evaluator qualities for undertaking such work include having good relationship- and trust-building skills, knowledge of evaluation methods and an ability to support children to choose suitable methods (Garcia-Quiroga & Agoglia, 2020). You may need to seek advice from specialist organisations (e.g. disability support organisations) or seek additional training (e.g. trauma-informed practice) before beginning your evaluation (Ethical Research Involving Children, 2021; Morris, Hegarty, & Humphreys, 2012).

Cultural safety

Evaluators should take steps to establish and practice cultural safety when working with Aboriginal and Torres Strait Islander and/or culturally and linguistically diverse children. Cultural safety is:

the positive recognition and celebration of cultures. It is more than just the absence of racism or discrimination and more than 'cultural awareness' and 'cultural sensitivity'. It empowers people and enables them to contribute and feel safe to be themselves (Commonwealth of Australia, Department of the Prime Minister and Cabinet, 2021, p. 7).

More information about cultural safety can be found in the Further resources section of this guide.

Ethical issues

Although we have discussed the ethical risks attached to excluding children from evaluation, evaluation also needs to be conducted safely and ethically so that children are not harmed in the process of participating (Graham & Fitzgerald, 2010; Hill, 2005; Sherwood & Parsons, 2021). Involving children for the right reasons; that is, so they can meaningfully contribute to the evaluation findings and recommendations, is an essential first step. Other key ethical considerations are described below.

Consent and confidentiality

To involve children in evaluation, you will need to obtain consent from participating children (where they have the capacity to independently make decisions about their involvement in research and evaluation) and their parent/s or carer/s. As part of this process, evaluators should develop and uphold child-centric protocols for assessing a child's ability to agree to be in research, obtaining consent and assent to participate in the research and to inform participants about their rights to withdraw from the research. Some researchers have sought to ensure a child-centric perspective by co-developing consent protocols and materials with children (Sherwood & Parsons, 2021).

A child's capacity to consent or assent to be in an evaluation is not entirely related to their age, although this should be considered. Individual circumstances and maturity, which vary from child to child, should also be considered (National Health and Medical Research Council, the Australian Research Council, and Universities Australia, 2018). The National Statement on Ethical Conduct in Human Research provides some guidance on when to seek consent directly from a child and whether to involve one or both parents/carers in obtaining consent. If you are unsure, seek advice from other program/service staff and/or relevant legal advisors.

Even when a child is deemed not to have decision-making capacity, and parents/carers are required to give consent, the child should still be asked to 'assent' (i.e. agree) to the evaluation. In such cases, it is best to have a plan for resolving disagreements when a child has agreed to participate but a parent/carer has not given their consent.

Regardless of who provides consent (the child or a parent/carer), you should still discuss the evaluation purpose, process and potential outcomes in a developmentally appropriate way with all children involved in the evaluation. Parents/carers, generally, should also be involved in these discussions; however, there may be cases where it is not safe to do so (e.g. parents/carers who are neglectful or abusive). And in cases where there is no appropriate parent or carer to ask (e.g. children living in residential out-of-home care), refer to the person with legal responsibility for the child (Garcia-Quiroga & Agoglia, 2020).

Managing privacy, confidentiality and potential disclosures of child abuse are other important ethical considerations. This means having protocols to ensure confidential data collection and management, and preparing for any disclosures of abuse and neglect (Thomas & O'Kane, 1998).

Addressing power imbalances

In addition to developing appropriate formal ethical procedures for obtaining consent or assent, evaluators should be conscious of the power imbalances that exist between adults and children (Knight & Kingston, 2021). Children typically do not have any say about the content or form of questions asked in surveys or research interviews or how their answers will be interpreted and used (if at all). Additionally, younger children often display high levels of social desirability and may 'aim to please' with their responses (Lewis & Lindsay, 2000). School-aged children may think there are right and wrong answers to questions and provide answers based on what they think they should say instead of what they genuinely think.

It is important to be clear that there are no right or wrong answers and to take the time needed to build trusting relationships (Naughton et al., 2010; Solberg, cited in Thomas & O'Kane, 1998). Sharon Sparks, author of the Engaging Children's Voices in the Early Years practice guidelines (2020b) says:

Keeping communication open with children about the processes, and where things are up to, builds trust, demonstrates that their voice is valued, and reflects the principle of transparent engagement.

Through training, evaluators can be made aware of their power and authority, address their biases about children's capabilities, work towards recognising children's strengths and skills and work more collaboratively with children (Ballonoff et al., 2021). Training can also help to improve evaluators' awareness of ways they may be inadvertently leading or influencing children's responses (Greig et al., 2007; Lewis & Lindsay, 2000).

Conversely, you should seriously reconsider involving children if there is any chance that they will be disempowered through the evaluation process. For example, if there is a possibility that children's views will be ignored or excluded from the analysis or reporting, it may not be appropriate to involve them or you may need to rethink the evaluation approach and goals (Kingston, 2016).

Some children may need a trusted adult or peer present to help them feel safe and/or confidently express themselves during evaluation activities; however, be aware that having an adult or peer present can inhibit a child's responses and prevent them from fully sharing their views and opinions (Gardner & Randall, 2012). It is also worth reemphasising that if children are not ready or willing to participate in an evaluation activity, then their wishes must be respected.

Evaluators may need to take a trauma-informed approach depending on the child's individual circumstances and the sensitivity of the evaluation topic. A trauma-informed approach involves creating an environment where children feel safe to express themselves, building trust and allowing children to speak in their own time, and being open and transparent about the purpose of the evaluation and how the child's contribution will be used (Emerging Minds, 2023).

Payments and incentives

Another consideration is whether to pay or reward children for their participation. Some Australian research has found that children aged 10-14 years old were more likely to participate in research when paid; with higher amounts resulting in higher levels of participation (Moore et al., 2021). An alternative to offering children money is providing other rewards and incentives such as toys, a certificate of achievement or a surprise reward that children receive after they participate (Laws & Mann, 2004; Taplin, Chalmers, Hoban, McArthur, Moore, & Graham, 2019). However, any payment or incentive should be appropriate to the burden of participating and not induce the child (or their parent/carer) to take risks they would not ordinarily take.

Data collection methods

Once you have decided to involve children in your evaluation, you will need to choose a suitable method to collect your data. Which data collection method you choose will depend on the questions you want answered and the resources you have available, but there are some additional considerations when working with children.

As noted above, data collection methods and strategies need to be age and developmentally appropriate, engaging and, ideally, fun (Coyle et al., 2007; Larsson, Staland-Nyman, Svedberg, Nygren, & Carlsson, 2018). Using a range of methods and tools can allow children to draw on their strengths and preferences, be more inclusive of children with different abilities and make the evaluation process more enjoyable (Sherwood & Parsons, 2021; Thomas & O'Kane, 1998). Try to use language that is plain, simple and can be understood by participating children. For ideas on how to communicate to children in age and developmentally appropriate ways, go to the Further resources section at the end of this guide.

The environment in which data collection takes place should also be considered. Some children prefer a 'natural' setting (e.g. at home, school or in playgrounds) over somewhere more formal or clinical, such as a formal interview room (Greig et al., 2007; Knight & Kingston, 2021). While it can be more difficult to maintain privacy in shared spaces (Greig et al., 2007), you should be guided by what children tell you about their preferences.

Regardless of the chosen method, it should be 'meaningful, empowering and valuable for both children and service providers' (Knight & Kingston, 2021).

A selection of data collection methods that are appropriate to use with children are outlined below. This is not an exhaustive selection but these are methods commonly used by evaluators who work directly with children. Some of the described methods overlap and/or might be used in combination as part of a single evaluation. For more detailed information about the methods below, see the Further resources section of this guide.

Surveys

Children aged 8 years old and above typically have the cognitive and language abilities to reliably respond to surveys and questionnaires (Coyle et al., 2007; Rebok et al., 2001; Rubie-Davies & Hattie, 2012). However, caution is needed in constructing surveys for young children as they can be confused by negatively worded survey items and tend to choose more positive responses on Likert-scale measures (including measures that use smiley-face scales instead of numbers or words) (Hall, Hume, & Tazzyman, 2016; Rubie-Davies & Hattie, 2012). The attention span of the age group you are working with should also be considered when developing surveys; that is, it is best to avoid long surveys.

Using characters or pictures to break up surveys can make them more appealing to children, while embedding surveys within puzzles or games can improve engagement with the survey (Coyle et al., 2007; Hall et al., 2016). One example is the Rumbles Quest game that measures child wellbeing. Rumbles Quest is a computer game for children that is used to measure social and emotional wellbeing. For more traditional survey techniques, some research suggests that survey response scales with written statements (e.g. strongly agree to strongly disagree) are more effective than numeric scales (e.g. 1-5). Frequency-based responses (e.g. never to regularly or often) can be particularly easy for children to understand (Mellor & Moore, 2013).

Regardless of what kind of survey measures and formats you choose, testing your tool with a small number of children will increase the likelihood of it being engaging, developmentally appropriate and effective. Reading surveys aloud to children can also be a good testing approach.

Diaries

Diaries are an effective way to explore children and young peoples' use of time and to document their thoughts and experiences. However, it is possible that some children will not engage with this method because of the similarities it has with schoolwork and/or if they have limited literacy (Flanagan, Greenfield, Coad, & Neilson, 2015). Audio diaries enable children and young people to provide immediate and intimate conversations and reflections (Sargent, cited in Flanagan et al., 2015). Regardless of whether the diary has a written, audio or visual format, you will need processes to manage what you do with (and how you store) entries that include very personal and intimate information (Flanagan et al., 2015).

Interviews

There are differing views in the literature about when children are developmentally ready to meaningfully participate in interviews. Some research suggests that children as young as 3 can express their views and opinions (Ponizovsky-Bergelson, Dayan, Wahle, & Roer-Strier, 2019). Other research suggests that the skills needed to participate in interviews develop around age 6 (Coyle et al., 2007). If you are unsure about interviewing children as part of your evaluation, it is best to be cautious and only do so with children aged 6 and above, or seek advice from an experienced child practitioner.

When interviewing children, use words of encouragement, open-ended questions and prompting (Ponizovsky-Bergelson et al., 2019). Generally, interviews can be more engaging for children when they incorporate the use of pictures and images, drawings, dolls or hypothetical situations (Urbina-Garcia, 2019). Visual aids can also help to build rapport between evaluators and children (Flanagan et al., 2015).

When child needs and preferences are considered, interviews can provide opportunities to empower children and mitigate adult-child power imbalances. For example, children can choose when they will be interviewed, operate audio-recording devices, and use diagrams and/or pictorial cards to start and stop interviews (Naughton et al., 2010; Tisdall, Davis, & Gallagher, 2009). Evaluators can also use interviews to identify non-verbal cues about how children feel and what is important to them (Greig et al., 2007).

Be aware that some children will not be comfortable being interviewed by someone they do not know (Coyne, 1998) and may need a trusted adult present, such as a parent/carer or a known staff member. Also, interviews may not be appropriate where families are concerned about child protection reports or with children who have had negative experiences with authority or institutions (Tisdall et al., 2009).

Focus groups

Focus groups are great avenues for generating ideas, problem solving and providing unique perspectives about social experiences (Christian, Pearce, Robertson, & Rotherwell, 2010; Greig et al., 2007). Research suggests that most children aged 6 years and older have the skills to participate in focus groups and that children younger than 6 years engage better in pairs or small groups (Coyle et al., 2007; Greig et al., 2007). Children under 10 also generally have shorter attention spans than older children and may need frequent breaks and/or a change of activity (Christian et al., 2010; Greig et al., 2007; Tisdall et al., 2009).

Focus groups with children are more likely to be effective if they incorporate a range of enjoyable and challenging activities, along with incentives for participating, time for relaxation, and food and drinks (Christian et al., 2010; Thomas & O'Kane, 1998). Effective focus group activities may include drawing and writing (Harwood & Copfer, 2015; Wahle, Ponizovsky-Bergelson, Dayan, Erlichman, & Roer-Strier, 2017), puppets to stimulate discussion (Almqvist Almqvist, 2015; Katz and McLeigh, 2017) and video-led discussions (Colliver and Fleer, 2016). As with one-on-one interviews, researcher-evaluators can empower children by letting them prioritise discussion topics, choose warm-up activities and lead the room set-up (Christian et al., 2010; Tisdall et al., 2009).

Art-based methods

Art-based methods can be used alongside more traditional data collection methods and have established efficacy for obtaining data from children (Flanagan, 2015). They also tend to be more participatory and oriented towards self-directed, self-paced, self-expression (Driessnack & Furukawa, 2011).

Drawing and photography (including techniques such as Photovoice) are commonly used in research and evaluation interviews as prompts for discussion and ways of engaging participants (Abma, Breed & Schrijver, 2022; Guillemin & Drew, 2010). Other art-based methods include painting, writing, theatre, storytelling using software and digital tools, and mapmaking (Driessnack & Furukawa, 2011; Larsson, 2018).

Use of these methods is not usually about interpreting the art produced but about what children have to say about their art. As such, many researchers choose to interview children in connection with their artwork. For example, with drawing, you can ask children to describe their pictures through 'draw and write' explanations or audio-recordings (Driessnack & Furukawa, 2011). Whatever method you choose, having children interpret their own work can help avoid mis-representing their views and improve the reliability of your findings (Driessnack & Furukawa, 2011; Sparks, 2020b).

Mosaic approach

One example of a research and evaluation approach specifically tailored to children is the mosaic approach (Clark & Moss, 2011). This is a multi-method process that combines traditional research methods with more participatory processes. It uses a variety of 'visual, spatial and physical tools' (Clark & Moss, 2011) to understand children's perceptions and world views, including:

• photographs/video
• guided tours of the setting being studied
• mapping
• interviews
• observation
• drawing
• role playing (Greig et al., 2007; Knight & Kingston, 2021).

It is a flexible and inclusive approach suitable for children of all abilities including very young and non-verbal children (Clark & Moss, 2011; Knight & Kingston, 2021). Adult views are also sought as part of the process and triangulated with data collected from children.

The mosaic approach was originally designed to be used in the evaluation of an early childhood service (Clark & Moss, 2011). It has since been used for a variety of purposes including research and evaluation to enhance children's learning environments, increase children's engagement in program activities (Akyol & Erkah, 2018), explore children's perceptions of play (Husar, 2009) and research perceptions of youth mentoring (Evans-Locke & Hsu, 2020). This method is well-suited to evaluations seeking to explore children's experiences of a program and changes/improvements that are needed for children to thrive. It can also increase children's participation in decision making and give them more control over the evaluation process (Akyol & Erkah, 2018; Clark & Moss, 2011).

The main strengths and limitations of all the above data collection methods are provided in Table 1. A discussion about children's capacity to give consent and participate in evaluative activities is detailed earlier in this guide in the Ethics section.

Source: Adapted from Flanagan et al., 2015

Communicating evaluation findings

Sharing evaluation findings, as well as any proposed actions that result, with the children who participated is a key part of involving children in evaluation. One option for doing this is to create child-friendly evaluation reports that include visual elements such as art, photographs and handwritten messages. For example, the researchers using the mosaic approach detailed in the Clark and Moss (2011) article created a book that weaved the evaluation findings into a story complete with pictures, children's comments and photographs. The Brotherhood of St Laurence (see the case study below) developed a report for children who participated in their listening tour designed to gather feedback from children about the HIPPY program (Sparks, 2020a). The colourful report used simple language and included pictures of children, evaluation activities, artwork and graphics representing what children said.

Children can also be involved in report writing and dissemination activities, and evaluators can help children to co-produce sections of the final report (e.g. key recommendations) or seek their feedback on early drafts. To facilitate dialogue, you can use the kinds of participatory methods discussed in this guide. And when it comes to dissemination, consider inviting children to co-present the findings at public events or conferences.

Concluding remarks

Involving children in evaluation has many potential benefits for service providers, policy makers and communities. It can provide information and perspectives that are not obtainable by other means and can be essential for improving the efficiency and effectiveness of services used by children and families. It is also a critical part of upholding the rights of the child because it means that children are enabled and encouraged to participate in discussions and decisions that affect them.

However, there can be significant challenges in meaningfully engaging children in evaluation and the feasibility of doing so will depend on several factors. These include access to material resources and skilled staff, the children's ages and development, the children's circumstances (e.g. whether additional considerations and supports will be required for children who have experienced trauma, for children with disabilities, for children for whom English is a second language, etc.), the ability to gain meaningful consent or assent to participate and the evaluator's internalised beliefs about children's vulnerability and incapacity.

If you are considering involving children in a current or future evaluation, there are fun and engaging data collection methods you can use, with options for supporting younger and older children to participate. There are also steps you can take to ensure that children are empowered throughout the evaluation process.

Finally, children's involvement in evaluation does not need to be limited to data collection processes; children can help with the co-production of consent materials and protocols, evaluation planning, analysis, reporting and actioning any recommendations made from the evaluation (Sherwood & Parsons, 2021; Tisdall et al., 2009)

Further resources

• National Statement on Ethical Conduct in Human Research - National Health and Medical Research Council (NHMRC)
• Ethical guidelines for research with Aboriginal and Torres Strait Islander Peoples - NHMRC
• AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research - AIATSIS
• First Nations Cultural Safety Framework - Australian Evaluation Society (AES)
• Keeping our kids safe: Cultural safety and the National Principles for Child Safe Organisations - National Office for Child Safety, SNAIIC, VAACA
• Safety of children from culturally and linguistically diverse backgrounds - Commission for Children and Young People
• Participatory Action Research - AIFS practice guide
• A kit of tools for participatory research and evaluation with children, young people and adults - Save the Children Norway
• Language development: Children 0-8 years - Raising Children Network information about how to communicate with children at different age and developmental stages
• Communicating with children: Principles and practices to nurture, inspire, excite, educate and heal - United Nations Children's Fund resource on communicating with children at different age and developmental stages
• Engaging children's voices in the early years: Practice guidelines - Brotherhood of St Laurence
• Involving children and young people (8-12 years old) in evaluation toolkit - Participation Works
• Kids Central Toolkit - Australian Catholic University
• Top tips for children and young people's participation - Office of the Advocate for Children and Young People
• Participation: Count me in! - Guide to conducting social research with children and young people from NSW Commission for Children & Young People
• Understanding and supporting children and young people's participation - NSW Advocate for Children and Young People
• Ethical Research Involving Children library

References

Abma, T., Breed, M., Lips, S., & Schrijver, J. (2022). Whose voice is it really? Ethics of photovoice with children in health promotion. International Journal of Qualitative Methods, 21. doi.org/10.1177/16094069211072419

Akyol, T., & Erkan, S. (2018). The effects of mosaic based approach on involvement levels of children. International Journal of Education Methodology, 4(4), 303-309.

Almqvist, A., & Almqvist, L. (2015). Making oneself heard: Children's experiences of empowerment in Swedish preschools. Early Child Development and Care, 185(4), 578-593.

Ballonoff, S., Ballard, P. J., Hoyt, L. T., & Ozer, E. J. (2021). Applying a developmental lens to youth-led participatory action research: A critical examination and integration of existing evidence. Youth & Society, 53(1), 26-53.

Christian, B. J., Pearce, P. F., Robertson, A. J., & Rotherwell, E. (2010). It's a small, small world: Data collection strategies for research with children and adolescents. Journal of Pediatric Nursing, 25, 202-214.

Clark, A., & Moss, P. (2011). Listening to young children: The mosaic approach (Second edition). London: National Children's Bureau.

Coyle, K. K., Russell, L. A., Shields, J. P., & Tanaka, B. (2007). Summary report: Collecting data from Children aged 9-13. ETR Associates & The Preteen Alliance.

Coyne, I. T. (1998). Researching children: Some methodological and ethical considerations. Journal of Clinical Nursing, 7(5), 409-416.

Driessnack, M., & Furukawa, R. (2011). Arts-based data collection techniques used in child research. Journal for Specialists in Pediatric Nursing, 17, 3-9.

Emerging Minds. (2023). In focus: Trauma-informed care. Adelaide: Emerging Minds

Ethical Research Involving Children. (2021). Ensuring research involving children with disabilities is inclusive, empowering and safe: What are the critical issues? Ethical Research Involving Children. Retrieved from www.childethics.com/blog/the-3-essential-principles-of-research-involving-children-with-disability

Evans-Locke, K., & Hsu, C.-I. (2020) Using participatory methods to evaluate the impacts of an early intervention programme on children and young people (CYP). Evaluation Journal of Australasia, 20(3), 176-188.

Flanagan S. M., Greenfield S., Coad J., & Neilson, S. (2015). An exploration of the data collection methods utilised with children, teenagers and young people. BMC Research Notes.

Garcia-Quiroga, M., & Agoglia, I. S. (2020). Too vulnerable to participate? Challenges for meaningful participation in research with children in alternative care and adoption. International Journal of Qualitative Methods, 19.

Gardner, H., & Randall, D. (2012). The effects of the presence or absence of parents on interviews with children. Nurse Researcher, 19(2), 6-10. doi: 10.7748/nr2012.01.19.2.6.c8902. PMID: 22338802.

Graham, A., & Fitzgerald, R. (2010). Children's participation in research: Some possibilities and constraints in the current Australian research environment. Journal of Sociology, 46, 133-147.

Greig, A., Taylor., J., & MacKay, T. (2007). Doing research with children (Second edition). London: Sage Publications Ltd.

Guillemin, M., & Drew, S. (2010). Questions of process in participant-generated visual methodologies. Visual Studies, 25(2), 175-188 doi:10.1080/1472586X.2010.502676

Hall, L., Hume, C., & Tazzyman, S. (2016). Five degrees of happiness: Effective smiley face likert scales for evaluating with children. Proceedings of the 15th International Conference on Interaction Design and Children, 311-321.

Harwood, D., & Copfer, S. (2015). 'Your Lunch Pail Is Silly!' Children's and Teachers' Views on Teasing. Journal of Research in Childhood Education, 29(1), 26-41.

Hill, M. (2005). Ethical considerations in researching children's experiences. In Greene, S., & Hogan, D. (Eds.), Researching children's experience: Approaches and methods (pp. 61-83). London: Sage Publications Ltd.

Kelly, L. (2017). Ethics and evaluative consultations with children in small to mid-sized Australian non-government organisations. Evaluation Journal of Australasia, 17(1), 4-11.

Khanum, A., & Trivedi, M. (2012). Take care: A study on usability evaluation methods for children. doi.org/10.48550/arXiv.1212.0647

Kingston K. (2016). Development of a Child Evaluation Framework for early childhood services using deliberative democratic evaluation and the mosaic approach. Evaluation Journal of Australasia, 16(3), 25-34.

Knight, R. L., & Kingston, K. L. (2021). Valuing beneficiary voice: Involving children living in out-of-home care in programme evaluation. Evaluation Journal of Australasia, 21(2), 69-84.

Lansdown, G. (2005). Can you hear me? The right of young children to participate in decisions affecting them (Working Paper 36). The Hague: Bernard van Leer Foundation.

Larsson, I., Staland-Nyman, C., Svedberg, P., Nygren, J. M., & Carlsson, I. (2018). Children and young people's participation in developing interventions in health and well-being: A scoping review. BMC Health Services Research. doi.org/10.1186/s12913-018-3219-2

Laws, S., & Mann, G. (2004). So you want to involve children in research? A toolkit to supporting children' meaningful and ethical participation in research relating to violence against children. Stockholm: Save the Children.

Lewis, A., & Lindsay, G. (2000). Researching children's perspectives. Buckingham: Open University Press.

Lundy, L., McEvoy, L., & Byrne, B. (2011), Working with young children as co-researchers: An approach informed by the United Nations Convention on the Rights of the Child. Early Education & Development, 22(5), 714-736.

Mellor, D., & Moore, K. (2013). The use of likert scales with children. Journal of Pediatric Psychology, 39.

Mertens, D., & Wilson, A. (2013). Program evaluation theory and practice: A comprehensive guide. London: Guilford Press.

Moore, T., McArthur, M., Graham, A., Chalmers., Powell, M. A., & Taplin, S. (2021). Children and young people's decision-making in social research about sensitive issues. Children's Geographies, 19(6), 689-704, doi:10.1080/14733285.2020.1866749

Morris, A., Hegarty, K., & Humphreys, C. (2012). Ethical and safe: Research with children about domestic violence. Research Ethics, 8(2), 125-139. doi.org/10.1177/1747016112445420

National Health and Medical Research Council, the Australian Research Council and Universities Australia. (2018). National statement on ethical conduct in human research (Updated edition). Canberra: National Health and Medical Research Council.

Naughton, G., MacNaughton, G., Rolfe, S., & Siraj-Blatchford, I. (2010). Doing early childhood research: International perspectives on theory & practice. Berkshire: Open University Press.

Pascal, C., & Bertram, T. (2009). Listening to young citizens: The struggle to make real a participatory paradigm in research with young children. European Early Childhood Education Research Journal, 17(2), 249-262.

Ponizovsky-Bergelson, Y., Dayan, Y., Wahle, N., & Roer-Strier, D. (2019). A qualitative interview with young children: What encourages or inhibits young children's participation. International Journal of Qualitative Methods, 18.

Rebok, G., Riley, A., Forrest, C., Starfield, B., Green, J., Robertson, J. et al. (2001). Elementary school-aged children's reports of their health: A cognitive interviewing study. Quality of Life Research, 10, 59-70.

Rubie-Davies, C., & Hattie, J. (2012). The dangers of extreme positive responses in Likert scales administered to young children. The International Journal of Educational and Psychological Assessment, 11, 75-89.

Scriven, M. (2003-04). Michael Scriven on the differences between evaluation and social science research. The Evaluation Exchange, IX(4). Retrieved from www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/reflecting-on-the-past-and-future-of-evaluation/michael-scriven-on-the-differences-between-evaluation-and-social-science-research

Seed, N. (2016). Photovoice: A participatory approach to disability service evaluation. Evaluation Journal of Australasia, 16(2), 29-35.

Sherwood, G., & Parsons, S. (2021). Negotiating the practicalities of informed consent in the field with children and young people: learning from social science researchers. Research Ethics, 17(4), 448-463.

Sparks, S. (2020a). Children's voices listening tour 2019. Melbourne: Brotherhood of St Laurence.

Sparks, S. (2020b). Engaging children's voices in the early years: Practice guidelines. Melbourne: Brotherhood of St Laurence.

Taplin, S., Chalmers, J., Hoban, B., McArthur, M., Moore, T., & Graham, A. (2019). Children in social research: Do higher payments encourage participation in riskier studies? Journal of Empirical Research on Human Research Ethics, 14(2),126-140. doi:10.1177/1556264619826796

Thomas, N., & O'Kane, C. (1998). The ethics of participatory research with children. Children and Society, 12, 336-348.

Tisdall, K., Davis, J., & Gallagher, M. (2009). Researching with children and young people: Research design, methods and analysis. London: Sage Publications Ltd.

United Nations. (1989). United Nations Convention on the Rights of the Child. Geneva: United Nations.

Urbina-García, M. (2019). Methodological strategies to listen to children's voices: A systematic critical review. Revista Colombiana de Educación, 77, 61-85. doi:10.17227/rce.num77-9650

Wahle, N., Ponizovsky-Bergelson, Y., Dayan, Y., Erlichman, O., & Roer-Strier, D. (2017). On the margins of race, immigration and war: Perspectives on risk and protection of young children from the Ethiopian community in Israel. European Early Childhood Education Research Journal, 25, 305-320.