What is ethical conduct?

Ethical conduct in an evaluation means that the evaluation is designed to respect the rights of participants and ensure any potential for harm is minimised. In particular, the evaluator needs to consider whether participants will be exposed to any physical or psychological risk, burden, inconvenience or breach of their privacy.

Professional guidelines

The National Health and Medical Research Council (NHMRC) sets out guiding principles for Australian researchers undertaking human research or evaluation studies. It also has guidelines for ethical behaviour in quality assurance and evaluation activities.

Some societies and professions have their own ethical guidelines, such as the Australian Evaluation Society's guidelines for ethical conduct in evaluation.

Important ethical considerations

This section outlines some of the key considerations when planning or conducting an evaluation. In some instances, a formal ethical review of the evaluation will be required. Even if a review is not necessary, these issues should still be considered to ensure the evaluation is conducted ethically.

Is the evaluation worth doing?

Taking part in an evaluation can place a burden on participants and potentially cause them harm or discomfort. Any likely benefit of the evaluation must therefore justify these risks to participants. The benefit could be to the participants (e.g. improving the outcomes for participants in an ongoing program that they are part of), the wider community (e.g. to improve the outcomes for a future program) or both. These benefits should be clearly specified.

To ensure an evaluation is worth doing, it must also have a clear purpose and be carefully designed so that it answers the identified evaluation questions. Good design includes choosing the most appropriate data collection methods and making sure your sample is representative of program participants, to reduce any bias in the findings.

Will the evaluation cause harm or distress?

If the evaluation involves asking participants about sensitive topics, it could be distressing or prompt unpleasant memories for some. The risk of this happening must be considered, as well as what actions should be taken if someone becomes distressed. This could involve stopping the interview or referring the person to an appropriate support service. Documented procedures for avoiding harm or distress, or what to do when a participant becomes distressed, should be in place and understood by everyone involved in implementing the evaluation.

Are participants voluntarily taking part and do they understand the risks involved?

All participants should be given the information they need to understand what will happen during the evaluation and how their data will be used. This information should be provided in an accessible, plain language format. Participants must be allowed sufficient time to consider the information and to raise any queries before deciding whether or not to participate. The information should include:

• the purpose of the evaluation
• the nature and duration of their involvement
• how data will be collected, used, stored and who will have access to it
• the measures in place to safeguard their anonymity, confidentiality and privacy.

An important aspect of participant consent is the absence of any pressure to take part. It is important that participants know that their participation is voluntary, that there will be no consequences to them if they choose not to take part, and that they are free to withdraw from the evaluation at any time once they have started.

For research that collects data through surveys and interviews, examples of the key points to include in participant information can be found here:

• Example survey introduction [Word, 22.8 KB]
• Example interview participant information sheet [Word, 27.21 KB].

Consent to participate in an evaluation should be recorded in some way. This can take the form of written or verbal consent. Written consent can be via email or the completion of a formal consent form. NHMRC templates to guide the development of written participant information and consent forms can be downloaded from this page [Ethical issues and resources | NHMRC].

Verbal consent can be obtained by asking if the person is willing to proceed with an interview and then making an audio recording of their response or recording their consent on paper (such as in a verbal consent form). Completion of an anonymous paper survey or an online survey is often considered to be a form of consent but the survey introduction should include information for participants to help them decide if they wish to complete the survey; for example, why the survey is being conducted and how the participant's information will be used. Information and consent can be made more explicit in online surveys by asking potential participants to read information about the project before clicking 'next' if they agree to participate in the survey. When surveying someone in person, formal consent (e.g. on a consent form) is usually necessary.

Seeking consent from children and young people requires particular attention. Further information about ethical considerations in research and evaluation with children and young people can be found in this article.

If the evaluation requires the participation of target populations that are difficult to recruit, payment or incentives may be offered to encourage participation, as long as adequate processes are in place to promote valid consent. Payment can take the form of reimbursement of costs to participants, such as travel, parking or their time. Participants could also be offered non-monetary incentives such as vouchers, lottery tickets or participation in prize draws.

Payment or incentives for participants should be equitable, proportionate to the burden placed on them and not encourage participants to accept a risk or burden that is more than they would have otherwise accepted. The degree to which payment or incentives could result in pressure on individuals to consent to participate should be considered. This is especially important when the evaluation involves more than a low risk of harm.

Further information can be found in the NHMRC's guidelines on payment of participants in research [Payment of participants in research: information for researchers, HRECs and other ethics review bodies | NHMRC]

Do participants feel comfortable giving honest feedback?

Program participants should be able to provide feedback without feeling pressure to provide positive responses. Ideally, feedback should be anonymous and qualitative data should be collected by someone who is not involved with running the program being evaluated.

Do all program participants have an opportunity to participate in the evaluation?

Program participants should have an opportunity to contribute to the evaluation. Even if you are only planning to speak to or survey a sample of program participants, as much as is possible, this sample should be representative of all participants (or of the range of participant types). Consider and address any language and literacy barriers to participation.

Is participant information protected?

Always respect participants' rights to confidentiality and privacy. This can be done by removing or not collecting information that identifies individuals. You should consider whether an individual could be identified from the information collected. Even without names and addresses, individuals in some settings may still be identifiable. For example, in a small group, characteristics such as age or country of birth could identify someone. If this is the case, you need to be clear with participants that it is possible that someone who knows them might be able to identify them even if their personal details are removed from any reporting on the evaluation.

Your organisation should treat employee and client data in accordance with relevant privacy legislation. Evaluation data should be treated in the same way. Details of participants should be stored securely and only the designated evaluators should have access to this information. For example, paper evaluation records could be stored in a locked filing cabinet and digital files stored on a password-protected computer or server. When the information is no longer required it should be properly discarded; for example, paper containing confidential information or data that could be used to identify someone should be shredded.

Information from an evaluation that has been approved by a formal ethical review process may need to be stored for a specific period of time (e.g. to allow people to check what data are held on them).

How will disclosure of serious wrongdoing be handled?

The evaluation team need to understand their legal obligations and develop protocols to respond to a disclosure of potential criminal activity or other serious harm or wrongdoing (e.g. allegations of child sexual abuse).

Do participants know who to contact if they have any concerns?

Participant information should include the contact details of the evaluation lead, so participants can report any concerns they have about the evaluation.

Is reporting accurate and honest?

Ensure that all evaluation findings, including positive, negative and unexpected findings, are reported accurately and honestly. Any limitations to the evaluation should also be reported; for example, noting that only a small proportion of clients participated in the evaluation or that there was limited participation by some client groups.

Human Research Ethics Committees

This section outlines how Human Research Ethics Committees (HRECs) operate and when they might be needed.

What is a Human Research Ethics Committee?

The role of HRECs is to review research and evaluation proposals and ensure they meet the requirements of the National Statement on Ethical Conduct in Human Research (National Health and Medical Research Council, the Australian Research Council and Universities Australia, 2018). HRECs are comprised of at least eight people with different backgrounds, as specified in the National Statement.

HRECs have different application processes but generally researchers are required to provide details of the project proposal and to demonstrate how it adheres to ethical principles. The committee is responsible for evaluating the proposal in terms of ethical principles and may ask the researchers to amend the project to meet requirements.

Not all organisations conducting research need to establish and maintain their own HREC. If your organisation does not have a HREC, it may be possible to submit an evaluation for ethical approval to an external HREC. Some HRECs charge a fee for the ethical review process. The NHMRC has published a list of accredited HRECS. Another option for obtaining formal ethical approval is to partner with evaluators from an organisation, such as a university, that has access to a HREC.

When is a formal ethical review needed?

Evaluation activities undertaken to monitor or improve the quality of programs often involve minimal risk to participants and may not require a formal review by a HREC. However, oversight of the evaluation and an assessment of burden or risk to participants is still necessary. This section outlines the issues to consider when deciding if a formal ethical review is necessary.

A formal ethical review is often not considered necessary in situations where the data are:

• already being collected by the organisation in the conduct of their usual work
• collected from program participants expressly for the purposes of monitoring or improving that program
• anonymous; that is, individuals cannot be identified.

A formal ethical review should be considered when:

• the activity potentially infringes on the privacy of participants
• data are used for a purpose other than what was originally intended; that is, participants have not given consent for their data to be used in this way. For example, if measures of client mental health collected at different time points to guide treatment are used for evaluation purposes without specific consent for the data to be used for this purpose.
• information is gathered from participants beyond what is collected routinely
• non-standard or innovative protocols are being tested
• cohorts are compared, which could mean that there is potential for individuals to be identified
• control groups are used, which could result in people who are in need of a service being denied access because they have been allocated to a control group
• there is participation from people from minority groups, vulnerable groups or people who have historically been subjects of research with little control of the outcomes. This can include, for example, children, refugees, Aboriginal and Torres Strait Islander people or those with an intellectual disability or mental illness. Additional considerations for vulnerable participants include choosing an appropriate method of consent and balancing the risks and benefits of participation.
• there is targeted analysis of data involving minority groups, vulnerable groups or Aboriginal and Torres Strait Islander people and these data are separated out of the main data collected for the evaluation.

Conclusion

Considering ethical issues is an integral part of evaluation planning and there should be a shared understanding of ethical principles among those involved in implementing the evaluation. Evaluation planning should include an assessment of the burden or risk to participants and, if necessary, submission of the evaluation proposal to a HREC for formal ethical review. Even if a formal ethical review is not necessary, clear information should be provided to evaluation participants, and data collection, usage and storage protocols should ensure that participant confidentiality and privacy are respected.

Resources

• Block, K., Warr, D., Gibbs, L., & Riggs E. (2012). Addressing ethical and methodological challenges in research with refugee-background young people: Reflections from the field. Journal of Refugee Studies, 26(1) 69-87.
• Lowitja Institute. Ethics Hub. Melbourne: Lowitja Institute.
• National Health and Medical Research Council, Australian Research Council, & Universities Australia. (2019). Payment of participants in research: Information for researchers, HRECs and other ethics review bodies. Canberra: National Health and Medical Research Council.
• National Health and Medical Research Council, the Australian Research Council and Universities Australia. (2018). National statement on ethical conduct in human research. Canberra: National Health and Medical Research Council (Updated edition). 
• National Health and Medical Research Council. (2014). Ethical considerations in quality assurance and evaluation activities. Canberra: The National Health and Medical Research Council.