 to pay my respects to the traditional custodians of the land I'm joining from, the Wurundjeri people of the Kulin nations, and pay my respect to elders past and present. I think it's particularly important in this context to acknowledge that we're joining from stolen land. Many of the formal ethics processes that I'm going to talk to you about today have their origins as a response to racist, exploitative research across the world. And I think it's really important to acknowledge that the potential for that power dynamic to continue to exist in research processes and to always be really mindful of that in the way that you approach ethical research. And also recognize the many, in Australia, the many Aboriginal researchers who are leading the way in terms of this thinking and particularly around things like data sovereignty and access to data and really thinking about how to do research in an ethical way. So there's immense wisdom that we can draw on there as well. So I'm going to talk about a number of things today. So hopefully answering some of the frequently asked questions that you might have about ethics processes. So things like when do you need ethics processes? How do you interpret the national statement and how do you work with human research ethics committees or HRX that I'll keep referring to? Hopefully you all at least vaguely know what those are, even if you've not had to deal with one directly. I'm famously terrible at sticking to time, but I'm really hopeful that we'll leave some time at the end for discussion. So if you have questions as we go through, feel free to pop them into the chat. Or if you're not a chat person at the end, we can just come off mute and have a chat about your questions. I guess a caveat as we go through this presentation today is that a lot of this comes from my own practice and I'm trying to navigate my way through these various processes and I'm hoping to try and give you some of the tips and tricks that I've learned along the way. I've had to engage with a number of different ethics committees across a number of different contexts in my practice and I find the process so frustrating that it's led me to do a PhD specifically on the topic. So I'm still relatively early days in that PhD and the scale of things. So I don't yet have all of the answers or even any of the answers, but I'm hoping to just share some of the experiences that I've had with you all. And in that spirit as well, I'm sure there are people in this room here who also have lots of experience of navigating these processes and maybe even having sat on hrex. And when it comes to the discussion, I really encourage you to share your experiences as well. And also to challenge me as I as I go through and go actually my experience is a bit different from that. So yeah, hopefully we can make this a really learning discussion at the end. So first of all, I'm going to introduce you to this document because I'm going to refer to it an awful lot as we go through this presentation today. I'm presuming that the most of the people who've dialed into this call are based in Australia. Apologies if you are based in New Zealand or further afield. I don't know your system and your infrastructure in regards to these things. So some of this advice might not be the most applicable to you. But in terms of working in an Australian context, it might have passed some of you by that the national statement on ethical conduct in human research, which is kind of the guide to formal ethics processes has just been reviewed and re released. So it's a new addition. Many of the sections completely rewritten was published in the middle of last year. And it's now effective from the 1st of January 2024. So when you're responding to ethics applications, this is the document that you should be referencing rather than the 2007 2018 version. But it's also important to note that some sections of this are still being updated. In particular, there was a public consultation on section four, which is the section that specifically talks about kind of particular quote unquote vulnerable populations that you might do research with. And also some of the framing at the start of the document was also part of that consultation. So what I guess I would just say keep your eyes peeled because some of that stuff is likely to change on the basis of that consultation. And I think in general, the sense I got from the public consultation document was taking a slightly less paternalistic approach to some of those populations. And we'll talk about that a little bit more as we go on. So the first thing I wanted to talk to you about was when do you need ethics processes? Because this is far and away the biggest question that comes up when I'm talking to clients about potential projects. They go, do we need ethics? When I'm talking to co-researchers, do we need ethics? And the short answer is it's very confusing about when you need ethics. But I'm hoping to provide a bit more detail that sits behind that. So at least you can understand where the confusion lies. So as I said before, the national statement is a bit of a Bible for these things. It's intended as it says there for use by any researcher conducting research with human participants and then all the people who kind of make decisions about research governance. Specifically, the national statement in terms of guiding people and requiring people to go through Human Research Ethic Committee review applies to research that's being done under the auspices of any of the bodies that develop the national statement. So they're listed there, the NHMRC, the Australian Research Council and Universities Australia. So essentially, if you're doing research with or in partnership with a university, aside from the caveats that we're going to talk about, you do have to go through a Human Research Ethics Committee formal approval process. If you're outside the bounds of those organizations, which many people working in consulting, working in government departments, working in not-for-profits sit in that gray space, that's where it starts to get a bit confusing about whether you actually need to go through formal A-track approval. So if you're not one of those bodies, the first thing to say is that the national statement still sets the standards for what the research is. So you should still be doing whatever you do in alignment with the things that are in the national statement, even if you don't go through a formal process. And the institutions that do research are responsible for the ethical review of that research and the processes that they need to go through in order to make sure that that happens. I can see somebody's put in the chat is evaluation considered as research and you're going to be happy to know. I have a whole slide on that coming up because that is absolutely one of the topics that comes up a lot. Before we get to that, one of the things I wanted to flag in terms of what needs reviewed and when does it need formal approval is the idea of risk. If you're super familiar with the 2018 national statement, you might see that this is one of the things that has changed with the 2023 national statement. It used to be that there were three categories of risk, which were negligible risk, low risk and greater than low risk. The new national statement instead has this kind of broad to categories of risk, which is basically lower risk and higher risk. And research risk exists on a continuum within that. And it says on the slide there, chapter two of the national statement provides a lot more detail about what fits into those categories. But the other thing that I would say is they're absolutely not definitive. So it's not like you can look at a piece of research and go tick, tick, tick, this is higher risk or tick, tick, tick, this is lower risk. It requires an element of judgment based on the kind of categories and the descriptions that exist within that section of the national statement. In terms of what risk and harm might involve, there's a number of different ways that harm is conceptualized. And it's also in the new statement conceptualized as different from burden and discomfort. So as it says in that sliding skill there, people might experience burden from taking part in research. So it might be, you know, they have to take time out of their day to complete a survey, but that doesn't inherently constitute a harm or a risk that makes it higher risk. It might be other things that are linked to what a harm might be such as psychological harm or physical harm. Obviously, that's probably less likely in the kind of work that we do, cultural harm, social harm, legal harm. So for example, in some of the work that I'm doing at the moment, which is working with people who are in prison, there's a risk of legal harm that they disclose something during an interview that the interview transcript could get subpoenaed and then have legal consequences from that. So I'm going to talk a little bit more about how you do like a statement of ethical principles as well, but it's really helpful to get familiar with this section on risks and harms. Not just that you can identify whether your research is lower or high risk, but also so you can understand the particular ins and outs of the risks that your research might face. And so you're able to respond appropriately to the risks that your research poses, rather than kind of just generically going, this might be harmful, because that's not always going to point you in the direction of the specific harms that you need to respond to in your risk mitigation approaches. So now we come to the slide that goes to Anne's question. And some of you might be, and again, this is one of those situations where I'm saying you will get the slides. So don't feel like you have to zoom right in and read all of the tiny script that's on this slide. But some of you might know that there's a document from 2014 that's kind of a companion to the national statement that's called ethical considerations in quality assurance and evaluation activities. And I think the intent of this document so I'm deeply confused by this document honestly is to explain that there are situations where the work that people are doing is quality assurance or evaluation. And that's distinct from research. And so it really defines quality assurance slash evaluation as an activity where the primary purpose is to monitor or improve the quality of service delivered by an individual or an organization. And so I interpret that to mean the collection and use of kind of day to day service data that is being used, you know, it might be used for board reporting or that kind of thing to help track day to day organizational performance. This document defines evaluation as a broader task than quality assurance that it is about the systematic collection and analysis of information to make judgments. And it both defines that as distinct from research but also in that document gives a number of instances where despite that it still meets the threshold to be classified as research and requiring of ethical scrutiny. So it's basically saying to the best of my interpretation and again this is where I would put that caveat that it's my interpretation based on sitting with it and thinking about it a lot. Which is that even though it recognizes that's an evaluation activity, it's still using many of the research methods and methodologies that would constitute it to be a piece of research that would require ethical scrutiny. So for me, the most important question when thinking about is this research and therefore does this require ethical scrutiny? Is it research or evaluation? It's other parts of the question about what's happening about the kind of work that you're doing because if you get stuck on this, is it research or is it evaluation question? You're probably not going to come up with a satisfactory answer about whether it requires ethical scrutiny. As a result, I've developed this nightmare of a flow chart to help make decisions about whether your project needs scrutiny and if it does what kind of scrutiny it requires. And again, you'll get the slides and this is particularly something that I'm really open to feedback on because there might be bits in the steps here that I've misinterpreted along the way. But I think obviously the first thing to consider is is it research? And I guess more importantly, is it human research? Because that's what the national statement covers. If it is research, then what is the level of risk that that covers? And you can see I have not updated this flow chart since the new national statement came out. Then we go to that question of, you know, is it being conducted in partnership with the partners that are covered by the national statement? So yes, obviously it needs to do a track of you. But I'm going to talk about some of the other options that might be covered in the next couple of slides as well. So there might be times where you aren't working in partnership with a university, but there are other reasons that you want to or often are asked to do if you're a consultant, an HREC review of a project. So some of those other reasons that you might seek HREC review include you want to publish in an academic journal that requires HREC review as part of the peer review process for a journal article. I would say it's really unusual, mostly if you're not a university academic to be thinking that at the end of a project, you're aiming for academic publication for any evaluation project. So just consider if that's actually a real end goal for you in terms of the research, because you might be forcing yourself to jump through hoops that you don't actually need to jump through. Other reasons that you might consider HREC review are peer and sector scrutiny. So I mean, I do a lot of work in the health sector and because most of the ethics review processes come from that history of quite clinical, you know, drug trials and that kind of thing. It's often just expected in the health sector that in order to do a study that's taken seriously, particularly if you're doing qualitative studies, it needs to go through this process so it has legitimacy. And finally, it might be part of your funding requirements. So your funder might say it's part of our expectation that you do some form of ethical review. And that's often the case in government departments. And then some government departments have their own HREC or have their own processes for ethical review. So if you've got to the part of the flow chart where you go, okay, we don't automatically need HREC, but there might be other reasons that we want to do ethical review. Then it's a question of what kind of ethical review do we want to do. Now, again, you can go to an HREC. Those are the bodies that are set up and designed to do this process. But as we're going to talk about a bit further on, there are some reasons why you might not want to go through an HREC cost timeliness. You might not have one that's available for you to access readily. And then it's important to think about on the basis of what your project needs. There might be other forms of ethical review that you want to take. So it could be that you, I mean, many not-for-profits or government departments have their own internal research review or ethical review process. That might be something that you go through. You might have like a CEO review and sign off process that they've signed off that this is an ethical process. I've seen that happen sometimes. And you might also have like specialist researcher sign off. So you develop your approach in-house and then you go to a specialist researcher to review it and get them to review whether it's an appropriate process that you've used. So it might be after, you know, after you've navigated through that flow chart, you either don't need formal ethical scrutiny. You need HREC review or you need another form of ethical review. It's also worth caveatting in there in terms of HREC review that you might alternatively seek AACIS review. And I've just had a complete mental blank on what it stands for because I just always call it that. And on the website, it also just says AACIS. So apologies for that. But it's essentially the body that views Aboriginal and Torres Strait Islander research and you use that through a lens of culturally appropriate ethical scrutiny as well. So I think the other important thing when thinking about this is the question is often do we need ethics? And that's become a default for do we need ethical review? And I just want to constantly remind you that through all of this, the purpose of ethical review is not just so that we can all have fun filling in an extremely boring form. But it's actually to make sure that what you're doing is ethical. So even if you don't need formal ethical review through an HREC or through another process, it doesn't mean you shouldn't be ethical. You should still be thinking about approval processes, your policies and guidelines that are guiding what you do as researchers and making sure that you're acting in accordance with the principles in the National Statement, the principles in the AACIS Code of Ethics. And any one of these other documents that I've got there on the right hand side. So there's the AACIS Code of Ethics, which I will touch upon in a second as well. The AES has a Code of Ethics and guidelines for ethical conduct of research. And there's also the Australian Code for the, it's probably ethical conduct of evaluation and that would be a typo. There's the Australian Code for Responsible Conductive Research. And it's also worth considering the other things that are part of that kind of ethical ecosystem. So there are things like having good policies and practice within your organization around things like remuneration, around things like safety and distress processes and making sure that everyone involves understands what they are and how they respond in an ethical way. And also making sure that yet your organizational policies are in alignment with those practices as well. As that comment there on the left says that the National Statement doesn't exhaust the ethical discussion of human research. There are both these other documents that exist, but there also might be issues that come up in your research that are new and evolving that the National Statement just hasn't been in a place to think about. And I think a lot of the stuff that in emerging technology and AI and all of those kinds of things, there are constantly ethical issues emerging that the National Statement just simply can't keep a brace off. So it's important to kind of have that ethical framework and ethical principles underpinning what you do, not just have we squared it with a document and a process. Because that's when things like the National Statement, things like ATREC processes risk just becoming a tick box exercise rather than actually a prompt for you to think about how to do your research better, which is what they should be doing. But I certainly know from my own experience that they often just become a hurdle that you've got to get through in order to do the research or at least that's what it feels like when you're in the form filling phase. So how do you interpret the National Statement? So I've included here the key values and principles that sit on the left there in the National Statement. So that talks about there that when you're conducting research, it's a balance between the risks and the harms that we talked about before, balanced with the benefits of doing the research. So it's really important to not just think about the harms and the risks that come from research, but also think about why are we doing this research, what's it going to add, what are the principles that underline how we do this in a positive and a good way. And so those principles are in the National Statement are around research merit and integrity. So again, making sure that you're doing research for a good reason and you're conducting it with integrity. And that you're just in your approach to conducting research that you consider the beneficence. So is it actually of benefit and adding value to the community or the people involved. And also that throughout the process you treat everyone with respect. I also want to include here the AIATSIS Code of Ethics, which has a lovely diagram and a slightly different set of principles for thinking about these things. So it has four core principles at the heart of this in terms of Indigenous leadership, impact and value, sustainability and accountability and Indigenous self-determination. So absolutely if you're doing a project that is working with Aboriginal people and communities, you should have this Code of Ethics front and centre of everything you do and be thinking about how does your research fulfill the priorities that sit within this. I would also argue that beyond that, most of the social research that happens in Australia should also be considering, most if not all should be considering this Code of Ethics in thinking about how you respond to ethics applications. Thinking about how you design research in an ethical and culturally safe way and thinking about how you build these principles into what you do. Again, I'm going to talk a bit more about how I use some of these principles in practice a little bit further on in terms of responding to evaluation, in terms of responding to formal ethics processes. At this point, I'm going to point out it's extremely hot in Melbourne and I turned my air conditioning off to make sure that you could actually hear me speak. And it's getting to the point where the combination of being front and centre and not having air conditioning is frying my brain. So we will keep going. I wanted to also flag about working with particular populations because far and away this is one of the things that you will see in ethics forms. There's usually sections that go, how are you dealing with working with this community? How are you dealing with working with this community? And so I think it's really important to consider the work that you're doing in response to this. I guess the first thing again I'd flag is that this section is under revision and it may well change in the coming months. It's already in the draft. So when the new version was released in 2023, there are already some significant changes to this section from the previous version. So particularly in terms of things like mental health disability, the language around those populations is quite significantly different from how it was discussed in the previous version. I think there is generally, as I said before, less of a paternalistic approach and more of an assumption of capacity. And I think based on the way that the consultation was framed, it's likely that those will see an even bigger focus on that in the revised section, but obviously make no promises because I don't know what other people put into the consultation. But in terms of the key things that you should be thinking about in terms of working with those populations are, first of all, who to involve and how. So if you're working with a particular population or you're studying the impact of a program on a particular population, have a think about who from that community you should be involving in your research and how you will involve them in a safe and ethical way. This was something that was really great to see in the revised national statement is the idea of not involving people. So the previous version had a real emphasis on the risks of involving people, but it didn't recognize that there are also risks when you don't involve people and there's a as it's called a dignity of risk. So giving people the risk to take to take part in something to make a bad decision to take part in a piece of research and gives people dignity to make choices about their own life. And even if they are in a kind of vulnerable population and and provided you're making that environment as safe as possible for them to make that choice into those dignity of giving choice. And so think about if you are being paternalistic and making assumptions about who should and shouldn't be involved in research and making sure that you're giving people options. Another thing to really consider is how to demonstrate capacity to participate. So there's a real focus in the national statement as there should be on making sure that you know people are able to take and have the capacity to take part in research, particularly when working with particular populations. But it's not enough to just state that you've thought about that you actually need to think about how you've demonstrated that capacity to participate. Again, I do a lot of work in the mental health sector and I'm really lucky in that sector to have a law in the Mental Health Act that defines what capacity is and defines the conditions under which people have capacity and the ability to make decisions. So I often just draw on on that law and say these are the conditions that exist within law and they apply in this context here. But you might in your sector need to have a good think about how do we actually demonstrate capacity to participate and what kind of what's the flip side of that so if you're in a situation where I've seen I've seen studies which you know when working with particularly high risk populations or you know people with intellectual disabilities or people with complex mental illness, it requires clinical sign off by somebody to demonstrate that that person is capable of participating in the research. Think about the kind of flip side of the impact that that creates on people's ability to participate and the kind of potential biases that you've baked into that kind of demonstration of capacity. You are just assuming that most people have you know taking it as a baseline assumption that people have the capacity to participate then also just be clear that that's what you've done and be really clear about who's making the determination is it an individual researcher sitting with somebody that's making the determination of capacity. And the final thing to really think about when you're working through responses to an application in this section is also thinking about demonstrating project safety. So how have you made it clear to Human Research Ethics Committee that you have considered the safety of your participants and how you are creating the safest environment for them. Again, it's not just enough to say we think our project safe, you actually have to step through what you've done to create that environment. And also something else that's flagged in both national statements but I think more so in the recent one is the safety of the research team as well so how are you making sure that they're safe as part of that process too. This is basically my PhD in one slide, which I'm just chucking in there because I think it's particularly interesting but it's, it's something I've faced in responses from ethics committees is sometimes some questions about whether particular groups are safe, or have the capability to participate in a particular piece of research. And so I just wanted to basically put a bit of a summary of the evidence that says broadly speaking people from vulnerable vulnerable groups again. Lots of problematic language in this space, but thinking about that kind of vulnerable language can they participate in research safely and the answer is broadly speaking yes. Most people participating in research don't find it distressing and any distress that they do have is generally not extreme or is short lived and also might lead to additional benefits if you're only viewing it through a risk frame you miss capturing the benefits for people of that particular part and again, giving them the dignity to weigh up whether that risk benefit calculation works for them as an individual. It is important to note though that some evidence suggests that the risks are higher among particular particular populations. And again in the area that I'm doing research, it's people who have a history of trauma. There is some evidence that people are more likely to be triggered or re traumatized by taking part in that research so it's really important that what you do is thinking about that upfront and designing safe and ethical research. So broadly speaking some of the things that you can do there are being mindful of the specific risks and benefits as they apply to your project which I like before when when talking about that so being really specific about in this case. You know the risk of traumatization is high in this project because we're dealing with this population and we're talking about this issue that they have an experience of and being really clear about how you've mitigated the specific risk. Working in collaboration with people with lived experience can help to mitigate some of that risk they can identify risks that you might not have considered and they can come up with strategies to address them. It's obviously really important to understand the context in which you're working and the strengths that people with lived experience or people from vulnerable populations can bring to that research. And finally as I flag before not assuming that they're vulnerable and that they don't have capacity to consent but instead building in processes to support people's capacity to consent. So last section I wanted to talk through and it does seem like I'm on time by a miracle is how to work with human research ethics committees. So if you have the choice and I would say first of all that's a big caveat because often you don't have the choice if you're working in a university to go typically with your university. If you're working in a department you typically have to go with the department that you're in. But if you do have the choice choose wisely. And my easiest suggestion for the choosing wisely is to chat with other people who've gone through the process who have used particular a tracks and try and work out the strengths and weaknesses of each committee that you might be working with. First of all have a think about which committee will actually accept your application. I've been knocked back by committed in the past they go we don't actually have the expertise to deal with the nuances of the particular issue and that you are interested in. So have a think about if that's the right committee or if there's a more appropriate one to go for if it's more appropriate for your project to get approval by a access or one of the state equivalents rather than an a track. That's something to consider. The next thing to consider is cost. There are some private eight tracks most notably Bellbury. They're quite expensive. And that's just a fact of working with them. And so sometimes that trade off is worth making because they are also often quicker. And so it might be that you're willing to pay the cost to get something back quicker. But that relates to that next point there of timeliness. And if you if you do your digging and work out how fast people how fast committees respond to things and basically do that that value judgment of which bit of the which bit of the the triangle of cost timeliness and adequacy is most important for you to make. And finally, that last point there is about skills and knowledge, which is kind of related to the first one about which a track will take your application. But certainly in my experience of applying across a range of different eight tracks, I've definitely come to understand that there are some that understand the context of the work that I'm doing better than others. So particularly in the work that I'm doing, which is often about lived experience led evaluation. There are definitely some eight tracks that are better understanding the way you work with a lived experience researcher the risks and benefits of that approach than there are than I have had with others. And so that factors into the decision about which a track to use if you have that choice. I flag statement of ethical issues before as a way of responding to a lot of the stuff that I've talked about today. This is something that I've started building into my research or research or evaluation protocols. Mainly because it initially helps me to organize my thoughts. But then it's also really helpful when you're filling in an eight track application form. Many of you have been through the process before will know that every eight track has its own unique form, its own unique format to respond to. And that makes it makes it really difficult to have kind of copy paste responses, not that you should have copy paste responses to because you should be thinking about this uniquely every time. But it is helpful to have that kind of answer bank to draw from. And so one of the things that I do is basically just put a statement of ethical issues within the protocol that says these are how I think this project addresses each of those principles that we talked about at the start. These are the particular risks that I've identified in this project and this is how I think they're mitigated. And yeah, as I say, it's a really helpful process for just gathering all your thoughts together for forcing you to go through and understand if your research protocol has addressed the points within the national statement. And it makes it five times quicker when it comes to filling in the form because at that point you can copy and paste from stuff that you've already written. And typically it's easier to review because you'll have written your statement of ethical issues probably in a Word document alongside your research protocol and not in one of the many forms that it's almost impossible to export your answers from and edit them in and work on as a group. So that's one of my saving graces and time savers of recent eight track applications is to do all of that stuff not in the form. And my final slide is on tidbits for working with eight tracks on these various issues. The first thing, and this is kind of related a little bit to working out which eight track to go with is to build good relationships with eight tracks. I think we often see them as this scary entity that we submit our form to and nine times out of 10 they tell us that our form was wrong in some way and we get involved in a protracted annoying back and forth with a trick. It doesn't have to be like that is what I'm saying. And if you build relationships, it's very unlikely that you'll build relationships with the committee and the committee members because they do have that arms length role, but the Secretariat can become your best friend. The Secretariat is a really helpful person to talk to before you're submitting an application because you can explain the nature of the research that you're planning to do. And you can say, hey, have any other projects a bit like this gone through the eight track before? What were some of the concerns that they had? And what are some of the challenges that we might face? And again, if you've got a good Secretariat, they'll go, oh, we actually had a piece like this two months ago and they were really concerned about the ability to demonstrate capacity consent among people with disability. So if you could make sure that that's covered off in your application, then I think it will get through with her basically. The other way, another thing that I find it really helpful is when those comments come back and the committee have highlighted the 101 things that they want you to edit. Sometimes those relate to quite technical things or they relate to a slight misinterpretation of how you're envisaging the research and the committee haven't quite got that that's what you meant. It can be really helpful to just pick up the phone to the Secretariat and say, this is the detail of that technical thing that I was trying to explain. I had this recently on with a Government Department eight track where there was just like a change to a technical recruitment criteria that I couldn't really explain without waving my hands around and talking to someone. So I wrote an email and then said, hey, can I give you a call to explain this? And what that means is then the Secretariat becomes the person who explained things at the meeting and goes, OK, committee, you found this challenge. It's all good. I have and Bridget is the person I was just referring to and who's commented in the chat. So it's really helpful because it doesn't mean that there's a constant back and forth between you and the committee mediated by the Secretariat. Instead, the Secretariat can say to say to the eight track, I think what the researcher meant was this. And so you don't have to have as many as many like the back and forth and it saves everyone a lot of time and frustration. The second point that I would put there is educate eight tracks about the context, the evidence and precedence. So it's really important to understand that you might be the expert on the ins and outs of your research environment that it's being conducted in. But most of the eight tracks that you're sending your research to are covering a vast array of research projects. And actually, because of the makeup of eight tracks as outlined in the national statement, you often have people on those committees who are by definition not experts. That's one of the reasons that they're on the committee and they're providing a diverse perspective. So it's really important that you view your application, not just as an explanation of the research, but an explanation of why the research is important and why it should be done in this way. And if you build some of that education in upfront when you're writing it. Again, you've kind of preemptively answered some of the questions that the eight track might have in their feedback to you. And again, saves everyone a lot of time and is gets the eight track a bit more on your side for answering for understanding and responding to what you're saying. The final thing is about challenging decisions that you disagree with. So I think there can often be a temptation when you get the feedback from an eight track to just go, okay, fine, we will just accept what they say because we're running three months behind the project and we just need to get it started. But I would really encourage you to think carefully about the comments that you just go, yep, sure, we'll change the footer. Yep, sure, we'll change that word. And the decisions that you actually think are really important to the integrity of your research and that you won't budge on. And again, if you've got a good relationship with the secretary and the committee, and you've done your education role or continue to do that education role, it becomes much easier to challenge the decisions that you disagree with in a really constructive way. Because you're not just the problematic person disagreeing contrarially with everything, you've actually got a reason to explanation and relationship about why you want to do things the way you want to do them. So those are my tips for how to work with eight tracks. And so far they're going okay, they're definitely better than it was in the past I still get the odd piece of feedback or comment that I go what the hell were they thinking. But most of the time this process has helped me to get through it a bit more easily. And I'm going to stop talking now and hopefully we've got some questions. We do Joe we have lots of great engagement in the chat, but probably the comment has the widest support is for you to have your airplane on if you want to and cool down a little bit people don't mind the background noise and we want to make sure you're okay. I'm all good. I cannot begin to describe how light it is. I think you would disagree if I turned up. I know I've got evaporative air, honest. I just pick out some of the questions that have come through the chat start, and we've got about 10 minutes left and then if other folks after that want to jump in with a question that would be great. We do have a question from Martina, which was what decision making criteria might a funder consider about whether they want a track. So I think they're broadly the same. The same decision making criteria that individual researchers would have and I think it often comes down to on one level policy so they might have a policy that has that written down. But also in particular reputation and scrutiny. I think funders are often conducting research and evaluation for a reason they want to show that their programs working they might be seeking funding in relation to that program. And so they might want to add an extra layer of scrutiny on top of that that says. Everyone knows that we did this evaluation because we want to seek more funding on the basis of it. But we also did it right. And so I think that's where an HREC can be really helpful, but I think it is also useful to note that there might be other ways that you can provide that similar level of scrutiny that might save everyone a lot of time and headache. Because one of the common experiences I'm sure we've all had is ethics has blown out our timeline. And yeah, so thinking about that strategically can really help your project over the long term. Thank you Beth. And ask if you could please comment on Aboriginal data sovereignty and ethics and sharing confidential data collected by interviews for evaluations. Yeah, so I'd first off and say, I probably not the best person to comment on this. I think there are a lot of other people doing this thinking a lot better than me. And I don't situate myself in Aboriginal research because I don't think it's my place to be doing that. I think in general, there are some things that you can do in terms of data sovereignty that are I think steps that you can take to build that into your approach that we often just don't even think of because this established practice in research. And so I would also say at the kind of like big data level, big data sets, that's also not something that I do in terms of my practice. I really speak to data sovereignty at that kind of level. Most of the stuff is what I do is around kind of smaller scale qualitative studies. But some of the things that I've started building into that are things like obviously making sure everybody who's done an interview gets a copy of their transcript and an opportunity to correct the transcript. As ownership over how they're referred to in the report, I think there's often a default assumption that people want to be anonymous, but particularly in the area that I work. People are really proud of their stories and they want to claim those stories publicly. So giving people the choice over that. It's an admin headache for you as an evaluator because you have to go back and check things. But it's it's worth it for that kind of ownership of that. And then obviously making sure that people get the data back in some form as well. So having reports that are publicly released. Again, it's really hard as a consultant to have ownership over that whole process because often it's up to a government department or an agency about whether they release something. But those are some of the things that I would be thinking about in terms of where I have influence. And calling that doesn't ethical certainly require some sort of farms. Really in reference to that internal reviews. And that's happening around ethics reviews internally and that this may be insufficient for high risk projects. I think it depends. I think it depends on the purpose of the research. And I think it also depends on the approach that you're taking to the evaluation. I think if you're, I think that, you know, often with internal evaluation, if it's if it's internal evaluation within a department within a not for profit, you don't have arms reach arms like anyway, because there's people involved directly in that process. And that might be a reason for having extra scrutiny because you've got people in the day to day doing and you want to make sure that they're doing that appropriately. But I think it also depends on the scale and the budget, the level of risk, the, how you're going to use the evaluation findings that you that come out of the project. I don't think it's as black and white as it must have arms like scrutiny, but I do think you should let you need to consider what's appropriate for your particular project. And sometimes it's just practically really difficult within the budget, the timeframe, the connections that you have on a project to put it through an hrec. And I guess I just wanted to provide some reassurance that that doesn't mean that what you're doing is not ethical and hasn't received kind of scrutiny. There are other ways that you can approach that challenge. Great. Thanks, Jo. And Stephanie and a lot of support for this question. How do you account for unfolding iterative design processes, particularly can you let evaluations we're doing in a fixed application? So my first response would be that this goes to choosing your hrec wisely and firstly trying to find an hrec that gets this and is used to doing work in this way. I think in my experience, I would are away from, if you can, are away from the hrecs that have a more clinical bent because they are the ones that find this stuff really difficult because they're used to nice neat RCT protocols which don't change in the way that iterative design work tends to. So first of all, try and find an hrec that that gets what you're doing. That would be my first piece of advice. Then I would have a chat with the Secretariat and go, I'm sure you've had this problem before. How have other projects approached this? What have you found that's worked and kind of build on that knowledge of what works? And the final thing is like once you've gathered that information, provide as much information as you can to the committee, but be really clear about where you have left intentional gaps. So, you know, saying the first phase of our project is to design our research guides. So we haven't included our research guides at this stage, but we'll send our research guides to you as soon as they've been designed. And so seeing if there's a kind of a level of ambiguity that that hrec are comfortable with approving through that process. I think it's also important to recognize that if you, you know, often iterative design processes, you could potentially get away with not having hrec scrutiny of that bit at all. You could say that that's the design process to inform the evaluation methodology and it's the evaluation methodology once designed that is the bit that you'll get hrec scrutiny for. So there's a few different ways that you can potentially manage the challenge, but again, information gathering and working with hrecs I think is always your best bet. Great. We just have a couple minutes left and people more questions coming in. There was a question here and I'm not sure if you're going to answer this but whether or not it's standard for all peer review journals to require hrec approval. No, it varies from journal to journal. It varies based on things like their impact factor, some of the more high impact factor ones that it's obligatory. And it also depends on like what they define as research. I think often evaluations get away with not needing scrutiny even when they sit within journals that would require that just because of how they view things. Again, it's one of those things where generally if you're writing a paper, I would always suggest that you write it with a journal in mind because you're much more likely to get it accepted because you're writing to the terms of what that journal requires. And so once you've got that figured out you can then go okay what kind of scrutiny with that that journal need. But again, sometimes journals what they're what they actually want to know is not have you gone through an hrec but have you thought about these things. So if you can prove in other ways that you've met the threshold for doing ethical research, then you don't necessarily need a trick unless it's a journal that says you need a trick. There's no clear answer to any of this stuff. That's the challenge but you've navigated us very well Joe there was just one question here about the timeframe for approval by an hrec. Now I know this very but I wonder what you might say about that. The answer is it varies. And so one of the things when thinking about the time in this is thinking about when they come if you're working with an hrec that has set committee dates. So that obviously restricts when things are going to go to committees when things are going to get approved in committees. I've had ethics approvals with private providers that have been approved in three weeks. I was saying to Aaron before we started that an amendment on my PhD application took nine months and the only feedback they said was change the text in the footers. So it varies hugely. And so if you can work out what you need and which committee is going to give you the best result in terms of what you're balancing then. Yeah, that's the best way through. Right. Thank you so much. I think we might come to a close now we're at the end and thank you so much for taking us through everything Joe and all the tips and the flow charts and folks like I said at the beginning if you weren't there at the start we will be sharing the slides and recording around as well so you can catch up on what you may have missed or other aspects of the wonderful slides that Joe has put together for us, including PhD on a page. I mean, that's always a win if they can put that together. Lots of appreciation for you in the chat. Joe anything final you'd like to say before we go. Well, just if you ever have any questions about this stuff feel free to to reach out I'm like a third of the way through my PhD so I fully expect my thoughts and understanding on this will evolve on the next few years so if you're keen to be part of the evolving nerdy conversation then just feel free to reach out. Okay, thank you and thank you everyone. I hope you all have a wonderful evening.