 We have next Tom Van Drimelen, a PhD researcher at Leiden University Medical Centre, and Tom is going to talk about researchers' decision making. Tom. Thank you. Very nice to see so many of you still here, especially seeing as the Eurovision Song Contest is on right now, hoping Holland gets through to the finals so that I can see the rest of it on Saturday. But I'm very happy to be here talking to you today about our research project on researcher discretion. Sorry. And when most people, when most researchers talk about researcher discretion, it's usually seen as something, researcher degrees of freedom, it's usually seen as something negative. It's a place where science is vulnerable, where biases, where conflicts of interest seep into the research through the researcher. So usually the idea is that we need to try to curb this researcher discretion as much as possible. So I've got here a quote from a fellow Dutchman, Bichertz, and his colleagues. They wrote that the ideal preregistration of a study provides a specific, precise and exhaustive story of the planned research. That is, it describes all steps, only one interpretation, and excludes all other possible steps. So on this board up here, it's very easy to do that. Bam. What does it look like in practice? Is it even possible? We found that quite a lot of the research that talks about this topic of researcher discretion engages with it on a theoretical level, what people should and can do. Quite a little, not a lot, is on an empirical level. So we wanted to change that. We set out to research, researcher discretion in practice to find out when researchers are required to exercise decision, what components come into these decisions, and how these decisions are actually made. So how do we do that? By doing ethnography in a research group. Ethnography is mainly based on participant observation, triangulated with document research, semi-structured interviews throughout. So basically, like Malinowski did about a hundred years ago off some islands off Papua New Guinea, but minus the obvious racist and colonial undertones, I hope. I spent 12 months at two different end of life care research groups embedded in their groups, joining formal informal meetings, taking note of observing and taking note of when researchers exercised discretion, when they made decisions about implementing the research plans. So what did we find? First of all, it kind of just slapped us in the face, just the sheer abundance of researcher discretion. How often these researchers had to figure out and still make decisions in the moment, and after long bouts of coding where we tried to identify and categorize these instances of researcher discretion, we found two categories under which this researcher discretion emerged. First of all, it emerged when plans were under-determined. So plans were under-determined in the sense that it failed to, it was not sufficiently precise to specify a specific action for the researchers to take. Something still had to be operationalized. Big circle, the researchers in the moment still had to focus it. So what this comes down to in practice is you have quite clear inclusion criteria. But what we saw is quite often these researchers still had to make decisions in the moment about whether a particular potential participant is wanted to be included in the study, for example, because their medical condition might not fit exactly with the inclusion criteria. Maybe you're not quite sure whether they understand the language, whether they sufficiently mastered the language. These are all things that had to be operationalized in the moment still. So where research and discretion emerged. So the answer at this point seems quite simple and we should indeed be following what Lichards and his colleagues said. We should be making plans that are very specific, very precise, and exhaustive. But that might not be a suitable solution for all either because, on the other hand, we had quite some moments where plans were over-determined as well. So over-determined plans were plans in which the protocol specifies, it's wonderful to specify specifically when, where, how something needs to happen, but things changed under circumstances at hand. Those are no longer desirable or even possible to execute. So you have to do something else. And again, research and discretion emerged. So even though in certain situations, and there might very well be situations in which you can specifically plan for actions in advance, probably this research and discretion will, will stay. But this is nothing new. We all know that plans change and that doesn't have to be a problem either because we've heard before, plans are plans and not prisons, so they can change and it's okay, this adaptation, operationalization is all right if you're just transparent about it. But our next finding kind of problematizes or makes things difficult for this assumption as well. Because we found it quite difficult to identify discretion sometimes. So occasionally in our data it was very clear researchers made a decision. They came together, they weighed different options and they picked one. But in many more other cases, there might have been different options available, but the researchers didn't consider any of them. They just went ahead, it was just implicit. So an example of this obviously is when we identified a decision in the data, but the researchers themselves didn't identify it as such. But interestingly, the researchers amongst themselves also express confusion, disagreement about whether an active decision was merited or whether a decision might have been made in the past. For example, we saw a couple of times where some person, researcher asked a question in the research, should we do this or that? Does this mean this or that? And another researcher just dismissed this, well, it's clear, right? It's right there, so we don't need to think about it anymore. So obviously there was these people identified their discretion differently. And if people identify their discretion differently, then that might mean that we're not really able to be transparent effectively about the things that really matter. Um, tying up a bit, I think tying up a bit. I want to say that maybe our research shows some indications that research discretion is an integral and unavoidable part of the research practice. What makes things a bit more difficult is that if we were just transparent about it, that would be great, but we might not be able to be effectively transparent because we don't know which decisions we've actually made. Now I'd like to tie this off with a nice quote by a call popper who wrote that institutions are like fortresses. They must be well-designed and properly met. And I would say so are plans, and that's why you see this nice double helix because I think we have two ways forward to go here. We need to design a fortress and we need to train the troops. Designing the fortress means that more follow-up research is necessary to contextualize and to qualify scientific rigor when it is necessary and when it is possible. But we may have to abandon fantasies of a protocol that states a specific action that is only interpretable in one way. But the next point is that training for responsible conduct of researchers might also have to include a bit more of an explicit acknowledgement that researchers regularly are required to exercise their discretion. And in order to do so, we might need to build capacity for these researchers to engage with these decisions with critical autonomy and reflexivity in order to responsibly manage the freedom and responsibility that discretion gives them. Thanks so much for your attention. My fellow researchers on the topic are Ria Reis, Niki Slachbaum, next bout there, Yeni von der Steen. There's so much more I'd like to talk to you about. Please let me know if you have any questions or tapping on the shoulder for the rest of the conference. Thanks so much. Great. Thank you, Tom, very much. Great talk and hopefully God is back more or less on time. If there are any troop movements to the microphones for questions, now is the time. To come forward. No, we're... Oh, there's one. Yep, quick, quick, quick. Yep, great. Thank you. This question will be very fast to ask, but potentially a very long conversation. It's part of my wondering about the relationship between metascience as a community and science and technology studies. Your study reminds me very much of Norcatina's study of epistemic cultures, Bruno Latoura's size and action, and this interesting sort of application to thinking about the issue of researcher degrees of freedom. So the question, I guess, is how do you make sense of, like, is this sort of an application of what we knew from STS into a metascience framing, or is there something, how do you make sense of the relationship between what you found and what we know from STS? Great. It's a nice question, and I think the fact that I kind of got through to you is a sign that I kind of succeeded in what I was trying to do. Indeed, I think both disciplines are kind of separated at the moment. I'm trying to combine them a little bit, but honestly, whether I've actually been successful about that is whether people, metascientists in the room, kind of find this credible, and STS people, what they think about this. So let me know. Great. Thank you, Tom, again, very much.