 All right, I'm going to go ahead and get started. I think a few more folks might trickle in, but we've got a good crowd. Welcome everyone to the Funding Consciousness Research using the Registry Reports format. This is, I'm going to share my screen and give a couple of introductions and we'll have plenty of time to go over the details of the initiative, what the workflow looks like for funding and publication and the great partnerships that are forming throughout this. So here we go. You should all see my PowerPoint and slides here. So again, thank you very much everyone for joining us today. The slides will be made available and links to all the resources that we're mentioning and instructions are coming to anybody registered to this webinar. I'd like to thank in particular our panelists, Lucy Charles and Zoltan Dines. I'll be giving a little bit more complete introductions in just a moment, but thank you everyone for participating and thank you to the Temple and World Charity Foundation for supporting this initiative and links to those resources will also be available on the website. Today we're going to talk about the initiative and how the Association for the Scientific Study of Consciousness is working with us on expanding the registered report format to this research community. Then in particular, how neuroscience of consciousness which has been accepting registered reports, Zoltan you might have to correct me, but for several years now, three or four years now I believe, how they manage the workflow and what to expect when submitting a registered report. Then I will take over and talk about some of the funding expectations, what to include in budget, how to submit pre-submission inquiries to us and what to do if you are looking to publish in one of the other partner journals in the peer community in registered reports. And then finally we'll have Q and A time at the end. If you have a clarifying question, please put that in the chat at any time. If you have any question, put that in the Q and A feature also at any time and we'll make sure to get to all the big questions at the end, but if there's any points that we are unclear about during the presentation, we'll interrupt it and clarify that right at that point. So again, thank you to Lucy Charles, the Executive Director of the Association for the Scientific Study of Consciousness and fellow at the British Academy. It's been a great partner in helping us figure out all the logistics and get a great advisory panel together to help us with our work in supporting this research and Zoltan Dines, Professor of Psychology at the University of Essex and the registered reports editor at the Armenian Journal Partner in Neuroscience of Consciousness. So thank you very much. My name is David Beller, I'm the Director of Policy here at the Center for Open Science in the USA. At this point, I'll pass the baton to Lucy. I'm going to stop sharing my screen and she can talk about the ASSC and the expertise that is joining us from them. Sure. Okay, let me share my screen. Okay, so welcome everyone. Thanks very much for joining. It's very nice to see not familiar faces, but familiar names. So thanks for the introduction, David. So as I said, as David said, I'm the Executive Director of the ASSC and I'm very, very happy today to be kind of introducing this scheme to you in collaboration with the Center for Open Science and the Temporal Foundation. So I'm just going to say a few words about the ASSC. Hopefully most of you know the association, but just for some of you who might not know, the ASSC was founded in 1994. And the core goal of this academic society is to promote rigorous research towards understanding the nature function and underlying mechanisms of consciousness. And really what's at the core of the society is pluridisciplinary. We have members working in the field of cognitive science, medicine, neuroscience, philosophy and many other relevant disciplines in the science and humanities. And the first conference of the ASSC happened in 1997 and since 2007 we have official journal. It changed across the years, but that's one of the core activity of the ASSC. And just to describe a bit more what we've been doing, mainly the ASSC organized a yearly conference. A last in-person conference was in 2019, quite a long time ago now in Canada, London, Ontario. And we had to cancel the 2020 meeting, but we had a wonderful virtual meeting in 2021 organized by the Tel Aviv Organizing Committee. And we're very pleased that this year we have again a first in-person meeting after a long time in Amsterdam that will be in July and next year will be in New York City. So if you're interested, please look at the website and register for this year's meeting. So really the core of the ASSC is organizing these conferences and again having many different scientists of different fields kind of meeting to discuss consciousness research. We also have the William James Prize that we give for paper every year. And this is for an outstanding contribution to the field of consciousness research. And again, what's at the core of this is pre-disciplinarity. And for that reason, the William James Prize is given alternatively each year either to theoretical or empirical contribution. And as I was saying, we also have official journal. This is Neuroscience of Consciousness. So that will tell you maybe a bit more about the journal later. And we don't directly publish the proceeding of the conference in the journal, but the idea is that it really, we work with the journal in close proximity and the members of the ASSC are encouraged to publish their work in the journal. And we both work closely with the editor, Anil Seth, as well for developing initiative through this. So these are the core activities of the ASSC that you might have heard about already, but you've probably seen that the fields of consciousness research has been evolving and there's been a lot of reflection about what we do as scientists working on consciousness in the recent years. You might have heard of the Consciousness Theory Study Database, for instance, which lists different, all the different papers on the topic of theories of consciousness across the years. And this has maybe revealed to us that there are some shortcoming or biases in the type of papers that are published on theories of consciousness. And I think this is not just this paper, but also a growing feeling that maybe there are things that we can do to improve consciousness research. And we're very much appreciative of the initiative of the Templeton Foundation for accelerating research on consciousness. And it's particularly the initiative of adversarial collaborations that you might have heard of with the idea of having different supporters of theories of consciousness confronting really the theories. The ASSC very much support these initiatives and indeed at this year's conference in Amsterdam, we're going to have the Great Consciousness Debate where several of these supporters of theories of consciousness are going to present these theories and debate directly to kind of really create progress in the field and not just kind of stay within one kind of community of thoughts about the theories of consciousness. So these are the initiatives that the ASSC participate in, but we were really, really excited to see the Center for Open Science and Templeton kind of contact us and link up to also favor open science in consciousness research and really develop an open science of consciousness with the idea of promoting transparency and rigor in consciousness research through registered reports. And I think the idea of this scheme and in particular to provide funding for registered reports is to empower consciousness researchers to use open science practices. And here I'm thinking in particular to junior scientists who might not have access to this more larger source of funding for consciousness research. And I think it's particularly important that junior scientists do submit some of their registered report to this school because you are really the next generation of scientists that are going to promote this higher standards and new standards in consciousness research. And the second thing that I wanted to say is that we very much hope that the ASSC in Amsterdam will be the occasion to also develop collaboration and submission for this scheme. So I think Zoltan is going to talk a tiny bit more about the registered reports, but I also wanted to point out that he's going to have a tutorial on registered reports at the conference. And I think David and myself and many more will be able to answer your questions at the conference as well. So that will be the occasion to hopefully create collaboration and have people to ask questions directly to encourage the submissions. So I think I'm probably over time a bit, but the last thing that I wanted to say is that we're really lucky to have several of our board members who agreed to be on the review committee for this scheme. And the idea is that these are scientists, of course, expert in the field of consciousness and will decide basically whether the registered reports are in the scope of consciousness research. So obviously, if your submission is going to neuroscience of consciousness, you don't really have a problem in the stem of the scope but because many other journals are part of this scheme, we think we saw that it would be good to have a kind of first review of the submissions before they go to the registered reports. So I'll stop here. I hope I wasn't too long and I'm very much looking forward to hearing your questions about this scheme. I think it was perfect. Thank you, Lucy. I'm gonna pass to Zoltan now to describe the registered report process at neuroscience of consciousness. Zoltan, take it away. If I may, I'll describe also peer community year in. Is that okay, David? Yeah, that's absolutely fine. Because when you submit, there's, there were two routes. You can either go to neuroscience of consciousness as the society's journal and obviously the remit or the journal is exactly the remit that is wanted for this scheme. Another thing you can do is to submit to peer community year in registered reports, which has been going about a year. And over the year, we've had about 50 submissions that we're dealing with. And what happens is peer community year in registered reports, in a way that's free for the author, free for institutions, free for the reader. It takes the submission through the registered reports process all the way using archived submissions. And then we have a list of PCI, our peer community in registered reports, friendly journals. There's more than 20 of them, which includes some that was just listed on previous slide. My psychology of consciousness is obviously rather than one. Royal Society, Open Science, Cortex, and many others that could be relevant. They have committed to accepting the papers that have been through the PCI process as registered reports. Pending though that any author processing charges, APCs and any other specific requirements of the journal like specific power levels or other criteria that we give have been satisfied. So what you can do is either submit to Neuroscience of Consciousness or you go to PCIRR. PCIRR takes you through the process and then you have the choice at the end of that as to which journals you may wish, if you wish. You can say in the PCIRR system that it's been fully reviewed. And edited recommended, as we say. But if you wish, you can then go to any of the 20 of journals. Now, David, I'm right in saying if it goes through that route, the PCIRR, that's when it goes to you and the ACC committee for vetting as to the top of the probe witness. Yeah, the, yeah. Yeah, the pre submission comes to us for vetting and focus just to make sure that the scope is within the right remit, correct? Yeah. So, I mean, the main thing I wanted to talk about is some of the things that are not in place in an initial registered reports mission just so that you know about that. But there are just a couple of features of the PCIRR system. I just want to highlight because it's different from anywhere else. One is that we do a scheduled review, PCIRR, meaning you put in a very short description. You send to us a very short description of what your stage one, your initial submission will look like. And we send it out to reviewers and say in six weeks time, in a five day window, do you commit to reviewing the paper in that time? Absolutely commit. There's no exceptions, yeah, in that five day window. And then in that six weeks, your authors have a chance to write up the manuscript, give it to us a week before that five day window. And then we send it out and we have guaranteed reviews back within a week or two. And about half our submissions of PCIRR have been through that process and it's worked as advertised. So that's one feature. Another feature I just want to mention because it could well be relevant to you is typically a registered report involves a single study. It might involve a pilot study, which is always recommended. But then the registered report part of that in which you're pre-registering this particular study is typically one of the most important features in one study. We have programmatic RRs where you can at the initial submission go through the review process through a set of studies which can each have the individual papers or be published all at once. Okay, so now when you submit a registered report I want to talk about registered reports in general now. I found a very common way psychologists and other people think about experiments is they say, I wonder what will happen if I do this? I wonder what will happen if I do manipulate this thing? Let's see what happens. Now, wondering the scientist is a fine thing of course. But that's not quite the sort of the attitude you want to have by the time you get to a registered report report. There might be appropriate say and good it's not that that's not good. That's fantastic and how science works. That would be an exploratory report in cortex. And it could be the basis of a pilot study which you do. And if you're thinking of going for a registered report through this system or any other it would be good to do a pilot study. But then what you want to have on the table is some clarity about what claims are at stake in terms of why you're doing this registered report. What are you putting on the line when you do this? So if you, there'll probably be some interesting general bold claim, there may not be but they're probably something you could regard as a substantial theorem. So something like that could be for example extrinsic motivation kills intrinsic motivation. That's sort of a general idea that's not quite testful as stated is a bit too abstract. But it's sort of a bold interesting idea. And then through some assumptions you can come to more specific claim like if I reward children for eating the broccoli they won't like it so much because of the reward. So you're now making a particular prediction. So now you define some particular dependent variables ratings of liking and what your manipulation will be what the rewards will be and so on. Now when you do this you're gonna make certain assumptions like for example that the rewards you use are genuine extrinsic motivations for the subjects in question. And there's going to be for a given reward a prediction like that there's gonna be you need to think about what's a specific test a specific test that would test that claim that prediction. So children given sweets for eating broccoli versus those not given sweets will like the broccoli less after two weeks. So that's some specific claim then you might say well I'll do a tea test to do that or whatever it is that you want to do. So what you need to think about is what is the specific test for each specific claim that you're gonna make. And as I said, whenever you do that you're making some assumptions and registered reports are pretty hot on manipulation checks and checks of assumptions. So if you wanted the data to count against the general claim or the specific claim about eating broccoli in sweets what is it that needs to be assumed and how can you check that? Like as I said that the in this case can you test that rewards are rewarding or extrinsic motivations for the subjects in question. Or if you're claiming that some distracted task is more difficult or demanding than another distracted task what will be your test that the task is more difficult than the other times. You need to list these and you probably will need some manipulation checks and more generally what we call outcome neutral tests. In other words, tests that make sure the experiment will do its job of testing substantial theory for the main claim that you're actually interested in. So in your registered report you need to list the specific hypotheses the manipulation checks and the main claims that you're testing. So be very clear about the claim that is tested. It's not, I wonder if this is going to happen it's going to be this group will be larger than that group on this variable. And those claims will then line up with specific tests. So if you're predicting a linear trend the higher the voltage goes in some experiment then the more something else will happen. The test of that is a linear trend. And in general you'll probably find not always but the test that you're interested in is a one degree of freedom test is most predictions amount to one degree of freedom. So the appropriate analysis for testing a linear trend is you don't need to do the omnibus overall left or the overall ANOVA. You just say you're going to do the linear trend you specified this is the claim there'll be a linear trend well then you that's what you're going to test that's what you're going to specify is the linear trend when you specify the analysis. So make sure you're very clearly listed each hypothesis including the manipulation checks and then you've associated with each of them the specific test that tests that claim. Now what you need to be able to say as well is what would count against the claim. So the experiment can only do his job of testing theory if data can count against it. And that applies to manipulation checks as well. We need to know what data would count against the manipulation. So if you're proposing equivalence of something two groups need to have the same level of expectation mindfulness, whatever it is. You need a test of equivalence to support it and test a difference to go against it. And if you're predicting a difference you need to be able to get evidence for equivalence to go against that that claim as well as evidence for a difference to go with it. So the sort of the rather standard practice of simply doing significance tests and finding something significant and something not and that's all you do isn't going to work that isn't going to do the job. So how can you do the job? Well, if you want to be able to assert where evidence for equivalence which is what's necessary here there's not many things to do and the three main ones is you can work with power. You can say I've got enough power to pick up what would be a minimally interesting effect. And the other is base factors and the other is some form of equivalence testing but those are three, that's not exhausted but those are the main three. So you'll probably be picking one of those three. And that means you need to show the sensitivity of your test with respect to finding equivalent results or different results. In other words, how severely are you testing these claims that you've laid out? Now power is used to control the error of saying the error of saying something is not there when it is there and it's controlled with respect to the effect size you put into the power calculation. So what that means is you can only make claims about what isn't there up to the limit of the effect size you put into the power calculation. What that means is you need to specify a minimally interesting effect size for your power calculation to make scientific sense in getting a non-significant result to count against the claim which I just meant to count against. For a base factor, you need to have some sense of what's the expected effect size you're trying to pick up. That's the main thing you'll need to specify. If you're doing some sort of equivalence test either phase-in or frequent test you need to specify a region of sort of meaninglessness. Effect size in this region are too small to be theoretically or practically interesting to question that you're dealing with the claim that you're dealing with. And that means you need to specify a minimally interesting effect size again. And then you're gonna say whether your confidence interval or credibility interval lies within that region at the end of it or outside of it or spans both and is inconclusive. So then you need to work out if you're doing that sort of equivalence testing is how many subjects would you need to have the confidence interval be able to, in principle, sit completely within the region. Then you run that amount of subjects. Same for a base factor. Having specified what effect is relevant in this context for the theory or claims that you're testing. So a default base factor or a default anything is not a sufficient grounds for doing it. So a default effect size, by default base factor, I mean you're using a default predicted effect size but there's no such thing as a default effect size in the scientific context. So any default models that you use, you have to say why that model is relevant to your particular experiment. You have to justify scientifically. Now when you actually do this, it typically turns out you need more subjects than is typically run in that field. And that shouldn't be terribly surprising giving all the papers that have actually come out decade after decade about how so many fields are radically underpowered. So the fact that when you get to think about it in advance and really do this well and have your reasoning checked by reviewers and editors and talked about to make sure it's solid, it's not just construct a post-op after running an experiment that you end up needing more subjects than people often typically put in the first submission and at least more than is typically done in that field. So I'll tell you that's the biggest thing to think about is making sure that the tests that you do are scientifically relevant to the claims that you're putting forward. And the way that such a good test become scientifically relevant is when you're modeling what could be there or what's interesting in a way that's relevant to the science that you're dealing with. And then that leads to you proposing a certain number of subjects which is gonna be really relevant to this initiative because you're gonna have to say how much money you need and that's gonna depend a lot on how many subjects you have to run. Have I run out of time now? That's a good segue because the budgetary question is one that's gonna come up quite frequently in terms of what the proposed sample sizes are based on the scientific relevance and how we'll be looking out for that through the review process. There are two questions that have come in, and I think it might be okay to... I think there's one question from Aaron Scherger that might be quite relevant to what you were just saying, Zoulton, so is asking are well-designed data-driven or exploratory studies out of the questions here? So how much a registered report can kind of be exploratory? What would be your answer? Yeah, I mean, it's not sort of designed to be exploratory but there is a sort of a new answer there. I mean, some people might think that exploratory involves estimating parameter sizes and not testing hypotheses and they have been registered reports where there wasn't hypothesis testing, neither Bayesian, but it was estimating the parameter. You just wanted to get an unbiased estimate of a certain parameter. I don't know if you count that as exploratory, but that's still a fine thing for registered report because you're trying to take the bias out of the procedure of which this parameter is estimated. So that certainly can be done. But if you were just saying, I've got this whole bunch of dependent variables and I'm wondering what happens if I do this manipulation, I would say that's not quite relevant for registered report. Yeah, or those preliminary exploratory findings can and should be used to justify the submission or part of the rationale for the submission and any exploratory results that come up throughout the process can and should be reported as exploratory or unpredicted tests, but now that's the main focus of the registered report. Yeah. So there is, so just to clarify, so there's room, so you can register report with the planned analysis and then still report in the final paper some postdoc analysis that might be clearly labeled as postdoc after you've reported the kind of core of the analysis that you registered. Yes, you can do any analysis you want in the final page too, but as you say, it's in a separate section, a non-pre-registered analysis section. What one provides though, is when it comes to summarizing results in the abstract, the pre-registered analytic protocol is the one that should maybe only but certainly primarily dominate the abstract and the conclusions there. And the same in the discussion, you should focus on what was registered so that that comes to guide the firm conclusions or as firm as they can be from the paper, but of course you can still discuss and conduct exploratory analysis, just they'll just be labeled as such. Right. I think there was another interesting question. So from Mattis Lobo who said he's a physicist, a mathematician and he asks, does consciousness research include data analysis coming up from dreams while sleeping and connecting to one's own consciousness and external facts somehow connected to those dream reports? My feeling is that that sounds perfectly relevant for consciousness research, but as editor in neuroscience of consciousness, Zoltan, would you think that would be suitable? Dreams would be the fine topic for consciousness research. Yeah. So I think suitable topic, the question would be, can you actually make it as a registered report as we discussed? A registered report as a case study involving yourself might be hard, I think, for reasons of being blind to the hypothesis and they're just sort of reasons of the experimental rigor. I mean, I don't know the details of it, of the study that you have in mind, it sounds like that would be a difficult one to be honest, but what you could do is have a case study on yourself as a sort of pilot and see if that motivates any further ideas to do with dreaming that you could run as sort of a more controlled study on other subjects. We do have more questions, but let me give a couple of more details about some of the workflow and requirements and then we'll make, we have all these captured so we'll make sure to either get them at the end or absolute worst case scenario, we'll write them down and post them on the website, but a lot of great questions coming in, so please make sure we'll make sure we get those answered. Oh, no, I messed up one more time. There we go. So I just wanted to emphasize that all the materials that I'm going to be pointing out in the next few minutes are available on our website, cos.io slash consciousness. Frequently asked questions, any additional questions that arise from the webinar that we're not able to answer live, we can post there as well and links to the pre-submission inquiry, budget template and all the participating journals are available there as well. When deciding to submit the first thing to do on that website is the pre-submission form and this just allows us a couple of different important checks to, of course, make sure that the content is relevant for the scope of work that are to be supported and that's where, I'll point out where in the workflow, that's where the advisory panel comes in. It also helps us keep an eye on the budgetary expectation. So we, I'll give some parameters for that in the next few slides, but we're hoping to hit the target on that. The pre-submission form, of course, has some demographic information about yourself, question about what the expected timeline for your study is, where you expect to submit your results to or your register report to. And the most important part of the pre-submission form is the structured abstract of a little bit of background, what the specific objective and hypotheses are, what methods you'll use. And then finally, some acknowledgement and understanding of what the expectations of the funding is will be involved with the review process, again, to be on the lookout for any changes that affect the proposed budget. And there also are not only rich reports, but high open science expectations and requirements for all materials and all data generated as part of these studies. So to the greatest extent that's allowed ethically without compromising subject confidentiality or anything like that, all of the underlying data and materials and or data analysis process has to be publicly available as part of the, as one of the conditions of funding for these. So the basic timeline is that you submit the application, the pre-submission inquiry. We will be checking to make sure that those are within the appropriate scope, thanks to the support of the ASSC advisory committee. And then that can go either directly to neuroscience of consciousness or to many of the partner journals in the peer community and register report process. As Zoltan mentioned, though that process has a couple of different options where you can schedule a review if you want quick feedback, but you have to commit to a six week timeline of getting it to them, or it can be through their normal recommendation process of posting it and then requesting review. And that goes through their process. Sorry, one moment. There's some graphic noise. Upon in-principle acceptance, either through the peer community in format or at neuroscience of consciousness is when funding will be made available. So if it has a approval from us from the pre-submission inquiry, we've been monitoring any changes that occurred during the review process that might affect budget. The award will be given at that stage one in-principle acceptance and urgent support the proposed and accepted research design. That will permit conducting of the study and then as normal, publish the results approximating line with the proposed timeline. Of course, subject to experimental realities. The funding and budget we anticipate between 30 and 75 awards being made upon in-principle acceptance. We expect most of those to be between 15 and 20,000 US dollars. There's an absolute maximum of 50,000 US dollars. And of course, that would limit the total number of awards. And we're encouraging budget templates to include mostly research materials and participant support, and less so on researcher time or salary support. The budget template is available. There's a link to it on the COS website and the template and other materials that are available for download are available on OSF for doing so. And there is option to store all digital materials, data and a local code or any supporting information on live OSF projects and upon in-principle acceptance, those can be registered. So that creates that permanent read-only version of the research plan through the OSF registry. I won't go into too much detail because Zoltan gave a fabulous overview of the peer community and register reports. Primarily, I'll be pointing to the great materials that they have available on their website, rr.peercommunityin.org. The workflow that they allow includes the submission of the research plan. Again, after that's given a pre-approval by us through the submission form. And then the preprint, it's managed through revisions of posted preprints. And so the stage one proposal is version one of a preprint and that might go through, Zoltan, you might correct me, but two or three or maybe occasionally more rounds of review as reviewer feedback comes in. I'm not sure what you would say is typical in stage one. Are you able to give an estimate there? Yeah, just like a normal submission as a journal ready, maybe three rounds is, yeah. Okay, yeah, that's what I've seen and expected also. And then again, upon recommendation by the PCI community, that is the equivalent of in principle acceptance, of course, and that's the point at which funds can be distributed. As Zoltan mentioned, also the key feature, one of the many key features of the PCI Registry Report Community is they've got a list of, a large list of PCI Registry Report friendly journals. They commit to accepting the proposed and the accepted manuscript without any further scientific review. So there might be a couple of additional requirements that a journal indicates about word limits or ensuring that the scope of work is appropriate for what they are able and willing to publish, but they do commit to generally accepting those PCI endorsed studies and manuscripts. And here's a complete list of those PCI friendly journals. And again, we'll be posting a link to that on the website so you have clarity on what all of those journals are. But many of these are obviously appropriate for consciousness research, but as we've mentioned a couple of times, it is important that we have the pre-submission check just to make sure that the proposed work is within scope of what we're able to support. It's a little bit redundant right now, but all information is going to be available and is available on the Consciousness Research website. So I'm going to stop sharing my screen now and we'll get back to, I think a lot of the great questions have been coming in. I think we've got a good backlog of five or six questions. So let me stop sharing and we can work through that. So I think there was a couple of questions about journals that maybe we could, so someone is asking whether any of the plus journal would be possible submission avenues, David, what do you think? Not at this time. If those do become available, we'll make sure that that's prominent listed on the Consciousness website. But for the time being, the ones listed on cos.io slash consciousness are the eligible outlets. And Ndidiska is asking, how about the PCI are interested journals? So I think journals that might be may be coming in too. Yeah. Exactly. And if you're familiar with the peer community in journals, there's another group of journals that look favorably upon those that have been recommended by the peer community in review process. Again, at this time, no only neuroscience of consciousness and the PCI friendly journals are those that are eligible. Those are great questions, by the way. So thank you for asking. I think I've got a couple of questions I noticed there. One was from Aaron. He asked, let's say you have a bunch of five theories or explanations and the experiment's going to distinguish between them. That there isn't any prediction, he says. I think that's a really interesting question because from a scientific point of view, I think what's relevant is the relationship between each of those explanations and the predictions they make. When you say there's no predictions, that means you don't have a prediction. But I personally have this rule. I never end an introduction by saying, I predict that because I regard that as scientifically irrelevant. I say based on this theory, this is predicted. But based on that theory, that could be predicted because that's what's scientifically relevant. So what I predict, who cares? So yes, that is perfectly suitable for a registered report. You say theory one predicts this, that theory two predicts this and theory three predicts that, that's fantastic. That's precisely what we want. And in fact, what I think we should all be aiming to do in every registered report we could do. Do you think David's theory of consciousness is not the relevant one to include there? Hopefully. But it was not. What was it? Was it another question for me? I think that's... Just wanna say something about qualitative methods. Yeah. I think one of the very first, on PCIR, one of the very first papers we accepted at stage one gave impuncible acceptance. Two was a qualitative methods paper. So from a registered reports point of view, that we can actually do that. But now from the point of view of this particular initiative and what the ASSE wants, I'll ask Lucy and David. So qualitative methods. It's not out of scope. I don't quite anticipate it, but it is certainly something that we would consider if you're proposing a qualitative methods study. I think that should be submitted at least as the pre-submission inquiry and we'll take a more specific look about both the scope and methods. And if it's something that can be published as a registered report, as Elton said, that is feasible. And if the discipline scope looks appropriate, yeah, we would accept. We would be very happy to have those submissions. Lucy, do you have any? No, I think I agree, yeah. And Razvan had a question, does he need to do a pilot if he's, let's say, following on from a particular study with some modification to test and theory? You don't need a pilot to do a registered report. But because you're committing yourself to a certain analytic protocol, I personally do recommend it. So you've had some experience in playing around with that sort of data, but it's not necessary, and lots of papers don't have to find it. Yeah, and I would encourage sort of pilot as feasibility. It's not a requirement. But as the submissions are, both asking for financial support and publish the final results as a registered report, some sort of demonstration that the proposed methods are feasible can be done and there is an ability to do so. Those would make a stronger submission. I think there are a couple of questions about the timeline and duration of projects. Maybe David, you can comment on. So the funding is available through 2024, but we obviously the, you know, we anticipate that the submissions will primarily occur this year. So July is the ASSE in-person conference. And as Lucy mentioned, we will be there to, A, answer more questions about the process and raise more awareness about it. And so we do anticipate that in the few months following the July ASSE conference would be the main window for submission. So it's a rolling submission window. So if you're interested to have July, August, September would be a prime time for submitting manuscripts and proposals. And through the registered report process, in-principle acceptance, sometime later this year or early in 2023 would be most appropriate and expected. Can you remind us what's the latest people can submit if they wanted to? Right now the... Is it the end of the year? You remind us I have to double check now. Yeah. And also, Zoltan, what's the... So you said that for the registered report itself, then there is a kind of guaranteed timeline with the submission, at least for the first round of reviews. That's for the schedule. That's a particular sort of submission. Okay. The scheduled review submission. Do bear in mind that as author, you are guaranteeing that you're going to get this done by the time window we give you, say five weeks. If you don't make that, then all bets are off. It goes back to being an ordinary submission. So you gain from it in terms of the quickness of the turnaround process, but you're committing yourself to getting something written up in a certain time. So there's cost and benefits to you. Summer 2023 is the absolute deadline by which we would... Close the acquisition. Yeah. Or until funds are exhausted, perhaps before then by the end of this year. Thank you, Liv, for reminding me. Someone said, can they do estimation? Just to reiterate what I said, I did focus on testing hypotheses, but also mentioned estimation as a possible thing to register for. So yes, you could do a register for estimating some relevant grant. And in kind of more kind of technical question as well, Manuel Hausch is asking, what proportion of funds is only distributed at the end? I think the funding will come at the time of the acceptance of the register reports. Am I correct? You are correct. I'll answer a couple. Are there any humanity-specific journals too? There are no humanity-specific journals, but several of the interdisciplinary ones that are PCI-friendly are likely to have humanities relevance. Someone said on the figure that some funds are distributed only at the end. That's what I thought I read as well. At the end of the stage one review process is when funds are distributed. Was there, and if I convey any mistake in information, we will correct that in writing. I think it meant at the end of the reviewing processes. But that means all funds are distributed once you've got IPA and it's been awarded. You can get all the funds as you understand it now. Another relevant question. How many applications can a researcher make? There's no pre-specified limit. That's a very good question. We have no regulation against multiple submissions. I would encourage focusing on one submission, at least one submission at a time. Someone asked about non-physical theories of consciousness. In the end, it has to fit the journal remit. That's really a question of the journals that we listed there. What would fit for those journals? I think again, it's the registered report constraints that will dictate mostly and the scope of the journal that will dictate the feasibility. Am I correct, Alton? Yes. I mean, on psychology of consciousness, I am dealing with a parapsychology, so to speak. So, psychology of consciousness does deal with testing non-physical theories. Yeah, so look at the journals that we've listed there and see what things they publish, along the lines that we think of. And I don't know if there's anything more that from Templeton or across ASC point of view. No, I mean, it's mostly within kind of the expectations of what's publishable or what's presentable at the Association for the Scientific Study of Consciousness. The remit of the work conducted within that community is the remit that we are focusing on. Yeah, so parapsychology doesn't actually fall into that. I think that would be probably not. Yeah, I agree. I don't expect that to be so. There is another good question about... Sorry, so systematic reviews and meta-analysis approaches, would that be appropriate? I think that would, but again, only in the registered reports format, right? I think is doable, but probably less common, Alton? Exactly. There have been registered report meta-analyses, but far less common, exactly. There is an issue, which I didn't mention, that in PCIR, with registered reports, we have different levels of bias control. So the standard registered report, the data is not being collected yet. And then you get your approval to run the study and collect the data and then you analyze it. But we do at PCIR deal with other cases, where there is the data out there, but there can be different degrees of control over whether you've seen it yet, whether it's available yet, whether you could possibly have time to analyze it in the way that you're thinking of analyzing, which would be relevant to meta-analysis. So from PCIR point of view, it's fine to deviate from the strict criteria of the data is not there yet. Yeah, so data could be there yet, as long as there's some level of control. And then we judge what that level of control is and give it a bias rating accordingly. But I don't know if from the ASSC or cost point of view, you want to say anything about any issues to do with that. I think again, following the guidelines of the journal and the registered reports would be, I think would be fine for the ASSC, even if the data is already exists. There's one question, David, about for what purpose can the fund be used? So is it possible to cover researcher PhD salaries? We encourage most funds to be participant support or research material support. And we discourage a majority of funds to be used for salary, but it's not a requirement one way or the other. So PhD salary probably would be difficult. It's permissible. Okay, but permissible. But research assistants, for instance, would be fine. Let's see. Oh, yeah. And there is another question about timeline as well, David. If the pre-submission is positive, should the stage one journal review with in principle acceptance should be also completed at the end of 2022 to ensure available funds? So I think the idea is that the submission for the per registered report should be happening hopefully in 2022, but then obviously you still have more time. The pre-submission inquiry takes two weeks. That's the expected maximum for that. That is a relatively quick check so that we can have accounting of budget expectations and the appropriate scope. But most of the time can and should be during that in principle or rather that stage one review process. And so focusing on late summer, 2022, early fall, 2022 for submission of stage one proposed manuscripts is the expectation. But if the review process would take longer, that wouldn't affect the availability of funding, hopefully. It should not. And that's what we will be keeping an eye on closely during that timeline. We are coming up on time, meaning at time. We will be saving a record of all the questions that have come in and double checking that we've answered them all correctly. I'll be double checking that question on the fund distribution and just to make sure that we're not distributing any fake news. So if any additional instructions will come either through your email and or the COS website. Thank you everyone for attending. Thank you our panelists and partners for your help and support. Thank you. Cheers.