 So, what happens after risk determination if you think back to the beginning? So you've heard a very nice talk outlining some of the basics of the IDE submission. So what I'm going to do is quickly sort of run through the rest of that flowchart I showed you at the beginning and just give some tips. And then if there's a little time, I just, you know, wanted to mention a couple of things about genomics and some of the things we've heard today. So just very briefly, so we've talked about these three, you know, being exempt or non-significant risk or significant risk. And so what does that mean if your study is exempt? Just very briefly, of course, at this point you know you can begin your study assuming of you have all, you've met all the other appropriate requirements you have in form of consent and IRB approval where it's required and things like that. If it's non-significant risk again, we come back to the abbreviated requirements that you've heard about and here's a list of, I think most of them. A lot of them are things you don't have to send, well you don't need an IDE submission and that's always the big thing people don't want to do. You have to, you do need to follow the other abbreviated requirements. Some of them are things you would do anyway like IRB approval and informed consent, monitoring in many cases. We do have requirements for records and reporting but you don't need to send annual and final progress reports. The really the only thing that I can think of that you would need to send into us would be any unanticipated adverse events so just as those arise. And in the case of in vitro diagnostics, those can be very difficult to identify even so often, you know, you won't even identify adverse events related to the test. But we don't need annual and final progress reports either in that case, so mostly it's internal paper, you know, internal record keeping that you need to do and a couple of other things. You are, what important point is it's not that you didn't need an IDE but you have, you're considered, once you have followed the abbreviated requirements or begin to follow them, that you are considered to have an investigational device exemption. So that still means you're covered under the regulation and you do have some requirements that I've just outlined. Now obviously if you are significant risk, you have to get, you have to receive an approval of an IDE submission on top of the other requirements and you will have to, you know, do annual and final progress reports. And so this is just a handy table that compares the two things, the two categories. And obviously investigator agreements is, well, that has to be taken into account in a significant risk study. So I wanted to run through the elements that you heard so nicely outlined before and just give a few tips based on, you know, a few tips because, you know, I think filling out, you know, figuring out what information to send to FDA, especially the first time you're doing an IDE but even later can be confusing. So you do need to sort of describe why you're doing the study but this is not an R01 so it doesn't need to be a full justification of a hypothesis. I mean more of a summary, I sort of think of it, it could be like an abstract or something but we want to know why you're doing the study and just briefly what the motivation for it is. A detailed description of the device under study. So now as you heard, we want to know all the components and how it works and certainly to the degree you can illustrate things in pictures and diagrams, that's very helpful. The previous studies, pre-clinical and clinical, and one thing is that this would include the analytical validation studies you've done, you know, and sort of why, you know, the evidence is leading you to put this in a trial. You don't, you know, things that aren't relevant that have been studied about the device or the technology that aren't relevant to the study are not as important. And this has to, we need enough analytical validation and information on the device that you're going to use in the study. Now one of the challenges here is of course that this, in genomics, the devices are changing rapidly and there can be many modifications over the course of a study. So the degree that's happening, you have to consider modifications and I'll talk about that in a little bit. The risk analysis, I mean basically we need to know how we are mitigating your mitigating risks and if the benefits outweigh the risks of the study. One thing, well, you know, there's patient monitoring and follow-up, inclusion and exclusion criteria. We do look at the informed consent document and the, in some cases the informed consent, like I can remember one study I saw where the informed consent document had nothing to do with the study that was submitted to us. So you know, there are, you know, it's nice to have another level of review in some cases. Sample size, number of study sites if that is relevant with the justification. Again the administrative tips. We have an e-copy program and actually, you know, the e-copy program allows you to submit things on CD and there has been a paper copy, if it hasn't gone away it could go away. So I would check before you submit something to see if you have to print out all the paper you heard about. One thing I would point out is that you should be ready to respond, you know, we have a 30-day review window. We got to make a decision within 30 days. That means that we will be asking, you know, if we need information, if we need to figure, get clarification on something, we're going to email or call you. You will specify a contact. You should be ready to respond to those and know that you may get a request saying, I need to know X and, you know, you may like need to get it to them tomorrow so that the decision can be made within the deadline. So just that's something to be prepared for so you're not taken by surprise and doesn't need an alternate contact. So if you're not available as a contact that somebody will be around to respond. And then if you're responding to deficiencies from a previous IDE or making changes in the IDE, you should provide red line copies so that we can see what has been changed. Formatting an IDE, some tips, I mean, first of all, there's the cover letter and there is no specific format, which I think is surprising. Well, I think a lot of people the first time go on the website and look for all the forms and can't find them. There are no forms that are required. But that means we can see all sorts of variations in what we see. And I would say just like for a grant, you know, we are going to read all that paper you send us. But, you know, my eyesight is going, you know, so make it readable, don't use tiny font. Make it well organized, make it logical sections and subsections. You don't have to, again, don't have to follow a particular format. But we need to be able to find the evidence. When you're reviewing a sizeable document, you know, that becomes very important. Annotate your tables and describe figures, use figure legends. You wouldn't believe how many times you just get a figure with nothing on it or tables and you don't know, you know, things aren't labeled. Going back to the table of contents with hyper, you know, going back to the idea of organization, a table of contents in the electronic form with hyperlinks makes it easier for us to find things. Or, you know, even without a table of contents, the PDF, if it's well bookmarked, different reviewers have different ways they like to find things. But these are all things that are very helpful. And what you're trying to do here is make it easy, reduce the, you know, make the review go more smoothly. You know, to the degree that you can explain things to us initially, we won't be coming back and asking questions. When you are presenting data, don't just present a table of numbers and maybe some summary statistics. Describe and interpret the experiments. You know, lead us to the result you're trying to, to the interpretation you're trying to say. I've also seen this where, you know, people just sort of present, you know, analytical validation, accuracy, here's the table, it's X, we don't know what it means, what the original sort of threshold was that was pre-specified. You may need to provide line data, not always, but sometimes. And so I think just like with anything else, a grant or a paper or anything else, you know, it would be helpful to have someone unrelated to the project read the thing and see if it makes sense to them. Because if it doesn't make sense to them, it may not make sense to us either. But on the other hand, I do want to say don't let that perfect be the enemy of the good. Which is that don't wait forever assuming, you know, trying to, you know, guess what it is we'll need and fill everything in. If we, you know, if you don't send us the information we need, we will ask for it. So I think that, you know, the consequences, and this is the benefit of interactive review, the consequences of not getting it quite right or not, it's not the end of the world. You know, it may take a little more work on the back end, but it's not, you know, it's not going to sink your IDE if you didn't get it exactly right. And if in doubt ask, you know, talk to the reviewer, talk, we have an IDE staff, I'll get into this at the end. We have a number of venues in which you can ask questions, and that's probably the best way. FDA actions on IDE. So you've submitted your IDE, it's been, it's day 30, and you receive notification that you are one of three things, approved. You can begin enrolling subjects once IRB approval is obtained. And when we, approved with conditions, so this means that you can begin your study once you have IRB approval, but you have 45 days to make certain changes, there could be changes in the informed consent document, other changes, they aren't going to be big things like analytical validation studies, but you know, oftentimes there are tweaks that FDA is going to request, but it's not going to hold up the study, you just have to provide that information. The third thing is, you may be, it's rare for IVDs, but you may be disapproved. And then if you are disapproved, you will receive a letter documenting the deficiencies, the reason for the disapproval, and you can essentially submit a supplement or a new IDE addressing those deficiencies. There's no limit to the number of IDEs you can send. Again, this is not a grant. So, you know, oftentimes, maybe sometimes people go through multiple, more than one cycle, but they will get, they do get to approval in the end, the, but still the disapproval is rare and usually it's for very major things. Now again, I want to reiterate these decisions are made based on safety, the risk as opposed to benefit, but when we're talking about benefit, we're not talking about having an effective IVD that has been demonstrated to be effective. I mean, in many cases, that's why you're doing the investigation in the first place. We want to know that it, again, operates well enough to protect patients within the study and that the study is not absolutely crazy thing to do. And that you've mitigated the risk of the study appropriately. The other thing I wanted to say about letters here is that we also recently added two sections called major and minor study design considerations. We cannot, we don't make a decision on the IDE based on the study design, but we can't help ourselves to make comments. And we do identify things that we think may be issues in the study design. And so we can provide those, but they are essentially an appendix to the letter there. For your consideration, you know, they shouldn't, they don't affect the trial, but they're just things we have identified and we want to let you know this is what we think about the study. After you have an IDE, there are a number of things you may end up submitting to us, obviously annual and final reports, amendments, supplements, which could be new studies using the same device that can be a supplement to the existing IDE. Or they can deal with modifications and it just depends. So just to discuss modifications, because I know this is an important point, in general, if you made a modification, you would have to submit to your test. You would have to submit or a change in the study. You'd have to submit that to us and we would have to review as a supplement. We would review it and approve it and then you could institute that. However, there are, there are different types of modifications and not all modifications are created equally. You know, what we're really concerned about with reviewing are major changes. Changes that are going to affect the safety of the trial, make major changes to the performance of the device. If certain types of changes that aren't going to be as significant, and this is language from the Code of Federal Regulations. The, can be provided to us in a five day notice. So developmental changes in the device that don't constitute a significant change in design, so on and so forth. The changes in the protocol that do not affect the validity of the data or information in the approved protocol, the scientific soundness of the investigational plan, or the rights safety and welfare of the human subject. So there are many changes you will be, you could make that won't, that will just require you to let us know within five days. And then some minor changes can be made in the following areas without, just in the annual reports, so not even a five day notice. If they don't affect the validity of the data or the information that was used to approve the protocol, again the safety of the patients and the scientific soundness of the investigational plan. Now, who makes this determination? Well, again, this is a determination made by the sponsor and probably the IRB in the context of the study. You have to make an honest assessment as to the significance of the change and then decide whether to notify us. And in many cases, if we find out and disagree, we'll ask you to submit the proper report, supplement or, you know, or supplement, depending. Now, just to close this part out, I just want you to remember that you know, this, and this is probably one of the most important points of this whole day, which is that we want to help you. And we've, over the years, have developed many mechanisms to do that. And the biggest one being the pre-submission process. Then the ability to be interactive. You can call or email us if you don't understand any aspect of our letters or other communications or requirements. And I had a couple of slides here that I guess must have dropped out on what just on resources. But I did want to point a couple of things out. So the pre-submission program covers everything. And we get pre-submissions from, I'm submitting this and I need to figure out what the right experiments are to really just like I've developed something and I'm not even thinking about a regulatory submission right now, or I'm not planning one right now, but I want to come and just tell you about my technology and we get all of those. It's a venue for you to get feedback. And the pre-submission allows us to respond formally, to consider the questions, and to assemble a team of experts that will respond to this questions. The other thing, we have a number of guidances and IDs. And I'll make sure these get posted. CDRH learn, device advice, you know, information on our website. The other resource I wanted to point out is our databases. Now for IDEs, we do not post any information, you know, but we do post information, what are summaries, review summaries, for devices that have received marketing approval on our website. And I often say this is probably when you're starting out and trying to figure out what to do, even though obviously those devices had to meet a much higher bar than yours will, if you have a similar device, go look and see what they did to get approval and it will give you a good idea of what FDA might care about in terms of validation. I think they're really, it's a really excellent resource. So, you know, and again, we'll make sure those links get posted for your use. So with that I will end this part and I just wanted to mention one other thing and I think I'm just something I've been thinking over over the, well, we've been thinking about for a long time that really came up over the course of the day which is that I think we've heard a lot of, you know, we've heard a lot of, first of all, I want to thank everybody for really great discussions and for being here. I think it really illustrates how important it is to be interactive in this process. We're dealing with a technology, you know, in genomics that is really paradigm-busting in so many ways. I think we're seeing, you know, return of results in a way that we haven't seen before, the ability to generate data and incidental findings in a way that we haven't seen before. And so this is a new ground for everybody. So, you know, I think, you know, you all post really great questions of situations that are arising that we have not yet necessarily seen at FDA but that we will have to consider. And obviously this will take a discussion with sponsors and with the community and, you know, working through things like this through any, she or I are in meetings like this are critical to us. And I think you should take heart in the fact that we don't necessarily have all the answers. We're in the same, everybody, you know, these are undecided questions. People don't agree. And this is something we expect to work out in the context of real studies and real submissions working with you and having that discussion with you. Now, having said that, I did also want to just very briefly mention because it came up a few times, the Precision Medicine Initiative. Because this is such a different type of technology and a different way of thinking but so critical to precision medicine. FDA has actually developed, you know, it's actually been involved in the Precision FDA in thinking about new ways in which we can approach this technology. So we've had discussions with the field about standards, analytical standards and that's something we're very interested in developing so everybody has a clear bar, knows what to do or how to develop tests without necessarily having to come to FDA. We've been very interested and have actually been working with ClinVar and ClinGen to better understand the use of genetic databases and sort of crowd source evidence and how to provide that evidence to patients and providers so they can benefit from that information and that work. And finally, Precision FDA, you also heard about those. So this is a cloud-based platform that we just started last December or released last December. It's only been around for about six months. But this is really a place where you can come and benchmark software, look at its performance, have a community discussion. And so we've been working with NIST to post reference files and we've been issuing, we just closed our second challenge, looking at benchmarking and measuring the reproducibility and accuracy of pipelines. So this is something I want to encourage all of you who are working on genomics or are working on programs to consider, you know, coming and talking to us about how it can be used because we do, you know, the more the science advances, the more we understand how to validate these and how these technologies operate, that will obviously be help us regulate this technology much more efficiently and it will benefit everybody else in terms of developing this and getting it to patients. So sorry, that was my unscheduled little sort of summary, but we can now, you know, questions.