 Thanks, Andrei. Hello, everybody, and I thank organizers for the opportunity to speak here. And what I'm going to do is to elaborate a little bit on the topic that David mentioned and his keynote about mutation calling and try to join him in voicing the concern, the message that it's not an easy problem and the problem that's not actually completely solved as yet. And what we are all trying to do is to characterize the disease and treat the disease. And we almost take for granted the fact that we sequence samples and we obtain mutations, let alone more complex variations. And in the ideal world, that's how it is supposed to look as in this figure where we have the upper panel is normal. The panel at the bottom is tumor. And I think everyone would agree that this is a very clear example of somatic heterozygous mutation. If we could get our sequencing to this point, then probably I wouldn't be giving this talk and we just sequence, read out the mutations, done. Now, there are many other examples where the outcome is not as clear as in here where we have quite a few issues in sequencing. First of all, the colored reads that indicates that they're made is somewhere on a different chromosome. And note that this occurs in normal as well. So most likely it's some mapping problem. We are hitting some region that has highly homologous region elsewhere. So we are not that certain about the mapping itself. If you look in the tumor, yes, we have a few reads that carry an alternative allele. But somehow all those reads go in one direction. This alternative base always close to the end of the read. So when you think about all those things, you start growing suspicious. And that's pretty much what we are all trying to do with all those mutation colors, to separate good and reliable events from all kinds of noises that could arise at the library construction level, at the sequencing level, at the mapping level. And the types of errors we are considering is pretty much no event when just the sequencing is noisy. As in this example, just ends of the reads are all Cs and accidentally it may look as somatic mutation. And actually every base is at risk, in a sense to. We could have the systematic artifact elsewhere. Alternatively, we can be over-calling. We can have, especially if we have low coverage and normal, we just do the random sampling. We might be unable to see the alternative allele in normal and erroneously call this germline event in the diagram too as somatic. And given all the complications that were discussed here many times, like presence of tumor in normal contamination, if it's an adjacent tissue. It's really hard to tell in this case. Is this normal germline event with just sampling bias? Or it's actually normal with little over-sampling. It's under-sampling of germline allele or over-sampling of contaminating tumor allele in normal. So we don't know. So this whole project within the TCJ, the cross-center comparison was initiated with the goal of comparing, evaluating and improving the mutation calling pipelines between different centers. And it's set up as selecting a set of reference samples and everybody runs their pipeline on this same set called mutations. Compare and contrast all the callers against each other. And it has multiple benefits and just one of them in particular, you can think that it's relatively easy when it can be painstaking to go through all your calls, manually review them, but at least that way you can weed out false positives. Now things that you didn't call, you don't know where to look for and it's extremely helpful when someone else made a call. You can go to that particular site, review it, ideally validate, of course. And that gives you the idea of your false negative modes. But in general, at the end of the day, of course we need validation data. We cannot just look at those sites and make decisions based on our emotions. So what we were trying to do, the proposal we are having here is that RNA-seq can provide an extremely useful vehicle for validation, for many reasons, including technical ones, such as now RNA-seq data are going to be produced simultaneously in parallel with DNA sequencing, which slashes probably three to four months that would be required for targeted re-sequencing after the initial round of DNA sequencing is done and mutation calling is performed. So from the third phase of this cross-central comparison exercise, we selected 20 lung squamous TCGA samples that also have RNA-seq data generated at UNC. And just like all other data sets in this cross-central comparison, those samples were analyzed at Broad Washington University, UCSC, and Baylor College. So we compare them and try to see if it's feasible to use RNA-seq for the validation. Now, just to start with what we are looking at and this is very similar to what David has shown, we can start from simply characterizing the mutation calls generated by different centers. Sort of an agnostic view, we don't know what is true, what is false positive, what is false negative for each caller, but at least we can see that yes, there is a decent overlap, so we agree more often than not, which is great. And there are some calls unique to each center and I'm showing only three centers here, I'm really not good at thinking in four dimensions, but the overlaps with Baylor are very similar, you can slice your data either way and the result will be qualitatively pretty much the same. Now we can try to go dig a little deeper and see what are the characteristics of those calls that are shared between the centers or that are unique to each particular center and again to keep things simple, I'm showing here only the calls from the center of the diagram, those 229 that are shared between all centers and three years that are unique to each center. I'm not showing pairwise comparisons just to avoid making the plot too busy, but even in this case you see that there is a central black part where everybody agrees and the unique calls made only by a single center, they do show some distinct patterns, you can see that Broad for instance, blue points tends to call at lower coverage and at lower allelic fraction and WashU for instance does go into lower coverage, not that much into lower allelic fraction but at the same time they call more events at, sorry, more events at a higher allelic fraction which again without validation data we don't know whether it's true or not, I can only tell that this particular sample has purity 0.6, so, but it doesn't necessarily rule out those events. Okay, so just another view at the same data actually is to try to see what's the distribution of different statistics, so again I'm just conveying the same information as in the previous plot here just in different ways, so what I'm showing here is depth of coverage, gray bars is depth of coverage for that center part of the diagram where everybody agrees and blue bars here are calls made only by Broad, so again it's biased towards lower coverage, same applies to WashU, so they also go to low coverage, UCSC is more uniform here, so just some information we can glean from this straightforward comparison, we can also look at allelic fractions and again see that yes there is some difference, there are some parts of this configuration space into different callers venture trying to make more calls. It is somewhat instructive to see what callers are telling about themselves when they provide the quality scores and that's a good sign that these are for instance all the quality scores reported by Broad caller for the center of the diagram and these are the quality scores for the calls that are unique to Broad, so we are making more calls somewhere where nobody did but we are less sure, which is sort of intuitive, again one would expect that where everybody agrees this is probably the most reliable set of the calls. So this is great and if you go into manual review and start looking into those calls made uniquely specifically by any given center, some of them look like noise or at least they're questionable, most are convincing, I will show a couple of examples, this comes from Broad where again upper panel is normal, middle panel is tumor and I'm just jumping a little bit ahead of myself, the bottom panel is RNA-seq and you can see that it does look like a somatic event except that there are many, many mapping qualities, zero reads in here, so there is definitely some, there are issues with alienability of this region. In RNA-seq the reads are shorter here and you see that not a single read got aligned with non-zero mapping quality and none of those carry the mutation. So it's a very questionable one. Now this is an example of Washington University call where again it's really hard to tell, you do have a couple of reads here in the normal carrying the same alternative alleles, it's really hard to distinguish whether it's sampling error, whether it's tumor in the normal or it's really a little bit under sampled germline event. The other way, this is just apparently a blunder from Baylor College, there is an error mode that occurs in couple, maybe three or four of their calls where clearly germline event is called a somatic. So some things are better questionable but mostly they are fine. So we need a lot of validation data to compare those tools. And again our proposal is to use RNA-seq as a validation set. It's not completely orthogonal but at least it uses independent library construction, different protocol. It is though the same sequence in technology and more often than not it's even the same aligner. So if you have some mapping errors due to mapping artifacts they may carry on to RNA-seq as well. Now what do we call a mutation? What do we call a validation event? One way to do it and extremely conservative way is to just do de novo calling in RNA-seq which is probably an overkill in this particular case. We already made those de novo calls from DNA sequencing. We were conservative and now the question is what's the probability to make a mistake in RNA-seq? Not elsewhere but just at that location. We have a prior. So in principle we can set a much lower threshold. Basically if we just observe any or some evidence for an event in RNA-seq we can call the original call validated. Now there is a question of power of course and we have been showing this diagram quite a few times. Depending on what coverage you get in RNA-seq or in DNA-seq for that matter you have a finite probability to observe the event at all. And it's a function of allylic fraction of course. So if your original call has allylic fraction at 10% you need really many reads to observe it. And even more reads to be able to absolutely confidently say no it's not there. So fortunately allylic fraction is not a huge issue. It is an issue but it's solvable in a sense that there is a beautiful correlation between allylic fraction in DNA-seq and RNA-seq. Which means if we make a call in DNA-seq we can take this allylic fraction and we can estimate the power given the, whatever coverage we got at that site in RNA-seq we can compute the probability to observe it. And basically I'm running out of time and just jumping to some preliminary results. So we can look again using not even statistical procedure but using cut-offs right now and asking for just again since allylic fraction is expected to be consistent we just look for a presence of at least two reads in RNA-seq carrying the alternative allylic. And you see that if you increase your coverage threshold so if we ask for at least five reads in RNA-seq then many sites are covered. Some are validated 80% actually. Many of those will be probably lost actually the validation rate might be higher if we did it statistically because there are many low allylic fraction events here that just didn't get sampled. If you increase the coverage for RNA-seq you can see that the validation rate grows because now we definitely have less sites covered that well but the validation rate grows so now we start seeing alternative allele for those low allylic fraction reads. Okay and that's pretty much all the message that I was trying to convey here that we have this framework and thanks to leadership of this whole project thanks to David and Gady and Washington University for making those things happen having this setup for comparing and improving the calls. We are working on validating mutations and it looks like RNA-seq provides a really good platform for getting a more accurate objective view at the quality of the mutations and I think I will conclude at that point. Okay, thanks Andrei, are there questions? Quick one, how many of these centers have made their callers available publicly and how close are the R&D calculations to what you get from say fair scan or something from one of these centers? Our caller is, well we are working on publishing it. Is that right Chris? And I think Washington University published their caller already. Ours is still pre-publication and we're still working on it. Basically the plan is definitely to make everything available but it's not yet. WashU set were generated from bar scan and somatic sniper, both of which are available. What was the second part of your question? How close, I couldn't hear your real, how close is whatever you shown here to the results obtained with publicly available versions? We don't run other center's callers actually per se. We exchange the results but I presume it's very close. Sorry, on the issue of coverage at ARNA level and the fraction of validation, is it done on the same genes on artificially different coverages or on different populations of genes? Because genes at lower expression have lower coverage and that population could also be enriched in false positives. I am not sure, I would argue that it's not necessarily the case. Low covered genes unless there is something special about their context and nucleotide content. I don't see why they should have more false positives. This can be easily checked if you just get out part of the data and see. Absolutely, yeah. We did actually some, there is less data available. There are some samples, they are not part of this exercise. We didn't share them with other centers but we do have couple of samples where we did exome and whole genome sequencing. And if you contrast exome calls, try to do the same kind of exercise, trying to validate exome calls in DNA and whole genome DNA sequencing. Your numbers are pretty close. So it's not a proof but reassuring. So we do lose coverage for some genes in RNA-seq. That's a given, but I don't think it's bias. I'm clear, so low coverage in the DNA is definitely a problem. Low coverage in the RNA-seq means that you can't use it to validate. Is that reasonable? I mean, that's kind of the perspective. That's what my impression is, yes, some genes cannot be assessed by RNA-seq but it doesn't introduce an additional bias. Okay, thanks a lot, Nandray.