 I'll just start with some acknowledgments, first of all, to say thank you for inviting me and particularly to Federique for organizing it, and a usual disclaimer that I'm quite a cynic about personalized medicine but many people I work with are not, they might be very anxious not to be associated with what I'm about to say, so these are my personal views. And also to acknowledge the grants behind this, this is supported by the European Union 7 framework program under the IDEAL project which was led by Dieter Hilgers in Aachen, he surprised, well why exactly what has, what have rare diseases got to do with personalized medicine? Well the reason is that if ultimately we have to personalize everything all diseases become rare, so the whole process of personalization is essentially making disease groups smaller and smaller, so the deaf and the ears are connected. So I'm going to start with an example, this example is the sort of thing that should give us a case for personalized medicine, this is a crossover trial in asthma and I used to work a lot in the field of asthma when I was in the pharmaceutical industry and one of the things you learn was that there is a gold standard for response in asthma using the measure of forced expiry to your volume in one second which the FDA always uses, they classify an individual who had a 15% or more increase from baseline as being a responder and one who did not achieve 15% as being a non-responder, so this is a way in which you can dichotomize the world for patients having been treated into responders and non-responders. So here we have a diagram, two treatments are being compared, it's a crossover trial, every patient got treatment A and got treatment B and so what I can do is I can plot a response in this two-dimensional space here and I've drawn those magic lines of 15%, vertical line of 15% being the boundary for response under treatment A, I have a horizontal line at 15% being the boundary for response under B and I've used a little color coding here, those who are in yellow are those who responded under neither treatment, those who have the gray triangles responded under both treatments, those who have the blue diamonds responded under B but not A and those who the red squares responded under A but not under B and so what we could say is looking at this particular diagram and certainly that's what many people would include that there's a clear case for personalized medicine here, clearly some patients do better under B, some patients who better under A because it doesn't really matter which and so surely the task is for us to try and find out how can we predict which patient should be given A and which patient should be given B. I shall come back to this example later as I say it must be a green light presents in mustn't it so part one of the problem is the following I'm going to maintain we don't have reliable figures on the scope for personalized medicine so here's an example for your question for you to test what you think what you think subjective yourself should correspond to response for headache so Ms Smith had her headache reduced from eight hours duration to six she would have had a headache of eight hours under placebo instead given paracetamol she had an headache of six hours did she or did she not have a response to treatment Mr Jones on your hand had his headache duration reduced 2005 minutes to one hour and 55 minutes this is a reduction of 10 minutes or 8% so the question is did Mr Jones have a response or not well actually I can tell you that the international headache society knows the answer to this they know that Mr Jones had a response and they know that Miss Smith did not have a response I because the magic boundary is two hours if your headache goes on beyond two hours it's irrelevant whether it lasted two hours or five minutes or it lasted for three days that really makes no difference to your suffering whatsoever it's absolutely completely irrelevant on the other hand if your headache was reduced from 2005 minutes to 155 minutes wow that's a response that really is something treatment did and at the bottom when you have a look at most claims in doing people's responders and non-responders you have some sort of boundary like this so I'm on Twitter a lot probably not a far more often than I ought to be that's certainly my daughter's opinion think but I point out to my daughter that it is the function of fathers to embarrass their daughter basically one of the reasons we're put on earth so this is what I happen to see on Twitter I follow the coffin UK and they said only 10% of people with tension type headaches get a bit of benefit from paracetamol and I'm a very nasty person and basically as soon as I see a statement like that I regard as being a challenge to prove it's wrong but basically I accepted a challenge how can I prove that this particular statement the coffin put out is wrong and when you had a look at the data on which was based as follows they done a meta-analysis of 6,000 individuals they had obtained data from a number of clinical trials for 6,000 individuals entered into the clinical trials who had the headaches treated either with placebo or with paracetamol and they found the following that 59% had no headache after two hours when treated with paracetamol 49% have no headache after two hours of treatment placebo the difference between 59% and 49% is 10% 1 over 10% is 10 this is known as a need to treat you would need to treat 10 people with paracetamol to have more response than you would have under placebo and then the conclusion is well that must mean mustn't it that in fact paracetamol only works for one patient in 10 so just to sum up meta-analysis of placebo controlled paracetamol intention headache with 23 studies 6,000 patients in total and the outcome measure was simply whether a patient was pain free or not after two hours but in fact the conclusion about the proportion of patients cannot be drawn from such studies I maintain you cannot separate headaches and patients and in fact the dichotomy causes confusion and here's a motto for you we tend to be the truth is in there but sometimes it isn't and the dangers we will find it anyway and it seems to me a lot of people involved in big data analytics are essentially engaged in finding truths that don't exist and then marketing them as solutions for mankind so I'm going to do the following I'm going to create a simple statistical model the simplest of all possible models for survival of headache over time how long can a headache last I'm going to use the exponential distribution the simplest model for time to event and I'm then going to assume that what happens to paracetamol it has a proportion effect on the duration that people would have under placebo I'm going to use a form of cell analysis model very very famous due to us at David Cox proposed at a red paper in 1972 the proportional hazard model and I'm going to then use a dichotomy I'm going to use the international headache society standard of two hours and based on this standard I'm going to classify every headache as being response or anonymous spots and I'm then going to show you the results and I'll show you that we'll be tempted to include that some don't benefit and some do but that this conclusion is false so the numerical recipe is as follows I shall generate 6,000 headaches under the secret you must remember I am the God of this universe I declare what happens in this universe I declare how long every single headache will be and I'm going to use it during the exponential model and I'm going to use an exponential distribution with a mean of just under three hours 2.97 hours to be exact then each such duration we multiplied by just over three quarters that's to say the placebo duration will be converted to a duration of the paracetamol by reducing that duration by one quarter by making sure that the new duration is three quarters of the placebo one then what I'm going to do is I shall take the 6,000 pairs one of which is a necessity counterfactual because I cannot observe the same headache treated twice I can only observe the same headache once I'm going to randomly erase one of the 6,000 one of the members of 6,000 pairs leaving with 3,000 placebo and 3,000 paracetamol headaches and I'm going to analyze the data so these are the two survival curves here this black curve is the curve under placebo and the time was chosen by me precisely because at two hours it would yield probability response of 49% of 0.49 exactly with a black copper and sore then what I did was I took every such headache every point on this particular curve and I shifted to the left by a certain amount increased the rapidity with which headache would be cured I reduced the time to resolution of headache and I created the paracetamol curve and this is the red curve here and you'll see that by doing that what I've done is I've also created exactly the probability that copper and sore I created a 59% probability response but the point you have to appreciate is that the red curve is created from that curve every point on the black curve has been shifted a particular proportionate amount to the left to create the red dashed curve and this is the paracetamol distribution here are some examples of some particular headaches that I could generate from the 6,000 there and what I've been able to do here is because as I say I'm the god of this universe I can see what would happen under placebo I could see what happened with the paracetamol these headaches all fall on a particular line this is a line of equality the diagonal red line is the line of equality dash red line the blue circles fall on another imaginary line that's the line that gives you the benefit of paracetamol that tells you by what proportion paracetamol reduces every headache now what I do is I apply the incredibly stupid way of looking at patients dichotomizing them according to a headache having gone at two hours or not as if nature cares about these particular human definitions that we use and what you will see is that a certain proportion of headaches then fall in the area in which they would respond paracetamol only these in the top right hand area would respond to neither and the neither paracetamol nor under placebo is the duration of the headache left in two hours these down here in the bottom left hand corner would respond on both and what we have is we have a number of headaches that was for respond under paracetamol well so far I've only gone halfway in my simulation recipe I haven't erased one member of the pair I'm now going to do that because one of the members of the pair is a necessity counterfactual I cannot observe placebo and paracetamol respond at the same time if I run a power group trial I have to allocate a patient to either get the paracetamol or to get the placebo so this is the idea behind it here we have a counterfactual experiment one I actually can't run the red circle the red open circle is the response of the headache and the placebo and the blue square gives you the response under paracetamol that's what ideally if I was a an all-knowing all seeing all predicting individual as what I would like to be able to tell you about but I can't do that I can only observe one of these and so I randomly erase one of these and we're now left with this particular confusing diagram the diagram on the right is typically what you see from a power group trial and you will see the considerable overlap between blue circuit between red circles and blue squares and people often make the stupid mistake of assuming that that means that not all individuals respond he doesn't it simply means it's very difficult to determine response at all and typically what you need is you need a large number of patients in a power group trial for the mean from the blue squares to differentiate itself from the means from the red circles in a reliable way in which you can conclude that paracetamol must work for at least some patients otherwise you wouldn't see a difference between them this is the same thing on the log scale here we have a constant difference in the left-hand diagram the difference between the two members of the counterfactual pair is always constant I always reduce the placebo headache by one quarter in order to get the paracetamol one that will lead to a constant difference on the log scale and the power group trial I simply can't cover this particular information but power group trials is what we run that's primarily what we have to base our impression of who responds and who doesn't in clinical trials on this now these now are the cumulative distribution curse the black one for placebo and the red one for paracetamol that's actually empirical plot on top of the data point that I got and you will see that it fits my theory very well and that means that it also produces pretty much exactly the response the coffin saw and what that means is that my theory is pretty good or either I made the same mistake in my simulation I made in my theory but I'm pretty confident that I have got here an economical explanation of the coffin collaboration results I want to make it quite clear I don't know that this explanation is true in fact I can even go further than that I can say I suspect it's not true I suspect it's far too simple and the reality is far more complicated than that all I'm saying is that nothing in the data presented in the coffin collaboration review tells me it's wrong for all anybody know me you them this is right and this corresponds to every single headache not just every single patient every single headache having had its duration reduced by one quarter by by the patient having been at paracetamol rather than placebo so these are the statistics for response under under paracetamol and placebo rather I get for this simulated example a mean of 0.48 remember the problem found 0.49 and I get for the simulated a paracetamol results mean of 0.459 which is exactly what so the simulation mimics exactly what copper found it does not justify any conclusion that only some of the patients benefit from treatment so to sum up they are consistent with paracetamol having the same effect in every single headache it doesn't have to be the case but we don't know it isn't the combination dichotomy is and responder analysis has great potential to mislead thinking of things in this particular way potentially it involves naive use of causality and it's a shortcut to thinking incorrectly about response responders are such as assuming that because some patients responded and some don't that this is an argument for personalised medicine to give you an example here we have the prime author of the current problem that the United Kingdom faces the mover behind Brexit a former prime minister David Cameron and this is what he has to say about progress in genetics the agreement will see the UK lead the world genetic research within yours I'm determined to do all I can support the health and scientific sector to unlock the power DNA that's the sort of things that politicians say power of DNA fantastic isn't it turning an important scientific breakthrough into something that will help deliver better tests better drugs and above all better care for patients really rousing stuff but we can go back further we can go back 20 years 1997 this is so Richard Sykes fellow of the World Society who was at the time the chief executive officer of the welcome a major pharmaceutical company now seemingly worried about the effects of exit by the way we have him and he was also about to become rector Imperial College in London one of the great University of the Kingdom although as a former UCL man it always pains me to have to say that sort of thing and he's talking here about genetic tests individuals respond paper to the drug based on their genotype smaller more effective clinical trials we all know that has happened anybody who looks at the bill for the money that's been paid by the pharmacy industry for clinical trials the first thing that strikes me is wow it's all got a lot cheaper since 1997 hasn't it an individual patient be targeted with specific treatment personalized dosy regimens etc etc here we have a paper not so long ago from Shork in nature 2015 and what he did was he took the 10 most best-selling drugs in the U.S. and took the industry in other words exactly the same sort of figure the carpools reporting for the headache data and he interpreted them straight away and it's a mistake he interpreted them trade way as telling you that between three and 24 people respond are needed to fail to improve the condition of three to 24 people in other words here for example we have a number due to four that means one person benefited and three other nonsense it simply does not follow a number units treat does not tell you this for the very simple reason I told you the example I gave you in the in the headache data in fact not only are these data do these data not tell you what I think they did but they're not even correct in his terms either for instance at their discusses not a treatment for asthma here this particular figure he quotes is for chronic obstructive pulmonary disease which is not the same disease and here actually the author for symbiotic depression the particular trial here I think of the method analysis the number needs to treat as far as I can tell was six to seven according to the source and not mine is short so he's a data that people don't really own in any particular way they just put out there but then subsequently people like me getting probably irritated because every time I say well you know where's the proof that we really have this got this vast proportion of patients don't respond they say well haven't you seen short major piece that's why I said we use a house here what I don't understand is why prove killer case but have a look at the FDA document of 2013 paving the way for personalized mention by the way they also considered a sort of headache a migraine is not quite the same as headache but some similar let's say in some of the symptoms and what they have here is they have another number of particular diseases which they're categorizing according to the number of people don't respond 38% don't respond in depression 40% not in asthma 48% don't this is in migraine apparently and so forth 75% in cancer 75% of people don't respond in cancer well is that a mixture of all cancers because that's a very strange statement to make that in that case if it takes a particular cancer thank goodness for pharmaceutical breakthroughs what we can now say is that the five-year survival rate I think is somewhat in excess of 95% and a high proportion of patients can effectively consider themselves could if we take a low cancer not so good so actually we already know this personalized in that particular way we already know that you don't necessarily treat testicular cancer the way same way treat cancer and we also know that the survival prospects are not exactly the same for these two particular diseases but here we have the FDA the premier regulatory agency in the world putting out these particular figures but the first thing I want to know on seeing something like this is how do they know how do the FDA know that these particular things are true well the FDA actually tell you they tell they got it from this particular article trends in molecular medicine now I have to give the FDA some credit here because here you'll see that the diseases are sorted alphabetically and the FDA have learned a good presentation result and that is that you should support by you sought by importance are not alphabetically so let's give the agents some credit for that particular thing the second thing is the second thing is that they've actually decided it is more important to concentrate on the need rather than that can solve successfully so what they've done is they subtracted these particular figures that you have here the efficacy rate they subtracted it from 100 so what is a 30% efficacy rate becomes a 70% failure to have efficacy rate so let's give the FDA the credit for that but what is this in the middle here we have a group of Irish dancers we're not allowed to put their head hands up and here we have a group of Scottish dancers who went out to around like this and this is supposed to tell us something but in the words of meatloaf two out of three ain't bad so I suppose I have to give the agents some credit but I'm here I'm interested about this how did spear he's cut see and how did they know what they knew and also why why is the FDA in 2013 citing data from 2001 to tell you what the problem is with the US pharmaceutical industry because basically does the FDA not think there's been some progress in 12 years if they don't then what on earth are they therefore how we should simply abandon drug development altogether but actually you can find out from spirit you cut see and how's where they got it but they get it from well here they studied the positions desk reference they studied the position desk reference they don't tell you how they send it there's no way you could take the position desk reference there's no way that you could actually check the figures that they give you a correct but I can tell you one thing although I'm no physician the position desk reference could not possibly tell you this and as far as I know where all attempt to try and establish the basis for the claims about the report of people to respond and don't respond they always end in something like this and these are what may be called zombie statistics they simply refuse to die you can talk about them as much as you like you could try and put a state of their but whatever you do with with the zombies you blow them the pieces or whatever something like that you can try it but you'll always find they come back again somebody at some stage in the future will again cite this FDA document for you is it gospel truth but this tells you the proportion of individuals who respond and don't respond and so what I say is that not only are the claims made by short on the one hand and the FT on the other not right they're not even wrong you wouldn't even know exactly pin down what it was made to say that you could say it doesn't mean that and even if the as I say even do a right in some numerical sense they wouldn't mean what does it mean and so here's a nice quote from big Reeves eighty eight point two percent of all statistics are made up on the spot and basically I think that's more or less true of all of these personalized medicine statistics they're just made up on the spot basically but in the meantime those in the room as a philosopher was like to say there's a particular thing that everybody knows is it really the case that the major source of variation in the health care system is patients no it's doctors variation in practice is so large that it cannot be justified by variation in patients and in fact Brent James is a physician who played very interested in the teachings of W. E. Deming the guru of quality control became convinced that what you needed to do need to understand the variation the system if you want to improve health care and the truth is that we're not even doing average medicine well because even where we have established what an average guideline should be you will find that many different positions different approaches to treating the same disease and they can't all be right so here's an example for you these are tonsillectomy rates for England by local authority and these are elected tonsillectomies in young persons persons age under 15 and these are the raw estimates and 95% confidence interval so you can work out assuming some sort of cross on variation you can work out what the confidence would be some of them are not very well established but nevertheless it looks as if the rate varies by more than just chance and here I've done that I've done a shrunk estimate so that's the line of equality and the crosses here of the shrunk estimate to the shrunk estimate on the y-axis and the raw estimate on the x-axis and what you find is the variation from minimum to maximum is almost 5 I put it to you that that sort of variation is almost impossible to explain in terms of difference in need some doctors are whipping out tonsils far too often or some of them not often enough and I have my the suspicions about which way round it was so Frederick mentioned first job which is working for a national health service my boss at the time very tiny position for Dr. Cichl a man I really respected he came in one day and I said to him dr. Cichl why are you looking so pale and he said I have to explain to some parents why their daughter died I said why did she die and they said well they decided that she should have her tonsils taken out for proper lack of purposes but it was a good idea and the consultant did it privately and he then left and she started to bleed and the registrar panicked didn't keep records in the over transfusion she died he had to tell her about that and we've known about the variation of tonsilectomy long before these particular data here we've known about this since the 1920s the 1930s and nothing happens about it a lot of variation the system is doctors here is another example here is a paper from Sheffield admittedly from over 10 years ago called in the towel and they looks to 11 unit region to try and see whether you could explain the difference in attitude what happens to a woman who has a cancer detected by a scan what happens to her then and what they found is that there's a huge variation in the ratio of observed to expect a metectomy so they're just using the average metectomy rate according to a particular model for patients to characteristics and what they find is they find a variation of lower rate down here to rate up here so a huge variation between one one particular unit which is doing it far more often than the other particular unit I wonder how many of the terrified women who present to their doctor at this particular stage know that the most important thing that determines what happens to the next is just the luck of which who happens to be their doctor that is the thing which determines what happens to the next something as serious as that so Brent James advice the doctors in the mountain health is guys more importantly with the same way than what you think is the right way the same way having been something that all the doctors together having looked at the evidence-based guidelines have decided the right to do things and the way in which they run health there is they give the doctors the freedom to do what they want to but they have regular case reviews every quarter or something like that and all cases reviewed and gradually what they find is that people converge to a common policy so some notes towards possible solutions I like to think of sources of variation in clinical trials there's between treatments that's why we do them we hope what we think we're trying to find out whether the treatment is different between patients main effects of patients we know that's the case we know that not all patients have got the same forced expulsion by the volume one second at baseline they differ in terms of the degree of asthma that they have patient by treatment to ration this is what everybody is fixated on in individualized therapy the idea the treatment that was good to you might have been so so you can actually find the right treatment for you the fact that there'll be a variation response depending on a treat who you are and the treatment you're given and this one that people tend to overlook within patients if you measure forced expulsion volume one second in a patient on separate occasions you find you don't get the same answer it varies from occasion to occasion these are what you can identify in a clinical trial if a parallel group trial these three sources of variation simply come together you cannot separate them from each other if a crossover trial each patient receives each treatment in one period then in that case you can separate out the between patient variation and of course the treatment effect which are trying to get but the between within patient variation the interaction will still be together however if what you can do in treat patients with each treatment in at least two periods and that's possible for some diseases and it would be possible in fact treating headaches imagine you actually decide to study four headaches for each patient two under placebo two under paracetamol then in that case you can separate out all of these particular sources of variation so i'm going to give you a thought experiment imagine a crossover trial on hypertension patients are going to be randomized to receive either an ACE2 inhibitor or a placebo in random order and then we're going to do it again so in each pair of periods the patient will receive cibo once and paracetamol one and there will be two such pairs then we can compare each patient response under ACE2 to placebo twice so these are the sequences we could use in every pair of periods a patient will receive a or b and so here's one possible sequence a followed by b and again a followed by b here's another one a followed by b but b followed by a and so forth those are the four possible sequences to randomize patients to and that means that in every pair of periods we could actually compare the treatment effect so here is a plot this is one particular patient here this is the difference to placebo diatolic blood pressure in millimeters of mercury and this particular patient's had a very dramatic reduction 15 millimeters on one occasion on the second occasion nearly 20 millimeters on the first occasion otherwise what i've done i've done a color coding again i've used here a response boundary of five millimeters that's rather more modest and often used in hypertension but remember this is a difference to placebo not a difference to baseline it's five millimeters compared to cibo and uh the people uh i've i've highlighted in blue are those who responded on both occasions those highlighted in red are those who responded on neither occasion and those highlighted in uh in orange responded on one occasion but not the other and here is a clear correlation i can see from the fact that i've studied the effect of of the ace inhibitor compared to placebo on two occasions i can see that the response itself is correlated and if i try and in statistics just using the dichotomy which i don't like but it's accepting that for the moment then in that case i can calculate the conditional probability of observed response uh 838 individuals responded on the first crossover and of those 781 responded on the second so that means that the conditional probability of your responding a second time given i saw you respond on the first time is uh nearly 80 sorry it's 90 percent excessive 90 percent but here's the surprise perhaps don't give up hope just because you didn't respond on the first occasion doesn't mean you won't on the second actually for the same of those who didn't respond under the second on the first didn't the second here's another example one same setup different result this time there's no particular correlation and now if i have a look what i find is that response on one occasion is completely unpredictable of response under a lot here it simply is that you have an 80 chance of responding but the fact that on the one occasion is not predictively responsible another essentially the patients are just completely interchangeable to the guy so what's this this is the marginal distribution from one crossover from one crossover and now the question is which is applied to is it applied to the first or is applied to the second case and the answer is i don't know it was only the fact that i was prepared to study patients more than once under each treatment which enabled me to produce the bivariate plot it's the bivariate plot that enables me to identify the responders just having marginal distributions like this won't work a crossover trial on its own is not good enough a power trial is certainly not good enough you need to have repeated use of treatment in order to identify the next so all i can say is that you need to have a sufficiently rich design uh even repeated measures would help that requires suitable model actually the uh pk modeling school back in the 1970s people like the shiner they started doing this they started putting non-linear random effect models in an attempt to try and tease out personal response they know that you can do this with enough effort and enough intelligence and enough statistical know-how but the message that they've been largely ignored i know having worked at the pharmaceutical industry it's incredibly difficult to get marketing to go down the route of using pharmaceutical netics as a way of deciding what those patients should get so a complex design in asthma i'm not gonna tell you more about my first example i gave you my first example i invited you to think that my first example showed that some patients responded under b are not under a and vice versa this is actually a design i helped a trial i helped to design in my during my time at seba gaigi we were trying to develop a new formulation uh mta of for mottrol for mottrol is a long-acting beta raganus it's used for patients of asthma it expands their lungs it able to breathe more easily it increases their force of biology volume in one second um and i uh we came to the inclusion we needed seven treatments we needed three doses of the existing formulation that's one and three doses of the new formulation the formulations were identical in the sense that they both had dry powder in them and the molecule was the same but the device used was different in one case it was like a single loading rifle you could have one shot and then you need to reprime the whole thing in the second case it was like a machine gun you could keep on pressing and you get more shots and so it was useful for the patient it's convenient but they have a second one but registration had been the first one and so to try and obtain registration for the second one we're going to a full development program what we did was we had a parallel assay design we came to inclusion we wanted seven treatments but then we got the bad news from the mathematics there's no way we can persuade people to come into the cynics at the time we can give you five periods that's all we can do and so then a nice challenge for me not that difficult but easily the mainstream and the standard of the stuff I had to find sequences balance such a design I found the design with 21 sequences for each patient was given a subset of the seven treatments five out of seven in one of the 21 sequences I was worried that our trial logistics department would not be able to do this so I telephoned them this is in the days before email and I said what's the maximum number of treatment groups you can handle each sequence of your group and I received the answer 26 wow that's fantastic they can manage 21 is the most complex designed and ever been owned by Ciba Geige in terms of number of groups I was really what's the phrase the bullshit bingo phrase I was pushing the envelope here or whatever something like that anyway so I was really pushing the envelope here but then I suddenly thought why 26 why after the question why 26 and they said they're only 26 letters in the alphabet so what they had was they had a packaging system which would give a label to a sequence and know the right mind could ever want more than 26 group in the clinical trial so just a letter of the outfit was enough and so it just goes to show that sometimes the smaller detail is important in knowing how to run a truck but anyway we in the end we had 158 patients four six bar to volume one second these are the time points which we measured this is just one course of one particular treatment this is ISF 24 micrograms over time the average effect over time for a number of patients and these are the measurement intervals here and we use a UC of f of u1 as the main outcome these by the way are the comparisons to placebo under the model having fitted a model with patient effect period effect in addition to treatment effect and this gave us an incredible shock this is a perfect dose response 6 12 24 the new formulation 6 12 24 from the old formulation what you'll see is the 24 micrograms of the new formulation is not as powerful as 6 of the existing formulation that particular project was casket and I felt very proud I had killed a product dead I did that but at least I had a big part in killing that particular product then and I can tell you from having worked in the pharmacy industry it's quite hard to kill projects there's always somebody sufficiently creative to find some sort of subgroup in which it appears that the treatment might have had an effect and it will limp on its misery but I actually killed that particular project why are my colleagues dead so now I'm going to reveal to you what these two particular treatments were B was ISF 24 it was this one here this is A and that's ISF 12 and now I can tell you it's almost inconceivable that a patient could find that ISF 12 gave them better broncholation than ISF 24 there would have to be something really weird about those response curves if that was the case and I really don't think that that is the case so then the question is how is it possible for patients to respond under ISF 12 when they didn't respond under ISF 24 and the reason is the following first of all stupid bias on dichotomies secondly stupid counterfactuals comparison to baseline thirdly ignoring within subject variation ignoring the fact that on occasion individuals can can vary I personally do not believe individuals exist who would respond better in terms of broncholation to 12 micrograms for model and 24 so here's my advice don't let the label responder infect your brain a responder is a patient who is observed to get better by some arbitrary standards a responder is not a patient who was caused to get better by the drug subsequent is not consequence things change anyway to establish who really responds who does not use you need to work very hard and never ever ever use an arbitrary dichotomy now my daughter the one I embarrass on Twitter is a geneticist another way I'm about to talk about genetics is that you should not talk about genetics you can't even pronounce anything correctly you tend to say phenotype you see phenotype you tend to say I don't know locus locus I forget which period one of those things all those things I get them all wrong you get the vocabulary wrong I shouldn't talk about genetics but like me she suffers from hay fever and maybe that genetic I don't know she and I both suffer from hay fever and so I was really interested to see her medicine she was taking I mean my daughter is not a little girl anymore I have to remind myself of this sometimes she's actually 35 years old now and here we have here we have the advice on her particular medicine packet what you give what it says is first of all if you're giving this medicine to children then you need to weigh your chop you're either child of nine years of age way about 30 kilograms that's four and a half so these are the wonderful imperial measures that the British are looking for looking forward to getting this back again they really want to know how you know what is it 16 ounces in a pound and 14 pounds in stone and you know something like that it's really wonderful system and so do not give to children weight less than 30 kilograms do look at children under two years as well as strange how he said that a nine years car will weigh about 30 kilograms suddenly we have somehow two years in 30 kilograms together in some particular way and so what is what about the dosing now this is personalized medicine at the cutting edge this is where it really happens this is where the personalization really takes place here adults and children are 12 years and over one tablet once a day oh okay good now what about the the younger the younger patient here children of two to 11 years who weigh more than 30 kilograms very very obese two-year-old children what sort of dose these children get oh well it turns out one tablet once a day they get exactly the same dose of the adults that's the same dose of the adults and then we have a look at her children of two to 11 years away less than 30 kilograms do not give this medicine that ladies and gentlemen that's the reality of personalized medicine out there and if someone who worked at the pharmaceutical industry I'm always baffled by this if you came to marketing you said we propose one dose in and another for men they say on far too complicated far far too complicated to dose by sex merc have one dose for everybody they will kill us in the market if we come for something more complicated than this but on the other hand the prime minister's knight came talking about unlocking the power of dna wow that's no really exciting so finally I leave you this particular thought supply of truth great exceeds its demand thank you