 So people, if you can just come over here, that would be good. So well, hello everyone and welcome to this breakout session, for me it's a pleasure today to introduce to Lod de Bulf, it sounds kind of weird when you hear yourself by the speakers. So Lod is a medical doctor by training and has been involved in working for the pharmaceutical industry for more than 50 years, if I'm correct, 15, sorry, 15 years, not 50. He's currently holding a position of patient officers in Sevier and today he's going to explain us about what I think is a very burning topic for the patient community, which is the clinical trial design and how to select the starting points, especially because we are increasingly as patient advocates being involved in this pathway of clinical research. So Lod, my pleasure, the floor is yours. Thank you very much Alfonso. So let me indeed start again, so my name is Lod de, the rest forget it. What else do you want to know about me? Yeah, but all we don't have time, we take long talks, you know, know what you want to know. It's a good point. Why did I go to pharma? Was it the money? No, it wasn't the money. In fact, I was making more before I went into pharma. I was a practicing clinician. The reason I went into pharma is because I love change and I could not see myself to either stay in the same specialty for my whole career or to, I was a GP in a village and I mean you take care of about 2,000, 2,500 people. I just couldn't imagine just having a circle of patients of only 2,000, 2,500 people and always the same and having to go to the church and having to go to the local football club and being popular and always having, I just couldn't bear with that. So that's the honest answer and so I've been very lucky and I have continued my clinical practice for almost 20 years because I really love it and for example I've combined my vacations with clinical practice. It's not that I didn't love it and afterwards yes the money got better but that was not because I was a doctor but because I started climbing the management. It's a good question actually. Why do people go? In fact, my father didn't speak to me for 3 years when I did that because he was a doctor. I was the only of 5. Having done medicine, he wanted me to succeed him. He was explaining the idea that the only way you can be a doctor is to see patients from morning to evening and so he was very, very disappointed and for 3 years if he ever asked how it was, how are things at the factory but later in life then he realized that you can also do good for patients on that side and so we were surprised. What other questions? So yeah, so I have actually been almost 30 years, not 50 but 30, 30 years in pharma. I started in, I went from clinical to pharma in 1989 and the end of 2006 in beginning of the 50,000 I stopped. At that time I had been 5 years chief patient officer and we could talk about what that means but I was at a point in my life that I needed to make a very big decision because my mother was in a very bad situation, also medically and health wise and here I was going around preaching about patient centricity but I wasn't really doing it so I said okay you got to live up to that, stop preaching, start practicing so I left the job, I left industry with the ID to never return and I for 6 months did half time to take care of my mother who I'm very happy to say is still alive and she's 84 just 2 weeks ago and the other half time I did something that I always wanted to do which is ONG, NGO. I started to work for doctors of the world and for the homeless people I gave medical consultations in the evening and in the winter so that means side of society the illegals but also a lot of refugees did that for 6 months and then other things happened and then actually a month, it's actually 6 weeks ago this today, 6 weeks ago I did join back to the industry because I created Corvallis because I was being asked to consult on patient and how to bring patients into clinical trials into the organization by pharma because I was quite well known and so I couldn't do that because I had no way to actually make build them so I created with my wife Corvallis and so then I had a building method but so Corvallis is asleep now because now I'm back into being an employee, anything else? No? All right so today we have about 15 minutes left and I'm going to start to clock so okay right to talk about clinical trial designs and there are several ways to do this in my until a few years ago before I really started talking a lot to patients I would have done of the doctor way which means I would have had a slide deck which I have at 67 slides and I would just have gone one by one by one and telling you everything I think you should know. I have in the last few years and certainly since I took my sabbatical I have understood this is not the best way it's much more difficult for me because I cannot prepare so well but I'm going to do what I did for myself I would like to know what you want to know and we start from that so what why did you come to here actually you're much more more people than I thought you would I thought all five people are going to be interesting so tell me I will write down the questions and I will try to start so and you can continue to ask but what do you want to know about clinical trials what is important for you to know yes sir how and also it's about endpoints and when you say how are they considered do you mean how are they created or how are they interpreted okay this selection of endpoints okay right good that's a very important one specifically okay it's a very good one yes please sorry drug being tested yes what do you want yes what's the question I'm not sure I understand the question so I need to clarify so you want to know at what point in a life a drug gets tested or okay okay yeah okay what sort of drug is tested in which trial they're different types and when I guess that's that's all yes case history okay cases that's a very good case history is okay series as they call it yep all right that's a good question yes please absolutely yep that's a good one yes sir how are decided so how many times you give the drug or how many times you study how many times use okay yeah okay so those and administration that's what you're asking okay good there was one more person I don't want to stop on you yes sir you had a question selection of trials what do you mean can you say a bit more you mean for patients decide which trial they go or for by you how to select a trial okay wait I need to you try and clarify his question is a separate question is that what you want to know when to do a clinical trial so that's that's a little bit also here okay good and then sir you have another and I will get started that's a good one very interesting wow wow we have a great so where shall we start maybe we can start let's see all right I'll start with this one so what is the case history and then we start from there a case history is basically exactly what the word says it is the description of one case and I hate the word case because already we doctors reduce people to their disease now if you then call it a case you totally depersonalize all right so but anyway that's the way the scientists look at it so that means that the number of people in it is equals one and equals one and what you do is you describe everything that you know about the case how the history was what symptoms what you did and then you see what what comes out of that all right now the question is what can you do with a case history how big evidence is this because science and approval and reimbursement which means access for you is based on two principles and that's very important one thing you remember and that's the thing that the basis of science science is actually a belief system it's not like chemistry or engineering it's a belief system how does it work there are two things that you need to do you need to have representability okay and these are two difficult words and the other one is reproducibility what does it mean all clinical studies end up in a publication also they should that's very important okay and we can talk about that separately that's not a clinical design issue and what they do is they spend a lot of time describing exactly what has been done right and then they describe what how they measured what they measured and what they saw and what they think that means right and if the other reasons the other scientists believe that what you have written if they would done the same experiment it would give the same results then they accept it so it's a little bit like I invent a new recipe for baking a cake and I describe exactly what flour I took what butter what this what that and I put it together and I report that hey it was a cake that recipe if I publish that recipe to the point that others say okay it's detailed enough that I can reproduce it yes and if I do it I will get this result then people accept okay this is a cake recipe they don't all well they're not all going to make it they basic accept okay this is how you make a cake and then their next question is in science how do I make the batter for the cake and so science moves on the moment we've accepted something and to accept it we first need to believe that the result is reproducible all right the second thing we need to know is okay this I believe that if this patient came to me this case history with this history if I had done this I believe I would have the same result then that's a very strong case but is this patient representable that's the big issue and this comes back to the statistical point if that we are all unique humans right and so that means that by default a case history is not representable because I give you five ladies of the same age each of them diagnosed with breast cancer that looks the same on the x-ray they will have five different outcomes even if I give them the same drug and so this representability is the most difficult one especially in rare diseases which comes back to the other question right now the bigger the effect is that you will get because okay so here's here's I'm giving you simple formulas all right there's a person okay what does this link to if I add to this person the drug okay then this person changes right if I add placebo which is nothing zero then the person is the chain the same understood so all scientific studies try to do that they will take half of the patients and give them nothing and then half of the patient they give what they don't know and then they will compare it to and this and this okay clear I'll make this one to be it's easier for you to understand that's two different people that person Patrick and Bart okay Patrick and Bart yeah but that makes it too difficult to go one it's the same person but now he's had drugs so you're absolutely right because that's why we need this group it could be that this is not exactly the same so we'll call him Patrick later and Bart later because maybe the study is two years later and if you didn't do something the disease moved on so the disease is no exactly made the same nor the same thing that's why you need this control group so the bigger D is drug the less patients I need to prove this that's logic if if if for example I have a disease hello all right anyway you hear me this without the microphone but they need for the recording so if if if let's take a very bad scenario you have a very very bad thing and two years later there are no bees more because everybody is gone right then if you have to change it with my job or the doctor they now know the doctor they're no longer afraid or they started doing a bit of exercise right so there's a lot of factors that could explain what what my effect is and then I need lots of patients so that's how statistics work and so this famous p value what does it say it tells you what is the chance that what I'm seeing is not related to D but simply chance right if it's less than 0.05 what we really say there's less than 5% that what I'm seeing and thinking is effect from drug is simply chance which already gives you the message that even a p-value is not perfect science is not perfect a p-value of 0.1 because now you say it's only 1% chance that what I'm seeing is not you okay so that's what the p-values and so what what you what you therefore do when you make a study sample you start with this and you say okay this comes back to when is the drug test what do I already know about drug D from the laboratory from measuring animals and from previous studies all right and so that means what effect size can I expect am I expecting a 10% improvement a 50,50 or 100 and then I start calculating how many patients would I need to make sure that I detect this difference right with certainty with the p-value because you then decide the power and usually go for power of 80% and a 5% p-value you can put the bar very high I can say I want to be 100% sure at 0.1 p-value the problem is you need more patients than there are inhabitants of Europe because it's an aesthetic so in the end you're going to have to make a decision and of course any researcher doesn't like a negative study negative study is a study that isn't doesn't mean that the result the drug doesn't work a negative study doesn't answer the question this is very important sometimes you want to prove that the drug does nothing for example no harm a negative study is only a study that doesn't answer the question and a good study starts with a very very clear question because if you try to answer more questions in the same you don't know what you're getting it's like I take go back to my my recipe right for my cake let's say my recipe has four ingredients if I change all four a little bit more flour a bit less test and my cake fails I have learned nothing because I don't know if it's the butter or the flour so that means to do the study see what is really important I need to start varying only the flour I do another I cook it one more time with more flour and the rest I keep the same and then I see if it works and then I will find out exactly what the values of each you see so it's it's a bit of a difficult a difficult point so let's see what we have answered so this comes to the when can we try when can we test the drug we can test the drug when we have enough reason to believe that it will not do more harm than benefit there is no such thing as harmless things that we take in if I drink right behind each other without stopping which very four five liters of water chances are that I'm dead within the hour and that's water so it's just about finding that that balance between benefit and risk so what does that mean we have what we call in drug development different phases all right and and oh I'm gonna take this one I'll turn it off so so the first thing we try to do is the best way to avoid harm to human beings is to not study human beings I mean seriously right so it's quite quite simple and so that means that there is a lot that is called the pre I don't know I'm doing red I hate red it's like everything's not correct it's the preclinical phase okay the preclinical phase and that has different stages there's chemistry just looking you can look the chemist can tell you if for example something is extremely acid then you already know that if you're gonna give that people are gonna get burning stomach and stuff like that so there's chemical things you can do then you can look at cells right specific cells is this drug will ultimately work on the liver then let's take liver cells put drugs put them in a glass with lots of liver cells we add a bit of the drug and we see is it not harming the liver cells okay and so we systematically test some cells liver is a very important one brain is a very important one right what else they were muscle you don't want things to go wrong with muscle nerve so we test in the lab different types of cells and this is one of the things where there is a credible progress happening incredible right down the road here 10 kilometers is an institute called iMac it's the worst the world's largest chip research laboratory so the chips that are in your iPhones or something's all that have been invented 10 years ago there and they're already working on the future ones and so what they have invented is chips they have programmed the total liver cell in a chip the size of a nail so it's one by one centimeter and you can now instead of taking a cell you can actually put the liver the drug in and let it go through the chip and the chip will tell you about toxic code and you it's amazing because what you can do is you can have the liver has the active liver cells but there's also fiber cells in there there are some blood vessels there this so you can actually recreate on a chip almost an organ and then you can run the drug there if you guys want to see this go and there's a TED talk about the how we will maybe able to avoid a lot of testing in the future so there's that step two cells step three and one reason to expand cells is already to avoid step three which look I have a cat two cats I have a dog our fish died last week but of age I'm happy to say animals there are just certain things that you cannot test in cells for example mental status confusion right so you have a labyrinth and you let a rat run in there and after a while the rat remembers the name the way to get from where it's in to get to the cheese and the computer just measures the time from going into the race to the cheese then you give them a drug and it suddenly they start bumping the sides and taking ten times well you know this is going to be a drug that's not going to be good for attention right so there's things like a same for exercise you can test the muscles but you can put a rat in a machine like in the hamster turn measure give drug see if you can still run like that so there's all kinds of things like that and I also don't like to do it and I can tell you everybody tries to avoid as much as possible to work with animals academic research first of all it's very difficult to keep them because you need to go back to the formula you need to make sure that the rats are the same so there are you buy actually specific rats or specific mice or even specific monkeys which have been bred generation over generation so they are comparable otherwise not reproducible not represent at all so these two in principles always so then at a certain moment you feel satisfied that you you have chemical proof that this drug for example binds the receptor or binds the gene etc all right you've proven that it doesn't harm cells right and you've proven that it doesn't harm animals and to give you an idea we usually go 1,000 times higher in animals in dose than we will ever go in humans just a number to remember 1,000 okay just to make sure that we're on the safe side before we go into the human and that is what's called phase 1 or F I H which is called first in human now the question I have for you if you are making this decision so I you've looked at all these data and you feel convinced and it's not just the researcher looks at that the ethics committee will look at that also to make sure that that the research is not biased because researchers want the question but here's the ethics committee so what would you test in a phase one what is the thing you want to test the first time you're going to give this molecule to a person what do you want to test safety not efficacy it's not at all about efficacy phase one is purely about safety now how do you do it step by step which means you first find healthy volunteers who want to do this all right and it's not easy because they have to agree to come for three weeks or whatever how long the studies how long you want to observe the study the first the first one is always a single dose which comes back to the question that you asked before a single dose you basically give one dose which you believe will could work because it's no point testing doses that are too low to work but that is at least a thousand times below what you've given as a maximum dose to the animal so that gives you an idea of how we find doses right and you give one dose and you wait you observe you measure everything you can measure you measure blood pressure you are asked them to do tests for attention you measure everything you can for these different organs you do heart film ECG you know if it's a brain drug you'll do an EEG so you do that's why these volunteers they spend quite a lot of hours because it's a lot of testing and actually you want to see nothing so it's a lot of money with the purpose of finding nothing it's funny right so when that happens you have two ways depends what you do you either go to multiple to higher dose higher single dose so that means you put up the dose first or you go to multiple dose which means you repeat the dose those are the two options now how do you decide which one to go if you are very afraid of your drug you will always first scale up the dose so that you are because you will start lower you may not start at a thousand times below the end is you may start at 10,000 times below so you may choose to be absolutely certain that you go for safety and you may start below the dose that you know this dose will not work but I first want to know how it safety it is and then you go up to the dose it's okay because you are and you go that in steps so never the same person the first person gets single dose low second person to get single dose a bit higher third person and you always wait for at least two weeks until you really know because there could be a late effect and you keep following the patient for the whole time so these studies are generally called step-up studies which is time is here you learn something and you follow this patient you add another one you add another one now you've done three so now you start back with the same dose but you'll do three of them at the same time so you can go faster right so it's step by step and every time it's a medical decision the ethics committee has to be agreeing it's all written out in the protocol so that's phase one so at the end of phase one what do we know this thing is safe in the doses that we think work good then we go to step phase two after one comes to you could have guessed that so here this is what we call pock those so this is F I H because we love abbreviations it makes us look so clever doctors love every issues by the way pock proof of concept basically this is the question where you want to show that the drug is actually doing what you wanted to do and it can be very basic I have developed an antibody for a cell maybe for my allergy cell right and I've looked at if I dose the antibody high enough it's well tolerated there's no problem now I need to prove that it actually binds to that cell and to bind well it if it's something that I eat well I first need to go it needs to go from my stomach to my blood so I first start seeing can I see it can I see it in the blood because if it's not in the blood then like it may bind to the cell in in a test tube but if it doesn't get there so this is also where we start looking at how the but the the traveling of the molecule is in the body how does it go from in through your gut wall into the blood through the organ where it needs to work how it gets back out where does it go for being broken down and ultimately where do you eliminate it because we don't want to keep drug in the body because and doses and doses that's like for example mercury which is a toxic thing you know that in the environment it stays in us over our life we ever ever more mercury and if you come to a certain toxic level we know that's not good so we want definitely drugs that are not accumulated that they are eliminated and there are two ways you can get out main ways that you can get out of number one is through your pee pee and number two is through your caca very simple pee pee is kidney and caca is liver and the kidney and the liver are very important elimination we call this kinetics that's from the Greek do we have Greek people in the in the in the word kinetic we know this part like if you go to the kinesis it's movement it's the movement of the drug and because we love our abbreviations we call it at me absorption distribution metabolism excretion doesn't that look much more fancy basically it's how the drug trials right so phase two is an incredibly important face because until here you have studied maybe 20 30 volunteers right and here maybe 20 30 ah would you take patients or volunteers here what would you do both are right depends what you want to measure if I just want to measure that the drug actually gets into the blood from the gut I can do that with a healthy for all the tears except if the if for example it's a gut disease because it may well be that the gut in the patient is working differently from the gut in the normal so you will have to have a really good reflection whether or not to work with patients here or not right in rare diseases we go faster with patients because we know that they are so very different genetically or whatever right so at the end of this we now have convinced ourselves that is safe and by the way we have expanded this knowledge because here we also look at safety of course but we have also a clue that it should work right and this is where the very important question of dose is is looked at where's dose here so dose we've already we're on the one okay and the dose because you are going to look at the dose range and what you're trying to find is LED and that's not the little lights that's the lowest effective dose which means that you need to find a dose where you see nothing and the ideal curve that all researchers dream of is that you have the dose here and here the effect is that you see nothing then you reach your dose you see the full effect right and it stays there and it stays there forever and that's what the dream looks like there's never ever ever the case okay so what you usually have is what we call a typical s-shaped curve so you'll see a part where things begin to move like the blood pressure begins to go down the number of cancer cells begins to go down right and then there's another part where it's flattening off which may not be the total result it may be that the blood cancer cells only go down by 50% but no matter how high you go you don't find any higher or the blood pressure goes by to 20 and then it stays right this ah wait that's why I was gonna get that's why I now read the red one the one we're all very afraid of is the one where if you go up suddenly the effect goes down and this is only effect because now I need to put on the same graph something else if I find my black pen which I lost here because this is the wanted effect but it could also be the unwanted effect so the adverse events what we always would like to see is that the the same curves are here right that they are right there because if it if it already starts going up adverse events here to the maximum adverse event this is quite narrow how do you decide to wigs those to pick because if I pick this dose there will be a few patients who already have the adverse event even though most patients will be good so what we really hope is that there's no overlap between those curves and so the end of phase 2 is summarized in a table like that and then you really sit down and you okay let's say that this is the curve what would really happen is that doctors would sit down and look at that and say okay there's something happening but is it really bad is it or is it just maybe itch is it headache is it diarrhea or is it something really bad so it's not just the numbers also what it is and then they take a decision and for me that's something of the past the only person who can decide what is worthwhile is the patient for both curves because you are the only ones who can say look I will not take a drug until I get a certain effect right and you're also the only ones who can say I will not accept more than this or there doesn't matter the black one and so this is what's increasingly happening now and what is the future of research is that the regulators who will ultimately approve the drug will ask patients three questions number one is what is your burden of disease and your burden of treatment that's number two but number three is what is your willingness to take risks and trade off what are you willing to trade off I'll give you an example there was a drug which was used and because it looked good and then a few adverse events came up and they were really bad ones because some people died right and on a thousand patients maybe there was two people but the manufacturer didn't want to take that risk and the regulator didn't want to take this so the drug was taken off the market but then the community of patients started reacting very very badly because the ones who were the 998 were having a lot of benefits and they were basically saying look we have a disease that's a severe disease that limits our life expectancy and this drug so significantly increases that that we are willing to take the 2% and that was the first time ever that regulators change their opinion because the regulator okay and they put the drug on back on the market but they're very clear the requirement to totally inform the patient you have a 2% more chance of dying if you take this drug than if you don't take this drug and so every individual patient can make the decision what the patient was saying was stop deciding for us and if I look at my mother she will happily trade quality of life for length of life where she is she's 84 and so sometimes she even says look if the quality is not good anymore I prefer it than easier but that's that's her so and this is something where coming to the end point somebody said about endpoint selection what we are beginning to learn is that what we as scientists have always used as endpoints may not have been the complete picture for example most cancer drugs are looking at two endpoints it's overall survival so if I go back to our number one which is the basic this in the group that all all Peters are there more Peters at the end than in the Barts that's overall survival and the other thing you do if it is a cancer that will come back anyway you look at the time until the cancer comes back that is called disease free progression so those are the two things that come back but they are very crude because what come back can either be more aggressive or less aggressive and we it's just very binary because it's easier to do statistic with simple numbers so and the easy thing to tell is number of death there's no discussion about that even recurrence is very difficult to measure because you have to have x-rays and all of that and you need people and you give the same radiograph to five radiologists they may interpret it a little bit differently that's the human factor and so that's why science is kind of pushing back against the patient community for what they call the soft endpoints but they're not soft happiness and quality of life are not soft so the whole point is how can we agree because I was discussing two days ago in Basel with a group of payers who even after drug is approved which means it's it's the regulators they approve the drug basically saying this drug is enough safe and brings enough benefit that we want our population to be able to have it that's one to say the next decisions for the payers say okay and I'm willing to pay for it two different decisions two different authorities they often don't talk to each other and they also don't use the same data alright but it's even worse than that because for example the payer in the United Kingdom which is nice they're everything but nice but it's the National Institute of Clinical Excellence nice actually looks beyond the classical medical data so they won't just look at survival and disease free progression if you have good data to show that the quality is better the independence and all that the data is good and remember what good is representable and reproducible they will look at that the Germans Kramkenkasse refused that and he said I will only look at it if it's an improved mortality and I said I disagree because mortality is the cheapest point I just paid a funeral six thousand including everything the coffin and everything it's a lot cheaper so you're even stupid so stupid has appeared that you're avoiding what is the cheapest cost so we had a bit of a discussion anyway so let me see oh so why there is so there was a lot of disagreements but the good news is in the same way as the patient community is beginning to organize which is great this one of things of MP all these little organizations come together fortunately we also see the regulators beginning to organize which and the big ones were always EMA for Europe and FDA for the U.S. they're now talking and they're beginning to use the same criteria which is very good you're laughing but I can tell you when I start my career in so in the early 90s every regulator had even for animals different requirements for some reason the Japanese like mice right but the Germans for the same test would accept beagle dogs but not mice right and so this and then one would accept 10 dogs and two monkeys and the other one without two monkeys there was no arrange no agreement and as a company it meant much more money because you have to do all these different tests for all the different markets much slower and for the patient it means waiting a few more years to get the drug so 25 years ago something big start is ICH the international conference for harmonization and today it's great that at least the preclinical part of the dossier this part is now harmonized in the world so you need to do it one time and the Japanese will accept it here when it comes to patients are different for example Korea still always requests 60 Korean patients you can come with 6000 patients from anywhere else in the world they will want 60 from Korea the Russians want 80 from Russia the US one 300 from us etc etc but at least this part is already harmonized now to the endpoints which is after this and I still have to go to phase 3 but I'm not yet there there's a very good movement that has started in 2014 another abbreviation for you to write down and it's called ICHOM and that's the international consortium for the harmonization of outcome measures ICHOM right it was started not by pharma which should make a lot of you feel more happy because it's more credible it was started by a very big consultancy company you have like the Lloyds and Belston console one of those I'm not going to name the name academia in particular it was the Karolinska Institute in in Sweden and Haunt came very very quickly afterwards but then and who was a third partner with Peter Stefan who's a third oh yes it was the University of Harvard the business school because they wanted to have a more standardization it's this Michael Porter Michael Porter very well known business guru who basically started with the idea that the health care should be based on the value right and what is value in economic terms because he's an economist value in economic terms is the price of the desired result divided by the cost of getting it which is quite right and that also shows why it's perfectly individual if I tell you what's the value of this it'll be different for you because you will tell me the price will be the same I know the cost will be the same I will buy a seller to you for 10 but maybe for you it's a lot more valuable right now that means that the desired outcome this one here it's everything but desired outcome divide he said we need to harmonize outcomes and so it was founded in 2014 with a very ambitious objective they wanted by 2017 which is three years later to have agreed outcome measures for 50% of the world's disease burden that's huge and you know what they did it yes they did so you can go to I chom dot org and you can look at which diseases they've already worked and for all the diseases they have a same approach and I've been part for two of them to help it create they create a wheel and they basically say and this is total simplification there is this what the doctors find important there is this what the scientists find important is this what the patients find important and there's this so they have these always the same coming dimensions and then they start working with these groups to find out what is the best outcome here and then they put percentages say okay in this disease this one is going to be 56% this one 28 this 110 I'm not going to put the number because I can't calculate it's 100 minus these okay and that allows you to know how to study that disease and this is great because for the first time in history academics industry researchers everywhere can start adopting the same language when we say this drug has value then you have to say what does it do here and it could be that it only works here which means your maximum effects going to be 56 could be that you do 30 of the 56 here 20 of see so it's going to be different different drugs can still do the same but against the same reference and this is very important specifically because there is this patient dimension in there right now which wasn't the case before all right and this is where you and your community can add value by becoming an active player in these discussions I do not know and I don't have a life internet connection here otherwise we could go to the website and see which of the disease you're interested in are there and what it looks like because everything that comes out is public knowledge which is also quite good so it's it's shared by everybody all right let's see where we are because we okay the endpoint selection I've talked about that so today most research five minutes so today most researchers will still take the medical point of view and sometimes that's really stupid for example there's a disease for the strength of the muscles where you progressively use the strength of your muscles until you can no longer breathe because it starts in the foot and it comes up and then basically you suffocate it's really bad disease and what the test that the doctors had always used is can you still do the six-minute walking test but six-minute walking test is ridiculous it's not I saw that with my father who took two years on his way out and it progress there's there it's not a linear curve eight meters is not better than six meters there are certain things for example from three to two meters means you can no longer go to the bathroom and from two to one means you can no longer reach the seat and they are much more valuable than being able to go 100 to 150 and so this is where patients come in in saying what is the value whereas classically statisticians have simply said it's linear 200 meters is double as good as hundred is double as good as 50 and that's not true so this is where I him is doing the work to kind of measure because what the parents said of these children is as long as they can use their hands they are not socially isolate because they can still go on Instagram YouTube and all of that but it's when they lose the power of their hands which is way after they stop walking so they're really that's what we want you to work on but then you have to start the regulators to convince that they should use it and that's why an independent organization so poor all right no more time the drug testing which when I've done zero preclinical one and two phase three is basically the proof this is where you put your money where your mouth is you think you know the dose you think you know how it works now you need enough remember representability reproducible you need enough people to show that it's not a chance and those studies can go from ten to ten thousand depending on what but this is for the company the most expensive one because they usually very very big and when I say expensive you have to calculate that a patient of what we call a fully loaded study participant so if you take all the costs of the study and you divide by the number and is now today for small molecules between 50 and 80 thousand and for anything which is new like antibodies or proteins you're between 100 and 300 thousand for every patient so if you need to do a 200 patient study you know exactly what budgets were talking about so we talked about that case history back to the case history where everything started there is a scale and I'll give you I'll send it through Alfonso two things a website with a slide deck of the things I talked more structured from the National Institute of Health all right that's in the US and it's publicly available and it's made for the lay public and so you can take your time go back and look at the slides but I thought it was better to have the discussion you that's rough there's a second one is an article also that goes and explains the process so you can at your ease read it up and I'll give it to you and then you can share it to everybody who wants you can even put it on the website because I wanted to take public things that you can always share with everybody why was I there okay anyway ah case history there's one slide on there which shows a letter and it goes for the the strength of the evidence always with the two criteria representability reproducibility so there's a case history one a case series ten of them and then you have a cohort which is a bigger group of people and so all the different types of studies are on the letter and on the top still the holy grail is randomized controlled clinical trials randomized means that chance decided whether you're in group from Peter or Bart it's blinded so the doctor doesn't know you don't know which all of this is to avoid bias but those are the very expensive ones so cases clinic trial sample size we talked about administration dose how to select a trial that's one to we talked about here this is a good one I really like it coming up because what we see at the end of the study is sometimes very striking that we see in some countries we see better results than in others and when we start digging deeper we find false data and the FDA and the regular will always go to two things it's the one country or the one side that if you take it out of the analysis changes the outcome so with statistics you can find out which side was the most important contributor to what you see they will always take that one out and do the study reanalysis without that side or that country and if that changes the result then they basically will not approve so what happens in some countries in some countries access to drugs is a real issue and so patients are desperate when they go into this new drug right and so they know that if they reported a lot of adverse events that maybe the doctor will stop them and so they actually start saying something different do you feel better oh yeah so much better that's strange because your blood count is different but I feel so much better right and and it goes both ways as on the Africa side but on the same in Japan the culture is one of extremely deep respect to the doctor and if you report an adverse event to the doctor you're basically telling him he didn't do a good job so in Japan your arm has to fall off before a patient reports an adverse event where in France they will report lots of issues so there's a the interpretation I could talk an hour about all these biases but let me say that this is a real problem but in the end this may help that individual but it can hurt all the others because if the study becomes negative and remember negative means it does not answer the question because if you need a thousand patients to get there and if this side has 20 patients you now only have 980 and maybe you just can't make the statistics and that's the worst there is so it's a real problem and this is why it's so very important to work with patients to design the clinical studies so that the studies are doable to create the trust and to explain very well that if you report adverse events you cannot be kicked out of a clinical study unless it is really harmful for yourself the only one who can kick you out of a study is yourself in the first place at any time you have the right and some patients want to stay in a study because they want to get my their patients a doctor angry you have the full right at any time to change but also if the company the sponsor or the academic institution see something that they feel is too dangerous they can unilaterally kick you out that's true but the intent of both is the right now life is not perfect so I know there's no rule that can prevent this from up okay have I answered most of the question I think I have all right so you're going to get two references by Alfonso one is a slide deck just goes more systematically than I did and one is the article and if you have any other question just send it to Alfonso who can contact me and then I'm happy to answer all right I think it's time for lunch I agree that all information for clinical part of times except