 Ken just showed us how learning sciences can help improve education and how we can embed that inside of software. But ultimately right now educational software has not mastered the art of teaching. And I believe that we and our world deserve no less than black belt ninja teaching. How can we do this by learning to improve learning? Well what does this involve? We need to think about the educational needs. We need students to become scientists and leaders and for this they need rich educational experiences. But unfortunately we just don't have the resources to give all learners these type of experiences in the traditional way through things like personalized tutoring or individual intention. Now current educational technology is trying to address this gap through things like virtual chemistry labs or avatar debates. But it's falling short. Why? Well can you imagine a teacher that has to try to understand their students by pouring through data logs trying to hunt for subtle signs of learning? That's kind of where we are right now. So this is the first opportunity to improve learning. We can use machine learning algorithms to take the data that's generated as students do things like play an educational game and use it to automatically predict student progress and goals. This is going to transform the capability of these types of educational tools which are so richly needed. For example students offer working groups and we're going to hear from Justine how important that is. But when they work in groups it's really hard for us to tell how the students are doing. But what we showed is that by looking at how say a pair of students interact with a chemistry tutor we can automatically predict things like post test performance. Okay so we're starting to have techniques and tools to try to uncover even open-ended learning tools and to predict student progress. But is that sufficient? Is measurement alone enough? If we can accurately predict that a student's about to drop out or fail are we satisfied? Of course not. What we need to do is change education so that students don't fail. This means that we're going to need to move beyond prediction of performance to excellent education. And this is going to require us to develop tools that can take the data that's generated and use it to automatically give new types of strategies. We're going to have to move to powerful pedagogy and we're going to need to be able to have things that efficiently scale up to thousands of students data. Now doing so doesn't seem trivial. Historically the way we improve education is we run a classroom experiment comparing just two instructional strategies. But Ken just told you there are over 200 trillion instructional strategies and doing pairwise comparison of these would take us centuries. But what if instead we could run millions of student experiments in parallel? What we've developed is algorithms to take data from a single classroom study and use it to efficiently evaluate many different potential strategies. This allows us to do things like optimize games like refraction where students learn how to put to slice and dice different laser beams in order to feed spaceships. What was the result? We improved engagement and concept completion by 30%. Okay now speaking of strategies I'm going to ask you a question. If you wanted your high schooler to learn calculus from an excellent teacher or the best textbook from 1999 which would you pick? I'm guessing it's going to be the teacher. And this is likely in part because you're hoping that that teacher has been learning and improving the way they teach over time compared to the textbook which is still exactly the same as the year it was issued. Now unfortunately right now educational technology is much more like the textbook than a teacher. It's a static resource that requires manual intervention to change just like this wooden pencil. But what if we could have more diligent software? Software that actually learns over time just like a teacher. How could we do this? Well think about it when you use a web search. Internet companies use algorithms to automatically identify which of those ads generate the most revenue. And I think we need to demand similar excellence from our educational software. We can use the same ideas to try to figure out over time which is the best pedological activity to give a student. Leading to better learning instead of just better profit. So think about this in the context of an educational game. There are often many different parameters we need to set. Things like how long until we have a timeout. And so we can use these ideas to automatically set these parameters allowing these tools like battleship number line to get better and better over time. The key idea is that we can use machine learning to close the loop between data and instruction. Allowing us to increasingly develop better and better pedagogical strategies. This new software will continue to get better and better at teaching as it interacts with more students by learning to improve learning. So these examples suggest a path forward that we can use things like learning sciences and machine learning to create truly mastery software. And in doing so it's going to allow us to transform learning not just in the classroom but during on the job training and lead to a better world for all of us. Thank you.