 Good afternoon, everyone. My name is Ted Chisholm or just call me Mike, I think it's easier to pronounce and it's easier to make friends with me as well. And so as you see from the power program from the OECD, just a very brief introduction of what the OECD is about, it's somebody called a think tank but we call it an international forum composed of 35 member countries and then it provides a forum and plays for different policy makers and academic and experts in the field to exchange good practices and to inform policy making. And one of the things some of you must have heard is PISA, which we do a lot of data collection at country members as well as our partner countries economies. And so as I said before, the OECD is pretty good at talking with ministries, or to ministers or policy makers. And so what really interesting for me today is that how we can learn to talk to people on the front line who are really working on a subject, especially that I'm from the director for education skills, that our main purpose is not just to serve the ministries of education but also to find out teachers and educators and how we can use data to empower them. And so PISA, maybe some of you have heard of this program for international student assessment that it takes every three years and we collect information from more than 70 countries and then we do policy comparison of the performance of each education system and all the data is online, all the reports are online that people can take a look at it easily. But the problem is that it doesn't really go to the school level. That our school are not very familiar with data and use of data for school improvement and this is why we have created a new project called the PISA-based test for schools that we implemented test directly as individual school level, meaning that school now have their own internationally-compowered data to inform their policy, school-level policy. And we think that it is not enough just to give teachers and educators data that we also created last year a pilot study, which I will go into detail right now, is like an online platform that we connect educators from all over the world. In the pilot that I chatted, 6,000 teachers, educators or policy makers from 172 countries to work together in a group as a project to come up with some teaching resources that are now available for all teachers in the world. So, more East PISA, you see in 2015, more than half a million students from seven, more than 70 countries took the test, but because we use a random sampling representative model, that actually those half a million students represented more than 28 million students. And it is a test of two hours and it's not just a traditional test, it's not like one plus one is equal to it, it's not just that, but it is the ability of students who apply the knowledge. And so it's not just the performance in the test itself, but we also connect information of the social economic background of students, teaching and learning environment at school, not just the student, but also teachers, parents and school leaders. So, based on the same model that PISA is working at the country level, that's random sampling schools at the country level, my approach at PISA based at first schools, we work with schools directly. So, instead of talking to ministries, we talk to individual school and provide them with data. So, after taking the test, what we got, you would get a report, huge, and 150 pages, and we have no time to go through all the details, but I will give you some highlights of what you can expect from the report or what teachers and school leaders can learn from the report. You can have the individual school results in reading mathematics and science, which are internationally comparable. Basically, for example, my school in Paris can compare my school to Tokyo, to Hong Kong, or to France as a whole. Oops, I'm going to pass it to one person. And so, as I said before, it allows you to do global comparison. It also allows you to do school-level comparison with your own country. For example, in this figure, you can see all the schools which took part in PISA 2015 in the United States. And then also, you will know more about the learning and teaching environment at your own school. So, it is not just a report card of how well you perform in reading, mathematics and science, but also the discipline, climate in the school, teacher-student relationship, how motivated are students, the reading strategy, et cetera, et cetera. So, in the X-axis, you can see, yes, yes, it stands for the socioeconomic status of the school. The more to the left-hand side means that the schools are more advantaged. And to the other side, it means less advantaged. So, you can see, again, all the schools in the USA in 2015 which took part in PISA. And here is the regression line. And then, for example, a school which took the test in 2015. And then, again, in 2016, then you can see there is a slight improvement. Again, 15. And so, you see improvement again and one more school. So, after you get the data, what we care is not how well we perform, but equity and room for improvement. As you can see here, for example, for this school in 2015 and move... I can see it. Okay, the green dot, sorry. The green dot from 15 to 16, you can see that it moves above the regression line, meaning that from performing below expected to above expected. So, what has the school done for such improvement? Of course, if I'm looking at the data, we don't know. There is exactly the reason why we created such an online platform to connect school leaders together so that we provide a place for them to talk. So, measure means taking the assessment. And after that, you get the report and you start to explore your own result. And after that, you act. How many things you can talk to your teachers from like an expert group within your school to talk about it. And one thing that I wanna highlight here is that we offer, as a project, we offer international peer learning opportunity for school improvement. What is it? It is a pilot with an external organization that we did it last year. It's called PISA for you. As the name suggests that it is PISA, but it's for you because we see the needs to democratize PISA data. Well, it is online, public, everyone can see it, but it is not in a way that people start talking about it at a front line at a school level. So, we want to connect educators around the globe that we did and we want to crowd source teaching resources making, you know, connecting all the white lines in the road to come up with solutions for some common challenges all over the road in the classroom. And we want to empower educators from a bottom-up approach instead of going from ministries to schools. So what it is about, it is, so for example, you go on the platform, which is a website and you subscribe to it. And according to your profile, for example, where you're from, which language you speak and which education level you are teaching at, and then we have algorithm to match you with other teachers all around the world because we are talking internationally, so we have to take it into account of the language, we have to take it into account of the time zone, et cetera, et cetera. So, once you are on a group, on a website, you assign to a group. And so you start talking to, with your group mates of what challenges you have. So on the website, we have library, we provide a lot of resources, like such as videos, bugs, and we also have mentors to help groups to define their own challenge. So once they have their challenge identified, they will start to kind of conceptualize what their challenges is about. And after that, they will work together to come up with a solution. Solution itself is not enough and they were implemented in their own classroom, and then they will evaluate their own solution. So you see that each team has support from mentors who are experienced educators. And it's not only about that, also teach them how to use certain functions on the platform. So it's a three to four month program. Last year in October, we threw peer review that we selected two teams. We invited them to come over to Paris and presented their teaching resources with us and share them with us. It is really international. For example, one team consists of a teacher from Colombia, a teacher from the Dominican Republic, and a teacher from the States. And the other one, a teacher from Colombia as well, again, and from Romania and then from the States. So it's truly international, which is interesting for us is to what we have learned. So in order to find out what we have learned, we did an evaluation, but it is more on the user-based evaluation. It's like, for example, how many people finish the whole program and number of participants, completion rate, and we also do like a focus group to see what they think about it. And in order to achieve a continuous improvement. So what have we learned from it? Some facts again, so 6,000 registered participants from 172 countries and finally 1,500 activity engaged in it and finished the whole program. So it was really a surprise for my service. When I first started planning the whole program with our partner, our idea was to have two, ideally 2,000 people registered. And then luckily enough, we have 20%, 30% people finish, but it was really went beyond our expectation. So you have also some complications on the resources that we already planned for, but finally we solved this problem. And we have 100 teaching resources produced. It comes from teaching mathematics to teaching STEM, to increasing motivation from students and all sorts of things. And most importantly is that all these resources are free to use by other teachers. Everyone here right now can log on to the website, register and you can see all the resources. For us it is very important because we believe in public good and we all believe that all resources should be shared and to teachers who lead them the most. So who were the participants? As you can see, 81% were teachers, 8% were school administrators and 2% government official and some other like students, journalists in academia, et cetera, et cetera. So you can see overwhelmingly they are teachers. And then we could also see a lot of communication happening among participants, 4,300 posts on the forum and then almost 40,000 comments and messages exchanged among themselves. So what have you learned from this exercise? Is that we learned there's a significant demand for international connection and professional development which is very interesting for us because when we look at different so-called teaching professional development it's mainly learning something new about your subject matter or how to manage a classroom but it is like more on a no-call and dramatic contest. Like such an international contest for like collaboration with other teachers was not very popular or was not very much offered by universities or by governments. So we found it quite interesting that there was a huge demand for that. And people really enjoyed working with teachers from other countries and because when they first started they had some doubt, come on, we have like from very different contests, we have very different education systems so the problems cannot be the same but after that, the feedback we got from them was actually we are facing the same challenges. So it really comes to the second lesson that we learned is that for the pilot the Lingua-Franka was English and then we found that it may not be very ideal and to wish all the teachers who need them are this kind of a professional development and international connection that for example the teacher from Colombia said that he tried to promote this platform to his fellow colleagues but that not many of the colleagues could do such a project in English. So we are planning to do it in different languages but we found that in terms of a technological problem there is some problem that we have itself and then we observed digital divide. I think as an presenter before said that it's a very interesting case in Africa that people impacted by Facebook and data in it that we found that in some countries teachers were using their cell phone directly they were not using computer to finish to complete the whole program so we have to think about what to do to satisfy these needs. So I think I have like four minutes left and this is the last slide. So again, so we did a pilot we found a lot of interesting things and it also provoked a lot of questions because what we learned so far is more on the user perspective, user-based research but we don't know anything about the impact we have no, because it was not part of the pilot that funded the impact. So we would do at least a plan to do at least three more ones. So these are questions that I'm trying to answer and to find to draw your brains as well is what would be the impact for a different user type? Because overwhelmingly the last in the pilot like most of them were teachers. So we didn't know much about the needs and desire from teachers, from school leaders, policymakers, parents, students and journalists. So we may not be able to cater for the needs and how can we find the needs? What can we find about the channels? And the second thing is we produce a lot of teaching resources and how can we evaluate the impact of those resources in the school classroom? And how can we make sure that we can disseminate those resources to other teachers who were not part of the platform? And more importantly is that is there any real impact in terms of performance, in terms of student engagement at school level? And how can we, what kind of information we can collect from the platform, from the behaviors of those users to come up with some meaningful policy level discussion? And so more research has to be done for example, the trend of discussion on the platform. What are the topics that most interest our teachers? And what are the different behaviors of each user type? For example, does it mean that policymakers are more concerned about the management or teachers are more concerned with the subject matter? I mean, what are the behaviors of each user type? And the pattern of communication, how they communicate with each other and how they use the resources produced after the program. And again, it's the impact of it that we are very interested in knowing. And then the last question that I'm asking myself, hope to find more inspiration from you guys later on is, well, how can we define success for such a platform and how we can measure success? Is success the same for different users? Is it success for the same from the OECD perspective, from the teachers perspective? What's the voice asset? So it is why I'm here. I look forward to discussing with you further in the cocktail or in the Q&A section. Thank you.