 ready? Yes. Well, I'm really delighted to welcome our next presenter. We are going to be hearing about the topic of numeracy training for everyone and of video, sorry, video is going to introduce the session from the University of Derby. Thank you very much. I would like to thank the previous speaker as well because I'd like to tell you that the best student we have in the third year in the mathematics course at the University of Derby has cerebral palsy and he's doing a fantastic job. He's the hardest working student and he's doing really, really well. So I think he benefited so far from support along this line. So I think you are doing a great job on this. My presentation is linked to a project which actually started in this building five years ago. I'll tell you at the right time. I'm working at the University of Derby for a number of years. We are focused on teaching quality, I think above everything else, student support, student employability, careers support. And we are supporting a wide range of learners. And I had to teach large classes of students. I had to teach mathematics. I am a mathematician, as you will see. I had to teach large classes of students who didn't want to be there. So I had to become familiar with items like mathematical anxiety. Then in order to provide inclusive learning, flexible learning, meaningful learning, assessment for learning, I had to build a range of resources. I had to build E assessment. And then this got me into this project where I worked with the technology company to develop digital support for mathematics. But this is mathematics for everyone, not just the mathematics for STEM students. And this is, I will show you the impact of not having the support in place at the society level. And then also how we try to support the learners in our university. And then also many learners abroad. Now why do we need to develop skills at the university level? So the office for students has many conditions of registration. So B3 is students provided with opportunities to develop an understanding of and necessary skills to demonstrate good academic practice. So this is something that we providers have to offer. D1 is organization provides opportunities for all students to develop skills that enable their academic, personal, and professional progression. And I think that's D3. Organization provides opportunities for all students to develop skills to make effective use of the learning resources provided and to provide the use of digital, so to make use of digital and virtual environments. Also, there are many institutional success measures which are linked to skills development. So relative performance in the test five scores, so related to NSS, the proportion of U.K. domicile students who are in highly skilled employability employment. So this is more employability is also a very important success measure. Then attainment gap in degree outcomes between white and black students and then continuation rate. And at the University of Derby, a decision was made to bring all this skills support under one umbrella and we built the project develop at Derby. And now every student from across the university can go to this one-stop shop to get access to all the skills support they need. And so this is a cross institutional collaboration. The aim is to help students develop a broad set of skills in a safe and supportive environment. Both their existing skills and experience apply those skills in new settings and challenge themselves to develop skills required for future success. Now there are a few boxes here. I'm not sure you can see very well, but this includes preparing for study, critical thinking, types of assessment, referencing, writing at university, digital skills. So we have a whole team dedicated to digital skills. Your future start now, academic well-being and among all these things, I'm responsible for maths and numeracy support. Feedback from students and also from teachers across the university has been very good because now we finally have seen, we used to have such support developed and provided to students in various colleges, schools in like smaller teams, but now this is provided across the university. And this is where my trajectory started to move towards this project. So I got into curriculum design and digital assessment for mathematics because I had to teach computational mathematics for year one students in computing. This included most of the mathematics they needed to know in order to do well in their degree. I had relatively large class, 130, 180 students, and SS feedback with this cohort was historically about 40-50%. So lots of students who had to learn, but they were not really happy with that. And even before the COVID pandemic, I took this course on in 2012 and I redesigned it completely. I wrote a textbook, teaching materials, recordings as soon as they became available. Math jokes because students are stressed enough having to learn mathematics, coping with the stress. Einstein had, for example, to say about mathematics, don't worry about your problems with mathematics. I assure you mine are far greater. And then I tried to comfort the students telling them that, look, Einstein had problems with mathematics. I had problems with mathematics because I'm a researcher and I always work on some new problem. And don't worry, if you have problems, if you struggle at some point with mathematics, I know exactly how you feel. So I'm here to support you. So for this reason, I developed E-assessment. So I developed like assessment for learning. Each topic is assessed using computer-based tests. I had to design about 800 questions. And then this really turned around student experience in the room, student results. Students responded straight away. I really liked how we were able to retake the online tests. This became the perfect revision tool because students could, because we had such a rich range of questions for each test, each question is randomly sampled from a pool of very similar questions. So this means that I assess the students at the end. So they know exactly, they get feedback on their attempt. They know what to revise. And then they can practice, they can take the test until they get the desire. So this is also a tool used to engage the students. So students say that it facilitated my learning and comprehension of all the material presented. Then the next year, I had, I always, I have some explicit comments as well. So I always sucked at maths, but even in, so even in school, I barely scrapped the seat. Strangely, I'm enjoying computational mathematics to the point where I enjoy, I enjoy doing some in the spare time. Then, because results were pretty good, I will show you more details about the results, the student results on this course. But this drew the attention of the college. And then in 2018, in March, we had the consultations about how to improve student performance, student entertainment, student retention in the maths intensive modules across the college of science, of engineering and technology, as it was that time. And I was familiar with this, with the impact of such digital tools of, so I had like lots of examples of good practice supported by student feedback. But I was not very keen to develop same 800 questions or so for each of the modules in the college that were heavy in terms of maths. But through this investigation on the student results in maths intensive modules, we all realized that quality maths education improves student entertainment and performance not only in mathematics, but we could just feel this. Then it can improve evaluations in NSS and it can support CAF and REF, and it can also boost student employability. So this was like, this was the hypothesis after these consultations. And then we designed and we formulated an ambition to provide maths support for all students in the university and for all staff. We created the maths at Darby community, we like designer clothes, like I mean, we designed with maths logos, we took students, we trained students, we selected students, we deployed them in schools to support maths activities. So we built like a mathematical community because we had very few staff. And this means that our main resource was the students. So we trained the students so that they can support local schools and outreach events and so on. So this means that we also to provide teaching support. So we recruited and trained student teaching assistants. Then this impacted outreach activities in local schools. And also the ambition is to support the wider society because we are in the Darby opportunity area, like lots of disadvantaged learners, some poor communities. We worked with some charities looking after children who vulnerable children who could have been involved with drug dealing if they didn't have adequate education. So they were working directly with the children, but we were trying to support them as well with tools. Then we have a range of activities. We have a maths café, which originates in the maths clinic at the University of Darby. I had to prepare the teaching excellence framework submission for mathematics. And this is when I was able to discover some of the papers published about the impact of maths clinic at the university. So this is an open session, this is a session booked in the everyone's timetable. And this is three hours a week. You can get some maths support. You go to a room and some maths lecturer will help you. And then I found the paper from 2003 describing the experience of running this clinic for 10 years at the University of Darby. So we realized that we were doing this for a long time already. So I knew that, but then I've seen this paper also. So we are now rebranding this as maths café. Class interventions and peer learning with student mentors and also especially throughout the pandemic. We pioneered the use of Microsoft Teams a few months after it was released for some group project with students. But then we set up a learning community for this maths café throughout the pandemic. So this means students could ask their maths questions somewhere we could have, we had these sessions online, like these maths interventions. But then I mentioned that this project actually I realized only after I parked the car, because it's the same car park where I let the car five years ago, when I attended the British Congress of Mathematical Education in 2018. And at this Congress of Maths Education, I bumped into a gentleman. This is a gentleman, Professor Graham Orput. And he greeted me. Hi, I'm Graham, and my dream is a world where everyone enjoys mathematics. I said, really, that's my dream as well. So let's collaborate. And then I paid in mind to organize a workshop on current issues in mathematical education. I had some speakers on my list. And then I invited him as well. I said, Graham, what do you think next week? Can you come to a workshop? He said, I need to call my wife. And he gave me the answer on the spot. He said, yes, I'll be with you next week. And then we organized this workshop, where we brought together teachers, researchers from the university. We have a very good group at Derby working on maths anxiety. So this is the fear of mathematics. If you ever ask someone, it happens to me every day. Today, I spoke to a school manager, and I talked to her. I'm doing mathematics, and she said, oh, I have problems with maths. I had problems with maths. I think in the UK, I think it's a cultural problem as well. And this maths anxiety is the fear of mathematics, which was proven to be a failure of the nerve, not the failure of the brain. So you are just afraid, and then you are freezing. But you are capable to do it, most likely. So we have a whole group working on maths anxiety. This person was an emeritus professor in maths education in Seneca College in Canada, but he was also the director of international programs at VRETA. And this was a technology company, which was just launched on the UK market. And then when he joined, I presented my research on maths anxiety and in impact maths anxiety has only in the UK, about only 15% of the students take mathematics beyond the age of 16. So A levels in maths. And I was mentioning how this impacts society. And he said, yes, it's true, but actually I don't expect more than 15, 20% of the people to need advanced mathematics. And he said, but I can tell you an even bigger problem. And this is that nothing is provided for the other 85%. Because for this 15%, there are lots of smart tools supporting mathematics in STEM, lots of software, lots of programs, but then nothing is done for the other 85% at that time. And this was for me an Eureka moment because I realized that, yes, it's these learners who need support as well. And we organized another support. And their project was informed by Research in Canada, where over a population of about 40,000 students from Ontario colleges, they proved that a third of them were in danger of failing their whole course because of their single maths course. Right. And then that's why, again, my research hypothesis was that if I'm able to support this third of the students who are struggling with maths, then this can make a huge impact at the institution level. Then the research, the next phase of this research was they went to secondary schools to see what was the problem. And then teachers in secondary schools said these are the materials, these are the tools we don't feel that anything is missing. But what we find out is that many of the students come with gaps from primary school. And then you have the snowball effect where at university, sometimes I had to remedy the problems from secondary and then secondary, they remedy from primary school. And then you realize that even students doing STEM subjects, some students understand very well the engineering concepts, they understand exactly forces, they understand how the how the different complex mechanisms work. But when they have to transform that into an equation, that they have to rearrange, then this is where they might struggle. And this is where we thought that we need to intervene. And also, Professor Orkut coined the term the numeracy gap. And this is the gap between the current level of numeracy and the level of numeracy required to be fully functional in the nowadays technological society. We did some research and the impact by national numeracy. So you can find some very sobering research done by the national numeracy. The impact on the UK economy in a study from 2019 was estimated at 20 billion pounds. So these are just small errors that maybe administrators make in some spreadsheet because the numbers don't make sense to them. They are not great with problem solving. In the States, the impact was maybe around 100 billion pounds. And then also this study from 2019 highlighted that about half of the adults in the UK are not numerated. And then you want to develop new technologies, new tools, innovation, AI, and so on. Why? What is the reason for this numeracy gap? Attitude, I think, especially in UK. So we have this culture that it's okay not to be good with maths. Don't worry, I'm also rubbish with maths. So we are both rubbish. But you never hear anyone saying, I cannot read. I really struggle with my reading. There are many myths like the maths gene. If you don't have the maths gene, you cannot do it. No matter how hard you try. The use of calculators and now we are going to see, even in these slides, the impact of COVID-19. I've seen quite a few studies about the students who missed out on their learning in college or like in secondary school. And I think this will hit us hard at the university as well. And then the technical solution to help the learners in Ontario was this program, Elevate My Math, designed around the principles of assessment for learning, flexible engaging assessment. And this was consisting of multiple stages. So first you have for each of the courses powered by Elevate My Maths, you had to sit a diagnostic test. Then based on the recommendations from the diagnostics, you had to take some upgrading modules, presenting your numbers in context. So this is numeracy for adults. It's not with rabbits and so on. And then finally a summative assessment to check the mastery level. These were multiple awards, including from the assessment association. And after I bumped into Graham, but BCME in Warwick in 2018, I said, let's collaborate. I want to have one of your products or two of your products available to my class. They kindly made this available free of charge. And I had the pilot project with 300 students from five modules. And some of these students were in my class, Computational Mathematics, but my class was running in two campuses in Greece as well. So I could get access to and this was like foundation mathematics, foundation here, calculus and computational mathematics. There are a few basic topics, so basic operations with numbers, fractions, percents, ratios, and so on. My main reason to do this was that I didn't, I wanted to provide students with a self-study course. They could practice in their own time to reduce their anxiety. They didn't have to ask, can you please remind me Pythagoras here? And so if they were struggling with something, they could get the remedy lessons. And I could see, I could through a dashboard, I could monitor the engagement of the students, so I could get like lots of lots of data. So I could check the engagement. So with the pre-test part, so with the diagnostics, the average was 102 minutes, the engagement with the remediation. So it seems that many of the students actually had to take remediation and then post-test with the summative. There are a few questions of 46 minutes on average. I could see improvement in the student scores between the diagnostic and summative. 14% in mathematics. I expected that to be lower because the students already had a pretty good level with mathematics. Then in year zero, 17%, but in computational mathematics this was 20%. So this was with the more anxious students. And then for up to almost 40% in the students in Athens, in Greece, because along with the mathematics they were also improving their technical English. Then I tried to make this experience as rewarding as possible. So I designed like certificates for the students. So this was an add-on from us. And then I was able to present the results of this pilot project at the Advanced HE STEM Conference in 2019. I collected feedback. So I'm always interested to engage with the learners to see what do they actually think about this. And especially students coming back into education after long breaks benefited greatly from this. So I had students who found it very useful after a break of 25 years and they had to go through the remedies. And then I had another student who was out of education for 14 years and again benefited. And this is a timeline of the partnership because for the next year, after this successful experience, we signed a partnership with the companies of the CEO visited our university. Then we ripped apart their whole resource. They had about 230 lessons and around those, with input from our academics, we designed 12 numeracy training courses aimed at students from psychology, from nurse education. So we tried to really make this available to everyone. And then we developed modular courses and combinations of these courses can be deployed in various places in Blackboard. So we integrated the tool with our learning platform. We developed new numeracy training patches, but now on the OpenBatchFactory so that the learners can share their achievement on LinkedIn, Twitter, Instagram if you want. And in 2021, we had a project sponsored by the European Union and we already had courses available. We had a platform and then I was presenting some mathematics to motivate the learners to care basically about improving their math skills. But then I was putting them through one of these courses giving them a specially designed badge and then this is how we were able to upskill the workforce in a few companies and also learning communities. So we reached to the Bosnia Herzegovina Society, Caribbean Society and Romania Society and others. And like every time you have learners through the platform this generates data that can be used for pedagogic research and for reporting at institutional level. So we refined all these things and then in 2022 last September, we signed a new partnership for four years with the company and we are seeking international expansion and we are also supporting the Rolls-Royce Nuclear Academy. But this tool, because we embedded lessons and our expertise in math anxiety, this means that we could get letters of support for case impact, case studies in REF for psychology. So it was not just like mathematics and so on. So this means that based on this collaboration, the company also was able to launch the next edition of their, so they refurbished the whole platform. So this was the launch of the new resource, Celebrate My Maths 2.0. So this has modular courses, each of them consisting of a few topics. Each topic has now diagnostic remedy and summative assessment. The completion of, so once you complete all these four topics, you can see the proficiency assessment and then once you complete this proficiency test, you also get the digital badge. We also designed dashboards. This is fully compliant with EDI requirements, with accessibility requirements because, so this is the core of what they do. If I was to develop this my own resource to be compliant with all these changing requirements, I didn't have the capacity and the resources to do it, but through the company we can do it. They do it anyway. Now to support you, this is how the topic looks like when it's completed. You have a dashboard view and you can provide feedback at a glance to your class. I do this every week and in two minutes I have my feedback because looking at this dashboard, so these are the results from the diagnostics. So in the top row we have diagnostics. This is the core D1, which is, so in computational mathematics, we labeled these courses from D1 to D11. And in computational mathematics, I'm using D1 and D2. For apprentices, they are using D1, D2, D3, D4. So in finance modules, we are using D6, D7 and so on. I'll show you the list. But on the front row, you have the results from diagnostics. So I see how many students actually completed the diagnostics. D1, D2, D3, D4. Then remedy from one to four summatives. And then here I have the summary with the results from the proficiency assessment. And I can see that. So looking at this bubble, I can tell how many students engaged. So I have 174 students engaged. Then 117 students have done a very good job. So this means that they reached the final summative. And then 74 students have completed the course fully and they have the digital badge. Now we also produce, so in order to make this resource available, we had to produce a math skills guide. So we have a website detailing the contents of all these courses. This is the list. So actually we have a basic numeracy skills. This was the basis for the, for a massive online open course made available to applicants throughout the pandemic to help them get up to speed with their learning. This was the basis for the course provided to companies. And then we have fundamentals. So this course has the maths that everyone across the university needs. So if you study humanities, you can take this one. So it's just like very basic mathematics. But then if you need financial mathematics, if you need algebra measurement for finance, last year we launched the course for biosciences. So now this will be available from this year to a few hundred students in biosciences. And much of this work was done in partnership with the students. So the students were involved in writing solutions, looking at data from the courses. This year I work with five students on interpreting the results. And also I had the feedback. This is Steve. I promised him that this was the present, this is the quote I asked for a presentation at Bette in London this year. And Steve is a mature student and he shared with me that he completed D2, the second course, with a score of 85%. And he was very happy with this because he's had to learn everything from scratch after 25 years. And this is I think the slide which motivated the university to invest for four years into this tool. So these are the results. This tool has in my own class computational mathematics over a number of years. So here I'm presenting the number of students taking the course between 2015 and 2021. I'll show you some more recent data as well. So this is the student numbers. So ranging 168, 120 to 180, 159 throughout the pandemic. Then I'm looking at some key metrics. So I'm looking at the first time pass rate. So this is here I give the numbers here I give the percentages. So pass rate before 2018-19 this was like 83, 86%. And the good grades. So the percentage of students scoring 60% or above this was about 60%. This was already pretty good. But then after I deployed this course, pass rate went across the whole cohort to 89%. And for students who actually completed this training, it was 93%. Good grades went to 64% with the completion of these courses. But the most impressive results followed throughout the pandemic when this pass rate stayed at 89%. But for students who actually completed their courses, this pass rate was 98, 96%. Therefore, in terms of goods course, they improved massively across the whole cohort to 70%. But for students who actually completed the training, this was 85%. So most of the students obtained digital batch they could share on social media. And my lesson from my key point was that numeracy training can help students build immunity against course failure. So if you get this maths aspect sorted, I think this is a first moral boost, because if you struggled all your life with maths for all sorts of reasons, if you managed to overcome this fear and show that you demonstrate to yourself that you can do better, then I think also mathematics is special as a subject, because you have that aha, you know, when you finally get it, I think it's very rewarding. And then I have some recent results. So these are results from this year for a cohort of 128 students. So I think most of them engaged, so 123 engaged properly, and then 100 obtained the digital batch. So results as percentage are better than the previous year even. And this is slightly worse results with the D2, but still better than the previous year. Then here I'm comparing some data. So this is more recent data from 2021-22, 2022-23. And this is data on the first column, we have data from University of Derby. So in 2021-22, I said 145 students. And then last year, I had 128. And then here I'm showing data from Med College Atems. So these are our partners where they run, of course, and Med College Thessaloniki class. And also in 2022-23, we launched this project in China as well. So we have a new BSEIT in collaboration with Shenyang University of Technology. And now we can have a look at the results. So again, pass rates are around 90% for Derby. I cannot compare the results yet with those in Athens and Thessaloniki. So I can see improvement in terms of pass rates, but they are not using the resource yet in the same way as we are using it. So they are using it just as a prescriptive resource. In my case, it's 10% of their grade. But then this year, I used this fully in China. And the pass rate was 100%. In China, different style. Student feedback was also very interesting. So it's the first time we run this course there. So they said, I think it's rich and meaningful. Don't be careless. What they mean is that if you don't get 80% in one diagnostic, you have to take all the remedy lessons, which maybe you should if you don't get 80%. So this is what they say. Don't be careless because if you don't get 80%, you have to sit and really spend maybe a few hours on the remedies. And after training, my ability has improved a lot. I found my shortcomings and improved. You can still improve your ability with more practice. So course satisfaction was also 94%. So now we are gathering more data. We are building an across university wide dashboard so that we can report all these things and create reports in real time. And this tool, which started in the classroom, actually had significant impact outside the classroom as well because we could train various communities. So we trained the community Sadaka, Bosnia-Herzegovina Romanian Society. This supported the project sponsored by the European Union. We have supported the Rolls-Royce Nuclear Academy. So they are using courses developed by us. And for this reason, last year we've been finalists in the E assessment awards in the best formative assessment project. Now, this is the footprint of the project. This year I presented about this project in Ethiopia in January. Then in Thailand last month. In China, we are now working with Shenyang. In Israel, last year I presented the CAT GME. So this is like digital tools in mathematical education. The project runs in Greece, in Romania. I presented in Portugal next week. I hope to be in Sicily again for CAT GME 2023. But I hope to get new collaborators on board. And now going back to developer Darby. So going back to developer Darby, I think this really became a pillar of this university-wide project. And I think one of the key advantages of what I'm selling is not just the tool, not the products. But what I'm selling here is the project as a whole. So I think working in collaboration with the technology company, so if you can build your resources, they can have huge impact across the institution. Thank you very much. Thank you very much. There was just so much information in there and so much work that you've covered over that time. And great to have those resources available as well for people to look at. Just to check if there are any questions. I know that we probably are at time because I know there's another session coming in here. So please feel free to get in touch. Yes, so please get in touch. I'm most happy to make you partners in this project. And I was going to fit that and say I don't have much time for it, because my last at my primary school was a maths graduate and he said positive that I actually have more dyslexia and spelling problems, but not maths. Yeah, I think attitude makes a huge difference. Absolutely. Yeah. Well, thank you so much. That was a brilliant presentation. And thank you all for joining us for this session. And I can probably hand over now to the next chair or presenter. There we go. Thank you so much.