 My name is Ann O'Shea, and I'm here with some of our team, and we're going to talk about our project, which is about assessment for learning, and it's about developing resources for first-year undergraduate maths modules. So there's a lot of tie-in with the work that Tom has been talking about, and also Lisa's project. OK, so the partners, as you'll see, are Minuth University, AIT, sorry, DKIT and DCU, and of course the National Forum has given us the money. Most of the people involved are either from mathematics departments or engineering departments in those universities, and we're working together to try and improve the resources available for formative assessment for first-year maths courses. So our aim was to develop an audience response system that we could use in classrooms. There are lots of these around, and people have already talked about clickers and so on, but unfortunately they don't work so well in maths classrooms. So it's a work to a certain extent, but what we really wanted to do was to be able to have students draw a diagram, draw a graph, give a little calculation, and the lecturer would be able to get that feedback straight away and then be able to respond to it on the spot. So Seamus, who's here from the engineering department in Minuth, had already developed a system like this, and most of the work in this part of the project is going to be to develop that further, and especially for use in both Android and iOS systems so that students on their smartphone just upload this kind of stuff instantly. So I'll talk a little bit more about that later. The other part of the project is to develop tasks and resources to look at conceptual understanding in first-year mathematics courses and see how technology can help us with that. So in order to figure out what we should concentrate on, we conducted a survey of staff and students to find out what it was that both staff and students wanted, what kind of areas they thought their students had difficulty in, and what kind of resources they wanted, and what kind of resources also they actually use so that we wouldn't be reinventing the wheel. The audience response system, we don't have the system yet. We actually spent most of this semester hiring a programmer because we wanted to get somebody who would have the skills to do this properly, and we held interviews. We had a very good set of applicants. We held interviews and we got our first choice applicant as decided to take the job. We will start work next Monday, actually, and she will spend the summer trying to develop this response system. We hope that at least a version of it will be ready in October to start trialling in classrooms next semester. We also had discussions within the group over the last few months about what we wanted from the system, what changes we wanted from the version that already existed, and we have identified some things that we will include in the next version. The Staff and Student Survey, Katrina, has done a lot of the analysis on that, and she will talk about that in a second, but I just wanted to say that we met, the group met a few times over the year. The first meeting we really concentrated on how to design the survey, what we wanted to know. We designed the survey, we piloted it, we ended up revising it a little bit after that, and then we administered it. We had students in the four institutions where we had about 450 responses, and we also had a staff survey which we sent around to people who were involved in teaching first-year maths modules, and we've had 32 responses to that so far. The analysis, and that's from people all over the country, you'll see in a second when Katrina talks about it in further detail. So the data from this project is one of our major outcomes so far. The tasks and resources that we wanted to work on, we decided to try and come up with things that were in two camps really. There's a lot, a huge amount of resources available online already, so we didn't want to go and do everything again. So what we want to do is try and package a lot of the things that we really like that's out there, and be able to give easy access to people, to both staff and students to that. But also then we wanted to develop new things, where are there gaps, what can we do about it. So some of these things, you'll see the first one, a web infantry of useful existing resources. A lot of that has come out of the survey from both staff and students saying what they use, what they really like. So we want to compile that all in one place. Using Khan Academy, Khan Academy came out when students were talking about what they use quite a lot and also with lecturers. So Fiona's going to talk a little bit in a minute about how we're going to package those resources in an easy to use way. We also want to design, use some of the features of Moodle to design lessons. These lessons are where you can combine content and tasks in the same thing, rather than having some content and then click somewhere else to do a quiz on it. It's all combined and students can develop their knowledge as they go through. We also wanted to look at screencast projects. Again, there had been use, we had thought maybe we'll create our own screencasts. But then we saw some other projects, including some in Dunduk, where the students created their own. And we thought, yeah, that's a good idea. We'll steal that idea. So we want our students to come up with screencasts to share with the rest of their class. And we think that there's two benefits. We'll have nice screencasts that students have developed themselves, but the students in developing them, we feel that that's the real learning is going to take place there. And we also want to develop a suite of interactive tasks based on conceptual understanding, trying to develop conceptual understanding in different areas of mathematics that we have identified from our survey. And Shaleid is going to talk, are going to show you some of those that we have worked on so far. So I think I've probably said that stuff already. This is the progress. So we've got some resources. We're going to make a website, which we haven't, our programmer, when she starts is going to be involved in doing that. The lessons in Moodle, we have some prototypes already. We're not going to show you any of those today. We'll focus on the Khan Academy and interactive tasks. And I've already said about the screencasts. So I'm going to hand you over to Katrina, who's going to talk about the data we've got. And then we'll see some of the other resources we've developed. Thanks, Ann. So as Ann said, there were two surveys carried out, one for students and one for lecturers. So in the student survey, they were essentially asked for three different types of information, their background information. Then there was a set of likered questions, 23 different question types that were asked to rate their ability to do and understand those questions. And then a set of open responses where they were asked what particular topics they found difficult in the year and what resources they used to overcome those difficulties and then what would their advice be for us in this project in developing resources. So what I've shown here in the graph really is what came out of that survey is the significance of the mathematics leaving certain level that the students had, whether they were higher or ordinary level students. And you can see from that there that the vast majority of the students in D.C. and Minuth had higher level maths taken and in AIT and DKIT their ordinary level and their responses to the questions were dependent on their level of maths. In the lecture survey, as Anne said, we got 32 responses, 16 from the IOTIs and 16 from the universities, and they were simply asked for open response questions similar to the students based on the concepts and procedures that they considered were most difficult and the resources they would advise us to use. So if we look at the student likered analysis, when you look at overall for the understanding do, 77% of the students reckon they can do and understand all the question types. So, and there was a significant difference there between what higher level students said and ordinary level students said. The main areas that came out of that where they might actually agree they have difficulties on our functions and drawing graphs and so on. So I also made a note there that they were asked about differentiation. None of them really indicated that they had a problem in differentiation in the likered analysis, but when you look at the open responses they're all talking about the problems they have with differentiation and they included integration which they weren't asked about. So some of the likered obviously didn't match up with the open responses. Now when you look across the two, evaluate the two surveys together you see that I just took one example here of the graffing basic functions because 20% of ordinary level students indicated their problems with it. 25% of the students in the open responses indicated their problems with functions and graphs. And we also saw that coming up in the lecturers responses. They find the students, but the lecture responses might give you a bit more detail of what exactly the problem was. For example, there were a number of lecturers who said it was compositions of functions or the students had problems with domain and ranges whereas the students themselves couldn't really pinpoint exactly what their problem is with the area. I thought as well it's worth saying there overall the students responses were heavily based towards calculus and stuff like that whereas the lecturers responses were mostly towards basic algebra, basic arithmetic but when you delve down into them you see a lot of them saying the same things. And because we asked them the easiest topics and something that kept coming over again is that the student, I put this up here, the students said the things they'd covered in leaving cert. That's what they found the easiest when they came to second level. When we asked the students and lecturers about their resources they found helpful they were spread across books, handouts, videos, websites. Students found handouts useful because they're step-by-step examples and this came up over again, step-by-step examples and solutions. Lecturers found them helpful because students seemed to like to look at them in class and helped retain information. They also liked the Khan Academy, Wood from Alpha and then the lecturers mentioned things like the Math Centre, the Math Tutor and GeoGebra and as Ann said we're taking all those on board and developing our resources. In the advice zone resources as well the students equally kind of advocated they like printed resources, websites and videos. They like a print out even if it's unmoved and they can print it out themselves they like something that they can take away and have to look at the step-by-step. From the lecturers point of view most of them want online resources so again I said this is what we're going to develop here. One of the resources that we're going to focus on is looking at existing resources and listening to the students' responses to the surveys Khan Academy came up again and again and we're also aware of the Mathleets challenge that is bringing Khan Academy into second level and primary level that's going on currently in Ireland. The problem with a lot of the resources is that students felt that they couldn't name the topics. Going out and trying to find the resource that they actually need is a problem. Our idea here was to take one resource that students highlighted was useful and try and package it, try and using the topics that they have named as problem topics put together what we call a Khan playlist so that they could a bit like this so that they can go to this moodle package, this moodle course and they can be led through resources and directly to resources, videos and quizzes that they can use. That's the main focus of this element of the project but as it came up earlier it's very important that students can see and use these things in context and that lecturers are able to facilitate student use of these resources so we also want to through this moodle page highlight some of the functionality inbuilt into Khan Academy, the monitoring facility that's available through their coach class setup, the possibility of creating playlists and predefined missions which can guide a class group to particular tasks and allow lecturers to see what students are interacting with and to highlight problem topics very early and also the recommendation structure which would be particularly useful in learning support environments where you're trying to tailor resources to particular student needs so that's kind of the essence of what this aspect is about. We're also developing some interactive tasks. No, great. Sorry, does anybody know how I can show this on the drive window? Oh yeah, great, thanks. Yeah, drive it right up. Oh, thank you. Thanks very much. So we had developed some interactive tasks. This is one using GeoDebra which is a freely available dynamics geometry software and the focus in this part, these tasks is on conceptual understanding, developing conceptual understanding and students' mathematical thinking skills. So in this particular task we're looking at graphs of functions which is something the students have highlighted as problematic and the effect of adding a constant to a function, to a function value or to the independent variable. And I suppose this is something that traditionally students would have been shown in class, would have been told and then they would be required to remember or reproduce it. But here we want the students to explore the effects for themselves and to describe them. So here there are a number of different mathematical skills involved for the students to complete the task. They need to experiment. So the idea is that if we change the value of A here you'll see that actually we have a graph of two different functions. So if I change it to 1 and we want to see how the graph changes as the value changes over. It's 1.1. So the graph, the red graph, which is x squared plus A as opposed to x squared, it's moved 1.1 units vertically. If we change the value of A again, so if I change it to 5 for instance, it moves 5 units vertically. So we want the students to be able to recognise this and articulate it. And there are different things they can do with this apple, but that's basically the idea. So I won't bother putting up our last slides, but it was really to talk about the national impact and the top three things that we hope come out of this. And what we really want is that we create a bank of resources that are tailored to the kind of courses that we teach in first years in universities and IOTs in Ireland and that these would be readily available to lecturers and students to use, that the research coming from the project is going to be available not just to people in Ireland but disseminated around the world. We're going to a conference in September in the UK to talk about the data coming from our surveys and we hope that that will continue with the evaluation. We want to evaluate everything we do. It all should be research-based so we know what works, what doesn't. And that the app, the app's a lecturer and student apps for the audience response system, we hope they'll be useful in lots of areas, not just mathematics but we could see them being used in lots of STEM subjects in particular and that they would be readily available to everyone in the country. So that's it really.