 So welcome everybody on a second session in the open source education track. This session is called Tutors Tech, a student-centric learning experience. It is presented by Lee Griffin and Colm Danfi. Folks are from Ireland, and I believe they will say something about themselves in a short while. So good luck. Perfect. Thank you, Marek. Good morning, everybody. Great to be back at DevConf. Hopefully next year in person, but the virtual experience will have to do today. So I'd like to do just a quick introduction and a very broad overview of why we think this is an important topic before passing it over to Colm. So my name is Lee Griffin. I'm a senior engineering manager in Red Hat based in Waterford, and this is Colm Danfi, who's a lecturer based in Waterford Institute of Technology, which is our local college. And Red Hat and WIT, as the acronym goes for the college, have a very strong relationship. We've hired a lot of graduates from it. We take a lot of interns in, and we tried to give something back, as we all know Red Hat is very community-facing. So we tried to give our expertise and our insight, and this is really the result of that expertise and insight. So what we're going to talk about today is a very student-centric learning experience and almost to show how the college have flipped the experience on the head to turn it into a paradigm of engagement and to mirror how students and graduates will interact in industry when they exit the college system. So I'll pass it over to Colm, and he's going to take us through the technical side of it, and you'll hear from me later on. Over to you, Colm. Thanks, Lee. So I'm a lecturer on the WIT, H-Dipping Computer Science. This is a program that's been running for 10 years. We've had three cohorts at any given time, currently 155 students. And they all have to be working and they're all mature students in order to take the course. And by doing that, they get 90% funding from the Irish government through a springboard program. This is the first fully online program at WIT. So we're very excited to be able to do that. And we're on our fifth online cohort. In moving online, we needed a new approach to academic delivery. Single-user operated online course production and delivery system and a similar workflow for both synchronous delivery and asynchronous delivery. And on the right, you can see what a course actually looks like when it's delivered. And really, the goal is to produce a full-stack-orientated developer within two years. Motivations, we wanted to improve the student experience, but also the lecturer experience in delivering online. We wanted MOOC quality. You would be familiar with programs like Coursera, but we didn't have the budget. So it had to be open. It had to be free or at least very low cost. So our technological choices are based on that. So we had to build and come up with a framework for blended and online and digital delivery. And in doing that, this meant unbundling the learning management system and the open management system that we use is Moodle. But we had to make it better. So effectively, what we came up with was a coherent tool set for teaching online. And that's called Cheuter Stack. And this is all the technologies that we've glued together, if you like, to come up with a solution. So as part of that stack, we have Cheuters. And that's developed by my colleague, Eamon De Lester, who is the lead developer. It's an open-source platform for enabling lectures to structure and publish rich learning materials more easily. It can be done in other systems, but we wanted to make it more easy. Basically supplying to the lecture templates that they could edit as text files and then produce nice looking courses like you see on the right. That has been extended to include Cheuters Live and Cheuters Time. Cheuters Live simulates the live classroom lab experience where you can find out who's here, what are they doing, but in an online context. It's easy to do that in the classroom, but in an online experience, that's a bit harder. So we've come up with Cheuters Live. And this really promotes being part of a community online. And of course, everybody at this conference will know about how important community is. Cheuters Time then is for student interaction and engagement patterns. We want to be able to see what are the students doing, when are they doing it, how long are they doing it for. And if they're not doing it, then we can facilitate or use this to facilitate early interventions for student success. So let's look at the feature set. This is the Cheuter Stack. These are the technologies we're using. They're all based on being open, free and low cost. Video is a big element in MOOC systems and we're a Spock if you like, a small private online course. So video is extremely important. We chose the YouTube platform because it was free and it gave us many, many additional features. In order to get our signal to YouTube, because we use YouTube Live or both Live and On Demand, we use the Open Broadcast Studio software. And that gives us many advantages. We can have different layouts, we can switch between them and we can use green screening, as you can see I'm doing here at the moment. We're actually delivering through OBS. For annotations we use Screenbrush or IPvo Annotator. Zoom is part of the solution as well and has become part of the solution and I'll talk about that a bit more in a second. Layer three is about assessment and feedback. So traditionally people deliver their notes in Moodle and they add some assessment later. We kind of removed all our notes from Moodle and we only use it for students submitting formal summative assessments. We use Socrative to do informal or if you like formative assessment through quizzes and it enables us to share on-screen while we're recording live classes answers from students because it's totally anonymous. And it also enables us to look at many more wrong answers. It fosters that community and people getting in the class, getting more involved in the live classes, which again leads to a better experience. Community is a massive part of the success of the platform and before we were online we'd use Slack for about two years and we had fantastic success with face-to-face classes and it was a first protocol for using it for community on this program. You need to have a back channel. It's no good having messages coming from everywhere. And when we did our research, students who had taken other classes, their biggest gripe was the amount of communication that happens from lots of different systems, particularly email and discussion forums. We put everything into Slack. We don't have our students using email at all. And then we had the additional benefits that it gave us which was that you could screen share, that you could do a voice call, a video call and you could do remote desktop and this was essential in a programming environment where you're trying to provide support online. And we're going back, you know, five years when we're talking about this. It's all doable in Zoom now, but back then it wasn't. As you know, Slack removed some of those features and they advised that you use Zoom. So that's what we've done. And finally, we then get to our own contribution which is tutors, the tutors system for delivering all our online materials. And as I mentioned, tutors live in time are part of that. Now, all these systems generate lots of data and analytics but we couldn't get a clear picture of how our students were getting on and we couldn't do it quickly and immediately. So the extension tutors live in time are designed to do that to give us insights into how our students are getting on. It also helps with community behavior and presence and allows us to see those student interaction patterns and also student progress. The goal was that we could make early interventions for student success. We know that students in a semesterized system if they miss three weeks in a course, they are very unlikely to succeed and pass that semester. So we wanted to get in early and that three week is a magic number. So here's tutors live. When you're online interacting with the course materials you can go live, tutors live. And what that does is present a card which shows your profile picture coming from GitHub and it shows what you're working on, what the current time is. So it's very visual you can see and it encourages peer-to-peer learnings and support because you can see all your classmates who might be online working as well. Now they could be working on different things and the cards enable you to see who's working on what you are interested in. And that then kind of promotes making connections and discussing as you would in a real classroom but we're now in a remote environment. Effectively it's giving you the ability to look over your shoulder remotely. Now you can switch it off if you don't want to be part of it but most students don't. So what does that mean then for the student? Well, they never know really how much time they're spending on the course when we've surveyed them in the past. They're gestinating. So what we try to do is to put numbers on that and color code it. And you might be familiar with this type of look from GitHub, it's modeled on that. So the student can see how much time they're spending per week each week on the program. The lecturer can also get a view and the lecturer gets to see what all students are doing. So you can see their interaction or engagement patterns on the left here with the calendar. But then that is all summarized on the right with a summary of how each student is doing. That's a heat map, red means bad. And as you can see towards the bottom on the right hand side we start to see the students are starting to fall behind. So this then enables us to make contact with them through Slack and see that they need a leg up or a hand and prompt them to re-engage. So let's talk about the underlying technology. It's open source from day one. It embraces the Jamstack initially authored in Python then migrated to JavaScript and then TypeScript. Node command line apps generating single page applications and currently it's a Svelte application with Tailwind implemented with TypeScript. When the user is using the system they're originally publishing to GitHub or Bitbucket and to do that they needed to know Git and originally using the terminal then using applications like SourceTree and Git desktop. Most recently in the last six months we've managed to open it up to a wider audience lecturers who are not techies can now just drag and drop onto Netlify but that doesn't inversion it. So if you wanna get involved with our open source project it's tutors.dev and this is the team lead developer Amy DeLester and myself and we have three students who are involved and interning two of them are interning at Red Hat and I'll just go through them. Ben Copper who's involved in the engineering Emma Kidney who's been doing the creative and UX side and Jordan who's actually doing both and Peter Wendell as well. So you can see all the, this is a landing page if you'd like to get involved with the project. This is what our front end looks like. It's a hierarchical card based system. You start with a course, you drill into the weeks or the topics that are being covered and when you click on a card like forms it shows you then the subtopics within that and that will then lead you to the materials, the slides or indeed the labs and labs are really important in the programming sense and we put a lot of effort into doing step-by-steps to enable our students to do labs. It's not about theory, it's about the hands-on. Okay, so how does that work? We've got a static site generator which basically uses local files and it generates JSON. We have a course reader to read that JSON to produce the site. It also produces the time reader so separate reader to do that or application. We have static site generators are reading these files in a folder structure that's predetermined. Basically we're writing markdown, supplying PDFs of notes and these get JSON encoded, hosted on Netlify or Versedel. The course reader picks that up and we have a Svelte TypeScript application for the front-end that we've seen and also for the time reader we need to also interface with GitHub because that's where our students log in and OAuth and then Firebase for storing time data. So all code and sample courses are on GitHub. I think we have 29 repos and these are just some of them how they're related to what I've just described. Tutors dash SDK on GitHub if you'd like to clone those repos or get involved. So over to you, Lee. Thanks, Colin. So we've seen the amazing custom development work that the lecturers and staff have taken on to try and create and engage the students but that's only half of the story. What I want to chat about for a couple of minutes is the Agile semester. So Agile is really important in the industry. It's how we build and ship our products to customers and the community in a rapid cycle because the engagement paradigm has changed and people want faster iterations and they want to play with it. So if we move to the next slide we'll see the originals if we can hit that column. Yeah, there we go. So we can see the traditional academic experience and it is waterfall in nature and this is what I would have experienced and probably what most students globally are experiencing where you have a typical 12 week semester and what you do is you go through the weeks you layer the knowledge and typically around four to six weeks in you'll get the first project specification which is delivered around a halfway point. Normally that coincides with a little bit of time off for the students for midterm and if you don't get that right and don't layer that knowledge the second and really the latter half of the semester becomes a struggle because the capstone project is delivered typically at the end of the 12 weeks and how semesters are organized certainly in Ireland the end of the 12 week semester coincides with the likes of Christmas holidays or summer holidays which means the delivery point can be beyond the 12 week mark on it and that way you lose access to the lecture and staff and it's a big bang approach so if you haven't layered and gotten the prerequisite knowledge as you go through and that becomes a huge problem and as we see on the next slide that's effectively an anti-pattern so from an agile perspective that is a total anti-pattern and effectively here based on the tooling developed it's magnified so we have this real snappy approach to content delivery for early interventions but we're still working on a 12 week semester and what this also is the impact and I'm speaking as an employer here when we take students in they realize we're shipping something every two to three weeks depending on the sprint duration and that movement, that agile movement it's dominant for the last 20 years but it hasn't been replicated at an academic level yet and just as a quick reminder of what the agile manifesto is which is on the next slide here the agile manifesto was published just over 20 years ago now and it's effectively a set of principles to guide how we should be building software and it doesn't say abandon documentation or abandon contract negotiations and having a plan it puts a little bit more emphasis on the items highlighted on the left so we want to have individuals and interactions we want to chat with developers we want to bring people together we don't want to focus exclusively on the tooling and an overarching process but what I love about it is working software we want to get something in the hands of a customer or a community member so they can test it and bring something back and more importantly, we involve the customer there are an active element of it and not someone to forget it out once you sign the contract and taking this approach we try to look at what the agile academic semester would look like so on the next slide here we have this reimagined specifically for a college environment so a manifesto for agile education and if you look at how any academic course is structured you have things like a quality manual an academic calendar and effectively deadlines that you have to hit but we want to still have those because you're governed the course by regulations and so on but we want to put students and their interactions first and foremost because at the end of it all college should be about a successful student experience that's how you graduate people and not stick rigidly to the initial plan that you have because as we say at the bottom stuff happens every so often and we need to be able to respond to that and that was really the value proposition that led us to which is on the next slide the agile semester and just before we get there did 12 agile principles just as a reminder so the agile manifesto is complemented by these principles and we've highlighted the ones that we feel were most relevant to the student experience the two big ones for me is motivated individuals in a measure of progress you want to have people engaged you want to have people see that they're learning that they're progressing that they're moving towards a target which in a student's case is graduation and we want to be able to metricize that along the way and not figure that out at the end of 12 weeks so what we came up with and Aiman has helpfully pasted it into the chat we published an academic paper it's actually going to be a book chapter that's going to be published shortly about the agile semester and this is centered around the three week courses column so elegantly put it earlier where we want to have the ability to intervene in and around that time frame to try and bring students along to try and rescue them so to speak so we have the formative assessments those small, lightweight kind of approaches to quizzes and knowledge checks as well as more formal assessments as we go through it and by breaking them down we model the scrum approach to retrospectives by using that as a means to assess, to test, to measure and figure out what the learning outcomes achieved and by dividing it into these blocks you're getting a very consistent experience that mirrors what happens in industry so if we move on to the next slide we'll talk briefly about the impact so from a student perspective and we've gathered data since semester one since we've rolled out the latest edition of it for these insights and 87% of students have completed all classes in semester one and if we compare that from a retention point of view 80% of those students have continued on into semester two and the national average for computing courses in Ireland is 55% so it's batting well above the average which is fantastic and hopefully we'll come back in a couple of years with more stats to really drill down on this because it is a course only one full semester that we've gathered this for and what's really welcoming here is the community peer learning opportunity because that's how it works in industry, right? We all learn, we deal with community we deal with inner teams and this is leading to higher marks overall and this is giving a serious leg up for students who undertake this course because they effectively are in employment when they're starting the program and they have career progression to look at job changes, completely new careers and new companies by taking on the course because it has a built-in internship aspect of it so they can see the belief they're getting input from industry like ourselves in Red Hat and what I love about it when I'm interviewing students from this course is the skills are there the remote working experience is there and to be fair this was even in place before the pandemic but even more so over the last 22 months or so that we've been experiencing this and they have an experience of agile which is fantastic so they don't have that disconnect when they land in on the students so I'll pass it back to Colin to talk about the lecture experience of this. So I suppose one of the best successes is the energy and excitement we get by working on a course and working with technologies and a team like the one on the HDIP the big success was that this was our first fully online program we were years ready to jump in but to finally get one launched fully online and then it's running every year it has been a real success for us and that has led to excellence awards both locally and nationally and then when the pandemic hit the college was actually scrambling as were many to know how they were going to deliver remotely so our framework was used with some of the enterprise level options that were already in place including Teams and Zoom and WIT Chuter Stack was applied so swapping out some of the technologies but using the layers to inform what would be used in order to deliver online we needed to create studios and we called them pods and again, no budget so they had to be done really cheaply but give the same professional look and even getting budget for one was quite a deal at the start but we now actually have 10 of these in place and they keep building them also the flexibility that we get from being online professional development and the openness and it's led to a lot of additional research funding and more opportunities with universities both in Europe, the US and Canada that we've been working with and the open source success we've had 160 courses now using tutors the pods picture there on the top right and how they've been used but not just in WIT that spec has been used by other colleges not just at third level but also at second level education additional funding from the National Forum for teaching and learning enhancement and the team has grown finally we now have six people involved as we showed you since September 21 and they've been working on aspects that we really needed to get to accessibility user experience, media, creating that svelte application and documenting the whole thing it's finally documented and GIT is no longer a prerequisite which was a turn off and that's no longer a prerequisite for lectures to use it because of our new Neclify drag and drop and we're planning to move this forward as a formal open source project with the help of our friends in Red Hat you can find us on Github, tutors SDK these are the links we built a tutorial site using the technology tutors docs, Neclify.app there's tutor stack resources where we've got links to all the details for the different layers on the stack each of those you don't have been 20 minute talks before and we collected all of those links and you can see them there and we mentioned about the agile semester so there's a quick way of getting to that book chapter and that brings us to the end of our talk so thank you very much there's links in there if you need to get in touch with us Super, we have one question from Luigi in the chat here just a slight question on the video part have you considered Twitch which is strong interaction support and open for integrations and it's been used for educational purposes too? Yes, in coming up with the stack we looked to gamers we looked to farm influencers and beauty influencers and the worship industry and we looked at all the technologies that we're using and we've tried out different flavors you know we've looked at many alternatives to Slack but we're not saying what we've got is watch everybody has to use we've come up with the framework and you can swap in and out those technologies but the one we've got is working well so we've originally started with Adobe Connect actually which was supposed to be the one stop shop for everything and I remember presenting a conference where I put a headstone and put Adobe Connect on the headstone so we do swap in and out the technologies we've looked at Twitch but we haven't switched to it And just to add in there any of those integrations so be it with Twitch or like Google Docs or other markdown kind of editors or even Pay Spins like they are perfect API integrations for open source contributions Maybe you'd like to help us and try out Twitch for us maybe get involved All right so thank you we are out of time unfortunately I would have some questions myself but we will have to leave it at thank you once again Kom, Lee for joining and for your support crew on the chat Thank you very much Super great experience thanks for having us folks Thanks, thank you