 Good afternoon, everyone. I hope you've been enjoying the sessions today. I have the pleasure now of introducing Charlotte Stevens and Sorry, I have the pleasure of Introducing Charlotte Stevens and Karen Hamilton from the Open University. Their presentation is titled the show must go on Reflecting on transitioning the program of face-to-face staff development events online They'll be finishing the presentation a bit early to allow for questions So if you've got any questions, please do add them to the chat and we'll store them up to ask them at the end Over to you Charlotte and Karen Hello there. I'm Charlotte Stevens And I'm Karen Hamilton and we are part of the Associate Lecturer support and professional development team at the Open University in the UK Over the past year. We have transitioned a program of face-to-face Associate Lecturer staff development events online and we are here to share our learning from this experience First of all some facts about the Open University The Open University or the OU as it's referred to is well known for being a pioneer of distance learning for the past 50 years It is one of the largest universities in terms of student numbers in Europe and offers distance education in England, Scotland, Wales and Ireland and Over a hundred countries worldwide Here are some key facts and figures The university has over 175,000 students in 2019 from 2019 to 20 over 30,000 students declaring a disability studied with us at the OU Having an open access policy means that 33% of students have one A level or a lower qualification at entry The vast majority of OU students are part-time and most start by registering for a qualification and then take a series of modules Study is mostly online with regular assessments Students can complete a degree in three years, although many study part-time over six years or longer Open University students are taught by a network of over 5,000 associate lecturers who act as tutors marking assignments providing detailed written feedback and offering support to students by telephone email or via online forums They also run tutorials and day schools Some associate lecturers or ALs as we call them are full-time members of staff But most are experts in their subject who combine their work with other academic or industry roles We are very fortunate today in that Karen is one of our ALs as well as being part of the associate lecturer support and professional development team The team works with colleagues for across faculties units and programs to deliver resources Processes and systems to help support the work of ALs We provide training and development opportunities appropriate to the experience of those ALs delivering resources for the new AL to help them understand what it means to be an AL on a module and Opportunities for those who are more experienced to develop within the AL role and beyond We are a source of information guidance for colleagues across the university We manage a development fund the applaud program leading to a advanced HE accreditation We create resources across the university We are probably best known for our face-to-face Development events which prior to COVID would bring ALs colleagues from across the faculties Other colleagues from across the university and external speakers together to discuss the important issues in HE explore ideas around supporting students Looking at new plans within the university anything related to teaching and learning prior to COVID unique planning teams organized 10 face-to-face conferences across the four UK nations and Supported 63 specific faculty events in the year however in 29 in Yeah, 2019-20 three conferences took place and we became the experts in cancelling and negotiating refunds But we did finish on a high with an online conference which attracted 600 attendees approaching The 20-20 21 academic year everyone was determined to offer a strong program to bring people together and to share practice Although we recognize that these events would be online and we followed Rosman and Durdon's our aim of providing an experience that would hold attendees attention create meaningful and supportive experiences Which must have worked as we had more than a thousand ALs turn up to these events Whilst we understood that picking up the content of a face-to-face event and just presenting it online Was unlikely to provide the learning experience. We aim for some colleagues took some persuading and Some saw the online Alternative as a budget saving exercise But while online costs less in terms of venue and travel we identified the need for investment in technology in technological infrastructure and Training and the use of the tools particularly amongst our colleagues not directly related to supporting students Those who were unfamiliar with Adobe connect, which is the you the university's chosen platform did struggle a bit So using Adobe connect for conferences would enable presenters to model good practice and where necessary develop their own skills a Challenge was maintaining some social contact With some planning teams deciding to use forum activities and others supporting faculty colleagues to run events both within and outside of the conference structure Bamba and Stefani say that evaluation is important and educational developers will increasingly be required to demonstrate value This is relevant to us in our work since events if expensive whether they are online or face-to-face So we need to be able to evidence the value in the staff development We offer for both quality assurance and quality enhancement How do we do this? Well over the past two years we have developed a coordinated approach to evaluation of our events We recognize that there can be a tendency for evaluation to focus on outputs How many people attend workshops or how many workshops are offered and these numbers are easily obtained We also recognize that there can be a tendency to focus on hygiene factors Particularly at face-to-face events such as the venue or the facilities on offer Whilst outputs and hygiene factors do have some value They tell us little about the impact of the staff development on those who participate in it The outcomes the learning the potential changes in behavior and whether the events are useful in helping our associate Lecturers develop in their role or what they might do differently to support students in the future as the result of attending We wanted these kinds of questions to be at the heart of our evaluation We developed a co-evaluation model presenters at the events and associate lecturers are invited to complete a survey immediately after the event takes place The survey questions are informed by relevant models For example Kirkpatrick's training evaluation model and Guski's five levels of CPD evaluation From the ALs we seek to gauge reactions as well as learning and plan changes in behavior Through questions focusing on expectations Reflections and interactions We ask ALs whether they recommend others attending future and we ask them for feedback on individual sessions Which we send to presenters We also ask them which topics would be most relevant to them based on how long they've been in post to help us with forward planning Presenters are asked how they think their sessions went what they might do differently in the future and whether presenting at the event was useful in contributing to their own professional development We use JISC online surveys and Envivo Plus as a qualitative data analysis tool We aim to offer presenters feedback on their sessions within two weeks of the survey's closing and we share full reports with senior teams Highlights from our own highlights with our own team and disseminate findings across the university thereafter So what is our evaluation told us and what have we learned as a team through transitioning to online only events? Let's start with the associate lecturer evaluation Over the past year. We have received feedback from 509 associate lecturers, which gives us a response rate of 47% 93% of respondents found the events useful in helping them develop in their AL role The word cloud here provides an overview of the qualitative data We received when we asked how the event was useful to them We can see words such as students sharing hearing practice support learning thinking and colleagues here 80% of respondents said that they reflected on ways to support students after interacting with others at the events We asked them how they would change the way they supported students as a result of attending and you can see here another word cloud Which captures the most used words? We see some similar words that were used before such as ideas and learning but also other words like tutorial tutorials feedback and forums Finally 95% of respondents Recommended that others attend a similar event in the future, which is very encouraging for us to organize We received responses from 117 presenters, which gives us a response rate of 52% When we asked presenters whether presenting at the event was useful in contributing to their own professional development You can see that the majority here said yes The reasons they gave were that it enabled them to engage with other stakeholders and learn from them It helped them progress projects or activities. They were undertaken Undertaking or informed their research It helped with developing confidence in designing a workshop Improved their online presentation skills gave them the opportunity to present collaboratively and gave them an understanding and appreciation of how associate lecturers use Adobe connect to teach So what have we learned as a team? Bambra's Defani suggests that taking a stand on Demonstrating the value of our work does require developers to be courageous And it wasn't just the developers who gained Courage the whole team developed new skills and new ways of thinking and working We discovered that we needed to look at new ways of delivering learning opportunities and not to be fearful of working in different ways We learned forgiveness and being patient. It is not necessarily quicker online We've learned to be a lot more adaptable and this is isn't just one way of doing things We can find lots Our audience the ales have been particularly enthusiastic about engaging with the events and very appreciative of everyone's efforts What can we do better? We'll be encouraging presenters to record sessions to enable those not able to attend on the day to access the learning We won't be so afraid of trying something different We'll be looking to offer training for all presenters to familiarize them with the adobe connect system Looking at better attention to detail and Make time to check each other's work as the spelling monster seems to hide around every corner And we will share the evaluation There are members of the university who have no idea what we can do to help with their projects We need to tell them Thinking about technology We accept that there are many different systems, but being consistent and using just one for the program seems to save time in production Looking to develop new skills and confidence across the team early in the Program year is something we're hoping to do Things will go wrong but having confidence and being able to practice the skills can help you cope when it does Technology is not a replacement for pace-to-face, but we don't need to be afraid of it Looking at the content We're offering a wide and varied program which draws more participants sharing practice ideas important sessions given Particularly by al's for al's not only sharing practice ideas, but developing bonds across faculty boundaries Being online also enabled us to have a more diverse range of presenters There is a need to preserve the content either by recording or making summaries and Then making sure that you advertise what is available and where it can be found 84% of our associate lecturers said that they would make use of materials on our VLE site This led us to think about what we provide in addition to session recordings and presentations And we started to ask presenters to provide a summary slide for their sessions, which we could collate into a book of slides This now sits as an interactive resource on our VLE site and we are reviewing how this was received by the AL community So what have we learned about format Feedback from associate lecturers has made us consider the timing and duration of events The timing of an event never suits everyone Regardless of whether it's online or face-to-face For some online is better because it's it's it's a better use of time than spending long hours traveling to a venue and Delegates can fit being at the event around their other busy schedules But a day online is a long day Some associate lecturers told us they struggled with this We have learned that we cannot expect them to attend all day all day as they did at face-to-face events They will drop in and out. So we need to readdress our expectations We received positive feedback for events split across an afternoon or and morning. So we're thinking about running half-day events Our evaluation has made us think about how busy our programs are and how we need to consider break times carefully and build in time for reflection In addition to the day itself There is a lead-up time and a follow-up follow-up time to an event that we must consider carefully There's a lot of work to be done in these spaces which can require a change of mindset amongst team members And what have we learned about interaction in the online space? Well associate lecturers told us that they appreciated being part of a shared online learning experience especially during an isolating time But it depends how involved the AL feels in that space We know that being an associate lecturer can be lonely for some being at an online staff development event Rather than a face-to-face one can mean that sense of isolation is felt more acutely Being face-to-face provides a real link to other staff especially for new associate lecturers It is especially important for ALs to meet with other members of their faculties. They tell us this in their feedback Associate lecturers have told us that they like to be able to interact with presenters and with each other at our events When we asked which session formats they prefer we get some interesting results as you can see here The most votes are for workshops where there are more opportunities to interact The second most popular option is our plenary style sessions Many associate lecturers told us that they appreciate receiving updates on the university as a whole But they still like to be able to join in and to ask questions Which brings us on to managing the chat facility during presentations, which has been tricky for us Sometimes presenters prefer the facility to be switched off because it's distracting However, ALs tell us they don't like the chat function being disabled because it feels like a top-down approach Yet the chat can get dominated by some individuals, so it needs to be well managed in order for it to work We receive many comments from associate lecturers who miss catching up with their colleagues face-to-face whether informally or to share ideas For some those micro interactions such as meeting in the corridor or talking over lunch in the face-to-face space Are what makes staff development for them the most impactful Over the past year we have offered associate lecturers the opportunity to engage in forums social and related to the event But with limited success We found that these need to be well advertised and well managed to succeed So we have no easy answers But over the next year we will be looking at experimenting with more social spaces and see how we get on Stefani cited in that can suggest we exercise leadership each time we push for change Introduce innovations or influence others and we have spent a considerable amount of time trying new things Influencing our colleagues and inviting them to join in and Communicating across the University in the last year for the forthcoming year our events will be online again Is that a bad thing? I don't think it is because it gives us the opportunity to expand on recent achievements and it keeps us safe, of course Annual reporting has enabled us to put forward some recommendations for the forthcoming year Which have been shared with planning teams and anyone developing Events in the coming year We are also looking to add value We are now in the process of conducting a research study Contacting associate lecturers at three six and twelve month stages after the event To evaluate their long-term impacts on the staff development we offer The show did go on and now we're ready for the first on call. Thank you Thank you Sorry, I am to think of you keeping my mic commuted. Thank you Karen and Charlotte that was really insightful and it was great to hear about all the work that you've been doing and Your approach really is very reflective and how you built on it I was really interested in your work around evaluation And I noticed that you said you did that evaluation after the event I wondered as well if you did if you did or if you're planning to do kind of any of that longer term Evaluation to find out whether those changes that people talk about put in place They've actually put it into place or indeed if there were new things that they did that they Had an initially thought they would do as a result of your sessions Yeah, I can answer that one. We're currently undertaking a research project to look at that and we're going back to consenting associate lecturers three six and twelve months after the event to Find out really what happened. So we've done our we've done our three months round We're about to start a six month and the results are good. It's looking good So we'll have some great research to to build on and and use going forward That's really good. And I have you Along the way have you been adapting the sessions that you do based on some of the feedback you've been getting? I know it's about really good feedback. Did you just sort of get it right to begin with? Were you just a good at sensing what you've needed? I'll give that one to Karen. I'd love to tell you we were perfect right from the beginning, but unfortunately That's not always the way is it So yes, we've done some useful things where we've found it It's been the adaptable bit that's been the challenge for us Some of our colleagues have said but we've always done it this way and You can't just pick up and make this way happen when you're using different technology So so yes, we have been adapting Even maybe five minutes before the start of an event And I think it I mean are certainly the co-evaluation models worked where we're offering presenters feedback from the associate lecturers themselves and We haven't tested this but I'm sure some of those presenters will come back Having reshaped their content on the basis of that feedback Would you say that Karen? Yes, and of course, we are just literally starting today planning for next year Those that have presented in the last year can be asked how things have changed in their presentation in their skills as part of our Interactions with them during the coming year Yeah That's that's really interesting and I guess a final question that I'd like to ask you before our time ends is around Kind of sense of community that you were talking about. So you were talking about students really valuing the chat Even some although some of the presenters didn't like the chat being enabled and I wondered if there were any informal Methods that you used or methods you used to create a more of an informal community You know the coffee room chat that you only get when you're faced with it We haven't in the last year and this is something Charlotte and I have been trying to work on part of our reason For coming to this conference is to see what ideas are available. See how other people are doing it and We're getting ideas, but as yet we haven't progressed that on That may be a thought for next week and although we've had forums. We've struggled to keep people engaged in them So it's it's finding out it's finding out what's out there and what fits us actually what you know, yeah So that's why we're here. I think as well. And that's brilliant. It's brilliant presentation. Thank you Thank you Karen and Charlotte. Thank you. Thank you