 Rwy'n meddwl. Felly yn y ddech chi'n ddeg llwy. Yn y ffwrdd, rwy'n meddwl, rwy'n meddwl iawn, ddim ddim ddim ddim ddim, dwi'n meddwl i chi ddweud, dwi'n meddwl i'n meddwl i chi ddim ddim i ddweud, a'u dru i'r ffasio ar y bwrdd yn ystafel yn mynd i ddweud dwi'n meddwl i ddweud ymgyrch yn y ffasio gwirio. A dyma. Er gyda'r ffasio, dwi'n meddwl i'n meddwl i'n meddwl i'r rydw i. My name is Keith Russell. I am the interim director outreach at the Australian research data Commons. The first thing I would like to do... It's a tough act to follow after Aunty Joan. But acknowledge the First Nations peoples and the land on which we are meeting today. That's the Gadagal people of the Oeron nations. I would like to extend my respects to elders past and present and also to all First Nations peoples here today. Felly, rhaid i'n arcofio ar yr ARDC, ymddir i'r fforddar ac pan wneud hynny, a fyddai sydd hodd yn fwy o'r ARDC yma yn parod, ond ar ôl ffordd ar ard instrumentsau ar ARDC er eu cynnig bobl yn dynnu'r ffordd yn yr nghaelat, gan gynllunol yng Nghyrch Aelodraeth o Gwrthwyneb Iurethiol, i'n frathrwynydol Cymru, a'r hoffau yn fawr mae gweld yn fawr ar hyd yn gyr warning cyffoedd. Rwy'r ddweud o'ch gwrthwyneb i wneud, dyma yn y rhan cyhoedd, Some of those are specific to disciplines, some of those are instruments, some of those are facilities. In the case of ARDC, it's very much across all disciplines and it's about data. So our focus is very much around looking at data and making sure that researchers can use, create, and reuse and make available high quality data assets. Thank you, that looks familiar. See if I can work around this. That's right. Ah, brilliant. Thanks, Mary. So it's great to have a wonderful team that helps you out. It's amazing. So yes, our purpose is to provide Australian researchers with a competitive advantage through data. Now one of the questions you get a lot about is research data commons. What does that actually mean? What do you mean by that? What does that look like? And we see it as a sort of a virtual place that brings together data and tools and assets and compute and models. But if you look at this diagram, one reason I pulled out this slide is because right smack bang at the top are people and skills. We think people and skills are very important elements in a research data commons. If you want researchers to be able to use, reuse and create data, then they need to have the skills to actually be able to do that. And they need to have staff to support them in that. Now we're not alone in thinking that. There's been multiple reports out there emphasizing the importance of skills in different aspects and in different perspectives. I've pulled out just two to provide a little bit of perspective. So one of those is a report that was published by RMIT Online 2021. So this is very much just coming out of COVID and thinking about Australia in a time post COVID. First of all, they identified that 87% of jobs require digital literacy skills, at least at some level. So this is crucial. It's so important to ensure that the Australian population has digital literacy at at least some level. And as they were talking about technology workers, talking especially about the importance of digital technology workers, and emphasising that we're going to need 156,000 more digital technology workers by 2025. That's only a few years away. So there's quite a big challenge for us out there to look at how we can increase those number of people out there that are available to deliver digital technology. Another report by the National Skills Commission looks at emerging skills. And as it looks at emerging skills, it identified that actually digital and data skills dominate. They are the most important, one of the most important areas that is where the demand lies. And they are seen as gateways for transition between jobs. So there's clearly a big issue and a big challenge for us out there. And if you look at, they mapped out the cutting edge skills being required and being asked for. And you see here a number of the, well, all of these are very much in the digital space. So cyber security, machine learning and data visualisation are identified as key areas that are being required and being requested. Now to get to these skills doesn't happen overnight. This is not something you can teach somebody from one day to next. What we need is pathways, pathways to get people upskilled and to gain that experience. So we need to think about this as a complete pathway running across K-12, vocational education, higher education, but also ongoing learning as people are employed through government and industry. And thinking about the emerging technologies as they change and as the needs change, making sure that people can adapt to that and they have the skills to adapt to those changing needs. So back to ARDC and where we're at. Well, we, from our perspective, we care a lot as a research infrastructure facility. There was a research infrastructure roadmap and in that research infrastructure roadmap, I talked a lot about skills for actually using the infrastructure. It's all very well to have all these wonderful instruments, but researchers need the skills to be able to understand the data they're getting off these instruments and being able to actually use these instruments and they need support. They need workforce skills and career pathways for those research infrastructure staff that can support them in using the data and analysing the data. So it works on both sides. Now ARDC has been working on this for several years now. We have a digital research skills agenda, which picks off a number of areas we're working on. I won't go into all of the detail here. A number of things I wanted to highlight, things I think are important. For one, I think skills summits are very useful and very valuable. We've done a few of them face-to-face. Then we had the COVID years, so we had to do them virtually, which was very sad. And it's lovely to have you all back here face-to-face and seeing everybody in person who will allow these rich conversations and discussions we're hoping for for the next two days. Also, we've been working on the upside-down carpentries. We normally do them the right way up, but we're quite flexible. The carpentries as a model for teaching digital skills to researchers and getting more universities on board with the carpentries model in using that. Another piece of work we've been working on is Dresa, digital research skills Australasia, a portal. We've built this together with a range of partners from around Australia. What we're doing with Dresa is trying to bring together the events, the materials, the training providers, and the actual training materials that are being made available. So researchers can find what is there and find the right training for them. We've got two action-packed days ahead of us. First, today is very much focused around success factors for training. First, we're going to get a virtual presentation by Jason Williams from Cold Spring Harbour in New York. I'm really looking forward to that. That should be fun. After that, a range of lightning talks and discussions. Hold on to those lessons. Learn that you're going to be picking up today because tomorrow we want to apply that and say, well, what does that mean for national skills policy? How can we apply that? How can we use that to narrow the skills gap? Thank you all for your attention. I would like to hand back to Catherine. First of all, I'd like to say a big thank you for organising all this and the team for making it all happen and getting all this sorted. Thank you. It gives me great pleasure to introduce our keynote speaker for the 2023 Skills Summit, Jason Williams. I must give another shout out to Christina Hall and Melissa Burke from Australian Biocommons for introducing me to Jason last year. I got to meet Jason in person because he was visiting Melbourne where he was hosting a number of workshops and bike principles consultations, which I managed to attend some of those too. Jason is an assistant director external collaborations at Cold Spring Harbour Laboratories' DNA Learning Centre where he works to spread hands-on biology education internationally. As education outreach and training lead for CYVERS, the US National Cyber Infrastructure for Life Science, Jason provides training and support to scientists and educators, helping them leverage the most advanced tools and best methods for research and education in data intensive biology. Jason organises, instructs, and speaks at more than a dozen bioinformatics workshops and conferences annually. Additionally, serves on several committees and boards for projects that advance science and science education, including his service on the steering committee of the Software Carpentry Foundation and was chair in 2016. And as an instructor for software and data carpentry, organisations that centre around scientists, teaching scientists, computational best practice. And Jason is co-creator and lead instructor of the Science Institute at Yeshiva. Did I pronounce it right? University High School for Girls. Now, I hope that bio was okay. You didn't tell me otherwise. So, yeah, cool. All right. Over to you. Let's get into it. So we even have time for questions and discussions. I'm going to share my screen here. And once again, thank you so much, Catherine, for the opportunity to present and thank all of you. For, I guess, thankfully you're in person. I am on Zoom. So I hope you get something out of this and enjoy. I'm putting here a QR code and a quick URL. So I believe that will work for you to be able to download slides. So feel free to have them. Also, it's an accessibility thing. Someone might prefer to read the slides on their own device. So I'm keeping that up there for just a moment more. And hopefully that works. I haven't had time to test it. And I also resisted the urge to change my slides once you post them and everything like that. As Catherine mentioned, I'm at Cold Spring Harbor Laboratory, which is almost on the opposite side of the Earth, but I hope you have a chance to visit one day. It's quite nice. I enjoy it. And within that, I'm at the Learning Centre. We do focus on education at many different levels. We have middle school programs, high school hands-on activities, all types of project-based and mentoring activities, and also faculty training, which is what I spent some time on. We have a whole collection of websites and online education, and we also have various centres in the New York area, as well as centres that we've either collaborated on, including, well, I think you're not in Melbourne today, but in Melbourne and elsewhere on the planet, we have the opportunity to really work with fantastic people all over the world. So I have a question for you, and you can think about it for a second. Take a second, although I mean, don't skip ahead and spoil my surprise. But what makes a superhero? It's an interesting question. If you indulge me for three or four minutes, it's a question to reflect on. And I'm asking that question, and it's a little bit inspired by this graphic, which I thought was cute, and which I dug up from some old slides. And it talks about superheroes, and actually they're sort of origin stories because the title of this talk is about what's the origin story for the modern researcher. And there, at least in whoever put this together, they said, well, there's a couple of different ways that you get to be a superhero, at least according to a comic book or a Marvel movie or something. You might have the right technology, or you might have some trauma in your life that you've gone through that turns you into a superhero and you hopefully have to become. You might be just rich. I don't know why exactly Scrooge McDuck is there. I guess I use him again. Or you might have had just a lot of training that enabled you to become a superhero. It's interesting, right? Batman is at the center of this, because I guess he had all of those. So that's an example of a superhero origin story. Now, part of my thesis here with this or my running thing with this is that we are obviously we exist within a structure, particularly an academic structure. And the argument that I'm making is that the structure, the way that academia is, I didn't put a warning or a warning label. The structure that we have in academia is built around, I'm saying, the origin story of its quote-unquote heroes. In other words, there's a picture of who is, what makes a good scientist, what makes a good academic. And even if that person is not real, we've actually built the system to reward that person and not necessarily the people who are ordinary people, who are all of us. Maybe there are some superheroes there. I find that to be a fantastically bad idea because I think when you give certain people power to make decisions on other people's lives about what good and bad is, you can very easily give that power to somebody who might not do the best things with it. So that could happen in this case. But also, I think people have seen the idea of the stereotype scientist and there's all sorts of things to talk about who is a scientist. So what am I talking about and bringing it in because those themes of treasury and trauma I'll expand for a moment and talk about it. This was a slide and a quote from NIH. There are a couple of things in here. Actually, I'm in the US, so I'm going to be a little bit talking about our own system. You know that in Australia. You have a more perfect system. I don't know, you can tell me. But it was very interesting. I pulled this quote from a blog post talking about where does the research workforce go once we train them. The quote from this, it was 2012, so maybe people are a bit more woke now if we want to use that term. But the person said I was quite surprised by the idea that the majority of our trainees do not end up in academia. Did this surprise you? Right? So I mean it's just a reflection. I can't quite pass judgment. But it's a reflection right that there are people in power and I'm not meaning this to everybody's hopefully gets their chance at the wheel to make decisions. But are we in a situation where the people who are making decisions about how things work in terms of workforce and training, they may not be connected to the reality of things. If you want to ask the question stereotypically or cynically I don't know about how to be a successful researcher what are your strategies going back to the superhero theme? Well you could just be rich. That might make you a successful researcher. Again Scrooge McDuck was noted on the earlier slide. I don't consider him a superhero but maybe he has so much money he was just on that first slide. This is a just a look at the funding situation at NIH. This is recent. This is 2021. And talking about what age do researchers today get their first major research reward and that time has definitely been creeping up there although people have been trying to counteract that. But in other words, if you happen to strike that early on, that could lead to a successful career. But as we all know, and this one is also NIH and it's also I think similar if you look at this different Australian funding mechanisms that the success rates are really low. So I'm going to argue that there's not going to be many of us who are going to become superheroes just because we happen to get a lot of grants because it's not going to be that many people were able to do that. Another strategy for being a successful academic could be just during the pain I'd argue that's not where we want to go. This one is looking at again the funding level and just looking at the fact that the success rates versus the time that you have succeed is really putting in danger our faculty, our younger faculty today. And then we all know the idea of well-being that it takes to be able to survive a PhD is something that's becoming really not tenable. If you go further and this one actually was actually taken from Australian Academy of Science, now when you talk about the career progression pipeline this was just an illustration from this particular paper talking about all of the barriers that we face even once we've gotten our degree or whatever the credential is that is required for our career everything from recruitment issues, it could be lack of equity, it could be discrimination of biases that really trim the path to becoming that senior level researcher which is the mythical academic that everyone's supposed to be a full professor that's well-funded in an endowed position. Another thing is you could have the technology that could make you a superstar academic. It's kind of related to funding as well but you could be really smart and be in the situation of which you have access to new technology methods because you're trained on them this is a look at machine learning publications as you could see if you're graduating more recently you could be in a situation of which you have access to technologies that are really hot right now maybe that will always happen but it puts you in a really interesting situation, I really love this paper talking about that particular when we think about machine learning or AI or things like that that you have challenges there but maybe that's something where some people succeed by virtue of the fact that they have access to that first and others are still waiting to get in. Those are all issues in academia, the funding levels the stress and the obstacles to actually getting that academic position and even the difference in technologies but I think a lot of these things have been talked about. I do not by any means say that they've been solved there's a lot of work to do with all of them but at the very least if I was to try to look for one positive to be able to see that there's at least conversations around improving and equalizing funding there are at least conversations around work-life balance there's at least conversations around diversity equity inclusion those are fantastic, those are great however I think that there's a lot more work to be done and there's something we haven't actually talked about except maybe at places like where you are which is a skill summit so we are talking about it what about training? How do you get a superhero who's well trained? I could think of beast because I guess he's actually known for being an academic and you know how do you actually think about training beyond the formal training that you get in the the ordinary course of your degree and my thesis and what I'm here to talk a little bit about is the idea that in order to really be successful as a modern researcher we need to really get serious and think of new ways and open up conversations about career spanning training really thinking about the fact that the shelf life of skills that we get is not the myth but the reality of what it's going to take to be more successful when we talk about our training today. We did this study a little while back which was discussing with and serving researchers funded in our case by the National Science Foundation and asking them what were the things that you most need to be more successful at dealing with big data in our case and all of them said that it was training that was the most important thing for them and the most unmet need in order to be able to deal with the realities of doing research today. I found this article interesting so I always cited this by actually a pair of economists who really characterized the way that skills work in STEM careers by saying that it's not actually they make the case I didn't quote it in this particular quote but they make the case that when we are always talking about the STEM pipeline that term gets used in many places but they made the case that the STEM pipeline is not because the STEM workers are shortage in shortage it's that the skills are in shortage and in STEM in our areas that people in this room cover those skills get old quick unfortunately I wish they didn't but it's kind of true this is the reality that yes not most of us are going into the traditional academic setting but quite a number of us are taking the skills that are learned in academia and taking them into many other different places and so it's really important to get away from the mythos of academia as being a single track of this mythical origin story where everything turns out comic but perfect to really thinking about the realities of what we need to be able to not only consider other factors like the training that you like how you're supported by funding agency or how we can improve the academic climate but also the training that we need to actually support people across their whole career span because if we don't I wonder if we're not getting to an area where you end up with a scientific digital divide where you just have a place where academics who were not able or unable to keep up as that interesting paper said it's impossible to keep up maybe in some areas it seems where we have a disconnect and how do we bridge that gap between that formal training which I argue will never keep up because things will always continue to change and how do we bring that into a sense of career spanning learning and actually prioritizing the idea of career spanning training because the brain that we had decades ago whoever graduated the people that are sort of making decisions now I'm starting to be allowed I'm from I guess from the 1980s and starting to be allowed to make decisions not quite but sometimes they let me do it how do we actually deal with scaling people because we all have the same brain more or less at least we get hopefully a chance to start on the same page how do we scale people to do that so I had mentioned this one I do bring back the quote it looks like where I say that rate of change stem that drives this and I brought that sorry that these slides got repeated there and it's also really important because it ends up being an issue of also equity and inclusion and justice and these are themes that we've talked about in other academic settings but it's going to turn out that the degree that you get from one university and the training that you get from one university may not be exactly the same as you could get from a different institution and not being able to address faculty needs whenever and wherever they manifest themselves is an issue there that we have to fight against so if I ask the question how many of you have taken a workshop to improve your skills I'm sure many of you your hands would go up and workshops are definitely one of the ways workshops boot camps trainings I think was mentioned in the intro of the Carpentries is one group that I am sure many of you have gotten involved in workshops and boot camps in short courses are what we call short format training is certainly a way that many of us actually look to be able to get more of the skills that we need and I'm sure it's also something that's being thought about in that room as a skills summit how do we deliver those skills to the people that need them when they need them as they need them but what stopped me a few years ago or at least prompted me to give a lot of thought to this question was this particular paper which goes into the idea that a lot of what we may do in order to accomplish that training may not be as effective as we imagine sometimes now this particular paper suggested and it's not the only one or not the only reason or not the end of the question that a lot of what happens in training might be ineffective they did a longitudinal survey of several hundred students across many institutions in this particular case looking at a whole range of training that the students got to to improve their computational skills to improve their research presentation skills their ability to write and deliver papers and talks and articles and they found that they couldn't find a difference between the students who had received those extra trainings and not and they come to the conclusion that a lot of what we do for that type of training is not working, it's not effective that is certainly something that could be explored and it's open to interpretation to some extent but it was something that definitely started me thinking and many other people too so the work that I'm here really to talk about after that lengthy introduction and trying to give a little bit of setting a stage here is that myself and a group of several other people have been taking it to heart of how do we improve training and the project that we're working on and hopefully soon to be published the manuscript draft is on my computer waiting for the last dottings of eyes and crossings of teeth is we've been working on a minimal set of principles for making that type of training effective, inclusive and career spanning the idea that we could take what all of us do and really approach it in a rigorous way to give us a higher degree of certainty both us as instructors and also for our learners as we are delivering them is really going to be helpful and really have a good sense that what we're delivering will make the impact that we hope and this is important because after your undergraduate degree the way that learning works is so different that it does merit a wide consideration as you go through your normal learning progression you complete your undergraduate degree in STEM maybe you have a postdoc which gives you an additional chance to learn and maybe even you have an internship or other opportunities but when we're talking about how people really learn after that undergraduate setting on the one hand I guess you could just assume everybody's smart we're all researchers, we're all scientists we're all smart you're just going to learn things but how do we actually learn and you could see if this resonates with you it's certainly based upon my own fears and aspirations and feelings so you can use this to analyze me but oftentimes the learning starts when I'm looking at a talk listening to a talk or reading a paper looking at a blog and I'm like wow that's interesting, that's important I don't know how to do that I need to know how to do that so that might be the first thing the motivation for learning appears and then I want to do something about it signing up for an online course, taking a tutorial maybe that will be your next step if you start to really get into it maybe you start to subscribe to a YouTube channel obviously we live in the age of those things completing any of those things are actually getting meaningful stuff it depends, sometimes you can but then what are other options because then you start to realize wait a second, I don't have even the time to work on this maybe I'll be able to work with a student or a collaborator maybe I can hire somebody who can do it maybe I can do it next year if I give myself more time let me just go ahead and find any little detail that will get me to what I want to do today and maybe we'll work about it next time the other thing that I would do if I'm trying to learn something is follow people on social media who are dealing with that challenge and maybe I can learn something from them but if there's no other choice then finally comes down to can I actually apply and will I be able to apply what I learn in any of those things because that's the kind of key thing, can I do that so that's very very different because in short format training I've only got hours if I'm taking a workshop or being exposed to that to get any of that learning I'm really focusing I really do get a chance to learn something but when it comes to how those short format training workshops work or if I'm thinking about my learners as an instructor I know that the people that are coming to me they're prerequisites they might be difficult to get people to understand what I need them to come to me with and I can't make them come to me with those things when I'm delivering short format training the learners that I have needs which are quite heterogeneous if I contrast that on the right to what we're calling long format or if you think about the type of instruction that you get an undergraduate I've got quarters and semesters to deal with people I have a lot of time for lecture to give them underlying concepts I can really make sure that they have prerequisites before they come to me and then I can specify what learners need to do before they come to me that's all on the learners side when it comes to me as an instructor or when it comes to the university that supports me or the institution that supports me on the left hand side again with short format training usually as an instructor I'm a domain expert and I don't necessarily know how to teach you hope not always the case but you hope that your instructors at the undergraduate or graduate setting do have some qualifications there when it comes to regulations or the actual policies that my university has it could be everything from policies on how curricular design to policies on how the room should be accessible that's unlikely to happen in short format training whereas in the normal classroom setting there's usually standards or sometimes even legal requirements there's all of these I've read half the slide but you get the idea that it's a really different setting to do short format training versus long format instruction and so the group of us including some wonderful folks in Australia came together considering all of these methods I put their names or all of these considerations all of these problems I'll put their names up briefly on the screen just to recognize them because this is not just my work but the work of many many people what we came together where were these principles as I mentioned because sometimes if you put 10 things down people will listen to you so writing down principles is a good thing to start with so what are they what are these principles that we came up with why should you listen why should you consider them well we hope that the principles really have taken in because if you looked back a moment ago on the screen of the different institutions and different groups that were part of this group the principles were really trying to be additional consensus to label and capture the best of what many training programs out there maybe not all of them but what many of them are doing and also the idea of these principles would be to provide a path for individuals or for groups so that we could develop an experience try to think about how do we shape short format training into something that anyone could anticipate how it would be structured that they could be I hate to use the word guaranteed but they could have assurances that what they're getting in that short format training is going to be of a certain quality and hopefully therefore of a certain impact and effectiveness and then also when we're thinking about short format training the idea here is to take values that we all have as instructors and practices which we all believe in and really make them highlighted because these things can get lost in the fact that if I tell you to take a semester's worth of content and try to deliver it in a couple of days you're going to have to make a lot of cut and while obviously yes you do have to give whatever you can do in the time allowed there's certain things that we don't want to lose because they're too important and we don't want them to be lost in sort of the short duration of the experience that we have to give them up so let me give you there's a set of core principles there's four of them and there's a set of community principles I'll explain them and then I'm finally going to wrap up by coming to this idea of how do we use all of this stuff to professionalize training which is one of our recommendations so let me go through the first four core principles first and we say that these are things that all short format training should do the very first one is this idea of evidence or best evidence that when we design training programs we need to ground them in what we know works from the education sciences and also because often times there are things that we know work because we have data but they may not always even be published how do we actually also use the evidence from formally evaluated instructions so the absolute bottom of the list of the foundation that we put together the second principle is this idea of what we developed term catalytic learning how do we when we're designing a training workshop how do we prepare learners to succeed when what we have to actually give them is not enough I think we often know that when we're training people we can manage to convey a few skills we can manage to convey a bit of knowledge but often times there's a lot more that they'll need to get after the classroom or after that experience and they're going to be on their own they may need to take other courses they may need to do some study or they may need to just get out there and try it so how do we prepare learners to be successful at self directed study after the experience we give them the next one is this idea of effectiveness this one always reminds me of the quote that you cannot change what you don't measure sometimes in short format training we don't do assessments in evaluations sometimes we do and some programs do or sometimes there is some type of evaluation but it's more of was the experience positive enjoy the workshop was it too fast was it too slow but this idea of effectiveness is the idea that we need to have the right assessment in place that actually demonstrates to the learner and to the instructor but in this case at least we're emphasizing the learner that the learner has moved from point A to B in whatever the objective was to learn so the learning goal or the program the programmatic goal may be things like we've created an inclusive space and learners when they take our courses say that they felt included say that they were able to access and use the materials so that's the idea of providing that evidence and showing that what you're doing is actually effective and the final core principle that we suggest all short format training should do is to be inclusive and that really means taking time and doing due diligence to make sure that learners are able to fully participate in and benefit from the learning experience now complementing those set of core principles there's a smaller set three of them of what we call community principles now these apply when the short format training we're designing is not a one off it's not something that you're just doing once but it applies when you actually are going to start to take that training and get others to use and reuse it to spread it and to work together as groups so that large numbers of people could be impacted by that training so the first of these three community principles is reach which is principles surrounding the idea that you need to include new types and larger audiences of learners so your original training format or materials or approach or methodology might need to be altered in order to reach new types of learners or larger numbers of learners and you have to consider in the process of doing that do you lose something what do you lose and can you afford to lose it because maybe you need to reconsider if you compromise the quality of the learning the next one is the idea of scaling which is how do you affect or impact or prepare the instructors and the instructional designers or developers so that they are able to take what you have built in one setting and scale it out and the idea of sustainability which could mean many many different things it could be the availability and usability and the relevance of training materials when do they expire or how are they kept up to date how reliable are they what was the expertise of the people who pulled them together and then how do you build infrastructures and those infrastructures that support training materials or training programs could be the trainers themselves but also the communities that allow them to be effective so all of these things and you might have heard illusion to earlier this idea of the bicycle principles the reason why we use the word or use the bicycles metaphor is because you could think of these as iterative processes that feed off that form a cycle and you could think of a unicycle as hey you know what if you were only doing a training or putting together an experience that's for a small group of people limited time only going to be done in one setting that's fine and you could stick to those core principles unicycle and that would be fine to do it that way but if you are going to scale up and scale out that training how can you then think about and scale and sustain those community principles to have the full bicycle which is good for going far in this illustration so you can take a look at this here's the second QR code because all of these things although the paper is in drafting all of the key findings are out on the bicycle principles website and I have just a little bit of time in the next few minutes about some of those recommendations and focus on one of them which is the idea of professionalizing what we do and how could that be achieved and what might be the incentives for doing something like that so some of the example recommendations that this group that I alluded to earlier came up with well each one of them when you do go into the site as I present a couple of examples summarizes what that recommendation is and gives you a very quick illustration of how this could work it also says how the recommendations which are specific instances of what we suggest are related back to the principles and then also the entire group's ideas on how could this benefit the learner so it's a bit learner centered intentionally how could this be incentivized for the people who actually need to do the work sometimes that will be individual instructors but a lot of times it might be groups of instructors and then what are potential barriers that we would need to overcome in order to make this real you can also while you're there visiting give us feedback as to what you think and do you think this could work and as mentioned earlier by Catherine we've done some focus groups is one more happening in a week or so where we're going around to groups of people trying to get as many different opinions as we can to develop the evidence base that will allow us to deploy this in many ways both we meaning me in my particular position but really anyone who's interested in this because this belongs to the community came from the community we love to support anybody who would like to take this and make it better so here are some recommendations and because the recommendations are somewhat abstract they are accompanied by images generated by artificial intelligence many of you played with these tools and rather than look for copyright free images I've just always put prompts in and so the prompt is underneath the image I only told that with three or four of these but one of the recommendations D they're not in any particular particular order but D is the idea of operationalizing equitable and inclusive practice as ethical as an obligation so again if we walk into the classroom to teach often times we've had the whole support of the university which has really thought about what an accessible classroom means has perhaps an office in the university that's dedicated to accessibility standards and has assistance for you as an instructor to make sure you meet those standards but what we often find I have data here unpublished to demonstrate that but I think for those of you who do instruction this will ring true is that often times when we do short format training we're not doing it within the formal university context although we might physically be in the building and so the idea of inclusion and equity may be up to us and if we're well versed in these areas and we're very conscious of them we may do it well but if we're not or especially if we're just putting on an informal workshop we might not have considered this but this recommendation says that it really should be an ethical application and it's something that merits discussion although we won't focus on this one today it's certainly worth a look this other one alludes to some of the issues I mentioned earlier the idea of deploying training to counter inequity and what is meant here is the idea of can we make sure that as we develop training programs or consider how we serve members of the community or colleagues how do we do that in such a way that we really serve those of our colleagues who are most at a disadvantage for not having gotten that training and it could affect us and impact us in so many ways for example you know people are busy training workshops happen how many workshops happen at a time of day or in a situation or context where our colleagues who are parents wouldn't be able to attend because they are taking care of family members how would that work have we considered that if we don't consider that are we perhaps disadvantages those who have children and perhaps even disadvantage putting at a more imbalance women who often times take more of that responsibility and that may be less able to participate or take advantage of the training opportunity that's just one example but there could be many many more of how the training needs to be thoughtfully deployed to make sure that we don't do it in a way that unfairly disadvantages one group or another another one and then I'll get to the one that I'll just focus on for a few moments is this idea of standards and the idea that can we communicate to our learners who might not really think about the quality of short format training other than that they hope it's good they hope it teaches them something and they hope it's about what they want to learn at that moment to get them that next paper how do we communicate to them that as we do things we have thought about inclusivity we have thought about quality standards without such labelling then really the learners only at the option to take whatever training is happens to be available and as services like ARDC and others start to compile training and compile opportunities how can we communicate not only necessarily the level of quality but it could be the relevance of a training experience or the fact that one training experience has been more recently updated it could be whatever you consider as an important factor that you want to communicate to the learner so the one that I I'll spend my last few minutes on is this idea of professionalization and what does that mean to actually take what all of you in this room do and make it even more professional because I think you're probably already professionals but what could it mean to actually think about training and short format training not just as we held a workshop not just as well Jason knows how to use this program and therefore we'll give him to teach something to the rest of us but something that really makes it a little bit a notch above how could we professionalize on the training well we give a couple of examples the first one is thinking about the network of people who do training how do we create avenues for those people to develop an identity with that with that role and to to realize and recognize other people who have that role that we have a common role and have a venue to disseminate our work to develop our practice and become better and also to reward those of us who make contributions to the space of training however we participate in that role and currently I think that many different groups or professional societies have some element of them or some section within them that is devoted to training instead of being a special interest group within a society or an organization the idea of short format training itself could be independent and cross cutting as a professional community these are elements that help professionalize and then if there were such a thing perhaps even short format training community of practice or a short format training professional society what are some of the roles that such a group would do because a lot of it would be not necessarily disciplinary specific it wouldn't matter if you're doing this with physics or biology or medicine or etc but together as we think about what does it mean to make to deliver impactful training that meets all the criteria of principles to our colleagues as we serve them in the role of instructor in training when together we could think about how do we that curate and maintain centralized resources how do we clarify identify address challenges that we all share and then as we do that how do we share those innovations so that we can all become more effective inclusive and career spanning there are a number of incentives that could make the effort that will take to reach that level of professionals and worthwhile one of them for instructors recognizing and celebrating the contributions of people in this room and for every instructor to say that the role that you have within the community of academics within the scientific community is invaluable because without the ability to be effectively trained for our career span and without the ability for everyone to be included in that process science is not going to advance as quickly as it could otherwise but as a community you can rely upon your peers to support and you can really have that community sense for people who are also including let's say the instructor or designers who might not be at the face of the instruction but who organize it or administration or other to do independently could be done more successfully together and how do we have new opportunities to develop better and more sustainable economic models because there are models that really depend upon free labor which is not a nice thing to do if we can avoid it but at the same time we need to think about the fact that it does cost time and in effort to deliver these things how could we do that better together and then for funders and other organizations who also have a role in supporting this training working with such a professional society or professional societies or groups could strengthen the justification for short format training and allow the funders to shape and raise the bar for quality for multiple groups but focus through the activities of a single organized professional group so the thought there really just painting that very very in the clouds vision but one that I think has some elements that really could be made real we need to take these ideas and we need to build communities around them we need to sense these problems as a group and then think of solutions that we can act on together community of practice is one of the ways that this happens there are methods but this is certainly one of them where by coming together and talking about these topics we can support each other in many different ways to make sure that we raise the level of quality and the impactfulness of our training I'll throw a selfish pitch for a way that I've tried to help contribute to this which is a group called Lifeside Trainers where we definitely try to realize some of this idea by bringing together people who do short format training across many different groups with the opportunity of making training better and we have a monthly call you can go and take a look at that it happens twice so that we include people in multiple time zones including in Australia so my final thought with that is that we definitely need to build community around these ideas so that we can sustain the solutions so I hope I've given you something to think about I know personally there's a lot there to grasp but at the end of the day I think that it's a critical tool for conversations that you're having and I was looking at the skill summit just amazed by the diversity of topics I see you're going to be addressing to really think about how do we do this across the career how do we do this as a group and what is the role of that shorter format of training experience that could be improved that could be made more impactful and that could really be delivered to everybody to accelerate science so with that said I hope this time for questions and I hope that it starts some ideas and conversations because what you do is so important and I hope that in some way the work of these groups that I've been able to present stimulates you to think a little bit further on that so thank you for the invitation and I'll take any questions that come from it Thanks Jason Absolutely fantastic presentation thank you so much for that and certainly you have raised some flags some very pertinent topics that no doubt we will all be discussing over the next couple of days Does anyone have a question? Well while you're thinking about a question or a couple of questions Jason you mentioned communities of practice and earlier in your talk you spoke about the effectiveness of short form training and I was just wondering do we measure the effectiveness of communities of practice and is that on people's radar because I know it is actually on the ARDCs radar and we have a community strategy that we've worked through and so I was just wondering from your perspective do you measure communities of practice and what impact they're actually having Okay I will admit that for whatever reason you were crystal clear a few minutes ago and then just now not so and it's not that you're not near the microphone but I do take it that your question was about community of practice and the potential for that to have an impact Is that sort of a gist of the question? Yeah so essentially how you measure the effectiveness of a community of practice Okay I thought maybe I got something there about how could you envision that happening if that's right otherwise someone will chat I see the chat opening so they will correct me and I'll enrich my answer as I start to talk about it so community of practice is an interesting how to measure the effectiveness of a community of practice oh great community of practice is an interesting thing because on the one hand it sounds so simple that groups of people get together and they get better at what they do but there are actually people and I can throw a shout out so to speak and in fact since I can share my screen let me just pull up one thing that I think is so important and so valuable because your question kind of revolves around the idea of how will we even know if we're succeeding we could put together a community of practice but if we don't know how to measure that and how to understand in a sort of rigorous way then we might do something that's positive and it might feel good but it may not actually get us somewhere can I throw a thought to the group and really celebrate of one of the persons that's actually involved with this group and this is the center for scientific information and community engagement who deal with among other things community of practice and I wanted to draw their attention and use this please in your conversations have a look at it and again I'm leaving after this talk but you are continuing on for for some days they have this interesting community participation model so communities evolve and actually there's a number of theory and papers on this idea where you can go through a cycle of people who have an idea like me today and you are at the stage of where I've conveyed something to you and you've taken it in great that's step one but at a certain point the community needs to change where now it's not about me broadcasting to you but it's about people now starting to actually come to make contributions and that's something that could be measured and then the next stage of that of that evolution is the idea of collaboration now people are working in small groups or in different units on those ideas and then ultimately it ends up with the idea of co-creation now those individual groups are creating something new and there are metrics and qualitative and even I would argue quantitative milestones along that progress so I think that community practice could be more than this sort of soft fuzzy thing that oh we are community of practice but could actually be made into both something that is qualitatively and quantitatively measurable and there are people who have expertise in that sort of even network science to measure by the outputs of the community and by the actual activities it's undertaken is this successfully evolving as a community of practice where we see that the outputs are what we intended and can we measure that or if it's going off course in certain ways can we intervene along that process to make sure that it's following the course and do so in a way that's convincing and measurable so I think that that is possible and I think that would be a wonderful thing for all of us myself included to think about how when we use that term how can we apply that standard so that it's not just a buzz word or a theme that gets labeled Thanks Jason Any questions? Hi Jason, thanks so much for your presentation today I have two questions and they're completely different questions and the second one you might not be able to answer my first question is is there any other way to engage a number of trainers across a campus apart from a community of practice because I'm kind of new to this area and when I was doing research I was finding there was a lot of the same training offered in a variety of different centres or faculties and it was very much a silo situation so that was my first question my second question is a lot of this short format training is offered free and I'm wondering if you have any advice on how to ensure people that sign up for the training actually attend because there is a problem with commitment and it kind of irks me to be honest because especially if you're paying for catering and they just don't yeah I don't know people don't have a conscious and then they just don't turn up and tell you so yeah yes okay let me start with the second one first I can give an answer and there's actually one of our recommendations which is on the list there is a recommendation G which is to clarify the economic models that enable SFT it's kind of sounds a little bit of a I don't know a politician speak but it does go into that idea that because something is free often times this perception that it's not valuable or that it's not worth your time or and not meant in the learner's time I mean like you said you've planned for it you've booked it you saw 50 registrations come in you paid for the really expensive coffee and then six people show up not to mention the fact that we have lots of things going on everything from pandemic burnout and how do you deal with that I mean my way in sometimes is actually you do charge and you charge a nominal amount just to have an investment that people come into it that they you know or they lose their $25 or whatever nominal fee that's an operational question that we could ask and people to come up with recommendations but the idea of the larger issue that we put around that idea that's what you've mentioned is one specific instance of the idea of learners didn't value it right and it lost it wasn't taken seriously so we do mention that and I would encourage you to read what's written there under G because our thing was really we need more work on that how do we do that in such a way that we honour the time and value of everybody more than what we do because on the other hand we need to balance what we did we want things to be accessible we don't want someone who couldn't afford even maybe the $25 to say no to the training we have some suggestions under item G and it's something that is a larger discussion I'll just leave that thought with saying it's connected to other discussions which had been well thought out which is everything from the idea that why is reviewing peer reviewing and academia free and what should you do about it so there is some thought about this we make recommendations and then it's up to people to say the people that want to do those recommendations as well we can work with you and the people who never want to implement them okay going back to your other question and see if I can restate it and remember it but it was essentially the idea of how do you deal with training happening in silos how do you deal with the idea of training being replicated in multiple different places and things aren't coordinated well to one extent that can also happen this goes back to academic structures diverse incentives where it's somebody's job to deliver the training and they just deliver it regardless of how well you know how much it's needed or not they delivered it but I think the deeper thing is if people aren't connected then they may not realize that what they're delivering is already been well put together by someone else and there's no need to repeat it or there's no need to reinvent the wheel there are let me try not to get deep into just jargon and speak or whatever because the paper is in my head that we're reading right now but one of the things that we found in education reform is that there are two ways that you get there's a thousand ways that you get people to change but there's two that come up most often one of them is community of practice is as a method for education reform that's one that comes up more often than others and the second one that comes up more often is the idea of diffusion of innovation going back to Rogers and some will be familiar or some not but if you've ever heard the phrase early adopter that idea that there's a cycle of when something new comes in some people adopt it people have to be persuaded to use it people have to have knowledge to use it and another one of those elements in the diffusion of innovation cycle is people need to know that what they want to use is customizable because they always want something that looks like what you have but just slightly different because I'm special and I've got something different and you may well be helping people to identify what's out there through for example I know Dresa is coming online the digital research these different portals that type of infrastructure could raise awareness and help other people know what has been offered or what's been done before a physicist instead of physicians and that might help to reduce some of the redundancy or the isolation of people's efforts thinking that there's only one who've come up with something or delivered something or customize it so if people can feel connected or know that there's that resource that documents those things and that's actually one of the principles documenting what you've done and sharing it making that easier hopefully that can reduce that and free up people to do something that is more worth their time when it comes to being creative rather than just not being aware and having created something or spent a lot of time that was already well done and could have perhaps been put together rather than independently I hope that answers some of your question if not my contact information is all over the thing, I'd love to have a chat Hi Jason, thank you for your presentation in the early that you presented there's a need for a lot of data skills and software development skills and this is an area where there's a lot of talk of microcredentials because of the need to develop to develop more skills at the faster rate although I haven't seen a successful implementation of how a microcredential system work sure so one of our recommendations C is supporting microcredentialing although in our case we're talking about providing microcredentialing for our instructors if you're talking about microcredentialing for learners in general yes that has happened here's my perception of it I've even had that discussion in Australia where part of it there you go back to the academic system it's almost like I guess I could say that without getting in trouble, there are some universities who don't think microcredentials that exist because why would you we're a famous university and we create well-rounded people who know everything so why microcredential so my perception having spoken to many people is that there are actually some universities which are hostile to the idea but there are other universities which are super excited about the idea of microcredentialing because it does exactly what I think of a microcredential do it recognizes that there are bits and pieces of really needed skills that aren't quite represented in a formal course of study in which we need to deliver to somebody because they're interested but also that person needs to be able to demonstrate for example to an employer that they are competent and qualified in a particular area so that the microcredential means something so I've definitely seen microcredentials the way that the system works here in the US and I don't know exactly the equivalent in Australia but here we would say that community colleges which are usually institutions to vote it to two-year degrees in which often times are very well connected with the needs of employers at the level of industry so that they work with an employer or with a group or with a specific field to say hey we want our students to be able to work at institution or job X what are the five things that you look for in your new employees and teach exactly to that and that is a system that I've seen work it takes time and I think it's becoming more popular but because it's so tightly coupled to a job that the learner has confidence will likely be there for them if they acquire that microcredential that the employer or employers have had input into so that they know that that microcredential is teaching exactly to the skills that they'd like to see those I think are fairly effective and I think there's growing evidence for it it's still maybe a relatively new concept but I think it's entirely possible to be successful but I will say that there are institutions that don't like that idea and so there may be structures or groups of people who resisted or downplayed but I think it's a valid way and if you think about again the whole idea short format training sorry to make this long is that most of us are not going to go for sabbatical in a new area we don't have the time but a really well designed microcredential when we need a little bit more than just a short course or webinar but we really actually want to be quite skilled in that area that might be a perfect solution to do this and as a researcher that might be something that we should be socializing and popularizing as a researcher maybe I need a microcredential and Bayesian statistics because I really didn't get that maybe I need to get this for this new type of analysis or for machine learning for biologists to do X, Y or Z that would be perfect I want to know exactly what I want to know as a biologist in this area and I want to be protected from all of the extraneous information and learning that I don't need because I'm not smart enough to know what I don't know and what I need going on I want somebody package that for me to microcredential so that's my particular view Thanks Jason we have now reached time thank you so much and look if you do have questions for Jason we're quite happy to collate them and send them through and we can sort of distribute those to everyone who's registered otherwise as Jason said reach out to him directly there's plenty of ways of doing that so thank you Thank you so much I know you're going to have a great conference and so I'm going to build a conference Without further ado we've got five presenters this morning four here in the room and one joining us via Zoom so I'm going to get them to introduce themselves and we're going to start with Pablo Franco Thank you very much and thanks for having me today Pablo I'm a postdoc at the Centre for Brain and Mind at Markins at the University of Melbourne as I briefly described before I have some experience teaching digital skills in particular at the University we've been basically a group of researchers teaching researchers to code and I'm now moving on and doing that at a faculty level but today I'm going to talk to you about assessing the effectiveness of training from a practical perspective and teaching in the context of teaching digital skills to researchers so it's going to be very brief the outline is and three different parts the first one is the theoretical framework and how my own experience maps into that framework and then I'm going to finalise with some ideas on how we can actually move forward and maybe things that you've already implemented and things that we'll be happy to discuss later on in the summit so to start with the theoretical framework there's a framework you've probably heard of it's called the Kirkpatrick model of training evaluation and it has four components basically says that you can assess effectiveness based on four pillars reaction which is the learners react to the training the relevance and usefulness of the training so that's their own subjective experience also then we have the learning so the participants the students reach their learning objectives so that can be objectively matched with tests for instance the third pillar is behaviour so the students or learners change their behaviour in their day to day work so for researchers they perform they're performing research differently have they improved their research the quality or efficiency of their research and in terms of results this is the last pillar and one which I think we need most work is measure objectively the impact that has at an institutional level so can we quantify the results to society and in terms as well as research output and this is like the standard carepatrick model of training evaluation that's quite widely used and I propose that for our for our okay sorry I don't know if you were seeing all four pillars a second ago I propose there's one more pillar that we should include for purposes which is outreach and that is the extent to which the training is being deployed so who are our researchers who are we reaching which disciplines are we reaching are we reaching people from all stage careers so in terms of outreach it kind of talks to the inclusivity and diversity of the people that we are teaching so with that modification of the carepatrick model training I can talk about my own experience at the university of Melbourne in my team where our team at research platforms at research bazar which we ran a few years ago but what I've done and my own experience talks to with four of those pillars that I just described one being the outreach pillar so in terms of how we measure that we looked at the number of researchers the diversity of the researchers the disciplines were they coming from who were we teaching we were looking at the career stages of the people we were teaching so we asked participants to fill in surveys about all this kind of things and then that helped us understand who we were not reaching so which disciplines needed more effort which career stages needed we needed to focus on reaching as well and this came out of also from the pandemic as we moved into e-learning we looked into assessing through number of views of either videos or number of reads of books or blog posts that we faced that we posted in terms of the other pillars in terms of the reaction we surveyed participants in terms of qualitative and quantitative feedback they could provide about how their experience was that helped us improve our own workshops and trainings and in terms of the learning pillar then we did during training exercises and quizzes that we actually were able to see the results we used cahood, bull everywhere there are probably many ways that you can do this and one which I think I'm most proud of is behaviour so for that we actually called people that had trained with us after a few months or even a year and asked them how did your how did this training I'll keep on anyway how did this training help you change your behaviour in terms of research so we had interviews in which they very detailed passion told us about how the training had actually changed their way of doing research and this is my experience but I think there's plenty of things that can and could should be improved and especially for assessing impact objectively on the last three pillars so in terms of learning this is something that was talked to today before as well we would need some objective measures of pre and post training test how much did we actually improve in terms of the learning objectives based on how they came into the classroom in terms of behaviour a suggestion here would be maybe do surveys and interviews not only with learners but also with stakeholders supervisors did supervisors notice that something changed in the behaviour of researchers colleagues people think that their colleagues are becoming more efficient as they are as they attend our trainings and also IT teams and here IT teams I'm being a very generic term to refer to people that might have access to information about the change in use of high computing services on software licences and so forth and finally in the pillar that we haven't worked on is in the pillar of results impact of outcomes and we can consider for instance improvement in the efficiency of output generation so quantity of research output a number of papers or the quality of that research output by looking at a number of citations or the altmetric metrics of papers and one idea that I had that might be interesting to look at was to encourage explicit acknowledgement of training on research output just like funding is in papers why wouldn't training be acknowledged that will help us quantify and argue for the importance of our services and with that these are the references of the curepatrick model and thank you and to receive any questions hello so we've got time for a couple of questions if anyone would like to ask Pablo yep hi thank you for that that was really cool I had two questions so the one was around essentially the scalability of using a model like this so do you do this for say every sort of training ground that you have or is this something that you do periodically like once a year once every two years or something like that in particular I mean some of the stuff I can see being applied very very easily like the engagement stuff at the beginning and stuff but in particular where you sort of interview the students and their supervisors and everything that would be a very labour intensive thing that I can't imagine doing all the time so that was the one question the second one is I'm maybe getting into the weed tier but too much but in terms of the sort of sort of the linking the I love the idea of linking the training but in the same way that you would link the funding and sort of assessing it around like things like citations and everything but I suppose the question is how do you draw how do you draw that sort of explicit link between this person that attended the training and that's sort of led to the higher citations because they might come to one training course be sort of a you know a high achiever originally but they learned something at the training course but they were going to get those citations anyway do you know what I mean? and I'm not trying to dispel the training or anything I'm just sort of curious as to how do you measure that I think that both are really good questions the first one I do think that it depends on the amount of resources that you have at hand so how much can you actually report on how much time you have to actually figure out and measure things what we did actually was like on a yearly basis to do the big interviews but we did have and the big interviews and the outreach kind of analysis but we did have surveys after each training on things that we could change for that training and we asked every time people attended their training about their demographics in terms of career stage, discipline and so forth but I see you like for a supervisor and all these surveys it might be quite time consuming so I wouldn't do it on more than a yearly basis of course if you have the resources yes, on the second question I think it's more tricky I don't have an answer to that I think it's the trickiest question because it's always going from the subjective value of our training based on what people think and if they put it in the research output great but they'd actually help them where it's just like that they think it helped so maybe some sort of research that is well designed that allows us to control for all of these other factors and look at two different groups would be a way of actually justifying to funding agencies for our services I think we'll just move on to the next speaker and then we might have some questions time at the end so thank you Pablo and up next we have Aiden Wilson from Intersect Thanks, thanks Doris I'm just going to switch across my presentation while that's happening I'll start introducing myself so my name is Aiden Wilson I'm the Intersects Digital Research Services Manager wait I might have to got it no hang on got it okay so I am Intersects Digital Research Services Manager and administration and sorry administration delivery of training is in my remit for that job before that I was digital research analyst for seven years working at a number of members but mostly Australian Catholic University and from when I started I got involved in the training systems administration from very early on so I'm speaking today from personal experience so at Intersect we run probably the largest research technology focused short course program in the country we deliver hundreds of courses each year to our member universities and other partners and they are attended by thousands of researchers training courses are primarily delivered oh I shouldn't mention the topic the topic I'm talking about sorry is running training at scale and succession planning hence talking about the size of the program so training courses are primarily delivered by our team of digital research analysts augmented by a large team of casual experts right around the country and I'll talk about them more in a moment our members continually tell us that they value our training program very highly along with the digital research services we provide via our digital research analysts so on that basis as well as some of the other evaluation metrics that are up on screen now that we collect both after every training course and also sometimes a year later in blocks going to Publars talk as well I can confidently say that we do a very good job of delivering training as you can see from this graph our training has scaled up dramatically since inception of the program in really 2013 10 attendees in 2012 and they were all intersect people by the way in those 10 people and so this was only possible through continual improvement of our training administration systems and when I came on board when we started scaling up and looking at more systems automation is a hobby of mine so when I came on board and we were doing things very manually with the DRAs booking in courses looking to fulfil them with trainers the intersect office admin creating event-bright events booking routes on campus and making travel arrangements of trainers that kind of manual operation just cannot scale and I saw that and said we need more systems for this so as we started growing from 2016 onwards we began organising courses using a system of sounds a bit janky but using a system of connected Google spreadsheets and applications that either semi or fully automate most of the effort in administering training courses we call this the combined training combined training forward schedule it's not a great not a catchy term or CTFS the CTFS provides our DRAs with an annual view of all their courses generates calendar entries event-bright pages and automatically pushes courses to our website for external visibility without systems like this we would be totally unable in the time that we have the people that helped me on this are also DRAs so they're doing this in their spare time from their university so we would be unable to run this many courses without the systems that we have in place but the computational systems are just one piece of the overall platform the vastly more important part is our people this includes the entire team of digital research analysts based onsite at our member universities who take on ownership and primary responsibility for planning and delivering the program of courses at their university as well as a subcommittee of DRAs that we call the training admin team led by me who oversee the planning and also manage the allocation of the the most important piece which is our team of casual research trainers we would be totally unable to deliver our courses without the assistance and often the leadership of our digital research trainers the trainers assist DRAs in running courses but often lead the courses themselves particularly when the DRA doesn't have the expertise in that particular tool which with a large catalogue as ours is actually very common no one person can be across the extent of the catalogue that we offer this team has grown over the years and this snapshot of 15 people is only those trainers currently working with us they are all current PhD students and we find that the work that we offer really suits PhD students very well training for us demands flexibility over the course of the year that PhD students are able to live with because if they've got training on a Tuesday they can do their research on the Wednesday and vice versa so this flexibility fits well with the requirements of completing a research degree more importantly the salary we pay for delivering training is commensurate with the casual tutoring at the university and the work itself encourages the trainers to build their computational skills that they can then apply to their own research in short our trainers are paid to develop their own skills in research relevant technologies as one trainer put it in a recent catch up the salary is really good and is on topics that are of interest some other PhD students are working as for example Uber drivers or doing casual work that has nothing to do with their degree their knowledge or interests and it allows us throughout their degrees which is also critically important for the succession it means that we don't have a lot of the churn that others may face in having to train up new starters so we have people that secretly for two three years sometimes and so they build a body of knowledge and they become the senior trainers the junior trainers over time and it's a model that works very well we go through rounds of recruitment particularly in tools like humanities tools like EnVivo to lighten the load on Paul Catherine who's doing a lot of EnVivo training for us for other universities so if you know anyone who wants to work for us delivering EnVivo training getting contact also many of our trainers eventually come to work for us full time as DRAs as data scientists and HPC support specialists after the completion of their degree and a model of career path in this as well so to summarise our experience of delivering training at scale and I have to say doing a good job of it relies on two important factors having mature systems in place that handle the day-to-day minutia of planning courses and having access to a workforce of confident competent and supported trainers who are remunerated fairly for their efforts providing continuity of service thanks thanks Aiden, any quick questions for Aiden before we go on to our next speaker no? thanks Aiden you talked about well I'll just ask the question I guess straight out e-research analysts as those do change over a period of time and they're working specifically in institutions so they're building a lot of institutional knowledge and that sort of thing how do you capture that knowledge is there a handover process or is there some way that analysts are documenting what they're doing to ensure that that knowledge stays within intersect yes, good question there is we tried as much handover as possible so the DRAs will report to a manager I'm the manager of some of them at the moment Jonathan here is the manager of the rest of the team so we've kept pretty closely clued into what they're doing on campus but also on campus they report to a someone largely in the research office maybe IT or both so we encourage them to be very transparent with their managers on campus and the handover is sort of a combination of intersect and the local university so I guess there's a number of ways of doing that there are occasionally instances where somebody leaves without doing a full handover that's unfortunate but it happens and sometimes we have to sort of start that process again building up that local expertise probably something we should work on to I guess another element of working on that is we have similar systems that I've been a part of building to track the stuff that DRAs are doing on campus not relating to training necessarily but research support projects and so on so we're tracking more of that and we report on those things annually we have been reporting on that annually for the better part of the last decade so in that reporting there is naturally a lot of handover so when somebody starts, one of the first things they do is read the last 10 years of annual reports to that university to get a feel for what has taken place I hope that answers and yes we've got an internal sort of wiki for documenting some of those things to build a sort of knowledge base over time as well Thanks Catherine Thanks Hayden, that was great So our next speaker will be Paula Martinez from the ARDC and she's joining us online so the next one will be a Zoom presentation Hi everyone, can you hear me? I'm here on echo Thanks for inviting me to give this presentation Paula Martinez of a project coordinator at the ARDC I'm here with my community managers had to talk to you about building community I'd like to acknowledge the traditional owners of the land in which I work and live the Brisbane River the Jagara and Tobol people If you have interacted with me in the past you might know that I funded and supported multiple communities throughout the years within my role at the ARDC in the software program we have identified a gap and envisioned a new community to discuss research software visibility I started with a landscape analysis of related communities and grouped them in a dual access graph so horizontally how much do these communities discuss research software and vertically how much influence do they have to promote change so we found our niche and defined our vision the visible research software interest group was formed to affect change by boosting the visibility of research software to improve how it is cited, published and fair once the landscape analysis is completed we need a participation model and this is defined by the type of community here are some examples and they all tackle challenges by a group of individuals the next step is building a repertoire of tools and methods that will enable our vision such as some of these ideas for example, how does community production and co-development is facilitated how does knowledge is shared disseminated and obligated this is all stimulated by deliberate coordination generating space for participative, interactive, spirited and fun interactions which motivate eager participation and celebrate small victories creating reporting the community in our case we also intended to be influencing and resending so we initiated our work by prioritising being inclusive, non-linear and resourceful enabling different models of participation where each individual is valued and recognised for example creating champion roles, ambassadors or advocates for all of this it's important to have a clear and accessible code of conduct and one thing that I like to spend on is the ability to look back to move forward here are some examples of the visible research software interest group success stories we managed to attract the right audience by forming an influencing cohort by clearly stating who should join the group then initially we created an open online discussion forum as the main platform to discuss actions in a public and transparent way however we created the infrastructure before getting the participants so this work is yet not in the active way that we intended so we changed we proactively invited people to communicate by an alternative way the main list we know that change takes time and we are still in the process of enabling participative interactive spirited and fun interactions to finish celebrating small victories in the almost one year since inception we have increased awareness of national and international guidance to improve the visibility of research software by sharing existing resources our initial report has been viewed more than 600 times and downloaded half of those and we also continue to work on five short term actions one of them being the report on how research is time software and in the last three months it's been downloaded 150 times we're also getting important collaborations to leverage in the future that's all for me and I'd like to point out a few resources that you should be aware when thinking about building communities one is the ARDC communities and groups the second one is FIVA B is the center of scientific collaboration community and engagement CSCE and the Carpentries communities if you haven't heard about them I'm sure you will learn more about them at the end of this summit thanks for your attention feel free to contact me via the general ARDC contact email and subscribe to our newsletter thanks Paula any quick questions for Paula before we move on okay Paula thank you Mark Crowe a couple of slides coming so hopefully they'll be arriving in a minute while they come up I'd just like to introduce myself I'm Mark Crowe, I'm the Skills Development Manager for QCIF based up in Queensland QCIF provides research computing support services to all the Queensland universities 20,000 to 30,000 researchers we cover and the skills development side of things we have trainers we have a team of research analysts very much parallels what Aiden was explaining earlier on about intersect we have a very similar model with QCIF and yes I agree they're the biggest providers of digital training in Australia we're probably second place but I'm not going to talk about workshop program today what I'm going to do is talk about another area we focus on in skills development which is RESBATS and in particular bringing oh, that's still I'm down in the program as bringing training to research communities and I'll just change my title on the way down to bringing training communities to research because really that's what RESBATS is all about it's about creating a community and the idea that community is a way people can learn and grow their skills together not only during the event but really importantly afterwards so taking those connections away to continue their learning and training opportunities oops that's what I mean by community so this is a few shots of our RESBATS events so Pablo has already mentioned RESBATS but didn't really explain a lot of detail what it was all about so those of you who aren't familiar with it it's short for research bizarre so research bizarre describes itself as a festival promoting digital literacy so made up of a a mixture of workshops, talks, networking events and social activities and it's not a conference it is a festival and that choice of the word festival really emphasises that it's bigger and there's more going on than just a conference specifically a festival is a community event you go to any festival it's a community activity and we feel it's that community mindset that really makes makes me think that RESBATS is so effective because it encourages everyone who's there to get actively involved but a conference people may just sit there and listen RESBATS is all about participation so I mean Queensland last year in November we had about 300 enthusiastic young researchers and me and a few other less young researchers and everyone there was working together they were learning together they were networking and really importantly they were making friends and so for me it's those social interactions which are the really important things from RESBATS and much more so than the specific training workshops they're going to what we want from RESBATS is to create the connections that these people are going to go away and these connections will grow into long-term relationships where people are training each other they're supporting each other and eventually they're collaborating together so just as an example of that situation that happened at the RESBATS in November I was chatting to one of the attendees there at 90 seconds of research impact event so this is a social event we have at RESBATS slight three minute thesis but for people with a short attention span and he was telling me about a workshop he'd been to earlier on in that day and what he said was that he was so enthusiastic about the workshop the topic was there straight after the event he went up and sought out the instructor had a chat with the instructor two weeks later he was in the lab of this instructor up in University Queensland discussing collaboration and project opportunities so really from that one single interaction created by RESBATS those people they could now be working together for months or years he'll be learning more from that instructor the instructor will be learning from him and probably people in their groups are going to be learning from each other as well so really that's how RESBATS is helping to build a training community and that's just one example I'm sure and I hope that there are many other ones happening throughout the whole three days that parallel that that people are going away with at the end so really that's why it's important to me as a trainer RESBATS is important it builds that training community and building a community is the only way Aidan's talked about scalability but to really scale the community is the only way we can do it we've got 30,000 researchers in Queensland you've got 50,000 researchers in New South Wales and so on there's big numbers up on Aidan's graph that doesn't become close to the numbers up there many of you have run training workshops and probably when you did it they were massively oversubscribed I think it was back in the research Australia in October someone said there was something like one support professional for every 500 researchers, maybe more than that we can't do all the training they want we can't help them with all the analysis they want but what we can do is we can bring them together bring those researchers together we can empower them to support each other and to learn from each other and to go away and solve their data analysis questions together and that's what RESBATS does it allows us to do this, it brings those connections and those relationships and so that's why QCIF is so enthusiastic about RESBATS and why I see it being something that we're going to carry on being involved in for the foreseeable future thank you very much thank you Mark, I'm a big fan of RESBATS as you know fantastic any questions for Mark? having participated in RESBATS Sydney I guess 100% and some, thank you so much oh sorry, there's one question sorry so we've tried to grow it out for the first three years RESBATS was run it was RESBATS Brisbane and then a couple of sessions ones ago we called it RESBATS Queensland we tried to have a satellite event we streamed a lot more talks but the reality is in Queensland particularly we've got JCU, we've got Central Queensland University there's people scattered all over the state and we haven't yet got an effective way of reaching out to those we are trying to do it one of the challenges you had is because they are so far away and they haven't got a culture of RESBATS even when we do online activities we're not aware of it so I think it's just a long term one of offering things online, offering satellite events and gradually building up awareness over time thanks great, thank you Liz is our final speaker for this morning so thank you Liz, another RESBATS champion I could throw out my talk and start talking to about RESBATS again but I'm not going to do that because I've got to talk Sonya is going to tell me when I'm going to find so hello, I work in the skills and workforce development team at the AI at DC and I'm here to talk to you about our Carpentries partnership so we've heard about this organisation a few times today the Carpentries are a global volunteer organisation initiative who teach programming and data science skills to research communities they provide openly licensed curriculum to do that and a lot of their materials are about teaching researchers to work in open and reproducible ways one of the main reason that we're running a partnership is to make it easier for research organisations in Australia to access the instructor training instructor training program which is a train the trainer program that the Carpentries run it gives skills trainers enough pedagogical theory to handle a classroom hi, Phillipa and also walk the talk when it comes to using inclusive practices to build a positive environment for learning so that researchers can you've got your handles on your handles on the stuff and you can go away and you know what you're confident enough to approach these software and these tools on your own and you know who you can come back to the next day when it borks and falls over and you need to you want to check in again so we ask how can we make our how can we help our motivated champions the people who want to do this training more effective this partnership is one way of making it easier for organisations to access that trainer training my definition of easier is making something cheaper and making something simpler so our partnership which already has our key training organisations in Australia such as CUSIF and Mark's here from CUSIF and Intersect Aden see more intersect people across the room yeah okay have been part of this partnership and so we've got a handful of other key universities across Australia who have joined to run this train train the trainer program so last year we trained about 100 people across like researchers, PhDs and professional support staff this year we're going to train about half of that because we're going to focus our attention more on enabling these newly trained instructors to get them through their certification and also set them up so they can start teaching their first workshops last year there were about 28 carpentries like badged carpentries workshops in Australia and the bulk of that really came from CUSIF and this is in addition to all of the workshops that intersect run which are very much carpentries adjacent in some respects so what I would really like to see in terms of success is seeing more evidence of carpentries workshops being run by across our communities beyond Queensland and beyond where intersect run their workshops as well for me that means investing in the social infrastructure and if you've heard me bang on about the carpentries before what I usually say is they are great at that social infrastructure they make it easy for people to go to connect to others and for you to actually find the people who are going to help you jump on their slack group oh my goodness you might jump join in because someone told you to just ask them see if you can find an answer and within 24 hours you will have a hand pool of people saying oh did you know XYZ and here I'm here to help and I'm the regional coordinator so it's a really welcoming community and I'm really behind them this year we are also going to kick off a series of bi-monthly Australian New Zealand community calls so the carpentries set this infrastructure up so that people can come together someone has already set up a meeting room and organised a host and some speakers you can join in and meet other people and find out what's going on in other corners of the carpentries verse because I guess it is a bit of a multiverse in that way I want to end with one final point about what I reckon success means in terms of this carpentries partnership and that is by bringing new organisations in research organisations into our partnership we increase the number of active instructors in Australia and the people who are being trained to do to do this to run these instructor workshops sorry these carpentries workshops are actually paid to do it so that we so it's part of their whether they are employed on a casual basis through training organisations or our institutions are able to supplement that with other people being paid to do this work it's great people join the carpentries because they're enthusiastic about what they've learnt and they really want to share it with others and the instructor training enables them to be able to do that safely and effectively and in at a greater scales than one on one personal connection so obviously competition for this partnership is going to be high but if you're enthused about this partnership and would like your institution organisation to join please come and talk to me there's a little poster up there on the back that has a bit more information on the carpentries and that's it from me yes