 Welcome to the session, the Test Automation of Scaling with On-Demand Agile Working, Workplace Learning by Rudolf Kreuz. Thank you, Rudolf, for joining and we are glad you can find a time to join us today. Okay, so, hello, ladies and gentlemen, I will share my screen. Give me one second. Welcome to the Selenium conference and my talk about the learning evolution at Riveson Bank International. Well, Darwinism. You know the theory from Charles Darwin that not the strongest survive, rather the who can adapt faster. What does this mean in the context of learning? We must learn new things faster and when we need it. Meaning just in time. In other words, the digital Darwinism and the WUKA world requires lifelong learning and adaptive learning strategies. In the following 40 minutes, I will explain how we at Riveson Bank International, short RBI, transformed our IT delivery division towards self-based lifelong learning. So, who am I? My name is Rudolf. As an HR engineering coach, I am helping our development teams in using appropriate HR engineering practices like continuous integration, continuous deployment, DevOps, test automation and so on. As a learning guide, I am supporting our colleagues to find the best learning approach and I am implemented during the last years in RBI, the HR workplace academy and the HR learning approach. And I am the Guild Lead for our Test and Test Automation Guild. Well, and I am the organizer of the HR Test Automation Meetup Group, with more than 1,400 members and organizing six times per year the so-called Test Busters Night, which is a mini-conference in Test and Test Automation. Well, yes, and you can contact me via LinkedIn. Let me tell you more about Riveson Bank International, short RBI, that you know the context of my talk. Riveson Bank International is a traditional bank with over 100 years of history. RBI considers Austria and Central Eastern Europe as home markets and we currently have network banks in 13 CE countries. In Austria, we are the leading corporate and investment bank for the top 1,000 commercial customers in the country and we have around 47,000 employees and a customer base of more than 16 million customers. As I said, RBI, a traditional, more or less conservative bank, is going through an adaptive transformation and we use a very nice metaphor in our organization for that. We are moving away from this big, slow, ruled by command and control containership towards fast, agile, self-organized speedboats. For us, adaptive is not the purpose itself, but a tool to realize a better business impact. And adaptive is our basis for remaining competitive in a fast-changing environment, better known as the WUKA world. Five years ago, we started our transformation and now we have around 500 speedboats with around 4,000 employees working in agile teams and all of them must learn new things at high speed to steer their speedboats through the rough banking business sea. And therefore, we needed a new learning approach. But why do we need a new learning approach? As I already said, we are living in a WUKA world, which represents our new business reality. WUKA stands for volatility, uncertainty, complexity, and ambiguity. It is a world in which you never know what to expect. Business in the WUKA world means that companies compete in an environment that is characterized by continuous, fast, and unpredictable development. In such a world, learning is of central importance for effective operations. And one of the most important ways that knowledge workers can interact with the WUKA world is through continuous learning and allowing employees to learn in action during their daily work, learning on the job is one of the most important steps toward readiness in a WUKA world. In RBI, we have different possibilities to learn new skills and competencies. From the traditional perspective, we have our individual trainings. That means if you find a training you consider relevant for yourself, you can go to your manager and hopefully you get permission. Then we have Charlie. Charlie is our learning management system where we have learning offerings maintained and organized centrally by our HR and learning department. This is more business-related soft skills, standard ID skills, and all this stuff. And then we have our talent program. This program is a structured program designed to find talented employees who have high potential for growth. And then last but not least, we have the GoID Academy. And this is the initiative maintained by our ID division, which I will tell you more about on the next slides. As the name already suggests, the GoID Academy is a group-wide IT learning and development initiative. We have 19 units involved and more than 560 participants in 2019. In its traditional approach, each program of the GoID Academy is fixed content, mostly delivered by external trainers with a fixed time schedule, and pre-selected participants. You can see the programs offered by the GoID and all of them covering strategically important competencies. This is our special program for all IT-related agile things, DevOps, agile, and so on. To support one of the RBI strategies, which is Upskilling People in Test Automation, we created a dedicated test automation program, basic and advanced, for the 2020 GoID Academy. The GoID program was very successful and we had huge demand. In early 2020, we launched four new programs on DevOps, Cloud, Agile Engineering, and Test Automation. The trainers were fixed, the participants were nominated and confirmed, the flights and hotels for the first module were booked. And then the pandemic broke out and turned the entire GoID Academy upside down. From one day to the other, everything changed. We had to decide whether to cancel all the programs or to offer them remotely. The decision to grow remote was more than just a challenge, it was an opportunity. With the elimination of additional expenses and opportunity costs as well as the absenteeism, the demand for the training tripled and we had to scale. So what got clear to us was that such an approach of fixed content and training schedule does not allow for flexibility and scaling and that we needed amplified learning formats to adhere to the academy. Or in other words, we recognize that we must create a new learning approach for the WUKA world for our small speedboats. The focus of learning in the WUKA world can no longer be the HR centrally controlled education program with its formalized modification offerings be in face-to-face or virtual. Learning is more. The 70-20-10 framework provides a good orientation. And this framework or this model says we have learned approximately 70% of what we know and can do through our work, in particular through new tasks, challenges, crisis and changes. And then approximately 20% will learn through meeting and interactions with others. That means with colleagues and customers. And then we learn around 10% through formal learning opportunities. That means seminars, qualification programs or structured online courses. And you can see which of these parts are generating the best return of investment. And we answered WUKA and the 70-20-10 model with two new approaches, with the HR Workplace Academy and the HR Learning Guide. The HR Workplace Academy brings together learning and work. The goal is to overcome practical day-to-day challenges. So that means we are not learning to acquire certifications or knowledge which you maybe will need in 12 months. The Workplace Academy enables our colleagues to solve concrete problems that they are facing during their individual daily work. And the Learning Guides help the learners to find their individual learning journey if necessary. By the way, this is Matthias. Remember this face. You will come across it many times in this talk. HR Learning is characterized by two aspects, the performance problem and new learning formats. HR Learning in the WUKA world is considered as performance support and primarily serves the purpose to solve problems and challenges during the daily work rather than do accumulating knowledge. We speak here of learning and applying and not learning and storing. After all, the actual purpose of HR Learning is to enable the employees to continuous improvement or, in other words, to do their jobs. What could be a typically performance problem in an HR development team? Imagine a team comes up in the retrospective with, we need automated tests, but we do not have the skills for it. This is a typically performance problem because the team cannot do their daily job. And the team agreed that Diana, the manual tester and Max, the Java developer should start in automating the tests. The plan is to start with web and Java and then proceed with API testing. But Diana and Max have no idea how to do it. In a traditional approach, Max and Diana would go to the learning management system, search for the right training, hopefully there is one, check if the dates fit his schedule, register for it and then wait for confirmation or to be added on the waiting list. In our agile approach, with our agile workplace academy, we give Max and Diana the opportunity to create their own learning journey, their own learning experience to learn at their own pace and along their own paths. If they want, we even provide learning guides for them to support them on their learning journeys. Learning which should solve a performance problem has to be close to the workplace. And therefore, we need learning formats that enable individual self-service learning with the possibilities to exchange regardless of time and place. And this can happen in a guided or an unguided way. The classic face-to-face classroom training alone is not long enough and here we need a blended learning approach. The API Workplace Academy is based on the idea of blended learning. And when we talk about blended learning, we mean the combination of learning formats that enables students to learn at their own pace along their own paths. And from a singular learning event, we go into the direction of a learning process that means to continuous learning. And work and learning moving closer together in terms of content and time. In combination of the different learning formats allows for more flexibility and individualized learning. And to strengthen the effect of the Workplace Academy, we paired it with the support from our HR learning guides. As I said, the goal of the Workplace Academy is not to acquire certifications or to stockpile knowledge for the future. The aim of the Workplace Academy is to master practical challenges from everyday working life and to combine learning and work. Does anything in this picture look familiar to you? Yes, here you can see Matthias again. To implement this blended learning approach, this Workplace learning, we created a new learning architecture. And this architecture is not only for test and test automation, you could use it for every other topic for DevOps, cloud, whatever. So regarding test and test automation, we have first we have learning topics. There we have test engineering, then test automation foundation, then test automation adapters. These are the things which we want to automate. This could be a web UI, API, a mobile app, a desktop app, whatever. And then we have the test automation development learning topic. This is bare adapter, bare programming language. That means, for instance, for web UI with Java C sharp JavaScript. And a learning topic has subtopics. As an example, test automation foundation has the subtopics test methods, test automation frameworks, and CI, CD DevOps. And then, of course, we have skill or competence levels. That means, you know, basic advanced expert. It is unnecessary that I go here into the details. And then next we have our learning elements. These are the trainings. And a learning element could be a e-learning course, a Udemy, a plural site linked in. It could be an RBI training. It could be a recording from an already delivered Go ID Academy training. It could be a blog post. It could be a book, an article on a website like medium.com, a YouTube video, or a one-to-one peer session with a learning expert or with a colleague. And then we have the key element. And the key element is our learning journey. And a learning journey has a learning outcome, a competency which you want to achieve at the end of your journey. This is a learning outcome for our test automation journeys. The learner can develop automated basic tests with the given programming language for a given adapter. A learning journey has a beginning and an end and supports the learner to solve their performance problem. In practice, however, such an approach does not always do justice to the complexity of blended learning. Because depending on the role and or the competence of the learner, there can be different starting points and different learning elements. But the impact, the outcome is the same. It can solve their performance problem. And to handle this diversity, we have the last brick in our learning architecture and this is a learning pass. The learning outcome for Diana would be Diana can develop automated basic tests with Java for web UI. And of course for Max, it would be the same. Tricky is that Diana and Max would have different learning paths. As I said, a learning path is an individualized learning journey. And let's have a look on a learning path with examples. Remember Diana, Diana a test engineer with no development skills and no test automation skills should automate web UI tests with Java. What would be Diana's learning paths? Diana would start with test automation foundation, followed by test automation adapters, followed by test automation development for web UI and Java. Then I do not need the test engineering topic since she is a tester and she has all this knowledge. Now the question how a learning path for Max could look like. Max is a Java developer with no testing skills and no test automation skills and should automate web UI tests with Java. This is the same learning outcome as Diana but the way differs because Max has another skill set. So what is the difference? Max first would need more or less the same as Diana. Test automation foundation, test automation adapters, test automation development. Only that the path starts with test engineering because Max has no testing skills. And from the web UI test automation, we do not need Java programming, the Java programming learning element because Max knows everything about Java. As you can see, the same goal, the same journey but another path. And now a tricky question. Six months later, Max should also automate API tests with Java. What would be the learning journey or what would be the learning path? Yes, Max needs only test automation adapters for API and test automation development for API. That's all. Not more, not less. I guess you know now what is the learning journey and what is the learning path and how this is connected to the learning initiatives. So what is the time? Okay, we are perfect. And now I show you an example with a learning sprint where the learner is supported by a learning guide. As I said, the learner could also execute the learning sprint without a learning guide but to bring the concept of the learning guides closer to you, we will do this example with a learning guide. But first, what is a learning guide? A learning guide supports the learners individually in their learning process. In addition, the guide supports the organization with the implementation of this new learning approach. And he is an ambassador for a deeper understanding of agile learning. As an example, currently together with two other colleagues, I support eight of our network units to implement this agile learning approach. The learning guide helps the learner in the planning phase of a sprint to choose realistic goals. The learning guide provides assistance in setting a clear precise and doing the realistic formulation of these goals. And the learning guide ensures the provision of the learning content, of the learning elements that fits the defined learning objectives. But also the individual learning behavior and needs of the learner he will consider. If necessary, that means that the learning guide may be organizing special learning formats such as job shadowing, coping groups and so on. The learning guide will be supported by the learning experts who are the specialists for a given topic if necessary. But to make it clear, the learning guide does not act as a staff manager or a line manager who said what the learner has to do. The learner drives his learning sprint in a self-organized, self-based way. And does anything in this picture look familiar to you? Yes, we have Matthias again. Then in the first step, the learning guide discusses together with the learner the performance challenge, the learning budget in hours per week and other details that provide the boundary for the learning sprint. And the outcome of this task is the learning epic. That is the strategic goal of the learner that we document in the form of a learning epic. For the learner, the learning epic could be following. As you can see, we define what is the current status, what is the current skill set. We define the expected outcome, the impact, and we define how the impact should be visible. And we are using this format to make the impact measurable to make it clear what is expected. Or in other words, the written word is more powerful than the spoken one. In the second step, the learning guide selects together with the learner the learning elements based on the learning journey from our learning portfolio and defines the learning path, which then is stored in the learning backlog. Our learning portfolio is currently an Excel sheet with more than 500 learning elements grouped by learning topics, subtopics, and by learning packages. This portfolio will be maintained and reviewed by our learning guides and the learning experts. The learning portfolio is our holy grail for learning test automation. And in the backlog, then you will see the learning package, what is the title of the element. If it is selected for the backlog or not, and in which sprint it will be executed, then the link to the content, the most important information of this course. And then what is the effort and the costs. Currently, beside YouTube videos, blog posts and articles from medium.com and other websites, sorry, we are using online courses for test automation from Udemy, PluralSide, LinkedIn Learning, O'Reilly, and of course from the Apple Tools Test Automation University. From the Apple Tools Test Automation University, we are using around 95% of the courses in our learning sprints. We are using so many courses from the Test Automation University because of the high quality and the excellent content of these courses. And another important fact, they are free of charge. And with more than 5,000 people in IT who must learn test automation, we save a lot of money. In fact, until now, we have only 5 commercial courses from Udemy and PluralSide in our eLearning portfolio that we are using. 95% of our learning sprints have costs of zero euros. You heard right, zero euros. And back to the learning sprint. Once the learning path is defined, the next step is to agree on transfer tasks. Transfer tasks are an important concept and key to learning in the workplace. For this purpose, given tasks from the project and from the daily work are defined in which the newly learned competencies are then used. Transfer tasks guarantee that the learner applies the new skills directly in everyday life and that the learning has a value for the team. This is pure learning by doing. Remember the 70-20-10 model. And this approach ensures that learning has an immediate impact to the team. Yeah, and then after the planning, the self-directed learning phase, the learning sprint begins in which the goal is to implement the transfer tasks after working on the learning element. And it is important to say the goal is not to watch the learning videos or do the courses. The goal is to deliver the transfer tasks with or without the help of the courses. During the learning sprint, the learner acts on his own, which does not mean that he is on his own. He works on the learning elements and the transfer tasks, maybe together with the help of the learning guide or a learning expert, or he discussing ideas with colleagues or takes part in learning communities. In any case, the learning guide and the learning expert are always available during the sprint to help if necessary. And does anything in the picture looks familiar to you? Yes, here we have Matthias again. And then at the end of the learning sprint, the learner presents his achievement for the learning sprint. The feedback from the team then could influence the next learning sprint. You can see we also leave this agile mindset in learning, inspect and adapt. As an example, during a learning sprint with a learner with the goal of test automation for REST APIs, the team recognized in the sprint review that they could use the automation for creating test data via the APIs. And because test data was a big performance problem for the team. And due to that, we adjusted the next sprint with test data management topics and defined transfer tasks to fit the new goal. As you can see, learning on demand just in time, learning what has an impact to the team. And last, but not least, in the last step, we have the sprint retrospective. And of course we have the usual questions, what went well, what went wrong, what we must improve. And then we have a fourth question, an additional question, which is the most important for us for the learning guides. The question is, have you learned something new that you can use in your daily work? And if the answer is no, then we must adjust the learning sprint or we have a problem. So, and now some agile learning success stories. Then we have Tom. Tom is a data engineer with no test and test automation knowledge. Whose team develops the RBI data science platform. This is going in the direction of Jupyter notebooks and so on for our RBI data engineers in the Amazon cloud. And the challenge, the performance problem was that Tom should automate infrastructure tests with inspect, which was considered the best solution at this time. And already in the first sprint, Tom realized that inspect was not the right solution. And in the second sprint, he did some BOCs for different infrastructure testing tools in order to automate the first test in sprint three with his found solution, which was the Amazon CDK. And then we have Maria. Maria was a business analyst and was tasked with introducing agile testing methodologies and the whole team approach to her team. The challenge was that testing was previously done by an external team that fell victim to a short term reorganization. The Walker world says hello. To make matters worse, a regulatory requirement had to be implemented a short notice, which also affected the topic of testing. And Maria did an excellent job and realized everything within her learnings, her learning epic. And in the meanwhile, she is also the testing chapter lead for six test engineers in her tribe. Well, and in the meanwhile, you should know this guy, you should know Matias. The question is why is Matias so omnipresent? Matias was the first person in RBI who got an agile learning guide. In this case, I was the learning guide due to the huge demand from the teams regarding test and test automation support. We had a bottleneck due to the fact that I was the only Agile engineering coach for test and test automation at this time. The challenge of Matias was he was a developer and wanted to join the Agile engineering team as an Agile engineering coach for test and test automation. But test and test automation was not his area of competence. And one day suddenly he stand by my desk and started the conversation with, oops, oops, sorry, now. And he started the conversation with, hi, I'm Matias and I want to learn everything about test and test automation. Can you help me? But the best would be that Matias tell us his success story. Hello, yeah, it's me, Matias. Yeah, thank you really first to bring me here on stage to give me the chance and also that you use me here as a role model. Yeah, it's already more than two years ago where we two sat together and agreed on my learning journey where I transformed my our upgrade myself from being a developer up to being a test engineer responsible for engineering or focus on engineering and testing. To be honest, it was really challenging. At the beginning it was so much to learn testing is a own world test automation and all these trends. It was incredible. And there are situations where I thought maybe I should stop or how to keep track. And to be honest, without the Agile learning journey which I had and of course without the guidance from Rudy, I would not be there where I am today. Today, I have so many teams to get better in their majority level of testing, test automation at the end, software development, which is incredible, which I'm very proud of. And I'm very thankful for this whole initiative, which is also then responsible by RBI. And, you know, nowadays I'm at the level where I can call myself as an expert because I know also evaluating new things, new trends, trying out new PUCs, new frameworks. And even at the level that I share what we're doing. Today I'm also here at the Selenium conference where I talk about continuous security testing, which is very, very trendy and highly important that recommended to join the talk. It will be even after the session. And this, everything was possible just by this big learning initiative where we are very proud of, where we drive it, where we spread it now in the company, I personally act as now as a learning guide and also learning expert. So I guide people, create their journeys and support them in getting better in the content. And what should I say, that's a cool success story. We're happy to be there. Wish you a great day. See you. So, thank you, Matias. Well, so that was our journey to bring an Asian learning approach to RBI. And last slide, what are our lessons learned? First, there is no one size fits all. In Asian learning, the aim is to give the learner as much freedom and autonomy for the learning process. Not each type of learning approach works for each learner. A generation set employer needs another approach, maybe as a baby boomer. Offering different learning offers is the key to success. Then, meet the cognitive skills. Asian learning combines self-organization and self-directed learning. That requires special metacognitive skills and competencies, because now the learner has to take on task that in the old way of learning, we're taken by the manager, by the teacher, by the seminar leaders, professors or lecturers. We have seen that one of the biggest challenges is that the majority of the learners fail in organizing their learning process on their own. And then, last but not least, dedicated learning time. No matter how much you embrace or reward skills development, if your team members feel like they do not have the time to develop new skills, you are not going to do it. Encourage the learner to block out specific times to concentrate on skill development is here the solution for that. We found out that this is the best solution when you block dedicated learning time in your calendar. So, ladies and gentlemen, we have reached the last slide or the end of my slides. Please feel free to contact me via LinkedIn if you have any additional questions or if you would like to talk to me in general. You won't see Matias on my slides now, but if you want to see more from him, I recommend his talk about DevSecOps, which is coming up right after me. I can highly recommend this talk. So, thank you so much for sharing your experience with us. It's an amazing session. We really liked it. Okay. Thank you. Bye-bye.