 Welcome, I'm Nancy Vential Underwirt and I'm co-presenting with Brian about successes in agile and medical devices. I'll just kick off by saying welcome and we wrote our book that we'll be speaking about some case studies from a book we've just created. And, you know, it's really about, I think the real stories are better and more interesting than anything a person could create as a novel or something of that nature. And there's twists and turns and things you just wouldn't expect and creative approaches to using agile. So that was inspiring to us. So I'm gonna give Brian a chance to go through some introductory slides and then we'll each talk. So just to give you a sense of who we are, I'm actually originally an analytical chemist, although I've done some tinkering around with some programming, I do not consider myself an expert developer. I've worked for 15 years in the area of clinical diagnostics. For that I mean the development of the tests that are used to measure this, that or the other thing in somebody's blood sample or in some other kind of a sample. I was supporting that kind of work with doing chemical analysis that I would end up actually developing assays. And then I ended up validating the software that was embedded on the instruments. So I did that, there was 15 years with that collection of stuff. And then another six years as a software quality manager, most of that in the clinical trial data management software. And I'll talk a little bit more about that in the first of the examples that we cover. I've been working independently as a validation consultant for medical regulated companies, those regulated by the Food and Drug Administration. Most of them are medical device, but some of the work has been in the clinical trial data management area. So the kinds of things that I work with or software validation evaluation of the plans people have for fulfilling the requirements of the part 11, 21 CFR part 11, the electronic records and signatures, regulation. I also help people set up their sell up or tweak their quality systems for developing software or take part in the audits that they have to undergo and do some training. So Nancy. Okay. I'm Nancy Vinci showing the work. I worked originally as a designer of electronics and doing software. And this was for industry applications originally flight simulations for commercial and military aircraft. I've also worked in factory automation and medical devices and some defense work too. So for over 15 years, I've worked in safety critical applications that involved hardware and software both. So I got to use both parts of my background. In late 1990s, early 2000s, a lot of the agile practices were beginning to surface before agile had a name. And I began using those with teams that I worked with and I was really impressed with how effective they are for both hardware and software. So I had a good four or five years experience by the time agile started getting talked about. Anyway, around about 2002, 03, I just decided that I wasn't gonna use one of all practices anymore, but there weren't any companies using agile. So I decided I would try to make that happen along with enough other people. And ever since then, I've been coaching agile teams and their managers. I have something of a specialty you might say with hardware and with medical devices when I team up with Brian especially. And that's pretty much it. Back to you, Brian. So what we wanna talk about in this session is that working from first principle, it's not just copying somebody else's methods, allows groups that work in the regulated medical industry to adopt agile methods while they still fulfill the mandates that they have from their regulatory bodies. So we'll take, and this is based on the set of examples that we wrote up in a book that we published recently where we agile methods for safety critical systems, case studies of medical product companies. And you see here the link for checking out the information about that book. Now, what prompted putting the book together was two key things. On the one hand, there's this belief in the quality and regulatory world that still seems to persist that agile's not allowed, that agile is a thing that, no, no, no, no, you can't go there. And the other is the management mentality that says, yep, if it's a process, plug it in, make everybody follow it and check off all the boxes. On the one hand, in concerning agile being allowed, the association for the advancement of medical instrumentation, Amy, put out a technical information report, TIR number 45, back in 2012. Putting together that report involved people from the agile community, people from the regulatory area, there were a couple of FDA people involved, and people from the medical device industry who all collaborated to explain how an agile approach can in fact be used in the regulated environment, how the two sides really can be reconciled with each other. But even despite that technical information report, there's still some quality assurance people say, yeah, yeah, yeah, I've heard of this TIR 45, but I really don't see what they're gonna be doing with that. So that is still persisting. Gradually, some people are starting to see the light. I mean, I saw this a number of years ago when Nancy and a couple of colleagues of hers first showed me a test-driven development approach using a development board. What I had been thinking for so long that the testing of a piece of software was something that was tacked on at the end. And if a schedule was being compressed and the managers would always cut the quality process, I realized that building the quality process in made much more sense. And that's why Nancy and I have been collaborating all this time. Too often in the check the boxes mentality, somebody says, yeah, yeah, write the SOP, you're done. Then everybody will follow it. And they impressed the process as a top-down thing. And we know better than that in the Agile community. So here are the people that we talk to to put together examples. A company called Open Clinica in the Boston area where I used to live that started up providing data for managing clinical trial data or software for managing clinical trial data as an open source tool. Some amount of their stuff is still open source. In PECO, an Italian company that does automations for large clinical laboratories where they actually run the tests that determine the concentrations of this, that of the other thing in blood samples and others. Sanity that I did some consulting with for several years. A medical device group within a large multinational pharmaceutical company. Neurometrics and atomic object working together on a pain relief device. And Blue Fruit Software, a contract software development company in Cornwall, England that does a lot of work in the embedded systems area and has moved into the medical device embedded software field. We're only gonna talk about two of the seven examples that are in our book in this session. And the key thing about all of the examples that we described is that although they applied Agile and Lean principles in different ways, certain basics were common among all of them. So we'll start off by looking at my friends at Open Clinica. I did, I helped them set up their quality system and have off and on worked with them over the last 10 or more years. A clinical trial is the set of experiments that people do in order to determine whether a drug or a medical device or a vaccine is both safe and effective for use in humans. So they have to have volunteer subjects who agree to be treated by the drug or the vaccine or the whatever. And they have to have a particular structure so that they check the volunteer on periodic basis and determine how well the person is responding. Every trial has a different structure. There's a certain flow of events, there's things like skip logic, end points, there are checks that have to take place to make sure the data are valid. You don't wanna have somebody in the database being a classed as male and yet in another place in the database being listed as being pregnant. That kind of stuff, those are some, that's one of the more obvious checks that needs to take place. New features are being added all the time in the open clinical product because data managers say, well, I need to be able to do this or I need to be able to do that. And in most cases, people have used the software in a hosted environment. There's a heavy emphasis on the design of the software for usability. So open clinical takes a very flexible approach to planning out what they're gonna do. They, on a regular basis meet, they take their product owner gets input from the various data managers that are handling that, that are actually doing the clinical trials, managing the data from clinical trials and they get together internally between their client services, their software quality assurance people and their developers and say, well, what are the kinds of things people are asking for? Then look through them, analyze them, break things down and come up with a list of feature requests and then rank those on a roadmap. And notice this is a process that goes on a regular basis so that roadmap gets adjusted. From that roadmap then, they have a plan. They say, okay, over the next three months, we're gonna work on these minimum-releasable features. And so the regular cycles take place. There's the cycle of the development and the internal cycles of the sprints that result in minimum-releasable features that are composed of a series of stories where each story has its own acceptance criteria. And those things are linked in JIRAs. Now they're stored in JIRA linked via the Cucumber tool. The company went through a training and practice process several years ago wherein they set up a behavior-driven development process and the Cucumber tool so that when they create their stories, they set them up using the gherkin given when then expression, which not only gives them the story, but also the tests and test everything via the Cucumber tool. What that gives them is documentation that comes out of that as a natural outcome. They have a report that's a list of stories, which are basically the same idea as software requirements. Those stories are linked to their respective tests, so they are producing their traceability matrix. So there's a couple of key parts of their regulatory required documentation that come out just as a natural consequence of their BDD process. And oh, by the way, because the tool is highly configurable, they can set up automated testing through their BDD structure so that they can test lots of different permutations and combinations. The company is committed to a continuous improvement process. Remember, that's one of the key things about a quality process that you're always looking for ways to improve and make things better. They never used to estimate their stories, but by doing so now in the last few years, they improve that process and are more able to predict when they'll have certain kinds of things finished in their development process, because they have story sizes. When they run retrospectives, CAPA stands for Corrective and Preventive Action. That's actually a required part of a quality process within the regulated industry. Corrective and Preventive Action says, okay, we're gonna say, we need to do something about this thing because either a problem occurred or we can see that there could be a problem. That's the preventive part. When they do a retrospective, they'll take a couple of the most important needs improvement areas and say, put them into their CAPA process. So right there, the regular rituals of the agile process feed their regulatory required process. Autonomation, automation with a human touch. The automation that they achieve by using their BDD process with the automated testing that comes out of the CAPA process allows them to cover many, many more permutations than they would otherwise do if they were doing these things manually. And oh, by the way, because their customers have to audit them for their quality processes, their auditors have been impressed with the requirements, the tests and the traceability that have been generated by these processes. So as a quick sum up, there's a flexible planning process baked in, the automation with a human touch allows them to test things much more thoroughly because it's a highly configurable product. Their BDD and cucumber have given them auditable documentation and their retrospectives have fed their required quality processes. So over to Nancy. Okay. Next, we're going to talk about leading through quality how Blue Fruit Software entered the medical device market. And first off, I'll say something about who they are. They're a group of embedded software specialists. They are based in Cornelon, England, around 70 employees now. Their client base tends to be in the automotive and industrial spaces so far, for the most part. And they've more recently started getting into medical device work. Their founder, Paul Massey, was a software developer who had been doing contract software work. And he was looking for a more positive environment for developers to work in, especially developers like himself, who really wanted to do high quality software. And he had a strong commitment that quality needs to be driven from the bottom up, you know, by the hands-on people and for the right reasons. The company recently became employee owned. And I'll say a little bit more about that as we go further. But one of my favorite topics about real stories is when there are surprise benefits. So we'll talk about those two. But back to the origin of the company. They've been growing about 50% per year, ever since their founding 20 years ago. And they have very low employee turnover, only about 4%, where the norm for their type of company is more like 10 to 13%. And before I get into telling the story, I need to set the stage a little bit by explaining what certifications are. And basically, I'm going to talk a little bit more about the company. And I'm going to talk a little bit more about the company. What certifications are. And basically, these are standards that are based in laws and so forth, or various companies. They're standards that are issued by regulatory bodies. And they can, and companies are audited on the basis of what these standards call for. I'll just point out a couple of them here. These are some very common ones that are encountered over at the left in this chart. You see there's IEC 62304. That has to do with software development life cycle. And Brian alluded to it a little bit earlier when he said that, you know, there's a myth that agile is not really permitted or is frowned upon. That's just not true. If you're actually reading the standard, agile is a better fit in many ways than waterfall. For what the standards looking for. And there are others. There's usability. There's risk management. You see over at the right hand edge of the screen. 21 CFR part 11. That one's actually a law, but that has to do with electronic signatures and electronic documentation, something that's playing a role in all companies now. Well, maybe not all, but you know, it's getting much more wide use than before. Just so that. Sorry. Go ahead. Many of you are probably familiar with ISO 9000, the quality regulation. The one shown here that says ISO 13485 is simply the medical device version of ISO 9000. Sorry, Nancy. Okay. Well, that was a good point to make. Anyway, another one, another thing to mention is that these certifications often make reference to having a quality management system. And if we could go to the next slide, a quality management system is something that companies need to have in place. Your software needs to have been created within the context of a quality management system. And I actually like that concept because it, I think it fits well with that job. And I'm going to ask Brian to say a few words about it because this is squarely in his area of expertise. You know, Brian, why don't you give us the definition of a QMS and why the regulate regulatory bodies insist on it. This is in fact why the comparison with the ISO 13485, ISO 9000 is apt in this circumstance because the quality management system sets out kind of your ground rules. Here's how we're going to go about these things. Here's what we expect. Here are the outputs we want to see created when you do something like a software development. Here's what you're going to do to make sure that you provide proper service and document the proper service when there's something that needs to be serviced at a customer's site or supported in some way. Here's how your management is going to check to make sure all the quality processes are working well and what things need to be improved. So all of those things together make up the quality management system. The key thing to remember in the eyes of any of the regulatory bodies or the notified bodies that do the reviewing and auditing, if it isn't documented, it wasn't done. So there's a documentation element as well as a practice element. Yeah. So thanks, Brian. So moving right along here. As Blue Fruit managers were thinking about maybe getting into medical device work because after all they were very committed to high quality work and lots of medical devices are embedded systems. And they gave some thought to getting involved in this. And the idea is, well, they felt a little hesitation because they thought, well, if we have some expert design a QMS for us that's compliant to the regs, they were called. You can bring somebody in. They can tell you what to do and then they can help guide you so that within a few months time you'll have what certifications you were aiming for. The feeling was, well, if we get some outside designed thing into our company, that's going to break empowerment or it could. The idea being that we want to have our engineers doing the right thing because they're committed to it and because they have a clear line of sight between what it is they're doing and how it serves the overall quality of the product. Empowerment is really core value at the company. And Paul, their founder, defined it like this to us. He said it means avoiding creating situations where management has to overrule teams in order to safeguard a larger or longer term interest. For instance, gaining a certificate. And that's, so Paul has a bit more philosophy in this nature too. Next slide, please, Brian. Here we go. Paul had this to say about empowerment and I really think it embodies good agile management. He said if it's my project, if it's something that I'm championing, whether part 11 or in the past it's been BDD or TDD, I've always said to myself that if the team doesn't buy in, then we're not doing it. It's not happening and I'm absolutely committed to that. The onus is on me to get that buy in. And what he really means here is he's got to sell the idea on the merits to the developers, get them to honestly buy in, not just order them to go through the motions. There's a big difference. Yeah, so anyway, so as the story goes on, it happened that they were approached by a company who needed to get their software redeveloped. Let me give you a little backstory about that. They were approached by a company because of their reputation for quality. Now this other company, this potential client, they had a product that they had created. They were somewhat new to the medical device field, so they didn't understand the whole QMS idea. And they had been advised by, you know, business people that they knew that they were going to have a hard time with the FDA because of the way their software had been developed in a, you know, traditional waterfall type of environment. And they'd been told that there's no way their software would pass an audit because there were too many telltale signs that it had not been created within a quality management system. For example, that anybody looking at the documentation would have been able to see that the unit tests were not created, contemporary to the code, what design reviews they had weren't really actually about people planning the behaviors of the software. So there were enough signs like this in the documentation, and they were told that they would just have to, it would be better to just redevelop the software. And in case, now what I want to do is pause just for a second here and let Brian give us a little more background and explanation about why this happens. And is this really common? Have you seen this before? Brian, you want to take a minute and give us a little explanation on this. Being forced to go back and rewrite the software completely is a little unusual. However, there are plenty of circumstances where a developer, there's too many opportunities when you're writing things down to falsify stuff. And there's too much temptation to write stuff down that may or may not have ever properly happened. So there's a lot of, there are rules around your documentation that needs to be attributable. That is, you know who put that information down, the legible, consistent and coherent, captured at the contemporaneous, captured at the time you actually did the work. And there's several other criteria along the way. So in a case like this, apparently, the documentation just did not fulfill the so-called ALCOA, the attributable legible contemporaneous, et cetera, et cetera, full criteria. It was too obvious that something was wrong and that maybe the documentation was correct, maybe not, there were too many big questions about it. So that's why this kind of stuff takes place. As I say, this is a pretty unusual one to have to go back and completely rewrite, but it's not impossible for that sort of thing to happen. Okay, thanks, Brian. And so, you know, you can, by what Brian just gave us as an explanation, you can almost see why Agile, when it's done properly, is really going to fit all these needs that the regulators have. This is one reason why I think Agile is a perfect fit. But anyway, let's go on with the story. So this client, they found blue fruit because they made inquiries around, you know, they started inquiring what sort of processes companies had internally, the companies that they were considering to rewrite their software. And now, since they had learned in the School of Hard Knocks, so to speak, what they had been missing, they were extra, how do I say, they really had their antenna up for noticing what things would pass regulatory scrutiny. So the use of BDD and TDD, those are highly documented actions and auditable pair programming as a way of reviewing code. You know, in short loops, blue fruit also had a blind hiring process that I haven't said anything about yet, but their hiring process screened first through online exercises, screened first for a person's ability to problem solve and to, you know, code. And so, you know, in other words, all of this was documented and could be, there was evidence contemporaneous to all these actions, you know, as somebody went through the hiring process, blue fruit could demonstrate. So all of these things served to show that blue fruit had a genuine bottom-up quality management system that could easily pass regulatory scrutiny. Next slide. Yeah, we go. But blue fruit then had another little hesitation of their own. They started to pull back a little bit. They started to think, well, geez, you know, if we get into this industry, do we really feel ready? Do we, you know, I mean, if we made a mistake, somebody could come to real harm. So how confident are we really? So they started to have this little feeling. And, but then they started looking at the code. The company showed them the code, you know, as part of their process of engaging with someone. And this is a quote from Paul. As they were looking at the code, he said, it kind of drew our team in. And so rather than feel anxious about that kind of responsibility, making sure that people couldn't come to harm if we get this wrong. It was more like, if we don't help these folks up, they're going to come to harm. Because they could see that, you know, there were issues with the code that just wouldn't happen when you're using test-driven development. Just wouldn't happen when you're using the various cucumber kind of tools and such. So in a funny way, it turned around their, their anxiety. They also had some concerns about getting into an industry that's known for being bureaucratic and slow and, you know, just lots of paperwork. But those issues melted away the more they got drawn in, as, you know, as the quote says here. So the teams felt they started to feel even more confident in their agile coding by looking at the contrast with what, you know, they would be replacing. And so they were pretty excited because they could really see what additional value they could create in this situation. So this was a boost for the morale. And, and I mentioned earlier, surprise benefits, one of my favorite topics. You know, another thing happening alongside of all this was their move to employee ownership. And of course they did a lot of looking ahead and a lot of, you know, testing the waters before they went. But it's funny how there are sometimes benefits no one expected. They just pop up anyway. And there were two of those. One of them is there being less chance that a competitor will buy a blue fruit software due to legal barriers to sell. You might ask, wait a minute, what's this? How is this a benefit? Well, let me say first how it works. Becoming an employee owned company. First of all, a decision to sell the company isn't just up to one person anymore, one owner. It's up to more than one person. So that's a little bit puts the brakes on a company being sold suddenly. Another reason is there are tax breaks in the way the legal system works in the UK. A company gets tax breaks when they go employee owned. And if they go private again, which you can do that, but you would have to pay back some amount of those tax breaks. So those things all turn to damp. They tend to dampen down the possibility of company will be sold. So why does that matter? Why is it a benefit to customers? Well, think about it. Let's say I'm a medical device company and I use blue fruit to build a couple of my products or their software. I might be hesitating to give blue fruit too big a chunk of my work because maybe they'll get sold to my competitor. And then all of a sudden I'll be heavily dependent on a competitor. So I might have been holding back. But now that I see they've gone employee owned and it's likely they're ever going to get bought. I might feel more free to give them a bigger chunk of work that I could give to them. And there's less chance of that competitor ceasing any support of the software that they've already developed for you. Yeah. And the other thing too is there without having outside shareholders, it means they don't need to build in an extra layer of costs that are there in order essentially to make sure that they don't have to build in an extra layer of money, dividends and profits to people who really aren't involved in generating value for the company. So those are a couple of benefits. They hadn't even anticipated and summing up here. Okay. I'm impressed by how this story has two principles of Agile really working in harmony, quality and empowerment. The company would was founded in order to do high quality work. It turned out very quickly that the best way to keep on doing high quality work is to keep empowerment. Alive every step of the way as the company grows and becomes more complex structurally just because it has to. This is part of the story that comes out in the case study as we tell in the book and we're telling today a tiny fraction of the story. So there's much more, but at every step of the growth, they needed to do things to really fight for and keep empowerment alive each step of the way. And so quality empowerment kept on this duality. All right, Brian, you want to sum up some more. So one of the key points that we've agreed upon and that that the book really shows is that blind copying was not any kind of a way to get to achieve an Agile process, rather use of basic principles. So the principles that we like to bring out about our different examples are collaboration based on trust. So many of the of the presentations at this conference have talked about various kinds of collaboration. Flexible planning. You saw in the open clinic, a case how they revisit their plan on a regular basis. Feedback loops. This this team, the group at this conference should should know all about feedback loops. And then the automation, the automation being monitored by a human to make sure that things are going right and that if something is wrong, that the line can be stopped. So our takeaways, each of the two cases we're talking about here have got some key takeaways. Open Clinica use a flexible planning approach to continuously update their roadmap. Their automation gave them both the multiple benefits not only of much more thorough testing of their multiple configurations, but also auditable documentation resulting naturally from the process and that their Agile process fed into fed information right into their expected quality processes like their corrective and preventive action. In the case of blue fruit software, their focus on quality really is what attracted the customers to them from the founding of the company doing high quality work in a positive environment. Their Agile practices all fit the quality management system concept. So it was easy for their clients to say, yeah, we can rely on these people. And even though they don't know what to do, they may not have gone through the whole process of getting the ISO 13-485 that ticket to hang from the wall, their quality process has attracted the clients whose non-conforming software had to be rewritten. That focus, as Nancy was just talking about, empowerment was absolutely baked into the overall culture of the company. And then becoming employee-owned pretty quickly because that's what we're talking about right now. We're talking about the fact that the company had to be baked into the overall culture of the company and then becoming employee-owned to preserve their Agile culture in the bottom-up commitment and that they had surprise benefits. I've been involved in cases where people had to... You want to say something? I just want to interject one more thing here. Blue fruit never did end up going and getting to be the certificate holder. And that makes those companies responsible to the regulatory bodies to make sure that any software they subcontract is also in line with the demands of the certifications. So they still are operating in that way. Just let me give a quick update. Well taken point in the regulated medical industry. Anybody who outsources parts of their work is responsible to review the quality processes of their vendors to make sure their vendors are doing quality work. So that was an important point. And the item about long-term support of code, I've been involved in cases where people had actually decided to escrow force the supplier to escrow the code being... That was developed in case that company that did the development ever went out of business or was bought and the software was not going to be supported any longer. In Blue Fruit's case, because their employee owned, there's a lot less chance of just the company disappearing tomorrow. So summing all of these things up is that understanding the principles is important but not sufficient. Understanding the context, how what's being developed and who's being served is important but not sufficient. That the practices that are being chosen have to realize a particular principle in a particular context. I like to use the image of choosing the right tool for the job, Ergo, the illustration here. You support a particular principle in a particular situation. So here's our services. We overlap but do different kinds of things. And distance is really not an issue for the people that we work with. And here's our contact information. So we'd be happy to take any questions at this point. Could you please give an example of a user story? What a user story would look like in your area? Nancy, you want to try that one? Sure. I mean, in embedded systems, a lot of times the user stories are very down in the tech. So as you're just starting to get going with some things that are down in the electronics, they can look like any other embedded system, things like bringing up flash memory or bringing up a display panel on a circuit board, things of that nature are possible, but things that are more customer facing might be things like verify patient identification, a screen for that, you know, or various types of screens that a practitioner would use as they're working with a device. You'll have stories around those things as development proceeds a little further. So we tend not to write stories that are whole giant features because they're just too big. But I mean, we'll have that concept, you know, some people call it epics or whatever. Some people say, no, no, you shouldn't have those. But I mean, you do have to have an overall concept, but I tend not to get people wrapped up in writing stories that are very large features that I know are going to only emerge over quite a few sprints. And in the case of the folks at OpenClinica, their kinds of user stories might read, given trial, thus and such and so forth, has been set up and that the patient is in the second visit of the sequence and that certain other edit checks are in place when I enter the such and such piece of information given when then the system shall skip over the following question because the question is not relevant. That's the kind of user story that, and they may have a given with a whole bunch of ads attached to it because the situation can be very, very specific. But that's why using the Gherkin notation is very, very powerful for them because they can specify right down to the exact thing that they want to have happen. Yeah, did that answer your question? Let's see if I can see what's going on in the chat. Hope that it, oh, okay, he says yes. Thank you. I'm glad that answered your question. Thank you, Nancy and by Nadine. Okay, super.