 Mr. Banzika will speak about the crashes and what led to the crashes of the most recent 737 model. He is a flight safety, he's an engineer and he also worked on flight safety and he analyzed plane crashes for a lot of time and a long time. And you have to keep in mind that this 737, although multiple models have been built, can be flown, all models can be flown with the same type rating since 1967. Which is one of the many root causes of the issues that led to the disaster that killed 346 people. Let's listen to Bernd and he'll enlighten us what else went wrong. Yes, thank you very much for the introduction. I see there are not quite as many people as with the Edward Snowden talk, but I'm not disappointed. Aviation safety has always been very important to me and I've done a lot of work on it and I'm happy to share my passion with so many of you. Thank you. So it's basically the outline of what I'm going to talk about. It's the Boeing 737 Max or 737 as some may say. I will briefly talk about the accidents, what we knew at the beginning, what went wrong and then what came to light later on. I will show our causal analysis method that we use very shortly, very briefly and the analysis, an overview of the analysis that I did of these accidents. Then talk about the infamous MCAS system, the maneuvering characteristics augmentation system, as it's called by its full name. Then I'll talk about certification, how aircraft certification works in the United States. It's very similar in Europe, although there are some differences, but I'm not going to talk about European details in this talk. So it's mostly about the FAA and aircraft certification across the pond. Some other things and an outlook how it is going to go on with the Boeing 737 Max. We currently don't know exactly what's going to happen, but we'll see. And if we have time, there are a few bonus slides later on. So the Boeing 737 Max, the star of the show as you may say, it's the fourth iteration, as the Herald already indicated, of the world's best-selling airliner. I looked it up just recently. I think there are almost 15,000 orders that have been for the 737 of all the series, the original, the classic, the NG, and now the Max. And the Max itself is the fastest-selling airliner of all time. So within months it had literally thousands of orders. It has now almost 5,000 orders, the 737 Max, and all the airlines in the world are waiting for the grounding to be lifted so they can receive and fly the aircraft. So the first accident was last year. It was a line air, Indonesian flag carrier. Actually, I think the second or third largest Boeing 737 Max customer in the world with a couple hundred, 250 or something aircraft, and it crashed relatively shortly after it entered service. And so we heard some strange things in the news and on the forums that deal with aviation safety. It seems that there had been uncommanded nose downtrim. So the tailplane is moved by an electric motor and it forces the nose of the aircraft down. The pilot can counter that movement with some switches on his control column. And apparently the stick shaker was active during the flight and there were difficulties in controlling the aircraft. We didn't know at the time exactly what it was. And then for the first time the abbreviation MCAS surfaced. And even 737 pilots, even 737 Max pilots, at least some of them said they'd never heard of it. It was a mystery. We later found that actually in some documentation it was very briefly mentioned that such a system existed, but not exactly why it was there. And I guess Boeing knew and the certification authorities, as it turned out, sort of knew a bit of the story, but not the whole story. But especially people in the West, in the US and in other countries said, oh, these are just poorly trained third world pilots and we expect that. And they weren't completely wrong. Lion air has a particularly bad safety record and it wasn't unknown to aviation safety investigators. There have been a number of crashes with lion air. So in the beginning we thought, okay, maybe it's a fluke, it's a one-off or maybe it's caused by poor maintenance or bad pilots or whatever. So several people on the other hand already began worrying because some flight data recorder traces became public and there were some very strange things which we will see shortly. And then only a few months later the second aircraft of exactly the same type and the same variant, Boeing 737 Max 8, also crashed. And you can see maybe on the picture on the left it left a rather big crater. It really dove into the earth quite fast. It turned out I think about between 700 and 800 kilometers per hour, so really fast. And not much was left. I think the biggest parts were about this size, I guess. So all small pieces of debris and the engine cores which are a bit bigger. And from that as well flight data recorder traces became public. The recorders had survived at least the memory in them and were readable. So we finally found out something and found some similarities, some rather disturbing similarities. We'll come to that in a moment, but I'll talk a little bit about the Boeing 737 family in general. So there were four, as I said, models. There was the original which had narrow engines under the wings. Not a lot of room between the ground and the engines. But it looked quite normal, you could say. It was one of the first short-haul airliners with underslung engines under the wings. Then new high bypass turbofront engines entered the market which were much more fuel efficient. We're talking about maybe some 15 to 20 percent lower fuel consumption, so it was a big deal. And the Boeing 737 was re-engined and became known as the classic. Big engines but still mostly analog mechanical instruments. And it was basically the same as the original instead that it had some bigger engines. They had to shape the cowling a little differently to accommodate the bigger engines, but more or less it worked for a while. And then as airlines demanded more modern avionics, so the cockpit electronics in aircraft, the next generation was conceived. It also got a new wing, new winglets, which again saved a lot of fuel. It had basically the same engines except that the engines now were also computer controlled by what we call FATIC, full authority digital engine control. And Boeing said, well, that's probably going to be the last one. And in the next few years we're going to develop an all new short and medium haul single aisle aircraft, which will be all new and super efficient and super cheap to operate all the promises that manufacturers always make. In the meantime, Airbus was becoming a major player with the A320. It was overall a much more modern aircraft. It had digital fly by wire. It always had digitally controlled engines. It had much higher ground clearance, so it was no problem to accommodate the larger engines in the A320. And Airbus then announced that it was going to re-engine the A320. And for the A320, that was the first time it got new engines. For a long time you had the choice of two types of engines for the A320. And then they said, we're going to install these new super efficient engines, which brought with it another optimization of fuel consumption. It was another 15% fuel saved per mile traveled, something on the order of that. So it was a huge improvement again. And many Airbus customers immediately ordered the so-called A320 Neo. And some Boeing customers also thought, well, this one is going to consume so much less fuel that we might consider switching to Airbus. Even though it's a major hassle, if you have a fleet entirely consisting of Boeing aircraft, if you then switch to Airbus, it's a huge hassle. And nobody really wants that, unless they're really forced to. But the promised fuel savings were so big that companies actually considered this and lots of them. And so Boeing said, we need something very quickly, preferably within two years, I think. So that's for airline development. That's very, very, very quickly. And they said, well, scrap all the plans about the new small airliner. We're going to change the 737 again. And now the new engines were going to be bigger again. And so actually there was no ground clearance to move them in the same way as on the NG. So they had to modify the landing gear to mount the engines even further forward and higher. And the engines were bigger. But the engines were on the hold. They were a very good new development. The same type of engines that you could get for the new Airbus by CFM International. And so, yeah, they decided to make the Boeing 737 fourth generation and called it the MAX. So when we analyzed accidents, we used a causal analysis method called YB-cause analysis. And we have some counterfactual test which determines if something is a cause or something else. We call it a necessary causal factor. And it's very simple. A is a causal factor of B. If you can say had A not happened, then B would not have happened either. So I mean, you need to show for everything that this, that there's a causal relationship and that all the factors that you have found are actually sufficient to cause the other event. So you can probably not read everything of it, but it's not really important. This is a simplified graph. And I will show the relevant details later. And this is the analysis that I made of these accidents. And you can see it's not a simple tree. As computer scientists, many of you are familiar with trees. And this is just a directed graph. And it can have branches and so on. And so some things are causal influence, causal factor of several different things. So some of the factors actually have an influence in multiple levels. For example, the airspeed influences the control forces. And it also influences the time the crew had to recover the aircraft before impact with the ground. So these are some of the things that I will look at in a bit more detail. So here's one of them. Uncommanded nose down trim. So what happened apparently on these accident flights was that you can see it in the flight data recorder traces. I don't know. Can you see the mouse pointer? Here there's the blue line. And that is labeled trim manual. And there's the orange line that is labeled trim automatic. And if they have a displacement to the bottom, that means that the aircraft is being trimmed nose down, which means in order to continue to fly level, you have to pull the control column with more force towards you. And what you can see is in the beginning, there are a few trim movements. And on this type, they are expected. It has an automatic trim system for some phases of flight which trims the aircraft to keep it flying stable. And then after a while it started doing many automatic nose down trim movements. Each of these lasts almost 10 seconds and there's a pause between them. And in every case, the pilots counter the nose down trim movement with the nose up trim movement. On the control yoke, there are switches that you operate with your thumb. And you can trim the aircraft that way and change the control forces and cause the aircraft nose to go up or down. So for a very long time this went on, the computer trimmed the aircraft nose down, the pilots trimmed the aircraft nose up and so on. Until at the very end, you can see that the trim, the nose up trim movements that the pilots made become shorter and shorter. And this line here, it says pritch trim position. That is the resulting position of the trim control surface which is the entire horizontal stabilizer on the aircraft. And it moves down and it doesn't really go up anymore because the pilot imports become very short. And that means the control forces to keep the aircraft flying level become extremely high and in the end it became uncontrollable and crashed, as you can see here. So the pilots for various reasons which I will highlight later, the pilots were unable to trim the aircraft manually and the nose down trim persisted and the aircraft crashed. And this is only the graph of one of the accidents but the other one is very similar. And so that's what we see. There is a known system which was already known before on the Boeing 737. I think it's available on all the old versions as well, which is called the speed trim system which in some circumstances trims the aircraft automatically. But the inputs that we see, the automatic trim inputs, don't really fit the so-called speed trim system. And so for the first time we hear the word AMCAS. And we'll talk a bit more about what made the Boeing 737 different from all the previous models and that is the bigger engines. As I said the engines were much bigger and to achieve the necessary ground clearance they had to had to be mounted further forward and they're also a lot bigger. Which means at higher angles of attack when the aircraft is flying against the stream of the oncoming air at a higher angle these engine cells produce additional lift in front of the center of gravity which creates a pitch up moment. And the certification criteria are quite strict in that and say exactly what the forces on the flight controls must be to be certified. And due to the bigger engines there were some phases or some angles of attack at which these certification criteria were no longer met. And so it was decided to introduce a small piece of software which would just introduce a small trim movement to bring it in line with certification criteria again. And one of the reasons this was done was probably so the aircraft could retain the same type certificate as was mentioned in the introduction. So pilots can change within one airline between the aircraft between the 737 NG and the 737 MAX they have the same type certificate. There's a very brief differences training but they can switch even in line operations between the aircraft from day to day. And another reason no other changes were made. Boeing could for example have made a longer main landing gear to create additional ground clearance to move the engines in a more traditional position that would have probably made it more aerodynamically in line with certification criteria. I hesitate to say the word to make it more stable because even as it is the Boeing 737 MAX is not inherently aerodynamically unstable. If all these electronic gimmicks fail it will just fly like an airplane and it is probably in the normal flight envelope easily controllable. But to make big mechanical changes would have delayed the project a lot and would have required recertification and what instead could be done with the airframe essentially the same. The certification could be what is known as grandfathered. So it doesn't need to fulfill all the current criteria of certification because the aircraft has been certified and has been proven in service and so only some of the modifications need to be recertified which is much easier and much cheaper and much quicker. So this is one of the certification criteria that must be fulfilled. It's even though I have removed some of the additional stuff that doesn't really add anything useful it's still rather complicated. It's a it's a procedure that you have to do where you slow down one knot per second and the stick forces need to increase with every knot of speed that you lose and things like that and it says it's stick force versus speed curve may not be less than one pound for each six knots and it's quite interesting if you look at the European certification criteria is that they took this exact paragraph and just translated the US units into metric units but really calculated the new values so the European certification have now very strange values like I don't know 11.79 kilometers per hour per second or something like that it's really strange so you can see where it comes from but they said we can't have knots even though the entire world except Russia and China basically flies in knots even Western Europe but the criteria in the certification specification need to be in kilometers per hour well I would have thought that you would even if you do the conversion you would use meters per second but it used kilometers per hour for whatever reason so due to the aerodynamic changes that were made the max did not quite fulfill the criteria to the letter so something had to be done and as I said mechanical redesign was out of the question because it would have taken too long would have been too expensive and maybe would have broken the type certificate commonality so they introduced just this little additional software in a computer that also existed already and so it measures angle of attack it measures airspeed and a few other parameters flap configuration for example and then it applies nose down pitch trim as it sees fit but it has a rather interesting design from a software engineering point of view can you read that is that there are flight control computers and one part of this flight control computer one additional piece of software is called the mcast the maneuvering characteristics augmentation system and the flight control computer actually gets input from both angle of attack sensors it has two one on each side for redundancy but the mcast algorithm only uses one of them at least in the old version in the new version it will probably use both if it ever gets recertificated and then if that angle of attack sensor senses a value that is too high then it introduces nose down trim and it may switch between flights between the left and the right sensor but at any given time for any given flight it only ever uses one so what could possibly go wrong and here we can see what went wrong it's the same graph as before and I may direct your attention to this red line that says angle of attack indicated left and the green line which says angle of attack indicated right so that is the data that the computer got from the angle of attack sensors both are recorded in the data recorder but only one is evaluated by the mcast and you can see here's the scale on the right you can see that one is indicating relatively normally around zero a bit above zero which is to be expected during takeoff and climb and the red value is about 20 degrees higher and of course that is above the threshold at which the mcast activates so it activates right and apparently in the old version of the software there were no sanity checks no cross checks with other air data values like airspeed and altitude or other things and it would be relatively easy to do not quite trivial you have to get it right in these kinds of things which influence influence flight controls but nothing too fancy but apparently that was also not done so the mcast became active so how could it happen and it's still to me a bit of a mystery how it could actually get so far that it could be certified with this kind of system and the severity of each failure the possible consequences have to be evaluated and the certification criteria specify five severities catastrophic hazardous major minor and no safety effect and that doesn't have to be analyzed any further but for catastrophic failures you have to do a very very complex risk assessment and see what you can do and what needs to be done to bring it in line to make it either mitigate the consequences or make it so extremely improbable that it is not to happen so here are the probabilities with which the certification criteria deal and it's different orders of magnitude there are usually two orders of magnitude between them it's from a probability of one times ten to the minus five per hour to one times ten to the minus nine per operating hour and this is the risk matrix many of you are probably familiar with those and it basically says if something is major then it may not happen with a probability of probable and if it's catastrophic the only probability that is allowed for that is extremely improbable which is less than once in a billion flight hours right and to put that into perspective the fleets with the most flight hours to date I think are in the low hundreds of millions of flight hours combined so we're still even for the 737 or the A320 we're still quite far away from a billion flight hours so you might have expected perhaps one of these events because statistical distribution being what it is the one event might happen of course and but certainly not two in less than two years and quite obviously the severity of these failures was catastrophic I think there's no there's no discussion about that and here's here's the relevant part actually about flight controls in the certification criteria which was clearly violated it says the airplane must be shown to be capable of continued safe flight for any single failure without further qualification any single system that can break must not make the plane unflyable or any combination of failures not shown to be extremely improbable and extremely improbable is these 10 to the minus 9 per hour and this hazard assessment must be performed for all systems of course and severity must be assigned to all these and the unintended AMCAS activation was classified as major and let's briefly look at that what's major a reduction in capability maybe some injuries major damage so nothing you can just drug off but certainly not an accident with hundreds of dead so and therefore there are some regulations would say which kind of kinds of specific analysis you have to do for the various categories on for major no big failure modes and effects analysis FMA was required and these are all findings from the Indonesian investigation board and they're all in the report that is publicly downloadable in the final version of the slides I'll probably put some of the sources and links in there so you can read it for yourselves it's quite eye-opening so only a very small failure and failure analysis was made comparatively small it probably took a few man hours but not as extensive as it should have been for the event had it been correctly classified as catastrophic and some of these things that could happen were not at all considered such as large stabilizer deflection so continued trim movement in the same direction or a repeated activation of the AMCAS system because apparently the only design of the AMCAS system that the FAA saw saw was limited to a 0.6 degree deflection at high speeds and to one single activation only and that was changed and it is still unclear how that could happen it was changed to multiple activations even at high speed and each activation could move the stabilizer as much as almost 2.5 degrees and there was no limit to how often it could activate and what was also not considered was the effect of the flight characteristics caused by large movements of the stabilizer or movement of the stabilizer to the limit of the AMCAS authority the AMCAS doesn't have authority to move the stabilizer all the way to the mechanical stop but only a bit short of that much more than the manual electric trim is capable of trimming the airplane on the aircraft and you can always trim back with the manual electric trim switches on the yoke but you cannot trim it nose down as far as AMCAS can so that's quite interesting that wasn't that that was not considered what was also not considered at least it wasn't in the report apparently that that the Indonesian agency had seen was that flight crew workload increases dramatically if you have to pull on the yoke continuously with about let's say four equivalent to 40 kilograms or 50 kilograms continuously otherwise if you let go you're going to go into a very steep nose dive and at the short at the low altitude that they were they would not have been able to recover the aircraft and in fact they weren't what was also not considered was an AOA sensor failure in the way that we have seen it in these two accidents although apparently they those had different causes the effect for the AMCAS was the same that one of the sensors showed a value that was about 22 and a half degrees too high and that was not considered in the analysis of the AMCAS system so I hope that is readable that is a simplified state machine of the AMCAS system and what we can see is that it can indeed activate repeatedly but only if the pilot uses the manual electric trim in between it will go into a dormant state if the pilot trims manually with the hand wheel or if the pilot doesn't use the trim at all it will go dormant after a single activation and stay that way until electric trim is used so that's the basic upshot of this of this state machine so when the pilot thinks he's doing something to counter the AMCAS and he's actually making it worse but this isn't documented in any pilot documentation anywhere it will probably be in the next way if it's still working like that but so far it wasn't so Boeing was under a lot of pressure to try to sell a new more fuel efficient version of their 737 and so I can't say for sure how it was internally between the FAA and Boeing but it's not unreasonable to assume that they were under a lot of pressure from management to accelerate certification and possibly take shortcuts I can't make any accusations here but it looks that not all is well in the certification department between Boeing and the Federal Aviation Authority so originally the idea of course is the manufacturer builds the aircraft analyzes everything documents everything and the FAA checks all the documentation and maybe even looks at original data and maybe even looks at the physical pieces that are being made for the prototype and approves or rejects the documentation there is already a potential conflict that is not there in many in most other countries because they have separate agencies but the FAA has a dual mandate it is supposed to promote aviation to make it more efficient but also to ensure aviation safety and there may be conflicts of interests I think so here's what this certification has been up until not quite sure 10 15 years ago so the FAA the actual government agency the aviation authority appoints a designated engineering representative the DER is employed and paid by Boeing but is accountable only to the FAA and the DER checks and documents everything that is being done there's usually more than one but for simplicity's sake let's say and the DER then reports the findings and all the documentation all the low-level engineering and analysis documentation that has been done to the FAA and the FAA signs off on that or asks questions and visits the company and looks at things that makes audits and everything like that and so that usually has been working more or less and has certainly improved the overall safety of airliners that have been built in in the last decades and this is the new version and so he's the person is now not called DER but is called AR the authorized representative is still employed and paid by Boeing that hasn't changed but is appointed by Boeing management and reports to Boeing management and the Boeing management compiles a report and sends that to the FAA and the FAA then signs off on the report they hopefully at least read it but they don't have all the low-level engineering details readily available and only rarely speak to the actual engineers so anyone seeing a problem here well you have to say that most aircraft that are being built have been built in the last years aren't really terrible right the 787 is a new aircraft the 777 has been one of the safest aircraft around at least looking at the flight hours that it has accumulated so it's not all bad but there's potential for real really bad screw ups I guess there's another factor maybe that I've briefly mentioned is that the Boeing 737 even in its latest version is not computer controlled it's not fly by wire although it has some computers as we have seen that can move some control for surfaces but mostly it's really it really looks like that I think that's an actual photo from a 737 has some corrosion on it so it's probably not a max an older version but it's basically the same which is also why the grandfathering certification still works so it's all cables and pulleys and even if both hydraulic systems fail so yes it is hydraulically assisted the flight controls but if both hydraulic systems fail with the combined forces of both pilots you can you can still fly it and you can still land it that usually works except when it doesn't and the cases where it doesn't work are when the aircraft is going very fast and has a very high stabilizer deflection and this is from a video some of you may have seen that it's from mentor pilot and he has actually tested that in a full flight simulator which represents realistic forces on all flight controls including the trim wheel you can be in the center console under the thrust levers there are these two shiny black wheels and they are the trim wheels you can move them manually in all phases of flight to trim the aircraft if electric trim is not available within the normal trim system would not do this okay it would require manual trim to get it away from this that's fine all right trim it backward trim it backward as you can so now he's trying to trim it nose up again after he has manually trimmed it nose down because the normal electric trim system cannot trim it so far nose down they have to do it manually and now he's trying to trim it back nose up from a position which is known from the flight data recorder that it was in in the accident flights and is trying to trim it manually because some people said oh turn off the electric trim the electric trim system and trim it manually that'll always work and they're trying to do that and it has representative forces to the real aircraft oh my god okay uh you on pass the red and you can see that the pilot on the left the captain can't even help him in theory both could turn the crank at the same time they have a handle on both sides because he has to hold the control column with all his force so you can't let go he must hold it with both arms otherwise it would go into the nose dive immediately and this is the physical situation with which the pilots were confronted in the accident flights and he now says press the red button in the simulator so end the simulation because it's clear that they're going to crash so there's another thing that came that came up after the accidents and 737 pilot said oh it's just a runaway trim runaway stabilizer trim there's a procedure for that and just do the procedure and and you'll be fine well runaway stabilizer trim is one of the emergency procedures that is trained at infinitum right that's something that every 737 pilot is aware of because there are some conditions under which the trim motor always gets electric current and doesn't stop running that just happens occasionally not very often but occasionally and every pilot is primed to recognize the symptoms saying oh this is runaway runaway stabilizer and you turn off the electric motors for the stabilizer trim and trim manually and that'll work but if you look at what are the actual symptoms of runaway stabilizer it says uncommanded stabilizer trim movement occurs continuously and MCAS movement isn't continuously MCAS trim movement is more like the speed trim system which occurs intermittently and then stops and then trims again for a bit and then stops again so most pilots wouldn't recognize this as a runaway trim because the symptoms are very different the circumstances are different so i guess some pilots might have recognized that there's something going on with the trim that is not right and will have turned it off but some didn't even though they know they all know about runaway stabilizer and yeah that's the second file that i have so that's the sound the stick shaker makes on a Boeing 737 and now imagine flying with that sound all the while shaking the control column violently violently flying with that going on for an hour and that's what the crew on the previous flight did they flew the entire flight of about an hour with the stick shaker going i mean that's quite that's quite interesting because the stick shaker says your wing is about to stall right but on the other hand they knew they were flying level they were flying fast enough everything was fine their craft wasn't about to stall because it was going fast and right so from an aerodynamics perspective of course they could fly the airplane because they knew it was nowhere near a stall but still i think in most countries and most airlines they would have just turned around and landed again and saying the aircraft is broken please fix it something is wrong but yeah so the stick shakers are activated by the angle of attack vane on each side and but the sticks are mechanically coupled so both of them will shake with activation from either side so is it going to fly again it's still somewhat of an open question but i suspect that it will because it's it's hard to imagine that letting these 460 airplanes or some something like that that have been built sometimes sitting around on on employee parking lots like here just letting them be scrapped or whatever i don't know almost 5000 have been ordered as i said neither airlines nor Boeing will be happy but it's not quite clear it's not yet being certified again so it's still un-airworthy so there's another little thing certification issues with new Boeing aircraft reminded me of this have you ever seen that so battery exhaust which aircraft has a battery exhaust i mean what do you do with that does anybody know yeah of course some know yeah Boeing 787 Dreamliner less than two years after introduction or after entering to service actually had two major battery fires they have two big lithium ion batteries uh lithium lithium cobalt i think i'm not sure the one that burns the brightest really because they wanted the the energy density really and that wasn't available in other packages if they had used nickel cadmium batteries instead they would have been like 40 kilograms heavier for two batteries that's almost a passenger so yeah they were on board fires and if you ask pilots what's your worst fear of something happening in flight they'll say flight control failure and fire so you don't want to have a fire in the air absolutely not and one of the fires was actually in flight with passengers on board one was on the ground shortly after disembarking and the lithium ion batteries because they are unusual and novel features as it's called have special certification conditions because they are not covered by the original certification criteria and it says here safe cell temperatures and pressures must be maintained during any foreseeable condition and during any failure of the charging system not shown to be extremely improved extremely remote sorry and extremely remote is actually two orders of magnitude more frequent than extremely improbable extremely remote is only less than once every 10 million flight hours but I think the combined flight hours for the 787 at that time were I'm not quite sure maybe a few hundred thousand at most so and also happened two times that was not really not really fun and then it says no explosive for Texas toxic gases emitted as the result of any failure may accumulate in hazardous quantities within the airplane I think they've neatly solved the third point by putting the battery in a stainless steel box really thick walls maybe I don't know eight millimeters or something like that and piping them to this hole in the bottom of the aircraft so the gases cannot accumulate in the aircraft obviously so yes and with that I'm at the end of my talk and there's no I think quite some time for questions thank you extremely punctual I have to say thank you for this interesting talk we do have the opportunity for quite some questions and healthy discussion please come to the microphones that we have distributed through the hall and while you queue up behind them do we have a question from the internet already do you signal angel is your microphone working no yes do you think extensive software tests could have solved the situation software tests in this case perhaps yes although software tests are really a problematic thing because to test software to these extreme reliability is required you really have to test them for a very very very very long time indeed so to achieve some confidence say of 99% that a failure will not occur in say 10 million hours you'll have to test it for 45 million hours really and you have to test it with the exact conditions that will occur in flight and apparently nobody thought of an angle of attack failure angle of attack sensor failure so maybe testing wouldn't have done a lot in this case thank you microphone number four yes thank you for the talk i have a question concerning grounding so what is your view that the fa waited so long until they finally ground the aircraft week after i think the chinese started with grounding yes that's a good point and i think it's an absolute disgrace that they waited so long even after the first crash they made an internal study and it was reported in the news some some weeks ago and estimated that during the lifetime of the 737 max probably around 15 aircraft would crash so say every two to three years one of them would crash and they still didn't ground it and waited until four days after the second accident yes it's a shame really thank you microphone number seven please thank you for your talk i have a question regarding the design decision to only use one a away sensor so i've read that bowing used the mk system before on a military aircraft and that used both sensors so why was the decision made to downgrade yeah that's a good question i'm not aware of that military system if that was really exactly the same but if that's the case yes that makes it even stranger that they chose to use only one in this case yes thank you okay uh microphone number two please um yeah thank you for your talk um so how do you actually test these requirements in practice so how you determine in practice if something is likely to fail every 10 to the minus nine as opposed to every 10 to the minus eight no that's that's obviously practically completely impossible why you can't as i said if you want to have a reasonable confidence that it's really the error rate is really so low you'd have to test it for four and a half billion hours in operation which is just impossible um what instead is done there are some um industry standards for aviation that is do 178 currently in revision c and that says if you have software that if it fails may have consequences of this severity then you have to use these very strict very formal methods for developing the software like doing very strict and formal requirements analysis specification uh in a formal language preferably and um if possible and some some companies actually do that formally prove your source code correct and in some languages that can be done but it's it's very it's it's a lot of effort and um that's how this should be done and this software obviously should have been developed to the highest level according to do 178 which is level a and quite obviously it wasn't thank you signal angel please the next question from the internet um your talk focused mostly on encas but someone noted that the plane was actually designed for engines below the wings and already and the ng model so the one before already had problems with the wing mounts and engine mounts do you think there will be mechanical problems with the max two i'm not sure there were really mechanical problems there were aerodynamic problems and apparently well i'm sure they have tested the ng um to the same standards to the same certification standards because obviously there were aerodynamic changes even with the ng and the ng apparently still fulfilled the formal criteria of the certification there are some acceptable means of compliance and quite specific descriptions how you test these stick forces versus airspeed and as far as i know the ng just fulfilled them and the max just didn't so for the max something was was required although even the classic which basically had the same engines as the ng even the classic had uh some problems there and that's where the speed trim system was introduced and uh yeah so it has a similar system and actually the m-class is just another little algorithm in the computer that also does the speed trim system please stay seated and buckled up until we have reached our parking position uh no um we are still in the q and a phase please stay seated and please be quiet so we can enjoy all of this talk and uh if you have to have to leave then be super quiet right now it's way too loud in here uh please the next question from microphone number one so considering lessons learned from this accident has the f a a already changed the certification process or are they about to change it or what about other agencies worldwide the f a is probably going to move very slow and i'm not aware of any specific changes yet but i haven't looked into into too much detail in that other certification agencies works works somewhat different and at least the a asa in europe and the chinese authorities have already indicated that in this case they are not going to follow the f a a certification but going to do their own and until now it was usually the case that if the f a a certified the airplane everybody else in the world just took that certification and said what the f a it is probably fine and vice versa when the a asa certified a Boeing airplane then the f a a would also certified and that is probably changing now thank you microphone number three so uh hi uh thank you for this talk two questions please were you part of the official of the investigation or this your own analysis of the uh facts and the other one i heard something about the software being outsourced to india can you comment on that please the first one no this is my own private analysis i have been doing um accident analysis for a living for a while but not for any official agency but always for for private customers um and uh about outsourcing to india i'm not quite sure about that i've read something like that um what i've read is that it was um produced by honeywell i think maybe wrong about that but i think it was honeywell and um who the actual programmers were sitting if it's done properly according to the methodologies um prescribed by d o 178 and fulfilling all those requirements then where the programmers sit is actually not that important and um i don't want to write indian programmers and i think um if it's done according to specification and analyzed with static code analyzes and everything else um vis-a-vis the specification then that would also be fine i guess but the problem is not so much really in the implementation but in the design of the system in the architecture thank you microphone number five please um hello uh um i may got your presentation wrong but for me the real root cause of the problem is the competition and uh hide that line from the management uh so the question for you is is there any um suggestions from you that process could be i don't know maybe changed in order to um in order to um avoid the uh the box in the um in the software and have the mission political systems saved yeah so we don't normally just talk about the cause of the root cause but there are always several causes basically you can say depending on where you stop with the graph where is it uh where you stop with the graph all the leaves all the leaves in the graph are root causes and but i've stopped relatively early and not not done not gone into any more detail on that but yeah the the competition between albus and bowing obviously was a big factor in this and um i don't suppose you you suggest that we abolish competition in the market but what needs to be changed i think is the way certification is done and that requires the fAA reasserting its authority much more and that will probably require a lot more personnel but with the good engineering background and um maybe that would require the fAA paying better wages so i don't know because currently probably all the good engineers will go to bowing instead of the fAA but the fAA ideally needs engineering expertise and lots of it thank you the next question we hear from microphone number four hi thank you for the talk um i've heard that there is or i've heard i've read that there's a version of the 737 max 8 that did allow for a third a oa sensor to be present that served as a backup for either sensors but that this was a paid option and i have not found confirmation of this do you know anything about this no i'm not aware of that as a as a as a paid option um there was something about an optional feature that was called a safety feature but i can't exactly remember what that was maybe it was an angle of attack indicator in the cockpit that is available as an option i think for the 737 for for most models because the sensor is there anyway um as for a third a oa sensor i'd be surprised if that was an option because that is a major change and requires a major change to all the system layout then you'd need an additional a data inertial reference unit which is a big computer box in the aircraft of which there are only two and that would have taken a long long time in addition to develop so i'm skeptical about that third angle of attack sensor at least i've not heard of it thank you signal and do we have more from the internet please one quick one um if we need a quick one would you ever fly with a 737 max again if it was ever cleared again i was expecting that question and actually i don't have an answer yet for that and that maybe would depend on on how i see the f a and the asa doing the certification um i've seen some people saying that the 737 max should never be re-certified i think that it will be and um i look at it in some detail seeing how the f a develops and how the asa is handling it and then maybe yes great okay in that case we would take one more very short question from microphone number five do you know why the important a oa sensor failed to give the correct values there are some theories about that but i but i haven't investigated that in any more detail now there were some stories that in the case of the indonesian the line air that it was actually mounted or reassembled incorrectly um that would explain why there was a constant offset it may also have been somebody calculated that it was actually exactly if you look at the raw data that is being delivered on the bus there was exactly one flipped bit which is also a possibility but i i don't really know but there were some implications in the report maybe have to read that section again from the indonesian authorities about um substandard maintenance as it's euphemistically called okay we have two more minutes so i will take another question from microphone number one hey i would have expected that modern aircraft would have some plug physical plug hermetic one that would disconnect any automated system isn't something that exists in our plans today now and especially modern aircraft can't just disconnect the automatics because if you look at modern fly by wire aircraft there is no connection between the flight controls and the control surfaces there's only a computer and the flight controls that the pilots handle are only inputs to the computer and there's no direct connection that is true for every airbus since the a320 for every Boeing since the triple seven so the triple seven and the seven eight seven are totally 100 fly by wire well i think 95 percent because there's one control service that is directly connected one spoiler on each side but basically there's there's no way and so you have to make sure that the flight control software is developed to the highest possible standards because you can't turn it off because that's everything that's well let me put it this way on the fly by wire aircraft only the computer can control the flight so the flight control surfaces yeah so yeah just hope that it's good think about that when you next enter a plane and also please give a big round of applause for our speaker band Zika thank you