 Dutch collectors of volunteer, helpful hackers, and we aim to make the digital world safer by reporting vulnerabilities we find in digital systems to people who fix them, who can fix them. We have a global reach, but we do it Dutch style, open, honest, and collaborative. And that sometimes makes people unhappy. So I'm a bit stressed out because this just happened. This is a statement from the vendor saying that that account that's named super admin is in fact not a real system administrator and is not subject to our testing group. With sort of abilities to carry out function tests, we are just informed once in early July and then took the actions immediately. They asked and then they asked us not to give this talk. So this can either be short or not. Open honest, collaborative, we're not going to propose this disclosure. We've given the vendor till 20 minutes ago to come up with a reaction. I've not yet received such reaction. I will check my phone at the end of the talk. So my name is Frank Bredijk. I'm case handler for DIVD case 2020-0009. I'm a crisis manager at the DEAVD and my day job is the CISO at Schubert Phyllis. This is how you reach me. And it all started with an honest tweet from Celestine, one of our DIVD researchers who noticed that her parents had solar panels that had omnic converters in them. And omnic went bankrupt and now the inverter is sending its data to SolarMan in China. Funny ha ha in a kind of GDPR way. This talk, although it is an issue that data of European citizens just gets sent off to China, this talk is not about GDPR. I was expecting a sigh of relief. But the tweet was followed up by a tweet from Yeller, smart, showing how he logged into SolarMan with the Superman user, super admin user. So who's Yeller? Yeller is him. He's not here. He's taking a vacation, well-deserved vacation, but he's the main researcher on this. And some people may know him as Schizoducky. Some people know him a little bit more than others. What does Yeller do? Well, he's a programmer architect, does DevOps, is good at finding passwords in things like public GitHub repositories. He's done hundreds of responsible disclosures in the last three years. If he knocks on your door, he's in trouble because he didn't have to knock. He could have just walked in. He had the key. And he's too surprised about all the brands he's been able to get into so far. And he needs to add one. So this happened in February. And yeah, now what? And back in February, sorry, back in April 2021, DIVD wasn't what it was now. We were still developing what we did, what kind of cases we took. And did this not qualify as a DIVD case then? It does now, by the way. So we reported it to SolarMan. We did help Yeller reach them. And although we never got a reply back, the password was changed a few days later. Case closed. Life goes on. And we fast forward to February this year and Yeller reads the blog post by Jan van Kampen. I don't know if you're here, Jan, but if you are a good post, about problems with connected inverters. And that made us wonder, or it made Yeller wonder, is SolarMan still okay? Not really. He could log into the super admin account again. So what have we got? What did we find in the GitHub repository? Yeah, that's SolarMan, but not this one. So SolarMan claims by themselves that they are professional remote monitoring and management is my addition, solution for devices. As from SolarMan, Solis, Omnic, and Ginglong. Mostly inverters, loggers, and batteries. And yes, these were the super admin credentials for the monitoring platform. So what could we do apparently with this account? Let's read the data of all the users. And we've confirmed that. We could see names. We could see addresses. We could see email addresses. We should see current and historical generation. We could create and delete users. Change configurations, calibration offsets. Read and clear errors in the converters. Download firmware versions. Upload new firmware. We have an upload screen. Yeah, basically this is the list of build your own inverter botnet. We haven't tested this because of ethics and our code of conduct. Maybe if you want to do responsible disclosure, you have to be your measures, your tests do not have to be worse than what you're trying to prevent. Also it has to be subsidiary. That means if you can use the smallest, a small hammer and a big hammer, you should use the smallest hammer possible. This would have been too big a hammer at the wrong time. But we did confirm the data in there. We did confirm that the GUI showed us these possibilities. So what numbers are we talking about? These are the numbers from February. Remember it's winter in February. Globally there were close to a million plants, locations with solar panels, of which 42,000 in Netherlands, 7,000 in Germany, 7,000 in Belgium and about 13,000 in the UK. That are producing in February 10.03 gigawatts. That's serious numbers. So what's the impact for the Netherlands? Well, if you take 40,000 plants with an average of between 4 and 10 kilowatt peak per plant, you add some bigger plants like the ones we've listed here. You combine, come up with a combined power of 400 megawatts. That's a little bit bigger as this thing here. And that's the VELSEN-25 electricity plant on the Hogehove, Rijn in VELSEN. Yeah, so what could you do if you had admin? Install custom firmware, create a botnet, use lateral movement. So move from the converter further down into the network. Physically damage or overload the converter. That could lead to breaking it or overloading the device. We could lock the vendor out. Use their password, see what would do that. Could you hurt the grid? I don't know. I'm not a grid expert. I know some people who know something or know people who know something. One plant shouldn't be a problem. And it's not a novel idea to hurt the grid. This was sent around during the farmer protests earlier this month, on the 4th of July, to all farmers and owners of panels and windmills. Let's all turn off our solar inverters on the 4th of July. Wind power, too. We want to make a statement that 20% of all green power comes from farms. If we do it all at the same time, we can cause outages. Turn off at 4th of July, mid of day, bam. This is a forward message to forward. Well, if a bunch of farmers can come up with this idea, I mean, I know farmers are smart, but nation states, I think they're smarter. And taking panels offline is one thing. But what if we go and use it to do aerobics exercise? Take it up and down and up and down. That would cause a serious problem on our grid. Or take all the panels in the north of the Netherlands offline, and then once powers ramped up there, turn them back on and turn all the power in the south of the Netherlands. That would be fun. So getting this fixed was a long and windy road. This credential was actually committed to GitHub on the 5th of August, 2019. We discovered them in April of 2021. It was changed. In February, we noticed it was working again. And we opened the case. By the way, we welcomed Jelle as a researcher in our midst in the I.V.D. We contacted the Dutch National Cyber Security Center. We notified the vendors on the 9th of February. We worked with the NCSC on the 20th of February. They worked together. They contacted NCCNL and ChinaCert. We mailed ChinaCert. We got a little help from the Dutch Embassy. We visited the Chinese Embassy ourselves. Finally, there was contact with ChinaCert. And then things went really quickly. ChinaCert really did their job. It's like 17th. We contacted them. Fender changed the password. Github repository was deleted. But this thing was exposed for 777 days, at least, because we don't know when it was turned back. What happened in between? I don't know. So it's fixed. Github repository gone. Password changed. Well, yes, we resolved a situation where everybody could potentially mess with these devices. But there has to be a super admin password somewhere. A part he has to control this. Is that a desired situation? Well, the Dutch Ministry of Economic Affairs and Climate has blocked the Chinese company from bidding on the construction of what they call the stop contact of say, a wall socket at sea. And they did this because they think having a Chinese company controlling this much power on the grid is a security risk. So what about those solar panels? You may wonder, why doesn't anybody do something about this? Well, grid operators and energy companies have no authority over what happens behind the smart meter. If you overload your local grid, your fuse burns, you're off the grid. Problem solved. They can't put any regulation or any enforcement in place. Building codes are about electrical safety. And we have an authority for IoT and IoT devices. But yeah, they can only interfere when devices are insecure. There's new regulation on the way that requires registering and certifying devices if you want to sell them to consumers, which you probably can circumvent by going to AliExpress directly, but different matter. But even if these devices themselves ever become 100% secure, you still have a foreign party that controls a major amount of power in the grid. So that brought me back to the opening talk that Miko Hippen and gave, where he said, we're doing security. We're no longer securing computers. We are securing society. So statement from the vendor. Let me check. I have a statement. Dear Frank, this is Wei and I'm responsible for marketing department. Thanks for your reply. And thanks for any way for Dutch ethical hacker finding an issue for us. At present, we're absolutely off the court with the upcoming disclosure and communication time left us is extremely limited. It's an opinion. For now, I can only say, firstly, the account is subject to testing group and not a super admin account. Which later we can provide proof in details. Secondly, uploading firmware via the account is fine, but no actual control function is workable as real operation needs a verification and subject to authorization from inverter manufacturers. Thirdly, the account is not related to any account of other customers and won't affect their PV plans. And last, the account is now in safe condition. I hope that means they implemented two factor authentication. Therefore, we believe that there's no effect to TV plant operation in the system. In case you want to go public with the case, please do keep above facts in your statement as we've done. And we expect not to mention the product name in disclosure too late and won't expect any exaggeration before clarification. I hope I didn't. Personally, I think we can create long term cooperation with your organization to offer services to our customer in much more secure way. I hope so too. So we're really grateful that they are reaching out, that we are in contact now. We want to thank the China cert for help brokering this contact because obviously good with everything coming from China. That's really important that we have good international cooperation on this. And with that, any questions? Thank you, Frank. We have about 10 minutes for Q&A. So you don't have to walk all the way to the tent of the IFD, which is over here. So if you have any questions beyond this talk, you can always find these guys over here. They have lots and lots and lots of war stories going on. So half of them are not disclosable at all, so it's a bit of a shame. Any questions? Nobody? Well, we have one question. Have you looked at the hardware architecture of the inverter and do you think it would be possible for the output AC voltage to shift the phase alignment relative to the grid? So according to the statement of the vendor, the firmware somehow can't magically control this. Have we taken... People kind of get pissed if you take their inverters because they have them for a reason. Anybody who has a solar man inverter, well, they changed the account, so we don't have the control now. So no, we didn't. This wasn't the hardware investigation for us. This was just a software investigation and a SaaS platform where we found the password from. So we haven't. We're really curious, though. So if somebody is willing to sacrifice their solar man controller, and I guess the likelihood just increased a little bit, please come by the village. Thank you very much. You're a big fan of Project Aurora. Okay. Thank you very much. Please give Frank a big applause and thank you for your...