 Welcome to this episode of the structural engineering channel podcast, a podcast focused on helping structural engineering professionals stay up-to-date on technical trends in the field and to help them succeed in their careers and lives. I'm your co-host Matt Picardle, I'm a licensed engineer at DCI Engineers practicing on structural projects in California with an undergraduate degree from Cal Poly Pomona and a master's in structural engineering from UC San Diego. And I'm your co-host Alexis Clark. I work in Hilties North American headquarters as the product manager of our chemical anchoring portfolio in the U.S. and Canada. I'm a licensed professional engineer in Texas, received my bachelor's in civil engineering from UT Austin and I'm currently an MBA candidate at Auburn. In this episode we talk to Dr. Badri Hiryer, Vice President and Director of Artificial Intelligence at Thornton Thomas City about the applications of AI and machine learning in the AEC industry. In his current position, Dr. Hiryer leads to the core.ai research and development group focused on developing applications that leverage machine learning and AI to transform various workflows and processes within the AEC center. He is also the founder and CEO of T2D2.ai, a technology startup providing cloud-based asset management that uses computer vision to detect and map deterioration and damage in structures using drone or mobile camera feeds. Prior to establishing core.ai at Thornton Thomas City, Dr. Hiryer spent several years as a computational scientist in the company's applied sciences practice where he developed high performance computing software used by the Navy for computational fluid dynamics simulations. Dr. Hiryer has a master's degree from John Hopkins University and a PhD from Columbia University. Now let's jump into our conversation with Badri. Badri, welcome to the structural engineering channel podcast. Thanks so much for being here. Could you tell our listeners, you know, just a little bit about your career journey, how you got to where you're at and ultimately what you do at Thornton Thomas City? Yes, certainly. Thank you very much for having me. It's great to be here. So yeah, so right now I'm a VP and director of AI at Thornton Thomas City. So I've been in this position for about eight years now. But going back to the beginning, I moved to this country back in 2001. I came here as a fresh graduate student at Johns Hopkins University. I got my undergrad degree in civil engineering. And then I got my master's in 2003 and then worked for a couple of years, a few years in Chicago, Midwest area, and then moved to the Bay area. I was working in industry product development research and testing positions. One was Clark Detrick was a metal framing company and then Simpson Strong Tie, which many of your listeners might be familiar with. They're a big manufacturer of structural connectors and so on. So back in 2008, I decided to actually go back to grad school to pursue my PhD in computational mechanics because that was a strong interest of mine. I wanted to get back more deep into that subject. So I went to Columbia University. I spent three and a half years there and I graduated in 2012. I worked with Sandia Labs during that time, working on high performance computing software for tractor mechanics. And then after my graduation, I joined this company called Widelinger Associates. Widelinger merged with Thornton Thomas in 2015. So I've been with this combined firm for about eight years now. So I was an applied scientist or a computational scientist, you know, developing software for the US Navy on computational fluid dynamics and high performance computing and so on. But over the past three to four years, I've been working on machine learning and artificial intelligence applications. And so that led me to my current role as the director of AI. And some of the projects that I have led have turned into separate products. And then, you know, so in my current role, I work on leading R&D efforts, especially in the AI and ML area and bringing about these new technologies into applications in the AEC industry. So that is my full time job. My second full time job is actually as the founder and CEO of T2D2, which I will get to hopefully later during the show. I want to first mention my third full time job as the father of two sweet little kids. Exactly, very impressive. I know that's that's another full time job like you were saying. That's really interesting. And what's really fascinating to me, you know, just to get into the bulk of this is, you know, a lot of people, you know, the industry is always asking, hey, how do we, what's the future going to look like with AI and stuff? It's kind of cool that, you know, your effects, your directly effect, like shaping it. So that's what's really cool. And that's how we, we found you, you know, we found this article in AISC magazine and a really interesting article. I think what impressed me was you, so let me just read this correctly. So Thornton Thomasetti's core studio developed an application called Asterik, which is basically a web based software package that allows you to do concept level schematic design level designs on building structures. Like, so you're using like data analytics rather than engineering analysis. So it's basically AI, correct me if I'm wrong, but could you tell us something a little more about that? Because that just seems really interesting. And I know the industry is really interested in things like that too. Yeah, absolutely. I think, you know, there are definitely a lot of applications for AI. And I'll get to that. We can probably also talk about the definition of AI and the distinction between narrow AI and general AI. But coming back to your question about Asterisk, yes, it's a great tool. So, you know, just imagine that, you know, you're a young engineer and you're designing a building, and you come up with some member sizes, and you go to this very senior engineer, and you ask her, okay, you know, based on your experience, you think this member size seems appropriate. And she'll probably tell you that, okay, you know, she's worked on the, you know, engineering tall buildings for decades, maybe 20, 30 years, and she can say, oh, based on my experience, I think that member size looks good. Okay, for that particular profile, but maybe over here, you know, this member size looks too light, maybe you should go back and recheck your calculations and so on. So how is she able to do that, you know, just based on your experience? It's because she's seen so many buildings, she's designed so many beams and columns and base, she knows this from experience. Now, the idea behind Asterisk is that we wanted to encapsulate that same experiential knowledge in a software program. And so we wanted to train a system that has seen thousands of designs and has captured, okay, what are the features, what are the parameters that lead to certain design decisions and so on. With Asterisk, you can take a, what your input is basically, like a very simple geometry, a massing, you know, height profile and so on. Just with that input, it first goes through a geometry service which breaks down into, you know, different base and columns and so on. And that goes to the core learn piece, which is the central brain behind designing the structure. You know, when I say brain, it's actually just what has seen before. It's just a supervised learning system that has seen lots of different designs before. So you have a beam designer, a column designer, a bay designer and so on. So these are the various components within it. And just based on the parameters that are input, it is going to give out, okay, these are most likely the members for these profiles. So because this is an inference service, it happens very quickly. So that means that, you know, you can quickly come up with like a first order design, you know, it's not something that has gone through a stability check or a strength check or a serviceability check. But, you know, it's a first order design. You know, it's most likely these member sizes are appropriate for this particular design that can also lead to computations of okay, this is the total amount of tonnage for this building. This is the amount of embodied carbon, because that's also an important consideration these days with, you know, carbon neutrality being a strong focus in the coming few, you know, coming decades. So, you know, these are questions that you can answer at, you know, the snap of a finger instead of having to go through like a very detailed design that might take, you know, hours or weeks. So it's a great tool for early stage optioning, as we call it. You want to evaluate maybe three or four different designs. You can quickly come up with a massing and a design and you can say, okay, this one, the tonnage is so much, this is the concrete tonnage, this is the embodied carbon involved. You change the parameters, you change the design, and you quickly come up with a new estimate. And so you can quickly evaluate a bunch of different, you know, designs and compare them across different parameters of interest. So I think it's a great tool, machine learning engine that is running the design service. Yeah, so important too. And, you know, especially during the early phases where the clients are always, could we do this? Can we do this? Or what if we do this? How do we optimize this? And you don't always have to keep going back and be running, I don't know, like an eTabs model or whatever. I just wanted to ask too, because I think you made a good point. Can you just do like a quick rundown of the differences between, you know, AI and machine learning? I get confused between that too, like, what's the difference? Is there a difference? Can you just do a quick definition of that? Certainly. So I mean, just in terms of AI in general, so how do you differentiate between AI and automation? In many cases, you know, you can automate a lot of things using computer programming. You know, how is AI different if you're automating something? The only difference is that, let's say for example, you want to go from A to E. If you write down the real rules, you go from A to B. If B, then go to C. If not, go to D. And then go to E. So that is a procedural rule that you write down. And if you write it down, procedurally, then that's just, you know, normal automation. But if you just show a bunch of examples to a program that show the program how to go, these are the cases where you go from A to E. And the program learns all the rules, all the internal rules by itself, that I would say is artificial intelligence. And if, you know, machine learning, I think, can be used interchangeably with artificial intelligence in this context. If you have a machine that is learning all of the rules, all of the internal rules, you know, to achieve a certain objective, then it's artificial intelligence. But I would also like to make the distinction between artificial general intelligence and artificial narrow intelligence, or narrow AI and general AI. Because a lot of times I think these, especially with all this buzz about AI, people tend to conflict AI with, you know, general artificial intelligence about robots having some sort of consciousness and having their own agenda and so on. But that's very different from narrow AI. I mean, the actual technological advances that we've seen today and the various applications that we see in AI is all right now, at least in narrow AI. So if you take the narrow functions of, let's say, visual recognition or audio recognition, natural language understanding, data analytics, so these are narrow functions, which can be, you know, certainly optimized or automated through AI. And this is where I think we see a lot of applications. So general AI is probably a few years or maybe even a few decades away, but, you know, we already see a lot of potential for narrow AI in various applications. I love it. I think you gave us such a great definition. So thank you for breaking that down, Matt. Great question. Because I was like, man, you know, I learned about this a little bit, but it's always good to have a refresher. As soon as you said general AI and narrow AI, I understand what you're speaking about. If any of our audiences ever watched the TV show Person of Interest, that is general AI. That is an all-knowing machine being of sorts, right? Is that kind of what we would define general AI as, Padri? Yes, certainly. You see, I mean, I think most of the instances that you see in popular media and in various movies and shows, you know, you see basically a general AI that, you know, that does a lot of different functions that can do natural language understanding, visual recognition, and that can take decisions and so on that, you know, so that's still a few years ago, although a lot of companies, I believe many big tech companies are working towards that goal, but absolutely. Yeah, what we are dealing with is narrow AI. Yes, perfect. Yeah, we're not dealing with Watson here. We're working with something very focused and with a specific need, which brings me to my next question, which you set me up for so nicely, which is, what are some of those most important applications of AI in structural engineering? Yes, certainly. So I mentioned to you already, like, you know, within narrow AI, there are, you know, these various functions. So one is visual recognition, there is natural language understanding, there is audio recognition, data analytics, and so on. Within structural engineering, I think, you know, you take each of those applications, there are various, you know, that there are various, you know, potential applications in the structural engineering world. And in general, in the engineering and construction world, there are applications on the construction side and there are applications on the engineer's desk. You know, on the engineer's desk, you can imagine for visual recognition, there are applications like, you know, being able to, you know, analyze construction drawings, various shop drawings, identifying various features within those. There are any anomalies being able to quickly identify those anomalies, or, you know, being able to quantify trends across different, you know, across different quantities of interest, and so on. So that's, you know, that's just from a drawing perspective. And then from a natural language understanding perspective, I think you can have, you can imagine various systems that can help make the engineer's job a lot easier. You should be, you know, just imagine a chat box that you can, that you can query. You can ask a lot of these questions instead of having to go to a reference, you know, you can have a system that can understand the query and go to the specific database and find the answer and come back to you. So a lot of these everyday applications, you know, that has more than just, you know, consumer applications that has, that could be specifically trained to serve an engineer's needs. On the construction side, you have, of course, various applications, you know, you have applications for robotics, you have applications for visual understanding that can identify, okay, if the job, you know, on track, what is the current status? It can automatically recognize if, you know, if there's a truck that came in that contained, you know, the steel, the steel members that were going to be used. What is the level of progress of construction? If there are maybe safety conditions that are being violated and so on. So there's just tons of applications on the construction side, there are applications on the engineering side. And then beyond just engineering and construction, if you think about the maintenance and the lifetime of the building, you know, you have tons of applications over there as well. For example, T2D2, one of, you know, the tool that I mentioned before. So that involves visual recognition of facade damage, for example. So over the lifetime of a building, you want to identify the deterioration or damage conditions on a facade, on a roof, or a structural member. You can do that visual recognition using a tool like T2D2. So, you know, just over the lifetime of a building, right, from inception all the way to, you know, to the end of life of building, I think there are just tons of applications for AI. And I didn't even mention data analytics, which has, you know, applications across all of the whole spectrum of engineering. Absolutely. Absolutely. So you've really given me a lot to think about here. And you did such a great job of breaking down all of these different facets that can affect not only, you know, conceptual design from actually, you know, cranking out calculations for beam sizing and all of these different small executions that we already do, as well as the impact on construction in the long run. I guess I have a couple of questions here. So you used this term earlier called optioneering, which I'm a huge fan of. I love that. And we kind of get this buzz in the engineering, structural engineering industry right now, about how do we put power back in the SES hands? How do we differentiate ourselves? How do we increase scope? How do we demand higher fees? And how do we become the prime on the project? And this seems like such a great opportunity for us to be able to take power back and to demonstrate our engineering expertise using this tool to show ourselves as a greater partner in conceptual phase to the client, to the eventual owner. Do you agree with that? Yeah, I certainly agree with that. I think, you know, that this can be... So I, you know, if you think about AI as, you know, as just transforming a certain section of the current process, that's probably a very limited view. I think the use of tools like this, the use of technology can actually transform the entire process, you know, instead of an engineer being involved only after the, you know, the whole, you know, architecture and the design has been, you know, has been finalized, you can have the engineer come in at a much earlier stage and be involved in the optioning or the, you know, the conceptual design stages as well. So I think, yes, there are definitely, you know, various applications of AI that can transform the entire process. Perfect, perfect. And the flip side of that coin is instead of this, you know, big industry topic right now, I want to focus, I want to shift really quickly to a focus of something that impacts all of us, regardless of really what industry we're in, which is spent, you know, being as effective as we can, as productive as we can in the office, and then getting to spend the rest of our weekdays spending time on things that we'd like to do, which is being at home with our families or spending hobbies or, you know, being active. And that is you keep mentioning how so many of these processes outside of the conceptual phase when we're actually executing different calculations and design iterations can make us much more efficient and much more effective in our work. So I'm also seeing that there's an opportunity to use these kind of tools to derive greater profit for the actual engineering firm by being able to demand the same amount of scope to do the same amount of work, but be able to get off maybe a couple of hours early, get seven hours of work done in five hours and then get home and do whatever you need to get done. Yeah, absolutely. So actually on that topic, you might there's actually there's a McKinsey study from maybe about five years ago. I see that in a lot of conferences and I'm sure you've seen that graphic too, that compares the level of productivity change across many industries over the past few years and decades. And there are lots of industries where they've seen like tremendous gains in productivity over the past few years. But construction was actually down at the very low end. The productivity was not very good over the past maybe what 50 years that the chart showed. I think with the new technologies that are coming into the picture now, I think that is going to change. I think you can have a significant productivity increase. These tools can help take away those hours doing mundane tasks and lots of repetitive tasks that can be automated and that can enhance the processes and give back some time to the engineers with the productivity gains. Badri, thanks for explaining that. I did want to get into, I know you mentioned this before, it's the T2D2, that's the Thornton-Thomasetti damage detector. That seemed really interesting just from the brief time that you mentioned it. Could you go more into that? That's something that I haven't heard of before, so it'd be really good to get into that. Yeah, definitely. Thank you. I'd love to get into that because as I mentioned, I'm also the founder and CEO of T2D2, which started off as like innovation project within Thornton-Thomasetti. But quickly because of the great potential, because of the great performance, turned into a product that could be spun off as its own startup. So now it's a startup that is part of the Twin Accelerator. So there's an internal TT accelerator that spun it off and right now it's a separate company. And just this morning, we had some great news because a DOB, the New York City Department of Buildings, had an innovation contest called Hack the Building Code Innovation Contest. So they were soliciting various new ideas to transform the building codes, the various prescriptive regulations that are in the building codes. How do we modernize building codes to use the latest and greatest technology? So T2D2 was one of the entries. It was among the finalists and then it was one of the winners. So we're very happy to announce that we're one of the winners of this novel contest. What T2D2 is, as the name mentioned, apart from an obvious nod to George Lucas and Star Wars, T2D2 stands for Thornton-Thomasetti Damage Detector. It uses computer vision to automatically detect deterioration and damage conditions in facade and building envelopes, facade structures and so on. And how do we do that? So because TT has been involved in renewal and forensics practices for many years, for decades actually. So we've been inspecting various buildings, bridges, tunnels and all of these structures and we have a large database of images that we've collected of various types of deterioration. We've used this vast data set. We've annotated them and we've trained advanced deep learning models and these are the same models that are used in image recognition, that could be for facial recognition, that could be for medical diagnosis. You should see various applications of image recognition, object detection and computer vision these days. So we use some of the same advanced state-of-the-art computer vision models. We've trained it on our large data sets that have been annotated for this purpose and we have a series of modules that does that task of identifying damage in structures using these models. And we've built a portal where you can actually visualize those results and we present sort of a digital twin to the client, to the end client that shows their asset and where all the conditions have been found and how they map to that structure. Especially with the use of drones, this can be if there's a really great technology. You can have a drone fly around a structure in a matter of a few hours instead of having to scaffold it in the building or have gondolas that are dropping down. It's a lot of expenses, a lot of time and a lot of effort. If you have a drone that captures all of the images, then we can quickly process them, identify all of the detections, have an engineer review those detections and present to the client like an overview of the state of their facade in a matter of a few hours. So that's I think a revolutionary technology. It's a great use of computer vision in our industry to manage the process of facade inspection. And that's not just for buildings that have applications in bridges, tunnels, nuclear reactors and endless opportunities there. Yeah, it sounds like definitely something of the future where you have a drone or kind of like those video games where it has those you're flying across and that it'll detect damage here, damage there. So that's pretty much what it's doing, right? You take a picture and it'll at least tell you based on all the data from all the subsets that you've given it before. It's like, oh, I've seen this type of damage before. This looks like it's corroding. And that's what the T2D2 will tell you will basically tell you analysis of what damage that it sees or doesn't see. Yeah, instead of yeah, instead of an engineer going through each and every picture, it'll already tell you once you take it. Yeah, it's crazy. That's correct. So it actually, so there are series of modules, like I said, so first it'll identify, okay, I'm looking at a brick masonry building. No, I'm looking at a stucco. I'm looking at concrete. And then depending on what material substrate it is, it's going to identify if it's a concrete building, it's, you know, you're seeing exposed rebar and corrosion, you're seeing spalling and so on. For a brick masonry building, it's going to identify these are the types of damage open mortar joints and cracked mortar joints and so on. So yeah, we have built like a sophisticated pipeline that can identify damage conditions for different types of materials. This could be really useful. We had a conversation, she's like a few episodes back at this point about actually it was our still bridge episode, that's what it was. We were talking about bridge inspection specifically. And that, you know, there is actually a huge talent drought right now in the inspection community because we have a large portion of our of our existing inspectors who are more tenured, we'll say, rather than than older, but they've been with us for a long time and they're within retirement age. And it's expected that some figure like 25 to 33% of the inspection community will retire within the next five to 10 years that there isn't a talent pool to backfill them effectively. And using this kind of intelligence in these kind of systems could could help offset the need for a human to be there. I guess I'm curious, what would the amount of capital take in order to substitute a human for a structure for to inspect and watch a structure you know, throughout its life cycle so that we can watch the damage that it may be going through? I think there are going to be tremendous cost savings, you know, if you replace an automated system like T2D2 compared to like a manual human based inspection, just in terms of the amount of time involved, you could say for setting up the scaffolding and the amount of time it would take to just visually observe all of the conditions on a structure on a typical building, it might take probably two to three days if you're allowed to fly drones and that can be done in the matter of two hours or less. And then if you had to process each of those images for a human that would take quite a bit of time and of course if you're just going through a bunch of images, there's also a lot of room for error, but our computer algorithms, our computer vision models, they never get tired and they never sleep. So they can process each of these images with the same amount of focus, you know, in a matter of seconds. So in terms of efficiency, I think it's going to be a very dramatic increase. Of course, is the performance of the model the same as a human inspector? Not yet. I think it's going to take some time. So right now we are having trained engineers and experts review the detections and make sure it marks the false policies and mark the false negatives. And I think eventually the models can get so good that the amount of human oversight needed would be much less. And at that point, I think it's going to make the inspector's job quite a bit easier. Yeah, instead of having to drop down from a scaffold or drop down from a rope axis, have a digital view of a building showing, okay, these are the areas that have potential damage. And maybe then it's, you know, I'm not saying that this will completely replace having to, you know, have a closer look and touch and feel the damage, you know, but you can quickly identify, okay, these are only the spots that I may need to go and take a closer look and probably feel, you know, touch the brick and see if it's, if it's okay or if there's a piece of facade that may be, that may be loose or something that's underneath that's not, you know, visual, it's not, that's not identifiable from the surface. So, you know, you can quickly identify focus areas where you can, you can direct resources instead of having to, you know, just inspect like an entire structure. That's just a very inefficient way of inspection. Absolutely. And I think you, your description right there just so very nicely spelled out all of the different safety aspects that we have, and we talked about productivity, we've talked about business opportunities, but from the safety aspect and keeping our inspectors or structural engineers or forensic engineers safe on the job site, I mean, the value is huge. And I think you also made a great point that this isn't necessarily always that upfront caught capital. We have to think about the longevity of the structure, the lifetime of, you know, the amount of the scope of inspection or whatever that looks like. So I guess my question to you is, as a structural engineer, as if someone, if you could speak directly to our audience really quick and maybe help them understand how can we start to talk about this in our work today, how do we start to implement this in the projects that we're working on? If I'm a really excited listener right now and I'm like, man, I want to get my hands on this. I've got the perfect project for this kind of technology. What's the first step? How do they get to include that in the project scope? Do they bring it to the owner? Are there some benefits that they should, you know, be able to explain really eloquently? Give us a roadmap. Certainly. I mean, you could, for T2D2, I would, you know, encourage any of the interest listeners to just visit our website. It's T2D2.ai. There's a good description about our technology, about our service and, you know, what are the various features we have in our product. And of course, you know, we can, we were happy to answer any further questions, if there are technical point questions about how specifically it would apply to a certain building condition if it's, you know, we're certainly available to answer those questions. I would definitely encourage them to bring it up with the building owner, if this is a technology that I think that if this technology would be useful for that site. And, you know, we have plenty of resources on our website and we're available to answer any questions, you know, to identify for any specific project condition, if it is applicable or not. And, Badri, I had one more question. This is my last question. So with, you know, all this technology is great, right? And it's really interesting to see, but then now if you're thinking of it from a structural engineer or an engineer, like, is this thing going to replace my job? So do you have any, I guess, how do you see the future going in terms of for maybe roles shifting instead of maybe engineers doing all of the repetitive tasks? Maybe their role can be more into maybe checking the results or how do you see this going in the future and in terms of people that are afraid of like losing their jobs to AI in general? Yes, that's a great question. And I think you mentioned it, you know, in your question itself, I think it's not mainly about losing jobs. It's mainly about how the jobs are going to change. So right now, if they're engineers who are, you know, doing a lot of repetitive tasks to get to something that they can answer quickly, I think they're going to find this to be very useful, right? So right now, you should think of AI as like a calculator or like a toolkit that's in your pocket that can help you answer a lot of questions that can help you as an engineer do your work much more efficiently. And I think eventually, of course, the role might shift and it's not that with AI, with the introduction of AI, you're going to see a lot of disruptive changes right away. It's going to start happening very incrementally. So, you know, if people are open-minded, if they're welcome to accepting new technologies and new ways of doing their things, I think, you know, they're going to find this to be a very useful trend and a very useful development in the industry. And over time, of course, you never know with them, you know, 20 years down the line, you can't say, you know, what's going to happen. But over at least in the next five years or the next 10 years, I think there's going to be lots of benefits to structural engineers to make their jobs a lot more easier. Awesome. So, Badri, leave us with the final note. What's happening in the future? What new projects are you working on? What's exciting that we should be aware of? Sure. You know, I think this is a great time. I mean, we're working on some fantastic new technologies. At Core AI, as the director of Core AI, I mean, we're working on a few fascinating things, including natural language processing, data analytics, computer division, and so on. You know, recently, we had an internal hackathon. We have those every once in a while. Recently, at one of the hackathons, we had, you know, prototyped a machine learning hub or a dashboard where we can host AI models that have been trained under the production ready that can serve the entire AAC industry for various applications. And this could be like a central warehouse for trained and production ready AI machine learning models. If there are application developers, if there are users of machine learning who do not know how to, for example, build a model, they can come to this marketplace, to this hub, and just connect their application to the model. So, this was an idea that we had internally brainstormed recently at the hackathon. We built a small prototype and we're working towards that. So, that's one of the cool things that we're working on these days, besides many applications, besides, you know, T2D2, ASTRS, and the various other projects that we're developing at Core. Yeah, thanks so much for that, Badri. It's really interesting. That seems like a lot of projects. And like I said before, it's really cool to see that you're actually having a hand in, you know, developing the future and you're actually making this stuff a reality. So, really, thank you for sharing with us your insights and what you're working on. And I know I learned a lot, and I'm sure our listeners learned a lot. So, thanks again for being on the show. Thank you so much for having me. It was great to, you know, to talk to you. And I know that these are really exciting topics and I'm happy to talk about these. Thank you for the opportunity.