 Hello everyone, I am Chris Samuel and today I will be representing my capstone, which is called Automotive HADSAP Dispay. So we'll have the following content, I'll provide a brief introduction, some background information, and then we'll dive into the technical description. I'll talk about the challenges, accomplishments, and we'll conclude the presentation. So first of all, what is a HADSAP Dispay? I believe many of you are familiar with this concept. It's basically a transparent display that presents data to the user with specific controllers and other equipment. It has a wide range of views, it can be used in the automotive industry, in the military, as you can see in the airplanes, fighter jets, and of course it's mostly used in the community, in the people that are often using them the most. Basically, this device is used for providing, illustrating, displaying the most important, essential data and information to the user. And the most important part of the device is that it prevents the loss of focus consciousness and it makes the user concentrate on their task, their role, and etc. So before I move on, I'd like to mention that what was the motivation for doing this project specifically? I'd like to say that it's very interesting to me, the automotive industry, and this technology I could see emerging very quickly. But besides that, it has also been shown that very long time ago, there were some experiments, some designs that had this exact idea in mind. And as time flies, I personally see that a lot of industries, a lot of car brands actually start developing this technology, which is also proving to be very effective. So let's move on. I would like to say that, first of all, let's see how does the heads up this day H&D system work. As you can see, there are a couple of key components to the system. First of all, it's the projection system. Here's an example of that. Then we have the combiner or the so-called plexiglass oven. Here is an example. Then we have the sensors and data sources, such as the sensors that collect information from the environment, like used for brightness adjustment, etc. Then we have the main control unit, which is also displayed here. And the optics adjustments, these are used for the user, for the easier access and usability. And finally, the user interface, which not most of them have, but also are used in some cases. And lastly, the connectivity, which is often wireless, but sometimes there are cases when wired connection is utilized. So let's move on to some technical information. I would like to say that the project is mainly constituted of hardware electronics, embedded programming, and some CAD modeling. So to begin with, I would like to say that these are the main processors. What I'll talk about later is the STM32 that has been utilized. And these are some pictures you can see of the codes, of the embedded folders files, and the configuration file, as well as you can see here. A specific software that I've used, I'll talk about later. Then you can see the clock configuration. It's also part of the Cubemax software and so on. So let's move on to the next part, which is the diagnostic, the connectivity part, as I mentioned previously. This is one of the most important parts since it's responsible for the data request and data providing, since there are only two ways of getting information from the vehicle. And it's one of them is just directly connecting to the main computer of the vehicle and requesting information which the computer has to provide in a specific manner. I'll talk about that later. So for this case, I have used the OBD2 scanner tool. It's a diagnostic tool, mainly utilized for finding the error sources, the damaged parts of the vehicle when it's diagnosed. Here is the scanner, the pinout that you can see here. The pin is quite small, but it's very difficult. It has the protocol and the pinout here, but this is not the part that I dialed too deep because it's standard and it feeds most of the vehicles. So the last picture that you can see here is the CANBAS request information. I have put this image here because I believe this is one of the most difficult parts of the project as well. Basically, this is approximately how the car computer sends the information that you request for. As you can see, the fuel can level, the engine equivalent, the speed, the RPM and the insect rights. We could say it's a JSON file that contains the most important information. And the way you access it is by their, let's say, name, their value. You access their name and they provide its value accordingly. Also, the middle picture here is one of the software mobile applications that are used for the testing of checking if the connection is secure between the car and the processor and if the data is being transferred or not. So, as I mentioned previously, the process consists of this main key components, the hardware electronics embedded and so on. Here are some brief pictures. So next key point is the CAD modeling, 3D modeling, which is also essential because initially I had in mind the fact that it needs to be compact and user friendly. But we'll see if I've achieved this goal or not. But this is what some designs look like. As you can see, the display attachment, the screen holders, the component housings and the LCD display holder. For the hardware, we don't have too much, but the key main key components here are the STM32 controller, the ST756 by display, the HC05 Bluetooth module, the ELM327 scanner and also a wired scanner of the same protocol. One important thing I want to mention here is the Bluetooth module. Initially, the project had to be completely wireless. This means that the Bluetooth module had to take the entire task of transferring and receiving data from the car and had to provide it to the controller to process it. However, since these modules often tend to be broken and they are not functioning very well and besides that, the rate is not satisfactory for this application as the sequence that the data is being transferred and received for this application has to be very fast since if you're driving and your speed is, let's say, less than you are currently going then it's not working, the system is not sufficient and it's not satisfactory for the user. So you mean the 327 isn't fast enough to send the information? No, the Bluetooth module is not fast enough, but the wired connection is immediately, it works very quickly. So that is the reason why I shifted into the wired connection mode. Essentially it's the same head unit as you can see here except it's wired and not wireless. So next up we'll talk about the electronics which is also not quite much here. The STM, the LCD connection pinout here, the Bluetooth I also included but then it's removed and also I used a couple of ST-Link debugger tools, USB header tools for pushing the codes into the STM controller and also debugging. Besides this, there was also an additional USB link which was used to give power to the STM only and to be able to communicate with the PC or when the code is being developed. I'll talk about that later when I show the design. So here's the design, apparently. And as you can see the partly disassembled part of the project here consists of mainly of the Plexiglas holder that goes into the holder, the breadboard which secures the STM controller, the LCD in its corresponding case and the entire housing for the components which is located on the right upside. This is also for storing all these components in one place. And this is the assembled product for the final product. Again, as I said, the Plexiglas display with its holder which is also adjustable for the user interface for easier use and accessibility. The screen holder, the LCD housing compartment, this is the entire upper case which fits most of the electronics and hardware. As I mentioned, there are two USBs. This is for the PSS-ST link which is used for pushing the code into the STM and this is used for the, let's say, supplementary supply. However, later I shifted to a different power supply which is a 4.5 volt battery and I used just in case if the system goes wrong and I have no power, I can at least provide it with some sufficient power so it can function. But regardless, the main power source for the device is going to be via the OBD scanner which connects directly to the computer of the car. Yeah, as I mentioned, here are some top views and front views of the system. Also, I can display it here. So as I mentioned, one of the key things that I was going for is the user friendliness and accessibility and this is important because I believe user has to have the access to adjust and have the optics as it wishes. As I mentioned, the size, the compactness I was going for, it's not compact but at least it's doing what it's supposed to do. So yeah, I'll put this here. Again, more photos and pictures as I mentioned, the OBD scanner. And also one thing that I took into consideration, however, I had a different design in mind, is the fact that it needs to fit as many vehicles as possible. What I mean by that is the design, the modeling has to be universal so one can just plug in and attach it to the dashboard of the vehicle and operate it. However, this meant that I had to have a universal resin type of material that could be bent and adjustable so it could take the form shape of the dashboard. But since this was not achieved, I just came up with a more simple solution which is just attaching a rubberized sticker which allows for a fairly good connection bonding and it can be used for every dashboard. You can just put it there and go. So the next part is the embedded programming, as I mentioned. Since this was the most difficult part of the project, since the libraries that caused most of the software part had to be selected specifically, accordingly, and from various sources. Since there are many multiple libraries for, let's say, the disk space for the STM as well. However, for the specific display that I chose, the ST7565, the libraries for this display are very rare and they are often unfunctional so it's almost impossible to get your desired result here. Nevertheless, I was still able to detect some libraries with some working codes and I, of course, modified them to meet my needs. So since the codes are very big, the libraries used are a lot. I included some of them here. Each directory contains numerous files. Codes include files, C files, etc. Here are just some of the brief pictures that I put the patch here so you can see for the fonts, for the source code, for the OBD scanner and for some graphics for displaying the numbers, etc. So finally, we move on to the testing part. As I mentioned previously, the STLink Connect programmer was used with the STM32 microcontroller and some Bluetooth devices were initially used to detect if a car can see the module, the processor or not, if there is an established connection or not. So as part of the testing, there are also errors and malfunctions and one of them was after plugging the OBD scanner into the car most of the time some errors came up in the dashboard. For example, the check engine went on or some other levels just went crazy and this goes to show that the program and everything has to be so universal that every car can feed this. However, later I did some research and found out that the OBD scanner is available for models above some year. I don't remember the year, but the least models do not work as well. So that was the solution to the errors. And the utilized software tools and other applications, as you can see, here are the solid works for the Kimberley design, the STM32Q IDE for the code integration and development and finally the QBemex for the controller, the cheap configuration, the pinout configuration. And the printing software tools, Zortrax and some supplementary embedded development tools such as KL which I didn't end up using because of lack of knowledge in this software. So some live testing here, this is an early protocol here. As you can see with no casing and nothing but the pure screen. So yeah, here you can also have a video. This is just to show that this page, when it's shifted around, user can still have access to see the figures correctly since while driving you have to have this knowledge and if your head is tilted or the car is tilted you should always have this view point in front of you. Some more testing here. As you can see, these are some initial experiments with no casing, with no modeling, but the main feature is working and the numbers are, their speed is visible. This is the first try of trying to film and see the speed. When is the time it doesn't work? It works, but since the display brightness is not very bright and the Plexiglas that is installed here, it's not made for, mostly it's made for the night time but for this case it's not very well visible. As I said, the Plexiglas has the property of absorbing light and allowing the user to see the screen more efficiently. Do you think there's any parameter to display or do you just have to? This is the easiest one to choose. Engine temperature? No, I'll get to it in a second, but to answer short, it's mostly PID controllers in the car and every time you need to request the function from the car to receive the data it takes a lot of time and it has protocols there but for the sake of just using an easier task and just showing the main goal here I just chose to show the other speed. For the challenges that I faced during the project most of the most difficult part was the work with STM to redo microcontroller since it has its own designated software and besides software there is another software that allows the user to do the configurations of the pinouts and this is necessary for the system to operate. Also, as mentioned previously, the LCD display had numerous libraries but most of them were not functional and it was very hard to find a working one and the Bluetooth data transfer, as I said, I moved, I shifted to a wired one and the Canvas system, which also took some time to get used to it and understand how to request and receive data however, the major aid here is that most libraries that work with the Canvas are already integrated with the functions, the necessary PID controllers and they are registers, if I'm not mistaken, that you need to request to receive the corresponding data and lastly, as I said, filming the prototype in action was quite hard with one hand and lastly, the acknowledgments here, I want to mention, firstly, my parents the AUA Engineering faculty and staff, AUA Engineering junior and senior students and all the people that you can see here Dr. Hiraj Magalian, who was the most essential, who had the most important part in the project Ms. Athenik and Vanesian and my other teammates, my colleagues, as you can see here also they all had a huge contribution into my work and thank you If you had a stronger LCD, could you display it on the windshield? Let me tell the difference when the car manufacturers include these heads up display into the dashboard the windscreen is also made differently the material, the characteristics are different because they absorb most of the light and when it's projected and often if it's not there, people just take some tape that is also absorbent and they just tape it there so the light can be absorbed and focused there and not be dispersed I actually like the presentation just that it's comprehensive it gives almost a complete overview of what it is but I want us to take a step back and have a wider look at this so in R&D projects it's almost always the case that there are different groups working on the same project, achieving the same goals as you also presented in your report in detail but each one of these groups, each one of these projects is focused on achieving at least one main goal having something that they believe they can do better than the other groups or doing something differently so I want to know in your opinion what's the single most important point that you think you're doing better or different than other groups it could be technical, it could be financial, the execution of the project, market application, it could be anything I believe probably the key aspect that differentiates this project is the cost because the main controllers, the main components are quite cheap the hardware only utilizes the STM controller and the wiring, the other OBD scanner tool they are also very accessible and they are not that expensive besides that I think the versatility, the universal point of view of the project is enormous because I have actually seen many such designs that are just registrationary and they do not allow the user to adjust, have a adjustability so they can fit into their custom design and their preference so probably I would say cost is an assembling also it's easy to assemble, just plug and go these are the key since you mentioned the cost factor in percentage what is speaking, how much of your project is open source like specifically the software aspect of it for the 3D printing files you designed this, you took them from somewhere else but again when it comes to the software and the embedded system aspects what was the percentage, half of it is open source or closed so the embedded design mentioned was the most difficult part since the libraries let me say that most libraries that work with the STM, the CANBUS and the LCD they are available publicly however adjusting them and fitting into your project is the most difficult one also the configuration of the chip, the Tube MX software is also a custom design because it's nowhere in the key tab or anywhere so I would say mostly 50-60% is open source in regards of the cost libraries but the rest of the configurations, the pinout design and etc are obviously on Mr. Makar but if it's required you can obviously share the cost and the references I would say it's very impressive work I was impressed by the research that has been done in the field and also the working prototype I have a question about have you measured the accuracy of the reading because I saw showing 8 km per hour on the dashboard and 12 km of head ups on the display and also the total latency of the system how quick it reads and scans at this place I haven't done the great testing of measuring the latency or they say the rate that they transferred because of the following reason and that is the OBD scanner it's not wireless design, it's wireless design provides I believe instant immediate response since it's wired and it travels with the sort of light and once you just accelerate it, it requests data however the thing with the latency that you noticed I believe it could be due to the delays in the embedded system, in the embedded coding by saying this I mean that while coding there have been many prototypes of testing debugging and I have put some delays like milliseconds into the code to actually see some result and then move on to the next part of the code so probably if I remove and make the code more optimal let's say remove all the unnecessary stuff I believe it could work more quickly but I have not taken any measurements about the rates I just didn't know how to do so and one small technical question so you've been working directly with OVC have you been using the CAN ILOs did you do the coupling like the couplers? couplers? because you've been using, you've been wiring the CANs to your STM board have you used any decoupling there? so this part I forgot to mention it's not wired to the STM since the end of the OBD scanner is I believe mini-USB or micro-USB I cannot remember and I could find a converter that has a female USB mini and the output is male micro-USB and that's how I can directly connect the STM's input source and provide power and the necessary information exactly from that port so you've been using it through the USB? yes