 Hello, everyone, and thank you for joining our on-demand webinar today. My name is Harrison Donahue, and I am a business development manager for the Qt Company. Today, we're going to be talking to you about the STM32 accelerating the HMI of things with Qt and Touch GFX. Let's get into it. To start, I want to talk about what exactly we mean when we say the HMI of things. So human machine interface, or HMI, is the technology that facilitates the control and display of information between human and machine. As the number of connected devices grows, so does the information collected by these devices. We need a way to display that data and interact with these machines, turn them off, turn them on, see their state, and automate their functions. Many years ago, we had to physically move from place to place to collect this data or control these devices manually. Now, through a centralized HMI, we can monitor and control many things in a more convenient, efficient, and safer way. As technology advances, so does the HMI. What used to be physical buttons and indicator lights move to segmented displays, color displays with consoles and keyboards, and then on to touch screens, and now entering the world of touchless gestures, VR, and voice. We've gone from the device to the cloud, and now have returned back to the edge. The transmission and computation of data is faster than ever. Along with this growth in data are the user expectations. With a smartphone in every hand, our users are expecting and pushing HMI in the direction of that smartphone look and feel. They want full color touch screens that provide a user interface consisting of modern design elements, and one that is feature rich and responsive. As technology updates, they want this experience to evolve over time with it. Everything is connected after all, so why not be updatable? So how can you as a developer, designer, or a team of both achieve these expectations while managing these three critical components you see here on the screen, cost, time to market, and scalability? And what exactly do we mean when we're talking about these three components? When we talk about cost, of course, we're talking about cost resources, and maybe the tools you use. It may be your actual team members, your human resources, the cost of them, the bill of materials, so the cost of your product. When we're talking about time, again, cost, time is money. Every time, rather every minute you're working on your UI is time spent not working on other projects or components. When we talk about time to market, in some industries, this means the difference between having a successful business or none at all. And again, when we talk about scalability, this is the ability to have your application and look and feel of your application, not only the same across different levels of hardware, but also providers. Say you're looking to move a product line off various chipsets and consolidate onto a single family of ST micros or NTUs. It's easier to maintain, easier to work on the same platform. It's ideal to save time through the reuse of application components. Additionally, as you grow your offering in either direction, that is moving from NTUs to MPUs or the other direction, you don't want to have to redo your UI work from scratch or lose performance. You need the right tools for the job in order to accomplish this. Enter Qt and touch GFX. Before I get into the benefits of Qt and who we are, I want to quickly pass this off to Mike Hartman with product marketing at SD Microelectronics. Hello, thanks for having me and thanks for listening. As an introduction, I'm Mike Hartman, product marketing for microcontrollers and microprocessors in ST's America's region. As technology evolves, a critical design challenge that must always be addressed is how humans interface with that technology. At ST, we call this the HMI of things, HMI standing for human machine interface. Graphics displays are all around us. We see them every day and there's probably one sitting next to you right now. As embedded intelligence continues to grow, there are smarter and connected devices everywhere, addressing all markets from consumer goods and appliances to wearables, medical and industrial. In the past, microcontrollers might have been limited to simple buttons for control and a segment LCD for display. Now we can offer feature-rich touch interfaces with color LCDs and animations with smartphone-like control. The STM32 product portfolio and ecosystem possesses the technology needed for today's advanced graphics applications. Thanks for listening to the ST portion of today's presentation. I'll hand it back over for the remainder of the session. Alright, thank you, Mike. So coming back to this, I want to talk about exactly what Qt is. Qt is a cross-platform framework with libraries and tools for designing, developing and deploying software. We have a cross-platform IDE with a large variety of tools and extensions to aid in your productivity. It includes visual 2D and 3D UI editors with ready-made components and third-party graphical design tools. As well as a complete set of libraries for UI development, backend application logic, and modules for connectivity. Our IDE is called Qt Creator. It's a full-featured IDE that is cross-platform and includes integrated version control. It has an integrated designer with drag-and-drop UI components for a very what-you-see-is-what-you-get design interface. You can build and run for all of your supported targets as well as automate many of the processes from project creation to deploying to your final hardware. Additionally, we have our QML language. QML is a declarative, high-level markup language that's easy to read with a JSON-like syntax. You can define your UI in easy to reuse dynamically-connectable components. And alongside that, we have the Qt Quick Library, which consists of many of the standard QML types, including customizable UI components, layouts, and application connectors. And there's a compiler included with that to bring your QML code down to bytecode for the highest level of performance. Additionally, for your design team, we have the Design Studio. Design Studio separates your design from the target-specific implementation. You can build out your UI without worrying about the backend and all of the other components specific to your hardware. You can reuse components and customize them from across different projects or different platforms, and we provide a really great feature called the Animation Timeline, which allows you to seamlessly create animation on properties around your UI components such as color changes, positioning, scale, and rotation while working with keyframes. It's similar to many other components' design tools that you may be familiar with, like After Effects. Additionally, we have desktop simulation and live preview so you can see your design without needing to build the entire application. Additionally, we offer third-party tools to make your design process a little bit more seamless. This includes our Photoshop Bridge and Sketch Plugin. You can export designs from their native environment direct from your designer's desk to meet their vision. You can use artboards and layers to group and translate directly to QML components. Components import is a collection of QML standard types, and child components come in as flattened PNGs, which can be very efficient for the lower-level hardware. This all helps to aid in closing the gap between your designer and your developer. You can reduce the number of iterations, test in real-time, and it facilitates rapid prototyping. Next, I want to get into best practices and optimizing your design. Let's start by talking about the typical workflow and pitfalls. I think we're all familiar with having these segmented development cycles between our design team and our development team. The typical designer workflow includes early sketches, an interactive design, and building out the visuals through tools like Adobe's Creative Suite Illustrator and Photoshop, or possibly Sketch. The specs are updated slowly over time as they pull in feedback, and then the collaboration begins. This is really where the gap begins to grow, and the process becomes pretty inefficient in some cases. The developer takes the UI according to the specs. They implement the back-end logic, deploy to the device, and begin to test. As they start to collaborate and exchange the work between the development team and the designers, oftentimes there's a significant amount of back and forth. Many iterations, small tweaks and changes, which can be difficult to turn around quickly, especially if you're building a large application. This is where Qt's tools really come into shine. As far as optimization goes, you can start with your designer. Using the Photoshop and Sketch third-party tools, bridges, and plugins, you can modularize your design through the use of artboards. You can export assets appropriately, either as components or children, which will allow you to bring in reusable chunks or fully flattened images for efficiency. If you provide QML-specific types during the export process of your assets, you'll suddenly have access to all of those native properties of the QML component when you bring it into Design Studio. Additionally, we recommend using the QML IDs and standardizing on a clear naming convention so that when you do get down to the code, you know what you're working with, and it's easy to move back and forth between the form editor and text editor. We recommend applying your gradients and clipping effects early in your design files, especially if you're looking to scale between MCU and MTU as doing these types of effects on the device tend to be pretty expensive. When you do bring it over to the Design Studio and ultimately Qt Creator, you want to focus on making it responsive. Design Studio and the integrated editor and Qt Creator allow you to easily make use of anchors, which allow you to keep your UI components margined away from each other. When you do bring your design into Design Studio and ultimately Qt Creator, you want to make it responsive and make it dynamic. You can use Design Studio to build out your UI separate from your hardware and backend. This will help your design scale across different screen sizes, especially if you make use of our anchors. Anchors allow you to keep your elements positioned off of the edges of your screen, as well as with respect to other elements on your artboard. You can leverage property bindings to automatically update elements throughout your UI based on a single property change. And then, of course, make use of the timeline animations to easily manipulate properties over a set duration, whether it's color, size, rotation, or other properties. I'm now going to pass it off to Corey for a quick demonstration. Thanks a lot, Harrison. I appreciate it. So, yeah, today we're going to take a look at the Design Studio and how you can employ some of these techniques to create responsive user interfaces. What we're looking at here is a boilerplate entry point for a new application in Design Studio, a sort of Hello World application. And so I'm going to start by bringing in a couple of basic types that we support in QML called a rectangle. I brought a dial in. Let's also bring the rectangle in. A rectangle has a height, width, and some color values. So I'm going to go ahead and make this one a red and resize it down. We'll go ahead and keep the dial here, maybe size it down a bit. And I'm going to bring a couple more rectangles in. Position this one on the side. We'll give it a blue tint and one more here. This one's going to get nice, cute green. You can see the designer gives us cues like alignment positions. And I'm also going to bring in an image. And we're going to drop that here inside this second rectangle. Now, this image I want, I don't want it to be hanging outside of that rectangle. So what I want to use is the anchors that Harrison just mentioned to position it internally. I'm going to keep it to preserving the aspect ratio. But over here in our layout tab, we have a few options for anchors where you can position all four sides individually or all at once, as well as the center lines, horizontal and vertical center lines. In my case, I want this image to fill the green rectangle. So I'm going to choose this. And by default, it's set to fill the parent. And so as you can see, it positions it directly inside of that rectangle. Over here in my navigator, you can see the IDs for the QML objects that I've been creating. And I'm going to give it something a little more meaningful so that I can keep track of these. Rectangle one is now my rectangle blue. And rectangle is now my rectangle red. Dial is a fine name. And as you can see, what we've been doing as I drag and drop is actually creating QML code. As Harrison said, the QML code is a declarative. It's a JavaScript JSON notation where you have properties and values. And you instantiate objects just simply by declaring them in line. And you composite parent and child objects by declaring one inside of the other. So my image is a child of my rectangle, which is a child of my root rectangle object. So in my form editor now, we can see that these objects are all positioned. And I have a dial here, which gives me some options as far as values that I can assign. It can range and give it a from or a to value and a current value right now it's zero. So what I'd like to do is create a UI that responds to the value that I assign with this dial. So what I'm going to do is do what we call a property binding. And to do that, we can in design studio, we can click on this little nut icon next to any property. And it gives us a pop up menu where we can reset to a default value, or we can set a binding. Editor. What I get is I have a static value or I can set this value based on another value. In my case, I want to set my blue value based on a property out of rectangle green. So I'll choose rectangle green and I will choose the property X. So now the X value of my blue rectangle will be bound to the X value of my green rectangle. If I stop here, we wouldn't see anything because the blue rectangle would be hidden behind the green, as you can see. If I give it a little more height, we can see it peek out from underneath the green. But I want to edit that binding and give it a little more information. So let's go ahead and add rectangle green again. And add the width property. I mean, change that. I'm just going to type it in. So we'll add rectangle green as I start to type it a lot of complete dot width. And this way, the blue rectangle will be positioned to the side of the green rectangle. And if I change the position of the green rectangle, you can see that the position for the blue rectangle remains consistent. That's the image I dragged instead of the green. If I drag the green rectangle, the blue rectangle will move to follow it because it's bound to the width and the height, the width and the X position of the green. Now there's a more efficient way to do this within QML, and that's the anchoring system that I mentioned. So in addition to anchoring to a parent, you can anchor to sibling objects within the scene. So if I've selected my blue rectangle and I want to anchor the left side of it, I can click this icon and I can say my target is the rectangle green. And I can anchor it to the left, middle or right of rectangle green. In this case, the right is what I have. That's what I wanted. And this is a much more efficient design because it doesn't involve the need to do the addition to add the width to the X value. So anchoring is an efficient way to create a UI that is aware of other objects within it. I can also do things such as binding the value of the dial property of the dial object to one of our properties. So if I choose my green rectangle and say, I'm going to set my X value, and let's multiply this 166 X value by the value of the dial dial dot value and hit OK. So now when the value is zero, the green rectangle will be positioned at an X value of zero. And if I go change my dial value to let's say 0.5, now it's positioned halfway to the point where I want it. Here at this point I wanted to show the live preview feature. So if I click this play button in the top of the object, what I get is a live preview of the UI. And so as I make changes to the dial, you can see how it animates and follows the dial value up to the maximum that we specified. But beyond that, as I make changes to the actual code, so say I change this rectangle to be a little bit more blue. My live editor updates dynamically as I make those changes. Maybe now I like the blue, we'll stick with the blue. And so this is a great way to see your changes run and still be able to do your interactions in the UI, and it's a lot faster than having to compile and load every single time you want to see a change. Now, this is all great when we have a single resolution, but a responsive UI needs to be able to adapt multiple resolutions. So what I've done is I've created a binding in my QML code to a value that is defined in a singleton file, a constants file. And so if I change my width and height here, I'm going to comment out and uncomment. So I change it to a 720p size and we'll go back and take another look at this. All of a sudden my UI doesn't look so great as I wanted to scale this up to a higher resolution device. And so this is where I really engage with anchors again to try to create a UI that looks nice on different sizes of displays. So the first thing I want to do is put my dial in a consistent position. So for me what I'd like to do is anchor this horizontally within the parent. So there it's anchored horizontally. And then I want to position it vertically. So I want to anchor it near the bottom. And so I position it and then I click the anchors button and it automatically fills in my margin, which is great. It's right where I put it, but it's also not very responsive. If I change my resolution, 98 pixels of margin off the bottom might put my dial right on top or off the top of my view. So instead I really want this margin to be, I'm going to reset my horizontal margin. I really want this to be a binding as well. And so what I'll do is I'll bind it to my root component, bind it to the height. And I'm going to make it, let's say 15% of the height is how much margin I'll add. That's a little bit too much. Let's adjust that binding. Let's make it just 10%. Okay, so 72 pixels is 10% of my 720p height. Makes sense. And if I change the resolution again, it'll remain 10% off the bottom. Okay, so how about this blue rectangle? So it was anchored to my green. I want these to be positioned so that my green rectangle positions right in the center of my view. So if I can get a hold of it, get a hold of the right object here. I'm just going to reset my binding and do it by hand. There we go. Now I can do this. So if I position it here, and I can do that with anchors again so that it's horizontally centered. And I want my top to be anchored so my top will always be anchored to the parent with zero margin. Let's make the blue top anchored to the green rectangles vertical center. And we'll reset the margin. So now the top of the blue is anchored to the middle of the green rectangle. And then as I move my green rectangle here, I want my margin to be based on my slider value. So if I say this is the halfway point, let's anchor my, that's my top, I need to anchor my horizontal center margin to the dial dot value so that when it is one, it will position exactly at zero. And when it is a value of zero, it would position off to the side. So what I want to do is change my margin. And I want my value on my dial to go from, since these are pixels, I'm going to set it to a maximum of 500 pixels. So from zero to 500. And that will allow me to adjust the margin of my green rectangle from zero when the dial is down up to 500. We'll just take a look at that. And because I'm previewing the component, it doesn't have a default size, which is why I have to reset, resize it here. What you can already see is that as I dragged the size of this window out, my dial remains centered and it remained anchored near the bottom. My green rectangle remained centered in the middle and the blue is again still anchored vertically to the center of the green. And so this kind of gives you a taste of how you can use anchors to position your UI in a dynamic and responsive way as the resolutions may change. And if I go ahead and close my preview, and I go over to and change my size for my constants back to my 480 by 272. I can build and run this. And you can see my dial is still centered properly. Now my pixels are static at 500. So as I go up to 500, the green and the blue go off the screen. But that's what I designed in this use case. The final thing I wanted to highlight was the timeline animation because it's such a powerful tool that Qt has added into our design tooling. And it works across the board for MCUs and in MPUs. So if I add a new timeline in here, I'm able to specify the number of frames that I want to manipulate. I'm able to specify the duration for the animation and then change it to three seconds. I'm going to give it a ping pong effect to go back and forth. So you can use these to create transitions that are smooth. You can create custom easing curves. It's just a super powerful tool. So what I'm going to do is I'm going to animate the value of the dial instead of having to manipulate it by hand. So I'm going to click the auto key record button. And at the zero frame, I want the value of the dial to be zero. So I'll scroll down to zero. And you'll see it added the dial. And here at this key frame, the value is zero. Now, if I go to the end of the timeline, frame 1000, I'll change the value to, oh, let's just make it 250. We'll keep it on screen, maybe a little more than that. Right, 300. We can play with it, get it right where we want it. 400. Actually, 500 is close on here. 450. We'll say 450. It takes us out to the edge. All right. So at 1000 frames, we'll hit the 450 value. At zero frames, we're at the zero value. So let's run this and we'll see what happens. So because I still have my small resolution set in the constants, we are using the small window here. But you can see it automatically animated the dial value, which then updated the position of the green and blue rectangle. We can go ahead and change our constants back and see that run on the full screen. 720p. Over three seconds, we go out and back again. And so it's a super powerful tool to create UIs that are responsive, that are dynamic with animations that are smooth. And we found that with cute for MCUs, we can often get up to 60 frames per second on some of these more powerful SD micro MCUs. And certainly 60 frames is our target on all of the microprocessors that we support. So it's a fantastic way to build out a UI that's responsive that can scale well for microcontrollers up to microprocessors and beyond. And with that, I'll hand it back to Harrison for his last thoughts. Great demo Corey. Thank you very much for sharing the use of all those optimizations. You know, in real time for everybody to see. Thank you for everybody who participated and watched our webinar today and thank you again to Mike Hartman from SD micro electronics for his portion on on touch gfx and their offering. If you guys are looking for additional information on cute, you can go to our contact us page at cute.io slash contact dash us or send an email to info at cute.io. Thank you again.