 Hello and welcome to Flutter and Agile DeepDive. My name is Joel Venarska and I work for Toyota Connected North America. Quick introduction. Spent a fair amount of time at Qualcomm and the Windows BSP org. Did a lot of interesting and important things there. After that I started at InRex just right after OpenCard had been acquired and I was brought in to do back-end native code and optimizations and commercialization focus. A lot of interesting things happened there. Spent a lot of time on Chromium and just the general design of the browser and what needed to be adjusted. And that's when I started looking at embedded Flutter as a viable solution to replace JavaScript-based UI UX. Not just for automotive but just in general. Pure watercraft. Spent some time bare metal, electric drivetrain, CAN bus. That was a lot of fun. Multibattery packs, etc. And then moving back to Microsoft Surface. I was present for the launch of the Surface Pro X which runs at Qualcomm 8CX. And I'm the author creator of Metaflutter. And the author and owner of IVI Home Screen. Also known as Flutter Auto to the AGL community at Toyota. Embedded Flutter. The components. There's a development environment, a build environment, and a target environment. The Flutter Engine is the core component of the Flutter technology. It's written on C++17. There's a common source tree for all platforms that are supported. Desktop, Mac, Windows, Linux, mobile, Android, iOS, Web, Fuchsia, and Custom Embedder. The build environment is based on Google GN. It's a combination of Dart and Skiya. Impeler is coming into the scene. It will be replacing most of Skiya short of some text layout. And the Custom Embedder back-end supports OpenGL, Software-Based Rendering, Metal, and Vulkan. Some of the benefits. You get a premium user experience. A developer experience that's consistent amongst most targets. You have access to a commodity talent pool. You reduce your NRE, non-recoverable expense, and improve your time to market. The development environment. So, while evolving the development environment for AGL, the goal is to have the shortest path to run the Flutter app on AGL. Something easy to change between Flutter SDK versions, support unique configurations, archive-friendly, also known as LTS, and Ubuntu 20 support. A solution I came up with is Flutter Workspace Automation. Show a quick demo of that. So, I'm running a Python script that's present in the Metaflutter layer. And what this does is it goes out and grabs all the components needed to develop with Flutter in an embedded Linux environment. And it installs all the packages, runtime dependencies. Depending on what the configuration is set for, it'll pull a variety of repositories and locate them into the workspace container. So, once the container is present, then we can start using it. It takes approximately two minutes on a reasonable Internet connection. And so, it'll pull down a debug Flutter engine and better build. That'll pull down the Flutter SDK. And as I mentioned, the application repositories, if defined. And then it'll pull down AGL QMU image, depending on your architecture. And then it will set up all the custom devices that we have defined in the configuration file. The configuration file is the JSON file. And that JSON file can be tailored and customized to an organization's needs and uses. So, if I have a team that focuses on a, you know, coffee maker project, and it's a particular target and I only have certain repositories, well then you can create a standalone configuration file. So, when a new team member joins, they can just run the script and they have the exact same environment as you do, et cetera. Or internally at a large corporation, you may have different groups focusing on different areas of embedded Flutter so they each would want their own environment. So, that's what Flutter Workspace Automation here is about. So, as soon as this is done, we will, there we go, it's installing Flutter Auto desktop build. I'm running this demonstration on Ubuntu 22. This is supported on 20, 22, et cetera, Fedora and Mac M1. We recently merged support for that and that's based on QMU opposed to the virtualization support that was recently introduced in Ventura, which was just recently came out. So, right now it's pulling down the QMU image. So, so far I've opened a terminal and effectively cloned a repository and executed a script so far. Now, once you've set up your environment, or once you've created the workspace, there's no need to rerun the script unless you want to put your environment into a known state. So, there we are. We're complete. Now, if we need to source our setup environment file, as we can see, it prints out a list of custom devices that are available. The ID parameter here is the name that we will use to run an application. So, if we go in and look at this, we can do flutter run dash desktop auto. What will happen is it's going to build and deploy and we will see a Vulkan back-end enabled flutter auto run on the desktop and it's getting the elements for the gallery. There are a fair amount of elements and what it'll do is it'll install those into the pubcache, which is located within the flutter workspace. So, that's downloaded and it's built a bundle. And now it's launching the flutter auto and primary screen. So, let's see. That's that. So, you can also change the default size of it, et cetera. And then from here, I can launch the debugger if I so desire. It asks me every time I do this. So now, here's the debugger. So, we're looking at the flutter hierarchy, performance, et cetera. This is all functioning on the host. This is running Ubuntu native and all the features that you have that you get on the target, you get on the host and vice versa. So, let's close that out. Queue to quit. Okay, that's the quick demo of what that looks like. And now back to this guy. So, what does it do? It creates a flutter workspace, clones the flutter SDK, sets up a local flutter SDK config, sets up a local pubcache, fetches defined artifacts and installs runtime dependencies, clones defined repositories and creates a setup environment. What's in a workspace? Flutter SDK, a sandbox flutter SDK config, sandbox pubcache, platform setup, runtime binary, required dependencies custom, excuse me, device config, and then development repositories and OBS code launch.json. So, one install method from the AGL source tree. So, there's a configuration on meta flutter that's specific to the targets that I build on meta flutter and the configuration is specific to meta AGL flutter when working with targets there. So, it's locked to the flutter SDK version that AGL is locked to. So, meta flutter is typically going to be locked to a newer version than AGL is. This is an install method to get the tip of tree. The script options are listed as follows. So, using this script, you can effectively just pass in the flutter version and it will update your flutter version for you and pull down the debug flutter engine required to run and allows you to quickly swap versions. So, if you have a question like say your team is fixed at 331 and you're not sure about compatibility, someone can run the script, bump it up to 337 and see if everything still runs and works. That's pretty handy. There are some setup caveats with this script. So, Flutter IDE tooling does some things like file watching and they do automatic triggering and it can't interact with if you're updating a repository to a different commit or something like that then all of a sudden the IDE which has potentially a different flutter in its path will then attempt to do some things with it on the wrong version of flutter while you're trying to install one. So, the approach is you basically create your environment externals here tooling. It's handy to create a terminal window and visual code but I suggest not doing that. Multiple flutter's entry in a system path. So, the script will remove the first occurring instance from the path that resolves. It's possible to fake it out and have like I've seen a scenario where someone had three flutter instances in their path. They weren't, you know, and so that was a problem and so we had to go through and clean out all the instances and then that fixes the problem. So, your host machine is expected to have hardware hypervisor support. This is old actually. We check in about a week or so ago, hit dynamically checks for the SVM CPU support and sets that automatically for x86 versions for ARM64 on M1. It's enabled by default. So, debugging on flutter auto desktop. The demo I provided a little bit ago showed that working. So, I'm in a, so GDM stands for known display manager and by default you can, it'll run in X11. If you have Fedora by default it'll run a Wayland session. So, you have to manually select Wayland depending on which version or Ubuntu you are. In the login screen there's a little gear symbol. You click on that and it'll say Wayland and then, you know, depending on your hardware you may have to play around with some parameters to get your Wayland session functioning with your video drivers. Thinkpads with Nvidia hard discreet hardware are a lot of fun, but it works. You just have to do some searching on the internet. So, you log in to your Wayland session. That's sort of expected for people who are working on Wayland to understand how to set that up. So, login via GDM Wayland session. Open a terminal and type source flutter workspace. It's implied that set if it's not, you would now, you know, select the directory and then set up your environment. Then navigate to your favorite app, flutter run, and then device name desktop auto. So, the demo I did is gallery. And then debugging on the AGL QMU image is a little different because you do have a SSH connection on a given port. So, you open a terminal, you source your environment, you type QMU run, you wait until QMU image reaches the login prompt, and then you make sure you have connectivity to the target at local host and make sure that your known hosts are happy. So, if we look at that scenario here, all right, go into an application I want to run on my AGL QMU. And so, if I want to find out which devices are present, I can do flutter doctor-v and it will list out at the end the connected devices. So, using that second column there is the device ID. So, if I say flutter run device AGL QMU, it will then say we're still working on this part. Looking at debugging, you've seen Visual Studio Code. So, looking at a command window that has it set, the environment, we can simply open that up directly and take a look at that. So, on this one, it's created a VS Code launch file and notice the different entries here. Pretty much it takes everything in the config file and makes a target for each given repository. So, if each whichever repo listing has pubspec path set, which means it gives it a valid path to the pubspec YAML, then it will generate a target. So, if we go into this workspace, now notice I have all these options. So, say I want to look at flutter animated background on desktop auto and debug anyway. It's building, launching, and there we go. We see that, and then I have it popped up on the other monitor. So, let's move that over to clickering. It's my dual monitor. There we go. Now, let's maximize it. And then there's controls here. Specified and men speed, max radius, fill, count. You're really good crazy with it. And that's a, there's a variety of different options on this. So, I talked about the VS Code launch. How custom devices work. This question has come up a couple times. So, in, I go to console, and if I type flutter doctor, there's verbose and very verbose. When I type dash v, I got a list of all the devices. If I type dash vv, I got a whole lot of more stuff. It becomes very verbose. Notice there's some commands intermittent in there. So, it, what that is is some of the output of the ping. So, inherently in the tool chain when you have, so if I say flutter config at the bottom, there are a series of settings. Notice that custom devices, enable custom devices is true as is Linux desktop. So, those are the only ones enabled. And by default, when you run this flutter workspace, it's going to persist, right? Because there's a sandbox.config folder, and then I update the xdg home path unique to this environment. And so, these are independent of what happens in the user config. So, in the home folder.config, it gives it a sandbox home, and that's where these exist. So, if I go and open a new terminal, and then I, let's just say we look at lsconfig. Let's do tree config. There's a lot of things that are stored here by the flutter tooling, right? And everything is to do this. So, there's some of that. And the other thing is, the pubcache is also sandboxed, right? Meaning, it's within the container of our workspace. So, the config that has to do with what I type flutter config, it's interacting. See, I'm in the, I haven't set my environment. So, we'll go back to where the environment is set and look at our config. So, I have multiple items enabled. So, as long as custom devices is enabled, it'll attempt to query and ping them. So, the best place to take a look at the interaction with custom devices is with visual studio code. So, let's take a look at that. So, here we have a view of the flutter workspace, the root. I've opened the root up, and this flutter workspace config is what I use to create my workspace. And we notice, for instance, in the top one, desktop auto, this is a custom device that I interact with. There's a section on here called, with the key of custom device, right here. And it's ID desktop auto. So, what happens is, the script will insert these into the flutter configuration of the custom devices. So, let's take a look at that under .config, flutter, we have a file, custom devices.json. And this file is managed and maintained by the flutter tooling. So, when I run the script here, it interacts directly and modifies and updates custom devices external to the tooling. And it leaves it in a state which is usable. So, if, for instance, you have, this is the configuration, this overwrite existing key value, if that's set to false, that means if it queries the custom device file and doesn't find, and finds the same entry, it won't overwrite it. So, that's just to provide some flexibility. But normally, it will just overwrite what was there already. So, if there's a change in this setup and you re-run the script, it would overwrite the existing one in your flutter workspace config. So, in here, this expands out exact, there's some name value, some value substitutions that happen in the script. So, custom device, as is, it won't show up as is. For instance, this one has flutter workspace variable. This one up here has bundle folder. These all get string substituted prior to writing to the configuration file. And then this represents the post-expanded strings, right? So, it's, this is the literal version that config just provides some shortcuts from having to type things in all the time. So, what happens, the interaction with custom devices, when flutter doctor-v is run, or flutter doctor-vv, when we see the verbose output, we see the tooling execute the ping command. And then implicitly, it's waiting for the success-rejects value. And if it matches, then it will list the custom devices being available in the flutter doctor output. And also, if you don't specify a device and there's more than one, it'll prompt with a list of all the custom devices that are available. And so, the availability is determined by this ping and success-rejects response. So, if I was running, in this case, if I were running a X11 session, and I issued this command that I'm expecting type equals Wayland, then it would, it would fail, right? So, it wouldn't list it as being available. So, for instance, on M1 Mac, when you run this, it'll execute the ping, and then, you know, it'll obviously fail on the M1, and so it never shows up as being available, even though the device is defined in the config. So, it's fine to have a config if it doesn't respond and it just isn't listed. So, in the case of AGL QMU, this one will check and see if port 2222 is open. Just, that's the gating check, right? If port 2222 is not open, that would indicate that your QMU session is not running. Okay, and in another scenario, we have this where it'll issue a ping with the hostname, right? This is, so, the Vahi daemon is configured on the image, this device sitting on the network, and so it will broadcast, you know, its name, and it will give resolve to an IP address via that, multicast DNS, and then, that way, you don't have to put an IP address. And, you know, yes, it does a query, but it also makes it portable. So, in that case, ping, this response here is what determines if it's there. And then, the same thing is going to be for this device, Pi 02W Weston. It just pings with a different hostname, right? Same pattern. This one here is saying it gives a different response. So, that's how the ping sort of establishes the configuration when you type flutter doctor dash B, or when you type flutter run and say you have more than one of these custom devices available. So, that will prompt you to enter that value. So, once the ping and response is present, then what happens when you say you type flutter run dash D, and in this case, if you specify desktop dot dash auto, then there would be the series of callbacks that would happen. You have post build, install, uninstall, and run debug. And so, in what I do in install is I effectively create the bundle folder, uninstallable, and I basically cache it and then I push it over to the temp folder. And this is in a host scenario, right? So, it's all happening in the temp folder. Another approach that I may expand that to is use some sim links that point to the flutter assets folder, which would be a better scenario than your hot reload would work on host scenario. In the case of uninstall, it just removes the folder. And for run debug, I'm invoking flutter auto with two flags. One is a JSON flag configuration, which you'll notice. We take a look at that scrolling over in the workspace. Notice that's in the workspace and dot flutter auto default config. So, let's take a look at that default config and it's passing in parameters of the cursor theme and the view size. Now, the other thing it's doing is passing in the bundle folder, right? So, that's easy. In the case of AGL QMU, it's a little more tricky. There's a couple of things going on. So, one is it's expected that you have an SSH connection available to the target to begin with. So, there's no input handling, bash input handling available with the flutter tooling. So, you need to externally pre-connect to the device and ensure that it's connecting. So, in this case, I created a copy of the bundle folder off into a location. And then I talk to the QMU instance and tell it to delete the password for AGL driver. So, this allows me to, because AGL driver is the user in the case of the OS where you log in, you have root and then you have AGL driver. And if you want to invoke applications and have it show up with the compositor, you need to be as user AGL driver because there's, you know, the XDG runtime directory and the Wayland display are set up. And the AGL compositor is running under AGL driver most importantly. So, and then the uninstall that just is running a batch command as user driver. And it just removes from the temp folder where it gets installed. And of course, when you're debugging on a device like this, hot reload isn't necessarily an option, right? Because you have a cache file system unless that gets updated. So, in the case of install, again, using the same pattern, talking to port 2222 as AGL driver, makes the folder and then copies it over using SCP and then good to go. And in the case of run, so this dash t dash t, double entry, what it does is when the session is lost, the TCPIP connection slash SSH connection is lost, it'll actually terminate the process, the SSH process. So without that entry, if you have a connection and then for some reason something you shut down one side of the connection, you have an instance running on the target and then if you try to reconnect, you have a failure because something already connected. So this dash t dash t is there to ensure that it cleans up nicely. And then when you reconnect, you don't have that problem. So in this case, we invoke two things. We have, again, a JSON config and then the app name. And then we have to set the observatory host to any IP address that allows us to talk to it remotely. And then it sets the observatory to a fixed port of 123. And then in this case in AGL, we have a default config and that one's a little different than the other. So it's context specific. We got the background pane and then it's setting it to the default size, you know, the standard AGL size, which is 1920 by 1080, and it sets it to full screen. So that will do a full screen for us. And then back to the Flutter config. What else can we show here on this pattern on the remote ones is the same effectively. We're just not specifying a predetermined port, right? So in the case of QMU, which is right on the host, you know, the QMU session, you have to set up the firewall such that this will talk to it. So in the case of Flutter workspace config, the other element is on the QMU session. We have it broken down between ARM64, Darwin, Ubuntu, and Fedora. And effectively the primary ARG. And so we have ARGs, X86, ARGs, ARM64, QMU image, so image name, a kernel name, in case of ARM64. So on the suffix determines the variant, right? So they're treated independently. And the commands that show up in setup underscore ENV.sh are derived out of these key values here, right? So the combination would be, you know, we have the base QMU command and then the ARGs that are specific to X8664 and then, you know, QMU image. So between those and then you also have, depending on what it's running on, then get some different flavored flags too. So if you're running on the M1, you get the base command, the ARGs, and the kernel name, and then the QMU image, and then you get the additional Darwin flag set. So the resultant output in the case of M1 looks very different than on the X86, etc. So that's how I'm handling the variations on that because each platform has unique requirements around QMU. So in the case of the base arguments here, this is the firewall setup, right? So we specify we open up port 2222 and we also open up port 1234, which is set up for the observatory. So between these two ports allows from, you know, on local host to talk to the QMU image. And those values are effectively the same in the case of the M1 setup right here, right? 22221234. So the axis pattern is the same when you're running on the local host QMU session between targets. So I think that covers the, you know, how custom devices somewhat work. Right presently, the debug scenario is only supported, so profile and release are not supported. That's on the author of the feature and myself have discussed this a bit and we're going to do some work on this in the upcoming quarter around supporting additional modes here. So right now, you know, when we run and debug and talk to a device, whether it's on host or remotely, it's to talk to it for debugging scenarios and the setup for profiling is largely interacting between the Dart VM and the web browser, right? So those, that activity really doesn't necessarily require the debugging. You're not going to single step and, you know, profile. The profiling is about running it full bore and looking at trace data, so sort of different use cases. And in the release case, you know, there's no debugging information, so you don't need to transport. You just execute it and run it on, you know, the device. So, and that, so we, yeah, custom device is used to update and set the config flutter custom devices. And then there's that interaction between the QMU and that's about it. Like the string substitution shows up in there just to make it smaller and easier to read and prevent having to type things over and over again. And yeah, so that covers that. Kirkstone tools is the authoritative repository for this script. Everything that, and ultimately what you would want to do is keep up to date with this script here, not your config. The config may evolve depending on what features get added. So if like, Artifactory support and some other things get pulled in there, then it might, you might want to update it against one of the reference configs. And here's the Garrett for Automotive Linux. The metagl develop. This is where you would find information on how to use Flutter workspace automation with Agl images. That's instructions specific to Agl. And then this last one is the Flutter like generic knowledge of using custom and betters. So the labs side of a Flutter workspace create Agl Flutter application debug it using CLN and debug it using Visual Studio. I didn't run through that a bit. I think I'll leave this up to the to the viewers. This is could be homework. Best way to learn something is by going through the motions and doing it. So it's well documented. Okay. And this is where I talk about the Linux GTK and better. I get a lot of questions about this. Well, I think there's a lot of confusion because there's so many sort of variants of the Flutter and betters. So the Linux GTK and betters. Yes, it's a what Google, the Google Flutter team considers it a first party platform. So canonical is the primary development partner for the GTK and better. The Flutter SDK only supports host only builds. So there's no cross compilation support in the Flutter tooling with the Linux GTK and better. Now, I met a Flutter I do cross compile the Linux GTK shared module. So in theory, one would be able to pass in the right parameters in the Flutter SDK. With a CMake tool chain file, you would be able to cross compile comments on the Linux GTK. I'm better is that the runtime library dependency list is very big. I would consider it applicable to desktop class processors. So there's a lot of extra stuff in it that is a bit overkill for an embedded application. While you could make it work, sure, it's quick and easy and you can do cross platform development and it all works great and you can use some plugins, but your performance on an embedded device is going to be less than stellar because it requires X11 and among other things. So it's not an optimized experience for embedded Linux. Flutter SDK support is missing to consume artifacts. So meaning when I generate, when I cross compile the Linux GTK components, desktop components and then Flutter, it generates an SO and gen snapshot and a couple other tools required to create an AOT. But the Flutter SDK tooling does not support that in any way yet. So I had some dialogues with the area owner on the Flutter team about this and basically the pushback was generate a design document so we can talk about this platform views. So platform views is sort of a quick and easy way to get you some sort of external surface interleaved with your Flutter hierarchy. Now there's been a longstanding issue that Chin made open a while back that discusses around the horrible performance of platform views. And effectively what happens is the, using a platform view can decrease your frame rate dramatically. So what happens is if you look at your widget hierarchy, depending on where the platform view is, one or two of them halfway down, it may have to stop that render pass, go out and fetch it and then continue the rendering. And depending on how it's done, you can typically see half of your frame rate drop. So if you were targeting 60 FPS, you can easily see 30 FPS with platform views and your CPU time would go up. My general recommendation is you want to avoid the usage of it and Flutter Auto does not support it for this reason. Flutter Auto is about being an integrator toolbox best in class sort of giving you an optimal path forward without compromises and we're not trying to please everybody with it right now as a solution. We're looking for creating experiences that will please and delight the consumer and sluggishness and other things like that are not on that list. Platform lit channels. So that's a dark and native code bridge and approximately adds around 10 milliseconds latency per message. So some things are, you know, if you think about your architecture of how you implement something having this everything serialized through this bridge and then having this overhead and it's not a really good combination of things. It's suitable for life cycle calls. So if you want to set up and tear down objects and things like that that don't happen in a high frequency or to support pre-existing platform constructs like tying into some platform channel things that, you know, are low frequency and maybe like platform info or, you know, a handful of things or trigger a launching the browser, things like that. Foreign function interface, FFI. This enables calling native C APIs from dark code and encourage zero latency. There's no message passing. In terms of platform channels, there's no async, await on dark, no garbage collection. This is the preferred model, right? So what you can do is you can use platform channels for the life cycle of your FFI or you can directly just FFI load it up depending on the actual use case. First party Linux plugins. So first party Linux plugins are only intended for Linux. GT came better. The use of the term Linux plugins was poorly chosen, in my opinion. In no way does it mean that first party Linux plugins work with any Flutter embedder that runs on Linux. That's where probably the number one confusion happens in the community. Like if there's a matrix support matrix on this plugin and it says Linux, what do you mean you don't support it? Well, what it should say is GTK better, you know, instead of Linux. So it's sort of a poor choice for how I'm branding that support. It should really be first party Linux GTK plugins. The fact that first party Linux plugin dark code runs and Flutter debug builds as a Flutter bug in its track here. Some solutions involve forking the Flutter SDK to support a custom plugin type, not a Linux. In my view, this is not a long-term solution. RIN drops the frequency of how that Flutter SDK changes. This is really not a good choice because it incurs such high maintenance overhead that it's not terribly practical. Huge amount of busy work. So it's basically cost to ownership goes really high. I look at solutions where you decrease your cost to ownership and you decrease your NRE and etc. So this is one here that I don't think is a good near-term. Well, it's a near-term solution, but it's also not a long-term solution. Bill environment. So the Flutter embedded build environment comprises of Yachto layers and some source code. So Yachto layers, Meta-AGL demo, Meta-AGL develop, Meta-AGL Flutter, Meta Flutter, and then Flutter Auto. So Toyota IVI Home Screen, AGL branch is what's currently used. Meta-AGL demo, a Flutter image, AGL IVI demo platform Flutter image has runtime equals release and a series of Flutter apps, Dashboard or HVAC. That's a typo. Navigation, media player, Meta-AGL develop. This holds a handful of images. AGL image, Flutter runtime, debug, runtime, profile, runtime release. So for these, you really like, if you're doing a debug workflow, you don't necessarily want any applications installed. So your debug build would have no apps installed, and they would all originate from the host via a custom device connection. If you have a runtime release, then either build the AOT externally on the host and then SCP it over to the target to run it. But what you, the general approach is you want to avoid mixing up when you're running a debug versus a release build. So release build, an AOT actually compiles the dark code down to machine code, whereas in a debug it's interpreted. So it's far less performant. And so if you're going to do a demo, you really want to keep these ideas separate and you really like, you know, upper management wants a demo. I'm not going to show them a debug build. Like, that would be not a good thing. You would only want to show them a release build. BBaPens and MetaAgeal Flutter. So it disables the GStreamer build flag for Flutter Auto due to the commercial licensing requirement. Enables network access for an archiver and adds a user service for Flutter Gallery. And then MetaFlutter, the other last yachto there. This is considered the upstream. Flutter have bundle structure. So what this is is a folder structure that all the embedders that are present on MetaFlutter support and it's modeled after the GTK runtime folder structure. So when you build a bundle, like Flutter build bundle on the command line, you end up with a build Flutter assets folder. And that is not the exact same structure that you need for running the device. So you can create this with some sembling. So there's a right of ways to do it. But this structure enables a number of things for each embedder. It allows you to do overriding system libraries and things like that. Easy swapping of versions, which I do take advantage of in the Flutter workspace automation. And the recipe is DevTools and MetaFlutter. We have DevTools, Flutter REST bridge example, membrane example, and then REST Proc2 support. This REST support item is the workaround to enable. Well, it's sort of seen as the official solution right now. Actually, there's been a long going issue on MetaREST on this. The approach is you do a BBA pen and you reassign the source to be the test standard lib, which has Proc2. So that is there. And the above two items are REST based projects and they require them. That's why that's there. In the graphics recipes, there's a handful of apps. Trying to pull in things that are relevant. So AGL Flutter apps are somewhat of a sandbox on things that might be changing rapidly. But that's where I tend to lock things. These are all locked. None of these are auto-revved. Flutter apps. Yeah, if one pops up, I will add it. If someone has a recommendation that they think is useful for a functional test, like the animated background one, it's great for just sort of showing a bit of how the efficient GPU is on the target and looking at your CPU load versus the performance, et cetera, on your frame rate. So the Flutter REST bridge, the membrane example, those showcase the model of using FFI with zero copy scenarios. So those are highly, membrane is highly performant. And that's what Toyota, we use as the baseline for our FFI work. The Flutter test video player is the first-party plug-in. And another example where I have some implementation in Flutter Auto that talks the language of the first-party plug-in. So secure storage video player are two of those. The other recipes are Flutter Engine, Flutter Pi, Flutter SDK, Sonia Batters, Toyota. So for the Toyota recipes, there's two. There's Flutter Auto, which correlates to the AGL branch. And IBI Homescreen, which correlates to the quarterly release. So IBI Homescreen on Toyota Connected's GitHub is moved. The main branch is moved to quarterly releases. And the AGL branch has no restriction. So there might be some cherry picking from the AGL branch to main. And vice versa, but it's just the cadence of that update. So major features will be coming in on IBI Homescreen, which then would get pulled into Flutter Auto, the AGL branch. So the tooling, Flutter Workspace Automation. And again, it's the authoritative repo on this script. The Kirkstone CI jobs, Dunfell CI jobs. Try to get a representative sort of swath of devices. There is some, there is no issue around IMX6. And I believe it's the Dart VM has a problem with the tuning that a particular DSP passes in. And so that, you know, IMX6. So there's always a trade-off. If you go as low-end as you can, you're going to have some NRE to deal with problems. And the general recommendation is, well, you can. And if someone forges that trail for you and sorts out the problem, then it's available. But yeah, the cheapest one on this list, I think, is the Raspberry Pi Zero 2W. And that can run a 2K display easy with no latency concerns or any problems. It's a very high value. And then the most expensive on this, I'll let you think about that. Honestor, this was the point where I was focusing on making Vulcan work on RPI 3 and 4. There's a driver that exists for 3 pre-4s. It's not 100% compliant, but it does most things. Like it'll play Quake and whatnot. All right, the flutter auto. Last component of the build area is ABA Home Screen. This is Wayland-based. So it supports the ADL shell. Up to V2, V3 is coming. Before the end of the quarter, XDG shell, stable. So the model here is the same code runs on desktop and on a target device. So there's no pound defines about either or the only difference then really is the graphics stack, right? That implements the, you know, from DRM, you know, in the drivers down basically is where the delta happens. And there's plenty of variance between implementations there. MultiView is a feature that allows single process, multiple engines per surface. So, well, an engine per surface. So you can have multiple surfaces all with its own engine instance all running within a single process. Similar to how fusion is it. Back-end support at compile time, not runtime detectable. The point is to shrink down the executable. EGL and Vulkan has a very flexible configuration front-end that involves JSON and bundle override logic. And recently, I've added a feature called Compositor Surface Plugin, which I'll have a demo of that. Here's an example of the Compositor Surface Unit Test Case in action. There's three surfaces, Wayland surfaces here. One is the gray, one is the flutter that's running a Vulkan back-end. The one on the left is EGL. It's a map box instance and the Vulkan PBR on the right. So two Vulkan surfaces, one EGL. In this scenario, I'm exercising the constructor and destructor of the map instance. And that's two EGL surfaces simultaneously in a single process. So, single process for everything. And more. So, some of the command line options for Flutter Auto. So, this is all pretty well documented in README for the source tree. And you can either do manual overrides on the command line or you can specify them in a JSON config file. And then any parameter you pass that is not consumed by the argument handlers, they get directly passed to the Dart VM. So, it's easy to pass things to the Dart VM directly. So, the parameter loading order for JSON. So, there's Vue global and then CLI. So, Vue is defining a window effectively. And within the window, you know, it's on context. And then you have the global umbrella over that and then CLI above that. So, first it'll look at Vue specific parameters. And then those get overwritten by global parameters in the JSON. And then all of that gets overwritten by CLI arguments. So, if you had a multi-view scenario with four windows and you wanted to change one parameter on that, you could do that via CLI. Just append the command to a command line argument list and it would override whatever was in your JSON. So, the bundle override logic will... So, you can basically create a structure where you kind of multiple versions of the same app or bundles. So, you can have one bundle that ran, you know, 3 through 7 and you have one bundle that you could run 3 through 1. So, each of those versions require a specific ICU DTLDAT and each one requires a little Flutter Engine version. So, those have to match what it's built with. Right? So, this allows you to sandbox each app independent of one another. It doesn't have to always use the Flutter Engine version in the system folder. If you built a fresh image with OS and you owned all the apps and nothing changed or moved, then it would make sense to just use a system-based Live Flutter Engine data. So, which is the default install. So, the default build flags for Flutter Auto. There's a few of them. So, it's componentized in a way that you can effectively just turn off all the code that you don't want or you don't need. And by generally, it's better to do that. Although, if, you know, you have an app now and then you want to introduce some functionality later that depends on one of these, then, you know, obviously we want to enable that as needed. So, the CI jobs on this are located here. And when building the source, the best reference for figuring out how to build it is looking at the CI job. And in labs here, running Flutter Apps, Auto Run Flutter App using a system service, MultiView and Run Flutter App and Runtime Profile Image. So, we have these laid out nicely. So, here's the case of MultiView. So, in the JSON, you specify, so each one of these views, it's an array of views, right? Each one here is specifying the bundle path, the window type. This is in the case of the AGL compositor. So, you can specify different panes in the AGL compositor which you wanted to pin to. So, in this one, this is a single process Flutter Auto instance with this configuration. So, they'll put an app on the background pane, the panel top and the panel left. And they're all independent apps and they all have different sizes. Accessibility features is just, that's saying, enable all the bits for accessibility. So, like, there's flags in there where you can turn off motion, like fast motion and things like that that are applicable to automotive. So, you can have a, and then you can dynamically change that as well. Here's an example here where the lab walks you through it. This is running the QMU at AGL image. So, you can see in the background, it has the pump fuel app. On the top, we have the animated background example right here. And on the left, we have the EGL texture test case, right? That's just rendering this with GL. So, and then this shows you how to do, set up a profiling instance on AGL. So, this is what you end up with. You can effectively profile it, right? So, under the CPU profile, you can enable it. And then it will enable, so what happens in the profile images, you have trace markers that are interspersed throughout the code which are enabled as part of the build flag. And so, that allows it to work with a CPU profiler. So, when you run the, you enable the CPU profiler and you run it, you get very interesting trace events. It's a listing, similar to like Windows ETL tracing or something like that. And then, it allows you to look at latencies and interactions at a much lower level. So, very useful troubleshooting tool when you're working on the finer nuances of a, you know, UI design. And here we are to the resources. I highly recommend docs.flutter.dev. That's a great Flutter development info. The Flutter Wiki also contains quite a bit of information about the Flutter engine that could be useful. And in general, Yachto, Metaflutter, heavy-eye home screen, et cetera. The AGL compositor information is really great. Documentation on automotivelinux.org, I recommend that. And that concludes the presentation. So, thank you for your time. I appreciate your attendance. And any questions, please feel to reach out to me at joel.vanarska at gmail.com. Thank you.