 Hi, everyone. I want to talk to you about the web perception toolkit, a JavaScript library for doing QR code, bar code, and image detection that we launched at Google I.O. The idea behind it is that lower friction experiences help everyone. For example, when you're out shopping, you're used to the idea that you could see a QR code, a product logo, or a bar code or something like that. And if you could scan it instead of doing thumb typing, well, that would help. And if we could teach our PWAs to look for those things, well, that would help everyone. Now, for doing things like QR codes and bar code detection, we have the brand new web platform API, the Shape Detection API. And it's awesome, but it's not everywhere because it's a brand new API. And in terms of doing things like real world logo scanning, well, there is no platform API for doing that either. Now, we've got two options here. We know it's a good idea. We could either give up, or we could try and fix it. And that's exactly what the web perception toolkit's trying to do. We could try and fix that problem, try and make it easy to make those kinds of experiences. So from the user's perspective, let's say they're using something with the web perception toolkit. What does that look like? So the user launches the experience by tapping on the button that fires up a camera feed that they can point at something in the real world. In this case, a logo. And the logo is the one from the Squoosh app. And we identify that it's that. And from there, the user can launch information about Squoosh in this case, or they could go directly to the Squoosh app. So in summary, the user flow hopefully is fairly simple. They can tap on the button to launch the experience, point the camera at something, and get information about it. But I'm a developer, so I wanted to give you the architectural overview as well for you. So the web perception toolkit comes in, broadly speaking, three parts. There's the sensing, the meaning making, and the rendering. The sensing is where we take in the camera feed. So we use GetUserMedia to get access to the camera. From there, we can take snapshots of particular frames. We can either do that automatically in a kind of passive mode, or a more active mode where the user taps to capture a frame. We pass those frames off to a couple of detectors, a barcode detector, and a planar image detector. Once we detect something, we pass that over to the meaning maker. And the meaning maker uses structured data to figure out what the object is and what information it should show in the third part, the UX and rendering part. So that's the broad overview. What's the developer flow look like? Well, firstly, you've got to install the toolkit, which you can do with an npm install, which is npm install perception-toolkit. The next thing you need to do is to embed some structured data inside of your pages. This will describe the objects that you're looking for and what information needs to be shown when that object is found. So for example, if we have something like a barcode at the top here, you can see that we have an object which has AR target and the type is barcode and the value of that is some value. So that could be a number. It could be whatever you expect that barcode to resolve to. Below that, we have the content that we want to show when that item is found. So in this case, it's a web page. It has a name, a description and an image. This is the information that's passed off to the card later on. Now, if you are using something like an image target, well, things are a little bit more involved. We have the type here as AR image target and then we encode the image into a binary format that we can pass off to the perception toolkit. More information on that is available in the documentation, a link to which you can find in the description below. After that, we've got to configure the toolkit which we do by putting an object into the global. Here, I've got a minimal config where we have the route which is where the toolkit has been installed, the button that we'd like to use to launch the experience and the container we'd like to use when we generate some of the cards. Finally, we need to embed the toolkit which we do by putting some CSS and some JavaScript into the page. Now, we do allow customization. You can change how the cards look and all those kinds of things. I don't have time to get into that today, but again, if you want to get more information on that, check out the documentation linked below and also there's a link at the end of this video which you can follow as well. Now, the good news is you can also try all of this today. If you go to io.perceptiontoolkit.dev, you'll be able to launch the PWA that we had at Google IO and it has a link there for you to download some targets that you can print out and just try scanning at home or in the office. If you do use the Web Perception Toolkit, please do provide feedback. You can file issues on the GitHub repo. Again, we'll link that in the description below. And with that, thanks for watching.