 Welcome to the Chrome Enterprise Technical Community Hour. Today we'll be talking about new features and APIs for Chrome web apps. My name is Rich, and I'll be your host for today's presentation. Joining us today, we have our main speaker, Ben Morse, who's a product manager for Chrome Browser. For today's agenda, I'll start with the recap of our previous webinar sessions. Then I'll jump into a quick overview of the Chrome Enterprise Recommended Program and the Technical Community Hour webinar series. Finally, I'll hand it over to Ben, and he'll cover today's topic and the call to action for our Chrome partners. In June, we started a three-part series around web applications. You can check out the Chrome Enterprise Developer Playlist on YouTube for two sessions that we had in June and July. One is about the foundations of PWAs and the other is about storage best practices for web applications. Today's Chrome Technical Community Hour is brought to you by the Chrome Enterprise Recommended Program, which is Google's partner program for third-party solutions that are optimized for Chrome OS or integrated with Chrome Browser. This webinar brings you the opportunity to engage with our team about new features and updates, enterprise development best practices, and our enterprise strategy. Now, without further ado, I'll hand it over to Ben, and he'll kick us off. Ben? Thanks, Rich. I guess I've got a presenter over here now, so let me do that. Okay, I'm taking over, apparently, from you. That's exciting. Yep. And here comes the window I'm going to present. And any minute now we're going to have the magic of presentation. There it is. All right. You see my slide over there? It looks like. I do. Okay. I will go on then. So I am Ben Morris, as Rich was saying. I'm a product manager for Chrome working on what we call Fugu, which is our code word for the APIs that make advanced applications possible on the web. Things like storage, audio, windowing, devices, peripherals, things like that. And I'm going to talk about unlocking new possibilities on the web. New things you can do that you couldn't do before, that you can do now, and maybe you want to try now. Who knows? So let's begin with the web. The web has come a long ways. It began as this very simple content site kind of thing. It gained popularity. Things like search engines happened. There was Yahoo, a fine website in its day. It still exists. And it just evolved from this simple kind of content site thing to basic applications to very elaborate applications of all sorts. For example, a couple of years ago, 2021, Photoshop launched a light web version of itself, which worked across browser, which is a big change. People took notice because Photoshop is a very complicated, elaborate, long-standing application, and suddenly a light version of it worked on the web. That was kind of neat. People thought, what else can we do? And by the way, you'll see various links over here on these various slides. These will be included in the notes for this video. So you need to write these things down as they come around or get screenshots or anything of that. So other things that happened, things like music synthesizers right there on the web, like Europa, making music on the web possible. Yes, it is. Even the venerable AutoCAD, which has been around 1982, has a version of itself for the web. There's one thing else on the web, which is popular these days. Video editing, like ClipChamp, which has at least 20 million users, a popular way to edit videos. Once again, working right there in your browser. Lots of new things are possible. And in fact, it's very easy to see a kind of application that doesn't actually exist on the web. So for example, there's things like office applications, documents, e-mail, spreadsheets and such. These things are commonly found on the web. You also have creative applications. It didn't actually advance. Come on, you can do it because I'm in the wrong place. There it is. Creative applications. Editing images, audio, video, that kind of stuff. Also video conferencing itself works on the web, of course. And things like game streaming, also gaming game platforms, but streaming games right there magically to your web browser. Also a thing you can do. Then there are IDEs, coder programming environments like VS Code, GitHub.dev, also possible on the web. Finance applications with multiple screens showing you all kinds of exciting things about stocks and bonds and whatever else you do in finance. Really, the sky is the limit. Lots of things are now possible on the web. So we're on this journey, it seems, to enable this bright new world where the web can do just about anything you can do anywhere else. We're not done here in Chrome. We're always busy making new things. New APIs, new features you can use to enable things that are new use cases, to do things that were previously impossible. So our goal here today is talking about some of the new things you can do on the web, new APIs, new features making new things possible, and also newish things from the last several years. Things perhaps you haven't noticed before, but we can do on the web now. You can think advantage of in your own applications. Things that are new, new, new, new. How does that sound? Sounds good? Let's try it. We're going to take this tour of a couple of things that are happening in various APIs, and I think the order would be, actually, we're going to start with WebAssembly. We're going to go to storage, I believe, if I recall correctly, then peripherals and devices, audio and windowing. So climb on board, let's take a tour. Although before you climb on board and take a tour, I took this image and I cropped out this guy from the left. Notice how he seems like he's not really enjoying himself. What's the story? I don't know. All right, let's go on. So WebAssembly, you may have heard of this before. WebAssembly is a new thing for the web. I mean, not new as a new in years, it's been around for years now, but it's a new thing that does not involve JavaScript. It's a binary instruction format and a virtual machine. You can compile languages down to WebAssembly. It's a compilation target, like assembly language for machines, but a bit higher level and available cross-platform. Quite useful, there is a tool called Unscripten, which allows you to port your C++ or C code to WebAssembly. Unscripten is useful because it doesn't let you just compile things, but also it contains the kinds of things you need in C and C++, the libraries you need to make things happen in the actual world. It's got POSIX file emulation, it can do audio, it can do graphics, it lets you work with things in the browser instead of having to deal with things in your C++ libraries, often transparently. It works quite nicely. A couple of things that have been done with Unscripten, for example, Figma, the popular online design software recently bought by Adobe, works thanks to Unscripten. Unity, the popular game engine, they take their C++ code and port it to WebAssembly using Unscripten. Then Google Earth, where you can travel around the Earth, see skyscrapers, trees, waters, oceans, all kinds of things. All it uses is Unscripten. There are more examples, there's only a few of the examples, and you can do even more with this WebAssembly, in fact. Now you can actually run Swift applications right in your browser. I'm going to move this window over here a little bit. There we go. Applications that are Swift applications, so iOS applications can also be brought to WebAssembly and brought to the browser, which is interesting. For example, and not advancing my slides properly, but now I am, GoodNotes has been an iOS app for the last 12 years. GoodNotes is now available cross-platform, Android, Windows, and the web. The web part is due to transpiling things down to Unscripten. They use the same code in many cases on different devices, different platforms, thanks to WebAssembly. So that's WebAssembly, which I mentioned because we're going to use that while talking about storage APIs. So, for example, storage APIs include lots of things that use the storage things on the web, many are old like cookies, local storage, system storage, some are quite new. For example, here we go, file system access. So browsers have for years allowed very limited access to user file system. You can upload files via a form, you can download files, but those are things under the user's control explicitly and they're very restricted. Later on you can drag and drop files into a browser, but there's almost never any programmatic control. With file system access, WebApp can open up a file picker and the user can choose files or folders to give access to from a web app. The application can then read and write files or read and write to certain folders. The app gets a handle and gets to actually read and write files. So things like IDEs, for example, where you can use various files, change things in files, and then run in a virtual environment are possible now. You see these things happening with GitHub.dev and VS Code, text editors. It allows for a lot of new use cases so you can now read and write the user file system. It's the thing which is done with a lot of restrictions so it's very safe for the user. It's also a version of this which is more safe even and available cross-platform, cross-browser, supported by Chrome and Safari and Firefox because it's a private file system. The origin private file system is a virtual file system where each origin gets its own virtual space to store virtual files. It has less privacy concerns because of the reason that it's a virtual file system and not actual user files that they can see easily. So this lets you use the basic primitive of file storage. You want to download video files, audio files, JavaScript, CSS, other kinds of things. You can do so with this and store them in this file system. You can combine this also with a thing called file system access handles which is a way to access these files and do low-level byte reading and writing and POSIX style byte-level file access. So with these things together, also with WebAssembly which we talked about a minute ago, you see I have this whole story here now we're getting with here, and in Scriptin which allows you to take C++ code and port it over to WebAssembly. Any application which relied on byte-level file access can be brought to the web now including things even like SQLite because all database is is writing and reading bytes to the disk in the end. We had to duplicate WebSQL because WebSQL was only in Chrome and not used very much and required a lot of maintenance. SQLite now, since last year, is available on the web. You partner with a team that makes SQLite at WebAssembly as a backend and now you can just download this library and in JavaScript, use your favorite relational database right there on the web. All thanks to file system access, the origin private file system and Scriptin WebAssembly and so on and so forth. And yes, it works across browser which was not true before for past SQL solutions. So in the past, the only real cross-browser database that was sophisticated was indexDB and now we have SQLite and really who knows what's going to happen next? Perhaps other databases will get ported to the web. Perhaps you will do this yourself. Now you, the user, the developer can port your favorite databases to the web and use them in your applications. What's coming next over here in this API? We're going to make it so that permissions can persist to give applications or to give users the power to let trusted applications remember which files and folders they have access to. So if you're writing your dream novel, for example, and you close the tab and come back the next week, do it again. You'll have access to it right away. And change observers so that if the user changes the files or folders locally outside of the web app, the app can become aware of those things and change state internally and also making indexDB faster because we realize it could be faster. That's it for right now for storage. Let's now go to the device APIs, which I think these are right now only in Chrome, but if you're a Chrome OS programmer, they're all available for you there in Chrome OS. These are the APIs that let you connect to peripherals and communicate with peripherals of many different kinds. For example, WebUSB lets you, if you can guess how this works, that's right. Like the USB peripherals via USB cable and USB protocols. The Web Bluetooth API lets you, that's right, connect to Bluetooth devices and then communicate with them. Web serial, oh, the one too many there. Web serial, okay, fine, we'll do that one too. We'll make it wait. Web serial, you're right, that lets you communicate via serial port, but also anything else which uses serial port emulation, RS232. So for example, many Bluetooth devices use serial port emulation. Often also people use serial port emulation over USB. I think this is how LEGO does it actually, is we use Web Serial API to connect via USB cable. Web serial is for serial ports or also for the serial protocol. Then there's WebHID, there it is finally, which is for human interface devices. This is for, on the Web, sort of non-standard functions of standard devices, extra features. Like for example, browsers already can talk to mice and keyboards and headsets and get basic events like mouse up, mouse down, keyboards, it gets keyboard events and so on, but if your headset has call controls, for example, WebHID can be used to access those. If your mouse is a gaming mouse and has a third button or an extra scroll wheel and lights that pop up when things happen in the game, those things are also good use cases for WebHID. Also the Gamepad API, which allows, yes, you guessed it, either here are things that game controllers are doing, also does that have to feedback back to game controllers, and last but not least WebNFC for near field communication. Recently, Capital One used this to let Android devices communicate with Capital One things in the world. That's a lot of words there. There's a lot of device APIs. I mentioned before I think Lego Spike, Lego Spike uses the serial Bluetooth APIs to let kids in classrooms use Chrome to talk to robots, not talk as in communication with human language, but as in to program robots using a scratch like interface and also using Python if they know Python. It lets them connect to the robots via Bluetooth or via a USB cable with the serial protocol and then send code to the robots, get information back from the robots in their location, some of them have sensors like light sensors for example. It's kind of neat. It works using our device APIs and we featured it at Google I.O. this year. Another use case is Logitech WebConnect. This is the web app the Logitech made. It lets you connect your Logitech Bolt and your Unifying and Wireless devices. You want to actually use these things. You want to connect your USB device. How do you do it? Our approach over here is to make APIs that are very low level. They do simple things and thus you can do more complicated things using simple methods. Because if you think about it, when you're communicating with a peripheral, what you're doing is you're sending bits and bytes to the peripheral and getting bits and bytes back from the peripheral. Whatever you do is going to transition via bits and bytes. These are what the APIs let you do. Instead of having the higher level of functions like listen to my call, it's more like things like connect or disconnect and send and receive data. The code looks kind of like this for example. Look at that first line there, control transfer out. This sends a value to a peripheral. Look at the last line there, transfer in. It gets bytes from the peripheral. That's how it works. There's a lot more to it, of course. This is USB. You can find more information again in links in these slides and also links you'll see in the use show notes. If you want to use these things, don't start from scratch because they've been around for a couple of years. Look around for existing libraries and implementations. For example, Libhid's Aliphany, a library made here at Google, allows you to communicate with headsets and use call control devices to create a major well-known headset. Another example is this nice guy here named Niels who runs a company called Salon Hub which runs payment solutions for hair salons, credit card taking, receipt printing, barcode scanning, all those kinds of things. He has kindly open sourced a lot of his work. Things like barcode scanning, receipt printing, other kinds of things. Start with his work, talk to Niels and you can find out what else you can do. There are a lot more things on GitHub. There's libraries out there and collections of things for USB, Bluetooth, for serial. A lot of the work already has been done. What's coming next here for these APIs? One of our goals is to make it very rare to have redundant permissions. So if you connect the peripheral and give permission to talk to your computer and vice versa, you can then make sure it remembers that peripheral when it's connected next time. We're also working to make the permissions dialogue nicer, friendlier, kind of given some UX feedback from partners and from people we've seen using it to make it easier to know what to do to make it look nicer. And also, BLE scanning, Bluetooth low energy scanning which you can use to be aware of nearby devices that are beacons. For example, containers in a warehouse or things that I guess objects in a warehouse can be scanned and you can wear these things things like bikes on the streets where are their bikes, where are they docked or even things like wayfinding in hospitals where there's various beacons around the hospital and your Android device or other device which can use BLE scanning can see what things are around use those to find your location on a map hospitals are very big wayfinding can be a problem and BLE scanning will help for people. That's a lot about device APIs. I have more though because we're going to talk about now audio. We said this was about new APIs but this is not a new API, it's been around for quite a long time. It's available mostly features available across browser on Firefox, Safari and Chrome because you have audio and music and music ads working at all these places and made a variety of cool features you can use today. Web Audio is used for various different kinds of audio related applications. Sound production, if you're trying to record people around the world via the web or create music or other kinds of sound it's a good way to do those kinds of things on the web. I mentioned this a minute ago but music creation also a thing on the web you can record music, you can share music with your friends, you can collaborate with people across the world via Web Audio Video production there's popular video editors you've mentioned Clipchamp before also frame.io products that use AI to help you make video more easily and you send to use Web Audio to produce the audio ad effects and edit and things like that video conferencing applications often use Web Audio as well gaming applications to Web Audio because if you're writing a game application you often want to have a certain sample you can play different pitches or change the volume or pan it from left to right or your left to my right those kinds of things Web Audio allows you to transform sounds in real time useful for gaming also things like speech recognition for those cool AI enabled apps where maybe something generates some text you want to pose a follow up question you can speak right there to your web application and it can then respond to you with the magic of AI the application is using Web Audio today there's a lot of them out there we'll just mention a couple like Zoom for example I mentioned video conferencing before use Web Audio to immipulate sound in real time make things sound better and sound nicer in various ways I mentioned game platforms like crazy games for example use Web Audio also session wire allows you to do music production and talk to people around the world and sound production on your browser and master channel I mentioned because they're kind of a cool new use case there's this guy in Scandinavia who is a mastering engineer mastering songs and had some successes and songs such things also computer programmer and you thought I could use AI, I could train a model to do what I do couldn't I? he's been doing that for a couple of years now master channel now lets you do mastering right there on the web these things are often on the web because they're easy to discover that way it works cross-platform and of course if you're working on Chromebooks or Chrome OS devices it works nicely on your Chromebook how does all this work there's two major ways to use the Web Audio API the first one audio nodes there audio nodes are a series of ways to transform audio and do basic operations on audio or more complicated operations for example to change the pitch, generating new audio, changing the volume adding reverb or compression other kinds of effects there's nodes for all these things you can also adjust instead of your sophisticated person out there and you've got some C++ code that already does audio stuff you can use inscription to bring it to WebAssembly use the audio worklet and just go ahead and change audio however you want to right there in your own code that link there bit.ly audio-worklet has more information on that you should just look for some demos though because a lot of fun demos are out there that show things you can do with audio there's a couple over here I'm not going to list off the long URLs but there are things you can do here on the first one there are Chrome experiments try oscillators it's quite nice try other things there second one there is hip-hop doodle Google did for hip-hop anniversary that's fun you can you can turn tables around and do things with samples right there in your browser all thanks to WebAudio a couple more examples of applications using WebAudio out there Soundtrap is a popular music creation application you can use samples recorded during audio, change audio around mix things and share with your friends Spotify bought this and recently sold it again it's popular in education which means popular also on Chromebooks BandLab another popular web application they've also got a mobile application which works on iOS and Android but also popular on the web I think you have 50 million plus users now you can record audio again using samples or using your own audio you record yourself or using synthesizers you can make tracks you can post tracks online it's a great way to make audio and get started with making music one more thing finally CleanFeed is an application on the web that uses WebAudio to let professional producers produce audio if you're recording people in various places around the world and want to record high quality wave audio CleanFeed lets you do this kind of stuff in fact CleanFeed is so popular now on the web that it went on Emmy recently for its use in production and blog posts on this they mentioned and thanked by name our own HongChan Choi who is a tech lead of WebAudio here at Chrome we're almost done with our tour here one more thing I want to mention is multi-stream window management that we launched last year it allows you to become aware of what screens the user has plugged into their device the size of the screens the orientation and place windows on these screens the next year or so we'll be doing more things possible we'll be letting you make full screen windows more easily make more windows with a single user gesture just more freely if user gives you permission to do so to make windows on your screens for creative applications financial applications video conferencing even presentation applications like this one over here we want to maybe present slides on one screen and show your own notes on a second screen many uses exist for this it is called multi-screen window management and that's all I have for you today if it wasn't enough for you a lot more things are happening on the web if you have questions please reach out to us, please get in touch and please look at the various links and the notes over here to try out some of the nice things that we have discussed today thank you very much now back over to Rich who will talk probably more slowly than I just did I crammed a lot there into 20 odd minutes I think we have another slide here for Rich one more person than I am thanks Ben please visit the Chrome Enterprise developer website for additional information to supplement your learning that concludes today's presentation