 Hi, my name is Thomas Steiner. I'm a developer relations engineer on the Chrome team at Google. Today, I want to talk about advanced APIs in real world apps. At previous events like last year's I.O., you may have heard me and others talk about Project Fugu, an effort to close to web's capabilities and thereby enabling new applications to run on the web. If we've talked about Project Fugu before, why would we talk about the project again? Well, because there's been lots of exciting stuff happening around the APIs that we've developed. For a start, you will see how image editor Photopia makes working with files a supernatural flow. You will learn how Blockbench, the app that was used for creating the goats in Minecraft, enables users to pick colors from anywhere on the screen. Next, you will see how the game editor Construct 3 makes sure you never lose any of the games you create. And you will understand how the SVG editor Boxer SVG lets the user pick any of the fonts they have installed locally, including Comic Sans MS. And last but not least, you will find out how Adobe made zooming even the biggest images in Photoshop a battery-smooth experience. Before we begin this epic journey, let's briefly reflect back. As I said before, the Project Fugu effort isn't new. In November 2018, on the Chromium blog, we made a commitment to a more capable web. In the blog post, we promised to close the gap. What gap, you may ask? Well, the capability gap between the things you can do on platform-specific apps and the things you can do on the open web. We worked with our partners and came up with a long list of missing capabilities. For example, in 2018, you could not edit files on the web. So we prioritized this and many other use cases based on partner and developer needs. We made our list of missing capabilities public on our bug tracker and labeled it with Proj Fugu. Fugu as the puffer fish. It's a delicacy if you cut it right and can be dangerous if you cut it wrong. Just like being able to edit an image file on the web is fantastic but having the power to delete a file that is critical for the functioning of your computer can have fatal consequences. By the way, we don't allow system files to be accessed from the web. So metaphorically speaking, we make sure you cut the Fugu right. So back to Project Fugu. It's a cross-company effort. In it, we have Samsung and Microsoft and ElectronJS and of course Google and Intel. Samsung makes a browser called Samsung Internet and Microsoft is the maker of Edge and Electron have less node libraries to depend upon when they can just use the web APIs exposed by Chromium. Google, as you may recall, has Chrome but why is chip maker Intel invested in Project Fugu? Well, turns out they looked at what people are doing on Intel machines and found out that they spent more than 60% of their time on the web platform. So there you have it. That's why Intel is a project partner. All the work of Project Fugu happens in the open. For example, you can look at our Fugu API tracker where we keep track of the shipped and in progress APIs. Actually, recently I looked at this tracker in detail which caused me to write an article in which I asked, is Project Fugu done? When I counted, we had shipped 55 APIs ranging from red-blue tooth to contact picker and from badging APIs to unadjusted movement pointer to lock, from web share to app shortcut menu and protocol handling to the forgetting of serial ports all the way down to web custom formats on the clipboard. 55 APIs, that's a lot. But as you might have guessed, we're not done yet. There's still plenty of use cases that need unlocking and some of the APIs need refinement now that more developers use them. And boy, developers do use our Fugu APIs. We have a refreshed showcase of apps people have built with our APIs. Be sure to browse the showcase, add your apps if they're missing and subscribe to the RSS feed to be informed of new apps. With more developers using our APIs, we've also identified a number of useful patterns for doing so that we have added to our patterns collection. They're all worth exploring, but I specifically want to highlight our advanced apps patterns collection, our cool collection of clipboard patterns and our fantastic collection of patterns for working with files. I could go on and on and on since these patterns are really, really useful, but I'm supposed to be talking about advanced web APIs in real-world apps, so back to that. The first app I want to look at is Photopea. Photopea is a free online photo editor. It has various image editing tools, including spot healing, a clone stamp, healing brush, and a patch tool. It supports layers, channels, selections, and much more. It's a project Fugu API showcase. And if you zoom in on its listing, you can see that the app uses a lot of Fugu APIs. Every single one of them is interesting, but today I want to focus on Photopea's use of file handling. File handling is the capability of registering and install PWA as a file handler with the operating system. Photopea is a photo editor, so logically it has registered itself as a handler for image formats like JPEG. With Photopea installed, I can now right-click a JPEG file and open it with Photopea. It could even go as far as promoting Photopea to the default file handler and open images with Photopea simply by double-click. So how did Photopea do this? On this slide, you can see its production web app manifest. The highlighted section shows the file handler's member, which contains an array of objects. Let's zoom in on one. Each of these objects contain an action key pointing at a URL and an accept key pointing at an object with pairs of mime types mapped to arrays of file extensions. For example, the mime type image slash PSD for Photoshop files is mapped to an array with one entry, the extension dot PSD. Photopea registers itself for 21 other image-related file types. After declaratively telling the operating system what file formats Photopea can handle, the app needs to imperatively do something with the past files. This is minified production code, but it's easy enough to follow. It stores window.launchq in a variable called n. And if it exists, sets its consumer to a function that gets passed a parameter w. This w has a property, files, by which the past files are accessible. If you want to implement Photopea's file handling behavior yourself, check out the how to handle files open from the File Explorer pattern. The file handling API is supported on Chromium-based browsers as a version 102. Next, I want to show you Blockbench. Blockbench is a low-poly 3D model editor. If you have ever played Minecraft and have encountered a goat, you've met something that was created with Blockbench. Blockbench, which uses Vue.js and 3.js, puts all the tools at your disposal to make the creation process of low-poly models as easy as possible. You can use Cuboids to get that Minecraft aesthetic or create complex, low-poly shapes using the mesh modeling tools. You can create, edit, and paint texture right inside the app. And obviously, it's also contained in the Project Fugu API showcase. Similar to Photopea, it uses several Fugu APIs. For Blockbench, I want to highlight a small but neat API called Eyedropper. In this example, I'm editing the texture of a 3D model of a train. With Blockbench's paint tool, I can freely modify the texture. What happens when I click the color picker? I get a magnifying glass that I can use to pick screen colors from anywhere, including from outside the application. So I can pick the lively orange from the gorgeous Mac OS Ventura background image. So how does this work? Pretty simple. Here's the relevant snippet from Blockbench's code base. You initialize an Eyedropper by calling new Eyedropper. Then you just await the result of a call to its open function, which is an sRGB hex color string. For more details on how to work with the Eyedropper, like Blockbench, see the article, picking colors of any pixel on the screen with the Eyedropper API. The Eyedropper API is supported on Chromium-based browsers as of version 95. You can talk about Minecraft codes and not dream of becoming a game developer. Guess what? My next example may make this dream come true. Construct 3 is a fantastic web-based game-making software. You can use it to create anything from platformers to puzzles to racing games and much more. If you don't want to, you don't even have to touch code for your game. Of course, Construct 3 is in a project Fugu API showcase. Zooming in on its card, you can see it uses a ton of Fugu APIs. There's a lot to potentially dive into, but I want to put the emphasis on Construct 3's file system access. Here, I'm editing one of the examples, PopLab. On the screen here, you can see that it says Z2Select. Hmm, let's change that to S as in SELECT. Let me save my changes by a menu, Save As, save a single file. Let's call the file poplap.c3p. Now, Construct 3 holds a reference to the file, which means I can just press Command plus S to save any further changes to the same file without having to go through a new save dialog every time. But what's really cool is that when I reload the app, I can continue where I left off. This works because Construct 3 has a stored serialized version of the file handle into indexed DB. Here, I show the code for saving a file. Construct 3 calls the window.showSiveFailPocket method. It uses an ID of saveProject file, which the browser will use to remember the working directory. With the types field, the Construct 3 developers give the user a hint about the type of file via a localized description and via the accept field, tell the browser that Construct 3 saves its files via the MIME type application slash x-construct3-project using the C3P file extension. To learn how to save files like Construct 3 in a way that works in all browsers, check out the aptly named pattern, how to save a file. The file system access API works in Chromium-based browsers as of version 86. Other browsers can use a fallback solution. If you build a game, you alternate game assets. Some of these game assets you might actually want to create as a scalable vector graphic, SVG for short. Boxer SVG is a great app for editing and creating SVGs. Boxer SVG is built around the idea that user interfaces should get out of the way. There's no crowded workspace with overlapping dialogues or dozens of open pallets and toolbars. The illustration takes the center of the stage. Surprise, surprise. Boxer SVG is in the product Fugu API showcase. As the other app so far, Boxer SVG 2 uses several Fugu APIs. Especially interesting is the local font access API. With Boxer SVG's text tool, I can add text to my drawing. In the typography panel, I can choose the font for the text. Boxer SVG integrates with the Google Fonts directory, but thanks to the local font access API, I can also get access to the fonts installed locally on my computer. Let's choose one of my favorite classics, Mark of Felt. The way this works is surprisingly simple. After feature detecting if query local fonts exists on window, all that remains is actually calling the function. It returns a promise with an area of the installed local fonts. It's an asynchronous function because if the user hasn't granted access to their fonts yet, it will show a prompt asking the user if they want to share their fonts. Remember that the list of installed fonts can be used as a tracking vector. So you only want to give it out when you trust the site. If you want to let your users pick their locally installed fonts like Boxer SVG, read the article, use advanced typography with local fonts. The API is supported on Chromium-based browsers as of version 103. The last app I want to look at is Photoshop. The Adobe Manage to get a version of Photoshop running on the web still blows my mind. Photoshop is an image creation and photo editing software developed by Adobe. The tool provides many image editing features for pixel-based raster graphics, but also vector graphics. For its UI, it makes heavy use of web components based on lit. Like all other tools mentioned so far, it too is contained in the project Fugu API showcase. And as a powerful app, it obviously uses several different Fugu APIs. I want to focus on Adobe's use of the Origin Private File System. The file system standard introduces an Origin Private File System, or OPFS for short, as a storage endpoint private to the origin of the page and not visible to the user that provides optional access to a special kind of file that is highly optimized for performance. Photoshop uses files in the Origin Private File System for the performance zooming of images, which is an expensive operation. To make zooming as smooth as possible, Photoshop stores the image at different pre-calculated zoom levels in so-called MIP maps. As you can imagine, whenever you make a change to an image, the MIP maps need updating as well. Adobe really needed the fastest possible way to get access to files. And this was through the Origin Private File System. The full implementation details are of course Adobe's secret. But using the OPFS Explorer extension for the Chrome DevTools, you can peek under the hood a bit and see how the source it is made. The files in a temp folder that begin with Photoshop are the MIP maps. You can see that they get quite big and here I'm only editing a small image. What is not a secret though, is how Photoshop gets access to the Origin Private File System. They first obtain a root directory by calling the asynchronous function navigator storage get directory. Next, they create a new file handle by calling the likewise asynchronous function get file handle on the root directory, passing it the name of the desired file and a boolean flag called create that says the file should be created if it doesn't exist. Finally, they obtain an access handle by calling create sync access handle, which is the last asynchronous function in the chain. All other operations like write, get size, and read, et cetera, are then synchronous. So Adobe can access them without overhead from M-scripten, which is what they have used to port Photoshop over to the web. To learn how to use the Origin Private File System like Adobe, read the article, the Origin Private File System. The Origin Private File System is supported on Chromium-based browsers as of version 86, Firefox as of version 111, and Safari as of version 15.2. Wow, five amazing apps that all use Fugu APIs. Let's briefly recap what we've seen. We saw image editing app PhotoPeer and how it uses the file handling API so PhotoPeer users can start editing images right from their file explorer. We looked at low-poly 3D model editor blockbench use of the idropa API to let the user pick colors from anywhere on their screen. We learned how Game Editor Construct 3 uses the file system access API to let users save their creations to disk and enable a natural, edit, save, edit flow. We experienced how SVG edited a boxy SVG lets the users pick the fonts they have locally installed on their computers. And finally, we don't forbid until how Photoshop uses the Origin Private File System for high performance access to metadata files. Now the big question is, what will we see next? What's the next amazing app that you build with Fugu APIs? Remember to check out the Project Fugu API Showcase for inspiration. Also, definitely see our collection of amazing patents to get you started in a way that works on all browsers, not just Chrome. Oh, and just to be sure and avoid any misunderstanding, you can use Fugu APIs from all frameworks or with just vanilla JS. With that, thank you very much for watching. I can't wait to see your apps get added to the Project Fugu API Showcase.