 Welcome back to WebDev Live as a Brit and tickled to be kicking off day two from an Amir friendly time zone Yesterday we spoke about governments doing great work getting the word out on the web And one of the most inspirational digital services is from the UK where I saw a government convene online for the first time Now we've all been learning how best to work and learn from home and at Google We're trying to get a deeper understanding of the needs of web developers Partnering with the community as one of our close partners I wanted to invite our friends at Mozilla to chat more about that work and also get information on what's new in their world Welcome Kadea and Victoria Hi, Dion. Great to be here. Hey, so Kadea Web developers know MDN really well and many may have participated in the last DNA report But can you get us up to speed a little bit and tell us about its history and what we're trying to do? Yeah, of course. So this really started in late 2017 Shortly after CSS Grid was shipped CSS Grid was a massive success But was also years in the making and layout had been an issue for web developers at least since the 90s when we Abused tables to implement our UI designs So we asked ourselves so can we have more of these wins and we talked to people who worked at the web platform At Mozilla about how they prioritize things and one thing really stood out because almost everyone said the same thing They said we need to hear more from developers and it makes so much sense because none of us can be successful without that part It's hard to prioritize the right thing without knowing about developer pain points and it's hard to find the right solution Especially if it and it's hard to get people to Use something if it's not solving their problem or not solving it in the right way so for all of those reasons we proposed a developer needs assessment and The DNA in short is meant to be a single and simple tool for harsh prioritization representing very diverse populations and a huge feature space and It's published on MDN and that's important because it's not owned by a single browser vendor We initially proposed this under the umbrella of the MDM product advisory board where we have representation from browser vendors Like Google and Microsoft and Samsung, but also the WSBC and industry stakeholders And as a community we need to have at least a common understanding of the facts when it comes to needs even if you draw different conclusions from them and For this iteration in 2019 more than 28,000 developers and designers took the 20 minutes necessary to fully complete the survey And that's from a hundred and seventy three countries total and that's about ten thousand hours of time contributed By developers and designers to help us understand what their pain points and needs are and We believe that makes the MDM web DNA the biggest web developer and designer focused survey Connected so far. Yeah. Well when you put it that way Just really want to say thank you to to the community and anyone that that took the time to Go through those, you know 20 minutes and getting us the feedback. It's it's incredibly useful to us But but thank you so much. So when you were going through the results Cadir the 2019 report. What kind of really kind of stood out to you? You know one thing really and that's web compatibility and interoperability the four of the top five issues were focused on exactly that topic and One of the biggest strengths of the web is that there is no single entity controlling the platform But that doesn't come for free web developers and designers are frustrated by not being able to use features By having to find workarounds fiddling with browser differences And also by the difficulty to verify that something that works in one browser will not break in another browser and Related to that. It was a bit of a surprise that the top five issues were extremely stable between Very different markets. So whether it's China India Japan The US or France the top issues for web developers revolve around web compatibility and interoperability Got it. So when you take this feedback in How does it actually kind of change the road map at Mozilla? Like what are you now looking to focus on based on this feedback? Yeah, so web complete web compatibility was already a focus at Mozilla even before this But we now have doubled down on it. So recently we made browser compatibility data machine readable on MDN And that's now starting to pay off. So if you use yes code very popular code editor The tooltips have compact data information when you write CSS and We also recently started a collaboration with can I use comm? To share the data that we have so we are all looking at the same browser compact information And I'm sure Victoria can say more about it in a moment But the Firefox dev tools say now come with compact data information built in got it Yeah, this the feedback has been really Helpful for us and the web platform team is really, you know working even deeper on stability Compat issues that you talk about Helping with testing layout really kind of understanding what what the developer needs are and kind of bringing it into our prioritization too So it's actually almost time for the next version of the DNA report. So what do we have in store for developers this time around? Yeah, so one thing we're super excited about this year is to see how things have changed year over year So we want to see that developer satisfaction go up or down and how I have the top Pain points changed. So this is really the first opportunity to see those trends got it Well, I can't wait to get that out to developers and I hope that everyone's watching Would do us a favor and take some time to to get that feedback in so we can like really know What we can prioritize in the future for our roadmaps now, Victoria I actually used to work on the on developer tools at Mozilla And not only was it the birth of a, you know, great tooling in the browser back from Joe Hewitt's, you know Firebug onwards, but it really continues to to push the bar So I'd love for you to be able to kind of catch us up on what's the latest what's new in In Firefox dev tools. Hi, Dan as Gadir explained We know that differing browser support for CSS features is a top issue for web devs So our team built the compatibility panel to make it easier to stay on top of this it lists all the CSS on your website That's unsupported in certain browsers as well as deprecated styles. You try it now in Firefox dev edition I really love the the UX touches there, especially the turtle on the like that's that's great And in general, you know a lot of great features that are landed there I'm curious if there's anything on the upcoming roadmap that you're excited about sharing We've also been working on the Firefox profiler. It's our performance tool which features shareable links for collaboration We've been integrating the recording UI into Firefox. So it's easy to get started This tool is also currently in dev edition as far as our debugger I want to highlight two unique features in 77. We added a type of break point That's new to browser tools the watch point it lets you pause when an object property is accessed or changed Also, we've made source mapped variables work when you pause in an original file We now reverse engineer the scope chain so that variables look correct in the scopes pane and work in the console This was six months of incredibly challenging work done by our teammate Logan who had deep knowledge for being tech lead on babble The year before we joke that he's the only person in the world who could have written this So recently I really embraced the open design process for a network panel redesign We set and we sent out a survey and posted early mock-ups to Twitter and got amazing input I Originally used bold to indicate large files and someone suggested it'd be more clear to have mouse and elephant icons for small and big That's how we ended up with the turtle for slow responses People also told us that when it comes to the domain column they mainly want to know if it's third party or not So in this condensed view with a sidebar, we hid the domain column and added an icon that indicates third party requests Originally, I made these brightly colored file type icons and some people love them and others said it was too much They look like candy, so I iterated tone down the colors and got to the result you see here Most of this has landed in the latest nightly. We hope everyone will try it out and give us more feedback Awesome Great. So Victoria Kadir. Thank you so much for taking the time to join us today We really appreciate the great work the Mozilla continues to do for the web Now you spend a lot of time both in your developer tools and also on the core task of building your app UI So to chat more about modern web UI, let's welcome Yuna. Hi Dian Hi Yuna Now we were just talking about the developer needs survey with Mozilla And there was plenty of feedback from developers on layout So I was just curious, you know, are there any recent additions to the platform that you think target these needs? Oh Yes, we've definitely been listening and CSS has been evolving so rapidly in the past few years in really months So tomorrow I'll be going over a ton of cool aspects of modern layout with CSS grid and flexbox Including how to harness the power of CSS functions like clamp fractional units auto placement the min max function Justification place items the repeat function and a lot more to create robust layouts breaking down how powerful a single line of CSS can be There are also some CSS properties coming down the pipeline that will help with a lot of user needs that haven't yet been met an Aspect ratio is one of them. This just landed in Chrome canary and it enables users set to find width to height ratios for media items Like images and video Previously the way to do this was a hack using padding and calculating a percentage But now you can set your ratios in a much more readable way I'm looking forward to this landing in browsers and making a lot of developers lives easier because I know this is something that I've run into a lot We're also getting the gap property in flexbox This one is exciting because of how many times we've just been styling a series of items and wanting there to be space between those items But not around those items Gap enables the parent element to control spacing not the children making it easier to style these items uniformly within that parent Currently you can use gap to create tracks with CSS grid But you'll be able to use a flexbox layout too meaning you can leverage all the benefits of gap with the greater choice and layout mechanism The web animations API is also getting a lot more robust in chromium 84 Now we have promises replaceable animations Composite modes partial keyframes and a way to access animations from CSS in JavaScript Check out the blog post on web.dev for more information about these updates and try them out in Chrome canary yourself The app property rule is also available behind a flag in canary And it's something that I am particularly excited about because this allows for semantic variables in CSS With app property, you can declare a CSS custom properties that have semantic typed values and fallbacks This is the part of the CSS Houdini effort specifically the properties and values API and Previously was possible in JavaScript with CSS dot register property as a part of Houdini But the app property declaration brings this into our CSS files Meaning a nice co-location of superpowered styles with the rest of your CSS The other Houdini APIs to keep an eye out for are the typed object model the paint worklets Animation worklet and the layout work with great So JavaScript seemed to kind of have its time with the addition of a sink away. Yes modules and the like It was great to see that evolution. It's really feeling like this is a big time for for CSS I love being able to get features that like gap that you made that you mentioned in the like to just make things a lot easier For us and super excited at how deep you can get into on the on the Houdini side, too But if I use these and I'm building these super rich designs and the like With this great power comes great responsibility. So how do you think about the role of accessibility here? I love that you bring up accessibility because accessibility should always come first I think you're absolutely right that that needs to be top of mind Your users need to be able to access your content and navigate your product. It is not an enhancement I think of accessibility as a core feature and prom 83 actually just launched with some new accessibility testing features Which are pretty neat because they allow for visual accessibility testing So now through dev tools you can examine if your UI works for users with the various vision deficiencies like blurred vision and Four different types of color blindness. Yeah, it's great We're actually gonna have Paul Lewis come on and kind of walk through that a little bit and so thanks so much I also have really been enjoying your new CSS podcast with Adam Argyle Not only are you both kind of like really fun to listen to on the like It's actually been really interesting to what you kind of go through step by step and kind of teach us the fundamentals There was a lot that I didn't really know about Yeah, honestly, we are learning so much as we are going through the fundamentals and we're having a lot of fun making these episodes So if you haven't seen it yet check out the CSS podcast Absolutely, so thanks so much for joining us, you know, and we'll see you later on the stream. Bye Now there are a few more dev tools features I'm really keen to show you and no one's better to show off a bit at all in than the ever supercharged Paul Lewis Hey, Dionne, how you doing? Not too bad. How you doing mate? Yeah, pretty good. Thanks. All right So one of the things that we've noticed is that DevTools puts out these console warnings as you can see on screen And if you're anything like me after a while you start to ignore them And the reason is that there just can be quite a lot of them So we've been thinking about that and what we decided to do is to bring in the issues tab Now if we detect issues on your page You'll see this bar across the top with a button in the top right-hand corner there that says go to issues If you click on that it'll take you through to the issues tab Now it might offer you the opportunity to reload the page to get more information If you click on one of these items it'll expand and you can see more information there As well as potentially some links to content for you to read up on what you could do to fix the issue So that's the issues tab. The other thing we've been looking at is web vitals So if you go to web.dev slash metrics You'll see a whole list of metrics here that affect the UX and things that we would like to optimize as web developers And we've been looking at ways of exposing this information to you inside of the DevTools UI So things like first contentful paint or largest contentful paint for example So if you go to the performance tab in DevTools and you take a recording in the performance tab You'll see something that looks like this now. There is a Timings tab there or timings row. Sorry. I should say across which you'll see these blocks and these relate to some of those metrics So you can see FCP and LCP first contentful paint largest contentful paint and so on so you can start to get information there on some of your metrics The other thing we started doing is to add candy striping to your long running tasks And you can see that here I have one task on my main thread that is 70 milliseconds long And what we're looking for is we're looking for tasks to remain under 50 milliseconds This means that the main thread stays responsive and hopefully we can respond to user interactions quickly So as you look around your performance recording if you see this candy striping effect and the red triangle in the corner You know that you've got a task that's running longer than 50 milliseconds What we've also added as well is we've added a total blocking time footer at the bottom What this tells you is if you like the amount of candy striping that you would see across the whole recording So as you're looking around if you see that that number is going up You might want to take a look and see if you have a lot of long running tasks on your main thread Bringing that down should hopefully help your user experience Another thing that we've added is this experience track and what's contained within this is layout shift information So for example when you've got buttons and so on on your page and they're perhaps moving around This can cause UX discomfort So what we want to do is we want to minimize the amount of moving elements on the page And so the layout shift here is going to tell you what elements are moving on the page and where and So on and the size that they were when they did it. So if I look at this I have a warning here, which tells me that cumulative layout shifts can result in poor user experiences And that's a link to more information as well as information on where it's moved from and to and if I roll over that I get an overlay on my screen shows me exactly where on my page The shift took place you can also get live information about layout shifts by going to the rendering tab and choosing layout shift regions here in the options now I should say that for people prone to photo sensitive epilepsy This might be a less suitable option because it can cause flashing of overlays on the screen But it is there as an option if it's suitable for you The next thing I want to talk about is web assembly debugging. It's an experiment So if you go into your dev tool settings go to the experiments tab and click on web assembly debugging You can switch it on there What this allows you to do is allows you to do things like setting breakpoints in your web assembly code So here I've compiled a C program. It's just a hello world program But what I've done is I've added a breakpoint on the line that says hello world So when this code executes and it hits that line inside of the web assembly it pauses execution just like it would inside the JavaScript and you can see here in the call stack that I can actually take a look at What's going on in that particular frame and I can go between my JavaScript and the C and so on and so forth So that's something that's coming down and it's currently in canary. So take a look at that Now the last thing I want to show you is color vision deficiency emulation inside of DevTools And there's no better way to do that than to actually give you a demo okay, so here I am in Chrome canary and I have a video here running of me and so I'm doing supercharged Yes, the year but you see I have the rendering tab open in DevTools and I can emulate various vision deficiencies such as blurred vision Or I can do proton opiate I can do due to an opiate Proton opiate. Oh, it's trying to try to know here A chromatops here as well And you see the live effect that it has on the page So these are physiologically accurate emulations of various vision deficiencies now a vision deficiency Isn't an on-off thing like we see here, but rather it's a spectrum So a person could have a milder form of a vision deficiency or a more acute form What we've chosen to implement inside of the DevTools UI is the most acute form the theory being that as you're optimizing your app for Accessibility in terms of color and contrast if you make it work for the most acute form then you'll include everything up to and including that as well So that's color vision deficiency emulation inside of DevTools Thanks, Paul. I'm really looking forward to seeing more later today Now one thing I've noticed as we work from home is how seriously people are taking their home setups whether it be playing with mics and cameras or virtual backgrounds and Recently, I saw a demo that would make you invisible on your video feed using TensorFlow.js So I really wanted to learn more So please welcome Jason Maze from the TensorFlow.js team. Hi there I'm Jason and thank you for the introduction there. It's pleasure to be invited to the show and Yes, I have created an invisibility cloak. So maybe you want to learn more about that. Yeah, Jason invisibility cloaks are pretty cool And so maybe you can show us how web developers can create superpowers like that with TensorFlow Sure, definitely So if I switch to my slides for just a second you can see what the invisibility cloak was that the old man was referring to and in this Demonstration on the right-hand side you can see as I get on the bed The bed is deforming in real time and I'm being removed from the bottom frame at the same time And this is running all in the web browser Now this is pretty cool because privacy is preserved as none of these images are being sent to the server side And that's super powerful, especially in today's climate where privacy is top of mind Now this was created in just under a day in fact so it's quite easy to get started with machine learning in the web browser and we'll see some more demos in just a second So on that note, I also created a Chrome extension that allows me to use the same stuff We saw before I was actually using body pics to create that which gives me this image segmentation of my body in real time And I can now join a Google Meet meeting as you can see shown on the slide right now And this could be combined with my previous demo so I can remove that second person who comes into frame halfway through the GIF and then it would appear as if it never actually happened Cool. So can you give us a few more details on how this all works? So essentially all this is using body segmentation And this is running in TensorFlow.js in the web browser and this can Distinguish 24 unique body areas across multiple bodies in real time You can see on the right-hand side that this works pretty well when all the settings are bumped up to high And you can even get the pose estimation showing on the bodies to which estimate where the skeleton is These can be used in delightful ways just as clothing size estimation, which you can see here This is another prototype I created and I don't know about you Dion But I am terrible at knowing what size clothing to buy in my once a year clothing purchase So absolutely here totally and here you can see I just enter my height and when less than 15 seconds I can get an estimate of my inner leg my chest and waist measurements Which the clothing site can then use to estimate what size I am a small medium or large Now this can even give you superpowers as you see on this next light And one of our community members from the USA has combined this with WebGL shaders to turn himself into Iron Man of sorts And he can shoot lasers from his eyes and mouth using our face mesh model So this is pretty cool and it runs buttery smooth at 60 frames per second in the web browser and You can even go further of course There's many web technologies out there that you might want to combine with machine learning such as WebXR WebGL and use very defensive OGS And if you do that you can get an example like this from another one of our community members in Paris, France who can essentially Scan a magazine and if there's a person in it They can bring that person into the living room full-size so you can walk up to them and inspect them in more detail Pretty cool technology But of course after seeing this I thought to go one step further and if I add webRTC I can then teleport myself anywhere in the world in real time and This is using complete rewrite using webRTC a frame 3.js and TensorFlow.js together to create this demo And it really does make a big difference when I'm Seeing someone in my room which I can walk up to and move around It's a massive difference compared to a rectangle that's solid on the screen So this could be the future of videoconferencing who knows but it's great to play these technologies and push the boundaries of the web That's really cool from invisibility cloaks to teleportation That's that's pretty cool stuff. Yeah, exactly This changes everything is actually so how should web developers if we kind of zoom out a second How do web developers Kind of generically think about the role of ML and TensorFlow.js and how it could fit into their web applications Yeah, that's a great question and obviously right now in fact Machine learning in JavaScript is still a pretty new thing right the very early stages But that's super exciting too because there's so much potential to be unraveled at this time as well So on that note, I had asked web developers to consider how machine learning might fit into their existing pipelines Maybe you're developing a content management system in that case. You could potentially use something like Automatic image cropping to detect where a human face is in that image So then you can make sure that it's cropped nicely when you're resizing with your CSS or Maybe you want to summarize a blog post article So you have one paragraph of text that shows in the search results That's now possible machine learning too and that can be done automatically So I think I encourage people to experiment and go outside of the regular box of thinking and of course on that note On this side you can see all the different areas JavaScript can run on the browser server side mobile native desktop native and even internet of things and TensorFlow.js supports all of these environments too So maybe you want to combine it with hardware if you can recognize an object Maybe you can trigger something to happen in the physical world or something on the server side that talks to a third-party service and on that note TensorFlow.js can essentially run retrained by a transfer learning or even Right allow you to write your own models from scratch if you so desire Now on that note, we have a ton of pre-trained models You can use to get started such as the body segmentation you saw just a little bit ago But also things like pose estimation speech commands base mesh hand pose and some cool natural language processing And just to dive into that a little bit more you can see how these models work here So here's the object recognition in action. This class allows you to recognize 90 pre-trained Objects like these dogs you can see here and you get the bounding boxes that come back at the same time Which is pretty neat or what about this face mesh just three megabytes in size and you can understand 468 unique landmarks on the face and This could be cool for making a face mask or some kind of AR experience such as the one you see on the right Moddy face, which is part of a loyal group has actually used this for AR makeup Try on and this lady on the right hand side is not actually wearing any makeup at all In fact, she's selecting the color of makeup She wants to try on and she can do that all in real time in the web browser in a much more hygienic way Which is pretty cool and then finally I just want to talk about some of the client-side superpowers you get if we think about running machine learning in the web browser The first one is privacy as we hinted at before Essentially because we're running in the web browser None of that data is ever being sent to a server for classification So that allows you to access the sensor data in a way that is great for privacy Linked to that of course is lower latency as there's no server involved That means there's no hundred millisecond or so round-trip time from the mobile device to the server You cut that out completely by running on the edge and of course lower cost you might spend tens of thousands of dollars Hiring BP GPUs and CPUs to run the machine learning models on your server side environments By running on the client side all of that goes away because you're using the hardware of the client to run instead Got it. So how can people get started? You've definitely piqued my interest now. I want to know when I start playing Sounds good So if there's one slide you want to kind of screenshot from today's talk It would be this one here on this site you can find out our website our models get hub code We are open source. So feel free to contribute There's a Google group for asking more technical questions And of course some great boilerplates to get started on co-pen and glitch and for those of you want to go really deep We recommend the deep learning with JavaScript book. That's very comprehensive and even if you have no background in machine learning As long as you know some JavaScript, it will walk you through everything step by step So how do you recommend checking that out and on that note? I would also suggest checking out teacher machine right after this show maybe in just two minutes You can use this website to learn to recognize any object in your room in just 30 seconds You just take a few images of that thing hit train and you'll get a classifier that can then Classify that object if you like it you can then export this model to a JSON format and then use it on any website you like to be more creative and Makes your own creations using that and then finally I'd ask you to come join the community We've got this made of TFJ hashtag that you can use that allows you to share what you've made So we can find it and we can invite you to our future showing towels And of course allow others to get inspired by your great work too. So finally, I just want to leave you with one more inspirational example This guy from Tokyo in Japan is actually a dancer But he's used temporary JS to make this cool hip-hop video So machine learning really is now for everyone and I'm super excited to see how creatives will start using this and not just the academics musicians Artists and much much more So if you do make something do use a made of TFJ hashtag so we can find it and share it for you And I look forward to seeing what you make And with that feel free to stay in touch using the following links if you say desire on Twitter and LinkedIn Great. Thanks so much for joining us Jason. Yeah, and thanks to everyone who joined me on the day to kickoff So now let's get to today's sessions where we focus on updates across our tools and the web platform To make developers more productive as well as the latest in the world of PWAs Please enjoy the show and remember the team is here to chat with you on web.dev slash live and via YouTube I'll see you there today, and we'll be back tomorrow morning for the day three kickoff