 So thank you all for coming out on a rainy Sunday morning appreciate it so Friday I was in Berlin where Mozilla was having its twice a year all hands and I was thinking about this presentation and a question came to mind which is how many at the median tabs are open in a Firefox session Now there's the answer from Twitter And I'm going to ask y'all so all right 20 or more tabs are open put your hand up all right if 10 to 20 tabs are open Okay Ten or less so I have a rule do not feign surprise I'll take questions at the end because we're compressed for time Do not feign surprise. It's a rule that's used by the whoops and Okay, thank you. I couldn't find my little slide progressor thing. So there okay So do not feign surprise. So I'm gonna show you what my Reaction was when I asked a member of our telemetry team What the what the median was? to at the median a You have two tabs a person will have two tabs open now caveats on this If you're using a distribution on Linux like I'm that most of us in the room are You probably have telemetry stripped out. So this data is skewed towards our Mac OS and Windows users But anyhow, what I'm here to talk to you about is how do you ship new features in a browser which people want? Serve diverse groups and do all of this without being creepy. So what we had to do To think about this Well, I like I said, I'm not used to having I I couldn't find my little slide advance or widget No, I've got it and that just Today's been a thing today has been a bit of a day. Okay. Anyhow, so what we had to do the do this Is that we had to replace our we have to replace our process? Thinking of this with one that Sorry, that was biased towards our current our old processes biased towards people who were had the time and energy and Privilege to be vocal With what they wanted So we're going to talk about how we changed how we thought about features and How we changed it and today I'm just talking about desktop Firefox We're trying to do much more work on mobile Firefox Mobile of course is where the majority of browsing is occurring these days But we've learned a lot with desktop Firefox So just a bit about myself. I work at Mozilla My title is bug master, which is a fancy term for engineering program manager I work with the team that works on bugzilla that works on fabricator that works on Lando I am primarily focused on Policy process and understanding how we handle bugs I love coffee and I love ice hockey So let's talk about features So there's lots of sources for features one source of course is our standards process People are making updates to JavaScript. We're making updates to hg to html I know and all of these processes going through w3c. So that's one source of features The past is also is that we get features from talking to users of people who are using Firefox And we have a UX team We also have the ability for people to request enhancements One of our volunteers co-hay Put in this great feature last year that allowed us to determine Mark bugs his task in bugzilla features or Actual defects so we get enhancement requests now Bugzilla has votes and In the past it's been used as a way for people to prioritize work. The problem with votes is it's a rotten way to prioritize I'm going to go back to lucid Lou Villa. You may have seen him yesterday at the track on Libre and and law But back in the day on the gnome list He mentioned that votes end up having no bearing on an actual bug validity. It's importance or it's very So a lot of people could vote for a bug But it only affects a small number of users or just one distribution Whereas there could be a bug that has no votes But it's broken all of our users like what happened last summer with add-ons So the problem is this like you use Firefox you use ad lines You don't know where to go to say add-ons are broken. I can't load them. I get the certificate error What the heck is going on on? But we know that that's a high severity issue We all lost a lot of sleep that weekend. So how many of y'all familiar with? Hitchhiker's guide to the galaxy So you know this story. This is something at core doctor a photograph that Corey Docker. I'll take it I believe University College London So you remember the beginning of the story is basically this guy named Arthur Dent who has this friend named Ford prefect Who is very strange because he's named after Carr that was sold in England in the 1970s and 80s and They wakes up one morning and finds out the earth has been scheduled for demolition and he goes and asked well Why what what we didn't know about this you didn't tell us about scheduling to destroy the earth for a hyperspace bypass and The aliens who are running the who are running the demolition fleet say look dude We had a notice up. It was an Alpha Centauri You just could have just gone over to Alpha Centauri gone to the basement of the office And found the demolition permit At the same time Arthur's house is about to be demolished and he asked the guy who's about to do it saying well, dude It was in the It was in the it was in the In the council office in the basement in the room that said where the leopard so long story short is People are not in the same places where we make decisions And this causes concern this causes this causes worry and stress And a lot of the time privilege determines this so privilege can be here is just I know what bugzilla is I know how to log into bugzilla. I know how to read a news group. I know how to read a mailing list List, you know, these are things that we as developers take for granted But everyday users don't so you have to know where do you find the discussion? Do you have time to be involved in that discussion? Think about the question the question that simple brought up is like get should github be your resume and Github is your resume. It means that you have all this time to work on projects in github But you could be a perfectly fine developer raising two kids at home With no time to work on side projects and nobody will know about you because you don't have a github repository that shows you working all these cool open projects So if you do enter the discussion will you be heard and also if you enter the discussion will you be harmed? Few years ago. I was working at Linden lab Linden lab open source the second-life web client or excuse me a second-life client second-life client notoriously difficult to build and A person on the open source mailing list mail saying what can we do to clean this up? What can we do to make it easier for people news to the project to build? Second the second-life viewer and a person immediately came in and said how dare you? What what I suffered through this? Why should anybody else have an easier time of it? And so that that's what I mean by asking a question and not getting harmed not getting yelled at I want to point out some work that Dylan Did in the audience he used to work on bugzilla for us. He's now over at Fastly and He did a lot of work to make it easier to build bugzilla You can run it in a container So that's the sort of thing that we want to do. So sorry to embarrass you Dylan Okay, so we're working on time so Infrastructure so let's talk about infrastructure for making efforts for deploying changes So Firefox ships on three release trains last year was six weeks now. We're doing it every four weeks There's a new Firefox coming out on Tuesday So I'm going to be very busy on a train between here in Paris on Monday trying to figure out Making sure that every all the switches are set for release So we have a nightly we're a feature where new features land beta it goes through goes through that and then it goes out on on on release and A lot of the time we used calling a lot of the time we use preferences Which you've seen in about prefs To manage that so here we go. So we land up each our nightly it goes on the beta it goes on the release boom It crashes and release remember we're releasing across all kinds of combinations of operating systems and hardware My friend charity natures talks about you're always shipping or you're always testing and release whether you want to or not So what happened when we did this well normally what happens then is we have to spin up a new release and That's expensive to spin up a new release. We have to back out to change We have to test the back out Then we have to get it to users and there's no guarantee the users will pick it up So anytime again you send a relations you are testing in production. So we talked about experiments At Mozilla has a way of getting around this and our first thing was using this thing called test pilot So test pilot was using the add-ons mechanism to try out new features in Firefox So again, this is great. It's opt-in If you want to try it However, you have to know what add-ons are You have to know how to install an add-on you have to understand that an add-on Changes the user interface of Firefox before you use it you know in order to use it, so We ended up picking up a Bias sample of users this way So we're picking up people who know Firefox. Essentially people kind of like us. Maybe not developers, but still Experts, thank you. So again, you have to know about it. We're the lover So you have to change how we think about features So this is a sticker by Laura Thompson who now runs our browser futures research at Mozilla and She referred to experiments as cupcakes So cupcakes, you know You can make a whole bunch of cupcakes and then you can experiment with toppings and fillings and frostings Because you don't want to make a chocolate cake and then do an anchovy frosting and throw away a whole cake It's just better to throw out the one cupcake and then try the next one with you know chocolate frosting instead of anchovy frosting Then we also grew our product team So we actually have people taking ownership of these of making decisions about the browser about the future of the browser about features And then we have a data science organization now When you bring in a data science organization people get immediately start asking questions about Telemetry and data from the users I'm going to defer you to our data collection Principles and this guides all the work that we do because we know we need to understand how people are using the browser But we also cannot Abuse that trust and then we have to change our tooling the ship code So remember when we said that we had things like turning on feature turning on features with preferences And the only way I could update the preferences. This was the ship a point release What if and I'm going to use a D&D metaphor here What if your browser is walking down the hall of the dungeon and you've got your sword and you got your torch And you got your thief and you got your wizard and you got your tank and you come along and then instead of a wandering monster there's a wandering feature and the referees ask Hey, you come across a wandering feature Roll on the encounter table So you roll the dice So what we do now is we say instead of are you a thief? Are you a rogue? Are you a wizard? Are you a cleric? Are you a paladin? We say are you Linux? Are you Windows? Are you Mac? Are you in an English language reason? Are you in the EU? You do you have a co-processor? Do you have a GPU? You and that tells us what table to roll on and Then we'll turn on the preference so what we're doing is Every few hours your browser if it's enabled We'll go back and ask hey referee Are there any wandering features on the encounter table for me to roll against and it says yes Roll on this table and you roll it rolls a die randomly and says says okay. I rolled six Okay, you get the feature so notice that we don't have to keep track of Who of who you are we just simply probabilistically assign you to features has we're going along So we so we So we apply the preference So most of this is mainly being done for you. I features Features because it's pretty easy to turn this stuff on and off at the feature level At the turn to turn off you turn on on on enough JavaScript We're figuring out how to do this for features that are in rest in C++ Because normally what we have to do now is turn on we hit we have to restart your browser to do it So what does this enable you to do so I? can get 90% of my target audience I can deploy a feature in in 72 hours Hours if you can talk to our servity to our preferences server, which we call Normandy you get the update You don't have to wait until you know You don't have to wait for a point release Release rolling back means all I have to do instead of backing out code and retesting and responding all this stuff is I just turn it off the feature and Most importantly, we are sending rules not code. It is very scary to send code to a browser Because if your code server sending code gets Hacked then I have poisoned my entire user population. So address much much rather send a preference So we also do some stuff around safety. Okay Preference server and client are designed to fail safely and then major releases always restore restore safe defaults So we do consistent prep flipping again. I thought from what I described with the library We call that that was developed by another one of our engineers years we Have a tooling site for it called experimenter that we use for managing Experimental preflips as well as production preflips So when we did firefox monitor last year Which we did in cooperation with have I been pwned Pwned this enabled us to really quickly turn on this turn on this feature for a select few people Understand how it was working and then decide okay. We'll go ahead and roll it out to a broader audience and again for this To work for for firefox monitor work all you had to know is what site they're looking at So we just basically look at it, you know, there's a there's a reference table We look at to anonymize this so we can just say hey you're on xkcd The forums have been hacked make sure you update your password and then the product process behind this is You know just going through saying which of these features you want and people said hey We want to be notified about site site releases So then we constructed an experiment to see hey will this work? Okay three minutes, okay? So doing this experimentally enabled us to get half a million people or near or 600,000 people nearly in the United States to try this out So something up What we had to do is recognize the biases that our product I had remember when I asked you again The first question is how many tabs do you have open and it turns out it's much less than you think it is Yes Start with the information that you have start with what we had Make it easy to update the products make it easy to fix and make the easy process easy to follow And again Tatiana Mac who's a great designer and developer pointing out trust your users lived experiences over your own presumptions so That that's why as much as we want to put all of this cool stuff in for tabs There's other problems that we need to work on to help our users and Remember in Firefox We have the add-ons mechanism So that gives us a great place for people to do those Customizations to support power users because if you end up supporting your power users You end up leaving the rest of your user base behind You don't want to just lock out your power users and say none. I'm not helping you, but what you can do is Create a mechanism that does that so that's why we have add-ons Mozilla.org Or I don't know if anybody from the add-ons team is speaking this afternoon Okay, but anyway So you can't data science your way to supporting diverse groups of users alone. You do have to talk to people You do have to advocate for users for preferably diversity reflecting your users. So that just means not just disability its language its background all of these things and Then you know and asking the users from those diverse backgrounds the questions and then validate and then validate your work so Thank you I have Mozilla stickers if you'd like them and I will be happy to take your questions out in the hall Thank you, and I you can we can do questions There's no problem. We can take five minutes to take questions if you attempt to stay a bit longer and