 When I was first building the Preact site I had it kind of all client-side rendered. We weren't doing any pre-rendering or anything we are now but at the time I you know I did that and we weren't super concerned with SEO at the time which immediately vanished and I was definitely worried when we first pushed it live like is this just gonna be like you know a title on Google search with no body content yeah so I was kind of confused with that and then there was other weird things like we were doing client-side routing and in response to the client-side routing we would change the meta tags for the description stuff yeah no idea whether that gets picked up in Google search hello and welcome to another episode of SEO booth busting with me today is Jason Miller I'm really excited to have you here so Jason what do you do for work I'm a dev rel on the Chrome team cool so what do you work on I work on a bunch of APIs in Chrome and sort of test them out cool and you build preact as well right do you build preact on the side yes that's a cool thing so now that I've got you here let's talk about SEO and frameworks so I guess there's like a bunch of misconceptions about how frameworks and SEO come together and it would be a perfect opportunity to get them all with busted there's definitely a conception that search engines don't run in JavaScript just across the board wow and I'm not a hundred percent sure where that comes from I think maybe you know back in the you know in the early days of the web it wasn't a thing so I mean there's still probably cases where you know some other engines don't so maybe that's like it's pervasive because of that that makes sense that makes sense I mean we do still have crawlers and engines out there that are not executing JavaScript is there anything that you wonder or worry about in terms of discoverability yeah definitely so like if you build an app where the HTML payload is a script tag in a style sheet you laugh this is every that is true that's what happens yeah yeah I mean besides the the user implications that this has because obviously the browser then has to fetch more content and you look at a blank page for a little longer that also does affect the crawlers a little bit especially the ones that are not actually executing JavaScript I can speak for those but I can say that Google but does execute JavaScript so we do see your content unless there's some good reason not to see your content like JavaScript errors or network errors happening I did see in the in the CDS talk you did mm-hmm there was a new feature in Search Console that you guys were talking about for errors yes yes so I don't know if you've ever used the affection render in Search Console beforehand to get a screenshot yes to get a screenshot so what do you do when you get a blank screenshot cry yeah that was yeah I feel you so one way of working around that is to use the mobile friendly test which is the tool that we brought out it does what it says it does in the name like it tells you if your page is mobile friendly which is fantastic but more importantly is it also gives you a list of the resources that we couldn't fetch for whatever reason and it gives you the console logs all including sec traces what yeah so you can actually debug so you get a blank screenshot and you're like oh no but then you can go to this the console and see why so what we are doing right now is we are working on the new Search Console that also contains a thing that we call the URL inspection tool okay that's part of the Search Console and for every page that you have on your site that you verified for because we don't want to search for that data to the public yes show me Google search data so we try to have like one stop solution for pretty much everything right now that includes when we last crawled you if it's on Google or not if there has been any issues with I think mobile friendliness or AMP but yes but it's gonna be extended to like structured data and also to JavaScript errors I think so wait we mentioned like the features thing like what features can I count on being available in the crawler so we do parse HTML really yeah that's a that's a mind-blowing one we are stateless hmm so that means we are basically pretending to be incognito browser that opens the tap for each URL that we indexing the first time for the storage that kind of storage well we do store cookies but basically when we visit when we go from one page to another these cookies are cleared so you can't really rely on cookies to be available across we're not running WebGL okay yeah I don't think we are rendering what's on the canvas I'm not sure about that really search it anyway right that's that's the thing exactly everything in there is inaccessible to pretty much every user who's using assistive technology and you can think of a crawler as a user that needs assistive technology to understand the content I'm curious if let's say like this is a super biased question but let's say I built a website using a framework that ends in ACT one of the many so let's say I you know I built a website using whatever technology client-side rendering right and I'm doing like push-date routing using the history API does Google pick up on push dates yes yes we do so because you give us a proper URL right right you have example.com slash blurb slash listing or something like that so we do pick those up if you use hash-based routing we don't pick them up so please do not use a push-date routing history API is the way to go so if you use proper history API URLs however don't fall for a thing that I have seen a few people fall for you might test by basically loading your page from the home page and then clicking through we are not doing that remember we're stateless so if I go straight to slash something so if you if you use a URL and use a link to that URL somewhere in your page that's fantastic but make sure that people can enter right there. Interesting. So you may have to make sure that the serve understands how to serve this one way of doing it is doing like using a server that actually serves the index HTML for all routes and then the JavaScript displays the right content that is perfectly fine just don't give us a four or four I found it or something like that. Right okay yeah so like a bunch of the services like I'm super lazy and deploy all my stuff to CDNs yeah so they let you do like a 200 dot HTML and it's just like it returns that content for any URL it's not a file. That is fantastic I did fell for something once you might use like a static site thing that you can yeah like GitHub pages or S3 or whatever it doesn't matter they also give you the possibility to do a custom four or four page right and I have abused that to like serve all because I didn't know all my routes up front so I'm like yeah you know if it finds a route that it doesn't know it serves this 404 HTML. Is the status code 404? Yes and that's why it's not being indexed then. So it's an error page that is actually very useful. Yeah but you shouldn't be doing that because if you give that to a crawler and say like this is not a page then we are not picking it up because you told us not to. So you said like if I'm using HTML links or whatever link between pages that's fine. What if I'm using don't judge me for this a button and the button has a click handler and that click handler calls push state. I'm not saying I'm doing this. Okay. I'm not saying I'm not doing this. Let me guess a friend of yours does that. Right yes what's a name Bob Bob. Bob uses buttons and on click handlers. Poor Bob. You clearly do not do that. No I would never do that. I'm very happy that you're not doing that because your friend really needs to use links for that because links are to go from A to B button triggers an action. I think as you can use buttons and on click handlers for all the things that are like I don't know doing a pop-up or something because yeah modals or even like trigger a function trigger a calm down whatever if it's an action that happens on the page that's a button and that's perfectly fine but if it takes you to another piece of content and you want to make this content accessible via search and make it discoverable in search engines if it has a URL it should be a link. Okay. Right. So like if I had a modal that had a like the high value content in it or whatever and it right now it's just a button with a click handler that shows a thing on the page if I want that to actually have like a representation in the search results. Yeah. Give it a URL. Make it a link. Yes. What you can always do is so I like the idea of exposing some of the application state in the URL so you can basically say like and we support parameters so you could basically like I don't know slash action question mark modal or pop-up equals true. And if you use a link to go to that one and then that means that it displays the thing basically you have to understand that search results work like as if you would take the URL copy that into your messenger and someone else tapping on the message on whatever device you don't know. So basically they start a fresh browser have never been to your page. Do they see what you're seeing if that's the case then that's an indexable URL. Right because otherwise like if I had a modal and let's say Google somehow crawled it if you click on a link from Google search how does the link open the modal. That's the thing. Exactly. It has to it has to make it happen because if that's mismatch then what's the point. Another situation like that is infinite scrolls right. Right. Don't you hate it when someone sends you a link and then goes like did you see like that bird and then you look at it and it's like what bird. Oh yeah you have to scroll for a while. You have to scroll 600 meters down the page or 400 feet or whatever. Yeah I use metric system. Thank you very much for. I think more people should do that. So yeah if you scroll down like 600 meters on a page to find the content that's an issue to like surface that. If you could link to it if there's like an anchor on it. Now you got it. Exactly. If it's an anchor and specifically if you're using push state to like update URL and then allow the JavaScript to pick that up when you come back into the site and load the right content for it you're off really well. So it's almost like like you're doing virtual scroll but the Google search bot almost sees it as like next and back buttons. Pretty much. Yeah that's kind of like the modality that exactly you give it an offset and then we're like okay so now our JavaScript starts at like page five and then you're good and then Google bot comes and goes like oh so there's a link here with page five. All right so in that case it surfaces these 25 images the bird being the third one off we go. That's a good way of doing it. So let's say I was like driving a page with a worker. Like I have like a worker that's doing all of my state management or you know even doing DOM rendering or something crazy and in order to actually get like pixels on the screen I have to round trip through that worker and get all the HTML and DOM manipulation back onto the main thread. Is the fact that that might take a little while going to be a problem for Google bot? Generally speaking no. I'm definitely encouraging to test that carefully using the tools that we've got available to make sure that everything works the way you expect. But basically if you see the content in the so we the mobile friendly test also gives you the rendered HTML that we are using for indexing so you can use that to see like if it appears that normally if it takes a little bit of time that doesn't matter that much because we're doing wonky interesting things with time anyways because we have to deal with like a bunch of interesting things. You just take a lot. Exactly. Yeah. So don't worry too much about the time it takes. If it's a little fragile you will find out using the tools. The HTML that it shows is that like if I've done DOM manipulation is that after the DOM manipulation? Yeah. Okay. So it's like a snapshot. Yeah. And if you're not seeing it then something has gone sideways. Okay. So even if I didn't have the error reporting thing I mean I want the error reporting thing but even if I didn't have the error reporting thing I can see. You still have the rendered HTML that you can deduct from what we're indexing so. Okay. That works. I can live with that. That's pretty cool. Anything else from the top of your head? Is there any difference so like when I'm picking a framework or trying to figure out how I'm going to build my front end stuff? Is there much of a difference between like framework sizes and approaches for how those might get crawled? So when we're talking about like crawl budget which means how often and how quickly we can crawl the frameworks don't have that much of a difference because we are not like discriminating or preferring any of those and we're not like looking at we're not to looking too much at the size really. So generally speaking the more requests you do to get the content together the more requests we count towards like that we have done a thousand requests and if like each page of your content makes 100 requests that's literally like 10 pages. Right. So it's also a usability thing basically if you think about it being on a flaky network. Right. Not nice to get half baked content or like get no content at all and only the empty shell so it makes sense to look into a server side rendering or hybrid rendering to minimize the amount of requests that it takes because that helps you with the crawl budget sometimes. But that being said crawl budget is a really tricky topic because we cache stuff so if we have the content already in the cache it doesn't count towards it. Yeah like if your home page loads all this stuff and then like all your product pages or whatever are just hitting the cache maybe you're slipping under the crawl budget. That might happen exactly and crawl budget adjusts all the time anyways because we want to make sure to not overwhelm your server so. Jason it has been a fantastic conversation. Thank you so much for for being here and really looking forward to what you're building next. And thanks a lot for being on the show. Thanks. In the next episode of SEO Mythbusting, Dionne Almer is going to join me to talk about the web platform and SEO and all the road that lies ahead of us. So do not forget to subscribe to the Webmaster's channel.