 All right. Welcome, everyone, to today's Google Webmaster Central Office Hours Hangouts on Air. My name is John Mueller. I am a webmaster trends analyst here at Google in Switzerland. And part of what we do are these Office Hours Hangouts with webmasters and publishers from around the world. Maybe first off, I wish you all a happy new year and hope that this year will be fantastic and even better than last year. And hope that soon you'll have data in Search Console to compare this year to last year. So that update should be happening in Search Console over the next couple of weeks. So some of you might already have it or have seen it for some sites. And it should be coming out for all sites, I think, in the next couple of weeks. So usually we roll these out slowly just to make sure that nothing breaks along the way, that everything works smoothly. So some of you might need to wait a little bit longer. And some of you might have a little bit earlier looking forward to seeing all of your feedback. As always, if there are any of you here who want to get started with the first question, feel free to jump on in now. Thank you, John. I have a question which is related to the XML sitemaps. So on a video XML sitemap, just like the classic URL sitemaps, the video links often contains 301 or 302 redirects because the clients either tend to use shorter versions of those URLs or those websites have switched from HTTP to HTTPS. Some things also applies for images. I know that standard rules are to have the final URLs placed within the XML sitemaps. Well, in order to search engine to index those videos, how about the videos? How is Google treating these redirected URLs to be found on XML sitemaps related to videos or images? Are you specifically talking about the landing page for the images and videos? Or is this like for the video files themselves? The landing page where the video resides, because on the XML sitemaps, usually, if you cannot link directly the video, because it's a CDN, you will link the page which resides the video. And when you click on the link, it will open the page, and the video will start playing automatically. OK. So in general, especially for the landing pages, so those would be pages that we would pick up with the web search crawler as well as a landing page for a video or an image. We do recommend to use the final destination URL in the sitemap file. Part of the reason for that is so that we can report explicitly on those URLs in Search Console. So especially in a new one, you can select the sitemap file, and you can look at the indexing information just for that sitemap file. And that's based on the exact URLs that you have there. The other reason we recommend doing that is that we use the sitemaps URLs as a part of trying to understand which URL should be the canonical for a piece of content. So that is the URL that we should show in the search results. And if the sitemap file says one URL and it redirects to a different URL, then you're giving us kind of conflicting information. You're saying, well, this is the one I want to have shown in the sitemap file. And that URL itself says, actually, I want you to choose this other URL instead. So we're in this situation like, do we trust the sitemap file? Do we trust the redirect target? Is there maybe a rel canonical that's even different? What is the internal linking like? Does that match any one of these? Or maybe even a third? And all of that makes it a little bit trickier for us to pick the canonical to show in the search results. So for ranking, that doesn't matter so much. It's not that we would rank these pages differently, depending on the canonical that we choose. But maybe you have a strong feeling and say, I want everyone to use this nice-looking URL instead of this ugly-looking URL. Then that's something that you should tell us with the sitemap file, with redirects, with internal linking, all of that. I understand. The thing is that these videos are the product videos. If it's a house, it's a presentation video of the house. And usually, in the page of the website, in the normal page, the video is linked with the short URL. But within the XML sitemaps, you say that we should use the final destination URL to reach that video, correct? Yeah. And in this way, Google will know to append this video to this property or whatever. Yeah. So that's, I think, for ranking point of view, it doesn't matter so much as long as we can crawl those URLs and find the right ones. But for indexing, if you do something like a site query, then you might see a mix of these different types of URLs because we're not really sure which one you actually want to have shown in search. So if you have a preference and you say, I really want everyone to look at this URL because it's the nice version and this one just has a code in it, then that's something you can tell us there. OK, I understand. Thank you. Sure. All right. Any other questions before we jump into the submitted ones? No? OK. Maybe some more questions will come up as we go along. Not for me. All right. Go for it. Hello, John. I appreciate everything what you do for all community for last year and for all years. And I just want to say thank you for everything because this is very much for us very important everything. My question is about the voice search because I see the data is very important. I see all crawl is very everything is very important with the voice. This is the future. But the most important thing here is I see my daughter, a six-year-old. She only searched with voice. This is the next generation. And my question is, what do you think for some recommendation for everybody in the SEO community for technical voice integration or for content strategy like semantic? What is more important or combination of both of them? What is most important for voice? I think that's really complicated because from Google's side, what we try to do is to understand your pages best and to figure out which type of voice queries match those pages. So that's something that you can help us with using structured data on the pages if you tell us a bit more about what this page is about. Something you can perhaps also tell us if you have information that could be combined into a voice snippet, that might be useful for some kinds of content that doesn't make sense. That's not possible. If you have a question, then the answer is this big thing or a table or a list of links. That's not something that really works with voice. But I think the general directions where we try to figure out what content fits well for voice. I know for some other kinds of voice assistants, they try to match the question more directly. So they're looking for maybe web pages that say, I don't know, what is the tallest mountain as a title? And then they read the first paragraph. I think for Google, that's probably overdoing it and quickly ends up in a situation where you basically create a doorway site with all of these question variations and a short piece of answer. And the pages themselves have really low value because themselves, they don't have a lot of information. They're just targeted for this one specific query. And that's something, so from that point of view, I think that's very short-sighted if you were to go in that direction. So I really focus more on trying to make it so that Google and other search engines can understand the context of the information a lot better. And to make sure that your content is written in a way that can be read aloud, which I think is a general guidance anyway. So the old school kind of keyword stuffing, if you read it out loud, it doesn't make that much sense. It sounds really unnatural. But if you write naturally and you write in a clear kind of language that's consistent across your website, across the type of queries you want to target, then that's the type of information that we could pick up for voice as well. I think the other thing that just recently happened is we sent out an email for a bunch of voice actions where we noticed that I think podcasts and news content, in particular, are things that tend to be used a lot with voice assistance. I don't have the details there, but that might be something if you do have podcast content or news content that you have available in AMP format or whatever format that might be something to look into as well. Either maybe you receive this email directly where you can verify your profile, or maybe that's something where someone else has already written a blog post about the details on what is happening there. But I think this is a topic that isn't going to go away. But it's also a topic where it's still very early days where we don't have a simple meta tag that you just put on your pages, and everything works for a search. There's still a lot of work to be put in from our side, from the ecosystem in general, from users. Like, what do they expect from search? How do they expect search to work on voice? All of that needs to be figured out. The other thing, if you're technical, if you like dabbling with programming, you can also look into the voice assistance things where I think from Google, from Amazon, there are a lot of really detailed examples and kind of APIs that you can use to create a kind of a voice assistant type thing where I think it's probably also still very early days, but it's something where if you think about this for a little bit, then maybe you'll run across a client at some point where you see, oh, the voice assistant would be the perfect compliment for their service. And then in that case, that might be something that you could put together. There are a lot of really neat services that make this a lot easier. It's still pretty hard and kind of complicated to get set up, but it's something where I think when you run across the right situation, maybe you find something that's like a perfect match and that's something to kind of have in your repertoire as something that's available. OK, thank you. So I guess that's a long answer for I don't really know, but there are lots of small things that you could be working on. And I think it's good to think about this already, but it's still pretty early days. Yeah, it's pretty early, but we'll become. Definitely. Yeah. Thank you, John. Sure. All right, let me run through some of the submitted questions. And as always, if you have questions in between, feel free to jump on in or if something is unclear, just let me know. All right, the first one is about hreflang. So if a site has hreflang implemented via the on-page markup and the XML sitemap, which one would Google pay attention to more? From our point of view, these are equivalent. So if you have hreflang markup, which is a way of letting us know which pages are equivalent across the website for different languages or different regions, you can specify that either with a meta tag on the page, with an HTTP header for the page, or with the sitemap file. And for us, all three methods are the same. So it's not that we say your site is better if you use it like this or like that, but rather we have different methods because people use it in different ways. And we want to give you the choice to use whatever makes sense for you. And with regards to which one would Google pay attention to more if you use multiple methods, I think that's something that you need to fix on your side in the sense that Google shouldn't have to make up its mind. It should have one consistent guide for this markup. You shouldn't say, like, on one version of the markup, it says this, on another version it says that. Because if it's inconsistent, then we don't know what to do. And we can't promise you that we'll do it either one of these ways. Maybe we'll even ignore both of them if we think we can't make up our mind here. So you can use whichever method you want as long as you make sure that you're not providing any kind of conflicting information there. On December 15, many official celebrity websites dropped in rankings. And prior to the 15, many were ranking number one. And afterwards, a bunch of sites dropped. What happened here? I don't have any detailed information on what specifically changed with the search results on that specific date or with those specific sites. In general, we make changes in search all the time. And these are kinds of changes that I think can happen over time. And it can also just reflect how we think users expect to be served in the search results. So sometimes if you're looking for official celebrity websites, maybe people are searching for those celebrity names and they don't necessarily want the official site. They want maybe a news site about the celebrity. Or maybe they want background information about these people. And if that's what we think from our algorithm point of view makes more sense, then maybe we should swap those rankings around. Just because something is the official site doesn't mean we should always show it number one. Maybe kind of information about that from a third party is actually more relevant to individual user needs at some point. So that's something where I wouldn't necessarily say anything went wrong or we did this incorrectly, but rather it seems like something that could just be a normal change in search, like we have normal changes in search all the time. The Google guidelines recommend using no follow for sites you're not sure about vouching for. Are there any plans to discourage websites from no following every outbound link? If not, how do you imagine that would be affecting Google's algorithm going forward? So kind of as alluded to here, I don't see us changing our stance in that regard in the sense that we would capitalize a site that is not linking to other people's websites. I think that's totally within the right of every website to decide for themselves how they want to treat links. And some of these might choose to no follow some links. Some of them might choose to no follow all links because they're tired of trying to figure out which one of these links is actually a paid link that got snuck in through a guest contributor or something like that. I totally understand it from the side of these websites that are having problems with the linking from their website, especially if they're multiple people working for content on a website. Then that might be hard to figure out, which one of these are actually natural links and which one of these are links that are purely there because of some connection with whoever is writing and the other website. And on the other hand, I also understand that this makes it a lot harder for those of you who are trying to kind of gain natural links for your website as well, because maybe you're featured in one of these other bigger websites in a natural way and they don't link to you because they're afraid of linking. From our point of view, I think that's something that our algorithms just have to deal with. I don't see us kind of discouraging websites from doing that. I think that's within the right of every website to kind of figure out how they want to create the content on their website and how they want to link to other websites. And sometimes they do it in a good way. Sometimes they do it in a way that doesn't work that well for users. But ultimately, that's kind of their decision to make. For us, it's only a problem when we see sites that are using unnatural links in a way to pass PageRank to other websites in a way that we think wouldn't comply with our Webmaster guidelines. So that's something where I think the web has been evolving, therefore, quite a long time. So when Wikipedia did this first, I think that was over 10 years ago. So it's not like something new that has just suddenly happened, but rather maybe there are other sites that are noticing that this is a problem and kind of not sure how to deal with this. Maybe they go to this extreme of no following every link and maybe in the end they'll settle down into kind of a middle area where they figure out, well, these are actually good sites that we do want to link to. These are contributors who are posting content on our website in a really good way that we can trust. Then that's something that could settle down over time. We're trying to use web tools in order to analyze our sites regarding to the better ads experience. We added the properties to the tool, but they're not scanned yet. Is there a way to initiate a scan manually and to get the results so that we can prepare in advance? As far as I know, there is no way to trigger a manual scan. That's something that we do periodically across the web, and we don't have a way of kind of like triggering a manual scan of a website, because it's still pretty involved and not something that we can just purely automate. There is a help form for the better ads experience. So if you're unsure about your site specifically or about specific techniques that you're using on your website, I would definitely go there and post. I noticed that the team members on Google side who are looking into this, they've been posting regularly and replying to pretty much every thread that is posted there. So chances are good that if you post a good question and you give some details around what you're trying to do or what you're currently doing or what you might do in the future, that you'll get some good advice from the team there, or at least from other webmasters as well, who are doing similar things. So hopefully that helps a bit. My question is about links. I saw a couple tweets where it said not to disavow spammy links on a website. So how do we avoid getting penalties for unnatural links if we don't keep our link profile clean? I think this is something, which is a tricky topic, of course, in the sense that we have algorithms on our side that try to recognize problematic links and just ignore them on our site. And that's something where we're trying to do the right thing for the large amount of users out there who really don't know about the disavow links tool, who don't know about paid links in general, who don't really have all of the details that a lot of the SEOs who are listening to this probably have. So for the general public, we really try to keep things as easy as possible and to avoid the situation where you have to know all of these arcane details in order to be visible in search in a natural way. So that's why our algorithms try to figure this out on their own. On the other hand, if you do know what you're talking about, if you do know what is happening here around links, if you've paid attention to your website and you know a previous SEO has been doing really sneaky things with your website with regards to links to the site, then that's something you can take action on, where you can use the disavow links tool to really manually tell us these links are links that I can't clean up and I don't want you to take them into account with regards to the algorithm. Or maybe these links are things that a competitor plays to my site and I'm worried that perhaps they have a negative effect. And you can disavow them and just tell us all of the links from this site to my site. I don't want Google's algorithm to even consider taking into account. And that way, you kind of have the problem narrowed down quite a bit. Google can still figure out which other links might be out there that it should be ignoring, but at least the ones that you're aware of that you're worried about, they're definitely taken out of the question. So when it comes to manual actions for links, that's something you can use to disavow links to clean this up. When it comes to kind of this worrying situation where you see those links and various linking tools and you're like, I really don't want Google to get the wrong picture about my website. I don't want to be associated with these links. Then you can just drop those into disavow links tool and we will take them out. And it's not the case that the disavow links tool is an admission of guilt or is in any way a bad thing. It's really just a purely technical thing from our point of view where you're saying, I know these links are pointing at my site, but I don't want you to use them. And we're like, OK, fine, we'll just ignore those and kind of focus on the rest. Wondering if we can expect any significant changes to the Google Search Console API with a new Search Console rolling out. When will the updated documentation will be available? Asking as we have internal data engineering resources building out our tool sets. So I think, first off, it's awesome to hear more people using the APIs and building out internal tools to use the data and the APIs. At the moment, we don't have anything to announce with regards to Search Console other APIs coming into kind of the API tool set there. I think the one thing that will be happening probably in the next couple of weeks or in a month or so is expanding the existence. Search Analytics API, so the existing API just to provide data from the longer time frame that we have available now in Search Console. So that's the one change that will definitely be happening with regards to other APIs coming to Search Console APIs. That's something I'm not aware of anything specific that we're going to announce at the moment. But if you do have recommendations for things that you definitely want to have in APIs, then feel free to let us know. Maybe post in the Webmaster Health Forum or send us a tweet so that we can pass it on to the engineering team here. Should I no-follow all links to a no-index page? Before you said, over time, Google will stop following links from a no-index page. So if I link to a no-index page, am I putting my link juice into a black hole? From our point of view, I recommend just linking normally within your website and not trying to use kind of these page rank sculpting techniques to try to funnel page rank through different parts of your website. So we've got a lot of practice dealing with normal websites, understanding normal websites, and figuring out which parts of a normal website are more important and less important. And for the most part, that's something that kind of just works. So it's not the case that you need to no-follow links to no-index pages or anything like that. We can crawl those pages. When you see the no-index, we can drop those pages from an index. When we process that no-index, that's perfectly fine. Can Google release a criterion for the top stories box? Is it only available for very large sites? What about mid-sized news websites that work very hard to meet your requirements? What can we offer there? At the moment, I don't think we have any kind of public guidance on the top search, top stories box. So it's essentially just an organic search results element like many others. And we try to figure out which of the pages that we have and our search injects are relevant to users, especially with regards to the top stories box is mostly kind of newsy type content. So that's something where we try to figure that out. But it's not the case that it's limited to very large websites. I see lots of small websites in there as well all the time. With regards to getting featured there, that's something where I think it's less an issue of purely meta tags type things that you need to put on your pages, but just generally us understanding that your page is about this topic that is currently something people are searching for, and it makes sense for us to show your pages in the search results there. So having essentially a clean website, clean pages with clean HTML that we can pick up quickly, that's kind of what we're looking for there. And it's more a matter about the content rather than kind of the form factor. How well does Google manage cross origin resource sharing? I don't really know. So I looked into this very briefly before the hangout to figure out what exactly I'm not aware of here. But I don't really know how this applies specifically to search. So that's something where I'll look into this a little bit more afterwards to figure out what I'm missing here. But also, if there's something more specific that you're worried about in this regard, maybe with regards to rendering or maybe with regards to kind of general crawling of a website, then feel free to add a question to the next hangout with a little bit more detail so that I understand what specifically you're kind of worried about or looking into. We're seeing Google Antsbot eat up a large percentage of our servers with over 2 million pages crawled per day. Does the Google Antsbot share data with other Google bots to optimize crawling? As far as I know, we don't share data across those two. Usually that makes kind of sense because you can have URLs shown in search that you don't necessarily use an ad. And likewise, the other way around, you can use URLs in ads that aren't necessarily shown in search. So as far as I know, those are kind of handled separately. One thing where I have seen happen with some sites that have talked about this in the forum, where I looked into the details with the team, is if you use a lot of tagged URLs within your ads, for example, if you set up some ad campaigns that essentially have dynamic parameters in the URLs, then that's something where we'll go off and try to crawl all of these URLs before we actually show them to double check what is shown there. And it might be that all of these URLs actually need to exactly the same content. So just because you're using all of these different tagged URLs that go to the same page doesn't mean that we'll skip crawling all of them and just focus on that page. But we'll actually try to crawl all of these. So if you have something essentially like a session ID that you're using in the ads there, then we'll go off and crawl all of these variations. And it could be millions of URLs that we're trying to crawl. Just to double check that for whatever the ad system needs to check there. So that might be one thing which you can look into where you can say, well, do I really need to use all of these tagged URLs? Or can I reduce it to a smaller set of URLs or a smaller set of tagged URLs so that I don't blow up all of crawling just to follow up on all of these URLs? Or if I do need to use a lot of tagged URLs, is there a way to otherwise limit the number of URLs that I expose in this way? So those might be some things that you could look at there. John, I have a question. Can I ask a question in the middle of your run? John, I have two questions. So I already submitted feedback on the new Search Console regarding this. But I have an additional question to that. In the new Search Console, there's this new index coverage area. And inside you have stuff like submitted URLs that were not selected as canonical. For example, if you go to the excluded URLs part that you have there. Regarding this, my feedback I already sent because I think it would be useful to give us, OK, if my URL was not selected as canonical, which one then was to try to understand why what is being seen as canonical? And my question then arises, as can a URL appear there and say that it was not selected as canonical and not have any canonical URLs associated with it, that you just excluded the result from Search? Yeah, I think there are two things there. I believe I don't know if this ended up being in the final version. At some point, we had it so that when you click on those URLs, you would go to an info query or a site query for that URL. And usually, if you do an info query for a URL that's not selected as canonical, where we have a different canonical, instead, we'll show that chosen canonical in the search results. So that's something you could do on an individual URL basis to get a bit more information about the URLs that we didn't select as canonical. What you don't see is why we didn't choose that one as a canonical. I think that would be useful, too. But it gets really complicated very quickly. So that's something that's a bit trickier. With regards to URLs where nothing was selected as a canonical, usually that's more of a two-step process, in a case like that, where first we run across those two URLs. And we say, oh, this one or that one. And we say, OK, we'll choose this one. And then at some later stage, we say, oh, well, actually, we don't even need this one anymore, either. So we drop that one afterwards. So it's not the case that we see, oh, two URLs. We'll choose this one. And we'll, oh, no, we don't want that one. At the same time, it's like a secondary thing that happens afterwards. So that's something where it's possible that we don't have a canonical for a specific URL. But it's not the case that we make the canonical decision and say, oh, we don't want to index anything from this. OK, thank you. But I think the whole canonicalization process is something that seems very simple. At first, when you look at it, you're like, oh, sure, Google has two URLs and it just picks one. But it gets really complicated when you look into the details. I see plenty of people shooting themselves in the foot with canonicalization. And that's why that's one of the reasons I am asking, because if it says there that, oh, we saw another canonical and one, you don't give me the example that was often of what you see as canonical and B, there is no canonical, then that would kind of add to the confusion a bit. But yeah, I already did my homework. I submitted the feedback there. So that's great. That's what I can do. Yeah, I think it's tricky. It's something that we've noticed over time that people have problems with when they say, like, I submit this URL as a sitemap. And then at the same time, it has a real canonical somewhere else. And we don't counter this index. And people might say, well, I put it in a sitemap and it's like, you indexed the content. It's like, why don't you counter this index? So we wanted to provide some kind of information about what is happening in between. And I think it's tricky to get the right level of information in there. It's something where it's tempting to simplify it a little bit too much. And then the critical part is kind of dropped away. Yeah, I understand. Thank you very much. John, comment. Hi. Sure. Hi, John. Yeah. So regarding the same question, basically, I'm also seeing this one of those section, which is the excluded URLs, which it says Google choose different canonical than user. So when I'm checking this one of the list and one of the URLs which I'm picking something, it is like showing the same URL in the search results as well as the canonical is also the same. Like, I do not understand, like, why is it coming into the excluded list? So probably there's a small difference between those URLs that we have listed in the list and which is actually indexed as. So that could be something like on dub, dub, dub, or non-dub, dub, dub, or HTTP, HTTPS. It could be the trailing slash or no trailing slash. All of those things could add up and say it's a different URL. And it looks like the same URL at first glance. But from a technical point of view, we treat it as a different URL. So what I would do there is perhaps take a screenshot of that and copy down the URL. And post it in the Webmaster Help Forum so that someone else can take a quick look. And maybe it's the case that you looked at it correctly and we just indexed it. And at some point in the past, it was like in this weird state. And now it's OK. That's kind of good. Maybe it's the case that it's actually correctly flagged. There might even be that there's a bug on our side that someone needs to escalate to the team. And that's something that can be done in the Help Forum as well. Usually one of the TCs will ping us and say, hey, there's this weird question in the forum. You should take a look. And we'll take a look and pass it on to the team. But there is something that we need to do there. Sure. Sure. All right, we'll do that. Sure. Cool. All right, let me grab a handful more questions. And then we can open things up for more questions from you all. We discovered over 40,000 product pages not indexed in Google. We submitted these as a text site map. Google indexed about 4,000 over a two-month period. Why could this be? That's tricky to say without more information, actually. So we don't guarantee indexing. So just because something is in a site map file isn't a guarantee that we will actually index it like that. That's kind of the first thing there. So it might be completely normal. That we don't actually index all of those pages. The other thing is with a text site map file, you're basically just submitting the URL itself. You're not giving us any metadata. So in particular, what would be useful is actually the date, the last modification date of some of these URLs. Because with that date, we can figure out, do we need to re-crawl these URLs to figure out what is new or what is different on these URLs? Or are these old URLs that basically we might already know about when we decided not to index? So what I would recommend doing there is creating an XML site map file with the dates, with the last modification dates, just to make sure that Google has all of the information that it can get about those URLs. And then to let that be processed. The other thing I would recommend doing there is double checking the URLs themselves and double checking how they're actually indexed in Google. So it might be that we don't actually index the URL as you listed in the site map file, but rather a slightly different version that is perhaps linked within your website. So like I mentioned before, the trading slash is very common, or dub, dub, dub, non-dub, dub, dub on the website. All of those are technically different URLs, and we wouldn't count that for the site map as being indexed if we index it with a slightly different URL. So those are kind of two things I would look at there. But I also keep in mind that even if you do everything technically correct, there is no guarantee that we will actually index everything. And oftentimes, there is no absolute need to actually go off and have everything indexed because if we figure out that it's not worth indexing these pages because we probably wouldn't be able to send you traffic anyway for those pages because they wouldn't show up in the search results because they're not that important, then that's something where it doesn't really matter to you if these pages are indexed or not. It doesn't change the traffic that you get from search. So in cases like that, the best thing you can do is kind of make sure that technically you have everything right, but then spend the bulk of your time actually working on your website to make sure that it's really good, that it's something that schools algorithms when they look at it, they're like, oh, of course this should be the first one in those social results. It shouldn't be together with all of these others that are kind of the same, like the same product page across 1,000 different websites. We don't need to put them in that book. We can just say this is really the most important page on this topic on the web. And if we don't rank it number one, then we're doing the user a disfavor. So those are kind of the directions I would have for there. Wanted to know what scenarios Google's showing other domain URLs when searching for site command for your domain. Is that something to do with shared hosting? So I'm not exactly sure what specifically you're seeing there. Having more exact example would be very useful there. This is probably something also where maybe posting in the Webmaster Help Forum can give you a little bit more information on what might be happening there. One thing I sometimes see is people putting a space after the colon from a practical point of view. That means you're just searching for those two words. And that could result in a bunch of different URLs. But maybe there is something specific that you're seeing with your site that I'm not aware of. So that's something maybe posting in the Webmaster Help Forum will give you a little bit more information. John, this question was from me. Basically, I've already posted in the forum itself. And I've seen it's not something on the first page. It's still when you're going, let's say, the third and the fourth page of the Google, then you will start seeing this on other domain, which is, I believe, having the same IP address which we have for my hosting. So I'm just thinking maybe it's because of that part or maybe something Google getting confused like, I mean, which domain belongs to which IP or something until at that. I don't think from shared hosting that should happen, at least not by design. But if you can send me a link to your forum thread, I can double check to see what specifically might be happening there. Sure, all right. Cool. All right. Do site maps help structure Google about crawling? We're seeing some sites get crawled 500,000 times a day while still having new product pages not indexed in Google. So sitemap files do help us to better understand the website and to better figure out which parts of a website need to be recrawled. So specifically, if you have information in the last modification date that I mentioned, that really helps us to figure out which of these pages are new or changed that need to be recrawled. So that helps us quite a bit. With regards to product pages not being indexed in Google, again, that's something where maybe that's essentially just working as intended, where we just don't index everything from any website. I think for most websites, if you go into the sitemap section or the indexing section, you'll see that we index just a fraction of all of the content on the website. I think for any non-trivialized website indexing all of the content would be a very big exception. And I would be very surprised to see that happen. So I guess those are kind of the two directions I would look at there. Would you recommend partitioning a server so that bots are allocated 50% and users are allocated 50% to help bots from impacting the site speed for users? I think that's something you can definitely think about doing. I think for most websites that doesn't make sense because we can crawl normally and we can recognize when a website gets bogged down and we slow our crawling then. There might be some edge case situations where this makes more sense. And there are things that you can do to slow down our crawling in general, which are kind of the normal things that we would watch out for with regards to a website that's getting bogged down. So on the one hand, if the responses start getting slower, then generally we assume that maybe we've been crawling too hard. We need to kind of notch that back a little bit. If we start seeing a lot of server errors, so that could be 500 errors or 503 errors, which are temporary errors, then that's also a sign for us that maybe we're crawling a little bit too hard and need to kind of hold back a little bit. So if you're seeing us crawling too much, those are some things you could do. You can also go into Search Console in the crawl rate settings. And there's a slider there that lets you temporarily adjust the crawl rate. That's something you could also slide back and say, I don't want you crawling as much as you've been crawling in before. And we will take that into account as well. So those are kind of the things to watch out for. In general, the crawl speed process on our side is something that's an automated process that's fairly well tuned, that tries to adapt to your server and to our perceived needs from your website. So if we think we need to crawl a lot from your website to kind of keep up, then we'll have this big batch of URLs we'll try to crawl. And at the same time, we'll balance out with what we think we can crawl without causing problems for your website. And that kind of settles down over time, where we figure out, well, we can crawl this much. Or on some days, we can crawl a lot more. So maybe we'll crawl more then, or we'll try to catch up then. And figure out how that works out over time. Let's see, question and chat here. Which content practice is better if we provide a service and cover different areas of a city to have a service page and mention all the areas we cover or to create a dedicated page for every area? And how will Google treat them as content will be very similar. So I think this is one of those topics where there is no absolute yes or no. It really depends on how you set that up on your website. If you have a handful of locations and you have unique, valuable content to provide for those individual locations, I think providing that on your website is perfectly fine. If you have hundreds of locations, then putting out separate landing pages for every city or every region is almost more like creating a bunch of doorway pages. So that's something I really discourage doing. But there's lots of room kind of in between, like having two or three pages and having 100 pages. And you really need to figure out where you can actually provide unique value on all of these pages so that you're not just copy and pasting the same content and swapping out keywords. Let's see. If I have an m.site and after the mobile first index, you said Google would be more interested to hit the m.pages rather than . . .pages, then adding an XML site map would be useful in m.or not. So I guess the question is, should I put my m.pages in the XML site map or not? From our point of view, that's something that hasn't really changed. If you have separate m.pages, you can definitely put those in the site map file. We will look at those and crawl them and figure out what we need to do there. If there's a clean connection between the m.bot version and the desktop version, then that connection helps us to figure out which page to show in which situation. Ideally, you would use the same URLs, though. So this is something we've kind of recommended from the start is if you can use something like responsive design or use the same URLs for desktop and mobile or any other kind of devices, then that's really the ideal situation, because then you don't have to make this decision on site map file or internal linking or meta tags and all of that. You really just have one version of the content on one URL. Let's see. In chat, there's another question here. I am seeing Google cache show different URLs for some of my sites. And there's a long URL link. You can see the cache is showing my demo site, but the visual is a different website. Any idea why this is happening? So I didn't double check the URL that you're linking to there. But usually, this is a sign that we feel that these pages are equivalent or were recently equivalent and that we've been able to kind of canonicalize one to the other. So if you do a look at the cache page and you see on top the URL is slightly different than probably what we're saying is the page that you asked for is chosen not as a canonical. And instead, we chose the one that we do show there. So that's something you do see in the new search console as well, but you see that here on the cache page in particular. You also see that if you do an info colon query for that non-canonical URL, we'll show you the one that we think is a canonical there. And oftentimes that comes from us not really knowing which of these URLs you actually want to use as a canonical. So if there are different signals telling us that maybe this is the one that you want and you have a real canonical through that one and maybe redirect to another URL and then one is in the sitemap file, all of these things kind of add up and we try to figure out which one of these is the best bet for your canonical URL. And if we have conflicting information or incomplete information, then sometimes we have to guess. And sometimes we guess wrong. On the other hand, if you provide us consistent information and you say, well, rel canonical points here, this is the internal link, same URL, the same URL is in the sitemap file, all of these things add up. And we say, well, obviously, this is the one that we want to add or the webmaster wants to have you show in search. So we'll try to use that as much as possible. It's not a guarantee that we'll always do that, but as much as possible, we'll try to do that. OK, the question goes on. It says these sites are no way connected other than that they're both on our platform. Sometimes this is tricky. So what sometimes happens here in some cases, I don't know about this particular one. I probably have to look into that in detail. But what sometimes happens is our algorithms assume that things are similar based on other URLs on the website. So what we've seen sometimes is you have maybe a shared hosting or shared platform on all of these sites. And depending on the URL parameters, if you use the same parameter across all of these different host names, you'll get exactly the same content. And if we run across a situation like that, we might assume that actually these different domains are all the same. Because if you specify the same URL parameters somewhere, then you do end up with the same content. Or it looks like, based on what we've seen across the website itself, from maybe other parts of the site, that these domains are actually the same or they're equivalent, then something like this can happen. What I'd recommend doing here is if you're sure that things like the rel canonical are set up properly, that you're actually definitely not sharing content across these different domains. And you don't have any URL parameters where it's like you query one page with ID equals one. You get the same result as if you query the other domain with ID equals one. Then I posted the Webmaster Help Forum to get some advice from other people there and also to get that escalated to us so that we can take a look at that to see if there's anything on our side where maybe we're misinterpreting what you're providing to us there. Hey, John, I got a quick question. Sure. So July of last year, we did a big website migration. And so some of our pages we have indexed right now is still showing the old URL in the Google search, but actually showing the new page title, the new meta description with that old URL. Is there a reason this happens? How are you seeing those? Like if you search for the text or if you search for the URL? If you search for a text, so if you do a search for a dome light, it comes up and shows our new information but directs to the old URL that in 301 redirects to the new URL. Yeah. So probably what's happening there is similar to some of these other things that we're getting confused with canonicalization. So that could be something as simple like putting a rel canonical on those pages to really kind of tell us, well, not just the redirect, but also the rel canonical confirms the new URL is the right one, making sure that internal linking is all the same. If you use something like hreflang markup, that that's all the same. All of these things just to make sure that they align and that when we look at that URL, we definitely notice which one to show. So even if it has a redirect to a different URL, we might say, well, probably this old URL is still the one that you want to have indexed based on other signals that we have. OK, thank you. All right. Any more questions from any of you? Oh, OK. Let me see. Maybe I can grab one or two more questions. And I need to head out a little bit early. But otherwise, we can see how far we can get here. We're using canonical tags on filtered pages pointing to the main categories. If the content is not exactly the same, we believe it's more relevant to the user. Is that a good practice or not? So in general, this is probably not the right use of the rel canonical. It's possible that we pick this up and we say, oh, this is good enough. But if the pages are not equivalent, if they're not really the same, then it's also possible that our algorithms look at this and say, well, this rel canonical is probably accidentally set like this, and we should ignore it. So my recommendation there would be more to try to figure out how you can use the rel canonical correctly there. And to try to use that more to guide indexing rather than to combine different things that are kind of similar, but not exactly the same. Hi, John. I was wondering if I could quickly jump in. OK. My question is, I've actually listed it, but you haven't got to it yet, and I'm hoping you will. Mine's around the change of address tool. So we're looking into a site merge at the moment where we have lots of different CCTLDs, and we want to put them under one single, the subdirectories. Now, the change of address tool doesn't really support that in the sense that a home page of a CCTLD would need to point to the top. And I didn't want Google to get confused about locations. So is there anything you would recommend in terms of an alternative to change of address tool if we're not able to use it? I would just use redirects. Just simple redones. So the change of address tool really needs to have this kind of one-to-one mapping of all of the URLs within the website. And when you're merging sites, that's not possible. So for cases like that, you really just need to use the redirects and kind of the normal things that you would do for migrating websites. And especially when you're talking about merging sites or splitting sites, it's also, I think, important to keep in mind that this is something that is a bit trickier for us to actually process compared to a normal move, where you're just moving everything from one URL to another URL. When you're migrating in a way that you're merging things together into one site or splitting one site into multiple sites, that takes quite a bit more time for us to actually re-process. So it's probably worth really thinking about when would be a good time where you're not dependent on search traffic for that site. OK, cool. Thank you. All right, let's see. On a website with millions of pages of user-generated content, a small percentage of the page are very high quality, whereas the rest are thin content. Should I add a no index to the low quality page to improve my SEO, if afterwards the user improves the content of a page, can I add it again? So with regards to user-generated content and the quality of the content, that's ultimately something that you have to decide. So you're providing this website to us, and you're essentially standing for the content that you're providing on your website. And regardless of where that content is coming from, we don't make a distinction between this is something some random user submitted and posted on this website, and this is something the webmaster themselves put on this website. Kind of after a week-long study, they created this piece of content and put this on the website. We look at all of the content that we see there. And if there are things that you don't want to have indexed, then that's up to you to say, well, I don't want to have this indexed with maybe a no index tag, or if there are things that you say, well, this is kind of mediocre content, but I want to improve it, or I want to help someone else to improve this, then that's something where you can also change your mind and say, well, I have a no index here. Now, while it's still kind of in work and once it's complete, I'll take that no index off and let it be indexed normally. And that's completely normal. So that's something where from my point of view, I wouldn't think of it so much as user-generated content or not, but rather, this is your website, and this is what you're providing to Google and to all of Google's users to kind of look at. And if that's something that you think is good that you're proud to provide to the web, then, of course, let it be indexed. But if it's something where you're like, well, some random spam or put this on my website, I don't really want to be associated with this, then maybe you should take it down, or maybe you should just put a no index on it to prevent it from kind of being seen as a part of your website when it comes to search. John, can I ask a question? Just a short one, because I need to head out. Yeah, sure. I have a page where I am using three or four images, but I don't want those images to view on the mobile version. Is it OK to do that? Sure, you can do that. I mean, it might be that if we index the mobile version, we will not see those images either. So that's something where you kind of have to think about, what is the long-term plan? Are these images important or not? And if they're important, then show them. If they're not important, then maybe not show them. And one more question. I have seen some pages like they are using for a five question, using H1 tag for those, and answering same five questions. Is it OK to do that? Sure, that's fine. OK, yeah. All right, I need to head out. Thank you all for joining, and I hope to see you all again one of the next sites. Thank you, John. Bye.