On February 26, 2009, Google software engineer Matt Cutts collected questions on Google Moderator and answered many of them on video.
Mark Lykle from Oslo, Norway asked:
When will Google create a software similar to Web Position so that SEOs, spam fighters and regular webmasters can check rankings etc. without violating the guidelines? Why not make a better product instead of going to war against these programs?
Add a esrun.txt file to your site with "Disallow esrun scraper" and I won't scrape the serps anymore :)
Google is the biggest scraper of them all. Constantly forcing people to add files to their site or look at the recent wifi scraping. We have to change the SSID of our wifi router just to stop Google from slurping information about it?
Fine. Have it that way. But if you don't want people to scrape Google then you'll have to follow our guidelines on blocking our scrapers :)
onlinehoster 2 months ago
:::Attention::: bot252 from ubotstudios com approves of GoogleWebmasterHelp's videos. That is all.
ubotstudios11 8 months ago
p.S i think question was aimed towards page rank checking.. NOT keywords ranking.
TdRaidX 1 year ago
@figurethis2 Thats true, true
milkfish87 1 year ago
I don't know about this video. Do you habe any others that can back it up?
expertreviewsdotcom 1 year ago
Mats presentation skills have really come along way since this vid... :0
figurethis2 1 year ago
@twparker1 You can check Wiki for "Web scraping" for the exact definition, but its pretty much when a website is created to go to google and copy (or scrape) the content and bring it back to the originating website. From there the data can either be displayed (for example, without their ads, or in a different format), or stored in a database. It slows server resources cause they can scrape 1000's of pages in a very short time, much faster than a human, and it can overload the servers.
louishcom 1 year ago
Klasse... Prima gemacht...
calabenirras 1 year ago
this was a very good answer
peterpixo 2 years ago
What does scrapping mean?
twparker1 2 years ago