Can I use robots.txt to optimize Googlebot's crawl?
Loading...
Top Comments
All Comments (16)
-
@iLovePalestineDotCom You can always remove the already indexed pages too.
-
Why you dont stop answering question from that 5 year old ?, It's your cousin or something?
-
He is funny. I find it educational and entertaining as well. He lifted me up from boredom.
-
nice video thanks for sharing very interesting .. god bless
-
@iLovePalestineDotCom good note but I think that is OK. I think he meant doing it on weekly basis, for long period of time, could confuse the google bot.
But technically, you should go live if you aren't ready
-
KINDA NOOOOOOOOOO VIDEOS fan club!!!
-
There were NO ROBOTS txting, as suggested by the title.
You're a nice guy Matt, but I'm done with you professionally. :-)
-
Google need the brand new Googlebot technology. Just like the new search algorithm.
1 year ago 2
-
@Sander33333 Go to the webmaster forum. If you're lucky, Matt will answer your question in a video.
-
How can i ask a question for a video?
1 year ago 3
- 2:30
Can a site's downtime affect its ranking?by GoogleWebmasterHelp7,569 views
- 2:17
Are there any plans to implement a "negative keyword" meta tag?by GoogleWebmasterHelp6,305 views
- 2:13
Should I block duplicate pages using robots.txt?by GoogleWebmasterHelp8,724 views
- 2:01
Is it better to have keywords in the URL path or filename?by GoogleWebmasterHelp20,869 views
- 9:56
Search Engine secrets - Remove indexed web pages from googleby proson1231,000 views
- 2:20
How do PageRank updates work?by GoogleWebmasterHelp19,535 views
- 5:00
How to create and submit an XML sitemap using free toolsby joeblstrn23,032 views
- 2:36
The Simulation Hypothesisby ADruskovich3,940 views
- 2:13
Uploading Your robots.txt Fileby NicheBloggingProfits2,695 views
- 1:58
Google does not use the keywords meta tag in web rankingby GoogleWebmasterHelp201,276 views
- 2:23
How Become the Google Botby greenyvidtutorials17,058 views
- 1:40
Are nofollow links irrelevant?by GoogleWebmasterHelp17,195 views
- 1:13
Site Maps and Robot.txt for SEOby htiyqa448888 views
- 2:19
How does Google handle special characters?by GoogleWebmasterHelp21,666 views
- 2:20
Advanced Robots.txt Generator Tutorial Vol1by basileios2,436 views
- 9:12
Web2Expo - Matt Cutts - What Google Knows About Spamby flmortgagepro1,105 views
- 4:08
What are some examples of SEO misinformation?by GoogleWebmasterHelp26,950 views
- 4:54
How to use Google Webmaster Toolsby AskJoanne58,475 views
- 2:09
Matt Cutts on Big Brand BlackHat Sites by ChrisDaviesCa Linkwheelby ameenaincredulityb623 views
Something that Matt did not touch on is the fact that when you present Google with a new robots.txt file restricting certain area of your site from being indexed although that is not what you really wanted to accomplish. You are in essence telling Google, if you have already indexed certain areas REMOVE THEM because NOW they are restricted. I was so hoping that Matt touches on that but he didn't. So we do not know if this act damage the status of already indexed pages or not.
iLovePalestineDotCom 1 year ago 11
Don't scare googlebot, come on man :(
prisontv 1 year ago 5