Can I disallow crawling of my CSS and JavaScript files?

On February 26, 2009, Google software engineer Matt Cutts collected questions on Google Moderator and answered many of them on video.

SEOmofo from Simi Valley asked:

If I externalize all CSS style definitions and JavaScript scripts and disallow all user agents from accessing these external files (via robots.txt), would this cause problems for Googlebot? Does Googlebot need access to these files?

Category:

Science & Technology

Tags:

• google
• css
• javascript

[yyzxyzzy]

In other words, "let us inspect your bag because if you don't, we'll assume you're smugging drugs." Nice.

[SEOMofo]

Thanks for the reply, Matt!

[TestShoot]

Does Google index javascript via the document.write() method? I have some sites that can not use includes and are on different servers so I created a navbar in javascript from one publish point using that method.

[wwwSEOWritersNet]

Custom blog writing services use professional writers. You need to be sure the content provided to you really is written by the writer and not copied from another custom written blog. This could only make you look bad and even get you blacklisted with the search engines. Always be sure the content provided by a blog writing service is unique and written by them.

[Webnauts]

There can't be a problem with the CSS and Javascripts, if you implement X-Robots with the directives "noindex,follow", or in the robots.txt the "noindex" directive.

[manfmnantucket]

interesting answer - at end u say that not blocking the css etc would allow fetching it in rare case eg of manual spam review needed. however ROBOTS.txt only blocks indexing bots, not others from seeing/using those files. anyone can find non-indexed files by reading robots.txt so u could do so for review... confused.