Home / server / Questions / 247650
Accepted
Ben K.
Asked: 2011-03-16 07:19:38 +0800 CST

Disallow xml robots.txt

  • 772

Google webmaster FAQs suggest that this will exclude all xml files from search:

User-agent: Googlebot
Disallow: /*.xml$

Is this legal for other bots as well?

User-agent: *
Disallow: /*.xml$
robots.txt

1 Answers

  1. Best Answer

    Yes, this is valid. Whether other bots actually understand and obey is completely up to them, though!

    • 1