User-Agent: * # Googlebot Disallow: /find-new/ Disallow: /search/ Disallow: /__memcached.php Disallow: /proxy.php Crawl-delay: 5