User-agent: * Disallow: cache/ Crawl-delay: 5 User-agent: BUbiNG Disallow: / User-agent: Yandex # will use all Yandex robots Disallow: / User-agent: cubbot Disallow: /