# robots.txt User-agent: rdfbot/1.0 Crawl-Delay: 1800 User-agent: * Disallow: