User-agent: Mediapartners-Google Disallow: User-agent: * # to select all crawling bots and search engines Disallow: /search* # to block all user generated query item within the website. Disallow: /20* # this line will disallow archieve section of Blogger. Disallow: /feeds* # this line will disallow feeds. Read instruction below Allow: /*.html # allow all post and pages of the blog #sitemap of the blog Sitemap: https://www.cagymind.me/sitemap.xml User-agent: Mediapartners-Google Disallow: User-agent: All browser and website inspector Disallow: https://draft.blogger.com/blog/pages/4384325925103688714 Allow: / Sitemap: https://www.cagymind.me/sitemap.xml User-agent: * Disallow: / User-agent: * Disallow: /calendar/ Disallow: /junk/ Disallow: /books/fiction/contemporary/ User-agent: Googlebot-news Allow: / User-agent: * Disallow: / User-agent: Unnecessarybot Disallow: / User-agent: * Allow: / User-agent: * Disallow: /useless_file.html Disallow: /junk/other_useless_file.html User-agent: * Disallow: / Allow: /public/ User-agent: Googlebot-Image Disallow: /images/dogs.jpg User-agent: Googlebot-Image Disallow: / User-agent: Googlebot Disallow: /*.gif$ User-agent: * Disallow: / User-agent: Mediapartners-Google Allow: / User-agent: Googlebot Disallow: /*.xls$