Robots.txt that makes sure Facebook and Twitter can crawl images on your site.
Robots.txt that makes sure Facebook and Twitter can crawl images on your site. - robots.txt
# Disallow everything. User-agent: *. Disallow: /. # Certain social media sites are whitelisted to allow crawlers to access page markup when links ...