Restrict web crawler bots to crawl only one branch per book?
Created by: seanmadsen
Sometimes I am searching for docs in other projects and the google results take me to a version-specific docs page, which sometimes is outdated. Other times google will display separate results for separate versions, both with the same content, which is somewhat confusing. Also my limited understanding of SEO tells me that this is bad SEO, too.
I wonder if it would be easy/possible/desirable for us to make civicrm-docs generate a robots.txt
file or something else that will tell search engines to only index one version of each book. Maybe "latest", maybe "stable".
This issue will only become relevant once the auto-generated docs home page is in place.
Low priority. Just a thought.