Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block version-specific docs from search engines #69992

Merged
merged 1 commit into from
Mar 16, 2020

Conversation

kornelski
Copy link
Contributor

@kornelski kornelski commented Mar 14, 2020

Latest stable, beta and nightly URLs remain accessible because their URLs don't start with a version number. Robots.txt uses simple path prefixes, so it's OK that the disallow rules aren't full directory paths.

Direct links to old docs remain accessible to users, because robots.txt only affects crawlers.

With this change old docs for specific old versions of Rust won't pop up in search results. This is good, because users won't be getting obsolete documentation by accident.

Stable, beta and nightly URLs remain
@rust-highfive
Copy link
Collaborator

r? @steveklabnik

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 14, 2020
@steveklabnik
Copy link
Member

Interesting. I think I feel positively about this; let me sleep on it :)

Copy link
Member

@steveklabnik steveklabnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think this is the right call. Thank you!

@steveklabnik
Copy link
Member

@bors: r+ rollup

@bors
Copy link
Contributor

bors commented Mar 15, 2020

📌 Commit a5a786d has been approved by steveklabnik

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 15, 2020
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Mar 15, 2020
Block version-specific docs from search engines

Latest stable, beta and nightly URLs remain accessible because their URLs don't start with a version number. Robots.txt uses simple path prefixes, so it's OK that the disallow rules aren't full directory paths.

Direct links to old docs remain accessible to users, because robots.txt only affects crawlers.

With this change old docs for specific old versions of Rust won't pop up in search results. This is good, because users won't be getting obsolete documentation by accident.
bors added a commit that referenced this pull request Mar 16, 2020
Rollup of 8 pull requests

Successful merges:

 - #69686 (Use `pprust` to print attributes in rustdoc)
 - #69858 (std: on Windows, use GetSystemTimePreciseAsFileTime if it is available)
 - #69917 (Cleanup E0412 and E0422)
 - #69964 (Add Node.js to PR CI image)
 - #69992 (Block version-specific docs from search engines)
 - #69995 (Add more context to the literal overflow message)
 - #69998 (Add long error explanation for E0634)
 - #70014 (Small fixes in rustdoc book)

Failed merges:

r? @ghost
@bors bors merged commit d34ec33 into rust-lang:master Mar 16, 2020
@kornelski kornelski deleted the robots branch March 16, 2020 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants