Intra Rustdoc Links

[killercup]

Add a notation how to create relative links in documentation comments (based on Rust item paths) and extend Rustdoc to automatically turn this into working links.

The important bits:

Additions to the documentation syntax

1. [Iterator](std::iter::Iterator)
2. [Iterator][iter], and somewhere else in the document: [iter]: std::iter::Iterator
3. [Iterator] or [`Iterator`] with automatically generated link reference definition

All additions are valid CommonMark syntax.

Rendered

Revisions

March 9: Dropped <Iterator>-style links, added implied reference links (#1946 (comment))

May 28: Add more possible extensions, add more syntax for dealing with path ambiguities (#1946 (comment))

[GuillaumeGomez]

Awesome! \o/

[fbstj]

Bikeshed: would it be useful to have an additional <std::iter::Iterator> style syntax/marker that desugars to [Iterator](std::iter::Iterator) for convenience?

[killercup]

an additional <std::iter::Iterator> style syntax/marker that desugars to [Iterator](std::iter::Iterator)

Hm, I'm wary of adding more cases, and even more so to add magical new syntax. I'd prefer to go with explicit links if you want to set the link text to something other than the full path.

(But: If Iterator is in scope, <Iterator> will be a valid link!)

[sgrif]

This will need a way to disambiguate between functions, traits, etc. It's completely valid to have a struct and function with the same name. Perhaps we could use URIs for this? function://path::to::fn, struct://path::to::struct, etc.

Also should this specify whether super works or not? Are the paths always relative to the crate root? Is there ever a case where it would matter whether the link is based on where the struct appears in the documentation vs where it was defined?

[kennytm]

Previous discussions 🚲

• RFC: Relative link syntax for rustdoc #1661 (Relative link syntax for rustdoc, closed)
  • <::/std/cmp/trait.Ord.html>
• RFC: Links to Rust items in documentation text #792 (Links to Rust items in documentation text, active issue)
  • <<<std::cmp::Ord>>
• https://internals.rust-lang.org/t/2720 (Pre-RFC: standard form to reference things)
  • {std::cmp::Ord#cmp}
  • [cmp]{@link std::cmp::Ord.cmp}
  • <rustdoc:std::cmp::Ord>
  • <rustdoc://std::cmp::Ord>
  • <rust:std::cmp::Ord>
  • ` std::cmp::Ord` (note leading space)
  • [std::cmp::Ord]
• https://internals.rust-lang.org/t/968 (Rustdoc: Link to other types from doc comments)
  • ``std::cmp::Ord``
  • <rust://std/cmp/Ord.trait>

[frewsxcv]

Relevant rust-lang/rust issues: rust-lang/rust#36417, rust-lang/rust#38400

[kennytm]

@sgrif It only needs to distinguish between values (functions, constants) and types (structs, enums, traits, modules, type-aliases, primitives). Maybe:

• <syntax::ptr::P> — First search for type-like items. If not found, search for value-like items
• <syntax::ptr::P()> — Only search for functions.
• <std::stringify!> — Only search for macros.

[killercup]

This will need a way to disambiguate between functions, traits, etc.

Oh, I had completely missed that this is valid (damn you, Rust!). Is there are an existing way to disambiguate this in paths? Or is it always resolved by the position it appears in?

Playing around on the playpen a bit, it seems that (assuming all have the same name):

• unit and tuple structs conflict with functions
• unit and tuple structs conflict with enums
• regular structs, enums, and traits conflict with each other (but not functions)

I think @kennytm idea with a () suffix to disambiguate functions (#1946 (comment)) works well. I propose it is only needed when the non-suffixed link is actually ambiguous. (I assume most code bases follow Rust's style lints and use snake_case functions names and PascalCase for type/trait names.)
Macros should alway use ! suffix.

Are the paths always relative to the crate root?

Good point, that's totally missing from the RFC. Path are relative to the position of item in whose documentation they appear. (See complex example for relative paths.)

Previous discussions

Thanks, @kennytm! Reading #1661 (comment), it only feels right to cc @JohnHeitmann.

[frewsxcv]

There is talk about extending CommonMark to support syntactic extensions, but it looks like progress has stalled a bit.

If we're going to design our own hyperlink destinations (as proposed in this RFC and the comments above), I'd strongly recommend staying within the bounds of what is acceptable for a "link destination" as defined by CommonMark.

[kennytm]

How will cross-crate links be handled? Like,

• linking from num-traits to <std::u32> (linking to standard library)
• linking from num-complex to <num_traits::One> (crates in the same workspace)
• linking from num-bigint to <serde::Serialize> (third-party crates, and only linked when a feature is defined)

---

BTW there is also this thread on talk.commonmark.org discussing the issue from the Markdown side, but it is pretty inconclusive (one suggestion to api:std.cmp.Ord and one to {std::cmp::Ord}).

[killercup]

How will cross-crate links be handled?

@kennytm, rustdoc is already able to generate links to other crates' types, and I would assume we can use this mechanism. For std (and core), it links to an official URL, e.g. https://doc.rust-lang.org/nightly/core/fmt/trait.Debug.html. For other external crates, it links to the docs it generated for the crate itself.

Update: Sorry, I totally missed

linking from num-bigint to serde::Serialize (third-party crates, and only linked when a feature is defined)

I'm not sure how to approach this best. I'd say we don't generate the link and issue a warning. Rendered docs should probably always contain all possible features. It will be difficult to do this when features are exclusive, i.e. you can't use both serde_json and json. This seems to be an unusual practice, though.

[killercup]

Updated the RFC with sections on

• Resolving paths
• Path ambiguities
• Linking to external crates

---

I don't think we need to use a potential Markdown extension for this.

If we're going to design our own hyperlink destinations (as proposed in this RFC and the comments above), I'd strongly recommend staying within the bounds of what is acceptable for a "link destination" as defined by CommonMark.
– #1946 (comment)

I think Rust paths already comply with that. Even the () suffix should not be problem, as the fall into part (b) of:

[…] includes parentheses only if (a) they are backslash-escaped or (b) they are part of a balanced pair of unescaped parentheses […]
– http://spec.commonmark.org/0.27/#link-destination

[steveklabnik]

/cc @jonathandturner who has been working on a rough rustdoc re-write

[durka]

Good point, that's totally missing from the RFC. Path are relative to the position of item in whose documentation they appear. (See complex example for relative paths.)

This is a problem for docs that get cross-included by Deref.

[killercup]

This is a problem for docs that get cross-included by Deref. (#1946 (comment))

Can you give an example? I found serde::bytes::ByteBuf which has a Deref impl, and the docs from trait Deref itself are rendered.

Including docs from traits should lead to the docs being evaluated relative to the trait item they are defined on, i.e. relative to core::ops::Deref, not serde::bytes::ByteBuf.

[nrc]

@killercup, @sgrif, @kennytm

This will need a way to disambiguate between functions, traits, etc. It's completely valid to have a struct and function with the same name. Perhaps we could use URIs for this? function://path::to::fn, struct://path::to::struct, etc.

It only needs to distinguish between values (functions, constants) and types (structs, enums, traits, modules, type-aliases, primitives)

Rust has three namespaces - values, types, and macros. Everything is in one of those three and you don't need any more namespaces than that (it is a bit more complicated because some things can be in more than one namespace, tuple structs, iirc can be used both as type names and as function names). There are already alternate URLs in Rustdoc that use this scheme, unfortunately via redirects (I would love to deprecate the crazy ones with trait etc in the name, but there were back compat worries), these are easy to synthesise, e.g., https://doc.rust-lang.org/nightly/std/path/Path.t.html for https://doc.rust-lang.org/nightly/std/path/struct.Path.html, so you don't need to know the random rustdoc categorisation of a thing, only its namespace.

[steveklabnik]

Thanks @killercup ! Here's some thoughts....

1. For me, the priority here is to make sure that we're within regular Markdown rules. It appears that we are. 👍
2. A significant proposal from before that doesn't seem to be addressed here is a rust:// URI. Putting this in alternatives at least should happen; it's not clear to me that we shouldn't do this.
3. In general, an RFC like this is largely split between two things: how do I define the links, and how does rustdoc actually figure them out. The former is definitely RFC-worthy for me, but I wonder if the latter can't be avoided as some sort of "you use the canonical path for the item linking to", and then free rustdoc to figure that out itself.
4. In contradiction to point 4, "how does rustdoc figure this out" is a pretty significant pain point, and the hardest part of the RFC, IMHO.
5. @jonathandturner and I (among others at various times) have wondered if rustdoc in the future shouldn't be built on top of RLS. If so, that split would mean "rustodc proper asks RLS what the URL is", I think.
6. thoughts about migrating to this new style might be helpful, I wonder how hard it would be to create a tool to convert existing links like this.

That's all (lol, I feel like it's a lot) I've got at the moment, I'm sure I'll have more eventually...

[durka]

@killercup

For example, there is a link from str::len to char, but this link is also present (and broken) on the String page because it is a relative link. There are open issues about this.

It's definitely solvable, but it could require "absolute" (i.e. crate-relative) paths in some places.

[nrc]

If so, that split would mean "rustodc proper asks RLS what the URL is", I think.

FWIW, the RLS already creates concrete URLs from paths like this so that when you have a reference to something from std, you can go to the rustdoc page. It might not work exactly how you need it to for this application though.

[kennytm]

How should Rustdoc know that <P> is an HTML tag or a link to a P item?

[sophiajt]

That's great!

Thanks for the ping. I think for a "rustdoc 2.0" idea, we're still a little ways off, yet. Yeah, it will probably consume metadata rather than trying to work against the compiler directly, but there's still a fair bit of research we need to do to explore what we can do there. That said, I'm currently doing the rls stuff, and what we're doing there will help make a "rustdoc 2.0" possible.

I say go for it and keep improving this rustdoc. It just sets the bar that much higher for the next one (which is a good thing!)

[eddyb]

@jonathandturner IMO, instead of trying to serialize piecewise, we could have sources for everything, which combined with warm incremental recompilation caches would allow introspecting entire crates.

Even if we don't get incremental reparsing, we could still skip parsing and expansion for, say, a libstd that was compiled once and distributed, so you get almost instant access to the HIR and type information.
To keep it slim, we could drop anything that's not exported (i.e. function bodies' MIR) from the caches.

[solson]

2. [Iterator][iter], and somewhere else in the document: [iter]: std::iter::Iterator

@killercup Markdown has another trick that would make this even sweeter at the use sites:

[Iterator], and somewhere else in the document: [Iterator]: std::iter::Iterator

[llogiq]

The 'easy' alternative is to auto-link all items in scope, as in [String]. This is trivially supported by pulldown-cmark with extending the links map.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period is now complete.

[killercup]

Just came up:

`[Iterator]`

should probably not be rendered as a link inside a code block.

[Manishearth]

We're proposing some amendments in #2285