Rustdoc builds should be mergeable using RFC3662

[cormacrelf]

I currently struggle to maintain a big set of hacks to merge rustdoc builds across a crate graph by parsing search index javascript files. It breaks on nearly every rust release. According to the text of RFC3662, Meta does exactly the same thing and struggles just as much.

That RFC, which dramatically improves this situation, was implemented more than a year ago now. Has there been any progress switching doc builds to use the unstable rustdoc --merge=... flag? There is no mention of this in the prelude, so I guess no. Is it likely you'll accept a PR that implements this or are the required changes too sweeping and you're planning to do it yourselves?

I note that:

• A merged rustdoc build needs a target set as input. You could have a rust_doc_bundle target but it's probably not that useful.
• Therefore the final merge step should be BXL. It would take a target set, request a doc build for each with --merge=none, then specify an action to --merge=finalize with all the targets' outputs.
• An important feature is the treatment of crates not in the requested target set. I handle this by adding a flag "-Zcrate-attr=doc(html_root_url = \"{}\")".format(docs_rs_url) on third-party dependencies via a macro. Might be nice if this were not something Buck users would have to spend 6 hours figuring out on their own. Perhaps an attr on rust_library that reindeer populates automatically.

[cormacrelf]

A bit of news, there was a pretty big perf regression in rustdoc recently and RFC3662 has been identified as the solution to it. It looks like Cargo itself will move to the mergeable strategy. They often use RUSTC_BOOTSTRAP in cargo for this kind of thing, so it's probably not going to be insta-stable, but it's good news for the feature maturing.

rust-lang/rust#146895