Unflatten use statements in HIR

[oli-obk]

The HIR and AST representations of use statements are now mirrored, instead of having the HIR flatten the AST representation and duplicating early parts of nested paths:

use a::b::{c, d};

used to get flattened in HIR to

use a::b::c;
use a::b::d;
use a::b::{};

which duplicated the a::b part several times (and many more if you have more nesting).

This alone is fine, it made some parts of the compiler simpler and some parts harder, but it also meant we were generating more items (which are also owners) and had to duplicate resolver information across the flattened items. All that is gone with this PR.

This PR is the minimal version we can land, but there are obvious further avenues for cleanups and improvements. I did some refactorings in rustdoc, but I think the import logic can generally be improved by not handling imports together with other items and instead having them in separate tables.

Similarly clippy can probably benefit from a check_use_tree method on LateLintPass, but they often need to track separate information anyway, so it may be better to just always manually recurse.

cc @cjgillot

r? @petrochenkov (or reassign)

should unblock #159760

see related discussion on #t-compiler > partial_res_map vs per-owner tables @ 💬

part of rust-lang/goals#620

first commit is from #161299

everything up to 6936535 can land in a separate PR first

[oli-obk]

@bors try @rust-timer queue

[rust-bors]

💔 Test for bb18a8a failed: CI. Failed job:

• try - dist-x86_64-linux-quick (web logs, enhanced plaintext logs)

[oli-obk]

The CI failure makes no sense, the only way that assert is reachable is by there being UB somewhere afaict:

rust/compiler/rustc_metadata/src/rmeta/decoder.rs

Lines 1447 to 1452 in b588ef3

	match def_key.disambiguated_data.data {
	DefPathData::Ctor => {
	// Structure and variant constructors don't have any attributes encoded for them,
	// but we assume that someone passing a constructor ID actually wants to look at
	// the attributes on the corresponding struct or variant.
	assert_eq!(def_key.disambiguated_data.data, DefPathData::Ctor);

Also I can't reproduce locally 😨

[WaffleLapkin]

The backtrace doesn't make sense either:

  12:     0x7f7d92df5b99 - core[8ec35bcd8078f253]::panicking::assert_failed::<rustc_hir[5841a66ff274cc3a]::definitions::DefPathData, rustc_hir[5841a66ff274cc3a]::definitions::DefPathData>
  13:     0x7f7d946407c1 - rustc_metadata[69b7fa7d7ff41b69]::rmeta::decoder::cstore_impl::provide_extern::attrs_for_def

The assert is in CrateMetadata::get_item_attrs, but the backtrace skips it, even though it isn't marked with #[track_caller]. 🤔

And attrs_for_def shouldn't be able to panic like that on its own:

rust/compiler/rustc_metadata/src/rmeta/decoder/cstore_impl.rs

Line 332 in f7d782a

attrs_for_def => { tcx.arena.alloc_from_iter(cdata.get_item_attrs(tcx, def_id.index)) }

rust/compiler/rustc_metadata/src/rmeta/decoder/cstore_impl.rs

Lines 138 to 163 in f7d782a

	($tcx:ident, $def_id:ident, $other:ident, $cdata:ident, $name:ident => $compute:block) => {
	fn $name<'tcx>(
	$tcx: TyCtxt<'tcx>,
	def_id_arg: rustc_middle::queries::$name::Key<'tcx>,
	) -> rustc_middle::queries::$name::ProvidedValue<'tcx> {
	let _prof_timer =
	$tcx.prof.generic_activity(concat!("metadata_decode_entry_", stringify!($name)));
	#[allow(unused_variables)]
	let ($def_id, $other) = def_id_arg.into_args();
	assert!(!$def_id.is_local());
	// External query providers call `crate_hash` in order to register a dependency
	// on the crate metadata. The exception is `crate_hash` itself, which obviously
	// doesn't need to do this (and can't, as it would cause a query cycle).
	use rustc_middle::dep_graph::DepKind;
	if DepKind::$name != DepKind::crate_hash && $tcx.dep_graph.is_fully_enabled() {
	$tcx.ensure_ok().crate_hash($def_id.krate);
	}
	let cstore = CStore::from_tcx($tcx);
	let $cdata = cstore.get_crate_data($def_id.krate);
	$compute
	}
	};

[oli-obk]

The assert is in CrateMetadata::get_item_attrs, but the backtrace skips it, even though it isn't marked with #[track_caller]. 🤔

probably got inlined. but the assert message says it's in a different line

compiler/rustc_metadata/src/rmeta/decoder.rs:1404

I freshly rebased, so I assumed it should be fine, very odd

[rust-log-analyzer]

The job aarch64-gnu-llvm-21-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

To only update this specific test, also pass `--test-args internal-lints/default_hash_types.rs`

error: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui-fulldeps/internal-lints/default_hash_types.rs" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui-fulldeps/internal-lints/default_hash_types" "-Znext-solver=coherence" "-A" "unused" "-W" "unused_attributes" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Z" "unstable-options"
stdout: none
--- stderr -------------------------------
error: prefer `FxHashMap` over `HashMap`, it has better performance
##[error]  --> /checkout/tests/ui-fulldeps/internal-lints/default_hash_types.rs:9:24
   |
---
To only update this specific test, also pass `--test-args internal-lints/non_glob_import_of_type_ir_inherent.rs`

error: 1 errors occurred comparing output.
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent" "-Znext-solver=coherence" "-A" "unused" "-W" "unused_attributes" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Z" "unstable-options"
stdout: none
--- stderr -------------------------------
error: non-glob import of `rustc_type_ir::inherent`
##[error]  --> /checkout/tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs:16:9
   |
LL |     use rustc_type_ir::inherent::Predicate; //~ ERROR non-glob import of `rustc_type_ir::inherent`
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^---------
   |                                  |
   |                                  help: try using a glob import instead: `*`
   |
note: the lint level is defined here
---

error: non-glob import of `rustc_type_ir::inherent`
##[error]  --> /checkout/tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs:20:9
   |
LL |     use rustc_type_ir::inherent::ParamEnv as _; //~ ERROR non-glob import of `rustc_type_ir::inherent`
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^-------------
   |                                  |
   |                                  help: try using a glob import instead: `*`

error: non-glob import of `rustc_type_ir::inherent`
##[error]  --> /checkout/tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs:24:9
   |
LL |     use rustc_type_ir::inherent; //~ ERROR non-glob import of `rustc_type_ir::inherent`
   |         ^^^^^^^^^^^^^^^^^^^^^^^- help: try using a glob import instead: `::*`

error: non-glob import of `rustc_type_ir::inherent`
##[error]  --> /checkout/tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs:25:9
   |
LL |     use rustc_type_ir::inherent as inh; //~ ERROR non-glob import of `rustc_type_ir::inherent`
   |         ^^^^^^^^^^^^^^^^^^^^^^^------- help: try using a glob import instead: `::*`

error: aborting due to 4 previous errors
------------------------------------------

---- [ui] tests/ui-fulldeps/internal-lints/non_glob_import_of_type_ir_inherent.rs stdout end ----

[oli-obk]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 00e1020 (00e1020c22d36a6629c7d764b1967a735a100e74)
Base parent: 526c36b (526c36b4c477994b7905ec6a084d6150e49076f5)

[rust-timer]

Finished benchmarking commit (00e1020): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 0.9%]	45
Regressions ❌ (secondary)	0.7%	[0.0%, 1.8%]	47
Improvements ✅ (primary)	-1.3%	[-18.7%, -0.1%]	80
Improvements ✅ (secondary)	-0.5%	[-1.4%, -0.1%]	12
All ❌✅ (primary)	-0.7%	[-18.7%, 0.9%]	125

Max RSS (memory usage)

Results (primary 1.0%, secondary 3.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.7%	[0.4%, 4.1%]	20
Regressions ❌ (secondary)	4.8%	[2.1%, 11.1%]	18
Improvements ✅ (primary)	-2.0%	[-11.4%, -0.4%]	12
Improvements ✅ (secondary)	-5.2%	[-7.7%, -2.7%]	2
All ❌✅ (primary)	1.0%	[-11.4%, 4.1%]	32

Cycles

Results (primary -5.9%, secondary 3.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.8%	[2.3%, 3.2%]	2
Regressions ❌ (secondary)	3.2%	[2.2%, 3.8%]	11
Improvements ✅ (primary)	-8.0%	[-16.0%, -2.4%]	8
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-5.9%	[-16.0%, 3.2%]	10

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 470.08s -> 468.116s (-0.42%)
Artifact size: 400.00 MiB -> 400.03 MiB (0.01%)