cache type sizes in type-size limit visitor
This is basically https://github.com/rust-lang/rust/pull/125507#issuecomment-2206813779 as lcnr can't open the PR now.
Locally it reduces the `itertools` regression by quite a bit, to "only +50%" compared to nightly (that includes overhead from the local lack of artifact post-processing, and is just a data point to compare to the 10-20x timings without the cache).
```console
Benchmark 1: cargo +stage1 build --release
Time (mean ± σ): 2.721 s ± 0.009 s [User: 2.446 s, System: 0.325 s]
Range (min … max): 2.710 s … 2.738 s 10 runs
Benchmark 2: cargo +nightly build --release
Time (mean ± σ): 1.784 s ± 0.005 s [User: 1.540 s, System: 0.279 s]
Range (min … max): 1.778 s … 1.792 s 10 runs
Summary
cargo +nightly build --release ran
1.52 ± 0.01 times faster than cargo +stage1 build --release
```
On master, it's from 34s to the 2.7s above.
r? compiler-errors
bors
cc8da78a03