Revert performance-sensitive change in #82436
This change was done in #82436, as an "optimization". Unfortunately I
missed that this code is not always executed, because of the "continue"
in the conditional above it.
This commit should solve the perf regressions introduced by #82436 as I
think there isn't anything else that could affect runtime performance in
that PR. The `Pick` type grows only one word, which I doubt can cause up
to 8.8% increase in RSS in some of the benchmarks.
---
Could someone with the rights start a perf job please?
bors
cd82e451a8