Use a `ParamEnvAnd<Predicate>` for caching in `ObligationForest`
Previously, we used a plain `Predicate` to cache results (e.g. successes
and failures) in ObligationForest. However, fulfillment depends on the
precise `ParamEnv` used, so this is unsound in general.
This commit changes the impl of `ForestObligation` for
`PendingPredicateObligation` to use `ParamEnvAnd<Predicate>` instead of
`Predicate` for the associated type. The associated type and method are
renamed from 'predicate' to 'cache_key' to reflect the fact that type is
no longer just a predicate.
Previously, we used a plain `Predicate` to cache results (e.g. successes
and failures) in ObligationForest. However, fulfillment depends on the
precise `ParamEnv` used, so this is unsound in general.
This commit changes the impl of `ForestObligation` for
`PendingPredicateObligation` to use `ParamEnvAnd<Predicate>` instead of
`Predicate` for the associated type. The associated type and method are
renamed from 'predicate' to 'cache_key' to reflect the fact that type is
no longer just a predicate.
Because it caused major performance regressions in some cases.
That PR had five commits, two of which affected performance, and three
of which were refactorings. This change undoes the performance-affecting
changes, while keeping the refactorings in place.
Fixes#67454.
In particular, it has bugged me for some time that `process_cycles` is
currently located before `mark_still_waiting_nodes` despite being called
afterwards.
`NodeState` has two states, `Success` and `Done`, that are only used
within `ObligationForest` methods. This commit removes them, and renames
the existing `Waiting` state as `Success`.
We are left with three states: `Pending`, `Success`, and `Error`.
`Success` is augmented with a new `WaitingState`, which indicates when
(if ever) it was last waiting on one or more `Pending` nodes. This
notion of "when" requires adding a "process generation" to
`ObligationForest`; it is incremented on each call to
`process_obligtions`.
This commit is a performance win.
- Most of the benefit comes from `mark_as_waiting` (which the commit
renames as `mark_still_waiting_nodes`). This function used to do two
things: (a) change all `Waiting` nodes to `Success`, and (b) mark all
nodes that depend on a pending node as `Waiting`. In practice, many
nodes went from `Waiting` to `Success` and then immediately back to
`Waiting`. The use of generations lets us skip step (a).
- A smaller benefit comes from not having to change nodes to the `Done`
state in `process_cycles`.
`process_obligations()` adds new nodes, but it does not process these
new nodes until the next time it is called.
This commit changes it so that it does process these new nodes within
the same call. This change reduces the number of calls to
`process_obligations()` required to complete processing, sometimes
giving significant speed-ups.
The change required some changes to tests.
- The output of `cycle-cache-err-60010.rs` is slightly different.
- The unit tests required extra cases to handle the earlier processing
of the added nodes. I mostly did these in the simplest possible way,
by making the added nodes be ignored, thus giving outcomes the same as
with the old behaviour. But I changed `success_in_grandchildren()`
more extensively so that some obligations are completed earlier than
they used to be.
By collecting the done obligations (when necessary) in the main loop.
This makes the code cleaner.
The commit also changes the order in which successful obligations are
returned -- they are now returned in the registered order, rather than
reversed. Because this order doesn't actually matter, being only used by
tests, the commit uses `sort()` to make the test agnostic w.r.t. the
order.
It's not necessary; cycles (which are rare) can be detected by looking
at the node stack.
This change speeds things up slightly, as well as simplifying the code a
little.
The name `waiting_cache` sounds like it is related to the states
`NodeState::Waiting`, but it's not; the cache holds nodes of various
states.
This commit changes it to `active_state`.
Now that all indices have type `usize`, it makes sense to be more
consistent about their naming. This commit removes all uses of `i` in
favour of `index`.
The size of the indices doesn't matter much here, and having a
`newtype_index!` index type without also using `IndexVec` requires lots
of conversions. So this commit removes `NodeIndex` in favour of uniform
use of `usize` as the index type. As well as making the code slightly
more concise, it also slightly speeds things up.
`Node` has an optional parent and a list of other descendents. Most of
the time the parent is treated the same as the other descendents --
error-handling is the exception -- and chaining them together for
iteration has a non-trivial cost.
This commit changes the representation. There is now a single list of
descendants, and a boolean flag that indicates if there is a parent (in
which case it is first descendent). This representation encodes the same
information, in a way that is less idiomatic but cheaper to iterate over
for the common case where the parent doesn't need special treatment.
As a result, some benchmark workloads are up to 2% faster.
This makes the code a little faster, presumably because bounds checks
aren't needed on `nodes` accesses. It requires making `scratch` a
`RefCell`, which is not unreasonable.
This commit removes the custom index implementation of `NodeIndex`,
which probably predates `newtype_index!`.
As well as eliminating code, it improves the debugging experience,
because the custom implementation had the property of being incremented
by 1 (so it could use `NonZeroU32`), which was incredibly confusing if
you didn't expect it.
For some reason, I also had to remove an `unsafe` block marker from
`from_u32_unchecked()` that the compiler said was now unnecessary.
This function is very hot, doesn't get inlined because it's recursive,
and the function calls are significant.
This commit splits it into inlined and uninlined variants, and uses the
inlined variant for the hot call site. This wins several percent on a
few benchmarks.
Tshepang Lekhonkhobe