When the value held by SCEVUnknown is removed, we will call forgetMemoizedValues() on it in https://github.com/llvm/llvm-project/blob/d5de568cc7375281b14bd2632576bff7f4afabc3/llvm/lib/Analysis/ScalarEvolution.cpp#L504.
After recent changes, this will also forget memoized values on recursive users. After D113349 this additionally clears the values from the ValueExprMap. This means that the nullptr SCEVUnknown will no longer be directly or indirectly part of ValueExprMap, and we can drop the validity check in getSCEV().
This simplifies our invariants (we essentially remove the notion of an "invalid" SCEV) and saves a bit of compile-time: https://llvm-compile-time-tracker.com/compare.php?from=38ff05f329cb3116c6c4f46a34318b421fda4331&to=20d2f513a21cfd9678df091499f6a6226d9fca89&stat=instructions