[NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions

[kmitropoulou]

Summary

Load coercion is done in two phases:

1. Collection of loads: First, we collect the loads that have a dependency with stores. The loads along with their dependencies are added in LoadCoercion map.

2. Code generation: Iterate over the loads of LoadCoercion map and replace the loads with the value that we extract from the stores. This is done in implementLoadCoercion().

New instructions are generated during load coercion. In order to eliminate them, we run value numbering for them. For this reason, the load coercion should be completed before the elimination phase.

[fhahn]

FWIW I am not sure if we should be adding new features to NewGVN at the moment. There are multiple correctness issues of varying degrees. To have a credible path for NewGVN to become useful eventually, I think we should first prioritize fixing the known issues.

llvm/lib/Transforms/Scalar/NewGVN.cpp

3908

I don't think modifying the IR directly here and re-running numbering afterwards really fits with the NewGVN model. If the load is congruent to a known expression, couldn't the load be moved to the same congruence class as that expression and then the existing replace and erase logic will take care of the rest?

llvm/test/Transforms/NewGVN/load_coercion_between_store_and_load.ll

2–3

please keep the RUN line so that the diff shows the functional changes in newgvn. As it is now, it is very hard to see the changes with newgvn

[kmitropoulou]

Updating D124071: [1/3][NewGVN][LoadCoercion] Add support for load coercion between store and load instructions.

[kmitropoulou]

llvm/lib/Transforms/Scalar/NewGVN.cpp

3908

Good point! This was my initial approach too. But, I ended up having some problems. NewGVN adds instructions with similar expressions in the same congruence class. In the elimination phase, the algorithm traverses all the congruence classes and it replaces each member of the congruence with the leader of the congruence class.

Let's assume that we have to do load coercion between a vector store and a scalar load. The current algorithm will not add the store and the load in the same congruence class. We have to do a small hack to do this.

Now, the store and the load are in the same congruence class. In the elimination phase, we can check if any of the members of the congruence class are in LoadCoercion map. If yes, then we can extract the loaded value from the leader of the congruence class (stored value) by generating the right sequence of instructions. This is done at the beginning of the elimination phase. Next, we remove the load from the congruence class because: i. we have to prevent the algorithm from processing this load again and ii. we have to let the algorithm eliminate the other members of the congruence class without a problem. In other words, the load is treated as a special case in the elimination phase. IMO, this does not make much sense since we do not use the elimination phase at all. It is much better to have a separate step that deals with all the loads that need to be optimized with load coercion. After that, we can run the elimination phase without any problem. This is what the current patch does.

Another problem is that it is not expected to emit instructions during the elimination phase. As a result, it is hard to optimize them. Load coercion emits a new sequence of instructions. For example, in test13, the two loads are replaced by %1 = trunc i32 %V1 to i8. The code has already %V4 = trunc i32 %V1 to i8. The elimination phase will not optimize the two truncates. Because, it traverses the congruence classes in reverse order. So, it first processes the %V4 = trunc i32 %V1 to i8. Next, it processes the congruence class with the two loads that should be optimized with load coercion. It emits the truncate and the value numbering adds the newly generated truncate in the same class as %V4. But, this class has already been processed. As a result, we miss to eliminate %V4 by %1 (or vice versa). To deal with this problem, we have to bookkeep the classes that were updated and next, we should run the elimination phase for these classes again. The current patch does not have this problem, because the newly generated instructions are added in the corresponding congruence classes before the elimination phase.

Another problem is that the elimination phase works without problems if the instructions have correct DFS numbers. If we put the load that needs to be optimized with load coercion in a congruence class, then we have to update the DFS numbers because of the newly generated instructions. This means that we might have to update the DFS numbers several times. This patch updates the DFS numbers only once.

IMO, this patch is more explicit and more clean since it tries to do the load coercion without complicating the elimination phase and by doing minimal changes.

[kmitropoulou]

FWIW I am not sure if we should be adding new features to NewGVN at the moment. There are multiple correctness issues of varying degrees. To have a credible path for NewGVN to become useful eventually, I think we should first prioritize fixing the known issues.

@fhahn : Can you please point which of these problems affect load coercion? I might try to address them.

[kmitropoulou]

Updating D124071: [1/3][NewGVN][LoadCoercion] Add support for load coercion between store and load instructions.

[kmitropoulou]

ping

[kmitropoulou]

[NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions.

[xbolva00]

llvm/test/Transforms/NewGVN/load_coercion_between_store_and_load.ll

355

GVN fails to remove dead instructions?

[kmitropoulou]

llvm/test/Transforms/NewGVN/load_coercion_between_store_and_load.ll

355

Yes, these instructions are removed by dce. This sometimes happens with NewGVN as well.

[RKSimon]

@fhahn Do you we have anyone leading the GVN/NewGVN work these days do you know? I can't see anything in CODE_OWNERS

[kmitropoulou]

Updating D124071: [NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions

[kmitropoulou]

ping

[kmitropoulou]

Updating D124071: [NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions

[kmitropoulou]

Updating D124071: [NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions

[kmitropoulou]

Updating D124071: [NewGVN][LoadCoercion][1/3] Add support for load coercion between store and load instructions

[asbirlea]

Quick note, since updates on this until now were offline.
After the fix in D130910 that Konstantina did, I was able to do some testing for NewGVN and haven't run into any crashes, which was a first. This doesn't mean that the current NewGVN is fully correct, just that in the testing I did nothing was obviously broken. This does mean I can test the correctness of this stack of patches on the same workloads.
I tested the previous iteration, followed up with reproducers for issues found and I'm testing the current diff.

Regarding:

In D124071#3585047, @RKSimon wrote:

@fhahn Do you we have anyone leading the GVN/NewGVN work these days do you know? I can't see anything in CODE_OWNERS

I'm currently focusing more on testing the GVN&MSSA patches by @momchil.velikov, than getting NewGVN up to parity.
After the above fix, I did some performance testing with NewGVN turned on vs GVN and it's pretty terrible. To give a rough idea, I'm seeing 20-70% regression for running time, and 30% compile-time regression (due to the compiler being bootstrapped; newgvn compiling the compiler makes the new compiler 30% slower). The biggest factor are the PRE optimizations that GVN does, but even comparing a single stage build NewGVN to GVN with PRE disabled, there are still a few test-suites that show up to 60% regressions.
On the flip side, comparing NewGVN vs GVN with no PRE, there are a couple of big improvements on ARM, so it's quite possible there are gains to harness for specific use cases and/or architectures, but there's a lot of work left to make it a replacement for GVN in general.

llvm/lib/Transforms/Scalar/NewGVN.cpp

 // issue.
 static cl::opt<bool> EnableStoreRefinement("enable-store-refinement",
                                            cl::init(false), cl::Hidden);
 
 /// Currently, the generation "phi of ops" can result in correctness issues.
 static cl::opt<bool> EnablePhiOfOps("enable-phi-of-ops", cl::init(true),
                                     cl::Hidden);
 
+// Enables load coercion for non-constant values.
+static cl::opt<bool> EnableLoadCoercion("enable-load-coercion", cl::init(true),
+                                        cl::Hidden);
+
 //===----------------------------------------------------------------------===//
 //                                GVN Pass
 //===----------------------------------------------------------------------===//
 
 // Anchor methods.
 namespace llvm {
 namespace GVNExpression {
 
#endif
   DenseMap<const Value *, unsigned> InstrDFS;
 
   // This contains the mapping DFS numbers to instructions.
   SmallVector<Value *, 32> DFSToInstr;
 
   // Deletion info.
   SmallPtrSet<Instruction *, 8> InstructionsToErase;
 
+  // Maps the loads to their depending instructions.
+  std::map<Value *, SmallPtrSet<Instruction *, 2>> LoadCoercion;
+
 public:
   NewGVN(Function &F, DominatorTree *DT, AssumptionCache *AC,
          TargetLibraryInfo *TLI, AliasAnalysis *AA, MemorySSA *MSSA,
          const DataLayout &DL)
       : F(F), DT(DT), TLI(TLI), AA(AA), MSSA(MSSA), AC(AC), DL(DL),
         PredInfo(std::make_unique<PredicateInfo>(F, *DT, *AC)),
         SQ(DL, TLI, DT, AC, /*CtxI=*/nullptr, /*UseInstrInfo=*/false,
            /*CanUseUndef=*/false) {}
private:
   }
 
   bool isCycleFree(const Instruction *) const;
   bool isBackedge(BasicBlock *From, BasicBlock *To) const;
 
   // Debug counter info.  When verifying, we have to reset the value numbering
   // debug counter to the same state it started in to get the same results.
   int64_t StartingVNCounter = 0;
+
+  // The following functions are used in load coercion:
+  // Try to add the load along with the depending instrunction(s) in
+  // LoadCoercion map.
+  void tryAddLoadDepInsnIntoLoadCoercionMap(LoadInst *, Instruction *) const;
+  // Eliminates the loads of the LoadCoercion map.
+  bool implementLoadCoercion();
+  // Run value numbering for the instructions that are generated during load
+  // coercion.
+  void runValueNumberingForLoadCoercionInsns(Instruction *);
+  // Extract from the depending instruction the value that will replace the
+  // load.
+  Value *getExtractedValue(LoadInst *, Instruction *);
 };
 
 } // end anonymous namespace
 
 template <typename T>
 static bool equalsLoadStoreHelper(const T &LHS, const Expression &RHS) {
   if (!isa<LoadExpression>(RHS) && !isa<StoreExpression>(RHS))
     return false;
const Expression *NewGVN::performSymbolicStoreEvaluation(Instruction *I) const {
   }
 
   // If the store is not equivalent to anything, value number it as a store that
   // produces a unique memory state (instead of using it's MemoryUse, we use
   // it's MemoryDef).
   return createStoreExpression(SI, StoreAccess);
 }
 
+void NewGVN::tryAddLoadDepInsnIntoLoadCoercionMap(
+    LoadInst *LI, Instruction *CurrentDepI) const {
+  // Add the load and the corresponding depending instruction in LoadCoercion
+  // map.
+  const_cast<NewGVN *>(this)->LoadCoercion[LI].insert(CurrentDepI);
+}
+
 // See if we can extract the value of a loaded pointer from a load, a store, or
 // a memory instruction.
 const Expression *
 NewGVN::performSymbolicLoadCoercion(Type *LoadType, Value *LoadPtr,
                                     LoadInst *LI, Instruction *DepInst,
                                     MemoryAccess *DefiningAccess) const {
   assert((!LI || LI->isSimple()) && "Not a simple load");
   if (auto *DepSI = dyn_cast<StoreInst>(DepInst)) {
     // Can't forward from non-atomic to atomic without violating memory model.
     // Also don't need to coerce if they are the same type, we will just
     // propagate.
     if (LI->isAtomic() > DepSI->isAtomic() ||
         LoadType == DepSI->getValueOperand()->getType())
       return nullptr;
     int Offset = analyzeLoadFromClobberingStore(LoadType, LoadPtr, DepSI, DL);
     if (Offset >= 0) {
       if (auto *C = dyn_cast<Constant>(
               lookupOperandLeader(DepSI->getValueOperand()))) {
         LLVM_DEBUG(dbgs() << "Coercing load from store " << *DepSI
                           << " to constant " << *C << "\n");
         return createConstantExpression(
             getConstantStoreValueForLoad(C, Offset, LoadType, DL));
+      } else if (EnableLoadCoercion) {
+        tryAddLoadDepInsnIntoLoadCoercionMap(LI, DepInst);
+        // Load coercion occurs before the elimination phase. The loads that
+        // will be eliminated with load coercion are not added in any congruence
+        // class. Thus, we do not create any load expression for them.
+        return nullptr;
       }
     }
   } else if (auto *DepLI = dyn_cast<LoadInst>(DepInst)) {
     // Can't forward from non-atomic to atomic without violating memory model.
     if (LI->isAtomic() > DepLI->isAtomic())
       return nullptr;
     int Offset = analyzeLoadFromClobberingLoad(LoadType, LoadPtr, DepLI, DL);
     if (Offset >= 0) {
#endif
   DFSToInstr.clear();
   BlockInstRange.clear();
   TouchedInstructions.clear();
   MemoryAccessToClass.clear();
   PredicateToUsers.clear();
   MemoryToUsers.clear();
   RevisitOnReachabilityChange.clear();
   IntrinsicInstPred.clear();
+  LoadCoercion.clear();
 }
 
 // Assign local DFS number mapping to instructions, and leave space for Value
 // PHI's.
 std::pair<unsigned, unsigned> NewGVN::assignDFSNumbers(BasicBlock *B,
                                                        unsigned Start) {
   unsigned End = Start;
   if (MemoryAccess *MemPhi = getMemoryAccess(B)) {
LLVM_DEBUG(dbgs() << "Block " << getBlockName(&F.getEntryBlock())
                     << " marked reachable\n");
   ReachableBlocks.insert(&F.getEntryBlock());
 
   iterateTouchedInstructions();
   verifyMemoryCongruency();
   verifyIterationSettled(F);
   verifyStoreExpressions();
 
+  // During load coercion, we replace the load instructions with a new sequence
+  // of instructions. Next, we run value numbering which adds the new
+  // instructions in the right congruence classes. In this way, any redundant
+  // instructions will be optimized out in the elimination phase.
+  if (EnableLoadCoercion && implementLoadCoercion())
+    // Update the newly generated instructions with the correct DFS numbers.
+    // TODO: Update DFS numbers faster.
+    ICount = updateDFSNumbers(ICount);
+
   Changed |= eliminateInstructions(F);
 
   // Delete all instructions marked for deletion.
   for (Instruction *ToErase : InstructionsToErase) {
     if (!ToErase->use_empty())
       ToErase->replaceAllUsesWith(PoisonValue::get(ToErase->getType()));
 
     assert(ToErase->getParent() &&
for (auto Member : *CC) {
     if (!MemberInst)
       return Member;
     if (DT->dominates(getBlockForValue(MemberInst), BB))
       return Member;
   }
   return nullptr;
 }
 
+// Run value numbering for the instructions that are generated during load
+// coercion. In this way, any redundant instructions will be removed in the
+// elimination phase.
+void NewGVN::runValueNumberingForLoadCoercionInsns(Instruction *I) {
+  TOPClass->insert(I);
+  ValueToClass[I] = TOPClass;
+  valueNumberInstruction(I);
+  updateProcessedCount(I);
+}
+
+// Extract the correct value from the depending instruction.
+Value *NewGVN::getExtractedValue(LoadInst *LI, Instruction *DepI) {
+
+  Type *LoadTy = LI->getType();
+  Value *NewValue = nullptr;
+  Instruction *InsertPtr = nullptr;
+  if (auto *Store = dyn_cast<StoreInst>(DepI)) {
+    int Offset = analyzeLoadFromClobberingStore(LoadTy, LI->getPointerOperand(),
+                                                Store, DL);
+    InsertPtr = Store->getNextNode();
+    // Emit the instructions that extract the correct value from store.
+    NewValue = getStoreValueForLoad(Store->getValueOperand(), Offset, LoadTy,
+                                    InsertPtr, DL);
+  } else if (LoadInst *Load = dyn_cast<LoadInst>(DepI)) {
+    int Offset = analyzeLoadFromClobberingLoad(LoadTy, LI->getPointerOperand(),
+                                               Load, DL);
+    InsertPtr = Load->getNextNode();
+    // Emit the instructions that extract the correct value from load.
+    NewValue = getLoadValueForLoad(Load, Offset, LoadTy, InsertPtr, DL);
+  }
+
+  if (!isa<Constant>(NewValue) && !isa<Argument>(NewValue))
+    for (Instruction *CurInsn = DepI->getNextNode(); CurInsn != InsertPtr;
+         CurInsn = CurInsn->getNextNode())
+      runValueNumberingForLoadCoercionInsns(CurInsn);
+
+  return NewValue;
+}
+
+// Iterate over the load instructions of LoadCoercion map and it replaces
+// them with the right sequence of instructions.
+bool NewGVN::implementLoadCoercion() {
+  bool AnythingReplaced = false;
+  for (const auto &P : LoadCoercion) {
+    LoadInst *LI = cast<LoadInst>(P.first);
+    SmallPtrSet<Instruction *, 2> DependingInsns = P.second;
+    Value *NewValue = nullptr;
+    Instruction *FirstDepI = *DependingInsns.begin();
+
+    if (DependingInsns.size() == 1 && DT->dominates(FirstDepI, LI))
+      // Extract the correct value from the depending instruction.
+      NewValue = getExtractedValue(LI, FirstDepI);
+
+    // Load coercion was rejected.
+    if (!NewValue)
+      continue;
+
+    // Remove the load and update all of its uses.
+    InstructionsToErase.insert(LI);
+    LI->replaceAllUsesWith(NewValue);
+    if (isa<PHINode>(NewValue))
+      NewValue->takeName(LI);
+    if (Instruction *I = dyn_cast<Instruction>(NewValue))
+      I->setDebugLoc(LI->getDebugLoc());
+    LLVM_DEBUG(dbgs() << "Load coersion: The load " << *LI
+                      << " was eliminated and its uses were replaced by "
+                      << *NewValue << "\n");
+    AnythingReplaced = true;
+  }
+
+  return AnythingReplaced;
+}
+
 bool NewGVN::eliminateInstructions(Function &F) {
   // This is a non-standard eliminator. The normal way to eliminate is
   // to walk the dominator tree in order, keeping track of available
   // values, and eliminating them.  However, this is mildly
   // pointless. It requires doing lookups on every instruction,
   // regardless of whether we will ever eliminate it.  For
   // instructions part of most singleton congruence classes, we know we
   // will never eliminate them.

llvm/test/Transforms/NewGVN/load_coercion_between_loads.ll

 ; OLDGVN-NEXT:    ret i8 [[V6]]
 ;
 ; NEWGVN-LABEL: @test12(
 ; NEWGVN-NEXT:    [[V1:%.*]] = load i32, i32* [[P1:%.*]], align 4
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
 ; NEWGVN-NEXT:    [[V3:%.*]] = trunc i32 [[V1]] to i8
 ; NEWGVN-NEXT:    store i32 [[V:%.*]], i32* [[P1]], align 4
-; NEWGVN-NEXT:    [[V4:%.*]] = load i8, i8* [[P2]], align 1
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V]] to i8
 ; NEWGVN-NEXT:    [[V5:%.*]] = add i8 [[V2]], [[V3]]
-; NEWGVN-NEXT:    [[V6:%.*]] = add i8 [[V4]], [[V5]]
+; NEWGVN-NEXT:    [[V6:%.*]] = add i8 [[TMP1]], [[V5]]
 ; NEWGVN-NEXT:    ret i8 [[V6]]
 ;
   %V1 = load i32, i32* %P1
   %P2 = bitcast i32* %P1 to i8*
   %V2 = load i8, i8* %P2
   %V3 = trunc i32 %V1 to i8
   store i32 %V, i32* %P1
   %P4 = bitcast i32* %P1 to i8*

llvm/test/Transforms/NewGVN/load_coercion_between_store_and_load.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S -gvn < %s | FileCheck %s -check-prefixes=GVN,OLDGVN
 ; RUN: opt -S -newgvn < %s | FileCheck %s -check-prefixes=GVN,NEWGVN
 
 define float @test1(i32 %V1, i32* %P) {
 ; OLDGVN-LABEL: @test1(
 ; OLDGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P:%.*]], align 4
 ; OLDGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = bitcast i32 [[V1]] to float
 ; OLDGVN-NEXT:    ret float [[TMP1]]
 ;
 ; NEWGVN-LABEL: @test1(
 ; NEWGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = bitcast i32 [[V1]] to float
 ; NEWGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
-; NEWGVN-NEXT:    [[V2:%.*]] = load float, float* [[P1]], align 4
-; NEWGVN-NEXT:    ret float [[V2]]
+; NEWGVN-NEXT:    ret float [[TMP1]]
 ;
   store i32 %V1, i32* %P
   %P1 = bitcast i32* %P to float*
   %V2 = load float, float* %P1
   ret float %V2
 }
 
 define float @test2(i64* %V1, i64** %P1) {
 ; OLDGVN-LABEL: @test2(
 ; OLDGVN-NEXT:    store i64* [[V1:%.*]], i64** [[P1:%.*]], align 8
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i64** [[P1]] to float*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = ptrtoint i64* [[V1]] to i64
 ; OLDGVN-NEXT:    [[TMP2:%.*]] = trunc i64 [[TMP1]] to i32
 ; OLDGVN-NEXT:    [[TMP3:%.*]] = bitcast i32 [[TMP2]] to float
 ; OLDGVN-NEXT:    ret float [[TMP3]]
 ;
 ; NEWGVN-LABEL: @test2(
 ; NEWGVN-NEXT:    store i64* [[V1:%.*]], i64** [[P1:%.*]], align 8
+; NEWGVN-NEXT:    [[TMP1:%.*]] = ptrtoint i64* [[V1]] to i64
+; NEWGVN-NEXT:    [[TMP2:%.*]] = trunc i64 [[TMP1]] to i32
+; NEWGVN-NEXT:    [[TMP3:%.*]] = bitcast i32 [[TMP2]] to float
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i64** [[P1]] to float*
-; NEWGVN-NEXT:    [[V2:%.*]] = load float, float* [[P2]], align 4
-; NEWGVN-NEXT:    ret float [[V2]]
+; NEWGVN-NEXT:    ret float [[TMP3]]
 ;
   store i64* %V1, i64** %P1
   %P2 = bitcast i64** %P1 to float*
   %V2 = load float, float* %P2
   ret float %V2
 }
 
 define i8 @test3(i32 %V1, i32* %P1) {
 ; OLDGVN-LABEL: @test3(
 ; OLDGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
 ; OLDGVN-NEXT:    ret i8 [[TMP1]]
 ;
 ; NEWGVN-LABEL: @test3(
 ; NEWGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
-; NEWGVN-NEXT:    ret i8 [[V2]]
+; NEWGVN-NEXT:    ret i8 [[TMP1]]
 ;
   store i32 %V1, i32* %P1
   %P2 = bitcast i32* %P1 to i8*
   %V2 = load i8, i8* %P2
   ret i8 %V2
 }
 
 define float @test4(i64 %V1, i64* %P1) {
 ; OLDGVN-LABEL: @test4(
 ; OLDGVN-NEXT:    store i64 [[V1:%.*]], i64* [[P1:%.*]], align 4
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i64* [[P1]] to float*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = trunc i64 [[V1]] to i32
 ; OLDGVN-NEXT:    [[TMP2:%.*]] = bitcast i32 [[TMP1]] to float
 ; OLDGVN-NEXT:    ret float [[TMP2]]
 ;
 ; NEWGVN-LABEL: @test4(
 ; NEWGVN-NEXT:    store i64 [[V1:%.*]], i64* [[P1:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i64 [[V1]] to i32
+; NEWGVN-NEXT:    [[TMP2:%.*]] = bitcast i32 [[TMP1]] to float
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i64* [[P1]] to float*
-; NEWGVN-NEXT:    [[V2:%.*]] = load float, float* [[P2]], align 4
-; NEWGVN-NEXT:    ret float [[V2]]
+; NEWGVN-NEXT:    ret float [[TMP2]]
 ;
   store i64 %V1, i64* %P1
   %P2 = bitcast i64* %P1 to float*
   %V2 = load float, float* %P2
   ret float %V2
 }
 
 define i8 @test5(i32* %P, i8* %T) {

 ; OLDGVN-LABEL: @test6(
 ; OLDGVN-NEXT:    store i64 [[V1:%.*]], i64* [[P1:%.*]], align 4
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i64* [[P1]] to i8**
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = inttoptr i64 [[V1]] to i8*
 ; OLDGVN-NEXT:    ret i8* [[TMP1]]
 ;
 ; NEWGVN-LABEL: @test6(
 ; NEWGVN-NEXT:    store i64 [[V1:%.*]], i64* [[P1:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = inttoptr i64 [[V1]] to i8*
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i64* [[P1]] to i8**
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8*, i8** [[P2]], align 8
-; NEWGVN-NEXT:    ret i8* [[V2]]
+; NEWGVN-NEXT:    ret i8* [[TMP1]]
 ;
   store i64 %V1, i64* %P1
   %P2 = bitcast i64* %P1 to i8**
   %V2 = load i8*, i8** %P2
   ret i8* %V2
 }
 
 define i32 @test7(double %V1, double* %P1) {
 ; OLDGVN-LABEL: @test7(
 ; OLDGVN-NEXT:    store double [[V1:%.*]], double* [[P1:%.*]], align 8
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast double* [[P1]] to i32*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = bitcast double [[V1]] to i64
 ; OLDGVN-NEXT:    [[TMP2:%.*]] = trunc i64 [[TMP1]] to i32
 ; OLDGVN-NEXT:    ret i32 [[TMP2]]
 ;
 ; NEWGVN-LABEL: @test7(
 ; NEWGVN-NEXT:    store double [[V1:%.*]], double* [[P1:%.*]], align 8
+; NEWGVN-NEXT:    [[TMP1:%.*]] = bitcast double [[V1]] to i64
+; NEWGVN-NEXT:    [[TMP2:%.*]] = trunc i64 [[TMP1]] to i32
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast double* [[P1]] to i32*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i32, i32* [[P2]], align 4
-; NEWGVN-NEXT:    ret i32 [[V2]]
+; NEWGVN-NEXT:    ret i32 [[TMP2]]
 ;
   store double %V1, double* %P1
   %P2 = bitcast double* %P1 to i32*
   %V2 = load i32, i32* %P2
   ret i32 %V2
 }
 
 define i8 @test8(i32 %V1, i32* %P1) {
 ; OLDGVN-LABEL: @test8(
 ; OLDGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; OLDGVN-NEXT:    [[P3:%.*]] = getelementptr i8, i8* [[P2]], i32 2
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = lshr i32 [[V1]], 16
 ; OLDGVN-NEXT:    [[TMP2:%.*]] = trunc i32 [[TMP1]] to i8
 ; OLDGVN-NEXT:    ret i8 [[TMP2]]
 ;
 ; NEWGVN-LABEL: @test8(
 ; NEWGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = lshr i32 [[V1]], 16
+; NEWGVN-NEXT:    [[TMP2:%.*]] = trunc i32 [[TMP1]] to i8
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; NEWGVN-NEXT:    [[P3:%.*]] = getelementptr i8, i8* [[P2]], i32 2
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P3]], align 1
-; NEWGVN-NEXT:    ret i8 [[V2]]
+; NEWGVN-NEXT:    ret i8 [[TMP2]]
 ;
   store i32 %V1, i32* %P1
   %P2 = bitcast i32* %P1 to i8*
   %P3 = getelementptr i8, i8* %P2, i32 2
   %V2 = load i8, i8* %P3
   ret i8 %V2
 }
 

 ; OLDGVN:       T:
 ; OLDGVN-NEXT:    ret double [[TMP0]]
 ; OLDGVN:       F:
 ; OLDGVN-NEXT:    ret double [[TMP0]]
 ;
 ; NEWGVN-LABEL: @test9(
 ; NEWGVN-NEXT:  Entry:
 ; NEWGVN-NEXT:    store i64 [[V:%.*]], i64* [[P1:%.*]], align 4
-; NEWGVN-NEXT:    [[P3:%.*]] = bitcast i64* [[P1]] to double*
+; NEWGVN-NEXT:    [[TMP0:%.*]] = bitcast i64 [[V]] to double
 ; NEWGVN-NEXT:    br i1 [[COND:%.*]], label [[T:%.*]], label [[F:%.*]]
 ; NEWGVN:       T:
-; NEWGVN-NEXT:    [[B:%.*]] = load double, double* [[P3]], align 8
-; NEWGVN-NEXT:    ret double [[B]]
+; NEWGVN-NEXT:    ret double [[TMP0]]
 ; NEWGVN:       F:
-; NEWGVN-NEXT:    [[C:%.*]] = load double, double* [[P3]], align 8
-; NEWGVN-NEXT:    ret double [[C]]
+; NEWGVN-NEXT:    ret double [[TMP0]]
 ;
 Entry:
   %A = load i64 , i64* %P1
   store i64 %V, i64* %P1
   %P3 = bitcast i64* %P1 to double*
   br i1 %cond, label %T, label %F
 T:
   %B = load double, double* %P3

 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to i8*
 ; OLDGVN-NEXT:    [[TMP2:%.*]] = trunc i32 [[V0]] to i8
 ; OLDGVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, i8 [[TMP2]], 0
 ; OLDGVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> [[I1]], float [[TMP1]], 1
 ; OLDGVN-NEXT:    ret <{ i8, float }> [[I2]]
 ;
 ; NEWGVN-LABEL: @test10(
 ; NEWGVN-NEXT:    store i32 [[V0:%.*]], i32* [[P:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V0]] to i8
+; NEWGVN-NEXT:    [[TMP2:%.*]] = bitcast i32 [[V0]] to float
 ; NEWGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
-; NEWGVN-NEXT:    [[V1:%.*]] = load float, float* [[P1]], align 4
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to i8*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
-; NEWGVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, i8 [[V2]], 0
-; NEWGVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> [[I1]], float [[V1]], 1
+; NEWGVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, i8 [[TMP1]], 0
+; NEWGVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> [[I1]], float [[TMP2]], 1
 ; NEWGVN-NEXT:    ret <{ i8, float }> [[I2]]
 ;
   store i32 %V0, i32* %P
   %P1 = bitcast i32* %P to float*
   %V1 = load float, float* %P1
   %P2 = bitcast i32* %P to i8*
   %V2 = load i8, i8* %P2
   %I1 = insertvalue <{i8, float}> poison, i8 %V2, 0
   %I2 = insertvalue <{i8, float}> %I1, float %V1, 1
   ret <{i8, float}> %I2
 }
 
 define <{i8, float}> @test11(i32 %V0, i32* %P, i1 %cond) {
 ;   Entry
 ;    /  \
 ;   T    F
 ;
-; OLDGVN-LABEL: @test11(
-; OLDGVN-NEXT:  Entry:
-; OLDGVN-NEXT:    store i32 [[V0:%.*]], i32* [[P:%.*]], align 4
-; OLDGVN-NEXT:    [[TMP0:%.*]] = trunc i32 [[V0]] to i8
-; OLDGVN-NEXT:    [[TMP1:%.*]] = bitcast i32 [[V0]] to float
-; OLDGVN-NEXT:    br i1 [[COND:%.*]], label [[T:%.*]], label [[F:%.*]]
-; OLDGVN:       T:
-; OLDGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
-; OLDGVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, float [[TMP1]], 1
-; OLDGVN-NEXT:    ret <{ i8, float }> [[I1]]
-; OLDGVN:       F:
-; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to i8*
-; OLDGVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> poison, i8 [[TMP0]], 0
-; OLDGVN-NEXT:    ret <{ i8, float }> [[I2]]
-;
-; NEWGVN-LABEL: @test11(
-; NEWGVN-NEXT:  Entry:
-; NEWGVN-NEXT:    store i32 [[V0:%.*]], i32* [[P:%.*]], align 4
-; NEWGVN-NEXT:    br i1 [[COND:%.*]], label [[T:%.*]], label [[F:%.*]]
-; NEWGVN:       T:
-; NEWGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
-; NEWGVN-NEXT:    [[V1:%.*]] = load float, float* [[P1]], align 4
-; NEWGVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, float [[V1]], 1
-; NEWGVN-NEXT:    ret <{ i8, float }> [[I1]]
-; NEWGVN:       F:
-; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to i8*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
-; NEWGVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> poison, i8 [[V2]], 0
-; NEWGVN-NEXT:    ret <{ i8, float }> [[I2]]
+; GVN-LABEL: @test11(
+; GVN-NEXT:  Entry:
+; GVN-NEXT:    store i32 [[V0:%.*]], i32* [[P:%.*]], align 4
+; GVN-NEXT:    [[TMP0:%.*]] = trunc i32 [[V0]] to i8
+; GVN-NEXT:    [[TMP1:%.*]] = bitcast i32 [[V0]] to float
+; GVN-NEXT:    br i1 [[COND:%.*]], label [[T:%.*]], label [[F:%.*]]
+; GVN:       T:
+; GVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
+; GVN-NEXT:    [[I1:%.*]] = insertvalue <{ i8, float }> poison, float [[TMP1]], 1
+; GVN-NEXT:    ret <{ i8, float }> [[I1]]
+; GVN:       F:
+; GVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to i8*
+; GVN-NEXT:    [[I2:%.*]] = insertvalue <{ i8, float }> poison, i8 [[TMP0]], 0
+; GVN-NEXT:    ret <{ i8, float }> [[I2]]
 ;
 Entry:
   store i32 %V0, i32* %P
   br i1 %cond, label %T, label %F
 
 T:
   %P1 = bitcast i32* %P to float*
   %V1 = load float, float* %P1

 ; OLDGVN:       F:
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to float*
 ; OLDGVN-NEXT:    [[I2:%.*]] = insertvalue <{ float, float }> poison, float [[TMP0]], 0
 ; OLDGVN-NEXT:    ret <{ float, float }> [[I2]]
 ;
 ; NEWGVN-LABEL: @test12(
 ; NEWGVN-NEXT:  Entry:
 ; NEWGVN-NEXT:    store i32 [[V0:%.*]], i32* [[P:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP0:%.*]] = bitcast i32 [[V0]] to float
 ; NEWGVN-NEXT:    br i1 [[COND:%.*]], label [[T:%.*]], label [[F:%.*]]
 ; NEWGVN:       T:
-; NEWGVN-NEXT:    [[P1:%.*]] = bitcast i32* [[P]] to float*
-; NEWGVN-NEXT:    [[V1:%.*]] = load float, float* [[P1]], align 4
-; NEWGVN-NEXT:    [[I1:%.*]] = insertvalue <{ float, float }> poison, float [[V1]], 1
+; NEWGVN-NEXT:    [[I1:%.*]] = insertvalue <{ float, float }> poison, float [[TMP0]], 1
 ; NEWGVN-NEXT:    ret <{ float, float }> [[I1]]
 ; NEWGVN:       F:
-; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P]] to float*
-; NEWGVN-NEXT:    [[V2:%.*]] = load float, float* [[P2]], align 4
-; NEWGVN-NEXT:    [[I2:%.*]] = insertvalue <{ float, float }> poison, float [[V2]], 0
+; NEWGVN-NEXT:    [[I2:%.*]] = insertvalue <{ float, float }> poison, float [[TMP0]], 0
 ; NEWGVN-NEXT:    ret <{ float, float }> [[I2]]
 ;
 Entry:
   store i32 %V0, i32* %P
   br i1 %cond, label %T, label %F
 
 T:
   %P1 = bitcast i32* %P to float*

 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
 ; OLDGVN-NEXT:    [[P3:%.*]] = bitcast i32* [[P1]] to i64*
 ; OLDGVN-NEXT:    [[V5:%.*]] = add i8 [[TMP1]], [[TMP1]]
 ; OLDGVN-NEXT:    ret i8 [[V5]]
 ;
 ; NEWGVN-LABEL: @test13(
 ; NEWGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
 ; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
 ; NEWGVN-NEXT:    [[P3:%.*]] = bitcast i32* [[P1]] to i64*
-; NEWGVN-NEXT:    [[V4:%.*]] = trunc i32 [[V1]] to i8
-; NEWGVN-NEXT:    [[V5:%.*]] = add i8 [[V2]], [[V4]]
+; NEWGVN-NEXT:    [[V5:%.*]] = add i8 [[TMP1]], [[TMP1]]
 ; NEWGVN-NEXT:    ret i8 [[V5]]
 ;
   store i32 %V1, i32* %P1
   %P2 = bitcast i32* %P1 to i8*
   %V2 = load i8, i8* %P2
   %P3 = bitcast i32* %P1 to i64*
   %V3 = load i64, i64* %P3
   %V4 = trunc i32 %V1 to i8
   %V5 = add i8 %V2, %V4
   ret i8 %V5
 }
 
 define i8 @test14(i32* %P1, i32 %V1) {
 ; OLDGVN-LABEL: @test14(
 ; OLDGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
 ; OLDGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
 ; OLDGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
 ; OLDGVN-NEXT:    [[V5:%.*]] = add i8 [[TMP1]], [[TMP1]]
 ; OLDGVN-NEXT:    ret i8 [[V5]]
 ;
 ; NEWGVN-LABEL: @test14(
 ; NEWGVN-NEXT:    store i32 [[V1:%.*]], i32* [[P1:%.*]], align 4
-; NEWGVN-NEXT:    [[P2:%.*]] = bitcast i32* [[P1]] to i8*
-; NEWGVN-NEXT:    [[V2:%.*]] = load i8, i8* [[P2]], align 1
-; NEWGVN-NEXT:    [[V5:%.*]] = add i8 [[V2]], [[V2]]
+; NEWGVN-NEXT:    [[TMP1:%.*]] = trunc i32 [[V1]] to i8
+; NEWGVN-NEXT:    [[V5:%.*]] = add i8 [[TMP1]], [[TMP1]]
 ; NEWGVN-NEXT:    ret i8 [[V5]]
 ;
   store i32 %V1, i32* %P1
   %P2 = bitcast i32* %P1 to i8*
   %V2 = load i8, i8* %P2
   %P3 = bitcast i32* %P1 to i8*
   %V3 = load i8, i8* %P3
   %V5 = add i8 %V2, %V3

llvm/test/Transforms/NewGVN/pr14166-xfail.ll

-; XFAIL: *
 ; RUN: opt -disable-basic-aa -passes=newgvn -S < %s | FileCheck %s
 ; NewGVN fails this due to missing load coercion
 target datalayout = "e-p:32:32:32"
 target triple = "i386-pc-linux-gnu"
 define <2 x i32> @test1() {
   %v1 = alloca <2 x i32>
   call void @anything(<2 x i32>* %v1)
   %v2 = load <2 x i32>, <2 x i32>* %v1