[MergedLoadStoreMotion] Merge stores with conflicting value types

[jrbyrnes]

Summary

Since memory does not have an intrinsic type, we do not need to require value type matching on stores in order to sink them. To facilitate that, this patch finds stores which are sinkable, but have conflicting types, and bitcasts the ValueOperand so they are easily sinkable into a PHINode. Rather than doing fancy analysis to optimally insert the bitcast, we always insert right before the relevant store in the diamond branch. The assumption is that later passes (e.g. GVN, SimplifyCFG) will clean up bitcasts as needed.

[nikic]

Please add tests for a) one store being ptr and the other i64 and b) one store being i64 and the other { i32, i32 }. The former requires the insertion of inttoptr or ptrtoint casts, the latter is not bitcastable. The correct predicate to use here is CastInst::isBitOrNoopPointerCastable() and then IRBuilder::CreateBitOrPointerCast() to materialize the cast.

It's probably simpler and cleaner to explicitly check this than trying to integrate checks into isSameOperationAs() in a way that makes sense.

[arsenm]

Also test pointers with different address spaces

[jrbyrnes]

Add tests for interesting cases.

Check isBitOrNoopPointerCastable when matching stores.

Combine isLoadStoreConflictingType , createBitCast => CreateBitOrPointerCast

Remove isTriviallyBitcasted (this is aleady caught by store->isSimple() and AA->isMustAlias.

[nikic]

The logic here looks fine to me now. I'm not a big fan of the change to isSameOperationAs(), please see the inline suggestion.

llvm/lib/IR/Instruction.cpp

599

With CompareIgnoringType this basically becomes the same as just haveSameSpecialState(). I'd suggest exposing that as a public method rather than adding this flag.

llvm/test/Transforms/MergedLoadStoreMotion/st_sink_conflict_type.ll

4

Probably not needed.

6

fastcc/unnamed_addr not needed

32

Drop the .i.i.i.i suffixes here.

[jrbyrnes]

Expose hasSameSpecialState to replace isSameLoadStoreOperationAs

[arsenm]

llvm/lib/Transforms/Scalar/MergedLoadStoreMotion.cpp

266

This should not introduce addrspacecast. I’m assuming this is filtered by isBitOrNoopPointerCastable

[jrbyrnes]

llvm/lib/Transforms/Scalar/MergedLoadStoreMotion.cpp

266

If the valueOperand of the store are ptrs of conflicting addrspace, they are not sink candidates. isBitOrNoopPointerCastable will return false due to explicit addrspace check (return SrcPtrTy->getAddressSpace() == DestPtrTy->getAddressSpace()). (caught by sink_addrspace test)

Potentially relevant is that MLSM finds sink candidates based on matching pointers of stores (S0->getPointerOperand() == S1->getPointerOperand()). So, we won't be able to match on addressspace casted pointers (sink_addrspace3 test)

[nikic]

LGTM

llvm/include/llvm/IR/Instruction.h

774

opcde -> opcode

I also suspect this doesn't follow clang-format due to overlong comments.

[jrbyrnes]

Comment & clang-format

llvm/include/llvm/IR/Instruction.h

public:
   /// types and any other factors affecting the operation must be the same. This
   /// is similar to isIdenticalTo except the operands themselves don't have to
   /// be identical.
   /// @returns true if the specified instruction is the same operation as
   /// the current one.
   /// Determine if one instruction is the same operation as another.
   bool isSameOperationAs(const Instruction *I, unsigned flags = 0) const LLVM_READONLY;
 
+  /// This function determines if the speficied instruction has the same
+  /// "special" characteristics as the current one. This means that opcode
+  /// specific details are the same. As a common example, if we are comparing
+  /// loads, then hasSameSpecialState would compare the alignments (among
+  /// other things).
+  /// @returns true if the specific instruction has the same opcde specific
+  /// characteristics as the current one. Determine if one instruction has the
+  /// same state as another.
+  bool hasSameSpecialState(const Instruction *I2,
+                           bool IgnoreAlignment = false) const LLVM_READONLY;
+
   /// Return true if there are any uses of this instruction in blocks other than
   /// the specified block. Note that PHI nodes are considered to evaluate their
   /// operands in the corresponding predecessor block.
   bool isUsedOutsideOfBlock(const BasicBlock *BB) const LLVM_READONLY;
 
   /// Return the number of successors that this instruction has. The instruction
   /// must be a terminator.
   unsigned getNumSuccessors() const LLVM_READONLY;

llvm/lib/IR/Instruction.cpp

const char *Instruction::getOpcodeName(unsigned OpCode) {
   case LandingPad:     return "landingpad";
   case CleanupPad:     return "cleanuppad";
   case Freeze:         return "freeze";
 
   default: return "<Invalid operator> ";
   }
 }
 
-/// Return true if both instructions have the same special state. This must be
-/// kept in sync with FunctionComparator::cmpOperations in
+/// This must be kept in sync with FunctionComparator::cmpOperations in
 /// lib/Transforms/IPO/MergeFunctions.cpp.
-static bool haveSameSpecialState(const Instruction *I1, const Instruction *I2,
-                                 bool IgnoreAlignment = false) {
+bool Instruction::hasSameSpecialState(const Instruction *I2,
+                                      bool IgnoreAlignment) const {
+  auto I1 = this;
   assert(I1->getOpcode() == I2->getOpcode() &&
          "Can not compare special state of different instructions");
 
   if (const AllocaInst *AI = dyn_cast<AllocaInst>(I1))
     return AI->getAllocatedType() == cast<AllocaInst>(I2)->getAllocatedType() &&
            (AI->getAlign() == cast<AllocaInst>(I2)->getAlign() ||
             IgnoreAlignment);
   if (const LoadInst *LI = dyn_cast<LoadInst>(I1))

 bool Instruction::isIdenticalToWhenDefined(const Instruction *I) const {
   if (getOpcode() != I->getOpcode() ||
       getNumOperands() != I->getNumOperands() ||
       getType() != I->getType())
     return false;
 
   // If both instructions have no operands, they are identical.
   if (getNumOperands() == 0 && I->getNumOperands() == 0)
-    return haveSameSpecialState(this, I);
+    return this->hasSameSpecialState(I);
 
   // We have two instructions of identical opcode and #operands.  Check to see
   // if all operands are the same.
   if (!std::equal(op_begin(), op_end(), I->op_begin()))
     return false;
 
   // WARNING: this logic must be kept in sync with EliminateDuplicatePHINodes()!
   if (const PHINode *thisPHI = dyn_cast<PHINode>(this)) {
     const PHINode *otherPHI = cast<PHINode>(I);
     return std::equal(thisPHI->block_begin(), thisPHI->block_end(),
                       otherPHI->block_begin());
   }
 
-  return haveSameSpecialState(this, I);
+  return this->hasSameSpecialState(I);
 }
 
 // Keep this in sync with FunctionComparator::cmpOperations in
 // lib/Transforms/IPO/MergeFunctions.cpp.
 bool Instruction::isSameOperationAs(const Instruction *I,
                                     unsigned flags) const {
   bool IgnoreAlignment = flags & CompareIgnoringAlignment;
   bool UseScalarTypes  = flags & CompareUsingScalarTypes;
 
   if (getOpcode() != I->getOpcode() ||
       getNumOperands() != I->getNumOperands() ||
       (UseScalarTypes ?
        getType()->getScalarType() != I->getType()->getScalarType() :
        getType() != I->getType()))
     return false;
 
   // We have two instructions of identical opcode and #operands.  Check to see
   // if all operands are the same type
   for (unsigned i = 0, e = getNumOperands(); i != e; ++i)
     if (UseScalarTypes ?
         getOperand(i)->getType()->getScalarType() !=
           I->getOperand(i)->getType()->getScalarType() :
         getOperand(i)->getType() != I->getOperand(i)->getType())
       return false;
 
-  return haveSameSpecialState(this, I, IgnoreAlignment);
+  return this->hasSameSpecialState(I, IgnoreAlignment);
 }
 
 bool Instruction::isUsedOutsideOfBlock(const BasicBlock *BB) const {
   for (const Use &U : uses()) {
     // PHI nodes uses values in the corresponding predecessor block.  For other
     // instructions, just check to see whether the parent of the use matches up.
     const Instruction *I = cast<Instruction>(U.getUser());
     const PHINode *PN = dyn_cast<PHINode>(I);

llvm/lib/Transforms/Scalar/MergedLoadStoreMotion.cpp

 // 2) Be more aggressive merging memory operations
 // Note that both changes require register pressure control
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/Scalar/MergedLoadStoreMotion.h"
 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/Analysis/GlobalsModRef.h"
+#include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
 
 using namespace llvm;
StoreInst *MergedLoadStoreMotion::canSinkFromBlock(BasicBlock *BB1,
   BasicBlock *BB0 = Store0->getParent();
   for (Instruction &Inst : reverse(*BB1)) {
     auto *Store1 = dyn_cast<StoreInst>(&Inst);
     if (!Store1)
       continue;
 
     MemoryLocation Loc0 = MemoryLocation::get(Store0);
     MemoryLocation Loc1 = MemoryLocation::get(Store1);
-    if (AA->isMustAlias(Loc0, Loc1) && Store0->isSameOperationAs(Store1) &&
+
+    if (AA->isMustAlias(Loc0, Loc1) &&
         !isStoreSinkBarrierInRange(*Store1->getNextNode(), BB1->back(), Loc1) &&
-        !isStoreSinkBarrierInRange(*Store0->getNextNode(), BB0->back(), Loc0)) {
+        !isStoreSinkBarrierInRange(*Store0->getNextNode(), BB0->back(), Loc0) &&
+        Store0->hasSameSpecialState(Store1) &&
+        CastInst::isBitOrNoopPointerCastable(
+            Store0->getValueOperand()->getType(),
+            Store1->getValueOperand()->getType(),
+            Store0->getModule()->getDataLayout()))
       return Store1;
-    }
-  }
   return nullptr;
 }
 
 ///
 /// Create a PHI node in BB for the operands of S0 and S1
 ///
 PHINode *MergedLoadStoreMotion::getPHIOperand(BasicBlock *BB, StoreInst *S0,
                                               StoreInst *S1) {
void MergedLoadStoreMotion::sinkStoresAndGEPs(BasicBlock *BB, StoreInst *S0,
   // Hoist the instruction.
   BasicBlock::iterator InsertPt = BB->getFirstInsertionPt();
   // Intersect optional metadata.
   S0->andIRFlags(S1);
   S0->dropUnknownNonDebugMetadata();
   S0->applyMergedLocation(S0->getDebugLoc(), S1->getDebugLoc());
   S0->mergeDIAssignID(S1);
 
+  // Insert bitcast for conflicting typed stores (or just use original value if
+  // same type).
+  IRBuilder<> Builder(S0);
+  auto Cast = Builder.CreateBitOrPointerCast(S0->getValueOperand(),
+                                             S1->getValueOperand()->getType());
+  S0->setOperand(0, Cast);
+
   // Create the new store to be inserted at the join point.
   StoreInst *SNew = cast<StoreInst>(S0->clone());
   SNew->insertBefore(&*InsertPt);
   // New PHI operand? Use it.
   if (PHINode *NewPN = getPHIOperand(BB, S0, S1))
     SNew->setOperand(0, NewPN);
   S0->eraseFromParent();
   S1->eraseFromParent();

llvm/test/Transforms/MergedLoadStoreMotion/st_sink_conflict_type.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 2
+; RUN: opt -passes=mldst-motion -S < %s | FileCheck %s
+
+define internal void @sink_conflict(ptr %this.64.val, half %val1, i16 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_conflict
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], half [[VAL1:%.*]], i16 [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    [[TMP0:%.*]] = bitcast half [[VAL1]] to i16
+; CHECK-NEXT:    [[TMP1:%.*]] = bitcast i16 [[VAL2]] to half
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    [[VAL2_SINK:%.*]] = phi i16 [ [[TMP0]], [[IF_THEN]] ], [ [[VAL2]], [[IF_ELSE]] ]
+; CHECK-NEXT:    [[VAL1_SINK:%.*]] = phi half [ [[TMP1]], [[IF_THEN]] ], [ [[VAL1]], [[IF_ELSE]] ]
+; CHECK-NEXT:    store i16 [[VAL2_SINK]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store half [[VAL1_SINK]], ptr [[TMP2]], align 2
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store half %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store i16 %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store half %val1, ptr %add.ptr.else, align 2
+  store i16 %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+define internal void @sink_volatile(ptr %this.64.val, half %val1, i16 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_volatile
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], half [[VAL1:%.*]], i16 [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store volatile half [[VAL1]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[TMP0:%.*]] = bitcast i16 [[VAL2]] to half
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    store volatile i16 [[VAL2]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    [[VAL1_SINK:%.*]] = phi half [ [[TMP0]], [[IF_THEN]] ], [ [[VAL1]], [[IF_ELSE]] ]
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store half [[VAL1_SINK]], ptr [[TMP1]], align 2
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store volatile half %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store i16 %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store half %val1, ptr %add.ptr.else, align 2
+  store volatile i16 %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+define internal void @sink_ptr_to_int(ptr %this.64.val, ptr %val1, i64 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_ptr_to_int
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], ptr [[VAL1:%.*]], i64 [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    [[TMP0:%.*]] = ptrtoint ptr [[VAL1]] to i64
+; CHECK-NEXT:    [[TMP1:%.*]] = inttoptr i64 [[VAL2]] to ptr
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    [[VAL2_SINK:%.*]] = phi i64 [ [[TMP0]], [[IF_THEN]] ], [ [[VAL2]], [[IF_ELSE]] ]
+; CHECK-NEXT:    [[VAL1_SINK:%.*]] = phi ptr [ [[TMP1]], [[IF_THEN]] ], [ [[VAL1]], [[IF_ELSE]] ]
+; CHECK-NEXT:    store i64 [[VAL2_SINK]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store ptr [[VAL1_SINK]], ptr [[TMP2]], align 2
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store ptr %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store i64 %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store ptr %val1, ptr %add.ptr.else, align 2
+  store i64 %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+
+
+define internal void @sink_not_castable(ptr %this.64.val, {i32, i32} %val1, i64 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_not_castable
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], { i32, i32 } [[VAL1:%.*]], i64 [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store { i32, i32 } [[VAL1]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[ADD_PTR_THEN:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store i64 [[VAL2]], ptr [[ADD_PTR_THEN]], align 2
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    [[ADD_PTR_ELSE:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store { i32, i32 } [[VAL1]], ptr [[ADD_PTR_ELSE]], align 2
+; CHECK-NEXT:    store i64 [[VAL2]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store {i32, i32} %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store i64 %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store {i32, i32} %val1, ptr %add.ptr.else, align 2
+  store i64 %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+define internal void @sink_addrspace(ptr %this.64.val, ptr %val1, ptr addrspace(1) %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_addrspace
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], ptr [[VAL1:%.*]], ptr addrspace(1) [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store ptr [[VAL1]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[ADD_PTR_THEN:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store ptr addrspace(1) [[VAL2]], ptr [[ADD_PTR_THEN]], align 2
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    [[ADD_PTR_ELSE:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store ptr [[VAL1]], ptr [[ADD_PTR_ELSE]], align 2
+; CHECK-NEXT:    store ptr addrspace(1) [[VAL2]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store ptr %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store ptr addrspace(1) %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store ptr %val1, ptr %add.ptr.else, align 2
+  store ptr addrspace(1) %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+
+define internal void @sink_addrspace2(ptr %this.64.val, ptr addrspace(1) %val1, ptr addrspace(1) %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_addrspace2
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], ptr addrspace(1) [[VAL1:%.*]], ptr addrspace(1) [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    [[VAL2_SINK:%.*]] = phi ptr addrspace(1) [ [[VAL1]], [[IF_THEN]] ], [ [[VAL2]], [[IF_ELSE]] ]
+; CHECK-NEXT:    [[VAL1_SINK:%.*]] = phi ptr addrspace(1) [ [[VAL2]], [[IF_THEN]] ], [ [[VAL1]], [[IF_ELSE]] ]
+; CHECK-NEXT:    store ptr addrspace(1) [[VAL2_SINK]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store ptr addrspace(1) [[VAL1_SINK]], ptr [[TMP0]], align 2
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store ptr addrspace(1) %val1, ptr %this.64.val, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store ptr addrspace(1) %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store ptr addrspace(1) %val1, ptr %add.ptr.else, align 2
+  store ptr addrspace(1) %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}
+
+define internal void @sink_addrspace3(ptr %this.64.val, half %val1, i16 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define internal void @sink_addrspace3
+; CHECK-SAME: (ptr [[THIS_64_VAL:%.*]], half [[VAL1:%.*]], i16 [[VAL2:%.*]], i32 [[VAL3:%.*]]) align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[THIS_AS_CASTED:%.*]] = addrspacecast ptr [[THIS_64_VAL]] to ptr addrspace(1)
+; CHECK-NEXT:    [[CMP_NOT_NOT:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    br i1 [[CMP_NOT_NOT]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store half [[VAL1]], ptr addrspace(1) [[THIS_AS_CASTED]], align 2
+; CHECK-NEXT:    [[TMP0:%.*]] = bitcast i16 [[VAL2]] to half
+; CHECK-NEXT:    br label [[IF_END:%.*]]
+; CHECK:       if.else:
+; CHECK-NEXT:    store i16 [[VAL2]], ptr [[THIS_64_VAL]], align 2
+; CHECK-NEXT:    br label [[IF_END]]
+; CHECK:       if.end:
+; CHECK-NEXT:    [[VAL1_SINK:%.*]] = phi half [ [[TMP0]], [[IF_THEN]] ], [ [[VAL1]], [[IF_ELSE]] ]
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[THIS_64_VAL]], i64 16
+; CHECK-NEXT:    store half [[VAL1_SINK]], ptr [[TMP1]], align 2
+; CHECK-NEXT:    ret void
+;
+entry:
+  %this.as.casted = addrspacecast ptr %this.64.val to ptr addrspace(1)
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  store half %val1, ptr addrspace(1) %this.as.casted, align 2
+  %add.ptr.then = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store i16 %val2, ptr %add.ptr.then, align 2
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %add.ptr.else = getelementptr inbounds i8, ptr %this.64.val, i64 16
+  store half %val1, ptr %add.ptr.else, align 2
+  store i16 %val2, ptr %this.64.val, align 2
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}

llvm/test/Transforms/PhaseOrdering/bitcast-store-branch.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 2
+; RUN: opt -O3 -S < %s | FileCheck %s
+
+%struct.ss = type { ptr, ptr, ptr, ptr, ptr, ptr, ptr, ptr, ptr }
+
+define internal void @phantomLoad(ptr %p, ptr %y, ptr %x) {
+entry:
+  %0 = load i32, ptr %x
+  store i32 %0, ptr %y
+  ret void
+}
+
+define ptr @parent(ptr align 8 dereferenceable(72) %f, half %val1, i16 %val2, i32 %val3) align 2 {
+; CHECK-LABEL: define nonnull ptr @parent
+; CHECK-SAME: (ptr readonly returned align 8 dereferenceable(72) [[F:%.*]], half [[VAL1:%.*]], i16 [[VAL2:%.*]], i32 [[VAL3:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] align 2 {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr i8, ptr [[F]], i64 64
+; CHECK-NEXT:    [[F_VAL:%.*]] = load ptr, ptr [[TMP0]], align 8
+; CHECK-NEXT:    [[CMP_NOT_NOT_I:%.*]] = icmp eq i32 [[VAL3]], 0
+; CHECK-NEXT:    [[TMP1:%.*]] = bitcast half [[VAL1]] to i16
+; CHECK-NEXT:    [[TMP2:%.*]] = bitcast i16 [[VAL2]] to half
+; CHECK-NEXT:    [[VAL2_SINK_I:%.*]] = select i1 [[CMP_NOT_NOT_I]], i16 [[TMP1]], i16 [[VAL2]]
+; CHECK-NEXT:    [[VAL1_SINK_I:%.*]] = select i1 [[CMP_NOT_NOT_I]], half [[TMP2]], half [[VAL1]]
+; CHECK-NEXT:    store i16 [[VAL2_SINK_I]], ptr [[F_VAL]], align 2
+; CHECK-NEXT:    [[TMP3:%.*]] = getelementptr inbounds i8, ptr [[F_VAL]], i64 16
+; CHECK-NEXT:    store half [[VAL1_SINK_I]], ptr [[TMP3]], align 2
+; CHECK-NEXT:    ret ptr [[F]]
+;
+entry:
+  call void @badChild(ptr align 8 dereferenceable(72) %f, half %val1, i16 %val2, i32 %val3) #4
+  ret ptr %f
+}
+
+
+define internal void @badChild(ptr align 8 dereferenceable(72) %this, half %val1, i16 %val2, i32 %val3) align 2 {
+entry:
+  %othergep = getelementptr inbounds %struct.ss, ptr %this, i64 0, i32 2
+  %load0 = load ptr, ptr %othergep, align 8
+  %x = alloca i32
+  %y = alloca i32
+  call void @phantomLoad(ptr %load0, ptr %x, ptr %y)
+  %val1.cast = bitcast half %val1 to i16
+  %cmp.not.not = icmp eq i32 %val3, 0
+  br i1 %cmp.not.not, label %if.then, label %if.else
+if.then:                                          ; preds = %entry
+  %0 = getelementptr inbounds %struct.ss, ptr %this, i64 0, i32 8
+  %1 = load ptr, ptr %0, align 8
+  store i16 %val1.cast, ptr %1, align 2
+  %add.ptr.i.i.i.i = getelementptr inbounds i8, ptr %1, i64 16
+  store i16 %val2, ptr %add.ptr.i.i.i.i, align 2
+  br label %if.end
+if.else:                                          ; preds = %entry
+  %2 = getelementptr inbounds %struct.ss, ptr %this, i64 0, i32 8
+  %3 = load ptr, ptr %2, align 8
+  %add.ptr.i.i.i.i7 = getelementptr inbounds i8, ptr %3, i64 16
+  store i16 %val1.cast, ptr %add.ptr.i.i.i.i7, align 2
+  store i16 %val2, ptr %3, align 2
+  br label %if.end
+if.end:                                           ; preds = %if.else, %if.then
+  ret void
+}