[MSP430] PR27500 CodeGen: Rework branch-select pass

[pftbest]

Summary

Branch instructions created during this pass now have a machine basic block as a target instead of a fixed offset. If there is no layout successor, the current MBB is split to create one. This fixes 27500.
Branch distance should be checked in words instead of a bytes, according to documentation. This prevents branch expansion from triggering too early.
Vector of block offsets is now used to calculate distances between blocks, this is more efficient when branch expansion is not needed, which is a common case.

[pftbest]

ping.
please review this change.

[llvm-commits]

ping

---

llvm-commits mailing list
[e-mail]
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits

[pftbest]

Hi! Any luck getting this diff reviewed? Is there some way I can help?

[asl]

lib/Target/MSP430/MSP430BranchSelector.cpp

31 ↗

(On Diff #56882)

Why these are ints? And not inside namespace { ?

49 ↗

(On Diff #56882)

Why this should be a member variable? It's effectively only used inside "expandBranches". Make it local there and you won't need all that error-prone BlockOffsets.clear() dance.

79 ↗

(On Diff #56882)

DisplacementBits is only used here. Why should it be static const variable after all?

102 ↗

(On Diff #56882)

This is pretty similar to measureFunction(). Please merge into a single one to reduce code duplication.

[pftbest]

lib/Target/MSP430/MSP430BranchSelector.cpp

31 ↗

(On Diff #56882)

WordSize is an int because it is involved in a calculations with negative numbers and that prevents nasty signed/unsigned promotion bugs. There is no reason for DisplacementBits to be signed. Both of them are used only inside isInRange function, so I can either move them there or move them inside a namespace and keep as global constants.

49 ↗

(On Diff #56882)

I can make it local inside runOnMachineFunction and then pass it as a reference to measureFunction and expandBranches

79 ↗

(On Diff #56882)

It was made a constant for a testing reasons, it's very hard to find a code that is so large to be split, so lowering this value was an easy way to test the code. If it is not OK, i will remove it.

102 ↗

(On Diff #56882)

Will do.

[asl]

lib/Target/MSP430/MSP430BranchSelector.cpp

31 ↗

(On Diff #56882)

please move inside isInRange.

49 ↗

(On Diff #56882)

This might also work, yes.

[pftbest]

Constants are moved inside isInRange function, BlockOffsets is now local and is passed as a reference to other functions, correctBlockOffsets function is merged with measureFunction.

[pftbest]

ping

[pftbest]

My thoughts on block renumbering:

1. It happens only once per function in common case
2. In a rare case when we need some branches to be expanded, block offsets are updated manually in the loop, so blocks are not renumbered
3. In extremely rare case when both successors are out of range, only then blocks are renumbered and remeasured.

So the point is:

1. This code is not slower than what is now in the trunk
2. But it is definitely better, because it does work and trunk is generating garbage that crashes mcu.

[pftbest]

ping

[asl]

LGTM. Thanks for doing this!

[asl]

Working on updating the patch due to some API changes.

[pftbest]

Updated diff due to API changes.

[pftbest]

Please review this diff.

llvm/trunk/lib/Target/MSP430/MSP430BranchSelector.cpp

 #include "llvm/ADT/Statistic.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Target/TargetMachine.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "msp430-branch-select"
+#define MSP430_BR_SELECT_NAME "MSP430 Branch Selector"
 
+static cl::opt<bool>
+    BranchSelectEnabled("msp430-branch-select", cl::Hidden, cl::init(true),
+                        cl::desc("Expand out of range branches"));
+
+STATISTIC(NumSplit, "Number of machine basic blocks split");
 STATISTIC(NumExpanded, "Number of branches expanded to long format");
 
 namespace {
-  struct MSP430BSel : public MachineFunctionPass {
-    static char ID;
-    MSP430BSel() : MachineFunctionPass(ID) {}
+class MSP430BSel : public MachineFunctionPass {
+
+  typedef SmallVector<int, 16> OffsetVector;
+
+  MachineFunction *MF;
+  const MSP430InstrInfo *TII;
+
+  unsigned measureFunction(OffsetVector &BlockOffsets,
+                           MachineBasicBlock *FromBB = nullptr);
+  bool expandBranches(OffsetVector &BlockOffsets);
+
+public:
 
-    /// BlockSizes - The sizes of the basic blocks in the function.
-    std::vector<unsigned> BlockSizes;
-
-    bool runOnMachineFunction(MachineFunction &Fn) override;
+  bool runOnMachineFunction(MachineFunction &MF) override;
 
-    MachineFunctionProperties getRequiredProperties() const override {
-      return MachineFunctionProperties().set(
-          MachineFunctionProperties::Property::AllVRegsAllocated);
-    }
 
-    const char *getPassName() const override {
-      return "MSP430 Branch Selector";
-    }
-  };
-  char MSP430BSel::ID = 0;
+  const char *getPassName() const override { return MSP430_BR_SELECT_NAME; }
 }
 
-/// createMSP430BranchSelectionPass - returns an instance of the Branch
-/// Selection Pass
-///
-FunctionPass *llvm::createMSP430BranchSelectionPass() {
-  return new MSP430BSel();
-}
-
-bool MSP430BSel::runOnMachineFunction(MachineFunction &Fn) {
-  const MSP430InstrInfo *TII =
-      static_cast<const MSP430InstrInfo *>(Fn.getSubtarget().getInstrInfo());
+static bool isInRage(int DistanceInBytes) {
+  // According to CC430 Family User's Guide, Section 4.5.1.3, branch
+  // instructions have the signed 10-bit word offset field, so first we need to
+  // convert the distance from bytes to words, then check if it fits in 10-bit
+  // signed integer.
+  const int WordSize = 2;
+
+  assert((DistanceInBytes % WordSize == 0) &&
+         "Branch offset should be word aligned!");
+
+  int Words = DistanceInBytes / WordSize;
+  return isInt<10>(Words);
+}
+
+/// Measure each basic block, fill the BlockOffsets, and return the size of
+/// the function, starting with BB
+unsigned MSP430BSel::measureFunction(OffsetVector &BlockOffsets,
+                                     MachineBasicBlock *FromBB) {
   // Give the blocks of the function a dense, in-order, numbering.
-  Fn.RenumberBlocks();
-  BlockSizes.resize(Fn.getNumBlockIDs());
+  MF->RenumberBlocks(FromBB);
 
-  // Measure each MBB and compute a size for the entire function.
-  unsigned FuncSize = 0;
-  for (MachineBasicBlock &MBB : Fn) {
-    unsigned BlockSize = 0;
-    for (MachineInstr &MI : MBB)
-      BlockSize += TII->getInstSizeInBytes(MI);
-
-    BlockSizes[MBB.getNumber()] = BlockSize;
-    FuncSize += BlockSize;
+  MachineFunction::iterator Begin;
+  if (FromBB == nullptr) {
+    Begin = MF->begin();
+  } else {
+    Begin = FromBB->getIterator();
   }
 
-  // If the entire function is smaller than the displacement of a branch field,
-  // we know we don't need to shrink any branches in this function.  This is a
-  // common case.
-  if (FuncSize < (1 << 9)) {
-    BlockSizes.clear();
-    return false;
-  }
+  BlockOffsets.resize(MF->getNumBlockIDs());
+
+  unsigned TotalSize = BlockOffsets[Begin->getNumber()];
+  for (auto &MBB : make_range(Begin, MF->end())) {
+    BlockOffsets[MBB.getNumber()] = TotalSize;
+    for (MachineInstr &MI : MBB) {
+      TotalSize += TII->getInstSizeInBytes(MI);
+    }
+  }
+  return TotalSize;
 
+/// Do expand branches and split the basic blocks if necessary.
+/// Returns true if made any change.
+bool MSP430BSel::expandBranches(OffsetVector &BlockOffsets) {
   // For each conditional branch, if the offset to its destination is larger
   // than the offset field allows, transform it into a long branch sequence
   // like this:
   //   short branch:
   //     bCC MBB
   //   long branch:
   //     b!CC $PC+6
   //     b MBB
   //
-  bool MadeChange = true;
-  bool EverMadeChange = false;
-  while (MadeChange) {
-    // Iteratively expand branches until we reach a fixed point.
-    MadeChange = false;
-
-    for (MachineFunction::iterator MFI = Fn.begin(), E = Fn.end(); MFI != E;
-         ++MFI) {
-      MachineBasicBlock &MBB = *MFI;
-      unsigned MBBStartOffset = 0;
-      for (MachineBasicBlock::iterator I = MBB.begin(), E = MBB.end();
-           I != E; ++I) {
-        if ((I->getOpcode() != MSP430::JCC || I->getOperand(0).isImm()) &&
-            I->getOpcode() != MSP430::JMP) {
-          MBBStartOffset += TII->getInstSizeInBytes(*I);
-          continue;
-        }
+  bool MadeChange = false;
+  for (auto MBB = MF->begin(), E = MF->end(); MBB != E; ++MBB) {
+    for (auto MI = MBB->begin(), EE = MBB->end(); MI != EE; ++MI) {
+      MBBStartOffset += TII->getInstSizeInBytes(*MI);
+
+      // If this instruction is not a short branch then skip it.
+      if (MI->getOpcode() != MSP430::JCC && MI->getOpcode() != MSP430::JMP) {
 
-        // Determine the offset from the current branch to the destination
-        // block.
-        MachineBasicBlock *Dest = I->getOperand(0).getMBB();
-
-        int BranchSize;
-        if (Dest->getNumber() <= MBB.getNumber()) {
-          // If this is a backwards branch, the delta is the offset from the
-          // start of this block to this branch, plus the sizes of all blocks
-          // from this block to the dest.
-          BranchSize = MBBStartOffset;
+      MachineBasicBlock *DestBB = MI->getOperand(0).getMBB();
+      // Determine the distance from the current branch to the destination
+      // block. MBBStartOffset already includes the size of the current branch
+      // instruction.
+      int BlockDistance =
+          BlockOffsets[DestBB->getNumber()] - BlockOffsets[MBB->getNumber()];
+      int BranchDistance = BlockDistance - MBBStartOffset;
 
-          for (unsigned i = Dest->getNumber(), e = MBB.getNumber(); i != e; ++i)
-            BranchSize += BlockSizes[i];
-        } else {
-          // Otherwise, add the size of the blocks between this block and the
-          // dest to the number of bytes left in this block.
-          BranchSize = -MBBStartOffset;
-
-          for (unsigned i = MBB.getNumber(), e = Dest->getNumber(); i != e; ++i)
-            BranchSize += BlockSizes[i];
-        }
+      // If this branch is in range, ignore it.
+      if (isInRage(BranchDistance)) {
+        continue;
 
-        // If this branch is in range, ignore it.
-        if (isInt<10>(BranchSize)) {
-          MBBStartOffset += 2;
+      DEBUG(dbgs() << "  Found a branch that needs expanding, BB#"
+                   << DestBB->getNumber() << ", Distance " << BranchDistance
+                   << "\n");
+
+      // If JCC is not the last instruction we need to split the MBB.
+      if (MI->getOpcode() == MSP430::JCC && std::next(MI) != EE) {
+
+        DEBUG(dbgs() << "  Found a basic block that needs to be split, BB#"
+                     << MBB->getNumber() << "\n");
+
+        // Create a new basic block.
+        MachineBasicBlock *NewBB =
+            MF->CreateMachineBasicBlock(MBB->getBasicBlock());
+        MF->insert(std::next(MBB), NewBB);
+
+        // Splice the instructions following MI over to the NewBB.
+        NewBB->splice(NewBB->end(), &*MBB, std::next(MI), MBB->end());
+
+        // Update the successor lists.
+        for (MachineBasicBlock *Succ : MBB->successors()) {
+          if (Succ == DestBB) {
             continue;
           }
+          MBB->replaceSuccessor(Succ, NewBB);
+          NewBB->addSuccessor(Succ);
+        }
 
-        // Otherwise, we have to expand it to a long branch.
-        unsigned NewSize;
-        MachineInstr &OldBranch = *I;
-        DebugLoc dl = OldBranch.getDebugLoc();
+        // We introduced a new MBB so all following blocks should be numbered
+        // and measured again.
+        measureFunction(BlockOffsets, &*MBB);
+
+        ++NumSplit;
+
+        // It may be not necessary to start all over at this point, but it's
+        // safer do this anyway.
+        return true;
+      }
+
+      MachineInstr &OldBranch = *MI;
+      int InstrSizeDiff = -TII->getInstSizeInBytes(OldBranch);
+
+      if (MI->getOpcode() == MSP430::JCC) {
+        MachineBasicBlock *NextMBB = &*std::next(MBB);
+        assert(MBB->isSuccessor(NextMBB) &&
+               "This block must have a layout successor!");
 
-        if (I->getOpcode() == MSP430::JMP) {
-          NewSize = 4;
-        } else {
-          // The BCC operands are:
-          // 0. MSP430 branch predicate
-          // 1. Target MBB
-          SmallVector<MachineOperand, 1> Cond;
-          Cond.push_back(I->getOperand(1));
+        // 0. Target MBB
+        // 1. MSP430 branch predicate
+        Cond.push_back(MI->getOperand(1));
 
-          // Jump over the uncond branch inst (i.e. $+6) on opposite condition.
-          TII->ReverseBranchCondition(Cond);
-          BuildMI(MBB, I, dl, TII->get(MSP430::JCC))
-            .addImm(4).addOperand(Cond[0]);
-
-          NewSize = 6;
-        }
-        // Uncond branch to the real destination.
-        I = BuildMI(MBB, I, dl, TII->get(MSP430::Bi)).addMBB(Dest);
+        // Jump over the long branch on the opposite condition
+        auto &NewInstr = BuildMI(*MBB, MI, dl, TII->get(MSP430::JCC))
+                             .addMBB(NextMBB)
+                             .addOperand(Cond[0]);
+        InstrSizeDiff += TII->getInstSizeInBytes(*NewInstr);
+
+      // Unconditional branch to the real destination.
+      MI = BuildMI(*MBB, MI, dl, TII->get(MSP430::Bi)).addMBB(DestBB);
+      InstrSizeDiff += TII->getInstSizeInBytes(*MI);
 
-        // Remove the old branch from the function.
-        OldBranch.eraseFromParent();
 
-        // Remember that this instruction is NewSize bytes, increase the size of the
-        // block by NewSize-2, remember to iterate.
-        BlockSizes[MBB.getNumber()] += NewSize-2;
-        MBBStartOffset += NewSize;
+      // The size of a new instruction is different from the old one, so we need
+      // to correct all block offsets.
+      for (int i = MBB->getNumber() + 1, e = BlockOffsets.size(); i < e; ++i) {
+        BlockOffsets[i] += InstrSizeDiff;
+      }
+      MBBStartOffset += InstrSizeDiff;
 
-        ++NumExpanded;
-        MadeChange = true;
-      }
-    }
-    EverMadeChange |= MadeChange;
-  }
+  return MadeChange;
 
-  BlockSizes.clear();
-  return true;
+bool MSP430BSel::runOnMachineFunction(MachineFunction &mf) {
+  MF = &mf;
+  TII = static_cast<const MSP430InstrInfo *>(MF->getSubtarget().getInstrInfo());
+
+  // If the pass is disabled, just bail early.
+  if (!BranchSelectEnabled)
+    return false;
+
+  DEBUG(dbgs() << "\n********** " MSP430_BR_SELECT_NAME " **********\n");
+
+  // BlockOffsets - Contains the distance from the beginning of the function to
+  // the beginning of each basic block.
+  OffsetVector BlockOffsets;
+
+  unsigned FunctionSize = measureFunction(BlockOffsets);
+  // If the entire function is smaller than the displacement of a branch field,
+  // we know we don't need to expand any branches in this
+  // function. This is a common case.
+  if (isInRage(FunctionSize)) {
+    return false;
+  }
+
+  // Iteratively expand branches until we reach a fixed point.
+  bool MadeChange = false;
+  while (expandBranches(BlockOffsets))
+    MadeChange = true;
+
+  return MadeChange;
+}
+
+/// Returns an instance of the Branch Selection Pass
+FunctionPass *llvm::createMSP430BranchSelectionPass() {
+  return new MSP430BSel();
 }

llvm/trunk/test/CodeGen/MSP430/BranchSelector.ll

+; RUN: llc < %s -march=msp430 | FileCheck %s
+target datalayout = "e-m:e-p:16:16-i32:16:32-a:16-n8:16"
+target triple = "msp430"
+
+@reg = common global i16 0, align 2
+
+define void @WriteBurstPATable(i16 %count) #0 {
+entry:
+  br label %while.cond
+
+while.cond:
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  %v0 = load volatile i16, i16* @reg, align 2
+  %lnot = icmp eq i16 %v0, 0
+
+; This BB should be split and all branches should be expanded.
+; CHECK-LABEL: .LBB0_1:
+; CHECK: jne  .LBB0_2
+; CHECK: br #.LBB0_1
+; CHECK: .LBB0_2:
+; CHECK: br #.LBB0_4
+; CHECK: .LBB0_3:
+
+  br i1 %lnot, label %while.cond, label %while.end
+
+while.end:
+  %i.0.i.0.1822 = load volatile i16, i16* @reg, align 1
+  %cmp23 = icmp ult i16 %i.0.i.0.1822, %count
+  br i1 %cmp23, label %for.body, label %for.end
+
+for.body:
+  br label %while.cond6
+
+while.cond6:
+  %0 = load volatile i16, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 19, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  br label %for.inc
+
+for.inc:
+  %1 = load volatile i16, i16* @reg, align 2
+  %cmp = icmp ult i16 %1, %count
+
+; This branch should be expanded.
+; CHECK-LABEL: .LBB0_4:
+; CHECK: jhs  .LBB0_5
+; CHECK: br #.LBB0_3
+; CHECK: .LBB0_5:
+
+  br i1 %cmp, label %for.body, label %for.end
+
+for.end:
+  ret void
+}
+
+define void @WriteSinglePATable() #0 {
+entry:
+  br label %begin
+begin:
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  store volatile i16 13, i16* @reg, align 2
+  store volatile i16 17, i16* @reg, align 2
+  store volatile i16 11, i16* @reg, align 2
+  %v2 = load volatile i16, i16* @reg, align 2
+  %lnot = icmp eq i16 %v2, 0
+
+; This branch should not be expanded
+; CHECK-LABEL: .LBB1_1:
+; CHECK: jeq  .LBB1_1
+; CHECK: BB#2:
+; CHECK: ret
+  br i1 %lnot, label %begin, label %end
+
+end:
+  ret void
+}