[BOLT][NFC] Add hash computation for basic blocks

[spupyrev]

Summary

Extending yaml profile format with block hashes, which are used for stale
profile matching. To avoid duplication of the code, created a new class with a
collection of utilities for computing hashes.

[spupyrev]

fixed assertion

[spupyrev]

Added a test and adjusted logging

[spupyrev]

fix

[spupyrev]

Rebasing...

This diff doesn't change existing behavior and used only to facilitate diffs down the stack. Hence, not adding tests here but please let me know if you prefer some.

[spupyrev]

nit

[Amir]

Looks good in general with a couple of nits. Testing with a full battery of tests.

bolt/lib/Core/BinaryFunction.cpp

3607–3619

bolt/lib/Profile/YAMLProfileWriter.cpp

123–128

127–129

[Amir]

Please retitle as "[BOLT][NFC] Add hash computation for basic blocks"

[Amir]

bolt/lib/Core/HashUtilities.cpp

128–130

Please use MIB->getShortBranchOpcode interface here

135–139

As a note: I want to switch to using instruction mnemonic instead of opcode value. It shouldn'

[Amir]

bolt/lib/Core/HashUtilities.cpp

135–139

It shouldn't affect the matching, but will make hash stable wrt opcode value changes.

[Amir]

bolt/include/bolt/Profile/ProfileYAMLMapping.h

129

Does it have to be a required field? This change breaks internal tests utilizing yaml profile which obviously don't have this field. I tried using an Optional instead and everything worked.

[maksfb]

LLVM has hash helpers under llvm/include/llvm/ADT/Hashing.h. Can we use them?

[spupyrev]

Addressing comments:

• block's hash is optional in yaml
• using MIB->getShortBranchOpcode

I agree that not relying on specific opcodes is a good idea. Maybe that could be a follow-up diff?

Regarding LLVM's Hashing.h, I dropped one method from the diff, as it is already implemented. Anything else you have in mind regarding the helpers?

[Amir]

The current state looks good to me. I think @maksfb is suggesting the use of ADT/Hashing helpers instead of rolling up our own hashing primitives but that could be in a follow-up diff IMHO.

I agree that not relying on specific opcodes is a good idea. Maybe that could be a follow-up diff?

I'll work on that as a follow-up.

[maksfb]

Looks good to me. Please address format nits by Amir. Thanks!

[spupyrev]

nit

bolt/include/bolt/Core/BinaryBasicBlock.h

private:
 
   /// Indicates if the block could be outlined.
   bool CanOutline{true};
 
   /// Flag to indicate whether this block is valid or not.  Invalid
   /// blocks may contain out of date or incorrect information.
   bool IsValid{true};
 
+  /// Last computed hash value.
+  mutable uint64_t Hash{0};
+
 private:
   BinaryBasicBlock() = delete;
   BinaryBasicBlock(const BinaryBasicBlock &) = delete;
   BinaryBasicBlock(const BinaryBasicBlock &&) = delete;
   BinaryBasicBlock &operator=(const BinaryBasicBlock &) = delete;
   BinaryBasicBlock &operator=(const BinaryBasicBlock &&) = delete;
 
   explicit BinaryBasicBlock(BinaryFunction *Function, MCSymbol *Label)
public:
 
   /// Return jump table if the block contains a jump table instruction or
   /// nullptr otherwise.
   const JumpTable *getJumpTable() const;
 
   /// Check if the block has a jump table instruction.
   bool hasJumpTable() const { return getJumpTable() != nullptr; }
 
+  /// Returns the last computed hash value of the block.
+  uint64_t getHash() const { return Hash; }
+
 private:
   void adjustNumPseudos(const MCInst &Inst, int Sign);
 
   template <typename Itr> void adjustNumPseudos(Itr Begin, Itr End, int Sign) {
     while (Begin != End)
       adjustNumPseudos(*Begin++, Sign);
   }
 
   /// Adds predecessor to the BB. Most likely you don't need to call this.
   void addPredecessor(BinaryBasicBlock *Pred);
 
   /// Remove predecessor of the basic block. Don't use directly, instead
   /// use removeSuccessor() function.
   /// If \p Multiple is set to true, it will remove all predecessors that
   /// are equal to \p Pred. Otherwise, the first instance of \p Pred found
   /// will be removed. This only matters in awkward, redundant CFGs.
   void removePredecessor(BinaryBasicBlock *Pred, bool Multiple = true);
 
   /// Set end offset of this basic block.
   void setEndOffset(uint32_t Offset) { InputRange.second = Offset; }
 
   /// Set the index of this basic block.
   void setIndex(unsigned I) { Index = I; }
 
+  /// Sets the hash value of the basic block.
+  void setHash(uint64_t Value) const { Hash = Value; }
+
   template <typename T> void clearList(T &List) {
     T TempList;
     TempList.swap(List);
   }
 
   /// Release memory taken by CFG edges and instructions.
   void releaseCFG() {
     clearList(Predecessors);

bolt/include/bolt/Core/BinaryFunction.h

   /// The caller can change this via passing \p OperandHashFunc function.
   /// The return result of this function will be mixed with internal hash.
   size_t computeHash(
       bool UseDFS = false,
       OperandHashFuncTy OperandHashFunc = [](const MCOperand &) {
         return std::string();
       }) const;
 
+  /// Compute hash values for each block of the function.
+  void computeBlockHashes() const;
+
   void setDWARFUnit(DWARFUnit *Unit) { DwarfUnit = Unit; }
 
   /// Return DWARF compile unit for this function.
   DWARFUnit *getDWARFUnit() const { return DwarfUnit; }
 
   /// Return line info table for this function.
   const DWARFDebugLine::LineTable *getDWARFLineTable() const {
     return getDWARFUnit() ? BC.DwCtx->getLineTableForUnit(getDWARFUnit())

bolt/include/bolt/Core/HashUtilities.h

+//===- bolt/Core/HashUtilities.h - Misc hash utilities --------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains functions for computing hash values over BinaryFunction
+// and BinaryBasicBlock.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef BOLT_CORE_HASH_UTILITIES_H
+#define BOLT_CORE_HASH_UTILITIES_H
+
+#include "bolt/Core/BinaryBasicBlock.h"
+#include "bolt/Core/BinaryContext.h"
+
+namespace llvm {
+namespace bolt {
+
+uint16_t hash_64_to_16(const uint64_t Hash);
+
+std::string hashInteger(uint64_t Value);
+
+std::string hashSymbol(BinaryContext &BC, const MCSymbol &Symbol);
+
+std::string hashExpr(BinaryContext &BC, const MCExpr &Expr);
+
+std::string hashInstOperand(BinaryContext &BC, const MCOperand &Operand);
+
+using OperandHashFuncTy = function_ref<typename std::string(const MCOperand &)>;
+
+std::string hashBlock(BinaryContext &BC, const BinaryBasicBlock &BB,
+                      OperandHashFuncTy OperandHashFunc);
+
+} // namespace bolt
+} // namespace llvm
+
+#endif

bolt/include/bolt/Core/MCPlusBuilder.h

public:
   virtual bool createDirectCall(MCInst &Inst, const MCSymbol *Target,
                                 MCContext *Ctx, bool IsTailCall) {
     llvm_unreachable("not implemented");
     return false;
   }
 
   virtual MCPhysReg getX86R11() const { llvm_unreachable("not implemented"); }
 
+  virtual unsigned getShortBranchOpcode(unsigned Opcode) const {
+    llvm_unreachable("not implemented");
+    return 0;
+  }
+
   /// Create increment contents of target by 1 for Instrumentation
   virtual InstructionListType createInstrIncMemory(const MCSymbol *Target,
                                                    MCContext *Ctx,
                                                    bool IsLeaf) const {
     llvm_unreachable("not implemented");
     return InstructionListType();
   }
 

bolt/include/bolt/Profile/ProfileYAMLMapping.h

struct BinaryBasicBlockProfile {
   }
 };
 } // end namespace bolt
 
 template <> struct MappingTraits<bolt::BinaryBasicBlockProfile> {
   static void mapping(IO &YamlIO, bolt::BinaryBasicBlockProfile &BBP) {
     YamlIO.mapRequired("bid", BBP.Index);
     YamlIO.mapRequired("insns", BBP.NumInstructions);
+    YamlIO.mapOptional("hash", BBP.Hash, (llvm::yaml::Hex64)0);
     YamlIO.mapOptional("exec", BBP.ExecCount, (uint64_t)0);
     YamlIO.mapOptional("events", BBP.EventCount, (uint64_t)0);
     YamlIO.mapOptional("calls", BBP.CallSites,
                        std::vector<bolt::CallSiteInfo>());
     YamlIO.mapOptional("succ", BBP.Successors,
                        std::vector<bolt::SuccessorInfo>());
   }
 };

bolt/lib/Core/BinaryFunction.cpp

 //===- bolt/Core/BinaryFunction.cpp - Low-level function ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the BinaryFunction class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "bolt/Core/BinaryFunction.h"
 #include "bolt/Core/BinaryBasicBlock.h"
 #include "bolt/Core/BinaryDomTree.h"
 #include "bolt/Core/DynoStats.h"
+#include "bolt/Core/HashUtilities.h"
 #include "bolt/Core/MCPlusBuilder.h"
 #include "bolt/Utils/NameResolver.h"
 #include "bolt/Utils/NameShortener.h"
 #include "bolt/Utils/Utils.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
size_t BinaryFunction::computeHash(bool UseDFS,
   SmallVector<const BinaryBasicBlock *, 0> Order;
   if (UseDFS)
     llvm::copy(dfs(), std::back_inserter(Order));
   else
     llvm::copy(Layout.blocks(), std::back_inserter(Order));
 
   // The hash is computed by creating a string of all instruction opcodes and
   // possibly their operands and then hashing that string with std::hash.
   std::string HashString;
-  for (const BinaryBasicBlock *BB : Order) {
-    for (const MCInst &Inst : *BB) {
-      unsigned Opcode = Inst.getOpcode();
-
-      if (BC.MIB->isPseudo(Inst))
-        continue;
-
-      // Ignore unconditional jumps since we check CFG consistency by processing
-      // basic blocks in order and do not rely on branches to be in-sync with
-      // CFG. Note that we still use condition code of conditional jumps.
-      if (BC.MIB->isUnconditionalBranch(Inst))
-        continue;
-
-      if (Opcode == 0)
-        HashString.push_back(0);
+  for (const BinaryBasicBlock *BB : Order)
+    HashString.append(hashBlock(BC, *BB, OperandHashFunc));
 
-      while (Opcode) {
-        uint8_t LSB = Opcode & 0xff;
-        HashString.push_back(LSB);
-        Opcode = Opcode >> 8;
-      }
+  return Hash = std::hash<std::string>{}(HashString);
 
-      for (const MCOperand &Op : MCPlus::primeOperands(Inst))
-        HashString.append(OperandHashFunc(Op));
-    }
+void BinaryFunction::computeBlockHashes() const {
+  for (const BinaryBasicBlock *BB : BasicBlocks) {
+    std::string Hash =
+        hashBlock(BC, *BB, [](const MCOperand &Op) { return std::string(); });
+    BB->setHash(std::hash<std::string>{}(Hash));
   }
-
-  return Hash = std::hash<std::string>{}(HashString);
 }
 
 void BinaryFunction::insertBasicBlocks(
     BinaryBasicBlock *Start,
     std::vector<std::unique_ptr<BinaryBasicBlock>> &&NewBBs,
     const bool UpdateLayout, const bool UpdateCFIState,
     const bool RecomputeLandingPads) {
   const int64_t StartIndex = Start ? getIndex(Start) : -1LL;

bolt/lib/Core/CMakeLists.txt

add_llvm_library(LLVMBOLTCore
   BinaryEmitter.cpp
   BinaryFunction.cpp
   BinaryFunctionProfile.cpp
   BinarySection.cpp
   DebugData.cpp
   DynoStats.cpp
   Exceptions.cpp
   FunctionLayout.cpp
+  HashUtilities.cpp
   JumpTable.cpp
   MCPlusBuilder.cpp
   ParallelUtilities.cpp
   Relocation.cpp
 
   DISABLE_LLVM_LINK_LLVM_DYLIB
   LINK_LIBS
   ${LLVM_PTHREAD_LIB}
   )
 
 target_link_libraries(LLVMBOLTCore
   PRIVATE
   LLVMBOLTUtils
   )

bolt/lib/Core/HashUtilities.cpp

+//===- bolt/Core/HashUtilities.cpp - Misc hash utilities ------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Computation of hash values over BinaryFunction and BinaryBasicBlock.
+//
+//===----------------------------------------------------------------------===//
+
+#include "bolt/Core/HashUtilities.h"
+#include "bolt/Core/BinaryContext.h"
+#include "bolt/Core/BinaryFunction.h"
+
+namespace llvm {
+namespace bolt {
+
+/// Hashing a 64-bit integer to a 16-bit one.
+uint16_t hash_64_to_16(const uint64_t Hash) {
+  uint16_t Res = (uint16_t)(Hash & 0xFFFF);
+  Res ^= (uint16_t)((Hash >> 16) & 0xFFFF);
+  Res ^= (uint16_t)((Hash >> 32) & 0xFFFF);
+  Res ^= (uint16_t)((Hash >> 48) & 0xFFFF);
+  return Res;
+}
+
+std::string hashInteger(uint64_t Value) {
+  std::string HashString;
+  if (Value == 0)
+    HashString.push_back(0);
+
+  while (Value) {
+    uint8_t LSB = Value & 0xff;
+    HashString.push_back(LSB);
+    Value >>= 8;
+  }
+
+  return HashString;
+}
+
+std::string hashSymbol(BinaryContext &BC, const MCSymbol &Symbol) {
+  std::string HashString;
+
+  // Ignore function references.
+  if (BC.getFunctionForSymbol(&Symbol))
+    return HashString;
+
+  llvm::ErrorOr<uint64_t> ErrorOrValue = BC.getSymbolValue(Symbol);
+  if (!ErrorOrValue)
+    return HashString;
+
+  // Ignore jump table references.
+  if (BC.getJumpTableContainingAddress(*ErrorOrValue))
+    return HashString;
+
+  return HashString.append(hashInteger(*ErrorOrValue));
+}
+
+std::string hashExpr(BinaryContext &BC, const MCExpr &Expr) {
+  switch (Expr.getKind()) {
+  case MCExpr::Constant:
+    return hashInteger(cast<MCConstantExpr>(Expr).getValue());
+  case MCExpr::SymbolRef:
+    return hashSymbol(BC, cast<MCSymbolRefExpr>(Expr).getSymbol());
+  case MCExpr::Unary: {
+    const auto &UnaryExpr = cast<MCUnaryExpr>(Expr);
+    return hashInteger(UnaryExpr.getOpcode())
+        .append(hashExpr(BC, *UnaryExpr.getSubExpr()));
+  }
+  case MCExpr::Binary: {
+    const auto &BinaryExpr = cast<MCBinaryExpr>(Expr);
+    return hashExpr(BC, *BinaryExpr.getLHS())
+        .append(hashInteger(BinaryExpr.getOpcode()))
+        .append(hashExpr(BC, *BinaryExpr.getRHS()));
+  }
+  case MCExpr::Target:
+    return std::string();
+  }
+
+  llvm_unreachable("invalid expression kind");
+}
+
+std::string hashInstOperand(BinaryContext &BC, const MCOperand &Operand) {
+  if (Operand.isImm())
+    return hashInteger(Operand.getImm());
+  if (Operand.isReg())
+    return hashInteger(Operand.getReg());
+  if (Operand.isExpr())
+    return hashExpr(BC, *Operand.getExpr());
+
+  return std::string();
+}
+
+std::string hashBlock(BinaryContext &BC, const BinaryBasicBlock &BB,
+                      OperandHashFuncTy OperandHashFunc) {
+  const bool IsX86 = BC.isX86();
+
+  // The hash is computed by creating a string of all instruction opcodes and
+  // possibly their operands and then hashing that string with std::hash.
+  std::string HashString;
+
+  for (const MCInst &Inst : BB) {
+    if (BC.MIB->isPseudo(Inst))
+      continue;
+
+    unsigned Opcode = Inst.getOpcode();
+
+    // Ignore unconditional jumps since we check CFG consistency by processing
+    // basic blocks in order and do not rely on branches to be in-sync with
+    // CFG. Note that we still use condition code of conditional jumps.
+    if (BC.MIB->isUnconditionalBranch(Inst))
+      continue;
+
+    if (IsX86 && BC.MIB->isConditionalBranch(Inst))
+      Opcode = BC.MIB->getShortBranchOpcode(Opcode);
+
+    if (Opcode == 0)
+      HashString.push_back(0);
+
+    while (Opcode) {
+      uint8_t LSB = Opcode & 0xff;
+      HashString.push_back(LSB);
+      Opcode = Opcode >> 8;
+    }
+
+    for (const MCOperand &Op : MCPlus::primeOperands(Inst))
+      HashString.append(OperandHashFunc(Op));
+  }
+  return HashString;
+}
+
+} // namespace bolt
+} // namespace llvm

bolt/lib/Passes/IdenticalCodeFolding.cpp

 //===- bolt/Passes/IdenticalCodeFolding.cpp -------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the IdenticalCodeFolding class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "bolt/Passes/IdenticalCodeFolding.h"
+#include "bolt/Core/HashUtilities.h"
 #include "bolt/Core/ParallelUtilities.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/Timer.h"
 #include <atomic>
 #include <iterator>
 #include <map>

 typedef std::unordered_map<BinaryFunction *, std::set<BinaryFunction *>,
                            KeyHash, KeyCongruent>
     CongruentBucketsMap;
 
 typedef std::unordered_map<BinaryFunction *, std::vector<BinaryFunction *>,
                            KeyHash, KeyEqual>
     IdenticalBucketsMap;
 
-static std::string hashInteger(uint64_t Value) {
-  std::string HashString;
-  if (Value == 0)
-    HashString.push_back(0);
-
-  while (Value) {
-    uint8_t LSB = Value & 0xff;
-    HashString.push_back(LSB);
-    Value >>= 8;
-  }
-
-  return HashString;
-}
-
-static std::string hashSymbol(BinaryContext &BC, const MCSymbol &Symbol) {
-  std::string HashString;
-
-  // Ignore function references.
-  if (BC.getFunctionForSymbol(&Symbol))
-    return HashString;
-
-  llvm::ErrorOr<uint64_t> ErrorOrValue = BC.getSymbolValue(Symbol);
-  if (!ErrorOrValue)
-    return HashString;
-
-  // Ignore jump table references.
-  if (BC.getJumpTableContainingAddress(*ErrorOrValue))
-    return HashString;
-
-  return HashString.append(hashInteger(*ErrorOrValue));
-}
-
-static std::string hashExpr(BinaryContext &BC, const MCExpr &Expr) {
-  switch (Expr.getKind()) {
-  case MCExpr::Constant:
-    return hashInteger(cast<MCConstantExpr>(Expr).getValue());
-  case MCExpr::SymbolRef:
-    return hashSymbol(BC, cast<MCSymbolRefExpr>(Expr).getSymbol());
-  case MCExpr::Unary: {
-    const auto &UnaryExpr = cast<MCUnaryExpr>(Expr);
-    return hashInteger(UnaryExpr.getOpcode())
-        .append(hashExpr(BC, *UnaryExpr.getSubExpr()));
-  }
-  case MCExpr::Binary: {
-    const auto &BinaryExpr = cast<MCBinaryExpr>(Expr);
-    return hashExpr(BC, *BinaryExpr.getLHS())
-        .append(hashInteger(BinaryExpr.getOpcode()))
-        .append(hashExpr(BC, *BinaryExpr.getRHS()));
-  }
-  case MCExpr::Target:
-    return std::string();
-  }
-
-  llvm_unreachable("invalid expression kind");
-}
-
-static std::string hashInstOperand(BinaryContext &BC,
-                                   const MCOperand &Operand) {
-  if (Operand.isImm())
-    return hashInteger(Operand.getImm());
-  if (Operand.isReg())
-    return hashInteger(Operand.getReg());
-  if (Operand.isExpr())
-    return hashExpr(BC, *Operand.getExpr());
-
-  return std::string();
-}
-
 namespace llvm {
 namespace bolt {
 
 void IdenticalCodeFolding::runOnFunctions(BinaryContext &BC) {
   const size_t OriginalFunctionCount = BC.getBinaryFunctions().size();
   uint64_t NumFunctionsFolded = 0;
   std::atomic<uint64_t> NumJTFunctionsFolded{0};
   std::atomic<uint64_t> BytesSavedEstimate{0};

bolt/lib/Profile/YAMLProfileWriter.cpp

 
 namespace {
 void convert(const BinaryFunction &BF,
              yaml::bolt::BinaryFunctionProfile &YamlBF) {
   const BinaryContext &BC = BF.getBinaryContext();
 
   const uint16_t LBRProfile = BF.getProfileFlags() & BinaryFunction::PF_LBR;
 
+  // Prepare function and block hashes
+  BF.computeHash(/*UseDFS=*/true);
+  BF.computeBlockHashes();
+
   YamlBF.Name = BF.getPrintName();
   YamlBF.Id = BF.getFunctionNumber();
-  YamlBF.Hash = BF.computeHash(/*UseDFS=*/true);
+  YamlBF.Hash = BF.getHash();
   YamlBF.NumBasicBlocks = BF.size();
   YamlBF.ExecCount = BF.getKnownExecutionCount();
 
   for (const BinaryBasicBlock *BB : BF.dfs()) {
     yaml::bolt::BinaryBasicBlockProfile YamlBB;
     YamlBB.Index = BB->getLayoutIndex();
     YamlBB.NumInstructions = BB->getNumNonPseudos();
+    YamlBB.Hash = BB->getHash();
 
     if (!LBRProfile) {
       YamlBB.EventCount = BB->getKnownExecutionCount();
       if (YamlBB.EventCount)
         YamlBF.Blocks.emplace_back(YamlBB);
       continue;
     }
 
for (const BinaryBasicBlock *BB : BF.dfs()) {
     }
 
     llvm::sort(YamlBB.CallSites);
 
     // Skip printing if there's no profile data for non-entry basic block.
     // Include landing pads with non-zero execution count.
     if (YamlBB.CallSites.empty() && !BB->isEntryPoint() &&
         !(BB->isLandingPad() && BB->getKnownExecutionCount() != 0)) {
+      // Include blocks having successors or predecessors with positive counts.
       uint64_t SuccessorExecCount = 0;
       for (const BinaryBasicBlock::BinaryBranchInfo &BranchInfo :
            BB->branch_info())
         SuccessorExecCount += BranchInfo.Count;
-      if (!SuccessorExecCount)
+      uint64_t PredecessorExecCount = 0;
+      for (auto Pred : BB->predecessors())
+        PredecessorExecCount += Pred->getBranchInfo(*BB).Count;
+      if (!SuccessorExecCount && !PredecessorExecCount)
         continue;
     }
 
     auto BranchInfo = BB->branch_info_begin();
     for (const BinaryBasicBlock *Successor : BB->successors()) {
       yaml::bolt::SuccessorInfo YamlSI;
       YamlSI.Index = Successor->getLayoutIndex();
       YamlSI.Count = BranchInfo->Count;
       YamlSI.Mispreds = BranchInfo->MispredictedCount;

bolt/lib/Target/X86/X86MCPlusBuilder.cpp

static cl::opt<bool> X86StripRedundantAddressSize(
     "x86-strip-redundant-address-size",
     cl::desc("Remove redundant Address-Size override prefix"), cl::init(true),
     cl::cat(BoltOptCategory));
 
 } // namespace opts
 
 namespace {
 
-unsigned getShortBranchOpcode(unsigned Opcode) {
-  switch (Opcode) {
-  default:
-    return Opcode;
-  case X86::JMP_2: return X86::JMP_1;
-  case X86::JMP_4: return X86::JMP_1;
-  case X86::JCC_2: return X86::JCC_1;
-  case X86::JCC_4: return X86::JCC_1;
-  }
-}
-
 unsigned getShortArithOpcode(unsigned Opcode) {
   return X86::getShortOpcodeArith(Opcode);
 }
 
 bool isMOVSX64rm32(const MCInst &Inst) {
   return Inst.getOpcode() == X86::MOVSX64rm32;
 }
 
assert((isCall(Inst) || isBranch(Inst)) && !isIndirectBranch(Inst) &&
            "Invalid instruction");
     Inst.getOperand(0) = MCOperand::createExpr(
         MCSymbolRefExpr::create(TBB, MCSymbolRefExpr::VK_None, *Ctx));
     return true;
   }
 
   MCPhysReg getX86R11() const override { return X86::R11; }
 
+  unsigned getShortBranchOpcode(unsigned Opcode) const override {
+    switch (Opcode) {
+    default:
+      return Opcode;
+    case X86::JMP_2:
+      return X86::JMP_1;
+    case X86::JMP_4:
+      return X86::JMP_1;
+    case X86::JCC_2:
+      return X86::JCC_1;
+    case X86::JCC_4:
+      return X86::JCC_1;
+    }
+  }
+
   MCPhysReg getIntArgRegister(unsigned ArgNo) const override {
     // FIXME: this should depend on the calling convention.
     switch (ArgNo) {
     case 0:   return X86::RDI;
     case 1:   return X86::RSI;
     case 2:   return X86::RDX;
     case 3:   return X86::RCX;
     case 4:   return X86::R8;