Reduce PGO Instrumentation binary and profile data size (Patch-1)

[davidxl]

Summary

The change includes three patches

1. LLVM changes (Patch-1)
2. Clang Driver changes (Patch-2)
3. compile-rt (patch-3)

Description of the changes:

1. The patch implements a new option -f[no]-profile-instr-with-names option. When -fno-profile-instr-with-names is turned on, the instrumentation will use function assembly name's md5 hash string (hex form) instead of the raw name as the profile identification key. With this change, binary and profile size is greatly reduced
2. refactor of the instrumentation and compile-rt code such that Magic and Version (raw, and indexed) values for profile data is specified in one centralized place: InstrProf.h. To assist this change, compile_rt code no longer needs to hard code the version/magic values and will never be out of sync with instrumentor
3. The 64bit version field in the header is partitioned into two parts a) the lower 32bit is used to represent the format version as before b) the upper 32bit is now used to encode feature/variants of the profile data that share the same format. In this case, it can be used to indicate if the profile contains raw name keys or md5hash keys. Another use of the bit is the late instrumentation profile data.

[silvas]

Fundamentally, this approach is not a long-term solution to reducing PGO data size. The correct long-term solution for reducing PGO data size requires passing the binary to llvm-profdata. The approach of using name hashes is also not an incremental step towards the long-term solution. So thinking about it a bit more, I'm on the fence about whether a quick solution like this one even makes sense to do. I am very apprehensive of adding complexity to make this approach work, in particular threading information through many layers, as this is likely to get in the way of future work towards the long-term solution.

My takeaways from the RFC thread (maybe I am misunderstanding) is that we are still using the same format just with one string vs. another. We shouldn't need to rev the version and there shouldn't need to be any changes in compiler-rt or llvm. Clang can easily diagnose (as an isolated piece of logic) if the user did the pgo training run with md5 mode enabled but the md5 mode flag was not passed or vice versa (it is easy to tell if input function names are md5 sums or not as a diagnostic heuristic). It is also inconsistent for llvm-profdata to be able to be passed -function=foo for an md5 profile; the user should have to pass (truncated) md5 of the function name.

test/Instrumentation/InstrProfiling/profiling_md5hash_key.ll

5–6

This is backward from a standard md5 sum in hex. We should maintain the invariant that the resulting symbol name is a substring of a standard md5 sum. Otherwise it is very difficult to explain how to get the hash for a function.

E.g. we should be able to say "the symbol name is effectively replaced by the first half of its md5 sum in hex":

Sean:~ % echo -n 'foo' | md5             
acbd18db4cc2f85cedef654fccc4a4d8
Sean:~ % echo -n 'foo' | md5 | head -c 16
acbd18db4cc2f85c

The symbol name string should be acbd18db4cc2f85c.

[thanm]

I see a some new warnings when I build the code (after "arc patch D12715"):

[35/42] Building CXX object tools/clan...s/clangCodeGen.dir/CodeGenModule.cpp.o
In file included from /ssd/llvm-trunk-profinst/llvm/tools/clang/lib/CodeGen/CodeGenModule.cpp:53:
In file included from /ssd/llvm-trunk-profinst/llvm/include/llvm/ProfileData/InstrProfReader.h:20:
/ssd/llvm-trunk-profinst/llvm/include/llvm/ProfileData/InstrProf.h:111:10: warning: unused function 'getMagic' [-Wunused-function]
uint64_t getMagic<uint64_t>() {

^

/ssd/llvm-trunk-profinst/llvm/include/llvm/ProfileData/InstrProf.h:124:10: warning: unused function 'getMagic' [-Wunused-function]
uint64_t getMagic<uint32_t>() {

^

include/llvm/ProfileData/InstrProf.h

59

typo fromat -> format

include/llvm/Transforms/Instrumentation.h

87

typo funciton => function

[davidxl]

Than, thanks for the review.

Currently there are two competing approaches to the problem with
different pros and cons. Looks like Sean and I are leaning towards the
more aggressive approach (not the one in this patch). I will write up
a summary at some point comparing the pros and cons of the two
approaches, and get feedback from others.

thanks,

David

include/llvm/ProfileData/InstrProf.h

 // frontend to feed PGO.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_PROFILEDATA_INSTRPROF_H_
 #define LLVM_PROFILEDATA_INSTRPROF_H_
 
 #include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/MD5.h"
+#include "llvm/Support/Endian.h"
+#include "llvm/Support/ErrorHandling.h"
 #include <cstdint>
 #include <system_error>
 #include <vector>
 
 namespace llvm {
 const std::error_category &instrprof_category();
 
 enum class instrprof_error {
enum class instrprof_error {
     count_mismatch,
     counter_overflow
 };
 
 inline std::error_code make_error_code(instrprof_error E) {
   return std::error_code(static_cast<int>(E), instrprof_category());
 }
 
+const uint64_t InstrProfFeature_Mask = 0xffffffffull << 32;
+
+enum class instrprof_feature {
+  // Use function name's md5 hash as index profile's key
+  md5hash_key = 32,
+  last = 63
+};
+
+/// A helper function to extract the fromat version value from the version
+/// field. The feature bits are filtered out.
+inline uint64_t getFormatVersion(uint64_t version) {
+  return (version & ~InstrProfFeature_Mask);
+}
+
+/// A helper function to test feature bit of the feature set
+inline bool hasFeature(instrprof_feature feature, uint64_t feature_set) {
+  return (feature_set & (1ull << (int)feature)) != 0;
+}
+
+inline uint64_t MD5Hash(StringRef Str) {
+  MD5 Hash;
+  Hash.update(Str);
+  llvm::MD5::MD5Result Result;
+  Hash.final(Result);
+  // Return the least significant 8 bytes. Our MD5 implementation returns the
+  // result in little endian, so we may need to swap bytes.
+  using namespace llvm::support;
+  return endian::read<uint64_t, little, unaligned>(Result);
+}
+
+/// Returns MD5 hash in hex string.
+inline StringRef MD5HashStr(StringRef Str) {
+  return utohexstr(MD5Hash(Str), true);
+}
+
 /// Profiling information for a single function.
 struct InstrProfRecord {
   InstrProfRecord() {}
   InstrProfRecord(StringRef Name, uint64_t Hash, std::vector<uint64_t> Counts)
       : Name(Name), Hash(Hash), Counts(std::move(Counts)) {}
   StringRef Name;
   uint64_t Hash;
   std::vector<uint64_t> Counts;
 };
 
+namespace RawInstrProf {
+
+const uint64_t Version  = 1;
+
+
+// Magic number to detect file format and endianness.
+// Use 255 at one end, since no UTF-8 file can use that character.  Avoid 0,
+// so that utilities, like strings, don't grab it as a string.  129 is also
+// invalid UTF-8, and high enough to be interesting.
+// Use "lprofr" in the centre to stand for "LLVM Profile Raw", or "lprofR"
+// for 32-bit platforms.
+
+template <class IntPtrT>
+static uint64_t getMagic();
+template <>
+uint64_t getMagic<uint64_t>() {
+  return
+    uint64_t(255) << 56 |
+    uint64_t('l') << 48 |
+    uint64_t('p') << 40 |
+    uint64_t('r') << 32 |
+    uint64_t('o') << 24 |
+    uint64_t('f') << 16 |
+    uint64_t('r') <<  8 |
+    uint64_t(129);
+}
+
+template <>
+uint64_t getMagic<uint32_t>() {
+  return
+    uint64_t(255) << 56 |
+    uint64_t('l') << 48 |
+    uint64_t('p') << 40 |
+    uint64_t('r') << 32 |
+    uint64_t('o') << 24 |
+    uint64_t('f') << 16 |
+    uint64_t('R') <<  8 |
+    uint64_t(129);
+
+}
+}
+
+namespace IndexedInstrProf {
+enum class HashT : uint32_t {
+  MD5,
+  Last = MD5
+};
+
+static inline uint64_t ComputeHash(HashT Type, StringRef K) {
+  switch (Type) {
+  case HashT::MD5:
+    return MD5Hash(K);
+  }
+  llvm_unreachable("Unhandled hash type");
+}
+
+const uint64_t Magic = 0x8169666f72706cff; // "\xfflprofi\x81"
+const uint64_t Version = 2;
+const HashT HashType = HashT::MD5;
+}
+
 } // end namespace llvm
 
 namespace std {
 template <>
 struct is_error_code_enum<llvm::instrprof_error> : std::true_type {};
 }
 
 #endif // LLVM_PROFILEDATA_INSTRPROF_H_

include/llvm/ProfileData/InstrProfReader.h

 };
 
 /// Base class and interface for reading profiling data of any known instrprof
 /// format. Provides an iterator over InstrProfRecords.
 class InstrProfReader {
   std::error_code LastError;
 
 public:
-  InstrProfReader() : LastError(instrprof_error::success) {}
+  InstrProfReader() : LastError(instrprof_error::success), Version(0) {}
   virtual ~InstrProfReader() {}
 
   /// Read the header.  Required before reading first record.
   virtual std::error_code readHeader() = 0;
   /// Read a single record.
   virtual std::error_code readNextRecord(InstrProfRecord &Record) = 0;
   /// Iterator over profile data.
   InstrProfIterator begin() { return InstrProfIterator(this); }
   InstrProfIterator end() { return InstrProfIterator(); }
 
 protected:
   /// Set the current std::error_code and return same.
   std::error_code error(std::error_code EC) {
     LastError = EC;
     return EC;
   }
 
   /// Clear the current error code and return a successful one.
   std::error_code success() { return error(instrprof_error::success); }
 
+  /// Raw or Indexed Profile Version value.
+  /// The version field contains two parts: Format version at lower 32 bit,
+  /// and feature set at upper 32bit.
+  uint64_t Version;
+
 public:
+  uint64_t getVersion() { return Version; }
   /// Return true if the reader has finished reading the profile data.
   bool isEOF() { return LastError == instrprof_error::eof; }
   /// Return true if the reader encountered an error reading profiling data.
   bool hasError() { return LastError && !isEOF(); }
   /// Get the current error code.
   std::error_code getError() { return LastError; }
 
   /// Factory method to create an appropriately typed reader for the given

 class IndexedInstrProfReader : public InstrProfReader {
 private:
   /// The profile data file contents.
   std::unique_ptr<MemoryBuffer> DataBuffer;
   /// The index into the profile data.
   std::unique_ptr<InstrProfReaderIndex> Index;
   /// Iterator over the profile data.
   InstrProfReaderIndex::data_iterator RecordIterator;
-  /// The file format version of the profile data.
-  uint64_t FormatVersion;
   /// The maximal execution count among all functions.
   uint64_t MaxFunctionCount;
 
   IndexedInstrProfReader(const IndexedInstrProfReader &) = delete;
   IndexedInstrProfReader &operator=(const IndexedInstrProfReader &) = delete;
 public:
   IndexedInstrProfReader(std::unique_ptr<MemoryBuffer> DataBuffer)
       : DataBuffer(std::move(DataBuffer)), Index(nullptr) {}

include/llvm/ProfileData/InstrProfWriter.h

 /// Writer for instrumentation based profile data.
 class InstrProfWriter {
 public:
   typedef SmallDenseMap<uint64_t, std::vector<uint64_t>, 1> CounterData;
 private:
   StringMap<CounterData> FunctionData;
   uint64_t MaxFunctionCount;
 public:
-  InstrProfWriter() : MaxFunctionCount(0) {}
+  InstrProfWriter() : MaxFunctionCount(0), MD5HashKey(false) {}
 
   /// Add function counts for the given function. If there are already counts
   /// for this function and the hash and number of counts match, each counter is
   /// summed.
   std::error_code addFunctionCounts(StringRef FunctionName,
                                     uint64_t FunctionHash,
                                     ArrayRef<uint64_t> Counters);
+  void setUseMD5HashKey() { MD5HashKey = true; }
   /// Write the profile to \c OS
   void write(raw_fd_ostream &OS);
   /// Write the profile, returning the raw data. For testing.
   std::unique_ptr<MemoryBuffer> writeBuffer();
 
 private:
   std::pair<uint64_t, uint64_t> writeImpl(raw_ostream &OS);
+  bool MD5HashKey;
 };
 
 } // end namespace llvm
 
 #endif

include/llvm/Transforms/Instrumentation.h

struct GCOVOptions {
   // all of the function body's blocks.
   bool ExitBlockBeforeBody;
 };
 ModulePass *createGCOVProfilerPass(const GCOVOptions &Options =
                                    GCOVOptions::getDefault());
 
 /// Options for the frontend instrumentation based profiling pass.
 struct InstrProfOptions {
-  InstrProfOptions() : NoRedZone(false) {}
+  InstrProfOptions() : NoRedZone(false), IncludeNamesInProfile(false) {}
 
   // Add the 'noredzone' attribute to added runtime library calls.
   bool NoRedZone;
 
+  // Include full funciton names in profile.
+  bool IncludeNamesInProfile;
+
   // Name of the profile file to use as output
   std::string InstrProfileOutput;
 };
 
 /// Insert frontend instrumentation based profiling.
 ModulePass *createInstrProfilingPass(
     const InstrProfOptions &Options = InstrProfOptions());
 

lib/ProfileData/InstrProfIndexed.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_PROFILEDATA_INSTRPROFINDEXED_H
 #define LLVM_LIB_PROFILEDATA_INSTRPROFINDEXED_H
 
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/MD5.h"
-
-namespace llvm {
-
-namespace IndexedInstrProf {
-enum class HashT : uint32_t {
-  MD5,
-
-  Last = MD5
-};
-
-static inline uint64_t MD5Hash(StringRef Str) {
-  MD5 Hash;
-  Hash.update(Str);
-  llvm::MD5::MD5Result Result;
-  Hash.final(Result);
-  // Return the least significant 8 bytes. Our MD5 implementation returns the
-  // result in little endian, so we may need to swap bytes.
-  using namespace llvm::support;
-  return endian::read<uint64_t, little, unaligned>(Result);
-}
-
-static inline uint64_t ComputeHash(HashT Type, StringRef K) {
-  switch (Type) {
-  case HashT::MD5:
-    return IndexedInstrProf::MD5Hash(K);
-  }
-  llvm_unreachable("Unhandled hash type");
-}
-
-const uint64_t Magic = 0x8169666f72706cff; // "\xfflprofi\x81"
-const uint64_t Version = 2;
-const HashT HashType = HashT::MD5;
-}
-
-} // end namespace llvm
+#include "llvm/ProfileData/InstrProf.h"
 
 #endif

lib/ProfileData/InstrProfReader.cpp

if ((Line++)->getAsInteger(10, Count))
       return error(instrprof_error::malformed);
     Record.Counts.push_back(Count);
   }
 
   return success();
 }
 
 template <class IntPtrT>
-static uint64_t getRawMagic();
-
-template <>
-uint64_t getRawMagic<uint64_t>() {
-  return
-    uint64_t(255) << 56 |
-    uint64_t('l') << 48 |
-    uint64_t('p') << 40 |
-    uint64_t('r') << 32 |
-    uint64_t('o') << 24 |
-    uint64_t('f') << 16 |
-    uint64_t('r') <<  8 |
-    uint64_t(129);
-}
-
-template <>
-uint64_t getRawMagic<uint32_t>() {
-  return
-    uint64_t(255) << 56 |
-    uint64_t('l') << 48 |
-    uint64_t('p') << 40 |
-    uint64_t('r') << 32 |
-    uint64_t('o') << 24 |
-    uint64_t('f') << 16 |
-    uint64_t('R') <<  8 |
-    uint64_t(129);
-}
-
-template <class IntPtrT>
 bool RawInstrProfReader<IntPtrT>::hasFormat(const MemoryBuffer &DataBuffer) {
   if (DataBuffer.getBufferSize() < sizeof(uint64_t))
     return false;
   uint64_t Magic =
       *reinterpret_cast<const uint64_t *>(DataBuffer.getBufferStart());
-  return getRawMagic<IntPtrT>() == Magic ||
-    sys::getSwappedBytes(getRawMagic<IntPtrT>()) == Magic;
+  return RawInstrProf::getMagic<IntPtrT>() == Magic ||
+         sys::getSwappedBytes(RawInstrProf::getMagic<IntPtrT>()) == Magic;
 }
 
 template <class IntPtrT>
 std::error_code RawInstrProfReader<IntPtrT>::readHeader() {
   if (!hasFormat(*DataBuffer))
     return error(instrprof_error::bad_magic);
   if (DataBuffer->getBufferSize() < sizeof(RawHeader))
     return error(instrprof_error::bad_header);
   auto *Header =
       reinterpret_cast<const RawHeader *>(DataBuffer->getBufferStart());
-  ShouldSwapBytes = Header->Magic != getRawMagic<IntPtrT>();
+  ShouldSwapBytes = Header->Magic != RawInstrProf::getMagic<IntPtrT>();
   return readHeader(*Header);
 }
 
 template <class IntPtrT>
 std::error_code
 RawInstrProfReader<IntPtrT>::readNextHeader(const char *CurrentPos) {
   const char *End = DataBuffer->getBufferEnd();
   // Skip zero padding between profiles.
   while (CurrentPos != End && *CurrentPos == 0)
     ++CurrentPos;
   // If there's nothing left, we're done.
   if (CurrentPos == End)
     return instrprof_error::eof;
   // If there isn't enough space for another header, this is probably just
   // garbage at the end of the file.
   if (CurrentPos + sizeof(RawHeader) > End)
     return instrprof_error::malformed;
   // The writer ensures each profile is padded to start at an aligned address.
   if (reinterpret_cast<size_t>(CurrentPos) % alignOf<uint64_t>())
     return instrprof_error::malformed;
   // The magic should have the same byte order as in the previous header.
   uint64_t Magic = *reinterpret_cast<const uint64_t *>(CurrentPos);
-  if (Magic != swap(getRawMagic<IntPtrT>()))
+  if (Magic != swap(RawInstrProf::getMagic<IntPtrT>()))
     return instrprof_error::bad_magic;
 
   // There's another profile to read, so we need to process the header.
   auto *Header = reinterpret_cast<const RawHeader *>(CurrentPos);
   return readHeader(*Header);
 }
 
-static uint64_t getRawVersion() {
-  return 1;
-}
-
 template <class IntPtrT>
 std::error_code
 RawInstrProfReader<IntPtrT>::readHeader(const RawHeader &Header) {
-  if (swap(Header.Version) != getRawVersion())
+  Version = swap(Header.Version);
+  if (getFormatVersion(Version) != RawInstrProf::Version)
     return error(instrprof_error::unsupported_version);
 
   CountersDelta = swap(Header.CountersDelta);
   NamesDelta = swap(Header.NamesDelta);
   auto DataSize = swap(Header.DataSize);
   auto CountersSize = swap(Header.CountersSize);
   auto NamesSize = swap(Header.NamesSize);
 
std::error_code IndexedInstrProfReader::readHeader() {
   using namespace support;
 
   // Check the magic number.
   uint64_t Magic = endian::readNext<uint64_t, little, unaligned>(Cur);
   if (Magic != IndexedInstrProf::Magic)
     return error(instrprof_error::bad_magic);
 
   // Read the version.
-  FormatVersion = endian::readNext<uint64_t, little, unaligned>(Cur);
-  if (FormatVersion > IndexedInstrProf::Version)
+  Version = endian::readNext<uint64_t, little, unaligned>(Cur);
+  if (getFormatVersion(Version) > IndexedInstrProf::Version)
     return error(instrprof_error::unsupported_version);
 
   // Read the maximal function count.
   MaxFunctionCount = endian::readNext<uint64_t, little, unaligned>(Cur);
 
   // Read the hash type and start offset.
   IndexedInstrProf::HashT HashType = static_cast<IndexedInstrProf::HashT>(
       endian::readNext<uint64_t, little, unaligned>(Cur));
   if (HashType > IndexedInstrProf::HashT::Last)
     return error(instrprof_error::unsupported_hash_type);
   uint64_t HashOffset = endian::readNext<uint64_t, little, unaligned>(Cur);
 
   // The rest of the file is an on disk hash table.
   Index.reset(InstrProfReaderIndex::Create(
       Start + HashOffset, Cur, Start,
-      InstrProfLookupTrait(HashType, FormatVersion)));
+      InstrProfLookupTrait(HashType, getFormatVersion(Version))));
   // Set up our iterator for readNextRecord.
   RecordIterator = Index->data_begin();
 
   return success();
 }
 
 std::error_code IndexedInstrProfReader::getFunctionCounts(
     StringRef FuncName, uint64_t FuncHash, std::vector<uint64_t> &Counts) {
-  auto Iter = Index->find(FuncName);
+
+  StringRef FuncNameOrHash = FuncName;
+  if (hasFeature(instrprof_feature::md5hash_key, Version))
+    FuncNameOrHash = MD5HashStr(FuncName);
+
+  auto Iter = Index->find(FuncNameOrHash);
   if (Iter == Index->end())
     return error(instrprof_error::unknown_function);
 
   // Found it. Look for counters with the right hash.
   ArrayRef<InstrProfRecord> Data = (*Iter);
   if (Data.empty())
     return error(instrprof_error::malformed);
 

lib/ProfileData/InstrProfWriter.cpp

std::pair<uint64_t, uint64_t> InstrProfWriter::writeImpl(raw_ostream &OS) {
 
   // Populate the hash table generator.
   for (const auto &I : FunctionData)
     Generator.insert(I.getKey(), &I.getValue());
 
   using namespace llvm::support;
   endian::Writer<little> LE(OS);
 
+  uint64_t Version = IndexedInstrProf::Version;
+  if (MD5HashKey)
+    Version |= (1ull << (int)instrprof_feature::md5hash_key);
   // Write the header.
   LE.write<uint64_t>(IndexedInstrProf::Magic);
-  LE.write<uint64_t>(IndexedInstrProf::Version);
+  LE.write<uint64_t>(Version);
   LE.write<uint64_t>(MaxFunctionCount);
   LE.write<uint64_t>(static_cast<uint64_t>(IndexedInstrProf::HashType));
 
   // Save a space to write the hash table start location.
   uint64_t HashTableStartLoc = OS.tell();
   LE.write<uint64_t>(0);
   // Write the hash table.
   uint64_t HashTableStart = Generator.Emit(OS);

lib/Transforms/Instrumentation/InstrProfiling.cpp

 
 #include "llvm/Transforms/Instrumentation.h"
 
 #include "llvm/ADT/Triple.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Transforms/Utils/ModuleUtils.h"
+#include "llvm/ProfileData/InstrProf.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "instrprof"
 
+static cl::opt<bool>
+InstrProfileWithNameHash("instr-prof-with-namehash", 
+       cl::desc("FE uses MD5 hash string in place of function names."));
+
 namespace {
 
 class InstrProfiling : public ModulePass {
 public:
   static char ID;
 
   InstrProfiling() : ModulePass(ID) {}
 
   InstrProfiling(const InstrProfOptions &Options)
-      : ModulePass(ID), Options(Options) {}
+      : ModulePass(ID), Options(Options) {
+    if (InstrProfileWithNameHash)
+      this->Options.IncludeNamesInProfile = false;
+  }
 
   const char *getPassName() const override {
     return "Frontend instrumentation-based coverage lowering";
   }
 
   bool runOnModule(Module &M) override;
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
private:
   GlobalVariable *getOrCreateRegionCounters(InstrProfIncrementInst *Inc);
 
   /// Emit runtime registration functions for each profile data variable.
   void emitRegistration();
 
   /// Emit the necessary plumbing to pull in the runtime initialization.
   void emitRuntimeHook();
 
+  /// Emit version and magic global variables
+  void emitHeaderGlobals();
+
   /// Add uses of our data variables and runtime hook.
   void emitUses();
 
   /// Create a static initializer for our data, on platforms that need it,
   /// and for any profile output file that was specified.
   void emitInitialization();
 };
 
if (GlobalVariable *Coverage = M.getNamedGlobal("__llvm_coverage_mapping")) {
     lowerCoverageData(Coverage);
     MadeChange = true;
   }
   if (!MadeChange)
     return false;
 
   emitRegistration();
   emitRuntimeHook();
+  emitHeaderGlobals();
   emitUses();
   emitInitialization();
   return true;
 }
 
 void InstrProfiling::lowerIncrement(InstrProfIncrementInst *Inc) {
   GlobalVariable *Counters = getOrCreateRegionCounters(Inc);
 
for (unsigned I = 0, E = Records->getNumOperands(); I < E; ++I) {
     // Move the name variable to the right section.
     Name->setSection(getNameSection());
     Name->setAlignment(1);
   }
 }
 
 /// Get the name of a profiling variable for a particular function.
 static std::string getVarName(InstrProfIncrementInst *Inc, StringRef VarName) {
-  auto *Arr = cast<ConstantDataArray>(Inc->getName()->getInitializer());
-  StringRef Name = Arr->isCString() ? Arr->getAsCString() : Arr->getAsString();
+  StringRef Name = Inc->getName()->getName();
+  Name = Name.drop_front(strlen("__llvm_profile_name_"));
   return ("__llvm_profile_" + VarName + "_" + Name).str();
 }
 
 GlobalVariable *
 InstrProfiling::getOrCreateRegionCounters(InstrProfIncrementInst *Inc) {
   GlobalVariable *Name = Inc->getName();
   auto It = RegionCounters.find(Name);
   if (It != RegionCounters.end())
void InstrProfiling::emitRuntimeHook() {
   IRBuilder<> IRB(BasicBlock::Create(M->getContext(), "", User));
   auto *Load = IRB.CreateLoad(Var);
   IRB.CreateRet(Load);
 
   // Mark the user variable as used so that it isn't stripped out.
   UsedVars.push_back(User);
 }
 
+void InstrProfiling::emitHeaderGlobals() {
+  const char *const VersionVarName = "__llvm_profile_version";
+  const char *const MagicVarName = "__llvm_profile_magic";
+
+  auto *Int64Ty = Type::getInt64Ty(M->getContext());
+  bool Is64bit = (M->getDataLayout().getPointerSizeInBits() ==
+                  Int64Ty->getPrimitiveSizeInBits());
+  uint64_t Magic = Is64bit ? RawInstrProf::getMagic<uint64_t>()
+                           : RawInstrProf::getMagic<uint32_t>();
+  auto *MagicVar =
+      new GlobalVariable(*M, Int64Ty, false, GlobalValue::LinkOnceODRLinkage,
+                         ConstantInt::get(Int64Ty, Magic), MagicVarName);
+  MagicVar->setVisibility(GlobalValue::HiddenVisibility);
+  UsedVars.push_back(MagicVar);
+
+  uint64_t Version = RawInstrProf::Version;
+  if (!Options.IncludeNamesInProfile)
+    Version |= (1ull << (int)instrprof_feature::md5hash_key);
+  auto *VersionVar =
+      new GlobalVariable(*M, Int64Ty, false, GlobalValue::LinkOnceODRLinkage,
+                         ConstantInt::get(Int64Ty, Version), VersionVarName);
+  UsedVars.push_back(VersionVar);
+}
+
 void InstrProfiling::emitUses() {
   if (UsedVars.empty())
     return;
 
   GlobalVariable *LLVMUsed = M->getGlobalVariable("llvm.used");
   std::vector<Constant *> MergedVars;
   if (LLVMUsed) {
     // Collect the existing members of llvm.used.

test/Instrumentation/InstrProfiling/profiling.ll

 ; RUN: opt < %s -instrprof -S | FileCheck %s
 
 target triple = "x86_64-apple-macosx10.10.0"
 
 @__llvm_profile_name_foo = hidden constant [3 x i8] c"foo"
 ; CHECK: @__llvm_profile_name_foo = hidden constant [3 x i8] c"foo", section "__DATA,__llvm_prf_names", align 1
 @__llvm_profile_name_bar = hidden constant [4 x i8] c"bar\00"
 ; CHECK: @__llvm_profile_name_bar = hidden constant [4 x i8] c"bar\00", section "__DATA,__llvm_prf_names", align 1
-@baz_prof_name = hidden constant [3 x i8] c"baz"
-; CHECK: @baz_prof_name = hidden constant [3 x i8] c"baz", section "__DATA,__llvm_prf_names", align 1
+@__llvm_profile_name_baz = hidden constant [3 x i8] c"baz"
+; CHECK: @__llvm_profile_name_baz = hidden constant [3 x i8] c"baz", section "__DATA,__llvm_prf_names", align 1
 
 ; CHECK: @__llvm_profile_counters_foo = hidden global [1 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
 ; CHECK: @__llvm_profile_data_foo = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
 define void @foo() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__llvm_profile_name_foo, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
 ; CHECK: @__llvm_profile_counters_bar = hidden global [1 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
 ; CHECK: @__llvm_profile_data_bar = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
 define void @bar() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([4 x i8], [4 x i8]* @__llvm_profile_name_bar, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
 ; CHECK: @__llvm_profile_counters_baz = hidden global [3 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
 ; CHECK: @__llvm_profile_data_baz = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
 define void @baz() {
-  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @baz_prof_name, i32 0, i32 0), i64 0, i32 3, i32 0)
-  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @baz_prof_name, i32 0, i32 0), i64 0, i32 3, i32 1)
-  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @baz_prof_name, i32 0, i32 0), i64 0, i32 3, i32 2)
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 0)
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 1)
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 2)
   ret void
 }
 
 declare void @llvm.instrprof.increment(i8*, i64, i32, i32)
 
 ; CHECK: @__llvm_profile_runtime = external global i32
 ; CHECK: @llvm.used = appending global {{.*}} @__llvm_profile_data_foo {{.*}} @__llvm_profile_data_bar {{.*}} @__llvm_profile_data_baz {{.*}} section "llvm.metadata"

test/Instrumentation/InstrProfiling/profiling_md5hash_key.ll

+; RUN: opt < %s -instrprof -instr-prof-with-namehash -S | FileCheck %s
+
+target triple = "x86_64-apple-macosx10.10.0"
+
+@__llvm_profile_name_foo = hidden constant [16 x i8] c"5cf8c24cdb18bdac"
+; CHECK: @__llvm_profile_name_foo = hidden constant [16 x i8] c"5cf8c24cdb18bdac", section "__DATA,__llvm_prf_names", align 1
+@__llvm_profile_name_bar = hidden constant [17 x i8] c"e413754a191db537\00"
+; CHECK: @__llvm_profile_name_bar = hidden constant [17 x i8] c"e413754a191db537\00", section "__DATA,__llvm_prf_names", align 1
+@__llvm_profile_name_baz = hidden constant [16 x i8] c"68bbf6b7a4fffe73"
+; CHECK: @__llvm_profile_name_baz = hidden constant [16 x i8] c"68bbf6b7a4fffe73", section "__DATA,__llvm_prf_names", align 1
+
+; CHECK: @__llvm_profile_counters_foo = hidden global [1 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
+; CHECK: @__llvm_profile_data_foo = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
+define void @foo() {
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @__llvm_profile_name_foo, i32 0, i32 0), i64 0, i32 1, i32 0)
+  ret void
+}
+
+; CHECK: @__llvm_profile_counters_bar = hidden global [1 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
+; CHECK: @__llvm_profile_data_bar = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
+define void @bar() {
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([17 x i8], [17 x i8]* @__llvm_profile_name_bar, i32 0, i32 0), i64 0, i32 1, i32 0)
+  ret void
+}
+
+; CHECK: @__llvm_profile_counters_baz = hidden global [3 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
+; CHECK: @__llvm_profile_data_baz = hidden constant {{.*}}, section "__DATA,__llvm_prf_data", align 8
+
+define void @baz() {
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 0)
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 1)
+  call void @llvm.instrprof.increment(i8* getelementptr inbounds ([16 x i8], [16 x i8]* @__llvm_profile_name_baz, i32 0, i32 0), i64 0, i32 3, i32 2)
+  ret void
+}
+
+declare void @llvm.instrprof.increment(i8*, i64, i32, i32)
+
+; CHECK: @__llvm_profile_runtime = external global i32
+; CHECK: @__llvm_profile_version = linkonce_odr global i64 4294967297
+; CHECK: @llvm.used = appending global [6 x i8*] [i8* bitcast ({ i32, i32, i64, i8*, i64* }* @__llvm_profile_data_foo to i8*), i8* bitcast ({ i32, i32, i64, i8*, i64* }* @__llvm_profile_data_bar to i8*), i8* bitcast ({ i32, i32, i64, i8*, i64* }* @__llvm_profile_data_baz to i8*), i8* bitcast (i32 ()* @__llvm_profile_runtime_user to i8*), i8* bitcast (i64* @__llvm_profile_magic to i8*), i8* bitcast (i64* @__llvm_profile_version to i8*)], section "llvm.metadata"

test/tools/llvm-profdata/raw-64-bits-le-md5.test

+RUN: printf '\201rforpl\377' > %t
+RUN: printf '\1\0\0\0\1\0\0\0' >> %t
+RUN: printf '\2\0\0\0\0\0\0\0' >> %t
+RUN: printf '\3\0\0\0\0\0\0\0' >> %t
+RUN: printf '\40\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\4\0\1\0\0\0' >> %t
+RUN: printf '\0\0\4\0\2\0\0\0' >> %t
+
+RUN: printf '\20\0\0\0' >> %t
+RUN: printf '\1\0\0\0' >> %t
+RUN: printf '\1\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\4\0\2\0\0\0' >> %t
+RUN: printf '\0\0\4\0\1\0\0\0' >> %t
+
+RUN: printf '\20\0\0\0' >> %t
+RUN: printf '\02\0\0\0' >> %t
+RUN: printf '\02\0\0\0\0\0\0\0' >> %t
+RUN: printf '\03\0\4\0\2\0\0\0' >> %t
+RUN: printf '\10\0\4\0\1\0\0\0' >> %t
+
+RUN: printf '\023\0\0\0\0\0\0\0' >> %t
+RUN: printf '\067\0\0\0\0\0\0\0' >> %t
+RUN: printf '\101\0\0\0\0\0\0\0' >> %t
+RUN: printf '5cf8c24cdb18bdace413754a191db537' >> %t
+
+RUN: llvm-profdata show %t -function=foo -counts | FileCheck %s
+
+CHECK: Counters:
+CHECK:   5cf8c24cdb18bdac:
+CHECK:     Hash: 0x0000000000000001
+CHECK:     Counters: 1
+CHECK:     Function count: 19
+CHECK:     Block counts: []
+CHECK: Functions shown: 1
+CHECK: Total functions: 2
+CHECK: Maximum function count: 55
+CHECK: Maximum internal block count: 65

tools/llvm-profdata/llvm-profdata.cpp

static void mergeInstrProfile(const cl::list<std::string> &Inputs,
 
   InstrProfWriter Writer;
   for (const auto &Filename : Inputs) {
     auto ReaderOrErr = InstrProfReader::create(Filename);
     if (std::error_code ec = ReaderOrErr.getError())
       exitWithError(ec.message(), Filename);
 
     auto Reader = std::move(ReaderOrErr.get());
+    if (hasFeature(instrprof_feature::md5hash_key, Reader->getVersion()))
+      Writer.setUseMD5HashKey();
     for (const auto &I : *Reader)
       if (std::error_code EC =
               Writer.addFunctionCounts(I.Name, I.Hash, I.Counts))
         errs() << Filename << ": " << I.Name << ": " << EC.message() << "\n";
     if (Reader->hasError())
       exitWithError(Reader->getError().message(), Filename);
   }
   Writer.write(Output);
static int showInstrProfile(std::string Filename, bool ShowCounts,
                             raw_fd_ostream &OS) {
   auto ReaderOrErr = InstrProfReader::create(Filename);
   if (std::error_code EC = ReaderOrErr.getError())
     exitWithError(EC.message(), Filename);
 
   auto Reader = std::move(ReaderOrErr.get());
   uint64_t MaxFunctionCount = 0, MaxBlockCount = 0;
   size_t ShownFunctions = 0, TotalFunctions = 0;
+  std::string FuncNameOrHashStr;
+  if (!ShowFunction.empty()) {
+    bool IsNameMD5HashString = hasFeature(instrprof_feature::md5hash_key,
+            Reader->getVersion());
+    if (IsNameMD5HashString)
+      FuncNameOrHashStr = MD5HashStr(ShowFunction);
+    else
+      FuncNameOrHashStr = ShowFunction;
+  }
+          
   for (const auto &Func : *Reader) {
     bool Show =
         ShowAllFunctions || (!ShowFunction.empty() &&
-                             Func.Name.find(ShowFunction) != Func.Name.npos);
+                             Func.Name.find(FuncNameOrHashStr) != Func.Name.npos);
 
     ++TotalFunctions;
     assert(Func.Counts.size() > 0 && "function missing entry counter");
     if (Func.Counts[0] > MaxFunctionCount)
       MaxFunctionCount = Func.Counts[0];
 
     if (Show) {
       if (!ShownFunctions)

unittests/ProfileData/InstrProfTest.cpp

TEST_F(InstrProfTest, write_and_read_one_function) {
   ASSERT_EQ(4U, I->Counts.size());
   ASSERT_EQ(1U, I->Counts[0]);
   ASSERT_EQ(2U, I->Counts[1]);
   ASSERT_EQ(3U, I->Counts[2]);
   ASSERT_EQ(4U, I->Counts[3]);
   ASSERT_TRUE(++I == E);
 }
 
+TEST_F(InstrProfTest, write_and_read_one_function_md5hash) {
+  Writer.setUseMD5HashKey();
+  Writer.addFunctionCounts("5cf8c24cdb18bdac", 0x1234, {1, 2, 3, 4});
+  auto Profile = Writer.writeBuffer();
+  readProfile(std::move(Profile));
+
+  auto I = Reader->begin(), E = Reader->end();
+  ASSERT_TRUE(I != E);
+  ASSERT_EQ(StringRef("5cf8c24cdb18bdac"), I->Name);
+  ASSERT_EQ(0x1234U, I->Hash);
+  ASSERT_EQ(4U, I->Counts.size());
+  ASSERT_EQ(1U, I->Counts[0]);
+  ASSERT_EQ(2U, I->Counts[1]);
+  ASSERT_EQ(3U, I->Counts[2]);
+  ASSERT_EQ(4U, I->Counts[3]);
+  ASSERT_TRUE(++I == E);
+
+  StringRef NameKey("foo");
+  std::vector<uint64_t> Counts;
+  ASSERT_TRUE(NoError(Reader->getFunctionCounts(NameKey, 0x1234, Counts)));
+  ASSERT_EQ(4U, Counts.size());
+  ASSERT_EQ(1U, Counts[0]);
+  ASSERT_EQ(2U, Counts[1]);
+  ASSERT_EQ(3U, Counts[2]);
+  ASSERT_EQ(4U, Counts[3]);
+
+}
+
 TEST_F(InstrProfTest, get_function_counts) {
   Writer.addFunctionCounts("foo", 0x1234, {1, 2});
   Writer.addFunctionCounts("foo", 0x1235, {3, 4});
   auto Profile = Writer.writeBuffer();
   readProfile(std::move(Profile));
 
   std::vector<uint64_t> Counts;
   ASSERT_TRUE(NoError(Reader->getFunctionCounts("foo", 0x1234, Counts)));