Add function entry count metadata.

[dnovillo]

Summary

This adds three Function methods to handle function entry counts:
setEntryCount(), hasEntryCount() and getEntryCount().

Entry counts are stored under the MD_prof metadata node with the name
"function_entry_count". They are unsigned 64 bit values set by profilers
(instrumentation and sample profiler changes coming up).

I had some internal debate about what to return when
Function::getEntryCount() is called without the function having its
metadata set. I don't want to return a special sentinel value, because
those are typically problematic down the road.

So, I added a Function::hasEntryCount() predicate. If
Function::getEntryCount() is called with no metadata present, it will
cause a compiler assertion.

[dnovillo]

Return Optional<uint64_t> from getEntryCount to indicate whether a count is available.

I'm getting several odd failures in the testsuite. Even though the new code
is only called from the unittest, I'm getting several related failures.
For example,

bin/opt -S test/Verifier/statepoint.ll -verify

Exit Code: 2

Command Output (stderr):

gc.statepoint mismatch in number of vararg call args

%safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)

gc.statepoint mismatch in number of vararg call args

%safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)

gc.statepoint mismatch in number of vararg call args

%0 = invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i8 addrspace(1)* %obj, i8 addrspace(1)* %obj1)
        to label %normal_dest unwind label %exceptional_return

It seems related to metadata, but I'm not sure how my patch could've triggered
this. Am I missing anything obvious?

Thanks.

[dnovillo]

This update includes the following changes:

• Return Optional<uint64_t> from getEntryCount to indicate whether a count is available.
• Move setEntrycount() and getEntryCount() to lib/IR/Function.cpp.
• Remove @brief from comments.
• Add IR verification to !prof nodes and tests.

[reames]

Reading through the code, I was a left a bit confused as to the actual structure. Could you update with the LangRef update so that it's easier to review? Also, tests for correct usage would be helpful.

lib/IR/Function.cpp

995 ↗

(On Diff #25488)

I'm mildly bothered by using the same name for two different pieces of metadata. Is there a reason not to use a different name here?

[reames]

Before going any further discussing this, I'd like to see an example or
LangRef documentation with an example. I'm not even sure I read the
code right and don't want to waste time discussing something which might
be a result of a misread.

[dnovillo]

• Add documentation for function_entry_count counter.

[reames]

LGTM w/ minor comments addressed. No further review needed.

docs/BranchWeightMetadata.rst

124 ↗

(On Diff #25641)

"function header"? What's that? I assume you mean the definition or declaration itself?

operand -> operator

lib/IR/Verifier.cpp

1480 ↗

(On Diff #25641)

cast rather than dyn_cast

test/Verifier/function-metadata.ll

3 ↗

(On Diff #25641)

I still want to see a positive test example. Your example from the documentation would be a good one. Something to show that the profiling is valid when used properly.

include/llvm/IR/Function.h

 #ifndef LLVM_IR_FUNCTION_H
 #define LLVM_IR_FUNCTION_H
 
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/IR/Argument.h"
 #include "llvm/IR/Attributes.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/CallingConv.h"
+#include "llvm/IR/Constants.h"
 #include "llvm/IR/GlobalObject.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/MDBuilder.h"
+#include "llvm/IR/Metadata.h"
 #include "llvm/Support/Compiler.h"
 
 namespace llvm {
 
 class FunctionType;
 class LLVMContext;
 
 template<> struct ilist_traits<Argument>
setAttributes(
                                  AttributeSet::FunctionIndex, Kind));
   }
   void addFnAttr(StringRef Kind, StringRef Value) {
     setAttributes(
       AttributeSets.addAttribute(getContext(),
                                  AttributeSet::FunctionIndex, Kind, Value));
   }
 
+  /// @brief Set the entry count for this function.
+  void setEntryCount(uint64_t Count) {
+    MDBuilder MDB(getContext());
+    setMetadata(LLVMContext::MD_prof, MDB.createFunctionEntryCount(Count));
+  }
+
+  /// @brief Return true if the function has a known entry count.
+  bool hasEntryCount() const {
+    MDNode *MD = getMetadata(LLVMContext::MD_prof);
+    if (MD && MD->getOperand(0))
+      if (MDString* MDS = dyn_cast<MDString>(MD->getOperand(0)))
+        return MDS->getString().equals("function_entry_count");
+    return false;
+  }
+
+  /// @brief Get the entry count for this function.
+  uint64_t getEntryCount() const {
+    assert(hasEntryCount() && "No entry count available for this function");
+    MDNode *MD = getMetadata(LLVMContext::MD_prof);
+    ConstantInt *CI = mdconst::extract<ConstantInt>(MD->getOperand(1));
+    return CI->getValue().getZExtValue();
+  }
+
   /// @brief Return true if the function has the attribute.
   bool hasFnAttribute(Attribute::AttrKind Kind) const {
     return AttributeSets.hasAttribute(AttributeSet::FunctionIndex, Kind);
   }
   bool hasFnAttribute(StringRef Kind) const {
     return AttributeSets.hasAttribute(AttributeSet::FunctionIndex, Kind);
   }
 

include/llvm/IR/MDBuilder.h

public:
   //===------------------------------------------------------------------===//
 
   /// \brief Return metadata containing two branch weights.
   MDNode *createBranchWeights(uint32_t TrueWeight, uint32_t FalseWeight);
 
   /// \brief Return metadata containing a number of branch weights.
   MDNode *createBranchWeights(ArrayRef<uint32_t> Weights);
 
+  /// \brief Return metadata containing the entry count for a function.
+  MDNode *createFunctionEntryCount(uint64_t Count);
+
   //===------------------------------------------------------------------===//
   // Range metadata.
   //===------------------------------------------------------------------===//
 
   /// \brief Return metadata describing the range [Lo, Hi).
   MDNode *createRange(const APInt &Lo, const APInt &Hi);
 
   /// \brief Return metadata describing the range [Lo, Hi).

lib/IR/MDBuilder.cpp

MDNode *MDBuilder::createBranchWeights(ArrayRef<uint32_t> Weights) {
 
   Type *Int32Ty = Type::getInt32Ty(Context);
   for (unsigned i = 0, e = Weights.size(); i != e; ++i)
     Vals[i + 1] = createConstant(ConstantInt::get(Int32Ty, Weights[i]));
 
   return MDNode::get(Context, Vals);
 }
 
+MDNode *MDBuilder::createFunctionEntryCount(uint64_t Count) {
+  SmallVector<Metadata *, 2> Vals(2);
+  Vals[0] = createString("function_entry_count");
+
+  Type *Int64Ty = Type::getInt64Ty(Context);
+  Vals[1] = createConstant(ConstantInt::get(Int64Ty, Count));
+
+  return MDNode::get(Context, Vals);
+}
+
 MDNode *MDBuilder::createRange(const APInt &Lo, const APInt &Hi) {
   assert(Lo.getBitWidth() == Hi.getBitWidth() && "Mismatched bitwidths!");
 
   Type *Ty = IntegerType::get(Context, Lo.getBitWidth());
   return createRange(ConstantInt::get(Ty, Lo), ConstantInt::get(Ty, Hi));
 }
 
 MDNode *MDBuilder::createRange(Constant *Lo, Constant *Hi) {

unittests/IR/MetadataTest.cpp

TEST_F(FunctionAttachmentTest, Verifier) {
   EXPECT_TRUE(verifyModule(*F->getParent()));
 
   // Functions with a body can.
   (void)new UnreachableInst(Context, BasicBlock::Create(Context, "bb", F));
   EXPECT_FALSE(verifyModule(*F->getParent()));
   EXPECT_FALSE(verifyFunction(*F));
 }
 
+TEST_F(FunctionAttachmentTest, EntryCount) {
+  Function *F = getFunction("foo");
+  EXPECT_FALSE(F->hasEntryCount());
+  F->setEntryCount(12304);
+  EXPECT_TRUE(F->hasEntryCount());
+  EXPECT_EQ(12304u, F->getEntryCount());
+}
+
 }