[CallSiteInfo][CallGraphSection] Extend CallSiteInfo for indirect call type ids

[necipfazil]

Summary

Compute numeric type identifiers for indirect calls using generalized type
identifiers passed from the front-end, and carry them to the back-end with
CallSiteInfo.

The numeric type identifiers will be used at the back-end in the call graph
section.

Original RFC: https://lists.llvm.org/pipermail/llvm-dev/2021-June/151044.html
Updated RFC: https://lists.llvm.org/pipermail/llvm-dev/2021-July/151739.html

[djtodoro]

Has the YAML format of the callSites: (MF attribute) changed? If yes, can you please add a test case that demonstrates that?

[morehouse]

Can we write a test?

llvm/include/llvm/CodeGen/MachineFunction.h

435

The cast looks unneeded.

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp

813

Nit: This variable can go closer to where it is used (or remove entirely).

llvm/lib/Target/X86/X86ISelLowering.cpp

3948

This pattern looks pretty similar across all the architectures. Can share more of this logic in CallSiteInfo?

Maybe a constructor that takes a CallBase * could do all of this.

[necipfazil]

Extract callsite type ids from operand bundles instead of from metadata

• Parent commit now passes callsite type ids through operand bundles. Adapt type id extraction to the changes.
• Fix a few nits

[vitalybuka]

llvm/include/llvm/CodeGen/MachineFunction.h

419

Try to extract separate NFC patch for SmallVector<ArgRegPair, 1> -> struct CallSiteInfo

llvm/lib/CodeGen/MIRPrinter.cpp

522

You can avoid many of them with begin() end() implementation

[vitalybuka]

llvm/include/llvm/CodeGen/MachineFunction.h

419

to clarify, extractNumericCGTypeId stuff should be in a separate patch with some tests

[morehouse]

llvm/include/llvm/CodeGen/MachineFunction.h

444

cast already has an isa assertion, so we can remove those here.

https://llvm.org/docs/ProgrammersManual.html#the-isa-cast-and-dyn-cast-templates

451

[necipfazil]

MipsISelLowering: Fill CallSiteInfo::ArgRegPairs only if -emit-call-site-info
ArgRegPairs was always filled but the container CallSiteInfo instance was only passed to DAG if -emit-call-site-info. Now we pass the CallSiteInfo instance to DAG when --call-graph-section as well. Avoid superfluous ArgRegPairs by filling them only when -emit-call-site-info.

[necipfazil]

Add typeId to callSites YAML format, add tests

• MIRPrinter/MIRParser: Parse/print CallSiteInfo::TypeId. This also enables better testing of the feature.
• Add tests: Test MIRPrinter/MIRParser. Test that call site type ids can be extracted and set from type operand bundles.
• Fix nits.

[necipfazil]

In D105915#2876501, @djtodoro wrote:

Has the YAML format of the callSites: (MF attribute) changed? If yes, can you please add a test case that demonstrates that?

It is now changed with the latest updates (tests included).

[necipfazil]

Refactor extracting and setting type id
Create a constructor for CallSiteInfo that takes a CallBase instance to encapsulate extracting and setting type id, and any warnings/assertions.

[morehouse]

llvm/include/llvm/CodeGen/MachineFunction.h

425

This constructor is implicit from the above TypeId = nullptr. We can just remove this.

430

We can also rely on the default member initializer here.

llvm/lib/Target/X86/X86FastISel.cpp

3584–3586

[necipfazil]

llvm/include/llvm/CodeGen/MachineFunction.h

425

I kept this since the default constructor is used elsewhere, which is implicity deleted because of the below constructor CallSiteInfo(const CallBase).

Besides, I removed the unnecessary initializers for TypeId.

llvm/lib/CodeGen/MIRPrinter.cpp

522

I am not sure if I understand this. Could you please elaborate?

[necipfazil]

This patch is split to 4 per @vitalybuka's feedback: D107108, D107109, D107110, D107111.

The issues raised in this revision are fixed there. Also, a few, smaller tests are added.

We can now abandon this patch.

llvm/include/llvm/CodeGen/MIRYamlMapping.h

struct MachineInstrLoc {
     bool operator==(const MachineInstrLoc &Other) const {
       return BlockNum == Other.BlockNum && Offset == Other.Offset;
     }
   };
 
   MachineInstrLoc CallLocation;
   std::vector<ArgRegPair> ArgForwardingRegs;
 
+  /// Numeric callee type identifier used for call graph section.
+  using TypeIdTy = Optional<uint64_t>;
+  TypeIdTy TypeId;
+
   bool operator==(const CallSiteInfo &Other) const {
     return CallLocation.BlockNum == Other.CallLocation.BlockNum &&
            CallLocation.Offset == Other.CallLocation.Offset;
   }
 };
 
 template <> struct MappingTraits<CallSiteInfo::ArgRegPair> {
   static void mapping(IO &YamlIO, CallSiteInfo::ArgRegPair &ArgReg) {

 namespace yaml {
 
 template <> struct MappingTraits<CallSiteInfo> {
   static void mapping(IO &YamlIO, CallSiteInfo &CSInfo) {
     YamlIO.mapRequired("bb", CSInfo.CallLocation.BlockNum);
     YamlIO.mapRequired("offset", CSInfo.CallLocation.Offset);
     YamlIO.mapOptional("fwdArgRegs", CSInfo.ArgForwardingRegs,
                        std::vector<CallSiteInfo::ArgRegPair>());
+    YamlIO.mapOptional("typeId", CSInfo.TypeId, None);
   }
 
   static const bool flow = true;
 };
 
 /// Serializable representation of debug value substitutions.
 struct DebugValueSubstitution {
   unsigned SrcInst;

llvm/include/llvm/CodeGen/MachineFunction.h

 #include "llvm/ADT/GraphTraits.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/ilist.h"
 #include "llvm/ADT/iterator.h"
 #include "llvm/Analysis/EHPersonalities.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/CodeGen/MachineMemOperand.h"
+#include "llvm/IR/Instructions.h"
 #include "llvm/Support/Allocator.h"
 #include "llvm/Support/ArrayRecycler.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Recycler.h"
 #include "llvm/Target/TargetOptions.h"
 #include <cassert>
 #include <cstdint>
public:
   /// register.
   struct ArgRegPair {
     Register Reg;
     uint16_t ArgNo;
     ArgRegPair(Register R, unsigned Arg) : Reg(R), ArgNo(Arg) {
       assert(Arg < (1 << 16) && "Arg out of range");
     }
   };
+
+  struct CallSiteInfo {
     /// Vector of call argument and its forwarding register.
-  using CallSiteInfo = SmallVector<ArgRegPair, 1>;
-  using CallSiteInfoImpl = SmallVectorImpl<ArgRegPair>;
+    SmallVector<ArgRegPair, 1> ArgRegPairs;
+    /// Callee type id.
+    ConstantInt *TypeId = nullptr;
+
+    /// Extracts a generalized numeric type identifier of a CallBase's type from
+    /// type operand bundle. Returned object can be used to set TypeId. Returns
+    /// null if such operand bundle cannot be found.
+    static ConstantInt *extractNumericCGTypeId(const CallBase &CB) {
+      // Call graph section needs numeric type id only for indirect calls.
+      if (!CB.isIndirectCall())
+        return nullptr;
+
+      auto Opt = CB.getOperandBundle(LLVMContext::OB_type);
+      if (!Opt.hasValue())
+        return nullptr;
+
+      // Get generalized type id string
+      auto OB = Opt.getValue();
+      assert(OB.Inputs.size() == 1 && "invalid input size");
+      auto *OBVal = OB.Inputs.front().get();
+      auto *TypeIdMD = cast<MetadataAsValue>(OBVal)->getMetadata();
+      auto *TypeIdStr = cast<MDString>(TypeIdMD);
+      assert(TypeIdStr->getString().endswith(".generalized") &&
+             "invalid type identifier");
+
+      // Compute numeric type id from generalized type id string
+      uint64_t TypeIdVal = llvm::MD5Hash(TypeIdStr->getString());
+      IntegerType *Int64Ty = Type::getInt64Ty(CB.getContext());
+      return ConstantInt::get(Int64Ty, TypeIdVal, /*IsSigned=*/false);
+    }
+  };
 
 private:
   Delegate *TheDelegate = nullptr;
   GISelChangeObserver *Observer = nullptr;
 
   using CallSiteInfoMap = DenseMap<const MachineInstr *, CallSiteInfo>;
-  /// Map a call instruction to call site arguments forwarding info.
+  /// Map a call instruction to call site arguments forwarding and type id.
   CallSiteInfoMap CallSitesInfo;
 
   /// A helper function that returns call site info for a give call
   /// instruction if debug entry value support is enabled.
   CallSiteInfoMap::iterator getCallSiteInfo(const MachineInstr *MI);
 
   // Callbacks for insertion and removal.
   void handleInsertion(MachineInstr &MI);
void setVariableDbgInfo(const DILocalVariable *Var, const DIExpression *Expr,
     VariableDbgInfos.emplace_back(Var, Expr, Slot, Loc);
   }
 
   VariableDbgInfoMapTy &getVariableDbgInfo() { return VariableDbgInfos; }
   const VariableDbgInfoMapTy &getVariableDbgInfo() const {
     return VariableDbgInfos;
   }
 
-  /// Start tracking the arguments passed to the call \p CallI.
-  void addCallArgsForwardingRegs(const MachineInstr *CallI,
-                                 CallSiteInfoImpl &&CallInfo) {
+  /// Start tracking the arguments passed to the call \p CallI and call type.
+  void addCallSiteInfo(const MachineInstr *CallI, CallSiteInfo &&CallInfo) {
     assert(CallI->isCandidateForCallSiteEntry());
     bool Inserted =
         CallSitesInfo.try_emplace(CallI, std::move(CallInfo)).second;
     (void)Inserted;
     assert(Inserted && "Call site info not unique");
   }
 
   const CallSiteInfoMap &getCallSitesInfo() const {

llvm/include/llvm/CodeGen/SelectionDAG.h

class SelectionDAG {
 
   /// Pool allocation for misc. objects that are created once per SelectionDAG.
   BumpPtrAllocator Allocator;
 
   /// Tracks dbg_value and dbg_label information through SDISel.
   SDDbgInfo *DbgInfo;
 
   using CallSiteInfo = MachineFunction::CallSiteInfo;
-  using CallSiteInfoImpl = MachineFunction::CallSiteInfoImpl;
 
   struct CallSiteDbgInfo {
     CallSiteInfo CSInfo;
     MDNode *HeapAllocSite = nullptr;
     bool NoMerge = false;
   };
 
   DenseMap<const SDNode *, CallSiteDbgInfo> SDCallSiteDbgInfo;
public:
 
   /// \returns true if \p N is any kind of constant or build_vector of
   /// constants, int or float. If a vector, it may not necessarily be a splat.
   inline bool isConstantValueOfAnyType(SDValue N) const {
     return isConstantIntBuildVectorOrConstantInt(N) ||
            isConstantFPBuildVectorOrConstantFP(N);
   }
 
-  void addCallSiteInfo(const SDNode *CallNode, CallSiteInfoImpl &&CallInfo) {
+  void addCallSiteInfo(const SDNode *CallNode, CallSiteInfo &&CallInfo) {
     SDCallSiteDbgInfo[CallNode].CSInfo = std::move(CallInfo);
   }
 
   CallSiteInfo getSDCallSiteInfo(const SDNode *CallNode) {
     auto I = SDCallSiteDbgInfo.find(CallNode);
     if (I != SDCallSiteDbgInfo.end())
       return std::move(I->second).CSInfo;
     return CallSiteInfo();

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp

 }
 
 /// Try to interpret values loaded into registers that forward parameters
 /// for \p CallMI. Store parameters with interpreted value into \p Params.
 static void collectCallSiteParameters(const MachineInstr *CallMI,
                                       ParamSet &Params) {
   const MachineFunction *MF = CallMI->getMF();
   const auto &CalleesMap = MF->getCallSitesInfo();
-  auto CallFwdRegsInfo = CalleesMap.find(CallMI);
+  auto CallSiteInfo = CalleesMap.find(CallMI);
 
   // There is no information for the call instruction.
-  if (CallFwdRegsInfo == CalleesMap.end())
+  if (CallSiteInfo == CalleesMap.end())
     return;
 
   const MachineBasicBlock *MBB = CallMI->getParent();
 
   // Skip the call instruction.
   auto I = std::next(CallMI->getReverseIterator());
 
   FwdRegWorklist ForwardedRegWorklist;
 
   const DIExpression *EmptyExpr =
       DIExpression::get(MF->getFunction().getContext(), {});
 
   // Add all the forwarding registers into the ForwardedRegWorklist.
-  for (const auto &ArgReg : CallFwdRegsInfo->second) {
+  for (const auto &ArgReg : CallSiteInfo->second.ArgRegPairs) {
     bool InsertedReg =
         ForwardedRegWorklist.insert({ArgReg.Reg, {{ArgReg.Reg, EmptyExpr}}})
             .second;
     assert(InsertedReg && "Single register used to forward two arguments?");
     (void)InsertedReg;
   }
 
   // Do not emit CSInfo for undef forwarding registers.

llvm/lib/CodeGen/MIRParser/MIRParser.cpp

if (!CallI->isCall(MachineInstr::IgnoreBundle))
                          "instruction. Instruction at bb:") +
                    Twine(MILoc.BlockNum) + " at offset:" + Twine(MILoc.Offset) +
                    " is not a call instruction");
     MachineFunction::CallSiteInfo CSInfo;
     for (auto ArgRegPair : YamlCSInfo.ArgForwardingRegs) {
       Register Reg;
       if (parseNamedRegisterReference(PFS, Reg, ArgRegPair.Reg.Value, Error))
         return error(Error, ArgRegPair.Reg.SourceRange);
-      CSInfo.emplace_back(Reg, ArgRegPair.ArgNo);
+      CSInfo.ArgRegPairs.emplace_back(Reg, ArgRegPair.ArgNo);
+    }
+    if (YamlCSInfo.TypeId.hasValue()) {
+      IntegerType *Int64Ty = Type::getInt64Ty(Context);
+      CSInfo.TypeId = ConstantInt::get(Int64Ty, YamlCSInfo.TypeId.getValue(),
+                                       /*isSigned=*/false);
     }
 
-    if (TM.Options.EmitCallSiteInfo)
-      MF.addCallArgsForwardingRegs(&*CallI, std::move(CSInfo));
+    if (TM.Options.EmitCallSiteInfo || TM.Options.EmitCallGraphSection)
+      MF.addCallSiteInfo(&*CallI, std::move(CSInfo));
   }
 
-  if (YamlMF.CallSitesInfo.size() && !TM.Options.EmitCallSiteInfo)
+  if (YamlMF.CallSitesInfo.size() &&
+      !(TM.Options.EmitCallSiteInfo || TM.Options.EmitCallGraphSection))
     return error(Twine("Call site info provided but not used"));
   return false;
 }
 
 void MIRParserImpl::setupDebugValueTracking(
     MachineFunction &MF, PerFunctionMIParsingState &PFS,
     const yaml::MachineFunction &YamlMF) {
   // Compute the value of the "next instruction number" field.

llvm/lib/CodeGen/MIRPrinter.cpp

for (auto CSInfo : MF.getCallSitesInfo()) {
     // Prepare instruction position.
     MachineBasicBlock::const_instr_iterator CallI = CSInfo.first->getIterator();
     CallLocation.BlockNum = CallI->getParent()->getNumber();
     // Get call instruction offset from the beginning of block.
     CallLocation.Offset =
         std::distance(CallI->getParent()->instr_begin(), CallI);
     YmlCS.CallLocation = CallLocation;
     // Construct call arguments and theirs forwarding register info.
-    for (auto ArgReg : CSInfo.second) {
+    for (auto ArgReg : CSInfo.second.ArgRegPairs) {
       yaml::CallSiteInfo::ArgRegPair YmlArgReg;
       YmlArgReg.ArgNo = ArgReg.ArgNo;
       printRegMIR(ArgReg.Reg, YmlArgReg.Reg, TRI);
       YmlCS.ArgForwardingRegs.emplace_back(YmlArgReg);
     }
+    // Get type id.
+    if (CSInfo.second.TypeId)
+      YmlCS.TypeId = CSInfo.second.TypeId->getZExtValue();
     YMF.CallSitesInfo.push_back(YmlCS);
   }
 
   // Sort call info by position of call instructions.
   llvm::sort(YMF.CallSitesInfo.begin(), YMF.CallSitesInfo.end(),
              [](yaml::CallSiteInfo A, yaml::CallSiteInfo B) {
                if (A.CallLocation.BlockNum == B.CallLocation.BlockNum)
                  return A.CallLocation.Offset < B.CallLocation.Offset;

llvm/lib/CodeGen/MachineFunction.cpp

try_next:;
   return FilterID;
 }
 
 MachineFunction::CallSiteInfoMap::iterator
 MachineFunction::getCallSiteInfo(const MachineInstr *MI) {
   assert(MI->isCandidateForCallSiteEntry() &&
          "Call site info refers only to call (MI) candidates");
 
-  if (!Target.Options.EmitCallSiteInfo)
+  if (!Target.Options.EmitCallSiteInfo && !Target.Options.EmitCallGraphSection)
     return CallSitesInfo.end();
   return CallSitesInfo.find(MI);
 }
 
 /// Return the call machine instruction or find a call within bundle.
 static const MachineInstr *getCallInstr(const MachineInstr *MI) {
   if (!MI->isBundle())
     return MI;

llvm/lib/CodeGen/SelectionDAG/ScheduleDAGSDNodes.cpp

if (Before == BB->end()) {
       // beginning of the block.
       MI = &Emitter.getBlock()->instr_front();
     } else {
       // Return first instruction after the pre-existing instructions.
       MI = &*std::next(Before);
     }
 
     if (MI->isCandidateForCallSiteEntry() &&
-        DAG->getTarget().Options.EmitCallSiteInfo)
-      MF.addCallArgsForwardingRegs(MI, DAG->getSDCallSiteInfo(Node));
+        (DAG->getTarget().Options.EmitCallSiteInfo ||
+         DAG->getTarget().Options.EmitCallGraphSection))
+      MF.addCallSiteInfo(MI, DAG->getSDCallSiteInfo(Node));
 
     if (DAG->getNoMergeSiteInfo(Node)) {
       MI->setFlag(MachineInstr::MIFlag::NoMerge);
     }
 
     return MI;
   };
 

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

AArch64TargetLowering::LowerCall(CallLoweringInfo &CLI,
   SmallVector<ISD::OutputArg, 32> &Outs = CLI.Outs;
   SmallVector<SDValue, 32> &OutVals = CLI.OutVals;
   SmallVector<ISD::InputArg, 32> &Ins = CLI.Ins;
   SDValue Chain = CLI.Chain;
   SDValue Callee = CLI.Callee;
   bool &IsTailCall = CLI.IsTailCall;
   CallingConv::ID CallConv = CLI.CallConv;
   bool IsVarArg = CLI.IsVarArg;
+  const auto *CB = CLI.CB;
 
   MachineFunction &MF = DAG.getMachineFunction();
   MachineFunction::CallSiteInfo CSInfo;
   bool IsThisReturn = false;
 
   AArch64FunctionInfo *FuncInfo = MF.getInfo<AArch64FunctionInfo>();
   bool TailCallOpt = MF.getTarget().Options.GuaranteedTailCallOpt;
   bool IsSibCall = false;
   bool IsCalleeWin64 = Subtarget->isCallingConvWin64(CallConv);
 
+  // Set type id for call site info.
+  if (MF.getTarget().Options.EmitCallGraphSection && CB &&
+      CB->isIndirectCall()) {
+    CSInfo.TypeId = MachineFunction::CallSiteInfo::extractNumericCGTypeId(*CB);
+    if (!CSInfo.TypeId) {
+      errs() << "warning: cannot find indirect call type id metadata for "
+                "call graph section\n";
+    }
+  }
+
   // Check callee args/returns for SVE registers and set calling convention
   // accordingly.
   if (CallConv == CallingConv::C || CallConv == CallingConv::Fast) {
     bool CalleeOutSVE = any_of(Outs, [](ISD::OutputArg &Out){
       return Out.VT.isScalableVector();
     });
     bool CalleeInSVE = any_of(Ins, [](ISD::InputArg &In){
       return In.VT.isScalableVector();
if (VA.isRegLoc()) {
                           [=](const std::pair<unsigned, SDValue> &Elt) {
                             return Elt.first == VA.getLocReg();
                           })
                 ->second;
         Bits = DAG.getNode(ISD::OR, DL, Bits.getValueType(), Bits, Arg);
         // Call site info is used for function's parameter entry value
         // tracking. For now we track only simple cases when parameter
         // is transferred through whole register.
-        llvm::erase_if(CSInfo, [&VA](MachineFunction::ArgRegPair ArgReg) {
+        llvm::erase_if(CSInfo.ArgRegPairs,
+                       [&VA](MachineFunction::ArgRegPair ArgReg) {
                          return ArgReg.Reg == VA.getLocReg();
                        });
       } else {
         RegsToPass.emplace_back(VA.getLocReg(), Arg);
         RegsUsed.insert(VA.getLocReg());
         const TargetOptions &Options = DAG.getTarget().Options;
         if (Options.EmitCallSiteInfo)
-          CSInfo.emplace_back(VA.getLocReg(), i);
+          CSInfo.ArgRegPairs.emplace_back(VA.getLocReg(), i);
       }
     } else {
       assert(VA.isMemLoc());
 
       SDValue DstAddr;
       MachinePointerInfo DstInfo;
 
       // FIXME: This works on big-endian for composite byvals, which are the

llvm/lib/Target/ARM/ARMISelLowering.cpp

ARMTargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
   SmallVectorImpl<SDValue> &OutVals     = CLI.OutVals;
   SmallVectorImpl<ISD::InputArg> &Ins   = CLI.Ins;
   SDValue Chain                         = CLI.Chain;
   SDValue Callee                        = CLI.Callee;
   bool &isTailCall                      = CLI.IsTailCall;
   CallingConv::ID CallConv              = CLI.CallConv;
   bool doesNotRet                       = CLI.DoesNotReturn;
   bool isVarArg                         = CLI.IsVarArg;
+  const auto *CB = CLI.CB;
 
   MachineFunction &MF = DAG.getMachineFunction();
   ARMFunctionInfo *AFI = MF.getInfo<ARMFunctionInfo>();
   MachineFunction::CallSiteInfo CSInfo;
   bool isStructRet = (Outs.empty()) ? false : Outs[0].Flags.isSRet();
   bool isThisReturn = false;
   bool isCmseNSCall   = false;
   bool isSibCall = false;
   bool PreferIndirect = false;
 
+  // Set type id for call site info.
+  if (MF.getTarget().Options.EmitCallGraphSection && CB &&
+      CB->isIndirectCall()) {
+    CSInfo.TypeId = MachineFunction::CallSiteInfo::extractNumericCGTypeId(*CB);
+    if (!CSInfo.TypeId) {
+      errs() << "warning: cannot find indirect call type id metadata for call "
+                "graph section\n";
+    }
+  }
+
   // Determine whether this is a non-secure function call.
   if (CLI.CB && CLI.CB->getAttributes().hasFnAttribute("cmse_nonsecure_call"))
     isCmseNSCall = true;
 
   // Disable tail calls if they're not supported.
   if (!Subtarget->supportsTailCall())
     isTailCall = false;
 
if (VA.needsCustom() && VA.getLocVT() == MVT::v2f64) {
         assert(VA.getLocVT() == MVT::i32 &&
                "unexpected calling convention register assignment");
         assert(!Ins.empty() && Ins[0].VT == MVT::i32 &&
                "unexpected use of 'returned'");
         isThisReturn = true;
       }
       const TargetOptions &Options = DAG.getTarget().Options;
       if (Options.EmitCallSiteInfo)
-        CSInfo.emplace_back(VA.getLocReg(), i);
+        CSInfo.ArgRegPairs.emplace_back(VA.getLocReg(), i);
       RegsToPass.push_back(std::make_pair(VA.getLocReg(), Arg));
     } else if (isByVal) {
       assert(VA.isMemLoc());
       unsigned offset = 0;
 
       // True if this byval aggregate will be split between registers
       // and memory.
       unsigned ByValArgsCount = CCInfo.getInRegsParamsCount();

llvm/lib/Target/Mips/MipsISelLowering.cpp

MipsTargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
   SmallVectorImpl<ISD::OutputArg> &Outs = CLI.Outs;
   SmallVectorImpl<SDValue> &OutVals     = CLI.OutVals;
   SmallVectorImpl<ISD::InputArg> &Ins   = CLI.Ins;
   SDValue Chain                         = CLI.Chain;
   SDValue Callee                        = CLI.Callee;
   bool &IsTailCall                      = CLI.IsTailCall;
   CallingConv::ID CallConv              = CLI.CallConv;
   bool IsVarArg                         = CLI.IsVarArg;
+  const auto *CB = CLI.CB;
 
   MachineFunction &MF = DAG.getMachineFunction();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   const TargetFrameLowering *TFL = Subtarget.getFrameLowering();
   MipsFunctionInfo *FuncInfo = MF.getInfo<MipsFunctionInfo>();
   bool IsPIC = isPositionIndependent();
 
   // Analyze operands of the call, assigning locations to each operand.
MipsTargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
   CCInfo.AllocateStack(ReservedArgArea, Align(1));
 
   CCInfo.AnalyzeCallOperands(Outs, CC_Mips, CLI.getArgs(),
                              ES ? ES->getSymbol() : nullptr);
 
   // Get a count of how many bytes are to be pushed on the stack.
   unsigned NextStackOffset = CCInfo.getNextStackOffset();
 
-  // Call site info for function parameters tracking.
+  // Call site info for function parameters tracking and call base type info.
   MachineFunction::CallSiteInfo CSInfo;
+  // Set type id for call site info.
+  if (MF.getTarget().Options.EmitCallGraphSection && CB &&
+      CB->isIndirectCall()) {
+    CSInfo.TypeId = MachineFunction::CallSiteInfo::extractNumericCGTypeId(*CB);
+    if (!CSInfo.TypeId) {
+      errs() << "warning: cannot find indirect call type id metadata for call "
+                "graph section\n";
+    }
+  }
 
   // Check if it's really possible to do a tail call. Restrict it to functions
   // that are part of this compilation unit.
   bool InternalLinkage = false;
   if (IsTailCall) {
     IsTailCall = isEligibleForTailCallOptimization(
         CCInfo, NextStackOffset, *MF.getInfo<MipsFunctionInfo>());
      if (GlobalAddressSDNode *G = dyn_cast<GlobalAddressSDNode>(Callee)) {
if (VA.isRegLoc()) {
 
       // If the parameter is passed through reg $D, which splits into
       // two physical registers, avoid creating call site info.
       if (Mips::AFGR64RegClass.contains(VA.getLocReg()))
         continue;
 
       // Collect CSInfo about which register passes which parameter.
       const TargetOptions &Options = DAG.getTarget().Options;
-      if (Options.SupportsDebugEntryValues)
-        CSInfo.emplace_back(VA.getLocReg(), i);
+      if (Options.EmitCallSiteInfo && Options.SupportsDebugEntryValues)
+        CSInfo.ArgRegPairs.emplace_back(VA.getLocReg(), i);
 
       continue;
     }
 
     // Register can't get to this point...
     assert(VA.isMemLoc());
 
     // emit ISD::STORE whichs stores the

llvm/lib/Target/X86/X86FastISel.cpp

if (CopyVT != VA.getValVT()) {
                                 TII.get(Opc), ResultReg + i), FI);
     }
   }
 
   CLI.ResultReg = ResultReg;
   CLI.NumResultRegs = RVLocs.size();
   CLI.Call = MIB;
 
+  // Add call site information (only call type id for call graph section).
+  if (TM.Options.EmitCallGraphSection && CB && CB->isIndirectCall()) {
+    auto *TypeId = MachineFunction::CallSiteInfo::extractNumericCGTypeId(*CB);
+    if (TypeId) {
+      MachineFunction::CallSiteInfo CSInfo;
+      CSInfo.TypeId = TypeId;
+      MF->addCallSiteInfo(CLI.Call, std::move(CSInfo));
+    } else {
+      errs() << "warning: cannot find indirect call type id metadata for call "
+                "graph section\n";
+    }
+  }
+
   return true;
 }
 
 bool
 X86FastISel::fastSelectInstruction(const Instruction *I)  {
   switch (I->getOpcode()) {
   default: break;
   case Instruction::Load:
for (MachineInstr::mop_iterator I = Result->operands_begin(),
     // Found the index reg, now try to rewrite it.
     Register IndexReg = constrainOperandRegClass(Result->getDesc(),
                                                  MO.getReg(), OperandNo);
     if (IndexReg == MO.getReg())
       continue;
     MO.setReg(IndexReg);
   }
 
+  if (MI->isCall())
+    FuncInfo.MF->moveCallSiteInfo(MI, Result);
   Result->addMemOperand(*FuncInfo.MF, createMachineMemOperandFor(LI));
   Result->cloneInstrSymbols(*FuncInfo.MF, *MI);
   MachineBasicBlock::iterator I(MI);
   removeDeadCode(I, std::next(I));
   return true;
 }
 
 unsigned X86FastISel::fastEmitInst_rrrr(unsigned MachineInstOpcode,

llvm/lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
   bool IsIndirectCall = (CB && isa<CallInst>(CB) && CB->isIndirectCall());
   const Module *M = MF.getMMI().getModule();
   Metadata *IsCFProtectionSupported = M->getModuleFlag("cf-protection-branch");
 
   MachineFunction::CallSiteInfo CSInfo;
   if (CallConv == CallingConv::X86_INTR)
     report_fatal_error("X86 interrupts may not be called directly");
 
+  // Set type id for call site info.
+  if (MF.getTarget().Options.EmitCallGraphSection && CB &&
+      CB->isIndirectCall()) {
+    CSInfo.TypeId = MachineFunction::CallSiteInfo::extractNumericCGTypeId(*CB);
+    if (!CSInfo.TypeId) {
+      errs() << "warning: cannot find indirect call type id metadata for call "
+                "graph section\n";
+    }
+  }
+
   bool IsMustTail = CLI.CB && CLI.CB->isMustTailCall();
   if (Subtarget.isPICStyleGOT() && !IsGuaranteeTCO && !IsMustTail) {
     // If we are using a GOT, disable tail calls to external symbols with
     // default visibility. Tail calling such a symbol requires using a GOT
     // relocation, which forces early binding of the symbol. This breaks code
     // that require lazy function symbol resolution. Using musttail or
     // GuaranteedTailCallOpt will override this.
     GlobalAddressSDNode *G = dyn_cast<GlobalAddressSDNode>(Callee);
if (VA.needsCustom()) {
       assert(VA.getValVT() == MVT::v64i1 &&
              "Currently the only custom case is when we split v64i1 to 2 regs");
       // Split v64i1 value into two registers
       Passv64i1ArgInRegs(dl, DAG, Arg, RegsToPass, VA, ArgLocs[++I], Subtarget);
     } else if (VA.isRegLoc()) {
       RegsToPass.push_back(std::make_pair(VA.getLocReg(), Arg));
       const TargetOptions &Options = DAG.getTarget().Options;
       if (Options.EmitCallSiteInfo)
-        CSInfo.emplace_back(VA.getLocReg(), I);
+        CSInfo.ArgRegPairs.emplace_back(VA.getLocReg(), I);
       if (isVarArg && IsWin64) {
         // Win64 ABI requires argument XMM reg to be copied to the corresponding
         // shadow reg if callee is a varargs function.
         Register ShadowReg;
         switch (VA.getLocReg()) {
         case X86::XMM0: ShadowReg = X86::RCX; break;
         case X86::XMM1: ShadowReg = X86::RDX; break;
         case X86::XMM2: ShadowReg = X86::R8; break;

llvm/test/CodeGen/AArch64/call-site-info-typeid.ll

+; Tests that call site type ids can be extracted and set from type operand
+; bundles.
+
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section -mtriple aarch64-linux-gnu %s -stop-before=finalize-isel -o - | FileCheck %s
+
+; ModuleID = 'test.c'
+source_filename = "test.c"
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+target triple = "aarch64-unknown-linux-gnu"
+
+define dso_local void @foo(i8 signext %a) !type !3 {
+entry:
+  ret void
+}
+
+; CHECK: name: main
+define dso_local i32 @main() !type !4 {
+entry:
+  %retval = alloca i32, align 4
+  %fp = alloca void (i8)*, align 8
+  store i32 0, i32* %retval, align 4
+  store void (i8)* @foo, void (i8)** %fp, align 8
+  %0 = load void (i8)*, void (i8)** %fp, align 8
+  ; CHECK: callSites:
+  ; CHECK-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+  ; CHECK-NEXT: 7854600665770582568 }
+  call void %0(i8 signext 97) [ "type"(metadata !"_ZTSFvcE.generalized") ]
+  ret i32 0
+}
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 7, !"frame-pointer", i32 2}
+!3 = !{i64 0, !"_ZTSFvcE.generalized"}
+!4 = !{i64 0, !"_ZTSFiE.generalized"}

llvm/test/CodeGen/ARM/call-site-info-typeid.ll

+; Tests that call site type ids can be extracted and set from type operand
+; bundles.
+
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section -mtriple arm-linux-gnu %s -stop-before=finalize-isel -o - | FileCheck %s
+
+; ModuleID = 'test.c'
+source_filename = "test.c"
+target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
+target triple = "armv4t-unknown-linux-gnu"
+
+define dso_local void @foo(i8 signext %a) !type !3 {
+entry:
+  ret void
+}
+
+; CHECK: name: main
+define dso_local i32 @main() !type !4 {
+entry:
+  %retval = alloca i32, align 4
+  %fp = alloca void (i8)*, align 8
+  store i32 0, i32* %retval, align 4
+  store void (i8)* @foo, void (i8)** %fp, align 8
+  %0 = load void (i8)*, void (i8)** %fp, align 8
+  ; CHECK: callSites:
+  ; CHECK-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+  ; CHECK-NEXT: 7854600665770582568 }
+  call void %0(i8 signext 97) [ "type"(metadata !"_ZTSFvcE.generalized") ]
+  ret i32 0
+}
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 7, !"frame-pointer", i32 2}
+!3 = !{i64 0, !"_ZTSFvcE.generalized"}
+!4 = !{i64 0, !"_ZTSFiE.generalized"}

llvm/test/CodeGen/MIR/X86/call-site-info-typeid-mir-printer-parser.ll

+; Test MIR printer and parser for type id field in call site info. It is used
+; for propogating call site type identifiers to emit in the call graph section.
+
+; Multiplex --call-graph-section and -emit-call-site-info as both utilize
+; CallSiteInfo and callSites.
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Test printer and parser with --call-graph-section only.
+
+; Test printer.
+; Verify that fwdArgRegs is not set, typeId is set.
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section %s -stop-before=finalize-isel -o %t1.mir
+; RUN: cat %t1.mir | FileCheck %s --check-prefix=PRINTER_CGS
+; PRINTER_CGS: name: main
+; PRINTER_CGS: callSites:
+; PRINTER_CGS-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+; PRINTER_CGS-NEXT: 7854600665770582568 }
+
+
+; Test parser.
+; Verify that we get the same result.
+; RUN: llc --call-graph-section %t1.mir -run-pass=finalize-isel -o - \
+; RUN: | FileCheck %s --check-prefix=PARSER_CGS
+; PARSER_CGS: name: main
+; PARSER_CGS: callSites:
+; PARSER_CGS-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+; PARSER_CGS-NEXT: 7854600665770582568 }
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Test printer and parser with -emit-call-site-info only.
+
+; Test printer.
+; Verify that fwdArgRegs is set, typeId is not set.
+; RUN: llc -emit-call-site-info %s -stop-before=finalize-isel -o %t2.mir
+; RUN: cat %t2.mir | FileCheck %s --check-prefix=PRINTER_CSI
+; PRINTER_CSI: name: main
+; PRINTER_CSI: callSites:
+; PRINTER_CSI-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs:
+; PRINTER_CSI-NEXT: { arg: 0, reg: '$edi' }
+; PRINTER_CSI-NOT: typeId:
+
+
+; Test parser.
+; Verify that we get the same result.
+; RUN: llc -emit-call-site-info %t2.mir -run-pass=finalize-isel -o - \
+; RUN: | FileCheck %s --check-prefix=PARSER_CSI
+; PARSER_CSI: name: main
+; PARSER_CSI: callSites:
+; PARSER_CSI-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs:
+; PARSER_CSI-NEXT: { arg: 0, reg: '$edi' }
+; PARSER_CSI-NOT: typeId:
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Test printer and parser with both -emit-call-site-info and --call-graph-section.
+
+; Test printer.
+; Verify both fwdArgRegs and typeId are set.
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section -emit-call-site-info %s -stop-before=finalize-isel -o %t2.mir
+; RUN: cat %t2.mir | FileCheck %s --check-prefix=PRINTER_CGS_CSI
+; PRINTER_CGS_CSI: name: main
+; PRINTER_CGS_CSI: callSites:
+; PRINTER_CGS_CSI-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs:
+; PRINTER_CGS_CSI-NEXT: { arg: 0, reg: '$edi' }, typeId:
+; PRINTER_CGS_CSI-NEXT:   7854600665770582568 }
+
+
+; Test parser.
+; Verify that we get the same result.
+; RUN: llc --call-graph-section -emit-call-site-info %t2.mir -run-pass=finalize-isel -o - \
+; RUN: | FileCheck %s --check-prefix=PARSER_CGS_CSI
+; PARSER_CGS_CSI: name: main
+; PARSER_CGS_CSI: callSites:
+; PARSER_CGS_CSI-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs:
+; PARSER_CGS_CSI-NEXT: { arg: 0, reg: '$edi' }, typeId:
+; PARSER_CGS_CSI-NEXT:   7854600665770582568 }
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+; ModuleID = 'test.c'
+source_filename = "test.c"
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local void @foo(i8 signext %a) !type !3 {
+entry:
+  ret void
+}
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local i32 @main() !type !4 {
+entry:
+  %retval = alloca i32, align 4
+  %fp = alloca void (i8)*, align 8
+  store i32 0, i32* %retval, align 4
+  store void (i8)* @foo, void (i8)** %fp, align 8
+  %0 = load void (i8)*, void (i8)** %fp, align 8
+  call void %0(i8 signext 97) [ "type"(metadata !"_ZTSFvcE.generalized") ]
+  ret i32 0
+}
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 7, !"frame-pointer", i32 2}
+!3 = !{i64 0, !"_ZTSFvcE.generalized"}
+!4 = !{i64 0, !"_ZTSFiE.generalized"}

llvm/test/CodeGen/Mips/call-site-info-typeid.ll

+; Tests that call site type ids can be extracted and set from type operand
+; bundles.
+
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section -mtriple=mips-linux-gnu %s -stop-before=finalize-isel -o - | FileCheck %s
+
+; ModuleID = 'test.c'
+source_filename = "test.c"
+target datalayout = "E-m:m-p:32:32-i8:8:32-i16:16:32-i64:64-n32-S64"
+target triple = "mips-unknown-linux-gnu"
+
+define dso_local void @foo(i8 signext %a) !type !3 {
+entry:
+  ret void
+}
+
+; CHECK: name: main
+define dso_local i32 @main() !type !4 {
+entry:
+  %retval = alloca i32, align 4
+  %fp = alloca void (i8)*, align 8
+  store i32 0, i32* %retval, align 4
+  store void (i8)* @foo, void (i8)** %fp, align 8
+  %0 = load void (i8)*, void (i8)** %fp, align 8
+  ; CHECK: callSites:
+  ; CHECK-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+  ; CHECK-NEXT: 7854600665770582568 }
+  call void %0(i8 signext 97) [ "type"(metadata !"_ZTSFvcE.generalized") ]
+  ret i32 0
+}
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 7, !"frame-pointer", i32 2}
+!3 = !{i64 0, !"_ZTSFvcE.generalized"}
+!4 = !{i64 0, !"_ZTSFiE.generalized"}

llvm/test/CodeGen/X86/call-site-info-typeid.ll

+; Tests that call site type ids can be extracted and set from type operand
+; bundles.
+
+; Verify the exact typeId value to ensure it is not garbage but the value
+; computed as the type id from the type operand bundle.
+; RUN: llc --call-graph-section -mtriple=x86_64-unknown-linux %s -stop-before=finalize-isel -o - | FileCheck %s
+
+; ModuleID = 'test.c'
+source_filename = "test.c"
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define dso_local void @foo(i8 signext %a) !type !3 {
+entry:
+  ret void
+}
+
+; CHECK: name: main
+define dso_local i32 @main() !type !4 {
+entry:
+  %retval = alloca i32, align 4
+  %fp = alloca void (i8)*, align 8
+  store i32 0, i32* %retval, align 4
+  store void (i8)* @foo, void (i8)** %fp, align 8
+  %0 = load void (i8)*, void (i8)** %fp, align 8
+  ; CHECK: callSites:
+  ; CHECK-NEXT: - { bb: {{.*}}, offset: {{.*}}, fwdArgRegs: [], typeId:
+  ; CHECK-NEXT: 7854600665770582568 }
+  call void %0(i8 signext 97) [ "type"(metadata !"_ZTSFvcE.generalized") ]
+  ret i32 0
+}
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 7, !"frame-pointer", i32 2}
+!3 = !{i64 0, !"_ZTSFvcE.generalized"}
+!4 = !{i64 0, !"_ZTSFiE.generalized"}