[BPF] Support for "btf_decl_tag" annotations for arguments of extern functions

[eddyz87]

Summary

This adds support for "btf_decl_tag" annotations for arguments of extern functions. The changes are split into several commits:

• Triple::isBPF() utility method -- adds a simple utility method for llvm::Triple class;
• preserve btf_decl_tag for parameters of extern functions -- modifies CLang frontend to generate DILocalVariable entries with attached "btf_dec_tag" annotations for parameters of extern functions;
• generate btf_decl_tag records for params of extern functions -- modifies BPF backend to process "btf_decl_tag" annotations for parameters of extern functions.

Below are descriptions for the last two commits:

Preserve btf_decl_tag for parameters of extern functions

Generate DILocalVariable entries for parameters of extern functions, the "annotations" field of DILocalVariable is used to link "btf_decl_tag" annotation with the parameter.

Do this only for BPF backend as there are no other users for this information. Final DWARF is valid as "Appendix A" is very much lax in
what is allowed as attributes for "DW_TAG_formal_parameter":

DWARF does not in general require that a given debugging information entry contain a particular attribute or set of attributes. Instead, a DWARF producer is free to generate any, all, or none of the attributes ... other attributes ... may also appear in a given debugging information entry.

DWARF Debugging Information Format Version 5,
Appendix A: Attributes by Tag Value (Informative)
Page 251, Line 3.

Generate btf_decl_tag records for params of extern functions

After frontend changes in the following commit: "BPF: preserve btf_decl_tag for parameters of extern functions" same mechanics could be used to get the list of function parameters and associated btf_decl_tag entries for both extern and non-extern functions.

This commit extracts this mechanics as a separate auxiliary function BTFDebug::processDISubprogram(). The function is called for both
extern and non-extern functions in order to generated corresponding BTF_DECL_TAG records.

[yonghong-song]

Please separate the patch to clang and llvm part.

clang/lib/CodeGen/CGDebugInfo.cpp

4218

Looks like this is for bpf only? Can we extend it to non-bpf as well? Currently, I think most use case, if any, will be in vmlinux (e.g., for struct_ops callback, etc).

[yonghong-song]

clang/lib/CodeGen/CGDebugInfo.cpp

4218

sorry, the above comment is not right about struct_ops, I thought this is for function pointer parameters, but it is for extern functions. Yes, for vmlinux, we don't need to support extern func arg decl tags since we always have real function definitions. So bpf only support is okay.

[eddyz87]

In D140929#4025118, @yonghong-song wrote:

Please separate the patch to clang and llvm part.

It's already split in three commits: two for clang, one for llvm.
Not sure if this is possible to see in the Phabricator UI, though.

[eddyz87]

In D140929#4025562, @eddyz87 wrote:

In D140929#4025118, @yonghong-song wrote:

Please separate the patch to clang and llvm part.

It's already split in three commits: two for clang, one for llvm.
Not sure if this is possible to see in the Phabricator UI, though.

However arc patch D140929 applies this revision as a single commit. Maybe I messed something up while creating revision.

[eddyz87]

Reorganize commits as a stack.

[eddyz87]

I think I messed up while creating this revision. I've created a proper stack of revisions with a tip at D140971. Sorry for all the spam in emails.

clang/lib/CodeGen/CGDebugInfo.cpp

void CGDebugInfo::EmitFunctionDecl(GlobalDecl GD, SourceLocation Loc,
   }
   unsigned LineNo = getLineNumber(Loc);
   unsigned ScopeLine = 0;
   llvm::DISubprogram::DISPFlags SPFlags = llvm::DISubprogram::SPFlagZero;
   if (CGM.getLangOpts().Optimize)
     SPFlags |= llvm::DISubprogram::SPFlagOptimized;
 
   llvm::DINodeArray Annotations = CollectBTFDeclTagAnnotations(D);
+  llvm::DISubroutineType *STy = getOrCreateFunctionType(D, FnType, Unit);
   llvm::DISubprogram *SP = DBuilder.createFunction(
       FDContext, Name, LinkageName, Unit, LineNo,
-      getOrCreateFunctionType(D, FnType, Unit), ScopeLine, Flags, SPFlags,
+      STy, ScopeLine, Flags, SPFlags,
       TParamsArray.get(), getFunctionDeclaration(D), nullptr, Annotations);
 
+  // Preserve btf_decl_tag attributes for parameters of extern functions
+  // for BPF target. The parameters created in this loop are attached as
+  // DISubprogram's retainedNodes in the subsequent finalizeSubprogram call.
+  if (IsDeclForCallSite && CGM.getTarget().getTriple().isBPF()) {
+    if (auto *FD = dyn_cast<FunctionDecl>(D)) {
+      llvm::DITypeRefArray ParamTypes = STy->getTypeArray();
+      unsigned ArgNo = 1;
+      for (ParmVarDecl *PD : FD->parameters()) {
+        llvm::DINodeArray ParamAnnotations = CollectBTFDeclTagAnnotations(PD);
+        DBuilder.createParameterVariable
+          (SP, PD->getName(), ArgNo, Unit, LineNo, ParamTypes[ArgNo], true,
+           llvm::DINode::FlagZero, ParamAnnotations);
+        ++ArgNo;
+      }
+    }
+  }
+
   if (IsDeclForCallSite)
     Fn->setSubprogram(SP);
 
   DBuilder.finalizeSubprogram(SP);
 }
 
 void CGDebugInfo::EmitFuncDeclForCallSite(llvm::CallBase *CallOrInvoke,
                                           QualType CalleeType,

clang/test/CodeGen/bpf-decl-tag-extern-func-args.c

+// RUN: %clang -target bpf -S -emit-llvm -g -O2 -o - %s  | FileCheck %s
+
+#define __decl_tag(x) __attribute__((btf_decl_tag(x)))
+
+extern void foo(int aa __decl_tag("aa_tag"), long bb __decl_tag("bb_tag"));
+extern void bar(char cc __decl_tag("cc_tag"));
+extern void buz(int __decl_tag("buz_arg_tag"));
+
+void root(void) {
+  foo(0, 1);
+  bar(0);
+  buz(0);
+}
+// CHECK: [[foo:![0-9]+]] = !DISubprogram(name: "foo", {{.*}}, retainedNodes: [[foo_nodes:![0-9]+]])
+// CHECK: [[foo_nodes]] = !{[[aa:![0-9]+]], [[bb:![0-9]+]]}
+
+// CHECK: [[aa]] = !DILocalVariable(name: "aa", arg: 1, scope: [[foo]], {{.*}}, annotations: [[aa_annot:![0-9]+]])
+// CHECK: [[aa_annot]] = !{[[aa_tag:![0-9]+]]}
+// CHECK: [[aa_tag]] = !{!"btf_decl_tag", !"aa_tag"}
+
+// CHECK: [[bb]] = !DILocalVariable(name: "bb", arg: 2, scope: [[foo]], {{.*}}, annotations: [[bb_annot:![0-9]+]])
+// CHECK: [[bb_annot]] = !{[[bb_tag:![0-9]+]]}
+// CHECK: [[bb_tag]] = !{!"btf_decl_tag", !"bb_tag"}
+
+// CHECK: [[bar:![0-9]+]] = !DISubprogram(name: "bar", {{.*}}, retainedNodes: [[bar_nodes:![0-9]+]])
+// CHECK: [[bar_nodes]] = !{[[cc:![0-9]+]]}
+
+// CHECK: [[cc]] = !DILocalVariable(name: "cc", arg: 1, scope: [[bar]], {{.*}}, annotations: [[cc_annot:![0-9]+]])
+// CHECK: [[cc_annot]] = !{[[cc_tag:![0-9]+]]}
+// CHECK: [[cc_tag]] = !{!"btf_decl_tag", !"cc_tag"}
+
+// CHECK: [[buz:![0-9]+]] = !DISubprogram(name: "buz", {{.*}}, retainedNodes: [[buz_nodes:![0-9]+]])
+// CHECK: [[buz_nodes]] = !{[[buz_arg:![0-9]+]]}
+
+// CHECK: [[buz_arg]] = !DILocalVariable(arg: 1, scope: [[buz]], {{.*}}, annotations: [[buz_arg_annot:![0-9]+]])
+// CHECK: [[buz_arg_annot]] = !{[[buz_arg_tag:![0-9]+]]}
+// CHECK: [[buz_arg_tag]] = !{!"btf_decl_tag", !"buz_arg_tag"}

llvm/include/llvm/TargetParser/Triple.h

public:
   }
 
   /// Tests whether the target is X32.
   bool isX32() const {
     EnvironmentType Env = getEnvironment();
     return Env == Triple::GNUX32 || Env == Triple::MuslX32;
   }
 
+  /// Tests whether the target is eBPF.
+  bool isBPF() const {
+    return getArch() == Triple::bpfel || getArch() == Triple::bpfeb;
+  }
+
   /// Tests whether the target supports comdat
   bool supportsCOMDAT() const {
     return !(isOSBinFormatMachO() || isOSBinFormatXCOFF() ||
              isOSBinFormatDXContainer());
   }
 
   /// Tests whether the target uses emulated TLS as default.
   bool hasDefaultEmulatedTLS() const {

llvm/lib/Target/BPF/BTFDebug.h

class BTFDebug : public DebugHandlerBase {
 
   /// Generate types for function prototypes.
   void processFuncPrototypes(const Function *);
 
   /// Generate types for decl annotations.
   void processDeclAnnotations(DINodeArray Annotations, uint32_t BaseTypeId,
                               int ComponentId);
 
+  /// Generate types for DISubprogram and it's arguments.
+  uint32_t processDISubprogram(const DISubprogram *SP,
+                               uint32_t ProtoTypeId,
+                               uint8_t Scope);
+
   /// Generate BTF type_tag's. If BaseTypeId is nonnegative, the last
   /// BTF type_tag in the chain points to BaseTypeId. Otherwise, it points to
   /// the base type of DTy. Return the type id of the first BTF type_tag
   /// in the chain. If no type_tag's are generated, a negative value
   /// is returned.
   int genBTFTypeTags(const DIDerivedType *DTy, int BaseTypeId);
 
   /// Generate one field relocation record.

llvm/lib/Target/BPF/BTFDebug.cpp

for (const Metadata *Annotation : Annotations->operands()) {
 
     const MDString *Value = cast<MDString>(MD->getOperand(1));
     auto TypeEntry = std::make_unique<BTFTypeDeclTag>(BaseTypeId, ComponentIdx,
                                                       Value->getString());
     addType(std::move(TypeEntry));
   }
 }
 
+uint32_t BTFDebug::processDISubprogram(const DISubprogram *SP,
+                                       uint32_t ProtoTypeId,
+                                       uint8_t Scope) {
+  auto FuncTypeEntry =
+    std::make_unique<BTFTypeFunc>(SP->getName(), ProtoTypeId, Scope);
+  uint32_t FuncId = addType(std::move(FuncTypeEntry));
+
+  // Process argument annotations.
+  for (const DINode *DN : SP->getRetainedNodes()) {
+    if (const auto *DV = dyn_cast<DILocalVariable>(DN)) {
+      uint32_t Arg = DV->getArg();
+      if (Arg)
+        processDeclAnnotations(DV->getAnnotations(), FuncId, Arg - 1);
+    }
+  }
+  processDeclAnnotations(SP->getAnnotations(), FuncId, -1);
+
+  return FuncId;
+}
+
 /// Generate btf_type_tag chains.
 int BTFDebug::genBTFTypeTags(const DIDerivedType *DTy, int BaseTypeId) {
   SmallVector<const MDString *, 4> MDStrs;
   DINodeArray Annots = DTy->getAnnotations();
   if (Annots) {
     // For type with "int __tag1 __tag2 *p", the MDStrs will have
     // content: [__tag1, __tag2].
     for (const Metadata *Annotations : Annots->operands()) {
void BTFDebug::beginFunctionImpl(const MachineFunction *MF) {
   }
 
   // Construct subprogram func proto type.
   uint32_t ProtoTypeId;
   visitSubroutineType(SP->getType(), true, FuncArgNames, ProtoTypeId);
 
   // Construct subprogram func type
   uint8_t Scope = SP->isLocalToUnit() ? BTF::FUNC_STATIC : BTF::FUNC_GLOBAL;
-  auto FuncTypeEntry =
-      std::make_unique<BTFTypeFunc>(SP->getName(), ProtoTypeId, Scope);
-  uint32_t FuncTypeId = addType(std::move(FuncTypeEntry));
-
-  // Process argument annotations.
-  for (const DINode *DN : SP->getRetainedNodes()) {
-    if (const auto *DV = dyn_cast<DILocalVariable>(DN)) {
-      uint32_t Arg = DV->getArg();
-      if (Arg)
-        processDeclAnnotations(DV->getAnnotations(), FuncTypeId, Arg - 1);
-    }
-  }
-
-  processDeclAnnotations(SP->getAnnotations(), FuncTypeId, -1);
+  uint32_t FuncTypeId = processDISubprogram(SP, ProtoTypeId, Scope);
 
   for (const auto &TypeEntry : TypeEntries)
     TypeEntry->completeType(*this);
 
   // Construct funcinfo and the first lineinfo for the function.
   MCSymbol *FuncLabel = Asm->getFunctionBegin();
   BTFFuncInfo FuncInfo;
   FuncInfo.Label = FuncLabel;
void BTFDebug::processFuncPrototypes(const Function *F) {
 
   // Do not emit again if already emitted.
   if (!ProtoFunctions.insert(F).second)
     return;
 
   uint32_t ProtoTypeId;
   const std::unordered_map<uint32_t, StringRef> FuncArgNames;
   visitSubroutineType(SP->getType(), false, FuncArgNames, ProtoTypeId);
-
-  uint8_t Scope = BTF::FUNC_EXTERN;
-  auto FuncTypeEntry =
-      std::make_unique<BTFTypeFunc>(SP->getName(), ProtoTypeId, Scope);
-  uint32_t FuncId = addType(std::move(FuncTypeEntry));
-
-  processDeclAnnotations(SP->getAnnotations(), FuncId, -1);
+  uint32_t FuncId = processDISubprogram(SP, ProtoTypeId, BTF::FUNC_EXTERN);
 
   if (F->hasSection()) {
     StringRef SecName = F->getSection();
 
     if (DataSecEntries.find(std::string(SecName)) == DataSecEntries.end()) {
       DataSecEntries[std::string(SecName)] =
           std::make_unique<BTFKindDataSec>(Asm, std::string(SecName));
     }

llvm/test/CodeGen/BPF/BTF/tag-extern-func.ll

+; RUN: llc -march=bpfel -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+; RUN: llc -march=bpfeb -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+
+; Source code:
+;   #define __tag(x) __attribute__((btf_decl_tag(x)))
+;
+;   extern void foo(int x __tag("x_tag"), int y __tag("y_tag")) __tag("foo_tag");
+;
+;   void root(void) {
+;     foo(0, 0);
+;   }
+; Compilation flag:
+;   clang -target bpf -O2 -g -S -emit-llvm test.c
+
+
+; Function Attrs: nounwind
+define dso_local void @root() local_unnamed_addr #0 !dbg !7 {
+entry:
+  tail call void @foo(i32 noundef 0, i32 noundef 0) #2, !dbg !12
+  ret void, !dbg !13
+}
+
+declare !dbg !14 dso_local void @foo(i32 noundef, i32 noundef) local_unnamed_addr #1
+
+attributes #0 = { nounwind "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+attributes #1 = { "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+attributes #2 = { nounwind }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C11, file: !1, producer: "clang version 16.0.0 (https://github.com/llvm/llvm-project.git 603e8490729e477680f0bc8284e136ceeb66e7f4)", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "/home/eddy/work/tasks/llvm-decl-tags/test.c", directory: "/home/eddy/work/tasks/llvm-decl-tags", checksumkind: CSK_MD5, checksum: "0a768161d04f21ca6734ef31f6258656")
+!2 = !{i32 7, !"Dwarf Version", i32 5}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = !{i32 1, !"wchar_size", i32 4}
+!5 = !{i32 7, !"frame-pointer", i32 2}
+!6 = !{!"clang version 16.0.0 (https://github.com/llvm/llvm-project.git 603e8490729e477680f0bc8284e136ceeb66e7f4)"}
+!7 = distinct !DISubprogram(name: "root", scope: !8, file: !8, line: 5, type: !9, scopeLine: 5, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !11)
+!8 = !DIFile(filename: "test.c", directory: "/home/eddy/work/tasks/llvm-decl-tags", checksumkind: CSK_MD5, checksum: "0a768161d04f21ca6734ef31f6258656")
+!9 = !DISubroutineType(types: !10)
+!10 = !{null}
+!11 = !{}
+!12 = !DILocation(line: 6, column: 3, scope: !7)
+!13 = !DILocation(line: 7, column: 1, scope: !7)
+!14 = !DISubprogram(name: "foo", scope: !8, file: !8, line: 3, type: !15, flags: DIFlagPrototyped, spFlags: DISPFlagOptimized, retainedNodes: !18, annotations: !25)
+!15 = !DISubroutineType(types: !16)
+!16 = !{null, !17, !17}
+!17 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!18 = !{!19, !22}
+!19 = !DILocalVariable(name: "x", arg: 1, scope: !14, file: !8, line: 3, type: !17, annotations: !20)
+!20 = !{!21}
+!21 = !{!"btf_decl_tag", !"x_tag"}
+!22 = !DILocalVariable(name: "y", arg: 2, scope: !14, file: !8, line: 3, type: !17, annotations: !23)
+!23 = !{!24}
+!24 = !{!"btf_decl_tag", !"y_tag"}
+!25 = !{!26}
+!26 = !{!"btf_decl_tag", !"foo_tag"}
+
+; CHECK:  .long 0                               # BTF_KIND_FUNC_PROTO(id = 1)
+; CHECK-NEXT:   .long 218103808                       # 0xd000000
+; CHECK-NEXT:   .long 0
+; CHECK-NEXT:   .long 1                               # BTF_KIND_FUNC(id = 2)
+; CHECK-NEXT:   .long 201326593                       # 0xc000001
+; CHECK-NEXT:   .long 1
+; CHECK-NEXT:   .long 0                               # BTF_KIND_FUNC_PROTO(id = 3)
+; CHECK-NEXT:   .long 218103810                       # 0xd000002
+; CHECK-NEXT:   .long 0
+; CHECK-NEXT:   .long 0
+; CHECK-NEXT:   .long 4
+; CHECK-NEXT:   .long 0
+; CHECK-NEXT:   .long 4
+; CHECK-NEXT:   .long 71                              # BTF_KIND_INT(id = 4)
+; CHECK-NEXT:   .long 16777216                        # 0x1000000
+; CHECK-NEXT:   .long 4
+; CHECK-NEXT:   .long 16777248                        # 0x1000020
+; CHECK-NEXT:   .long 75                              # BTF_KIND_FUNC(id = 5)
+; CHECK-NEXT:   .long 201326594                       # 0xc000002
+; CHECK-NEXT:   .long 3
+; CHECK-NEXT:   .long 79                              # BTF_KIND_DECL_TAG(id = 6)
+; CHECK-NEXT:   .long 285212672                       # 0x11000000
+; CHECK-NEXT:   .long 5
+; CHECK-NEXT:   .long 0
+; CHECK-NEXT:   .long 85                              # BTF_KIND_DECL_TAG(id = 7)
+; CHECK-NEXT:   .long 285212672                       # 0x11000000
+; CHECK-NEXT:   .long 5
+; CHECK-NEXT:   .long 1
+; CHECK-NEXT:   .long 91                              # BTF_KIND_DECL_TAG(id = 8)
+; CHECK-NEXT:   .long 285212672                       # 0x11000000
+; CHECK-NEXT:   .long 5
+; CHECK-NEXT:   .long 4294967295
+
+; CHECK:  .ascii  "x_tag"                         # string offset=79
+; CHECK:  .ascii  "y_tag"                         # string offset=85
+; CHECK:  .ascii  "foo_tag"                       # string offset=91