[Assignment Tracking][7/*] Add assignment tracking functionality to clang

[Orlando]

Summary

The Assignment Tracking debug-info feature is outlined in this RFC. This first series of patches adds documentation, the changes necessary to start emitting and using the new metadata, and updates clang with an option to enable the feature. Working with the new metadata in the middle and back end will come later. There are still a few rough edges but I'm putting these patches up now hoping to get feedback on the design and implementation from the upstream community.

---

This patch plumbs the AssignmentTrackingPass (AKA declare-to-assign), added in the previous patch in this set, into the optimisation pipeline from clang. clang/test/CodeGen/assignment-tracking/assignment-tracking.cpp is the main test for this patch.

Note: while clang (with the help of the declare-to-assign pass) can now emit Assignment Tracking metadata, the llvm middle and back ends don't yet understand it.

[jmorse]

This looks good -- however the last two tests running "opt" are probably running too much of LLVM to put into clang. Do these parts really need to be tested from the frontend, or can they be put in llvm?

This returns us to that perennial topic, end-to-end and cross-project tests, which I think for now we'll need to defer as "let's not think about that".

clang/test/CodeGen/assignment-tracking/assignment-tracking.cpp

1–2

Small risk that this is running "too much" of LLVM, and spurious changes will cause clang tests to fail. To me, it feels alright though.

I suspect that if it's in the clang CodeGen dir, then you should be using %clang_cc1 or whatever the macro is, and the cc1 command line to run this test. Same with the other tests.

[Orlando]

In D132226#3772338, @jmorse wrote:

This looks good -- however the last two tests running "opt" are probably running too much of LLVM to put into clang. Do these parts really need to be tested from the frontend, or can they be put in llvm?

I could remove the opt call and keep the tests there? The tests would function as intended still. The verify check just felt like a free win to me, but I hadn't considered the layering of the tools.

This returns us to that perennial topic, end-to-end and cross-project tests, which I think for now we'll need to defer as "let's not think about that".

SGTM

clang/test/CodeGen/assignment-tracking/assignment-tracking.cpp

1–2

My thinking was that source code tests add extra comfort and surface area to catch problems if (/when) we change "how dbg.assign / DIAssignID are generated".

I suspect that if it's in the clang CodeGen dir, then you should be using %clang_cc1 or whatever the macro i

SGTM, will do.

[jmorse]

(Hitting the big red button to return this to your "revision" queue).

[Orlando]

+ use %clang_cc1 substitution in tests (and add test assignment-tracking-opts.c which uses %clang)

[jmorse]

Hmmmm. I think -O0 is also running a reasonable amount of llvm. On the other hand, if it turns out to be a noisy test then we can come back to it at a later date -- we've done our best to ensure there's a small surface that can change, and the check lines are mostly for metadata rather than anything else. LGTM

clang/lib/CodeGen/BackendUtil.cpp

 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/BitcodeReader.h"
 #include "llvm/Bitcode/BitcodeWriter.h"
 #include "llvm/Bitcode/BitcodeWriterPass.h"
 #include "llvm/CodeGen/RegAllocRegistry.h"
 #include "llvm/CodeGen/SchedulerRegistry.h"
 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/IR/DataLayout.h"
+#include "llvm/IR/DebugInfo.h"
 #include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/LTO/LTOBackend.h"
 #include "llvm/MC/MCAsmInfo.h"
void EmitAssemblyHelper::RunOptimizationPipeline(
   PrintPassOpts.Indent = DebugPassStructure;
   PrintPassOpts.SkipAnalyses = DebugPassStructure;
   StandardInstrumentations SI(CodeGenOpts.DebugPassManager ||
                                   DebugPassStructure,
                               /*VerifyEach*/ false, PrintPassOpts);
   SI.registerCallbacks(PIC, &FAM);
   PassBuilder PB(TM.get(), PTO, PGOOpt, &PIC);
 
+  if (CodeGenOpts.EnableAssignmentTracking) {
+    PB.registerPipelineStartEPCallback(
+        [&](ModulePassManager &MPM, OptimizationLevel Level) {
+          MPM.addPass(AssignmentTrackingPass());
+        });
+  }
+
   // Attempt to load pass plugins and register their callbacks with PB.
   for (auto &PluginFN : CodeGenOpts.PassPlugins) {
     auto PassPlugin = PassPlugin::Load(PluginFN);
     if (PassPlugin) {
       PassPlugin->registerPassBuilderCallbacks(PB);
     } else {
       Diags.Report(diag::err_fe_unable_to_load_plugin)
           << PluginFN << toString(PassPlugin.takeError());

clang/test/CodeGen/assignment-tracking/assignment-tracking.cpp

+// RUN: %clang -g -O0 -emit-llvm -S -Xclang -fexperimental-assignment-tracking %s -o - \
+// RUN: | FileCheck %s --implicit-check-not="call void @llvm.dbg"
+
+// Based on llvm/test/DebugInfo/Generic/track-assignments.ll - check that using
+// -Xclang -fexperimental-assignment-tracking results in emitting (or, as it is
+// set up currently, telling llvm to create) assignment tracking metadata.
+//
+// See the original test for more info.
+
+struct Inner { int A, B; };
+struct Outer { Inner A, B; };
+struct Large { int A[10]; };
+struct LCopyCtor { int A[4]; LCopyCtor(); LCopyCtor(LCopyCtor const &); };
+int Value, Index, Cond;
+Inner InnerA, InnerB;
+Large L;
+
+void zeroInit() { int Z[3] = {0, 0, 0}; }
+// CHECK-LABEL: define dso_local void @_Z8zeroInitv
+// CHECK:       %Z = alloca [3 x i32], align 4, !DIAssignID ![[ID_0:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_0:[0-9]+]], metadata !DIExpression(), metadata ![[ID_0]], metadata [3 x i32]* %Z, metadata !DIExpression())
+// CHECK:        @llvm.memset.p0i8.i64{{.*}}, !DIAssignID ![[ID_1:[0-9]+]]
+// CHECK-NEXT:   call void @llvm.dbg.assign(metadata i8 0, metadata ![[VAR_0]], metadata !DIExpression(), metadata ![[ID_1]], metadata i8* %0, metadata !DIExpression())
+
+void memcpyInit() { int A[4] = {0, 1, 2, 3}; }
+// CHECK-LABEL: define dso_local void @_Z10memcpyInitv
+// CHECK:       %A = alloca [4 x i32], align 16, !DIAssignID ![[ID_2:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_1:[0-9]+]], metadata !DIExpression(), metadata ![[ID_2]], metadata [4 x i32]* %A, metadata !DIExpression())
+// CHECK:        @llvm.memcpy.p0i8.p0i8.i64{{.*}}, !DIAssignID ![[ID_3:[0-9]+]]
+// CHECK-NEXT:   call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_1]], metadata !DIExpression(), metadata ![[ID_3]], metadata i8* %0, metadata !DIExpression())
+  
+void setField() {
+  Outer O;
+  O.A.B = Value;
+}
+// CHECK-LABEL: define dso_local void @_Z8setFieldv
+// CHECK:       %O = alloca %struct.Outer, align 4, !DIAssignID ![[ID_4:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_2:[0-9]+]], metadata !DIExpression(), metadata ![[ID_4]], metadata %struct.Outer* %O, metadata !DIExpression())
+// CHECK:       store i32 %0, i32* %B, align 4,{{.*}}!DIAssignID ![[ID_5:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i32 %0, metadata ![[VAR_2]], metadata !DIExpression(DW_OP_LLVM_fragment, 32, 32), metadata ![[ID_5]], metadata i32* %B, metadata !DIExpression())
+
+void unknownOffset() {
+  int A[2];
+  A[Index] = Value;
+}
+// CHECK-LABEL: define dso_local void @_Z13unknownOffsetv
+// CHECK:       %A = alloca [2 x i32], align 4, !DIAssignID ![[ID_6:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_3:[0-9]+]], metadata !DIExpression(), metadata ![[ID_6]], metadata [2 x i32]* %A, metadata !DIExpression())
+
+Inner sharedAlloca() {
+  if (Cond) {
+    Inner A = InnerA;
+    return A;
+  } else {
+    Inner B = InnerB;
+    return B;
+  }
+}
+// CHECK-LABEL: define dso_local i64 @_Z12sharedAllocav
+// CHECK:       %retval = alloca %struct.Inner, align 4, !DIAssignID ![[ID_7:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_4:[0-9]+]], metadata !DIExpression(), metadata ![[ID_7]], metadata %struct.Inner* %retval, metadata !DIExpression())
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_5:[0-9]+]], metadata !DIExpression(), metadata ![[ID_7]], metadata %struct.Inner* %retval, metadata !DIExpression())
+// CHECK:     if.then:
+// CHECK:       call void @llvm.memcpy{{.*}}, !DIAssignID ![[ID_8:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_4]], metadata !DIExpression(), metadata ![[ID_8]], metadata i8* %1, metadata !DIExpression())
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_5]], metadata !DIExpression(), metadata ![[ID_8]], metadata i8* %1, metadata !DIExpression())
+// CHECK:     if.else:
+// CHECK:       call void @llvm.memcpy{{.*}}, !DIAssignID ![[ID_9:[0-9]+]]
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_4]], metadata !DIExpression(), metadata ![[ID_9]], metadata i8* %2, metadata !DIExpression())
+// CHECK-NEXT:  call void @llvm.dbg.assign(metadata i1 undef, metadata ![[VAR_5]], metadata !DIExpression(), metadata ![[ID_9]], metadata i8* %2, metadata !DIExpression())
+
+Large sret() {
+  Large X = L;
+  return X;
+}
+// CHECK-LABEL: define dso_local void @_Z4sretv
+// CHECK:       llvm.dbg.declare
+
+void byval(Large X) {}
+// CHECK-LABEL: define dso_local void @_Z5byval5Large
+// CHECK:       llvm.dbg.declare
+
+LCopyCtor indirectReturn() {
+  LCopyCtor R;
+  return R;
+}
+// CHECK-LABEL: define dso_local void @_Z14indirectReturnv
+// CHECK:       call void @llvm.dbg.declare
+
+// CHECK-DAG: ![[VAR_0]] = !DILocalVariable(name: "Z",
+// CHECK-DAG: ![[VAR_1]] = !DILocalVariable(name: "A",
+// CHECK-DAG: ![[VAR_2]] = !DILocalVariable(name: "O",
+// CHECK-DAG: ![[VAR_3]] = !DILocalVariable(name: "A",
+// CHECK-DAG: ![[VAR_4]] = !DILocalVariable(name: "B",
+// CHECK-DAG: ![[VAR_5]] = !DILocalVariable(name: "A",

clang/test/CodeGen/assignment-tracking/flag.cpp

+// RUN: %clang %s -Xclang -fexperimental-assignment-tracking -O0 -g -o - -emit-llvm -S \
+// RUN: | FileCheck %s --check-prefixes=FLAG
+// RUN: %clang %s -Xclang -O0 -g -o - -emit-llvm -S \
+// RUN: | FileCheck %s --check-prefixes=NO-FLAG
+
+// Check some assignment-tracking stuff appears in the output when the flag
+// -fexperimental-assignment-tracking is used, and that it doesn't when
+// the flag is not used (default behaviour: no assignment tracking).
+
+// FLAG: DIAssignID
+// FLAG: dbg.assign
+
+// NO-FLAG-NOT: DIAssignID
+// NO-FLAG-NOT: dbg.assign
+
+void fun(int a) {}

clang/test/CodeGen/assignment-tracking/memcpy-fragment.cpp

+// RUN: %clang -c -g -emit-llvm -S -Xclang -fexperimental-assignment-tracking  %s -o - \
+// RUN: | opt -S -verify -o - \
+// RUN: | FileCheck %s
+
+// Check that the (debug) codegen looks right with assignment tracking
+// enabled. Each fragment that is written to should have a dbg.assign that has
+// the DIAssignID of the write as an argument. The fragment offset and size
+// should match the offset into the base storage and size of the store. Each of
+// the scenarios below results in slightly different arguments generated for
+// the memcpy.
+
+#include "string.h"
+#include <cstdint>
+
+// Test write a complete struct field only.
+void fragmentWhole()
+{
+ struct Record {
+   uint32_t num;
+   char ch;
+ };
+
+ Record dest;
+ char src = '\0';
+ memcpy(&dest.ch, &src, sizeof(char));
+}
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 %ch, i8* align 1 %src, i64 1, i1 false), !dbg !{{[0-9]+}}, !DIAssignID ![[memberID:[0-9]+]]
+// CHECK-NEXT: call void @llvm.dbg.assign(metadata{{.*}}undef, metadata !{{[0-9]+}}, metadata !DIExpression(DW_OP_LLVM_fragment, 32, 8), metadata ![[memberID]], metadata i8* %ch, metadata !DIExpression())
+
+// Write starting at a field and overlapping part of another.
+void fragmentWholeToPartial()
+{
+ struct Record {
+   uint32_t num1;
+   uint32_t num2;
+ };
+
+ Record dest;
+ char src[5]="\0\0\0\0";
+ memcpy(&dest.num1, &src, 5);
+}
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 %1, i8* align 1 %2, i64 5, i1 false), !dbg !{{[0-9]+}}, !DIAssignID ![[exceed:[0-9]+]]
+// CHECK-NEXT: call void @llvm.dbg.assign(metadata{{.*}}undef, metadata !{{[0-9]+}}, metadata !DIExpression(DW_OP_LLVM_fragment, 0, 40), metadata ![[exceed]], metadata i8* %1, metadata !DIExpression())
+
+// Write starting between fields.
+void fragmentPartialToWhole()
+{
+ struct record {
+   uint32_t num1;
+   uint32_t num2;
+   uint32_t num3;
+}; 
+
+ record dest;
+ char src[5]="\0\0\0\0";
+ memcpy((char*)&(dest.num2) + 3, &src, 5);
+}
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 1 %add.ptr, i8* align 1 %2, i64 5, i1 false), !dbg !{{[0-9]+}}, !DIAssignID ![[addendID:[0-9]+]]
+// CHECK-NEXT: call void @llvm.dbg.assign(metadata{{.*}}undef, metadata !{{.*}}, metadata !DIExpression(DW_OP_LLVM_fragment, 56, 40), metadata ![[addendID]], metadata i8* %add.ptr, metadata !DIExpression())

clang/test/CodeGen/assignment-tracking/nested-scope.cpp

+// RUN: %clang -g -emit-llvm -S -Xclang -fexperimental-assignment-tracking %s -o - \
+// RUN: | opt -S -verify -o - \
+// RUN: | FileCheck %s
+
+// Check that dbg.assign intrinsics get a !dbg with with the same scope as
+// their variable.
+
+// CHECK: call void @llvm.dbg.assign({{.+}}, metadata [[local:![0-9]+]], {{.+}}, {{.+}}, {{.+}}), !dbg [[dbg:![0-9]+]]
+// CHECK-DAG: [[local]] = !DILocalVariable(name: "local", scope: [[scope:![0-9]+]],
+// CHECK-DAG: [[dbg]] = !DILocation({{.+}}, scope: [[scope]])
+// CHECK-DAG: [[scope]] = distinct !DILexicalBlock
+
+void ext(int*);
+void fun() {
+  {
+    int local;
+  }
+}
+