[dfsan] Support passing non-i16 shadow values in TLS mode

[stephan.yichao.zhao]

Summary

This is a child diff of D92261.

It extended TLS arg/ret to work with aggregate types.

For a function
  t foo(t1 a1, t2 a2, ... tn an)
Its arguments shadow are saved in TLS args like
  a1_s, a2_s, ..., an_s
TLS ret simply includes r_s. By calculating the type size of each shadow
value, we can get their offset.

This is similar to what MSan does. See __msan_retval_tls and __msan_param_tls
from llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp.

Note that this change does not add test cases for overflowed TLS
arg/ret because this is hard to test w/o supporting aggregate shdow
types. We will be adding them after supporting that.

[stephan.yichao.zhao]

update

[morehouse]

llvm/lib/Transforms/Instrumentation/DataFlowSanitizer.cpp

1074

This is a duplicate of the declaration comment. Let's remove this one. (Same for functions below).

1096

Nit: omit curly braces for one-line bodies. (Same elsewhere)

1104

Nit: This comment can fit on the same line as the break. (Same below)

1117

Nit: Base is a confusing name since it sounds like the base pointer to the TLS, not the offset. Let's rename it something else, maybe ArgShadowPtr?

llvm/test/Instrumentation/DataFlowSanitizer/arith.ll

7

Should we verify that align is 8?

8

IIUC, this adds 8 to __dfsan_arg_tls to get b's shadow. This is because we have 8-byte alignment, right?

Why do we need 8 byte alignment?

12

The ST and VT names are a little confusing to me. What do they mean?

llvm/test/Instrumentation/DataFlowSanitizer/select.ll

9

Should we verify offsets in the TLS for each argument?

[stephan.yichao.zhao]

addressed comments

[stephan.yichao.zhao]

llvm/test/Instrumentation/DataFlowSanitizer/arith.ll

8

This flows MSan's setting. https://github.com/llvm/llvm-project/commit/d2bd319adc7cf19d6e47d3caf3b6f508f299ea57 introduced 8-byte alignment for MSan. It sounds like sometimes the origin arg's alignment can be too large. This makes arg_tls run out of space.

12

renamed ST to SHADOWTYPE, and VT to ARGTLSTYPE

[stephan.yichao.zhao]

rebased after submitting D92459

[morehouse]

Looks like load.ll and phi.ll need to be updated.

Also, what is the impact on code size with this change?

[stephan.yichao.zhao]

updated load.ll and phi.ll.

Evaluated code size.

1. for large binaries with original size 2G, the additional overhead is within 0.2%.

2. We also examined a small case.

Loading TLS arguments and storing TLS ret inside a function have the same overhead
between the two approaches.

At callsite, if all parameters are variables, their overheads are also the same.
When any parameter is a constant, the i16-shadow shadow can be of less overhead.

At a callsite this diff stores shadows for the 6 parameters like this:

mov 0x0(%rip),%rax
xor %ecx,%ecx
mov %cx,%fs:(%rax)
mov %cx,%fs:0x8(%rax)
mov %cx,%fs:0x10(%rax)
mov %cx,%fs:0x18(%rax)
mov %cx,%fs:0x20(%rax)
mov %cx,%fs:0x28(%rax)

The i16-shadow stores shadows for the 6 parameters like this:

mov 0x0(%rip),%rax
movq $0x0,%fs:(%rax)
movl $0x0,%fs:0x8(%rax)

Because of the 8-byte alignment codegen cannot zero-out consecutive memory addresses
with fewer instructions. If this turns out to be a common case, we may want to revisit this.

[morehouse]

Why do we need the 8 byte alignment at all? It seems a little silly to have gaps in the TLS shadow.

Currently, when there's 6 params we get the following TLS pattern:

xxxxxx11xxxxxx22xxxxxx33xxxxxx44xxxxxx55xxxxxx66

where each x is an unused byte.

[stephan.yichao.zhao]

Hm, I checked the diff that introduced 8-byte alignment for MSan again.

It seems that this is specific to MSan because MSan's shadow type is the same to its original type: m128i's shadow type is also m128i. If m128i is set with a large alignment,
so does its shadow type. However DFSan's approach always uses i16 for all primitive types: if m128i exists, its shadow type is still i16.

So it is not necessary to follow MSan's. I will update.

[stephan.yichao.zhao]

switched to 2-byte alignment to be consistent with 16bit shadow values.

With this alignment, its binary code about TLS is the same as the code before this diff.
For large binaries with original size > 2G, the additional overhead is -0.1%. So its
overall code size overhead is slightly smaller than the current codegen.

[morehouse]

LGTM

compiler-rt/lib/dfsan/dfsan.cpp

 
 static const uptr kNumLabels = 1 << (sizeof(dfsan_label) * 8);
 
 static atomic_dfsan_label __dfsan_last_label;
 static dfsan_label_info __dfsan_label_info[kNumLabels];
 
 Flags __dfsan::flags_data;
 
-SANITIZER_INTERFACE_ATTRIBUTE THREADLOCAL dfsan_label __dfsan_retval_tls;
-SANITIZER_INTERFACE_ATTRIBUTE THREADLOCAL dfsan_label __dfsan_arg_tls[64];
+// The size of TLS variables. These constants must be kept in sync with the ones
+// in DataFlowSanitizer.cpp.
+static const int kDFsanArgTlsSize = 800;
+static const int kDFsanRetvalTlsSize = 800;
+
+SANITIZER_INTERFACE_ATTRIBUTE THREADLOCAL u64
+    __dfsan_retval_tls[kDFsanRetvalTlsSize / sizeof(u64)];
+SANITIZER_INTERFACE_ATTRIBUTE THREADLOCAL u64
+    __dfsan_arg_tls[kDFsanArgTlsSize / sizeof(u64)];
 
 SANITIZER_INTERFACE_ATTRIBUTE uptr __dfsan_shadow_ptr_mask;
 
 // On Linux/x86_64, memory is laid out as follows:
 //
 // +--------------------+ 0x800000000000 (top of memory)
 // | application memory |
 // +--------------------+ 0x700000008000 (kAppAddr)

llvm/lib/Transforms/Instrumentation/DataFlowSanitizer.cpp

 #include <memory>
 #include <set>
 #include <string>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 
+static const Align kShadowTLSAlignment = Align(8);
+
+// The size of TLS variables. These constants must be kept in sync with the ones
+// in dfsan.cpp.
+static const unsigned kArgTLSSize = 800;
+static const unsigned kRetvalTLSSize = 800;
+
 // External symbol to be used when generating the shadow address for
 // architectures with multiple VMAs. Instead of using a constant integer
 // the runtime will set the external mask based on the VMA range.
-const char kDFSanExternShadowPtrMask[] = "__dfsan_shadow_ptr_mask";
+static const char *const kDFSanExternShadowPtrMask = "__dfsan_shadow_ptr_mask";
 
 // The -dfsan-preserve-alignment flag controls whether this pass assumes that
 // alignment requirements provided by the input IR are correct.  For example,
 // if the input IR contains a load with alignment 8, this flag will cause
 // the shadow load to have alignment 16.  This flag is disabled by default as
 // we have unfortunately encountered too much code (including Clang itself;
 // see PR14291) which performs misaligned access.
 static cl::opt<bool> ClPreserveAlignment(
struct DFSanFunction {
   DFSanFunction(DataFlowSanitizer &DFS, Function *F, bool IsNativeABI)
       : DFS(DFS), F(F), IA(DFS.getInstrumentedABI()), IsNativeABI(IsNativeABI) {
     DT.recalculate(*F);
     // FIXME: Need to track down the register allocator issue which causes poor
     // performance in pathological cases with large numbers of basic blocks.
     AvoidNewBlocks = F->size() > 1000;
   }
 
-  Value *getArgTLS(unsigned Index, Instruction *Pos);
+  /// Computes the shadow address for a given function argument.
+  ///
+  /// Shadow = ArgTLS+ArgOffset.
+  Value *getArgTLS(unsigned ArgOffset, IRBuilder<> &IRB);
+
+  /// Computes the shadow address for a retval.
+  Value *getRetvalTLS(IRBuilder<> &IRB);
+
   Value *getShadow(Value *V);
   void setShadow(Instruction *I, Value *Shadow);
   Value *combineShadows(Value *V1, Value *V2, Instruction *Pos);
   Value *combineOperandShadows(Instruction *Inst);
   Value *loadShadow(Value *ShadowAddr, uint64_t Size, uint64_t Align,
                     Instruction *Pos);
   void storeShadow(Value *Addr, uint64_t Size, Align Alignment, Value *Shadow,
                    Instruction *Pos);
+
+private:
+  /// Returns the shadow value of an argument A.
+  Value *getShadowForTLSArgument(Argument *A);
 };
 
 class DFSanVisitor : public InstVisitor<DFSanVisitor> {
 public:
   DFSanFunction &DFSF;
 
   DFSanVisitor(DFSanFunction &DFSF) : DFSF(DFSF) {}
 
bool DataFlowSanitizer::runImpl(Module &M) {
   if (ABIList.isIn(M, "skip"))
     return false;
 
   const unsigned InitialGlobalSize = M.global_size();
   const unsigned InitialModuleSize = M.size();
 
   bool Changed = false;
 
-  Type *ArgTLSTy = ArrayType::get(ShadowTy, 64);
+  Type *ArgTLSTy = ArrayType::get(Type::getInt64Ty(*Ctx), kArgTLSSize / 8);
   ArgTLS = Mod->getOrInsertGlobal("__dfsan_arg_tls", ArgTLSTy);
   if (GlobalVariable *G = dyn_cast<GlobalVariable>(ArgTLS)) {
     Changed |= G->getThreadLocalMode() != GlobalVariable::InitialExecTLSModel;
     G->setThreadLocalMode(GlobalVariable::InitialExecTLSModel);
   }
-  RetvalTLS = Mod->getOrInsertGlobal("__dfsan_retval_tls", ShadowTy);
+  Type *RetvalTLSTy =
+      ArrayType::get(Type::getInt64Ty(*Ctx), kRetvalTLSSize / 8);
+  RetvalTLS = Mod->getOrInsertGlobal("__dfsan_retval_tls", RetvalTLSTy);
   if (GlobalVariable *G = dyn_cast<GlobalVariable>(RetvalTLS)) {
     Changed |= G->getThreadLocalMode() != GlobalVariable::InitialExecTLSModel;
     G->setThreadLocalMode(GlobalVariable::InitialExecTLSModel);
   }
 
   ExternalShadowMask =
       Mod->getOrInsertGlobal(kDFSanExternShadowPtrMask, IntptrTy);
 
if (ClDebugNonzeroLabels) {
       }
     }
   }
 
   return Changed || !FnsToInstrument.empty() ||
          M.global_size() != InitialGlobalSize || M.size() != InitialModuleSize;
 }
 
-Value *DFSanFunction::getArgTLS(unsigned Idx, Instruction *Pos) {
-  IRBuilder<> IRB(Pos);
-  return IRB.CreateConstGEP2_64(ArrayType::get(DFS.ShadowTy, 64), DFS.ArgTLS, 0,
-                                Idx);
+// Computes the shadow address for a given function argument.
+//
+// Shadow = ArgTLS+ArgOffset.
+Value *DFSanFunction::getArgTLS(unsigned ArgOffset, IRBuilder<> &IRB) {
+  Value *Base = IRB.CreatePointerCast(DFS.ArgTLS, DFS.IntptrTy);
+  if (ArgOffset)
+    Base = IRB.CreateAdd(Base, ConstantInt::get(DFS.IntptrTy, ArgOffset));
+  return IRB.CreateIntToPtr(Base, PointerType::get(DFS.ShadowTy, 0), "_dfsarg");
+}
+
+// Computes the shadow address for a retval.
+Value *DFSanFunction::getRetvalTLS(IRBuilder<> &IRB) {
+  return IRB.CreatePointerCast(DFS.RetvalTLS, PointerType::get(DFS.ShadowTy, 0),
+                               "_dfsret");
+}
+
+// Returns the shadow value of an argument A.
+Value *DFSanFunction::getShadowForTLSArgument(Argument *A) {
+  unsigned ArgOffset = 0;
+  const DataLayout &DL = F->getParent()->getDataLayout();
+  for (auto &FArg : F->args()) {
+    if (!FArg.getType()->isSized()) {
+      if (A == &FArg) {
+        break;
+      }
+      continue;
+    }
+
+    unsigned Size = DL.getTypeAllocSize(DFS.ShadowTy);
+    if (A != &FArg) {
+      ArgOffset += alignTo(Size, kShadowTLSAlignment);
+      if (ArgOffset > kArgTLSSize) {
+        // ArgTLS overflows, uses a zero shadow.
+        break;
+      }
+      continue;
+    }
+
+    if (ArgOffset + Size > kArgTLSSize) {
+      // ArgTLS overflows, uses a zero shadow.
+      break;
+    }
+
+    Instruction *ArgTLSPos = &*F->getEntryBlock().begin();
+    IRBuilder<> IRB(ArgTLSPos);
+    Value *Base = getArgTLS(ArgOffset, IRB);
+    return IRB.CreateAlignedLoad(DFS.ShadowTy, Base, kShadowTLSAlignment);
+  }
+
+  return DFS.ZeroShadow;
 }
 
 Value *DFSanFunction::getShadow(Value *V) {
   if (!isa<Argument>(V) && !isa<Instruction>(V))
     return DFS.ZeroShadow;
   Value *&Shadow = ValShadowMap[V];
   if (!Shadow) {
     if (Argument *A = dyn_cast<Argument>(V)) {
       if (IsNativeABI)
         return DFS.ZeroShadow;
       switch (IA) {
       case DataFlowSanitizer::IA_TLS: {
-        Value *ArgTLSPtr = DFS.ArgTLS;
-        Instruction *ArgTLSPos =
-            DFS.ArgTLS ? &*F->getEntryBlock().begin()
-                       : cast<Instruction>(ArgTLSPtr)->getNextNode();
-        IRBuilder<> IRB(ArgTLSPos);
-        Shadow =
-            IRB.CreateLoad(DFS.ShadowTy, getArgTLS(A->getArgNo(), ArgTLSPos));
+        Shadow = getShadowForTLSArgument(A);
         break;
       }
       case DataFlowSanitizer::IA_Args: {
         unsigned ArgIdx = A->getArgNo() + F->arg_size() / 2;
         Function::arg_iterator i = F->arg_begin();
         while (ArgIdx--)
           ++i;
         Shadow = &*i;

 }
 
 void DFSanVisitor::visitReturnInst(ReturnInst &RI) {
   if (!DFSF.IsNativeABI && RI.getReturnValue()) {
     switch (DFSF.IA) {
     case DataFlowSanitizer::IA_TLS: {
       Value *S = DFSF.getShadow(RI.getReturnValue());
       IRBuilder<> IRB(&RI);
-      IRB.CreateStore(S, DFSF.DFS.RetvalTLS);
+      unsigned Size = getDataLayout().getTypeAllocSize(DFSF.DFS.ShadowTy);
+      if (Size <= kRetvalTLSSize) {
+        // If the size overflows, stores nothing. At callsite, oversized return
+        // shadows are set to zero.
+        IRB.CreateAlignedStore(S, DFSF.getRetvalTLS(IRB), kShadowTLSAlignment);
+      }
       break;
     }
     case DataFlowSanitizer::IA_Args: {
       IRBuilder<> IRB(&RI);
       Type *RT = DFSF.F->getFunctionType()->getReturnType();
       Value *InsVal =
           IRB.CreateInsertValue(UndefValue::get(RT), RI.getReturnValue(), 0);
       Value *InsShadow =
case DataFlowSanitizer::WK_Custom:
         return;
       }
       break;
     }
   }
 
   FunctionType *FT = CB.getFunctionType();
   if (DFSF.DFS.getInstrumentedABI() == DataFlowSanitizer::IA_TLS) {
+    unsigned ArgOffset = 0;
+    const DataLayout &DL = getDataLayout();
     for (unsigned i = 0, n = FT->getNumParams(); i != n; ++i) {
-      IRB.CreateStore(DFSF.getShadow(CB.getArgOperand(i)),
-                      DFSF.getArgTLS(i, &CB));
+      unsigned Size = DL.getTypeAllocSize(DFSF.DFS.ShadowTy);
+      // Stop storing if arguments' size overflows. Inside a function, arguments
+      // after overflow have zero shadow values.
+      if (ArgOffset + Size > kArgTLSSize)
+        break;
+      IRB.CreateAlignedStore(DFSF.getShadow(CB.getArgOperand(i)),
+                             DFSF.getArgTLS(ArgOffset, IRB),
+                             kShadowTLSAlignment);
+      ArgOffset += alignTo(Size, kShadowTLSAlignment);
     }
   }
 
   Instruction *Next = nullptr;
   if (!CB.getType()->isVoidTy()) {
     if (InvokeInst *II = dyn_cast<InvokeInst>(&CB)) {
       if (II->getNormalDest()->getSinglePredecessor()) {
         Next = &II->getNormalDest()->front();
       } else {
         BasicBlock *NewBB =
             SplitEdge(II->getParent(), II->getNormalDest(), &DFSF.DT);
         Next = &NewBB->front();
       }
     } else {
       assert(CB.getIterator() != CB.getParent()->end());
       Next = CB.getNextNode();
     }
 
     if (DFSF.DFS.getInstrumentedABI() == DataFlowSanitizer::IA_TLS) {
       IRBuilder<> NextIRB(Next);
-      LoadInst *LI = NextIRB.CreateLoad(DFSF.DFS.ShadowTy, DFSF.DFS.RetvalTLS);
+      const DataLayout &DL = getDataLayout();
+      unsigned Size = DL.getTypeAllocSize(DFSF.DFS.ShadowTy);
+      if (Size > kRetvalTLSSize) {
+        // Set overflowed return shadow to be zero.
+        DFSF.setShadow(&CB, DFSF.DFS.ZeroShadow);
+      } else {
+        LoadInst *LI = NextIRB.CreateAlignedLoad(
+            DFSF.DFS.ShadowTy, DFSF.getRetvalTLS(NextIRB), kShadowTLSAlignment,
+            "_dfsret");
         DFSF.SkipInsts.insert(LI);
         DFSF.setShadow(&CB, LI);
         DFSF.NonZeroChecks.push_back(LI);
       }
     }
+  }
 
   // Do all instrumentation for IA_Args down here to defer tampering with the
   // CFG in a way that SplitEdge may be able to detect.
   if (DFSF.DFS.getInstrumentedABI() == DataFlowSanitizer::IA_Args) {
     FunctionType *NewFT = DFSF.DFS.getArgsFunctionType(FT);
     Value *Func =
         IRB.CreateBitCast(CB.getCalledOperand(), PointerType::getUnqual(NewFT));
     std::vector<Value *> Args;

llvm/test/Instrumentation/DataFlowSanitizer/arith.ll

 ; RUN: opt < %s -dfsan -S | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define i8 @add(i8 %a, i8 %b) {
   ; CHECK: @"dfs$add"
-  ; CHECK-DAG: %[[ALABEL:.*]] = load{{.*}}__dfsan_arg_tls, i64 0, i64 0
-  ; CHECK-DAG: %[[BLABEL:.*]] = load{{.*}}__dfsan_arg_tls, i64 0, i64 1
-  ; CHECK: %[[UNION:.*]] = call{{.*}}__dfsan_union(i16 zeroext %[[ALABEL]], i16 zeroext %[[BLABEL]])
-  ; CHECK: %[[ADDLABEL:.*]] = phi i16 [ %[[UNION]], {{.*}} ], [ %[[ALABEL]], {{.*}} ]
+  ; CHECK-DAG: %[[ALABEL:.*]] = load [[ST:.*]], [[ST]]* bitcast ([[VT:\[.*\]]]* @__dfsan_arg_tls to [[ST]]*), align [[ALIGN:.*]]
+  ; CHECK-DAG: %[[BLABEL:.*]] = load [[ST]], [[ST]]* inttoptr (i64 add (i64 ptrtoint ([[VT]]* @__dfsan_arg_tls to i64), i64 8) to [[ST]]*), align [[ALIGN]]
+  ; CHECK: %[[UNION:.*]] = call zeroext [[ST]] @__dfsan_union([[ST]] zeroext %[[ALABEL]], [[ST]] zeroext %[[BLABEL]])
+  ; CHECK: %[[ADDLABEL:.*]] = phi [[ST]] [ %[[UNION]], {{.*}} ], [ %[[ALABEL]], {{.*}} ]
   ; CHECK: add i8
-  ; CHECK: store i16 %[[ADDLABEL]], i16* @__dfsan_retval_tls
+  ; CHECK: store [[ST]] %[[ADDLABEL]], [[ST]]* bitcast ([[VT]]* @__dfsan_retval_tls to [[ST]]*), align [[ALIGN]]
   ; CHECK: ret i8
   %c = add i8 %a, %b
   ret i8 %c
 }
 
 define i8 @sub(i8 %a, i8 %b) {
   ; CHECK: @"dfs$sub"
   ; CHECK: load{{.*}}__dfsan_arg_tls

llvm/test/Instrumentation/DataFlowSanitizer/call.ll

 ; RUN: opt < %s -dfsan -S | FileCheck %s
 ; RUN: opt < %s -passes=dfsan -S | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; CHECK-LABEL: @__dfsan_arg_tls
-; CHECK: = external thread_local(initialexec) global [64 x i16]
+; CHECK: = external thread_local(initialexec) global [100 x i64]
 
 ; CHECK-LABEL: @__dfsan_retval_tls
-; CHECK: = external thread_local(initialexec) global i16
+; CHECK: = external thread_local(initialexec) global [100 x i64]
 
 declare i32 @f(i32)
 declare float @llvm.sqrt.f32(float)
 
 ; CHECK-LABEL: @"dfs$call"
 define i32 @call() {
   ; CHECK: store{{.*}}__dfsan_arg_tls
   ; CHECK: call{{.*}}@"dfs$f"

llvm/test/Instrumentation/DataFlowSanitizer/callback.ll

define void @store8(i8* %p, i8 %a) {
   
   store i8 %a, i8* %p
   ret void
 }
 
 define i1 @cmp(i8 %a, i8 %b) {
   ; CHECK: call void @__dfsan_cmp_callback(i16 %[[l:.*]])
   ; CHECK: %c = icmp ne i8 %a, %b
-  ; CHECK: store i16 %[[l]], i16* @__dfsan_retval_tls
+  ; CHECK: store i16 %[[l]], i16* bitcast ({{.*}}* @__dfsan_retval_tls to i16*)
   
   %c = icmp ne i8 %a, %b
   ret i1 %c
 }
 No newline at end of file

llvm/test/Instrumentation/DataFlowSanitizer/fast16labels.ll

 ; Test that -dfsan-fast-16-labels mode uses inline ORs rather than calling
 ; __dfsan_union or __dfsan_union_load.
 ; RUN: opt < %s -dfsan -dfsan-fast-16-labels -S | FileCheck %s --implicit-check-not="call{{.*}}__dfsan_union"
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define i8 @add(i8 %a, i8 %b) {
   ; CHECK-LABEL: define i8 @"dfs$add"
-  ; CHECK-DAG: %[[ALABEL:.*]] = load{{.*}}__dfsan_arg_tls, i64 0, i64 0
-  ; CHECK-DAG: %[[BLABEL:.*]] = load{{.*}}__dfsan_arg_tls, i64 0, i64 1
+  ; CHECK-DAG: %[[ALABEL:.*]] = load [[ST:.*]], [[ST]]* bitcast ([[VT:\[.*\]]]* @__dfsan_arg_tls to [[ST]]*), align [[ALIGN:.*]]
+  ; CHECK-DAG: %[[BLABEL:.*]] = load [[ST]], [[ST]]* inttoptr (i64 add (i64 ptrtoint ([[VT]]* @__dfsan_arg_tls to i64), i64 8) to [[ST]]*), align [[ALIGN]]
   ; CHECK: %[[ADDLABEL:.*]] = or i16 %[[ALABEL]], %[[BLABEL]]
   ; CHECK: add i8
-  ; CHECK: store i16 %[[ADDLABEL]], i16* @__dfsan_retval_tls
+  ; CHECK: store [[ST]] %[[ADDLABEL]], [[ST]]* bitcast ([[VT]]* @__dfsan_retval_tls to [[ST]]*), align [[ALIGN]]
   ; CHECK: ret i8
   %c = add i8 %a, %b
   ret i8 %c
 }
 
 define i8 @load8(i8* %p) {
   ; CHECK-LABEL: define i8 @"dfs$load8"
   ; CHECK: load i16, i16*

llvm/test/Instrumentation/DataFlowSanitizer/select.ll

 ; RUN: opt < %s -dfsan -dfsan-track-select-control-flow=1 -S | FileCheck %s --check-prefix=TRACK_CONTROL_FLOW
 ; RUN: opt < %s -dfsan -dfsan-track-select-control-flow=0 -S | FileCheck %s --check-prefix=NO_TRACK_CONTROL_FLOW
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define i8 @select8(i1 %c, i8 %t, i8 %f) {
-  ; TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 2)
-  ; TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; TRACK_CONTROL_FLOW: %3 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; TRACK_CONTROL_FLOW: %3 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; TRACK_CONTROL_FLOW: %4 = select i1 %c, i16 %2, i16 %1
   ; TRACK_CONTROL_FLOW: %5 = icmp ne i16 %3, %4
   ; TRACK_CONTROL_FLOW: %7 = call {{.*}} i16 @__dfsan_union(i16 {{.*}} %3, i16 {{.*}} %4)
   ; TRACK_CONTROL_FLOW: %9 = phi i16 [ %7, {{.*}} ], [ %3, {{.*}} ]
   ; TRACK_CONTROL_FLOW: %a = select i1 %c, i8 %t, i8 %f
-  ; TRACK_CONTROL_FLOW: store i16 %9, i16* @__dfsan_retval_tls
+  ; TRACK_CONTROL_FLOW: store i16 %9, {{.*}}@__dfsan_retval_tls{{.*}}
   ; TRACK_CONTROL_FLOW: ret i8 %a
   
-  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 2)
-  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; NO_TRACK_CONTROL_FLOW: %3 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; NO_TRACK_CONTROL_FLOW: %3 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: %4 = select i1 %c, i16 %2, i16 %1
   ; NO_TRACK_CONTROL_FLOW: %a = select i1 %c, i8 %t, i8 %f
-  ; NO_TRACK_CONTROL_FLOW: store i16 %4, i16* @__dfsan_retval_tls
+  ; NO_TRACK_CONTROL_FLOW: store i16 %4, {{.*}}@__dfsan_retval_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: ret i8 %a
 
   %a = select i1 %c, i8 %t, i8 %f
   ret i8 %a
 }
 
 define i8 @select8e(i1 %c, i8 %tf) {
-  ; TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; TRACK_CONTROL_FLOW: %3 = icmp ne i16 %2, %1
   ; TRACK_CONTROL_FLOW: %5 = call {{.*}} i16 @__dfsan_union(i16 {{.*}} %2, i16 {{.*}} %1)
   ; TRACK_CONTROL_FLOW: %7 = phi i16 [ %5, {{.*}} ], [ %2, {{.*}} ]
   ; TRACK_CONTROL_FLOW: %a = select i1 %c, i8 %tf, i8 %tf
-  ; TRACK_CONTROL_FLOW: store i16 %7, i16* @__dfsan_retval_tls
+  ; TRACK_CONTROL_FLOW: store i16 %7, {{.*}}@__dfsan_retval_tls{{.*}}
   ; TRACK_CONTROL_FLOW: ret i8 %a
   
-  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: %a = select i1 %c, i8 %tf, i8 %tf
-  ; NO_TRACK_CONTROL_FLOW: store i16 %1, i16* @__dfsan_retval_tls
+  ; NO_TRACK_CONTROL_FLOW: store i16 %1, {{.*}}@__dfsan_retval_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: ret i8 %a
 
   %a = select i1 %c, i8 %tf, i8 %tf
   ret i8 %a
 }
 
 define <4 x i8> @select8v(<4 x i1> %c, <4 x i8> %t, <4 x i8> %f) {
-  ; TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 2)
-  ; TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; TRACK_CONTROL_FLOW: %3 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; TRACK_CONTROL_FLOW: %3 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; TRACK_CONTROL_FLOW: %4 = icmp ne i16 %2, %1
   ; TRACK_CONTROL_FLOW: %6 = call {{.*}} i16 @__dfsan_union(i16 {{.*}} %2, i16 zeroext %1)
   ; TRACK_CONTROL_FLOW: %8 = phi i16 [ %6, {{.*}} ], [ %2, {{.*}} ]
   ; TRACK_CONTROL_FLOW: %9 = icmp ne i16 %3, %8
   ; TRACK_CONTROL_FLOW: %11 = call {{.*}} i16 @__dfsan_union(i16 {{.*}} %3, i16 zeroext %8)
   ; TRACK_CONTROL_FLOW: %13 = phi i16 [ %11, {{.*}} ], [ %3, {{.*}} ]
   ; TRACK_CONTROL_FLOW: %a = select <4 x i1> %c, <4 x i8> %t, <4 x i8> %f
-  ; TRACK_CONTROL_FLOW: store i16 %13, i16* @__dfsan_retval_tls
+  ; TRACK_CONTROL_FLOW: store i16 %13, {{.*}}@__dfsan_retval_tls{{.*}}
   ; TRACK_CONTROL_FLOW: ret <4 x i8> %a
 
-  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 2)
-  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 1)
-  ; NO_TRACK_CONTROL_FLOW: %3 = load i16, i16* getelementptr inbounds ([64 x i16], [64 x i16]* @__dfsan_arg_tls, i64 0, i64 0)
+  ; NO_TRACK_CONTROL_FLOW: %1 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; NO_TRACK_CONTROL_FLOW: %2 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
+  ; NO_TRACK_CONTROL_FLOW: %3 = load i16, {{.*}}@__dfsan_arg_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: %4 = icmp ne i16 %2, %1
   ; NO_TRACK_CONTROL_FLOW: %6 = call {{.*}} i16 @__dfsan_union(i16 {{.*}} %2, i16 {{.*}} %1)
   ; NO_TRACK_CONTROL_FLOW: %8 = phi i16 [ %6, {{.*}} ], [ %2, {{.*}} ]
   ; NO_TRACK_CONTROL_FLOW: %a = select <4 x i1> %c, <4 x i8> %t, <4 x i8> %f
-  ; NO_TRACK_CONTROL_FLOW: store i16 %8, i16* @__dfsan_retval_tls
+  ; NO_TRACK_CONTROL_FLOW: store i16 %8, {{.*}}@__dfsan_retval_tls{{.*}}
   ; NO_TRACK_CONTROL_FLOW: ret <4 x i8> %a
 
   %a = select <4 x i1> %c, <4 x i8> %t, <4 x i8> %f
   ret <4 x i8> %a
 }
 No newline at end of file