[HWASan] Add basic stack tagging support for LAM.

[morehouse]

Summary

Adds the basic instrumentation needed for stack tagging.

Currently does not support stack short granules or TLS stack histories,
since a different code path is followed for the callback instrumentation
we use.

We may simply wait to support these two features until we switch to
a custom calling convention.

[morehouse]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

Can we avoid creating ClUntagPointer for now? I am able to test locally with QEMU, and I'm also setting up a buildbot to ensure new patches don't break the LAM functionality.

192

Can we wire ClUsePageAlias up to fsanitize-hwaddress-experimental-aliasing in this patch? Then we can make ClUsePageAlias default to false and can remove this TODO.

193–194

268

Since getTargetTagShift and getTargetTagMask are used only during initializeModule, and since they are both tiny functions, let's remove them and inline the logic in initializeModule.

270

Since this is no longer constexpr, let's remove the k prefix.

499–504

For now we want to keep InstrumentWithCalls == true for x86 in general, even in LAM mode (no aliases). Please update this line.

557–559

We also want to avoid instrumenting globals for now, so please update this line to apply to x86 in general.

715

Please keep this check here for now. I want to avoid reintroducing untagging to x86.

981

The function name is confusing. Can we name it something like applyTagMask?

1016

This mask of the base tag seems unnecessary since we also mask the final tag.

1047

It seems like overkill to move the logic into this function. Let's leave it as part of tagPointer.

1054

Let's avoid implementing the kernel case for LAM, since we don't have a good way to test yet.

1528

Let's avoid doing this for now. I'd like to focus just on stack support in this patch. Future patches can address global support.

llvm/test/Instrumentation/HWAddressSanitizer/X86/stack.ll

5 ↗

(On Diff #346931)

Instead of making a new test, can we reuse the alloca tests for aarch64 and update the expected instrumentation?

[xiangzhangllvm]

Hello @morehouse, thanks very much for your careful review, I'll fix all your comments, just for "ClUntagPointer", I feel it very useful in my develop.

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

Yes, of course, we can. But here ClUntagPointer is disabled in default (false), no affect our current tests.
I and back this option, because I find It is very helpful for me in developing HWASAN without hardware or simulator supported. I can run most simple test on my local machine.

1016

make sense, it seem will handle at retagTargetTag fucntion.

[morehouse]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

It may be inconvenient for now, but as part of setting up a buildbot I'm also automating the process of setting up the custom QEMU and kernel required to test LAM. If all goes well, it should land later this week or next week. Then you can use that instead of the ClUntagPointer option.

For now, could you keep the ClUntagPointer patch locally for your testing purposes, and remove it from the patch we actually submit?

[xiangzhangllvm]

NO problem, Thanks!

[xiangzhangllvm]

Sorry for late update, a little busy in these 2 month : )

[morehouse]

Thanks for the update. I'll take a closer look tomorrow and try testing locally. You can also use the new buildbot script to test in QEMU (see inline comment below).

@eugenis @vitalybuka If you could take a look as well that would be helpful. I'm not too familiar with the stack retagging instrumentation.

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

Thanks. FYI, the buildbot is now live: https://lab.llvm.org/buildbot/#/builders/169

You can test locally using the buildbot script here. You can comment out the Scudo stuff to avoid testing that.

You'll need a QEMU image located at /b/qemu_image. You can build one using this script.

193–194

This comment wasn't addressed.

llvm/test/Instrumentation/HWAddressSanitizer/X86/stack.ll

5 ↗

(On Diff #346931)

I see this test is gone now, but I don't see any alloca tests copied into the X86 directory. Maybe you forgot to add them in the patch?

[xiangzhangllvm]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

Thanks for your work! let me do a try.

193–194

Oh, yes, I should use "hwasan-experimental-use-page-aliases", thank you very much!

llvm/test/Instrumentation/HWAddressSanitizer/X86/stack.ll

5 ↗

(On Diff #346931)

Did you mean the llvm/test/Instrumentation/HWAddressSanitizer/alloca.ll ?
It already at the parent directory of this test (llvm/test/Instrumentation/HWAddressSanitizer/X86)
So, the check-all can test it.

[morehouse]

llvm/test/Instrumentation/HWAddressSanitizer/X86/stack.ll

5 ↗

(On Diff #346931)

I meant that one and the other tests with alloca in the name. Those are testing instrumentation for aarch64. I want to also check the instrumentation for them on x86, since we have differences between the architectures.

[xiangzhangllvm]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

How can I run it online for this patch ? (it seems jobs in https://lab.llvm.org/buildbot/#/builders/169 are on line )

(For locally test, I can't download the QEMU in my company's server, for network permission and without root authority)

[morehouse]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

Can't run it online unless its been submitted.

Sounds like your setup isn't well suited for working on LAM support. I was planning to work on stack and globals support anyway, so perhaps I will commandeer these patches and work on them myself.

[eugenis]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

967

retagMaskX86

You could also make this non-static and get rid of the second argument. Or apply the mask in retagMask.

1016

It seems more efficient to mask StackTag, and then make sure that all retag constants are within the mask bits. This way you'll only need to apply the mask once per function.

1037

Why not simply replace 0xFF here with TagMaskByte?

[xiangzhangllvm]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

190

OK, no problem, feel free to use this patch, Let co-push the HWASAN work.
I'll continue to refine it (finish you and eugenis's comments).

1037

Because "xor" will let Tag > TagMask again : )

[morehouse]

I've addressed most comments locally; commandeering to prevent wasting Xiang's time addressing the feedback.

Will upload with tests tomorrow.

[morehouse]

• Rename flag as experimental.
• Refactor and simplify code.
• Apply mask to base tag only.
• Enable some compiler-rt stack tests on x86.
• Add IR tests.

[morehouse]

• Privatize new member variables.

[vitalybuka]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

504

As is it's not technically correct

518

maybe move up and combine with other IsX86 checks about

960

const

[morehouse]

• - Address nits.

[morehouse]

• Fix clang test failure on Windows.

clang/lib/Driver/SanitizerArgs.cpp

if (!TsanFuncEntryExit) {
     CmdArgs.push_back("-mllvm");
     CmdArgs.push_back("-tsan-instrument-func-entry-exit=0");
   }
   if (!TsanAtomics) {
     CmdArgs.push_back("-mllvm");
     CmdArgs.push_back("-tsan-instrument-atomics=0");
   }
 
+  if (HwasanUseAliases) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-hwasan-use-page-alias=1");
+  }
+
   if (CfiCrossDso)
     CmdArgs.push_back("-fsanitize-cfi-cross-dso");
 
   if (CfiICallGeneralizePointers)
     CmdArgs.push_back("-fsanitize-cfi-icall-generalize-pointers");
 
   if (CfiCanonicalJumpTables)
     CmdArgs.push_back("-fsanitize-cfi-canonical-jump-tables");

clang/test/Driver/fsanitize.c

 // CHECK-DIVBYZERO-NORECOVER-NOT: "-fsanitize-recover=float-divide-by-zero"
 // CHECK-DIVBYZERO-TRAP: "-fsanitize-trap=float-divide-by-zero"
 
 // RUN: %clang -fsanitize=float-divide-by-zero -fsanitize-minimal-runtime %s -### 2>&1 | FileCheck %s --check-prefixes=CHECK-DIVBYZERO,CHECK-DIVBYZERO-MINIMAL
 // CHECK-DIVBYZERO-MINIMAL: "-fsanitize-minimal-runtime"
 
 // RUN: %clang -fsanitize=undefined,float-divide-by-zero %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-DIVBYZERO-UBSAN
 // CHECK-DIVBYZERO-UBSAN: "-fsanitize={{.*}},float-divide-by-zero,{{.*}}"
+
+// RUN: %clang -fsanitize=hwaddress -fsanitize-hwaddress-experimental-aliasing %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-HWASAN-ALIAS
+// CHECK-HWASAN-ALIAS: "-mllvm" "-hwasan-use-page-alias=1"

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

const char kHwasanShadowMemoryDynamicAddress[] =
     "__hwasan_shadow_memory_dynamic_address";
 
 // Accesses sizes are powers of two: 1, 2, 4, 8, 16.
 static const size_t kNumberOfAccessSizes = 5;
 
 static const size_t kDefaultShadowScale = 4;
 static const uint64_t kDynamicShadowSentinel =
     std::numeric_limits<uint64_t>::max();
-static const unsigned kPointerTagShift = 56;
 
 static const unsigned kShadowBaseAlignment = 32;
 
 static cl::opt<std::string>
     ClMemoryAccessCallbackPrefix("hwasan-memory-access-callback-prefix",
                                  cl::desc("Prefix for memory access callbacks"),
                                  cl::Hidden, cl::init("__hwasan_"));
 
static cl::opt<bool> ClInstrumentPersonalityFunctions(
     "hwasan-instrument-personality-functions",
     cl::desc("instrument personality functions"), cl::Hidden, cl::init(false),
     cl::ZeroOrMore);
 
 static cl::opt<bool> ClInlineAllChecks("hwasan-inline-all-checks",
                                        cl::desc("inline all checks"),
                                        cl::Hidden, cl::init(false));
 
+// Clang option "-fsanitize-hwaddress-experimental-aliasing" will enable it.
+static cl::opt<bool> ClUsePageAlias("hwasan-use-page-alias",
+                                    cl::desc("Use page aliasing in HWASan"),
+                                    cl::Hidden, cl::init(false));
+
 namespace {
 
 /// An instrumentation pass implementing detection of addressability bugs
 /// using tagged pointers.
 class HWAddressSanitizer {
 public:
   explicit HWAddressSanitizer(Module &M, bool CompileKernel = false,
                               bool Recover = false)
       : M(M) {
     this->Recover = ClRecover.getNumOccurrences() > 0 ? ClRecover : Recover;
public:
   void instrumentMemIntrinsic(MemIntrinsic *MI);
   bool instrumentMemAccess(InterestingMemoryOperand &O);
   bool ignoreAccess(Value *Ptr);
   void getInterestingMemoryOperands(
       Instruction *I, SmallVectorImpl<InterestingMemoryOperand> &Interesting);
 
   bool isInterestingAlloca(const AllocaInst &AI);
   bool tagAlloca(IRBuilder<> &IRB, AllocaInst *AI, Value *Tag, size_t Size);
+  Value *getTargetTagPtr(IRBuilder<> &IRB, Value *PtrLong, Value *Tag);
   Value *tagPointer(IRBuilder<> &IRB, Type *Ty, Value *PtrLong, Value *Tag);
   Value *untagPointer(IRBuilder<> &IRB, Value *PtrLong);
   bool instrumentStack(
       SmallVectorImpl<AllocaInst *> &Allocas,
       DenseMap<AllocaInst *, std::vector<DbgVariableIntrinsic *>> &AllocaDbgMap,
       SmallVectorImpl<Instruction *> &RetVec, Value *StackTag);
   Value *readRegister(IRBuilder<> &IRB, StringRef Name);
   bool instrumentLandingPads(SmallVectorImpl<Instruction *> &RetVec);
   Value *getNextTagWithCall(IRBuilder<> &IRB);
   Value *getStackBaseTag(IRBuilder<> &IRB);
   Value *getAllocaTag(IRBuilder<> &IRB, Value *StackTag, AllocaInst *AI,
                       unsigned AllocaNo);
   Value *getUARTag(IRBuilder<> &IRB, Value *StackTag);
 
   Value *getHwasanThreadSlotPtr(IRBuilder<> &IRB, Type *Ty);
+  Value *applyTagMask(IRBuilder<> &IRB, Value *OldTag);
   void emitPrologue(IRBuilder<> &IRB, bool WithFrameRecord);
 
   void instrumentGlobal(GlobalVariable *GV, uint8_t Tag);
   void instrumentGlobals();
 
   void instrumentPersonalityFunctions();
 
+  unsigned PointerTagShift;
+  uint64_t TagMaskByte;
+
 private:
   LLVMContext *C;
   Module &M;
   Triple TargetTriple;
   FunctionCallee HWAsanMemmove, HWAsanMemcpy, HWAsanMemset;
   FunctionCallee HWAsanHandleVfork;
 
   /// This struct defines the shadow mapping using the rule:
   ///   shadow = (mem >> Scale) + Offset.
   /// If InGlobal is true, then
   ///   extern char __hwasan_shadow[];
   ///   shadow = (mem >> Scale) + &__hwasan_shadow
   /// If InTls is true, then
   ///   extern char *__hwasan_tls;
   ///   shadow = (mem>>Scale) + align_up(__hwasan_shadow, kShadowBaseAlignment)
   struct ShadowMapping {
     int Scale;
     uint64_t Offset;

 void HWAddressSanitizer::initializeModule() {
   LLVM_DEBUG(dbgs() << "Init " << M.getName() << "\n");
   auto &DL = M.getDataLayout();
 
   TargetTriple = Triple(M.getTargetTriple());
 
   // x86_64 uses userspace pointer aliases, currently heap-only with callback
   // instrumentation only.
-  UsePageAliases = TargetTriple.getArch() == Triple::x86_64;
-  InstrumentWithCalls = UsePageAliases ? true : ClInstrumentWithCalls;
+  bool IsX86_64 = TargetTriple.getArch() == Triple::x86_64;
+  UsePageAliases = ClUsePageAlias && IsX86_64;
+  if (IsX86_64)
+    InstrumentWithCalls = true;
+  else
+    InstrumentWithCalls = ClInstrumentWithCalls;
   InstrumentStack = UsePageAliases ? false : ClInstrumentStack;
 
   Mapping.init(TargetTriple, InstrumentWithCalls);
 
   C = &(M.getContext());
   IRBuilder<> IRB(*C);
   IntptrTy = IRB.getIntPtrTy(DL);
   Int8PtrTy = IRB.getInt8PtrTy();
   Int8Ty = IRB.getInt8Ty();
   Int32Ty = IRB.getInt32Ty();
 
   HwasanCtorFunction = nullptr;
 
+  if (IsX86_64) {
+    PointerTagShift = 57;
+    TagMaskByte = 0x3FLL;
+  } else {
+    PointerTagShift = 56;
+    TagMaskByte = 0xFFLL;
+  }
+
   // Older versions of Android do not have the required runtime support for
   // short granules, global or personality function instrumentation. On other
   // platforms we currently require using the latest version of the runtime.
   bool NewRuntime =
       !TargetTriple.isAndroid() || !TargetTriple.isAndroidVersionLT(30);
 
   UseShortGranules =
       ClUseShortGranules.getNumOccurrences() ? ClUseShortGranules : NewRuntime;
void HWAddressSanitizer::initializeModule() {
   InstrumentLandingPads = ClInstrumentLandingPads.getNumOccurrences()
                               ? ClInstrumentLandingPads
                               : !NewRuntime;
 
   if (!CompileKernel) {
     createHwasanCtorComdat();
     bool InstrumentGlobals =
         ClGlobals.getNumOccurrences() ? ClGlobals : NewRuntime;
-    if (InstrumentGlobals && !UsePageAliases)
+
+    // TODO: We need instrumenting globals for x86 in the future.
+    if (InstrumentGlobals && !UsePageAliases && !IsX86_64)
       instrumentGlobals();
 
     bool InstrumentPersonalityFunctions =
         ClInstrumentPersonalityFunctions.getNumOccurrences()
             ? ClInstrumentPersonalityFunctions
             : NewRuntime;
     if (InstrumentPersonalityFunctions)
       instrumentPersonalityFunctions();

 static size_t TypeSizeToSizeIndex(uint32_t TypeSize) {
   size_t Res = countTrailingZeros(TypeSize / 8);
   assert(Res < kNumberOfAccessSizes);
   return Res;
 }
 
 void HWAddressSanitizer::untagPointerOperand(Instruction *I, Value *Addr) {
   if (TargetTriple.isAArch64() || TargetTriple.getArch() == Triple::x86_64)
     return;
 
   IRBuilder<> IRB(I);
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   Value *UntaggedPtr =
       IRB.CreateIntToPtr(untagPointer(IRB, AddrLong), Addr->getType());
   I->setOperand(getPointerOperandIndex(I), UntaggedPtr);
 }
 
IRB.CreateCall(Intrinsic::getDeclaration(
                        M, UseShortGranules
                               ? Intrinsic::hwasan_check_memaccess_shortgranules
                               : Intrinsic::hwasan_check_memaccess),
                    {ShadowBase, Ptr, ConstantInt::get(Int32Ty, AccessInfo)});
     return;
   }
 
   Value *PtrLong = IRB.CreatePointerCast(Ptr, IntptrTy);
-  Value *PtrTag = IRB.CreateTrunc(IRB.CreateLShr(PtrLong, kPointerTagShift),
+  Value *PtrTag = IRB.CreateTrunc(IRB.CreateLShr(PtrLong, PointerTagShift),
                                   IRB.getInt8Ty());
   Value *AddrLong = untagPointer(IRB, PtrLong);
   Value *Shadow = memToShadow(AddrLong, IRB);
   Value *MemTag = IRB.CreateLoad(Int8Ty, Shadow);
   Value *TagMismatch = IRB.CreateICmpNE(PtrTag, MemTag);
 
   if (HasMatchAllTag) {
     Value *TagNotIgnored = IRB.CreateICmpNE(
if (Size != AlignedSize) {
       IRB.CreateStore(JustTag, IRB.CreateConstGEP1_32(
                                    Int8Ty, IRB.CreateBitCast(AI, Int8PtrTy),
                                    AlignedSize - 1));
     }
   }
   return true;
 }
 
-static unsigned RetagMask(unsigned AllocaNo) {
+static unsigned retagMaskArmv8(unsigned AllocaNo) {
   // A list of 8-bit numbers that have at most one run of non-zero bits.
   // x = x ^ (mask << 56) can be encoded as a single armv8 instruction for these
   // masks.
   // The list does not include the value 255, which is used for UAR.
   //
   // Because we are more likely to use earlier elements of this list than later
   // ones, it is sorted in increasing order of probability of collision with a
   // mask allocated (temporally) nearby. The program that generated this list
   // can be found at:
   // https://github.com/google/sanitizers/blob/master/hwaddress-sanitizer/sort_masks.py
   static unsigned FastMasks[] = {0,  128, 64,  192, 32,  96,  224, 112, 240,
                                  48, 16,  120, 248, 56,  24,  8,   124, 252,
                                  60, 28,  12,  4,   126, 254, 62,  30,  14,
                                  6,  2,   127, 63,  31,  15,  7,   3,   1};
   return FastMasks[AllocaNo % (sizeof(FastMasks) / sizeof(FastMasks[0]))];
 }
 
+static unsigned retagX86(unsigned AllocaNo, uint64_t TagMaskByte) {
+  return AllocaNo & TagMaskByte;
+}
+
+static unsigned retagMask(unsigned AllocaNo, Triple TargetTriple,
+                          uint64_t TagMaskByte) {
+  if (TargetTriple.getArch() == Triple::x86_64)
+    return retagX86(AllocaNo, TagMaskByte);
+  return retagMaskArmv8(AllocaNo);
+}
+
+// The whole pass imply the Tag is 8-bits size, because it is first
+// implemented by aarch64 whose tag is happen 1 byte. We should diff
+// it frome other targets now.
+Value *HWAddressSanitizer::applyTagMask(IRBuilder<> &IRB, Value *OldTag) {
+  if (TargetTriple.getArch() == Triple::x86_64) {
+    Constant *TagMask = ConstantInt::get(IntptrTy, TagMaskByte);
+    Value *NewTag = IRB.CreateAnd(OldTag, TagMask);
+    return NewTag;
+  }
+  return OldTag;
+}
+
 Value *HWAddressSanitizer::getNextTagWithCall(IRBuilder<> &IRB) {
   return IRB.CreateZExt(IRB.CreateCall(HwasanGenerateTagFunc), IntptrTy);
 }
 
 Value *HWAddressSanitizer::getStackBaseTag(IRBuilder<> &IRB) {
   if (ClGenerateTagsWithCalls)
     return getNextTagWithCall(IRB);
   if (StackBaseTag)
Value *HWAddressSanitizer::getStackBaseTag(IRBuilder<> &IRB) {
 
   // Extract some entropy from the stack pointer for the tags.
   // Take bits 20..28 (ASLR entropy) and xor with bits 0..8 (these differ
   // between functions).
   Value *StackPointerLong = IRB.CreatePointerCast(StackPointer, IntptrTy);
   Value *StackTag =
       IRB.CreateXor(StackPointerLong, IRB.CreateLShr(StackPointerLong, 20),
                     "hwasan.stack.base.tag");
+
   return StackTag;
 }
 
 Value *HWAddressSanitizer::getAllocaTag(IRBuilder<> &IRB, Value *StackTag,
                                         AllocaInst *AI, unsigned AllocaNo) {
   if (ClGenerateTagsWithCalls)
     return getNextTagWithCall(IRB);
-  return IRB.CreateXor(StackTag,
-                       ConstantInt::get(IntptrTy, RetagMask(AllocaNo)));
+
+  Value *Tag = IRB.CreateXor(
+      StackTag, ConstantInt::get(
+                    IntptrTy, retagMask(AllocaNo, TargetTriple, TagMaskByte)));
+  return applyTagMask(IRB, Tag);
 }
 
 Value *HWAddressSanitizer::getUARTag(IRBuilder<> &IRB, Value *StackTag) {
   if (ClUARRetagToZero)
     return ConstantInt::get(IntptrTy, 0);
+
   if (ClGenerateTagsWithCalls)
     return getNextTagWithCall(IRB);
-  return IRB.CreateXor(StackTag, ConstantInt::get(IntptrTy, 0xFFU));
+
+  Value *Tag = IRB.CreateXor(StackTag, ConstantInt::get(IntptrTy, 0xFFU));
+  return applyTagMask(IRB, Tag);
 }
 
+Value *HWAddressSanitizer::getTargetTagPtr(IRBuilder<> &IRB, Value *PtrLong,
+                                           Value *Tag) {}
+
 // Add a tag to an address.
 Value *HWAddressSanitizer::tagPointer(IRBuilder<> &IRB, Type *Ty,
                                       Value *PtrLong, Value *Tag) {
   assert(!UsePageAliases);
   Value *TaggedPtrLong;
+  Value *ShiftedTag = IRB.CreateShl(Tag, PointerTagShift);
+  // Kernel addresses have 0xFF in the most significant byte.
   if (CompileKernel) {
+    // TODO: Currently, X86_64 don't tag in kernel state.
+    if (TargetTriple.getArch() == Triple::x86_64) {
+      TaggedPtrLong = PtrLong;
+    } else {
       // Kernel addresses have 0xFF in the most significant byte.
-    Value *ShiftedTag = IRB.CreateOr(
-        IRB.CreateShl(Tag, kPointerTagShift),
-        ConstantInt::get(IntptrTy, (1ULL << kPointerTagShift) - 1));
+      ShiftedTag = IRB.CreateOr(
+          ShiftedTag,
+          ConstantInt::get(IntptrTy, (1ULL << PointerTagShift) - 1));
       TaggedPtrLong = IRB.CreateAnd(PtrLong, ShiftedTag);
+    }
   } else {
-    // Userspace can simply do OR (tag << 56);
-    Value *ShiftedTag = IRB.CreateShl(Tag, kPointerTagShift);
+    // Userspace can simply do OR (tag << PointerTagShift);
     TaggedPtrLong = IRB.CreateOr(PtrLong, ShiftedTag);
   }
   return IRB.CreateIntToPtr(TaggedPtrLong, Ty);
 }
 
 // Remove tag from an address.
 Value *HWAddressSanitizer::untagPointer(IRBuilder<> &IRB, Value *PtrLong) {
   assert(!UsePageAliases);
   Value *UntaggedPtrLong;
   if (CompileKernel) {
     // Kernel addresses have 0xFF in the most significant byte.
     UntaggedPtrLong =
         IRB.CreateOr(PtrLong, ConstantInt::get(PtrLong->getType(),
-                                               0xFFULL << kPointerTagShift));
+                                               0xFFULL << PointerTagShift));
   } else {
     // Userspace addresses have 0x00.
-    UntaggedPtrLong = IRB.CreateAnd(
-        PtrLong,
-        ConstantInt::get(PtrLong->getType(), ~(0xFFULL << kPointerTagShift)));
+    UntaggedPtrLong =
+        IRB.CreateAnd(PtrLong, ConstantInt::get(PtrLong->getType(),
+                                                ~(0xFFULL << PointerTagShift)));
   }
   return UntaggedPtrLong;
 }
 
 Value *HWAddressSanitizer::getHwasanThreadSlotPtr(IRBuilder<> &IRB, Type *Ty) {
   Module *M = IRB.GetInsertBlock()->getParent()->getParent();
   if (TargetTriple.isAArch64() && TargetTriple.isAndroid()) {
     // Android provides a fixed TLS slot for sanitizers. See TLS_SLOT_SANITIZER
for (unsigned N = 0; N < Allocas.size(); ++N) {
 
     AI->replaceUsesWithIf(Replacement,
                           [AILong](Use &U) { return U.getUser() != AILong; });
 
     for (auto *DDI : AllocaDbgMap.lookup(AI)) {
       // Prepend "tag_offset, N" to the dwarf expression.
       // Tag offset logically applies to the alloca pointer, and it makes sense
       // to put it at the beginning of the expression.
-      SmallVector<uint64_t, 8> NewOps = {dwarf::DW_OP_LLVM_tag_offset,
-                                         RetagMask(N)};
+      SmallVector<uint64_t, 8> NewOps = {
+          dwarf::DW_OP_LLVM_tag_offset,
+          retagMask(N, TargetTriple, TagMaskByte)};
       auto Locations = DDI->location_ops();
       unsigned LocNo = std::distance(Locations.begin(), find(Locations, AI));
       DDI->setExpression(
           DIExpression::appendOpsToArg(DDI->getExpression(), NewOps, LocNo));
     }
 
     size_t Size = getAllocaSizeInBytes(*AI);
     tagAlloca(IRB, AI, Tag, Size);
for (uint64_t DescriptorPos = 0; DescriptorPos < SizeInBytes;
     Descriptor->setMetadata(LLVMContext::MD_associated,
                             MDNode::get(*C, ValueAsMetadata::get(NewGV)));
     appendToCompilerUsed(M, Descriptor);
   }
 
   Constant *Aliasee = ConstantExpr::getIntToPtr(
       ConstantExpr::getAdd(
           ConstantExpr::getPtrToInt(NewGV, Int64Ty),
-          ConstantInt::get(Int64Ty, uint64_t(Tag) << kPointerTagShift)),
+          ConstantInt::get(Int64Ty, uint64_t(Tag) << PointerTagShift)),
       GV->getType());
   auto *Alias = GlobalAlias::create(GV->getValueType(), GV->getAddressSpace(),
                                     GV->getLinkage(), "", Aliasee, &M);
   Alias->setVisibility(GV->getVisibility());
   Alias->takeName(GV);
   GV->replaceAllUsesWith(Alias);
   GV->eraseFromParent();
 }
void HWAddressSanitizer::instrumentGlobals() {
   MD5 Hasher;
   Hasher.update(M.getSourceFileName());
   MD5::MD5Result Hash;
   Hasher.final(Hash);
   uint8_t Tag = Hash[0];
 
   for (GlobalVariable *GV : Globals) {
     // Skip tag 0 in order to avoid collisions with untagged memory.
     if (Tag == 0)
       Tag = 1;
     instrumentGlobal(GV, Tag++);
   }
 }
 
 void HWAddressSanitizer::instrumentPersonalityFunctions() {
   // We need to untag stack frames as we unwind past them. That is the job of
   // the personality function wrapper, which either wraps an existing