[HWASAN] Instrument globals

[evgeny777]

Summary

This patch enables access checks for global variables in HWASAN.
The algorithm is fairly simple:

For each global variable (GV) we can instrument

• Replace it with one aligned to 1 << kDefaultShadowScale
• Generate tag based on global name: hash_value(GV->getName())
• Replace all uses of pointer to this variable with tagged pointer, i.e:

i64 *@foo -> i64* bitcast (i8* getelementptr (i8, i8* bitcast (i64* @foo to i8*), i64 N) to i64*)
where N is a tag shifted by 56 bits

Patch adds calls to __hwasan_register_globals and __hwasan_unregister_globals to tag and untag globals memory.

Everything else is handled HWASAN function pass after that.

[eugenis]

hash_value(GV->getName())

This would break interoperability with non-instrumented translation units and DSOs.

AFAIK the proper implementation of global tagging needs to involve the dynamic loader, which can emit tagged pointers into GOTs. The compiler would need to route every access to a global through GOT, too, even for internal / non-preemptible globals.

I wonder if the same could be achieved with a clever use of ifunc?

lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

1275

This "all globals in one" approach breaks -Wl,-gc-sections, which is rather bad for binary size. You need to do the whole comdat + !associated thing, same as ASan.

[eugenis]

@pcc

[evgeny777]

even for internal / non-preemptible globals.

Why so? The only case I can imagine is ThinLTO promotion, but in such case instrumented code is imported as well

I wonder if the same could be achieved with a clever use of ifunc?

Do you mean generating ifunc for each instrumented global and tagging a pointer there?

[eugenis]

In D56672#1357619, @evgeny777 wrote:

even for internal / non-preemptible globals.

Why so? The only case I can imagine is ThinLTO promotion, but in such case instrumented code is imported as well

OK, not internal, but external non-preemptible for sure because they can be accessed directly from outside, and then the only source of truth for the variable's tag would be the GOT entry.

Internal variables can use a different approach, maybe based on a hash of the name (if there is one!) like in this patch.

I wonder if the same could be achieved with a clever use of ifunc?

Do you mean generating ifunc for each instrumented global and tagging a pointer there?

Yes, but I'm having doubts about how well this case (data-ifunc) is supported in the toolchain(s). If at all.

[evgeny777]

This comment was removed by evgeny777.

[evgeny777]

I wonder if the same could be achieved with a clever use of ifunc?

Do you mean generating ifunc for each instrumented global and tagging a pointer there?

I noticed that HWASAN forces compilation of position independent object files (this happens in SanitizerArgs.cpp). This means
that uninstrumented objects/libraries will also likely be position independent, beacuse linking PIE vs non-PIE is problematic

That said we can probably use ordinary ifuncs like it's already done with __hwasan_shadow. The lld linker can't handle this
at the moment because it can't create R_*_IRELATIVE relocs in .rela.dyn section, but GNU linkers can.

Your thoughts?

[evgeny777]

BTW, does it make sense to make this patch work for static variables first?

[pcc]

An alternative idea to ifunc is to compute a tag for each global at compile time and store it in the global's virtual address in the symbol table. Although the tags wouldn't be randomized per run, maybe this would be enough.

You can do this by transforming the globals from:

@foo = global i32 123

to:

@foo.data = private global {i32, [12 x i8]} {i32 123, [12 x i8] zeroinitializer}
@foo = alias inttoptr(add(ptrtoint(@foo.data), 0x4200000000000000))) ; tag is 0x42

To tag the globals at load time, you can create a section of (tagged address, size) pairs. The runtime would tag the range (address, address + size) with the tag (address >> 56). The tagged address could use a 64-bit relative relocation (R_AARCH64_PREL64 on AArch64 or R_X86_64_PC64 on x86_64) to avoid needing the section to be dynamically relocated.

One possible downside is that the tagged virtual addresses in the symbol table could confuse tools (e.g. objdump), but we might be able to live with it.

[evgeny777]

Ok, I finally have some time to work on this again
Changes

• External globals are instrumented via alias creation (suggested by @pcc). Compiler forces access to them via GOT even if they're dso_local
• Internal globals are instrumented by means of GEP pointer tagging (same as before)
• Interposable globals are not instrumented, but marked with special attribute, so they're accessed via GOT
• Hidden globals are not instrumented. They can't be accessed via GOT, because this will make such variables preemptive and not making them preemptive causes relocation overflow at link time
• Pointers to globals are stored in __hwasan_globals section. The pointer to section contents is passed to registration function.

Testing

To test the patch(es) I used RPI3 board with slightly modified Linux kernel (__user pointer untagging in syscalls). I ran two different test suites

• LLVM/clang compiled with -DLLVM_BUILD_LLVM_DYLIB=ON and -DLLVM_LINK_LLVM_DYLIB=ON. Toolchain was deployed on board and lit tests were run. Looks good so far

• LLVM test suite tests. Those triggered a number of TAG mismatch errors, one of them in hbd.test:

// decomp.cpp:
Exp *stack[8];
Exp **stkptr;
// .....
int decompileblock(Classfile *c, method_info_ptr mi) {
  // ...
  stkptr = stack;
  // ... pushunop is called later in this function ...
}

// d6-arith.cpp
int pushunop(Classfile *c) /* push unary operation, popping operand e.g. lneg */
{
  Exp *e1 = *(stkptr-1);
  // ....
}

There are quite a few other read overflows, for example in lencod and ldecod. I haven't investigated them thoroughly, but may be this would make sense if decide to proceed with this patch.

Issues

• Linking with ld.bfd requires -Wl,--no-relax. I haven't investigated this yet (ld.lld works fine).
• Linking uninstrumented code with weak or common symbols with instrumented code containing strong versions of those symbols will not work. This probably can be fixed at least for AArch64 by means of untagging symbol addresses on the linker side.

[evgeny777]

Ping

[eugenis]

sorry for the delay, I'll take a look later this week

[pcc]

lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

1297

Could this use a relative relocation so that we aren't paying the cost of a dynamic relocation for each global?

[evgeny777]

Could this use a relative relocation so that we aren't paying the cost of a dynamic relocation for each global?

@pcc Sorry I missed your comment

I think this is not possible to do for shared objects, because GVs could be preempted. Still this looks possible for executables where I can set
visibility of instrumented globals to hidden. I'll try this out

[pcc]

In D56672#1506944, @evgeny777 wrote:

Could this use a relative relocation so that we aren't paying the cost of a dynamic relocation for each global?

@pcc Sorry I missed your comment

I think this is not possible to do for shared objects, because GVs could be preempted.

I think it should be possible if you make the relocation point to the object itself (which will have local linkage) and not the alias that you're creating. Of course you'll need to add the tag to the address in the same way as when you create the alias.

Still this looks possible for executables where I can set
visibility of instrumented globals to hidden. I'll try this out

[evgeny777]

I think it should be possible if you make the relocation point to the object itself (which will have local linkage) and not the alias that you're creating.

It doesn't seem to work. Dynamic loader doesn't know anything about aliases - alias and aliasee can point to different memory locations if aliasee is preempted.
Also let's say I have external declaration of variable foo. How do I know if foo is instrumented in another TU or not. If not then I think, I can't tag it's address

cmake/modules/HandleLLVMOptions.cmake

 endmacro()
 
 # Turn on sanitizers if necessary.
 if(LLVM_USE_SANITIZER)
   if (LLVM_ON_UNIX)
     if (LLVM_USE_SANITIZER STREQUAL "Address")
       append_common_sanitizer_flags()
       append("-fsanitize=address" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
+    elseif (LLVM_USE_SANITIZER STREQUAL "HWAddress")
+      append_common_sanitizer_flags()
+      append("-fsanitize=hwaddress" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
     elseif (LLVM_USE_SANITIZER MATCHES "Memory(WithOrigins)?")
       append_common_sanitizer_flags()
       append("-fsanitize=memory" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
       if(LLVM_USE_SANITIZER STREQUAL "MemoryWithOrigins")
         append("-fsanitize-memory-track-origins" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
       endif()
     elseif (LLVM_USE_SANITIZER STREQUAL "Undefined")
       append_common_sanitizer_flags()

include/llvm/InitializePasses.h

 void initializeGlobalDCELegacyPassPass(PassRegistry&);
 void initializeGlobalMergePass(PassRegistry&);
 void initializeGlobalOptLegacyPassPass(PassRegistry&);
 void initializeGlobalSplitPass(PassRegistry&);
 void initializeGlobalsAAWrapperPassPass(PassRegistry&);
 void initializeGuardWideningLegacyPassPass(PassRegistry&);
 void initializeHotColdSplittingLegacyPassPass(PassRegistry&);
 void initializeHWAddressSanitizerPass(PassRegistry&);
+void initializeHWAddressSanitizerModulePass(PassRegistry &);
 void initializeIPCPPass(PassRegistry&);
 void initializeIPSCCPLegacyPassPass(PassRegistry&);
 void initializeIRCELegacyPassPass(PassRegistry&);
 void initializeIRTranslatorPass(PassRegistry&);
 void initializeIVUsersWrapperPassPass(PassRegistry&);
 void initializeIfConverterPass(PassRegistry&);
 void initializeImplicitNullChecksPass(PassRegistry&);
 void initializeIndVarSimplifyLegacyPassPass(PassRegistry&);

include/llvm/Transforms/Instrumentation.h

 // this is the context senstive instrumentation.
 ModulePass *createInstrProfilingLegacyPass(
     const InstrProfOptions &Options = InstrProfOptions(), bool IsCS = false);
 
 ModulePass *createInstrOrderFilePass();
 
 FunctionPass *createHWAddressSanitizerPass(bool CompileKernel = false,
                                            bool Recover = false);
+ModulePass *createHWAddressSanitizerModulePass(bool CompileKernel = false);
 
 // Insert DataFlowSanitizer (dynamic data flow analysis) instrumentation
 ModulePass *createDataFlowSanitizerPass(
     const std::vector<std::string> &ABIListFiles = std::vector<std::string>(),
     void *(*getArgTLS)() = nullptr, void *(*getRetValTLS)() = nullptr);
 
 // Options for sanitizer coverage instrumentation.
 struct SanitizerCoverageOptions {

lib/Target/TargetMachine.cpp

static TLSModel::Model getSelectedTLSModel(const GlobalValue *GV) {
   case GlobalVariable::InitialExecTLSModel:
     return TLSModel::InitialExec;
   case GlobalVariable::LocalExecTLSModel:
     return TLSModel::LocalExec;
   }
   llvm_unreachable("invalid TLS model");
 }
 
+static bool isHWAsanInstrumentedGlobal(const GlobalValue *GV) {
+  // With -hwasan-instrument-globals interposable global variables
+  // should always be referenced via GOT, otherwise we may overflow
+  // relocations.
+  if (const GlobalVariable *GVar = dyn_cast<GlobalVariable>(GV))
+    return GVar->hasAttribute("hwasan-interposable-global");
+
+  // Aliases can't have attributes so we check aliasee name to
+  // figure out if aliasee points to instrumented global. If
+  // that's true we should reference alias via GOT.
+  if (const GlobalAlias *GA = dyn_cast<GlobalAlias>(GV))
+    if (const GlobalAlias *Aliasee = dyn_cast<GlobalAlias>(GA->getAliasee()))
+      return Aliasee->getName().endswith(".hwasan");
+  return false;
+}
+
 bool TargetMachine::shouldAssumeDSOLocal(const Module &M,
                                          const GlobalValue *GV) const {
   // If the IR producer requested that this GV be treated as dso local, obey.
   if (GV && GV->isDSOLocal())
     return true;
 
   // If we are not supossed to use a PLT, we cannot assume that intrinsics are
   // local since the linker can convert some direct access to access via plt.
bool TargetMachine::shouldAssumeDSOLocal(const Module &M,
   assert(TT.isOSBinFormatELF());
   assert(RM != Reloc::DynamicNoPIC);
 
   bool IsExecutable =
       RM == Reloc::Static || M.getPIELevel() != PIELevel::Default;
   if (IsExecutable) {
     // If the symbol is defined, it cannot be preempted.
     if (GV && !GV->isDeclarationForLinker())
-      return true;
+      return !isHWAsanInstrumentedGlobal(GV);
 
     // A symbol marked nonlazybind should not be accessed with a plt. If the
     // symbol turns out to be external, the linker will convert a direct
     // access to an access via the plt, so don't assume it is local.
     const Function *F = dyn_cast_or_null<Function>(GV);
     if (F && F->hasFnAttribute(Attribute::NonLazyBind))
       return false;
 

lib/Transforms/Instrumentation/AddressSanitizer.cpp

for (size_t i = 0; i < n; i++) {
     if (G->getName().empty()) continue;
     GlobalsToAddToUsedList.push_back(G);
   }
   appendToCompilerUsed(M, ArrayRef<GlobalValue *>(GlobalsToAddToUsedList));
 
   std::string ELFUniqueModuleId =
       (UseGlobalsGC && TargetTriple.isOSBinFormatELF()) ? getUniqueModuleId(&M)
                                                         : "";
-
   if (!ELFUniqueModuleId.empty()) {
     InstrumentGlobalsELF(IRB, M, NewGlobals, Initializers, ELFUniqueModuleId);
     *CtorComdat = true;
   } else if (UseGlobalsGC && TargetTriple.isOSBinFormatCOFF()) {
     InstrumentGlobalsCOFF(IRB, M, NewGlobals, Initializers);
   } else if (UseGlobalsGC && ShouldUseMachOGlobalsSection()) {
     InstrumentGlobalsMachO(IRB, M, NewGlobals, Initializers);
   } else {

lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

static cl::opt<bool>
     ClInstrumentMemIntrinsics("hwasan-instrument-mem-intrinsics",
                               cl::desc("instrument memory intrinsics"),
                               cl::Hidden, cl::init(true));
 
 static cl::opt<bool> ClInlineAllChecks("hwasan-inline-all-checks",
                                        cl::desc("inline all checks"),
                                        cl::Hidden, cl::init(false));
 
+static cl::opt<bool>
+    ClInstrumentGlobals("hwasan-instrument-globals",
+                        cl::desc("instrument global variables"), cl::Hidden,
+                        cl::init(false));
+
 namespace {
+// Get the section name for frame descriptions. Currently ELF-only.
+inline const char *getFrameSection() { return "__hwasan_frames"; }
+inline const char *getFrameSectionBeg() { return "__start___hwasan_frames"; }
+inline const char *getFrameSectionEnd() { return "__stop___hwasan_frames"; }
+
+// Get the section name for globals descriptions. Currently ELF-only.
+inline const char *getGlobalsSection() { return "__hwasan_globals"; }
+inline const char *getGlobalsSectionBeg() { return "__start___hwasan_globals"; }
+inline const char *getGlobalsSectionEnd() { return "__stop___hwasan_globals"; }
 
 /// An instrumentation pass implementing detection of addressability bugs
 /// using tagged pointers.
 class HWAddressSanitizer : public FunctionPass {
 public:
   // Pass identification, replacement for typeid.
   static char ID;
 
private:
   FunctionCallee HWAsanMemmove, HWAsanMemcpy, HWAsanMemset;
 
   // Frame description is a way to pass names/sizes of local variables
   // to the run-time w/o adding extra executable code in every function.
   // We do this by creating a separate section with {PC,Descr} pairs and passing
   // the section beg/end to __hwasan_init_frames() at module init time.
   std::string createFrameString(ArrayRef<AllocaInst*> Allocas);
   void createFrameGlobal(Function &F, const std::string &FrameString);
-  // Get the section name for frame descriptions. Currently ELF-only.
-  const char *getFrameSection() { return "__hwasan_frames"; }
-  const char *getFrameSectionBeg() { return  "__start___hwasan_frames"; }
-  const char *getFrameSectionEnd() { return  "__stop___hwasan_frames"; }
-  GlobalVariable *createFrameSectionBound(Module &M, Type *Ty,
-                                          const char *Name) {
-    auto GV = new GlobalVariable(M, Ty, false, GlobalVariable::ExternalLinkage,
-                                 nullptr, Name);
-    GV->setVisibility(GlobalValue::HiddenVisibility);
-    return GV;
-  }
 
   /// This struct defines the shadow mapping using the rule:
   ///   shadow = (mem >> Scale) + Offset.
   /// If InGlobal is true, then
   ///   extern char __hwasan_shadow[];
   ///   shadow = (mem >> Scale) + &__hwasan_shadow
   /// If InTls is true, then
   ///   extern char *__hwasan_tls;
private:
   Type *IntptrTy;
   Type *Int8PtrTy;
   Type *Int8Ty;
   Type *Int32Ty;
 
   bool CompileKernel;
   bool Recover;
 
-  Function *HwasanCtorFunction;
-
   FunctionCallee HwasanMemoryAccessCallback[2][kNumberOfAccessSizes];
   FunctionCallee HwasanMemoryAccessCallbackSized[2];
 
   FunctionCallee HwasanTagMemoryFunc;
   FunctionCallee HwasanGenerateTagFunc;
   FunctionCallee HwasanThreadEnterFunc;
 
   Constant *ShadowGlobal;
 
   Value *LocalDynamicShadow = nullptr;
   GlobalValue *ThreadPtrGlobal = nullptr;
 };
 
+class HWAddressSanitizerModule : public ModulePass {
+  bool CompileKernel;
+  Function *HwasanCtorFunction;
+  Type *Int8Ty, *Int8PtrTy, *Int64Ty, *Int32Ty, *VoidTy;
+  bool globalWasGeneratedByCompiler(GlobalVariable *G);
+  bool shouldInstrumentGlobal(GlobalVariable *G);
+  GlobalAlias *createTaggedAlias(GlobalVariable *New, StringRef Name);
+  GlobalVariable *createInstrumentedGlobal(GlobalVariable *Orig);
+  Constant *replaceGlobal(GlobalVariable *New, GlobalVariable *Orig);
+  GlobalVariable *createSectionBound(Module &M, Type *Ty, const char *Name,
+                                     bool Weak);
+
+public:
+  // Pass identification, replacement for typeid
+  static char ID;
+
+  explicit HWAddressSanitizerModule(bool CompileKernel = false)
+      : ModulePass(ID) {
+    this->CompileKernel = ClEnableKhwasan.getNumOccurrences() > 0
+                              ? ClEnableKhwasan
+                              : CompileKernel;
+  }
+
+  bool runOnModule(Module &M) override;
+  bool doInitialization(Module &M) override;
+  StringRef getPassName() const override { return "HWAddressSanitizerModule"; }
+};
 } // end anonymous namespace
 
 char HWAddressSanitizer::ID = 0;
+char HWAddressSanitizerModule::ID = 0;
 
 INITIALIZE_PASS_BEGIN(
     HWAddressSanitizer, "hwasan",
     "HWAddressSanitizer: detect memory bugs using tagged addressing.", false,
     false)
 INITIALIZE_PASS_END(
     HWAddressSanitizer, "hwasan",
     "HWAddressSanitizer: detect memory bugs using tagged addressing.", false,
     false)
+INITIALIZE_PASS(
+    HWAddressSanitizerModule, "hwasan-module",
+    "HWAddressSanitizer: detect memory bugs using tagged addressing."
+    "ModulePass",
+    false, false)
 
 FunctionPass *llvm::createHWAddressSanitizerPass(bool CompileKernel,
                                                  bool Recover) {
   assert(!CompileKernel || Recover);
   return new HWAddressSanitizer(CompileKernel, Recover);
 }
 
+ModulePass *llvm::createHWAddressSanitizerModulePass(bool CompileKernel) {
+  return new HWAddressSanitizerModule(CompileKernel);
+}
+
 /// Module-level initialization.
 ///
 /// inserts a call to __hwasan_init to the module's constructor list.
 bool HWAddressSanitizer::doInitialization(Module &M) {
   LLVM_DEBUG(dbgs() << "Init " << M.getName() << "\n");
   auto &DL = M.getDataLayout();
 
   TargetTriple = Triple(M.getTargetTriple());
 
   Mapping.init(TargetTriple);
 
   C = &(M.getContext());
   CurModuleUniqueId = getUniqueModuleId(&M);
   IRBuilder<> IRB(*C);
   IntptrTy = IRB.getIntPtrTy(DL);
   Int8PtrTy = IRB.getInt8PtrTy();
   Int8Ty = IRB.getInt8Ty();
   Int32Ty = IRB.getInt32Ty();
 
-  HwasanCtorFunction = nullptr;
-  if (!CompileKernel) {
-    std::tie(HwasanCtorFunction, std::ignore) =
-        createSanitizerCtorAndInitFunctions(M, kHwasanModuleCtorName,
-                                            kHwasanInitName,
-                                            /*InitArgTypes=*/{},
-                                            /*InitArgs=*/{});
-    Comdat *CtorComdat = M.getOrInsertComdat(kHwasanModuleCtorName);
-    HwasanCtorFunction->setComdat(CtorComdat);
-    appendToGlobalCtors(M, HwasanCtorFunction, 0, HwasanCtorFunction);
-
-    // Create a zero-length global in __hwasan_frame so that the linker will
-    // always create start and stop symbols.
-    //
-    // N.B. If we ever start creating associated metadata in this pass this
-    // global will need to be associated with the ctor.
-    Type *Int8Arr0Ty = ArrayType::get(Int8Ty, 0);
-    auto GV =
-        new GlobalVariable(M, Int8Arr0Ty, /*isConstantGlobal*/ true,
-                           GlobalVariable::PrivateLinkage,
-                           Constant::getNullValue(Int8Arr0Ty), "__hwasan");
-    GV->setSection(getFrameSection());
-    GV->setComdat(CtorComdat);
-    appendToCompilerUsed(M, GV);
-
-    IRBuilder<> IRBCtor(HwasanCtorFunction->getEntryBlock().getTerminator());
-    IRBCtor.CreateCall(
-        declareSanitizerInitFunction(M, "__hwasan_init_frames",
-                                     {Int8PtrTy, Int8PtrTy}),
-        {createFrameSectionBound(M, Int8Ty, getFrameSectionBeg()),
-         createFrameSectionBound(M, Int8Ty, getFrameSectionEnd())});
-  }
-
   if (!TargetTriple.isAndroid())
     appendToCompilerUsed(
         M, ThreadPtrGlobal = new GlobalVariable(
                M, IntptrTy, false, GlobalVariable::ExternalLinkage, nullptr,
                "__hwasan_tls", nullptr, GlobalVariable::InitialExecTLSModel));
-
   return true;
 }
 
 void HWAddressSanitizer::initializeCallbacks(Module &M) {
   IRBuilder<> IRB(*C);
   for (size_t AccessIsWrite = 0; AccessIsWrite <= 1; AccessIsWrite++) {
     const std::string TypeStr = AccessIsWrite ? "store" : "load";
     const std::string EndingStr = Recover ? "_noabort" : "";
return (AI.getAllocatedType()->isSized() &&
           // inalloca allocas are not treated as static, and we don't want
           // dynamic alloca instrumentation for them as well.
           !AI.isUsedWithInAlloca() &&
           // swifterror allocas are register promoted by ISel
           !AI.isSwiftError());
 }
 
 bool HWAddressSanitizer::runOnFunction(Function &F) {
-  if (&F == HwasanCtorFunction)
+  if (F.getName().startswith("__hwasan"))
     return false;
 
   if (!F.hasFnAttribute(Attribute::SanitizeHWAddress))
     return false;
 
   LLVM_DEBUG(dbgs() << "Function: " << F.getName() << "\n");
 
   SmallVector<Instruction*, 16> ToInstrument;
if (ClMappingOffset.getNumOccurrences() > 0) {
     InTls = true;
     Offset = kDynamicShadowSentinel;
   } else {
     InGlobal = false;
     InTls = false;
     Offset = kDynamicShadowSentinel;
   }
 }
+
+bool HWAddressSanitizerModule::globalWasGeneratedByCompiler(GlobalVariable *G) {
+  StringRef Name = G->getName();
+  return Name.startswith("__hwasan") || Name.startswith("llvm.") ||
+         Name == getFrameSectionBeg() || Name == getFrameSectionEnd();
+}
+
+bool HWAddressSanitizerModule::shouldInstrumentGlobal(GlobalVariable *G) {
+  Type *Ty = G->getValueType();
+  LLVM_DEBUG(dbgs() << "GLOBAL: " << *G << "\n");
+
+  if (!Ty->isSized() || G->isDeclaration())
+    return false;
+  if (globalWasGeneratedByCompiler(G))
+    return false; // Our own globals.
+  // We don't support thread local variables for now.
+  if (G->isThreadLocal())
+    return false;
+  // Values with interposable linkage can be replaced by the linker.
+  // With appending linkage we don't know size until the final link.
+  // Hidden globals are not instrumented to allow non-instrumented code
+  // with external hidden declaration to link against instrumented code
+  // where that hidden variable is defined.
+  return !(G->hasAppendingLinkage() ||
+           G->hasAvailableExternallyLinkage() || G->hasHiddenVisibility());
+}
+
+GlobalVariable *HWAddressSanitizerModule::createSectionBound(Module &M,
+                                                             Type *Ty,
+                                                             const char *Name,
+                                                             bool Weak) {
+  auto Linkage = Weak ? GlobalVariable::ExternalWeakLinkage
+                      : GlobalVariable::ExternalLinkage;
+  auto GV = new GlobalVariable(M, Ty, false, Linkage, nullptr, Name);
+  GV->setVisibility(GlobalValue::HiddenVisibility);
+  return GV;
+}
+
+bool HWAddressSanitizerModule::doInitialization(Module &M) {
+  if (CompileKernel)
+    return false;
+
+  std::tie(HwasanCtorFunction, std::ignore) =
+      createSanitizerCtorAndInitFunctions(M, kHwasanModuleCtorName,
+                                          kHwasanInitName,
+                                          /*InitArgTypes=*/{},
+                                          /*InitArgs=*/{});
+  Comdat *CtorComdat = M.getOrInsertComdat(kHwasanModuleCtorName);
+  HwasanCtorFunction->setComdat(CtorComdat);
+  appendToGlobalCtors(M, HwasanCtorFunction, 0, HwasanCtorFunction);
+
+  IRBuilder<> IRBCtor(HwasanCtorFunction->getEntryBlock().getTerminator());
+  Int8Ty = IRBCtor.getInt8Ty();
+  Int8PtrTy = IRBCtor.getInt8PtrTy();
+  Int64Ty = IRBCtor.getInt64Ty();
+  Int32Ty = IRBCtor.getInt32Ty();
+  VoidTy = IRBCtor.getVoidTy();
+  // Create a zero-length global in __hwasan_frame so that the linker will
+  // always create start and stop symbols.
+  //
+  // N.B. If we ever start creating associated metadata in this pass this
+  // global will need to be associated with the ctor.
+  Type *Int8Arr0Ty = ArrayType::get(Int8Ty, 0);
+  auto GV = new GlobalVariable(M, Int8Arr0Ty, /*isConstantGlobal*/ true,
+                               GlobalVariable::PrivateLinkage,
+                               Constant::getNullValue(Int8Arr0Ty), "__hwasan");
+  GV->setSection(getFrameSection());
+  GV->setComdat(CtorComdat);
+  appendToCompilerUsed(M, GV);
+
+  IRBCtor.CreateCall(
+      declareSanitizerInitFunction(M, "__hwasan_init_frames",
+                                   {Int8PtrTy, Int8PtrTy}),
+      {createSectionBound(M, Int8Ty, getFrameSectionBeg(), false),
+       createSectionBound(M, Int8Ty, getFrameSectionEnd(), false)});
+  IRBCtor.CreateCall(
+      declareSanitizerInitFunction(M, "__hwasan_init_globals",
+                                   {Int8PtrTy, Int8PtrTy}),
+      {createSectionBound(M, Int8Ty, getGlobalsSectionBeg(), true),
+       createSectionBound(M, Int8Ty, getGlobalsSectionEnd(), true)});
+
+  return true;
+}
+
+static uint64_t getTagForName(StringRef Name) {
+  // Use Knuth multiplicative hash to get tag value
+  return (hash_value(Name) * 2654435761) % 255 + 1;
+}
+
+GlobalAlias *HWAddressSanitizerModule::createTaggedAlias(GlobalVariable *New,
+                                                         StringRef Name) {
+  uint64_t Tag = getTagForName(Name) << 56;
+  Constant *Aliasee = ConstantExpr::getIntToPtr(
+      ConstantExpr::getAdd(ConstantExpr::getPtrToInt(New, Int64Ty),
+                           ConstantInt::get(Int64Ty, Tag)),
+      New->getType());
+  GlobalAlias *Alias = GlobalAlias::create(
+      New->getValueType(), New->getAddressSpace(), GlobalValue::PrivateLinkage,
+      Name + ".hwasan", Aliasee, New->getParent());
+  GlobalAlias *Result =
+      GlobalAlias::create(Alias->getValueType(), Alias->getAddressSpace(),
+                          New->getLinkage(), "", Alias, Alias->getParent());
+  return Result;
+}
+
+GlobalVariable *
+HWAddressSanitizerModule::createInstrumentedGlobal(GlobalVariable *G) {
+  auto &DL = G->getParent()->getDataLayout();
+  Type *VTy = G->getValueType();
+  uint64_t SizeInBytes = DL.getTypeAllocSize(VTy);
+
+  // Align the size to shadow granularity. If already aligned then add
+  // extra 1 << kDefaultShadowScale bytes for padding
+  uint64_t Mask = (1 << kDefaultShadowScale) - 1;
+  uint64_t NewSize = (SizeInBytes & ~Mask) + (1 << kDefaultShadowScale);
+
+  // Create padding
+  Type *PaddingTy = ArrayType::get(Int8Ty, NewSize - SizeInBytes);
+  StructType *NewTy = StructType::get(VTy, PaddingTy);
+  Constant *NewInitializer = ConstantStruct::get(
+      NewTy, G->getInitializer(), Constant::getNullValue(PaddingTy));
+
+  // Create new global variable
+  Module &M = *G->getParent();
+  auto *NewGlobal =
+      new GlobalVariable(M, NewTy, G->isConstant(), G->getLinkage(),
+                         NewInitializer, "", G, G->getThreadLocalMode());
+  NewGlobal->copyAttributesFrom(G);
+  NewGlobal->setDSOLocal(G->isDSOLocal());
+  NewGlobal->setAlignment(
+      std::max(G->getAlignment(), 1u << kDefaultShadowScale));
+  // Don't fold globals with memory tagging
+  NewGlobal->setUnnamedAddr(GlobalValue::UnnamedAddr::None);
+  return NewGlobal;
+}
+
+Constant *HWAddressSanitizerModule::replaceGlobal(GlobalVariable *New,
+                                                  GlobalVariable *Orig) {
+  auto ReplaceGV = [](Constant *Expr, GlobalValue *New, GlobalVariable *Old) {
+    Old->replaceAllUsesWith(Expr);
+    New->takeName(Old);
+    Old->eraseFromParent();
+  };
+
+  size_t Tag = getTagForName(Orig->getName()) << 56;
+  if (New->hasLocalLinkage()) {
+    Value *Indices[] = {ConstantInt::get(Int64Ty, Tag)};
+    auto *TaggedPointer = ConstantExpr::getGetElementPtr(
+        nullptr, ConstantExpr::getPointerCast(New, Int8PtrTy), Indices);
+    auto *Expr = ConstantExpr::getPointerCast(TaggedPointer, Orig->getType());
+    ReplaceGV(Expr, New, Orig);
+    return TaggedPointer;
+  }
+
+  assert(New->getLinkage() == Orig->getLinkage());
+  GlobalAlias *TaggedAlias = createTaggedAlias(New, Orig->getName());
+  New->setName(Orig->getName() + ".hwasan.data");
+  New->setLinkage(GlobalValue::PrivateLinkage);
+  TaggedAlias->setVisibility(Orig->getVisibility());
+
+  auto *Zero = ConstantInt::get(Int32Ty, 0);
+  Value *Indices[] = {Zero, Zero};
+  ReplaceGV(ConstantExpr::getGetElementPtr(TaggedAlias->getValueType(),
+                                           TaggedAlias, Indices, true),
+            TaggedAlias, Orig);
+  return ConstantExpr::getPointerCast(TaggedAlias, Int8PtrTy);
+}
+
+bool HWAddressSanitizerModule::runOnModule(Module &M) {
+  if (CompileKernel || !ClInstrumentGlobals)
+    return false;
+
+  SmallVector<GlobalVariable *, 16> GlobalsToChange;
+
+  for (auto &G : M.globals())
+    if (shouldInstrumentGlobal(&G)) {
+      if (G.isInterposable())
+        // Force interposable vars to be accessed via GOT.
+        // Otherwise we can overflow relocations.
+        G.addAttribute("hwasan-interposable-global");                        
+      else
+        GlobalsToChange.push_back(&G);
+    }
+
+  size_t N = GlobalsToChange.size();
+  if (N == 0)
+    return false;
+
+  IRBuilder<> IRB(M.getContext());
+  // Global description structure type:
+  //  - tagged global pointer: i8*
+  //  - size: i64
+  // size is aligned to (1 << kDefaultShadowScale) bytes
+  StructType *GlobalStructTy = StructType::get(Int8PtrTy, Int64Ty);
+  for (size_t i = 0; i < N; i++) {
+    GlobalVariable *G = GlobalsToChange[i];
+    if (!G->hasName())
+      G->setName("__hwasan_anon_global");
+
+    std::string Name = G->getName();
+    size_t SizeInBytes = M.getDataLayout().getTypeAllocSize(G->getValueType());
+    Comdat *C = G->getComdat();
+    GlobalVariable *NewGlobal = createInstrumentedGlobal(G);
+    NewGlobal->setComdat(C);
+    MDNode *Assoc =
+        MDNode::get(M.getContext(), ValueAsMetadata::get(NewGlobal));
+    Constant *GVPtr = replaceGlobal(NewGlobal, G);
+
+    auto *GVRefInitializer = ConstantStruct::get(
+        GlobalStructTy, GVPtr,
+        ConstantInt::get(Int64Ty,
+                         alignTo(SizeInBytes, 1 << kDefaultShadowScale)));
+    auto *GVRef = new GlobalVariable(M, GlobalStructTy, false,
+                                     GlobalVariable::PrivateLinkage,
+                                     GVRefInitializer, Name + ".hwasan.ref");
+    GVRef->setComdat(C);
+    GVRef->setSection(getGlobalsSection());
+    GVRef->setMetadata(LLVMContext::MD_associated, Assoc);
+  }
+  return true;
+}

lib/Transforms/Instrumentation/Instrumentation.cpp

void llvm::initializeInstrumentation(PassRegistry &Registry) {
   initializePGOInstrumentationGenLegacyPassPass(Registry);
   initializePGOInstrumentationUseLegacyPassPass(Registry);
   initializePGOIndirectCallPromotionLegacyPassPass(Registry);
   initializePGOMemOPSizeOptLegacyPassPass(Registry);
   initializeInstrOrderFileLegacyPassPass(Registry);
   initializeInstrProfilingLegacyPassPass(Registry);
   initializeMemorySanitizerLegacyPassPass(Registry);
   initializeHWAddressSanitizerPass(Registry);
+  initializeHWAddressSanitizerModulePass(Registry);
   initializeThreadSanitizerLegacyPassPass(Registry);
   initializeSanitizerCoverageModulePass(Registry);
   initializeDataFlowSanitizerPass(Registry);
 }
 
 /// LLVMInitializeInstrumentation - C binding for
 /// initializeInstrumentation.
 void LLVMInitializeInstrumentation(LLVMPassRegistryRef R) {
   initializeInstrumentation(*unwrap(R));
 }

test/Instrumentation/HWAddressSanitizer/globals-aarch64.ll

+; REQUIRES: aarch64-registered-target
+; RUN: opt -hwasan-module -hwasan-instrument-globals %s -S -o %t.ll
+; RUN: llc -filetype=obj %t.ll -o %t.o
+; RUN: llvm-readobj -relocations %t.o | FileCheck %s --check-prefix=RELOCS
+
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+target triple = "aarch64-none-linux-gnu"
+
+@q = dso_local local_unnamed_addr global i64 2, align 8
+@c = common global i64 0, align 8
+
+declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i1)
+
+; Function Attrs: nounwind uwtable
+define dso_local i32 @main() local_unnamed_addr {
+  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @q to i8*), i8 0, i64 32, i1 false)
+  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @c to i8*), i8 0, i64 32, i1 false)
+  ret i32 0
+}
+
+; RELOCS:       Section (3) .rela.text {
+; RELOCS-NEXT:      R_AARCH64_ADR_GOT_PAGE q 0x0
+; RELOCS-NEXT:      R_AARCH64_ADR_GOT_PAGE c 0x0
+; RELOCS-NEXT:      R_AARCH64_LD64_GOT_LO12_NC q 0x0
+; RELOCS-NEXT:      R_AARCH64_LD64_GOT_LO12_NC c 0x0
+; RELOCS-NEXT:  }
+

test/Instrumentation/HWAddressSanitizer/globals.ll

+; RUN: opt -hwasan-module -hwasan-instrument-globals %s -S -o %t.ll
+; RUN: cat %t.ll | FileCheck %s
+; RUN: llc -filetype=obj %t.ll -o %t.o
+; RUN: llvm-readobj -relocations %t.o | FileCheck %s --check-prefix=RELOCS
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; For each externally visible global we create an alias with tagged adddress
+@q = dso_local local_unnamed_addr global i64 2, align 8
+
+; We don't create alias for static variables, because
+; those can't be referenced outside the TU
+@f = internal global i64 2, align 8
+
+; We don't instrument interposable globals, because those can
+; be preempted at link time and we may get ODR violation
+@c = common global i64 0, align 8
+
+; CHECK: @q.hwasan.data = private global { i64, [8 x i8] } { i64 2, [8 x i8] zeroinitializer }, align 16
+; CHECK: @__start___hwasan_globals = extern_weak hidden global i8
+; CHECK-NEXT: @__stop___hwasan_globals = extern_weak hidden global i8
+; CHECK-NEXT: @q.hwasan.ref = private global { i8*, i64 } { i8* bitcast ({ i64, [8 x i8] }* @q to i8*), i64 16 }, section "__hwasan_globals", !associated !0
+; CHECK-NEXT: @f.hwasan.ref = private global { i8*, i64 } { i8* getelementptr (i8, i8* bitcast ({ i64, [8 x i8] }* @f to i8*), i64 8718968878589280256), i64 16 }, section "__hwasan_globals", !associated !1
+
+; We create alias to alias to recognize instrumented variables when emitting relocs.
+; CHECK: @q.hwasan = private alias { i64, [8 x i8] }, inttoptr (i64 add (i64 ptrtoint ({ i64, [8 x i8] }* @q.hwasan.data to i64), i64 8718968878589280256) to { i64, [8 x i8] }*)
+; CHECK-NEXT: @q = alias { i64, [8 x i8] }, { i64, [8 x i8] }* @q.hwasan
+
+; Function Attrs: nounwind uwtable
+define dso_local i32 @main() local_unnamed_addr {
+  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @q to i8*), i8 0, i64 32, i1 false)
+; CHECK:  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast ({ i64, [8 x i8] }* @q to i8*), i8 0, i64 32, i1 false)
+  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @f to i8*), i8 0, i64 32, i1 false)
+; CHECK-NEXT: tail call void @llvm.memset.p0i8.i64(i8* align 8 getelementptr (i8, i8* bitcast ({ i64, [8 x i8] }* @f to i8*), i64 8718968878589280256), i8 0, i64 32, i1 false)
+  tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @c to i8*), i8 0, i64 32, i1 false)
+; CHECK-NEXT: tail call void @llvm.memset.p0i8.i64(i8* align 8 bitcast (i64* @c to i8*), i8 0, i64 32, i1 false)
+  ret i32 0
+}
+
+declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i1)
+
+; CHECK: define internal void @hwasan.module_ctor() comdat {
+; CHECK-NEXT:  call void @__hwasan_init()
+; CHECK-NEXT:  call void @__hwasan_init_frames(i8* @__start___hwasan_frames, i8* @__stop___hwasan_frames)
+; CHECK-NEXT:  call void @__hwasan_init_globals(i8* @__start___hwasan_globals, i8* @__stop___hwasan_globals)
+; CHECK-NEXT:  ret void
+; CHECK-NEXT: }
+
+; We mark interposable globals to emit GOT relocations to them
+; CHECK: attributes #0 = { "hwasan-interposable-global" }
+
+; CHECK:      !0 = !{{\{\{}} i64, [8 x i8] }* @q.hwasan.data}
+; CHECK-NEXT: !1 = !{{\{\{}} i64, [8 x i8] }* @f}
+
+
+; RELOCS:     R_X86_64_GOTPCREL q 0xFFFFFFFFFFFFFFFC
+; RELOCS:     R_X86_64_GOTPCREL c 0xFFFFFFFFFFFFFFFC
+