[HWASan] Build separate LAM runtime on x86_64.

[morehouse]

Summary

Since we have both aliasing mode and Intel LAM on x86_64, we need to
choose the mode at either run time or compile time. This patch
implements the plumbing to build both and choose between them at
compile time.

[morehouse]

Rebase

[morehouse]

• Add comment regarding new prctl() API.

[vitalybuka]

Would it be simple if module constructor of initialized code just sets some global or call init function to switch HWASAN into LAM so we can have the same runtime?

[xiangzhangllvm]

compiler-rt/lib/hwasan/hwasan_linux.cpp

125

Do we use this way to enable INTEL_LAM ?

[morehouse]

In D102286#2752765, @vitalybuka wrote:

Would it be simple if module constructor of initialized code just sets some global or call init function to switch HWASAN into LAM so we can have the same runtime?

I have another patch that does this. It is actually much more complicated than this. The tradeoff is that we lose most of our constexpr stuff. kAddressTagShift, kNumBits, and even heap size become non-constexpr and need to be loaded on callbacks. Significant changes to the primary also need to be made to handle non-constant heap sizes.

Considering we want to optimize in future patches with, for example, a custom calling convention, we may need to switch back to constexpr stuff in the future anyway.

compiler-rt/lib/hwasan/hwasan_linux.cpp

125

prctl is what the current public LAM patch uses: https://lore.kernel.org/linux-mm/[e-mail]/

Kirill said the API is subject to change, but he wasn't sure when we'll know what the final API is. So for now we're using what the only interface we know about and can test in QEMU.

[eugenis]

compiler-rt/lib/hwasan/CMakeLists.txt

171

I wonder if "lam" would look better as an arch suffix:

libclang_rt.hwasan-x86_64-lam  (or x86_64_lam)

unless it creates complications that I don't yet see.

compiler-rt/lib/hwasan/hwasan_linux.cpp

153

Could we put this stuff undef INTEL_LAM? I don't want this experimental calls to be done on non-LAM targets, where it is known not to work.

[morehouse]

compiler-rt/lib/hwasan/CMakeLists.txt

171

Just spent 30 minutes trying this. It seems to be more complicated since adding _lam to ${arch} requires all the RT* object libraries to be built with a _lam suffix, including non-hwasan ones.

If you still want to do this, I can keep working on it and upload a new diff, but I think it will be more complex.

[morehouse]

• Guard new prctl() API under INTEL_LAM.

[eugenis]

LGTM

[morehouse]

• Rebase onto D102288.
• Refactor to make LAM the default runtime.
• Use alias runtime if specified.

[gulfem]

We started seeing test failures in our Windows builds in Fuchsia after this patch.

FAILED: compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o 
C:\b\s\w\ir\x\w\staging\llvm_build\.\bin\clang++.exe --target=aarch64-unknown-linux-gnu --sysroot=C:/b/s/w/ir/x/w/cipd/linux  -DHWASAN_WITH_INTERCEPTORS=1 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IC:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/.. --target=aarch64-unknown-linux-gnu -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -ffile-prefix-map=C:/b/s/w/ir/x/w/staging/llvm_build/runtimes/runtimes-aarch64-unknown-linux-gnu-bins=../staging/llvm_build/runtimes/runtimes-aarch64-unknown-linux-gnu-bins -ffile-prefix-map=C:/b/s/w/ir/x/w/llvm-project/= -no-canonical-prefixes -Wall -std=c++14 -Wno-unused-parameter -O2 -g -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -nostdinc++ -fno-rtti -fPIC -ffreestanding -DHWASAN_ALIASING_MODE -UNDEBUG -MD -MT compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o -MF compiler-rt\lib\hwasan\CMakeFiles\RTHwasanAliases.aarch64.dir\hwasan_interceptors.cpp.o.d -o compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o -c C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan_interceptors.cpp
In file included from C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan_interceptors.cpp:18:
C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan.h:41:6: error: Aliasing mode is only supported on x86_64

The error message to the full build:
https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket.appspot.com/8846968772132677360/+/u/clang/build/stdout

The CMake commands:
https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket.appspot.com/8846968772132677360/+/u/clang/configure/execution_details

@morehouse do you have any idea?

[morehouse]

In D102286#2764203, @gulfem wrote:

We started seeing test failures in our Windows builds in Fuchsia after this patch.

FAILED: compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o 
C:\b\s\w\ir\x\w\staging\llvm_build\.\bin\clang++.exe --target=aarch64-unknown-linux-gnu --sysroot=C:/b/s/w/ir/x/w/cipd/linux  -DHWASAN_WITH_INTERCEPTORS=1 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IC:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/.. --target=aarch64-unknown-linux-gnu -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -ffile-prefix-map=C:/b/s/w/ir/x/w/staging/llvm_build/runtimes/runtimes-aarch64-unknown-linux-gnu-bins=../staging/llvm_build/runtimes/runtimes-aarch64-unknown-linux-gnu-bins -ffile-prefix-map=C:/b/s/w/ir/x/w/llvm-project/= -no-canonical-prefixes -Wall -std=c++14 -Wno-unused-parameter -O2 -g -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -nostdinc++ -fno-rtti -fPIC -ffreestanding -DHWASAN_ALIASING_MODE -UNDEBUG -MD -MT compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o -MF compiler-rt\lib\hwasan\CMakeFiles\RTHwasanAliases.aarch64.dir\hwasan_interceptors.cpp.o.d -o compiler-rt/lib/hwasan/CMakeFiles/RTHwasanAliases.aarch64.dir/hwasan_interceptors.cpp.o -c C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan_interceptors.cpp
In file included from C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan_interceptors.cpp:18:
C:/b/s/w/ir/x/w/llvm-project/compiler-rt/lib/hwasan/hwasan.h:41:6: error: Aliasing mode is only supported on x86_64

The error message to the full build:
https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket.appspot.com/8846968772132677360/+/u/clang/build/stdout

The CMake commands:
https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket.appspot.com/8846968772132677360/+/u/clang/configure/execution_details

@morehouse do you have any idea?

Looks like the same issue I fixed in https://reviews.llvm.org/rGd97bab651185. Let me know if you have the issue after that commit.

[gulfem]

Looks like the same issue I fixed in https://reviews.llvm.org/rGd97bab651185. Let me know if you have the issue after that commit.

Thanks, that resolved the build issue.

compiler-rt/lib/hwasan/CMakeLists.txt

add_compiler_rt_object_libraries(RTHwasan_cxx
   DEFS ${HWASAN_DEFINITIONS})
 add_compiler_rt_object_libraries(RTHwasan_dynamic
   ARCHS ${HWASAN_SUPPORTED_ARCH}
   SOURCES ${HWASAN_RTL_SOURCES} ${HWASAN_RTL_CXX_SOURCES}
   ADDITIONAL_HEADERS ${HWASAN_RTL_HEADERS}
   CFLAGS ${HWASAN_DYNAMIC_CFLAGS}
   DEFS ${HWASAN_DEFINITIONS})
 
+# Compile a different runtime for x86 LAM mode.
+set(HWASAN_LAM_RTL_CFLAGS ${HWASAN_RTL_CFLAGS})
+list(APPEND HWASAN_LAM_RTL_CFLAGS -DINTEL_LAM)
+set(HWASAN_LAM_DYNAMIC_CFLAGS ${HWASAN_DYNAMIC_CFLAGS})
+list(APPEND HWASAN_LAM_DYNAMIC_CFLAGS -DINTEL_LAM)
+add_compiler_rt_object_libraries(RTHwasanLam
+  ARCHS ${HWASAN_SUPPORTED_ARCH}
+  SOURCES ${HWASAN_RTL_SOURCES}
+  ADDITIONAL_HEADERS ${HWASAN_RTL_HEADERS}
+  CFLAGS ${HWASAN_LAM_RTL_CFLAGS}
+  DEFS ${HWASAN_DEFINITIONS})
+add_compiler_rt_object_libraries(RTHwasanLam_dynamic
+  ARCHS ${HWASAN_SUPPORTED_ARCH}
+  SOURCES ${HWASAN_RTL_SOURCES} ${HWASAN_RTL_CXX_SOURCES}
+  ADDITIONAL_HEADERS ${HWASAN_RTL_HEADERS}
+  CFLAGS ${HWASAN_LAM_DYNAMIC_CFLAGS}
+  DEFS ${HWASAN_DEFINITIONS})
+
 file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/dummy.cpp "")
 add_compiler_rt_object_libraries(RTHwasan_dynamic_version_script_dummy
   ARCHS ${HWASAN_SUPPORTED_ARCH}
   SOURCES ${CMAKE_CURRENT_BINARY_DIR}/dummy.cpp
   CFLAGS ${HWASAN_DYNAMIC_CFLAGS}
   DEFS ${HWASAN_DEFINITIONS})
 
-foreach(arch ${HWASAN_SUPPORTED_ARCH})
-  add_compiler_rt_runtime(clang_rt.hwasan
+# If use_lam is TRUE, adds the HWASan runtime built with LAM support.
+# Otherwise adds the runtime without LAM support.
+function(add_hwasan_runtimes arch use_lam)
+  set(hwasan_object_lib RTHwasan)
+  set(hwasan_object_dyn_lib RTHwasan_dynamic)
+  set(hwasan_runtime clang_rt.hwasan)
+  set(hwasan_rtl_flags ${HWASAN_RTL_CFLAGS})
+  set(hwasan_dyn_flags ${HWASAN_DYNAMIC_CFLAGS})
+  if(use_lam)
+    set(hwasan_object_lib RTHwasanLam)
+    set(hwasan_object_dyn_lib RTHwasanLam_dynamic)
+    set(hwasan_runtime clang_rt.hwasan_lam)
+    set(hwasan_rtl_flags ${HWASAN_LAM_RTL_CFLAGS})
+    set(hwasan_dyn_flags ${HWASAN_LAM_DYNAMIC_CFLAGS})
+  endif()
+  add_compiler_rt_runtime(${hwasan_runtime}
     STATIC
     ARCHS ${arch}
-    OBJECT_LIBS RTHwasan
+    OBJECT_LIBS ${hwasan_object_lib}
                 RTInterception
                 RTSanitizerCommon
                 RTSanitizerCommonLibc
                 RTSanitizerCommonCoverage
                 RTSanitizerCommonSymbolizer
                 RTUbsan
-    CFLAGS ${HWASAN_RTL_CFLAGS}
+    CFLAGS ${hwasan_rtl_flags}
     PARENT_TARGET hwasan)
-  add_compiler_rt_runtime(clang_rt.hwasan_cxx
+  add_compiler_rt_runtime(${hwasan_runtime}_cxx
     STATIC
     ARCHS ${arch}
     OBJECT_LIBS RTHwasan_cxx
                 RTUbsan_cxx
-    CFLAGS ${HWASAN_RTL_CFLAGS}
+    CFLAGS ${hwasan_rtl_flags}
     PARENT_TARGET hwasan)
 
   if (UNIX)
-    add_sanitizer_rt_version_list(clang_rt.hwasan-dynamic-${arch}
-                                  LIBS clang_rt.hwasan-${arch} clang_rt.hwasan_cxx-${arch}
+    add_sanitizer_rt_version_list(${hwasan_runtime}-dynamic-${arch}
+                                  LIBS ${hwasan_runtime}-${arch} ${hwasan_runtime}_cxx-${arch}
                                   EXTRA hwasan.syms.extra)
     set(VERSION_SCRIPT_FLAG
-         -Wl,--version-script,${CMAKE_CURRENT_BINARY_DIR}/clang_rt.hwasan-dynamic-${arch}.vers)
+         -Wl,--version-script,${CMAKE_CURRENT_BINARY_DIR}/${hwasan_runtime}-dynamic-${arch}.vers)
     set_property(SOURCE
       ${CMAKE_CURRENT_BINARY_DIR}/dummy.cpp
       APPEND PROPERTY
-      OBJECT_DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/clang_rt.hwasan-dynamic-${arch}.vers)
+      OBJECT_DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${hwasan_runtime}-dynamic-${arch}.vers)
   else()
     set(VERSION_SCRIPT_FLAG)
   endif()
 
 
-  add_compiler_rt_runtime(clang_rt.hwasan
+  add_compiler_rt_runtime(${hwasan_runtime}
     SHARED
     ARCHS ${arch}
     OBJECT_LIBS
-            RTHwasan_dynamic
+            ${hwasan_object_dyn_lib}
             RTInterception
             RTSanitizerCommon
             RTSanitizerCommonLibc
             RTSanitizerCommonCoverage
             RTSanitizerCommonSymbolizer
             RTUbsan
             RTUbsan_cxx
             # The only purpose of RTHWAsan_dynamic_version_script_dummy is to
             # carry a dependency of the shared runtime on the version script.
             # Replacing it with a straightforward
             # add_dependencies(clang_rt.asan-dynamic-${arch} clang_rt.asan-dynamic-${arch}-version-list)
             # generates an order-only dependency in ninja.
             RTHwasan_dynamic_version_script_dummy
-    CFLAGS ${HWASAN_DYNAMIC_CFLAGS}
+    CFLAGS ${hwasan_dyn_flags}
     LINK_FLAGS ${HWASAN_DYNAMIC_LINK_FLAGS}
               ${VERSION_SCRIPT_FLAG}
     LINK_LIBS ${HWASAN_DYNAMIC_LIBS}
     DEFS ${ASAN_DYNAMIC_DEFINITIONS}
     PARENT_TARGET hwasan)
 
   if(SANITIZER_USE_SYMBOLS)
-    add_sanitizer_rt_symbols(clang_rt.hwasan
+    add_sanitizer_rt_symbols(${hwasan_runtime}
       ARCHS ${arch}
       EXTRA hwasan.syms.extra)
-    add_sanitizer_rt_symbols(clang_rt.hwasan_cxx
+    add_sanitizer_rt_symbols(${hwasan_runtime}_cxx
       ARCHS ${arch}
       EXTRA hwasan.syms.extra)
-    add_dependencies(hwasan clang_rt.hwasan-${arch}-symbols
-                          clang_rt.hwasan_cxx-${arch}-symbols)
+    add_dependencies(hwasan ${hwasan_runtime}-${arch}-symbols
+                          ${hwasan_runtime}_cxx-${arch}-symbols)
+  endif()
+endfunction()
+
+foreach(arch ${HWASAN_SUPPORTED_ARCH})
+  add_hwasan_runtimes(${arch} FALSE)
+  if(${arch} MATCHES "x86_64")
+    add_hwasan_runtimes(${arch} TRUE)
   endif()
 endforeach()
 
 add_compiler_rt_resource_file(hwasan_ignorelist hwasan_ignorelist.txt hwasan)
 
 add_subdirectory("scripts")
 
 # if(COMPILER_RT_INCLUDE_TESTS)
 #   add_subdirectory(tests)
 # endif()

compiler-rt/lib/hwasan/hwasan.h

 
 #ifndef HWASAN_REPLACE_OPERATORS_NEW_AND_DELETE
 #define HWASAN_REPLACE_OPERATORS_NEW_AND_DELETE HWASAN_WITH_INTERCEPTORS
 #endif
 
 typedef u8 tag_t;
 
 #if defined(__x86_64__)
+#  if defined(INTEL_LAM)
+// Tags are done in upper bits using Intel LAM.
+constexpr unsigned kAddressTagShift = 57;
+constexpr unsigned kTagBits = 6;
+#  else
 // Tags are done in middle bits using userspace aliasing.
 constexpr unsigned kAddressTagShift = 39;
 constexpr unsigned kTagBits = 3;
 
 // The alias region is placed next to the shadow so the upper bits of all
 // taggable addresses matches the upper bits of the shadow base.  This shift
 // value determines which upper bits must match.  It has a floor of 44 since the
 // shadow is always 8TB.
 // TODO(morehouse): In alias mode we can shrink the shadow and use a
 // simpler/faster shadow calculation.
 constexpr unsigned kTaggableRegionCheckShift =
     __sanitizer::Max(kAddressTagShift + kTagBits + 1U, 44U);
+#  endif  // defined(INTEL_LAM)
 #else
 // TBI (Top Byte Ignore) feature of AArch64: bits [63:56] are ignored in address
 // translation and can be used to store a tag.
 constexpr unsigned kAddressTagShift = 56;
 constexpr unsigned kTagBits = 8;
 #endif  // defined(__x86_64__)
 
 // Mask for extracting tag bits from the lower 8 bits.

compiler-rt/lib/hwasan/hwasan_allocator.h

struct HwasanMapUnmapCallback {
   }
 };
 
 static const uptr kMaxAllowedMallocSize = 1UL << 40;  // 1T
 
 struct AP64 {
   static const uptr kSpaceBeg = ~0ULL;
 
-#if defined(__x86_64__)
+#if defined(__x86_64__) && !defined(INTEL_LAM)
   static const uptr kSpaceSize = 1ULL << kAddressTagShift;
 #else
   static const uptr kSpaceSize = 0x2000000000ULL;
 #endif
   static const uptr kMetadataSize = sizeof(Metadata);
   typedef __sanitizer::VeryDenseSizeClassMap SizeClassMap;
   using AddressSpaceView = LocalAddressSpaceView;
   typedef HwasanMapUnmapCallback MapUnmapCallback;
struct HeapAllocationRecord {
   u32  requested_size;
 };
 
 typedef RingBuffer<HeapAllocationRecord> HeapAllocationsRingBuffer;
 
 void GetAllocatorStats(AllocatorStatCounters s);
 
 inline bool InTaggableRegion(uptr addr) {
-#if defined(__x86_64__)
+#if defined(__x86_64__) && !defined(INTEL_LAM)
   // Aliases are mapped next to shadow so that the upper bits match the shadow
   // base.
   return (addr >> kTaggableRegionCheckShift) ==
          (__hwasan_shadow_memory_dynamic_address >> kTaggableRegionCheckShift);
 #endif
   return true;
 }
 
 } // namespace __hwasan
 
 #endif // HWASAN_ALLOCATOR_H

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp

 
 }  // namespace __hwasan
 #else
 namespace __hwasan {
 
 void InitShadowGOT() {}
 
 uptr FindDynamicShadowStart(uptr shadow_size_bytes) {
-#if defined(__x86_64__)
+#  if defined(__x86_64__) && !defined(INTEL_LAM)
   constexpr uptr kAliasSize = 1ULL << kAddressTagShift;
   constexpr uptr kNumAliases = 1ULL << kTagBits;
   return MapDynamicShadowAndAliases(shadow_size_bytes, kAliasSize, kNumAliases,
                                     RingBufferSize());
-#endif
+#  endif
   return MapDynamicShadow(shadow_size_bytes, kShadowScale, kShadowBaseAlignment,
                           kHighMemEnd);
 }
 
 }  // namespace __hwasan
 
 #endif  // SANITIZER_ANDROID

compiler-rt/lib/hwasan/hwasan_linux.cpp

 uptr kLowMemEnd;
 uptr kLowShadowEnd;
 uptr kLowShadowStart;
 uptr kHighShadowStart;
 uptr kHighShadowEnd;
 uptr kHighMemStart;
 uptr kHighMemEnd;
 
-uptr kAliasRegionStart;  // Always 0 on non-x86.
+uptr kAliasRegionStart;  // Always 0 when aliases aren't used.
 
 static void PrintRange(uptr start, uptr end, const char *name) {
   Printf("|| [%p, %p] || %.*s ||\n", (void *)start, (void *)end, 10, name);
 }
 
 static void PrintAddressSpaceLayout() {
   PrintRange(kHighMemStart, kHighMemEnd, "HighMem");
   if (kHighShadowEnd + 1 < kHighMemStart)

 }
 
 void InitPrctl() {
 #define PR_SET_TAGGED_ADDR_CTRL 55
 #define PR_GET_TAGGED_ADDR_CTRL 56
 #define PR_TAGGED_ADDR_ENABLE (1UL << 0)
   // Check we're running on a kernel that can use the tagged address ABI.
   int local_errno = 0;
   if (internal_iserror(internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0),
                        &local_errno) &&
       local_errno == EINVAL) {
-#if SANITIZER_ANDROID || defined(__x86_64__)
+#  if SANITIZER_ANDROID || (defined(__x86_64__) && !defined(INTEL_LAM))
     // Some older Android kernels have the tagged pointer ABI on
     // unconditionally, and hence don't have the tagged-addr prctl while still
     // allow the ABI.
     // If targeting Android and the prctl is not around we assume this is the
     // case.
     return;
 #else
     if (flags()->fail_without_syscall_abi) {
       Printf(
           "FATAL: "
           "HWAddressSanitizer requires a kernel with tagged address ABI.\n");
       Die();
     }
 #endif
   }
 
   // Turn on the tagged address ABI.
   if ((internal_iserror(internal_prctl(PR_SET_TAGGED_ADDR_CTRL,
                                        PR_TAGGED_ADDR_ENABLE, 0, 0, 0)) ||
-       !internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0)) &&
-      flags()->fail_without_syscall_abi) {
+       !internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0))) {
+    // Try the new prctl API for Intel LAM.  The API is based on a currently
+    // unsubmitted patch to the Linux kernel (as of May 2021) and is thus
+    // subject to change.  Patch is here:
+    // https://lore.kernel.org/linux-mm/[e-mail]/
+    int tag_bits = kTagBits;
+    int tag_shift = kAddressTagShift;
+    if (internal_iserror(
+            internal_prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE,
+                           reinterpret_cast<unsigned long>(&tag_bits),
+                           reinterpret_cast<unsigned long>(&tag_shift), 0))) {
+      if (flags()->fail_without_syscall_abi) {
         Printf(
             "FATAL: HWAddressSanitizer failed to enable tagged address syscall "
             "ABI.\nSuggest check `sysctl abi.tagged_addr_disabled` "
             "configuration.\n");
         Die();
       }
+    } else {
+      CHECK_EQ(tag_bits, kTagBits);
+      CHECK_EQ(tag_shift, kAddressTagShift);
+    }
+  }
 #undef PR_SET_TAGGED_ADDR_CTRL
 #undef PR_GET_TAGGED_ADDR_CTRL
 #undef PR_TAGGED_ADDR_ENABLE
 }
 
 bool InitShadow() {
   // Define the entire memory range.
   kHighMemEnd = GetHighMemEnd();
bool InitShadow() {
   // High shadow takes whatever memory is left up there (making sure it is not
   // interfering with low memory in the fixed case).
   kHighShadowEnd = MemToShadow(kHighMemEnd);
   kHighShadowStart = Max(kLowMemEnd, MemToShadow(kHighShadowEnd)) + 1;
 
   // High memory starts where allocated shadow allows.
   kHighMemStart = ShadowToMem(kHighShadowStart);
 
-#if defined(__x86_64__)
+#  if defined(__x86_64__) && !defined(INTEL_LAM)
   constexpr uptr kAliasRegionOffset = 1ULL << (kTaggableRegionCheckShift - 1);
   kAliasRegionStart =
       __hwasan_shadow_memory_dynamic_address + kAliasRegionOffset;
 
   CHECK_EQ(kAliasRegionStart >> kTaggableRegionCheckShift,
            __hwasan_shadow_memory_dynamic_address >> kTaggableRegionCheckShift);
   CHECK_EQ(
       (kAliasRegionStart + kAliasRegionOffset - 1) >> kTaggableRegionCheckShift,