[compiler-rt][hwasan] Support for Intel LAM v6 API

[glider]

Summary

Version 6 of Intel LAM kernel patches
(https://lore.kernel.org/all/[e-mail]/)
introduces arch_prctl(ARCH_GET_MAX_TAG_BITS), which (unlike
ARCH_GET_UNTAG_MASK) can be used to determine if the kernel supports pointer tagging.

This patch also refactors InitializeOsSupport(), making it more
readable.

[glider]

compiler-rt/lib/hwasan/hwasan_linux.cpp

144

I actually don't see any point in checking for EINVAL here. Shouldn't we bail out in the case of a different error?

173

As far as I understood, previously the semantics of has_abi was "This platform does not have the tagged ABI".

194

I am not very proud of this construct. On the other hand, building a string on the fly looks excessive, and putting line 195 under #if SANITIZER_ANDROID is too hacky.

202

Do we really need to undef these constants? This is a .cpp file, after all.

[vitalybuka]

Thanks for the patch, I was planing to do that today myself.
I've already updated https://lab.llvm.org/buildbot/#/builders/sanitizer-aarch64-linux-bootstrap-hwasan to LAM_v6 patchset, so you can land the patch without breaking it.

I would recommend to move fixes of preexisted issues and introduction of new functions into a separate patches.

compiler-rt/lib/hwasan/hwasan_linux.cpp

144

do you know why don't have x86_64 of PR_GET_TAGGED_ADDR_CTRL in kernel, seems like it more gereric than ARCH_GET_MAX_TAG_BITS?

144

I actually don't see any point in checking for EINVAL here. Shouldn't we bail out in the case of a different error?

Yes, looks weird

173

As far as I understood, previously the semantics of has_abi was "This platform does not have the tagged ABI".

yes, looks like a typo

202

i don't think so

[vitalybuka]

compiler-rt/lib/hwasan/hwasan_linux.cpp

126

static

150

static

[glider]

Thanks for the comments, I'll split this patch into a series then.

compiler-rt/lib/hwasan/hwasan_linux.cpp

144

do you know why don't have x86_64 of PR_GET_TAGGED_ADDR_CTRL in kernel, seems like it more gereric than ARCH_GET_MAX_TAG_BITS?

I don't remember for sure. There's some discussion in https://lore.kernel.org/lkml/[e-mail]/T/#u, the second RFC (https://lore.kernel.org/lkml/[e-mail]/) was already using the arch_prctl() interface. The second thread also points out that the original ARM interface wasn't generic enough (with reasons being explained).

compiler-rt/lib/hwasan/hwasan_linux.cpp

static uptr GetHighMemEnd() {
   return max_address;
 }
 
 static void InitializeShadowBaseAddress(uptr shadow_size_bytes) {
   __hwasan_shadow_memory_dynamic_address =
       FindDynamicShadowStart(shadow_size_bytes);
 }
 
-void InitializeOsSupport() {
+void MaybeFailWithoutTaggingAbi(const char *message) {
+  if (!flags()->fail_without_syscall_abi)
+    return;
+  Printf("FATAL: %s\n", message);
+  Die();
+}
+
 #  define PR_SET_TAGGED_ADDR_CTRL 55
 #  define PR_GET_TAGGED_ADDR_CTRL 56
 #  define PR_TAGGED_ADDR_ENABLE (1UL << 0)
-#  define ARCH_GET_UNTAG_MASK 0x4001
 #  define ARCH_ENABLE_TAGGED_ADDR 0x4002
-  // Check we're running on a kernel that can use the tagged address ABI.
-  int local_errno = 0;
-  bool has_abi;
+#  define ARCH_GET_MAX_TAG_BITS 0x4003
+
+bool CanUseTaggingAbi() {
 #  if defined(__x86_64__)
-  has_abi = (internal_iserror(internal_arch_prctl(ARCH_GET_UNTAG_MASK, 0),
-                              &local_errno) &&
-             local_errno == EINVAL);
+  int num_bits = 0;
+  // Check for x86 LAM support. This API is based on a currently unsubmitted
+  // patch to the Linux kernel (as of August 2022) and is thus subject to
+  // change. The patch is here:
+  // https://lore.kernel.org/all/[e-mail]/
+  if (internal_iserror(internal_arch_prctl(ARCH_GET_MAX_TAG_BITS,
+                                           reinterpret_cast<uptr>(&num_bits))))
+    return false;
+  if (num_bits != kTagBits)
+    return false;
+  return true;
 #  else
-  has_abi =
-      (internal_iserror(internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0),
-                        &local_errno) &&
-       local_errno == EINVAL);
-#  endif
-  if (has_abi) {
+  int local_errno = 0;
+  // Check for ARM TBI support.
+  if (internal_iserror(internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0),
+      local_errno == EINVAL)
+    return false;
+  return true;
+#  endif // __x86_64__
+}
+
+bool EnableTaggingAbi() {
+#  if defined(__x86_64__)
+  // Enable x86 LAM tagging for the process.
+  if (internal_iserror(internal_arch_prctl(ARCH_ENABLE_TAGGED_ADDR, kTagBits)))
+    return false;
+  return true;
+#  else
+  // Enable ARM TBI tagging for the process. If for some reason tagging is not
+  // supported, prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE) returns
+  // -EINVAL.
+  if (internal_iserror(internal_prctl(PR_SET_TAGGED_ADDR_CTRL,
+                                      PR_TAGGED_ADDR_ENABLE, 0, 0, 0)))
+    return false;
+  // Ensure that TBI is enabled.
+  if (internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0) !=
+      PR_TAGGED_ADDR_ENABLE)
+    return false;
+  return true;
+#  endif // __x86_64__
+}
+
+void InitializeOsSupport() {
+  // Check we're running on a kernel that can use the tagged address ABI.
+  bool has_abi = CanUseTaggingAbi();
+
+  if (!has_abi) {
 #  if SANITIZER_ANDROID || defined(HWASAN_ALIASING_MODE)
     // Some older Android kernels have the tagged pointer ABI on
     // unconditionally, and hence don't have the tagged-addr prctl while still
     // allow the ABI.
     // If targeting Android and the prctl is not around we assume this is the
     // case.
     return;
 #  else
-    if (flags()->fail_without_syscall_abi) {
-      Printf(
-          "FATAL: "
-          "HWAddressSanitizer requires a kernel with tagged address ABI.\n");
-      Die();
-    }
+    MaybeFailWithoutTaggingAbi(
+        "HWAddressSanitizer requires a kernel with tagged address ABI.");
 #  endif
   }
 
-  // Turn on the tagged address ABI.
-  if ((internal_iserror(internal_prctl(PR_SET_TAGGED_ADDR_CTRL,
-                                       PR_TAGGED_ADDR_ENABLE, 0, 0, 0)) ||
-       !internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0))) {
-#  if defined(__x86_64__) && !defined(HWASAN_ALIASING_MODE)
-    // Try the new prctl API for Intel LAM.  The API is based on a currently
-    // unsubmitted patch to the Linux kernel (as of July 2022) and is thus
-    // subject to change.  Patch is here:
-    // https://lore.kernel.org/linux-mm/[e-mail]/
-    if (!internal_iserror(
-            internal_arch_prctl(ARCH_ENABLE_TAGGED_ADDR, kTagBits))) {
-      return;
-    }
-#  endif  // defined(__x86_64__) && !defined(HWASAN_ALIASING_MODE)
-    if (flags()->fail_without_syscall_abi) {
-      Printf(
-          "FATAL: HWAddressSanitizer failed to enable tagged address syscall "
-          "ABI.\nSuggest check `sysctl abi.tagged_addr_disabled` "
-          "configuration.\n");
-      Die();
-    }
-  }
+  if (EnableTaggingAbi())
+
+#  if SANITIZER_ANDROID
+  MaybeFailWithoutTaggingAbi(
+      "HWAddressSanitizer failed to enable tagged address syscall ABI.\n"
+      "Check the `sysctl abi.tagged_addr_disabled` configuration.");
+#  else
+  MaybeFailWithoutTaggingAbi(
+      "HWAddressSanitizer failed to enable tagged address syscall ABI.\n");
+#  endif
+
 #  undef PR_SET_TAGGED_ADDR_CTRL
 #  undef PR_GET_TAGGED_ADDR_CTRL
 #  undef PR_TAGGED_ADDR_ENABLE
-}
+#  undef ARCH_ENABLE_TAGGED_ADDR
+#  undef ARCH_GET_MAX_TAG_BITS
 
 bool InitShadow() {
   // Define the entire memory range.
   kHighMemEnd = GetHighMemEnd();
 
   // Determine shadow memory base offset.
   InitializeShadowBaseAddress(MemToShadowSize(kHighMemEnd));