[unittest] Skip W+X MappedMemoryTests when MPROTECT is enabled

[mgorny]

Summary

Skip all MappedMemoryTest variants that rely on memory pages being
mapped for MF_WRITE|MF_EXEC when MPROTECT is enabled on NetBSD. W^X
protection causes all those mmap() calls to fail, causing the tests
to fail.

[krytarowski]

unittests/Support/MemoryTest.cpp

18 ↗

(On Diff #172505)

sys/param.h before sys/types.h according to the NetBSD style.

Please decorate this include section with
// clang-format off
and
// clang-format on

38 ↗

(On Diff #172505)

Using assert() might have a negative impact on functionality... because under some conditions this code might not be compiled at all and sysctl(3) won't be called.

I propose to go for:

if (sysctl(mib, 3, &paxflags, &len, NULL, 0) != 0)
  err(EXIT_FAILURE, "sysctl");

(and include <err.h>)

91 ↗

(On Diff #172505)

I would drop : \ here. Some compilers will complain about redundant ;.

223 ↗

(On Diff #172505)

Flags & Memory::MF_EXEC?

[krytarowski]

+ hfinkel for JIT API reference.

[mgorny]

Implemented all comments + moved CHECK_UNSUPPORTED below faster R+W checks.

[krytarowski]

unittests/Support/MemoryTest.cpp

90 ↗

(On Diff #172507)

I would further refactor this into:

// MPROTECT prevents W+X mmaps
#define CHECK_UNSUPPORTED(f) \
	​  do { \
	​    if ((f & (Memory::MF_WRITE | Memory::MF_EXEC)) && \
	​        IsMPROTECT()) \
	​      return; \
	​  } while (0)

And use CHECK_UNSUPPORTED(Flags) or CHECK_UNSUPPORTED(Flags | Memory::MF_WRITE) for EnabledWrite.

[mgorny]

unittests/Support/MemoryTest.cpp

90 ↗

(On Diff #172507)

Nah, that's too smart. Would go against KISS.

[krytarowski]

unittests/Support/MemoryTest.cpp

90 ↗

(On Diff #172507)

How about?

#define CHECK_UNSUPPORTED_RAW(f) \
  ​  do { \
  ​    if ((f & (Memory::MF_WRITE | Memory::MF_EXEC)) && \
  ​        IsMPROTECT()) \
  ​      return; \
  ​  } while (0)

and

#define CHECK_UNSUPPORTED() CHECK_UNSUPPORTED_RAW(Flag)

I would like to get rid of open-coding check in EnabledWrite.

In EnableWrite it would be: CHECK_UNSUPPORTED_RAW(Flags | Memory::MF_WRITE).

[mgorny]

unittests/Support/MemoryTest.cpp

90 ↗

(On Diff #172507)

Same problem — it says 'write' when it checks for 'exec'. We could get away by making an extra function with RHS specified via parameter with a default but I don't think it's worth the hassle. The purpose of the macro is to avoid repeating the most common version, not solve all the problems. Note that some functions have additional 'skip conditions' which we leave intact.

[krytarowski]

unittests/Support/MemoryTest.cpp

90 ↗

(On Diff #172507)

OK, I defer this to others in review.

[krytarowski]

@hfinkel I would still like to drop W|X support from MemoryBlock for everybody, restricting this class to swapping between RW and RX in JIT code. For those who need RWX allocations it's more convenient to add a dedicated allocator (and NetBSD can support it). It's just a matter of time when new features on security hardened OSes will break and lose feature parity.

[krytarowski]

@hfinkel any comments on this? If not I recommend to land this patch as it is.

llvm/trunk/unittests/Support/MemoryTest.cpp

 //===- llvm/unittest/Support/AllocatorTest.cpp - BumpPtrAllocator tests ---===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Support/Memory.h"
 #include "llvm/Support/Process.h"
 #include "gtest/gtest.h"
+#include <cassert>
 #include <cstdlib>
 
+#if defined(__NetBSD__)
+// clang-format off
+#include <sys/param.h>
+#include <sys/types.h>
+#include <sys/sysctl.h>
+#include <err.h>
+#include <unistd.h>
+// clang-format on
+#endif
+
 using namespace llvm;
 using namespace sys;
 
 namespace {
 
+bool IsMPROTECT() {
+#if defined(__NetBSD__)
+  int mib[3];
+  int paxflags;
+  size_t len = sizeof(paxflags);
+
+  mib[0] = CTL_PROC;
+  mib[1] = getpid();
+  mib[2] = PROC_PID_PAXFLAGS;
+
+  if (sysctl(mib, 3, &paxflags, &len, NULL, 0) != 0)
+    err(EXIT_FAILURE, "sysctl");
+
+  return !!(paxflags & CTL_PROC_PAXFLAGS_MPROTECT);
+#else
+  return false;
+#endif
+}
+
 class MappedMemoryTest : public ::testing::TestWithParam<unsigned> {
 public:
   MappedMemoryTest() {
     Flags = GetParam();
     PageSize = sys::Process::getPageSize();
   }
 
 protected:
bool doesOverlap(MemoryBlock M1, MemoryBlock M2) {
 
     return (unsigned char *)M1.base() + M1.size() > M2.base();
   }
 
   unsigned Flags;
   size_t   PageSize;
 };
 
+// MPROTECT prevents W+X mmaps
+#define CHECK_UNSUPPORTED() \
+  do { \
+    if ((Flags & Memory::MF_WRITE) && (Flags & Memory::MF_EXEC) && \
+        IsMPROTECT()) \
+      return; \
+  } while (0)
+
 TEST_P(MappedMemoryTest, AllocAndRelease) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(sizeof(int), nullptr, Flags,EC);
   EXPECT_EQ(std::error_code(), EC);
 
   EXPECT_NE((void*)nullptr, M1.base());
   EXPECT_LE(sizeof(int), M1.size());
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
 }
 
 TEST_P(MappedMemoryTest, MultipleAllocAndRelease) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(16, nullptr, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(64, nullptr, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(32, nullptr, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
 
TEST_P(MappedMemoryTest, MultipleAllocAndRelease) {
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, BasicWrite) {
   // This test applies only to readable and writeable combinations
   if (Flags &&
       !((Flags & Memory::MF_READ) && (Flags & Memory::MF_WRITE)))
     return;
+  CHECK_UNSUPPORTED();
 
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(sizeof(int), nullptr, Flags,EC);
   EXPECT_EQ(std::error_code(), EC);
 
   EXPECT_NE((void*)nullptr, M1.base());
   EXPECT_LE(sizeof(int), M1.size());
 
   int *a = (int*)M1.base();
   *a = 1;
   EXPECT_EQ(1, *a);
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
 }
 
 TEST_P(MappedMemoryTest, MultipleWrite) {
   // This test applies only to readable and writeable combinations
   if (Flags &&
       !((Flags & Memory::MF_READ) && (Flags & Memory::MF_WRITE)))
     return;
+  CHECK_UNSUPPORTED();
+
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(sizeof(int), nullptr, Flags,
                                                 EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(8 * sizeof(int), nullptr, Flags,
                                                 EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(4 * sizeof(int), nullptr, Flags,
TEST_P(MappedMemoryTest, MultipleWrite) {
   // Verify that M2 remains unaffected by other activity
   for (int i = 0; i < 8; i++) {
     EXPECT_EQ(i, y[i]);
   }
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, EnabledWrite) {
+  // MPROTECT prevents W+X, and since this test always adds W we need
+  // to block any variant with X.
+  if ((Flags & Memory::MF_EXEC) && IsMPROTECT())
+    return;
+
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(2 * sizeof(int), nullptr, Flags,
                                                 EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(8 * sizeof(int), nullptr, Flags,
                                                 EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(4 * sizeof(int), nullptr, Flags,
TEST_P(MappedMemoryTest, EnabledWrite) {
   x = (int*)M4.base();
   *x = 4;
   EXPECT_EQ(4, *x);
   EXPECT_FALSE(Memory::releaseMappedMemory(M4));
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, SuccessiveNear) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock M1 = Memory::allocateMappedMemory(16, nullptr, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(64, &M1, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(32, &M2, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
 
TEST_P(MappedMemoryTest, SuccessiveNear) {
   EXPECT_FALSE(doesOverlap(M1, M3));
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
   EXPECT_FALSE(Memory::releaseMappedMemory(M3));
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, DuplicateNear) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock Near((void*)(3*PageSize), 16);
   MemoryBlock M1 = Memory::allocateMappedMemory(16, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(64, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(32, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
 
   EXPECT_NE((void*)nullptr, M1.base());
   EXPECT_LE(16U, M1.size());
   EXPECT_NE((void*)nullptr, M2.base());
   EXPECT_LE(64U, M2.size());
   EXPECT_NE((void*)nullptr, M3.base());
   EXPECT_LE(32U, M3.size());
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
   EXPECT_FALSE(Memory::releaseMappedMemory(M3));
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, ZeroNear) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock Near(nullptr, 0);
   MemoryBlock M1 = Memory::allocateMappedMemory(16, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(64, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(32, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
TEST_P(MappedMemoryTest, ZeroNear) {
   EXPECT_FALSE(doesOverlap(M1, M3));
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
   EXPECT_FALSE(Memory::releaseMappedMemory(M3));
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, ZeroSizeNear) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock Near((void*)(4*PageSize), 0);
   MemoryBlock M1 = Memory::allocateMappedMemory(16, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M2 = Memory::allocateMappedMemory(64, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
   MemoryBlock M3 = Memory::allocateMappedMemory(32, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
TEST_P(MappedMemoryTest, ZeroSizeNear) {
   EXPECT_FALSE(doesOverlap(M1, M3));
 
   EXPECT_FALSE(Memory::releaseMappedMemory(M1));
   EXPECT_FALSE(Memory::releaseMappedMemory(M3));
   EXPECT_FALSE(Memory::releaseMappedMemory(M2));
 }
 
 TEST_P(MappedMemoryTest, UnalignedNear) {
+  CHECK_UNSUPPORTED();
   std::error_code EC;
   MemoryBlock Near((void*)(2*PageSize+5), 0);
   MemoryBlock M1 = Memory::allocateMappedMemory(15, &Near, Flags, EC);
   EXPECT_EQ(std::error_code(), EC);
 
   EXPECT_NE((void*)nullptr, M1.base());
   EXPECT_LE(sizeof(int), M1.size());