[JIT/llvm-rtdyld] Don't waste cycles invalidating the Instruction cache

[davide]

Summary

We call InvalidateInstructionCache() twice here and I think the second call is just wasted work. Hopefully my guess is correct.

[lhames]

lib/Support/Unix/Memory.inc

266–276

Whatever we do here should be consistent for all platforms. I haven't taken a good look at the icache invalidation code - I'll have a think about this tomorrow.

[davide]

Lang, what are your thoughts about this?
Should we invalidate the cache unconditionally instead?

Thanks!

[davide]

Updated after our discussion on IRC. Thanks!

[lhames]

Hi Davide,

I spoke with Jim Grosbach about this and we agreed that 'setExecutable' should do everything required to make the memory executable on the host processor. That means unconditionally setting the permissions and invalidating the icache (on processors that need that). The problem is with clients who call InvalidateInstructionCache after calling setExecutable: that should be redundant. Clients should either call setExecutable alone, or call both protectMappedMemory and InvalidateInstructionCache manually.

So - we should remove the redundant calls, and modify setExecutable where necessary to properly set the permissions.

[lhames]

This comment was removed by lhames.

[lhames]

LGTM, pending the removal of the icache invalidation for the data sections.

tools/llvm-rtdyld/llvm-rtdyld.cpp

432

The instruction cache doesn't need to be invalidated for data sections, so this loop should be removed.

lib/Support/Unix/Memory.inc

#if defined(__APPLE__) && (defined(__arm__) || defined(__arm64__))
   kern_return_t kr = vm_protect(mach_task_self(), (vm_address_t)M.Address,
     (vm_size_t)M.Size, 0, VM_PROT_READ | VM_PROT_WRITE);
   return KERN_SUCCESS == kr;
 #else
   return true;
 #endif
 }
 
 bool Memory::setExecutable (MemoryBlock &M, std::string *ErrMsg) {
-#if defined(__APPLE__) && (defined(__arm__) || defined(__arm64__))
   if (M.Address == 0 || M.Size == 0) return false;
   Memory::InvalidateInstructionCache(M.Address, M.Size);
+#if defined(__APPLE__) && (defined(__arm__) || defined(__arm64__))
   kern_return_t kr = vm_protect(mach_task_self(), (vm_address_t)M.Address,
     (vm_size_t)M.Size, 0, VM_PROT_READ | VM_PROT_EXECUTE | VM_PROT_COPY);
   return KERN_SUCCESS == kr;
-#elif defined(__arm__) || defined(__aarch64__)
-  Memory::InvalidateInstructionCache(M.Address, M.Size);
-  return true;
 #else
   return true;
 #endif
 }
 
 bool Memory::setRangeWritable(const void *Addr, size_t Size) {
 #if defined(__APPLE__) && (defined(__arm__) || defined(__arm64__))
   kern_return_t kr = vm_protect(mach_task_self(), (vm_address_t)Addr,
                                 (vm_size_t)Size, 0,
                                 VM_PROT_READ | VM_PROT_WRITE);
   return KERN_SUCCESS == kr;
 #else

tools/llvm-rtdyld/llvm-rtdyld.cpp

public:
 
   void *getPointerToNamedFunction(const std::string &Name,
                                   bool AbortOnFailure = true) override {
     return nullptr;
   }
 
   bool finalizeMemory(std::string *ErrMsg) override { return false; }
 
-  // Invalidate instruction cache for sections with execute permissions.
-  // Some platforms with separate data cache and instruction cache require
-  // explicit cache flush, otherwise JIT code manipulations (like resolved
-  // relocations) will get to the data cache but not to the instruction cache.
-  virtual void invalidateInstructionCache();
-
   void addDummySymbol(const std::string &Name, uint64_t Addr) {
     DummyExterns[Name] = Addr;
   }
 
   RuntimeDyld::SymbolInfo findSymbol(const std::string &Name) override {
     auto I = DummyExterns.find(Name);
 
     if (I != DummyExterns.end())
uint8_t *TrivialMemoryManager::allocateDataSection(uintptr_t Size,
   std::string Err;
   sys::MemoryBlock MB = sys::Memory::AllocateRWX(Size, nullptr, &Err);
   if (!MB.base())
     report_fatal_error("MemoryManager allocation failed: " + Err);
   DataMemory.push_back(MB);
   return (uint8_t*)MB.base();
 }
 
-void TrivialMemoryManager::invalidateInstructionCache() {
-  for (auto &FM : FunctionMemory)
-    sys::Memory::InvalidateInstructionCache(FM.base(), FM.size());
-
-  for (auto &DM : DataMemory)
-    sys::Memory::InvalidateInstructionCache(DM.base(), DM.size());
-}
-
 static const char *ProgramName;
 
 static void Message(const char *Type, const Twine &Msg) {
   errs() << ProgramName << ": " << Type << ": " << Msg << "\n";
 }
 
 static int Error(const Twine &Msg) {
   Message("error", Msg);
for (auto &File : InputFileList) {
 
     // Load the object file
     Dyld.loadObject(Obj);
     if (Dyld.hasError()) {
       return Error(Dyld.getErrorString());
     }
   }
 
-  // Resolve all the relocations we can.
-  Dyld.resolveRelocations();
-  // Clear instruction cache before code will be executed.
-  MemMgr.invalidateInstructionCache();
-
+  // Resove all the relocations we can.
   // FIXME: Error out if there are unresolved relocations.
+  Dyld.resolveRelocations();
 
   // Get the address of the entry point (_main by default).
   void *MainAddress = Dyld.getSymbolLocalAddress(EntryPoint);
   if (!MainAddress)
     return Error("no definition for '" + EntryPoint + "'");
 
   // Invalidate the instruction cache for each loaded function.
   for (auto &FM : MemMgr.FunctionMemory) {
+
     // Make sure the memory is executable.
+    // setExecutable will call InvalidateInstructionCache.
     std::string ErrorStr;
-    sys::Memory::InvalidateInstructionCache(FM.base(), FM.size());
     if (!sys::Memory::setExecutable(FM, &ErrorStr))
       return Error("unable to mark function executable: '" + ErrorStr + "'");
   }
+  for (auto &DM : MemMgr.DataMemory)
+    sys::Memory::InvalidateInstructionCache(DM.base(), DM.size());
 
   // Dispatch to _main().
   errs() << "loaded '" << EntryPoint << "' at: " << (void*)MainAddress << "\n";
 
   int (*Main)(int, const char**) =
     (int(*)(int,const char**)) uintptr_t(MainAddress);
   const char **Argv = new const char*[2];
   // Use the name of the first input object module as argv[0] for the target.