[WebAssembly] Add new pass to lower int/ptr conversions of reftypes

[pmatos]

Summary

Add new pass LowerRefTypesIntPtrConv to generate trap
instruction for an inttoptr and ptrtoint of a reference type instead
of erroring, since calling these instructions on non-integral pointers
has been since allowed (see ac81cb7e6).

[pmatos]

Unfortunately, I am still getting some terminator issues. I have a feeling my basic block splitting is not working properly. Still need to investigate further. Not ready to land.

[tlively]

Another approach would be to lower to DEBUG_UNREACHABLE, which is the normal unreachable instruction but not modeled as a terminator (see WebAssemblyInstrControl.td). I imagine you could use more standard ISel patterns for that lowering rather than introducing a separate pass.

[pmatos]

In D107102#2914633, @tlively wrote:

Another approach would be to lower to DEBUG_UNREACHABLE, which is the normal unreachable instruction but not modeled as a terminator (see WebAssemblyInstrControl.td). I imagine you could use more standard ISel patterns for that lowering rather than introducing a separate pass.

Didn't know about DEBUG_UNREACHABLE, thanks. However, I notice this is too low in the compiler. I need to generate IR instructions and DEBUG_UNREACHABLE seems to be a machine instruction. Still don't think however that using normal patterns would work. The issue has to do with the fact that inttoptr and ptrtoint instructions are lowered during ISel Lowering and don't reach pattern matching. So, once you go into SelectionDAGISel::runOnMachineFunction(MF);, the function visitPtrToInt will try to generate a TRUNCATE for a reference type which will assert because of the zero size. So anything we do here, should be done PreISel as far as I understand.

[tlively]

Oh right, that makes sense. I wonder if a custom DAG combine would be too late, too. At any rate, if you use the @llvm.debugtrap intrinsic in the IR replacement pass, it will lower to unreachable without needing to split any basic blocks.

[pmatos]

Generate a trap instruction to replace inttoptr and ptrtoint instead of unreachable

[pmatos]

@tlively wdyt?

[tlively]

Nice! LGTM with those cleanups applied.

llvm/lib/Target/WebAssembly/WebAssemblyLowerRefTypesIntPtrConv.cpp

63

dyn_cast should be pretty cheap, so I think it would be ok to just unconditionally do this second dyn_cast as well.

64–67

It's probably worth making a isRefType helper that checks both of these conditions.

[pmatos]

Cleanup code as suggested by @tlively

[pmatos]

I should have definitely used debugtrap instead of trap, which is still a terminator and expensive checks at the machine code level checked that there was a terminator in the middle of a basic block. *sigh*. Replacing with debugtrap as suggested by @tlively

llvm/lib/Target/WebAssembly/CMakeLists.txt

add_llvm_target(WebAssemblyCodeGen
   WebAssemblyFixFunctionBitcasts.cpp
   WebAssemblyFrameLowering.cpp
   WebAssemblyISelDAGToDAG.cpp
   WebAssemblyISelLowering.cpp
   WebAssemblyInstrInfo.cpp
   WebAssemblyLowerBrUnless.cpp
   WebAssemblyLowerEmscriptenEHSjLj.cpp
   WebAssemblyLowerGlobalDtors.cpp
+  WebAssemblyLowerRefTypesIntPtrConv.cpp
   WebAssemblyMachineFunctionInfo.cpp
   WebAssemblyMCInstLower.cpp
   WebAssemblyMCLowerPrePass.cpp
   WebAssemblyNullifyDebugValueLists.cpp
   WebAssemblyOptimizeLiveIntervals.cpp
   WebAssemblyOptimizeReturned.cpp
   WebAssemblyPeephole.cpp
   WebAssemblyPrepareForLiveIntervals.cpp

llvm/lib/Target/WebAssembly/WebAssembly.h

 
 // LLVM IR passes.
 ModulePass *createWebAssemblyLowerEmscriptenEHSjLj(bool EnableEH,
                                                    bool EnableSjLj);
 ModulePass *createWebAssemblyLowerGlobalDtors();
 ModulePass *createWebAssemblyAddMissingPrototypes();
 ModulePass *createWebAssemblyFixFunctionBitcasts();
 FunctionPass *createWebAssemblyOptimizeReturned();
+FunctionPass *createWebAssemblyLowerRefTypesIntPtrConv();
 
 // ISel and immediate followup passes.
 FunctionPass *createWebAssemblyISelDag(WebAssemblyTargetMachine &TM,
                                        CodeGenOpt::Level OptLevel);
 FunctionPass *createWebAssemblyArgumentMove();
 FunctionPass *createWebAssemblySetP2AlignOperands();
 
 // Late passes.

 void initializeWebAssemblyCFGSortPass(PassRegistry &);
 void initializeWebAssemblyCFGStackifyPass(PassRegistry &);
 void initializeWebAssemblyExplicitLocalsPass(PassRegistry &);
 void initializeWebAssemblyLowerBrUnlessPass(PassRegistry &);
 void initializeWebAssemblyRegNumberingPass(PassRegistry &);
 void initializeWebAssemblyDebugFixupPass(PassRegistry &);
 void initializeWebAssemblyPeepholePass(PassRegistry &);
 void initializeWebAssemblyMCLowerPrePassPass(PassRegistry &);
+void initializeWebAssemblyLowerRefTypesIntPtrConvPass(PassRegistry &);
 
 namespace WebAssembly {
 enum TargetIndex {
   // Followed by a local index (ULEB).
   TI_LOCAL,
   // Followed by an absolute global index (ULEB). DEPRECATED.
   TI_GLOBAL_FIXED,
   // Followed by the index from the bottom of the Wasm stack.

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

 #include "llvm/CodeGen/SelectionDAGISel.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Function.h" // To access function attributes.
 #include "llvm/IR/IntrinsicsWebAssembly.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/KnownBits.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/raw_ostream.h"
+
 using namespace llvm;
 
 #define DEBUG_TYPE "wasm-isel"
 
 //===--------------------------------------------------------------------===//
 /// WebAssembly-specific code to select WebAssembly machine instructions for
 /// SelectionDAG operations.
 ///
 namespace {
 class WebAssemblyDAGToDAGISel final : public SelectionDAGISel {
   /// Keep a pointer to the WebAssemblySubtarget around so that we can make the
   /// right decision when generating code for different targets.
   const WebAssemblySubtarget *Subtarget;
 
 public:
   WebAssemblyDAGToDAGISel(WebAssemblyTargetMachine &TM,
                           CodeGenOpt::Level OptLevel)
       : SelectionDAGISel(TM, OptLevel), Subtarget(nullptr) {
   }
 
   StringRef getPassName() const override {
     return "WebAssembly Instruction Selection";
   }
 
-  void checkForInvalidNodes(const Function &F) {
-    // This function will check for uses of ptrtoint on reference types and
-    // report a fatal error if these are found.
-    for (const BasicBlock &BB : F) {
-      for (const Instruction &I : BB) {
-        if (const PtrToIntInst *PTI = dyn_cast<const PtrToIntInst>(&I)) {
-          const Value *V = PTI->getPointerOperand();
-          if (WebAssemblyTargetLowering::isFuncrefType(V->getType()) ||
-              WebAssemblyTargetLowering::isExternrefType(V->getType()))
-            report_fatal_error("ptrtoint not allowed on reference types");
-        } else if (const IntToPtrInst *ITP = dyn_cast<const IntToPtrInst>(&I)) {
-          if (WebAssemblyTargetLowering::isFuncrefType(ITP->getDestTy()) ||
-              WebAssemblyTargetLowering::isExternrefType(ITP->getDestTy()))
-            report_fatal_error("inttoptr not allowed on reference types");
-        }
-      }
-    }
-  }
-
   bool runOnMachineFunction(MachineFunction &MF) override {
     LLVM_DEBUG(dbgs() << "********** ISelDAGToDAG **********\n"
                          "********** Function: "
                       << MF.getName() << '\n');
 
-    checkForInvalidNodes(MF.getFunction());
-
     Subtarget = &MF.getSubtarget<WebAssemblySubtarget>();
 
     return SelectionDAGISel::runOnMachineFunction(MF);
   }
 
   void PreprocessISelDAG() override;
 
   void Select(SDNode *Node) override;

llvm/lib/Target/WebAssembly/WebAssemblyLowerRefTypesIntPtrConv.cpp

+//=== WebAssemblyLowerRefTypesIntPtrConv.cpp -
+//                     Lower IntToPtr and PtrToInt on Reference Types   ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Lowers IntToPtr and PtrToInt instructions on reference types to
+/// Trap instructions since they have been allowed to operate
+/// on non-integral pointers.
+///
+//===----------------------------------------------------------------------===//
+
+#include "WebAssembly.h"
+#include "WebAssemblySubtarget.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/Pass.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "wasm-lower-reftypes-intptr-conv"
+
+namespace {
+class WebAssemblyLowerRefTypesIntPtrConv final : public FunctionPass {
+  StringRef getPassName() const override {
+    return "WebAssembly Lower RefTypes Int-Ptr Conversions";
+  }
+
+  bool runOnFunction(Function &MF) override;
+
+public:
+  static char ID; // Pass identification
+  WebAssemblyLowerRefTypesIntPtrConv() : FunctionPass(ID) {}
+};
+} // end anonymous namespace
+
+char WebAssemblyLowerRefTypesIntPtrConv::ID = 0;
+INITIALIZE_PASS(WebAssemblyLowerRefTypesIntPtrConv, DEBUG_TYPE,
+                "WebAssembly Lower RefTypes Int-Ptr Conversions", false, false)
+
+FunctionPass *llvm::createWebAssemblyLowerRefTypesIntPtrConv() {
+  return new WebAssemblyLowerRefTypesIntPtrConv();
+}
+
+bool WebAssemblyLowerRefTypesIntPtrConv::runOnFunction(Function &F) {
+  LLVM_DEBUG(dbgs() << "********** Lower RefTypes IntPtr Convs **********\n"
+                       "********** Function: "
+                    << F.getName() << '\n');
+
+  // This function will check for uses of ptrtoint and inttoptr on reference
+  // types and replace them with a trap instruction.
+  //
+  // We replace the instruction by a trap instruction
+  // and its uses by null in the case of inttoptr and 0 in the
+  // case of ptrtoint.
+  std::set<Instruction *> worklist;
+
+  for (inst_iterator I = inst_begin(F), E = inst_end(F); I != E; ++I) {
+    PtrToIntInst *PTI = dyn_cast<PtrToIntInst>(&*I);
+    IntToPtrInst *ITP = PTI ? 0 : dyn_cast<IntToPtrInst>(&*I);
+    if (!(PTI && (WebAssemblyTargetLowering::isFuncrefType(
+                      PTI->getPointerOperand()->getType()) ||
+                  WebAssemblyTargetLowering::isExternrefType(
+                      PTI->getPointerOperand()->getType()))) &&
+        !(ITP &&
+          (WebAssemblyTargetLowering::isFuncrefType(ITP->getDestTy()) ||
+           WebAssemblyTargetLowering::isExternrefType(ITP->getDestTy()))))
+      continue;
+
+    UndefValue *U = UndefValue::get(I->getType());
+    I->replaceAllUsesWith(U);
+
+    Function *TrapIntrin =
+        Intrinsic::getDeclaration(F.getParent(), Intrinsic::trap);
+    CallInst::Create(TrapIntrin, {}, "", &*I);
+
+    worklist.insert(&*I);
+  }
+
+  // erase each instruction replaced by trap
+  for (Instruction *I : worklist)
+    I->eraseFromParent();
+
+  return !worklist.empty();
+}

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

public:
 
   FunctionPass *createTargetRegisterAllocator(bool) override;
 
   void addIRPasses() override;
   bool addInstSelector() override;
   void addPostRegAlloc() override;
   bool addGCPasses() override { return false; }
   void addPreEmitPass() override;
+  bool addPreISel() override;
 
   // No reg alloc
   bool addRegAssignAndRewriteFast() override { return false; }
 
   // No reg alloc
   bool addRegAssignAndRewriteOptimized() override { return false; }
 };
 } // end anonymous namespace
void WebAssemblyPassConfig::addPreEmitPass() {
   // Fix debug_values whose defs have been stackified.
   if (!WasmDisableExplicitLocals)
     addPass(createWebAssemblyDebugFixup());
 
   // Collect information to prepare for MC lowering / asm printing.
   addPass(createWebAssemblyMCLowerPrePass());
 }
 
+bool WebAssemblyPassConfig::addPreISel() {
+  TargetPassConfig::addPreISel();
+  addPass(createWebAssemblyLowerRefTypesIntPtrConv());
+  return false;
+}
+
 yaml::MachineFunctionInfo *
 WebAssemblyTargetMachine::createDefaultFuncInfoYAML() const {
   return new yaml::WebAssemblyFunctionInfo();
 }
 
 yaml::MachineFunctionInfo *WebAssemblyTargetMachine::convertFuncInfoToYAML(
     const MachineFunction &MF) const {
   const auto *MFI = MF.getInfo<WebAssemblyFunctionInfo>();

llvm/test/CodeGen/WebAssembly/externref-inttoptr.ll

-; RUN: not --crash llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types 2>&1 | FileCheck %s
 
 %extern = type opaque
 %externref = type %extern addrspace(10)*
 
 define %externref @int_to_externref(i32 %i) {
   %ref = inttoptr i32 %i to %externref
   ret %externref %ref
 }
 
-; CHECK-ERROR: LLVM ERROR: inttoptr not allowed on reference types
+
+; CHECK-LABEL: int_to_externref:
+; CHECK-NEXT: .functype       int_to_externref (i32) -> (externref)
+; CHECK-NEXT: .local externref
+; CHECK-NEXT: unreachable
+; CHECK-NEXT: local.get 1
+; CHECK-NEXT: end_function

llvm/test/CodeGen/WebAssembly/externref-ptrtoint.ll

-; RUN: not --crash llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types 2>&1 | FileCheck %s
 
 %extern = type opaque
 %externref = type %extern addrspace(10)*
 
 define i32 @externref_to_int(%externref %ref) {
   %i = ptrtoint %externref %ref to i32
   ret i32 %i
 }
 
-; CHECK-ERROR: LLVM ERROR: ptrtoint not allowed on reference types
+; CHECK-LABEL: externref_to_int:
+; CHECK-NEXT: .functype       externref_to_int (externref) -> (i32)
+; CHECK-NEXT: .local i32
+; CHECK-NEXT: unreachable
+; CHECK-NEXT: local.get 1
+; CHECK-NEXT: end_function

llvm/utils/gn/secondary/llvm/lib/Target/WebAssembly/BUILD.gn

sources = [
     "WebAssemblyFrameLowering.cpp",
     "WebAssemblyISelDAGToDAG.cpp",
     "WebAssemblyISelLowering.cpp",
     "WebAssemblyInstrInfo.cpp",
     "WebAssemblyLateEHPrepare.cpp",
     "WebAssemblyLowerBrUnless.cpp",
     "WebAssemblyLowerEmscriptenEHSjLj.cpp",
     "WebAssemblyLowerGlobalDtors.cpp",
+    "WebAssemblyLowerRefTypesIntPtrConv.cpp",
     "WebAssemblyMCInstLower.cpp",
     "WebAssemblyMCLowerPrePass.cpp",
     "WebAssemblyMachineFunctionInfo.cpp",
     "WebAssemblyMemIntrinsicResults.cpp",
     "WebAssemblyNullifyDebugValueLists.cpp",
     "WebAssemblyOptimizeLiveIntervals.cpp",
     "WebAssemblyOptimizeReturned.cpp",
     "WebAssemblyPeephole.cpp",