WIP: [clang-repl] Basic WebAssembly support for running inside a JS engine

[argentite]

Summary

This commit introduces support for running clang-repl and executing C++
code interactively inside a Javascript engine using WebAssembly when
built with Emscripten. This is achieved by producing WASM "shared
libraries" that can be loaded by the Emscripten runtime using dlopen()

[argentite]

This cannot built yet because it depends on lld which does not seem to usable from clang directly. I am looking for feedback/ideas on how to proceed.

[v.g.vassilev]

This is quite exciting! Most of it looks already good enough to me. See inline comment.

clang/lib/Interpreter/WASM.cpp

79

I am not sure how we can solve that dependency here. Worst case scenario, could we check if lld is installed and make a system call?

[argentite]

clang/lib/Interpreter/WASM.cpp

79

AFAIK we can't really exec() within Emscripten.

[v.g.vassilev]

Let's add more reviewers to see if we can figure out how to move forward here.

[argentite]

Here's a demo of it in action with clang and lld linked statically out of tree. https://wasmdemo.argentite.me

[sbc100]

Very nice idea!

clang/lib/Interpreter/CMakeLists.txt

21

Elsewhere in LLVM either use WebAssembly (spelled out). If you really prefer to keep it short you should use Wasm (not all caps).

Also, I wonder if we should call this Emscripten given that it seems emscripten specific.. then you could have another one called WASI or WebAssembly for the less specific one?

clang/lib/Interpreter/WASM.cpp

79

This looks its using the in-process call to the linker library, rather then an exec of the linker process.. which could work. But is it OK to have the compiler depend on the linker like this?

[sbc100]

I suppose you could build a similar thing that runs under node and uses that wasi-unknown-unknown target?

[v.g.vassilev]

clang/lib/Interpreter/WASM.cpp

79

Well, this is what we would like to avoid. Somehow. Initially, I thought we could move the relevant lld bits in the ORC infrastructure but maybe that’s not a good idea after all. Is there a suitable place where we could move that lld logic in LLVM?

[sbc100]

ood idea after all. Is there a suitable place where we could mo

clang/lib/Interpreter/WASM.cpp

79

Perhaps you could wrap all of this in if __EMSCRIPTEN__ or even something like if STATICALLY_LINKED_LLD so it would only be available in your special build of the compiler?

[v.g.vassilev]

clang/lib/Interpreter/WASM.cpp

79

@argentite could we try @sbc100's proposal?

[argentite]

Yes, I think that's the simplest solution. __EMSCRIPTEN__ probably won't work because somebody may want to build for Emscripten without lld. STATICALLY_LINKED_LLD would probably be better. But I could not figure out how to do that in the build system.

clang/lib/Interpreter/CMakeLists.txt

 
 add_clang_library(clangInterpreter
   DeviceOffload.cpp
   IncrementalExecutor.cpp
   IncrementalParser.cpp
   Interpreter.cpp
   InterpreterUtils.cpp
   Value.cpp
+  WASM.cpp
 
   DEPENDS
   intrinsics_gen
   ClangDriverOptions
 
   LINK_LIBS
   clangAST
   clangAnalysis

clang/lib/Interpreter/IncrementalExecutor.h

 class IncrementalExecutor {
   using CtorDtorIterator = llvm::orc::CtorDtorIterator;
   std::unique_ptr<llvm::orc::LLJIT> Jit;
   llvm::orc::ThreadSafeContext &TSCtx;
 
   llvm::DenseMap<const PartialTranslationUnit *, llvm::orc::ResourceTrackerSP>
       ResourceTrackers;
 
+protected:
+  IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC);
+
 public:
   enum SymbolNameKind { IRName, LinkerName };
 
   IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC, llvm::Error &Err,
                       const clang::TargetInfo &TI);
-  ~IncrementalExecutor();
+  virtual ~IncrementalExecutor();
 
-  llvm::Error addModule(PartialTranslationUnit &PTU);
-  llvm::Error removeModule(PartialTranslationUnit &PTU);
-  llvm::Error runCtors() const;
+  virtual llvm::Error addModule(PartialTranslationUnit &PTU);
+  virtual llvm::Error removeModule(PartialTranslationUnit &PTU);
+  virtual llvm::Error runCtors() const;
   llvm::Error cleanUp();
   llvm::Expected<llvm::orc::ExecutorAddr>
   getSymbolAddress(llvm::StringRef Name, SymbolNameKind NameKind) const;
 
   llvm::orc::LLJIT &GetExecutionEngine() { return *Jit; }
 };
 
 } // end namespace clang
 
 #endif // LLVM_CLANG_LIB_INTERPRETER_INCREMENTALEXECUTOR_H

clang/lib/Interpreter/IncrementalExecutor.cpp

 
 // Force linking some of the runtimes that helps attaching to a debugger.
 LLVM_ATTRIBUTE_USED void linkComponents() {
   llvm::errs() << (void *)&llvm_orc_registerJITLoaderGDBWrapper
                << (void *)&llvm_orc_registerJITLoaderGDBAllocAction;
 }
 
 namespace clang {
+IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC)
+    : TSCtx(TSC) {}
 
 IncrementalExecutor::IncrementalExecutor(llvm::orc::ThreadSafeContext &TSC,
                                          llvm::Error &Err,
                                          const clang::TargetInfo &TI)
     : TSCtx(TSC) {
   using namespace llvm::orc;
   llvm::ErrorAsOutParameter EAO(&Err);
 

clang/lib/Interpreter/Interpreter.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Interpreter/Interpreter.h"
 
 #include "DeviceOffload.h"
 #include "IncrementalExecutor.h"
 #include "IncrementalParser.h"
+#include "WASM.h"
 
 #include "InterpreterUtils.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Mangle.h"
 #include "clang/AST/TypeVisitor.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CodeGenAction.h"
IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
   return CreateCI(**ErrOrCC1Args);
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
 IncrementalCompilerBuilder::CreateCpp() {
   std::vector<const char *> Argv;
   Argv.reserve(5 + 1 + UserArgs.size());
   Argv.push_back("-xc++");
+#ifdef __EMSCRIPTEN__
+  Argv.push_back("-target");
+  Argv.push_back("wasm32-unknown-emscripten");
+  Argv.push_back("-pie");
+  Argv.push_back("-shared");
+#endif
   Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end());
 
   return IncrementalCompilerBuilder::create(Argv);
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
 IncrementalCompilerBuilder::createCuda(bool device) {
   std::vector<const char *> Argv;
if (llvm::Error Err = IncrExecutor->cleanUp())
           toString(std::move(Err)));
   }
 }
 
 // These better to put in a runtime header but we can't. This is because we
 // can't find the precise resource directory in unittests so we have to hard
 // code them.
 const char *const Runtimes = R"(
-    void* operator new(__SIZE_TYPE__, void* __p) noexcept;
     void *__clang_Interpreter_SetValueWithAlloc(void*, void*, void*);
     void __clang_Interpreter_SetValueNoAlloc(void*, void*, void*);
     void __clang_Interpreter_SetValueNoAlloc(void*, void*, void*, void*);
     void __clang_Interpreter_SetValueNoAlloc(void*, void*, void*, float);
     void __clang_Interpreter_SetValueNoAlloc(void*, void*, void*, double);
     void __clang_Interpreter_SetValueNoAlloc(void*, void*, void*, long double);
     void __clang_Interpreter_SetValueNoAlloc(void*,void*,void*,unsigned long long);
     template <class T, class = T (*)() /*disable for arrays*/>
Interpreter::Parse(llvm::StringRef Code) {
   // Tell the interpreter sliently ignore unused expressions since value
   // printing could cause it.
   getCompilerInstance()->getDiagnostics().setSeverity(
       clang::diag::warn_unused_expr, diag::Severity::Ignored, SourceLocation());
   return IncrParser->Parse(Code);
 }
 
 llvm::Error Interpreter::CreateExecutor() {
+  llvm::Error Err = llvm::Error::success();
+#ifdef __EMSCRIPTEN__
+  auto Executor = std::make_unique<WASMIncrementalExecutor>(*TSCtx);
+#else
   const clang::TargetInfo &TI =
       getCompilerInstance()->getASTContext().getTargetInfo();
-  llvm::Error Err = llvm::Error::success();
   auto Executor = std::make_unique<IncrementalExecutor>(*TSCtx, Err, TI);
+#endif
   if (!Err)
     IncrExecutor = std::move(Executor);
 
   return Err;
 }
 
 llvm::Error Interpreter::Execute(PartialTranslationUnit &T) {
   assert(T.TheModule);

clang/lib/Interpreter/WASM.h

+//===------------------ WASM.h - WASM Interpreter ---------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements interpreter support for code execution in WebAssembly.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LIB_INTERPRETER_WASM_H
+#define LLVM_CLANG_LIB_INTERPRETER_WASM_H
+
+#include "IncrementalExecutor.h"
+
+namespace clang {
+
+class WASMIncrementalExecutor : public IncrementalExecutor {
+public:
+  WASMIncrementalExecutor(llvm::orc::ThreadSafeContext &TSC);
+
+  llvm::Error addModule(PartialTranslationUnit &PTU) override;
+  llvm::Error removeModule(PartialTranslationUnit &PTU) override;
+  llvm::Error runCtors() const override;
+
+  ~WASMIncrementalExecutor() override;
+};
+
+} // namespace clang
+
+#endif // LLVM_CLANG_LIB_INTERPRETER_WASM_H

clang/lib/Interpreter/WASM.cpp

+//===----------------- WASM.cpp - WASM Interpreter --------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements interpreter support for code execution in WebAssembly.
+//
+//===----------------------------------------------------------------------===//
+
+#include "WASM.h"
+#include "IncrementalExecutor.h"
+
+#include <llvm/IR/LegacyPassManager.h>
+#include <llvm/IR/Module.h>
+#include <llvm/MC/TargetRegistry.h>
+#include <llvm/Target/TargetMachine.h>
+
+#include <clang/Interpreter/Interpreter.h>
+
+#include <dlfcn.h>
+
+namespace clang {
+
+WASMIncrementalExecutor::WASMIncrementalExecutor(
+    llvm::orc::ThreadSafeContext &TSC)
+    : IncrementalExecutor(TSC) {}
+
+llvm::Error WASMIncrementalExecutor::addModule(PartialTranslationUnit &PTU) {
+  PTU.TheModule->dump();
+
+  std::string ErrorString;
+
+  const llvm::Target *Target = llvm::TargetRegistry::lookupTarget(
+      PTU.TheModule->getTargetTriple(), ErrorString);
+  if (!Target) {
+    return llvm::make_error<llvm::StringError>("Failed to create WASM Target: ",
+                                               llvm::inconvertibleErrorCode());
+  }
+
+  llvm::TargetOptions TO = llvm::TargetOptions();
+  llvm::TargetMachine *TargetMachine = Target->createTargetMachine(
+      PTU.TheModule->getTargetTriple(), "", "", TO, llvm::Reloc::Model::PIC_);
+  PTU.TheModule->setDataLayout(TargetMachine->createDataLayout());
+  std::string OutputFileName = PTU.TheModule->getName().str() + ".wasm";
+
+  std::error_code Error;
+  llvm::raw_fd_ostream OutputFile(llvm::StringRef(OutputFileName), Error);
+
+  llvm::legacy::PassManager PM;
+  if (TargetMachine->addPassesToEmitFile(PM, OutputFile, nullptr,
+                                         llvm::CGFT_ObjectFile)) {
+    return llvm::make_error<llvm::StringError>(
+        "WASM backend cannot produce object.", llvm::inconvertibleErrorCode());
+  }
+
+  if (!PM.run(*PTU.TheModule)) {
+
+    return llvm::make_error<llvm::StringError>("Failed to emit WASM object.",
+                                               llvm::inconvertibleErrorCode());
+  }
+
+  OutputFile.close();
+
+  std::vector<const char *> LinkerArgs = {"wasm-ld",
+                                          "-pie",
+                                          "--import-memory",
+                                          "--no-entry",
+                                          "--export-all",
+                                          "--experimental-pic",
+                                          "--no-export-dynamic",
+                                          "--stack-first",
+                                          OutputFileName.c_str(),
+                                          "-o",
+                                          OutputFileName.c_str()};
+  int Result =
+      lld::wasm::link(LinkerArgs, llvm::outs(), llvm::errs(), false, false);
+  if (!Result)
+    return llvm::make_error<llvm::StringError>(
+        "Failed to link incremental module", llvm::inconvertibleErrorCode());
+
+  void *LoadedLibModule =
+      dlopen(OutputFileName.c_str(), RTLD_NOW | RTLD_GLOBAL);
+  if (LoadedLibModule == nullptr) {
+    llvm::errs() << dlerror() << '\n';
+    return llvm::make_error<llvm::StringError>(
+        "Failed to load incremental module", llvm::inconvertibleErrorCode());
+  }
+
+  return llvm::Error::success();
+}
+
+llvm::Error WASMIncrementalExecutor::removeModule(PartialTranslationUnit &PTU) {
+  return llvm::make_error<llvm::StringError>("Not implemented yet",
+                                             llvm::inconvertibleErrorCode());
+}
+
+llvm::Error WASMIncrementalExecutor::runCtors() const {
+  // This seems to be automatically done when using dlopen()
+  return llvm::Error::success();
+}
+
+WASMIncrementalExecutor::~WASMIncrementalExecutor() = default;
+
+} // namespace clang