[mlir][test] Require JIT support in JIT tests

[ro]

Summary

A number of mlir tests FAIL on Solaris/sparcv9 with Target has no JIT support. This patch fixes that by mimicing clang/test/lit.cfg.py which implements a host-supports-jit keyword for this. The gtest-based unit tests don't support REQUIRES:, so lack of support needs to be hardcoded there.

Tested on amd64-pc-solaris2.11 (check-mlir results unchanged) and sparcv9-sun-solaris2.11 (only one unrelated failure left).

[mehdi_amini]

The dependency on clang-repl does not seems like something we should have here.
I'd rather define this host-supports-jit by checking native and filtering the known unsupported targets.

[mehdi_amini]

Alternatively, what about adding the host-supports-jit feature from clang-rep to mlir-cpu-runner itself? Seems like a handful of lines from what I can see.

[ro]

In D131151#3699052, @mehdi_amini wrote:

Alternatively, what about adding the host-supports-jit feature from clang-rep to mlir-cpu-runner itself? Seems like a handful of lines from what I can see.

Seems like the better option to me, rather than having multiple (possibly diverging) hardcoded exclude lists. I'll have a look.

[ro]

• Move --host-supports-jit to mlir::JitRunnerMain.
• Use it via mlir-cpu-runner.

Had to do it this way because mlir::JitRunnerMain calls llvm::cl::ParseCommandLineOptions: an earlier call in mlir-cpu-runner.cpp didn't work.

[ro]

Ping? There's been no response since I updated the patch as requested almost two weeks ago. It would be good to get this into the 15.x branch, too. Thanks.

[mehdi_amini]

Apologies: I wrote this inline comment and I thought I approved but didn't click "submit" at the time apparently.

mlir/lib/ExecutionEngine/JitRunner.cpp

91

Can you add a description for the flag?

[mehdi_amini]

Also: LLVM policy is to ping patches every week to get updates, but feel free to ping me more quickly: in particular when there is a timely concern like a release branch involved.

[stellaraccident]

Thanks and sorry for missing this. Ditto to what Mehdi said about pinging for things that have direct impacts like this.

[ro]

In D131151#3728436, @mehdi_amini wrote:

Apologies: I wrote this inline comment and I thought I approved but didn't click "submit" at the time apparently.

No worries: I'm on vacation right now and have been away for a week.

In D131151#3728440, @mehdi_amini wrote:

Also: LLVM policy is to ping patches every week to get updates, but feel free to ping me more quickly: in particular when there is a timely concern like a release branch involved.

I know, but sometimes forget. On top of that, flang/mlir are currently tested only for releases, thus they aren't as crucial as the rest of LLVM.

mlir/lib/ExecutionEngine/JitRunner.cpp

91

Sure. However, it will have no visible effect given the use of llvm::cl::Hidden, which I took from clang-repl.

[ro]

Add description for --host-supports-jit.

mlir/lib/ExecutionEngine/JitRunner.cpp

 #include "mlir/ExecutionEngine/OptUtils.h"
 #include "mlir/IR/BuiltinTypes.h"
 #include "mlir/IR/MLIRContext.h"
 #include "mlir/Parser/Parser.h"
 #include "mlir/Support/FileUtilities.h"
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ExecutionEngine/Orc/JITTargetMachineBuilder.h"
+#include "llvm/ExecutionEngine/Orc/LLJIT.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/LegacyPassNameParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileUtilities.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/StringSaver.h"
 #include "llvm/Support/ToolOutputFile.h"
struct Options {
   llvm::cl::opt<bool> dumpObjectFile{
       "dump-object-file",
       llvm::cl::desc("Dump JITted-compiled object to file specified with "
                      "-object-filename (<input file>.o by default).")};
 
   llvm::cl::opt<std::string> objectFilename{
       "object-filename",
       llvm::cl::desc("Dump JITted-compiled object to file <input file>.o")};
+
+  llvm::cl::opt<bool> hostSupportsJit{"host-supports-jit",
+                                      llvm::cl::desc("Report host JIT support"),
+                                      llvm::cl::Hidden};
 };
 
 struct CompileAndExecuteConfig {
   /// LLVM module transformer that is passed to ExecutionEngine.
   std::function<llvm::Error(llvm::Module *)> transformer;
 
   /// A custom function that is passed to ExecutionEngine. It processes MLIR
   /// module and creates LLVM IR module.

 /// standard C++ main functions.
 int mlir::JitRunnerMain(int argc, char **argv, const DialectRegistry &registry,
                         JitRunnerConfig config) {
   // Create the options struct containing the command line options for the
   // runner. This must come before the command line options are parsed.
   Options options;
   llvm::cl::ParseCommandLineOptions(argc, argv, "MLIR CPU execution driver\n");
 
+  if (options.hostSupportsJit) {
+    auto J = llvm::orc::LLJITBuilder().create();
+    if (J)
+      llvm::outs() << "true\n";
+    else {
+      llvm::consumeError(J.takeError());
+      llvm::outs() << "false\n";
+    }
+    return 0;
+  }
+
   Optional<unsigned> optLevel = getCommandLineOptLevel(options);
   SmallVector<std::reference_wrapper<llvm::cl::opt<bool>>, 4> optFlags{
       options.optO0, options.optO1, options.optO2, options.optO3};
 
   MLIRContext context(registry);
 
   auto m = parseMLIRInput(options.inputFilename, &context);
   if (!m) {

mlir/test/CAPI/execution_engine.c

 //===- execution_engine.c - Test for the C bindings for the MLIR JIT-------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM
 // Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 /* RUN: mlir-capi-execution-engine-test 2>&1 | FileCheck %s
  */
-/* REQUIRES: native
+/* REQUIRES: host-supports-jit
  */
 
 #include "mlir-c/Conversion.h"
 #include "mlir-c/ExecutionEngine.h"
 #include "mlir-c/IR.h"
 #include "mlir-c/RegisterEverything.h"
 
 #include <assert.h>

mlir/test/lit.cfg.py

llvm_config.with_environment('PYTHONPATH', [
         os.path.join(config.mlir_obj_root, 'python_packages', 'mlir_core'),
         os.path.join(config.mlir_obj_root, 'python_packages', 'mlir_test'),
     ], append_path=True)
 
 if config.enable_assertions:
     config.available_features.add('asserts')
 else:
     config.available_features.add('noasserts')
+
+def have_host_jit_feature_support(feature_name):
+    mlir_cpu_runner_exe = lit.util.which('mlir-cpu-runner', config.mlir_tools_dir)
+
+    if not mlir_cpu_runner_exe:
+        return False
+
+    try:
+        mlir_cpu_runner_cmd = subprocess.Popen(
+            [mlir_cpu_runner_exe, '--host-supports-' + feature_name], stdout=subprocess.PIPE)
+    except OSError:
+        print('could not exec mlir-cpu-runner')
+        return False
+
+    mlir_cpu_runner_out = mlir_cpu_runner_cmd.stdout.read().decode('ascii')
+    mlir_cpu_runner_cmd.wait()
+
+    return 'true' in mlir_cpu_runner_out
+
+if have_host_jit_feature_support('jit'):
+    config.available_features.add('host-supports-jit')

mlir/test/mlir-cpu-runner/lit.local.cfg

 import sys
 
 # FIXME: llvm orc does not support the COFF rtld.
 if sys.platform == 'win32':
     config.unsupported = True
 
 # MSAN does not work with JIT.
 if 'msan' in config.available_features:
   config.unsupported = True
 
 # Requires native execution.
-if 'native' not in config.available_features:
+if 'host-supports-jit' not in config.available_features:
     config.unsupported = True
 
 config.available_features.add(
         config.root.native_target.lower() + '-native-target')

mlir/test/python/execution_engine.py

 # RUN: %PYTHON %s 2>&1 | FileCheck %s
-# REQUIRES: native
+# REQUIRES: host-supports-jit
 import gc, sys
 from mlir.ir import *
 from mlir.passmanager import *
 from mlir.execution_engine import *
 from mlir.runtime import *
 
 
 # Log everything to stderr and flush so that we have a unified stream to match

mlir/unittests/ExecutionEngine/Invoke.cpp

 #include "mlir/Pass/PassManager.h"
 #include "mlir/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.h"
 #include "mlir/Target/LLVMIR/Export.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/raw_ostream.h"
 
 #include "gmock/gmock.h"
 
+// SPARC currently lacks JIT support.
+#ifdef __sparc__
+#define SKIP_WITHOUT_JIT(x) DISABLED_##x
+#else
+#define SKIP_WITHOUT_JIT(x) x
+#endif
+
 using namespace mlir;
 
 // The JIT isn't supported on Windows at that time
 #ifndef _WIN32
 
 static struct LLVMInitializer {
   LLVMInitializer() {
     llvm::InitializeNativeTarget();
     llvm::InitializeNativeTargetAsmPrinter();
   }
 } initializer;
 
 /// Simple conversion pipeline for the purpose of testing sources written in
 /// dialects lowering to LLVM Dialect.
 static LogicalResult lowerToLLVMDialect(ModuleOp module) {
   PassManager pm(module.getContext());
   pm.addPass(mlir::createMemRefToLLVMPass());
   pm.addNestedPass<func::FuncOp>(
       mlir::arith::createConvertArithmeticToLLVMPass());
   pm.addPass(mlir::createConvertFuncToLLVMPass());
   pm.addPass(mlir::createReconcileUnrealizedCastsPass());
   return pm.run(module);
 }
 
-TEST(MLIRExecutionEngine, AddInteger) {
+TEST(MLIRExecutionEngine, SKIP_WITHOUT_JIT(AddInteger)) {
   std::string moduleStr = R"mlir(
   func.func @foo(%arg0 : i32) -> i32 attributes { llvm.emit_c_interface } {
     %res = arith.addi %arg0, %arg0 : i32
     return %res : i32
   }
   )mlir";
   DialectRegistry registry;
   registerAllDialects(registry);
TEST(MLIRExecutionEngine, SKIP_WITHOUT_JIT(AddInteger)) {
   // The result of the function must be passed as output argument.
   int result = 0;
   llvm::Error error =
       jit->invoke("foo", 42, ExecutionEngine::Result<int>(result));
   ASSERT_TRUE(!error);
   ASSERT_EQ(result, 42 + 42);
 }
 
-TEST(MLIRExecutionEngine, SubtractFloat) {
+TEST(MLIRExecutionEngine, SKIP_WITHOUT_JIT(SubtractFloat)) {
   std::string moduleStr = R"mlir(
   func.func @foo(%arg0 : f32, %arg1 : f32) -> f32 attributes { llvm.emit_c_interface } {
     %res = arith.subf %arg0, %arg1 : f32
     return %res : f32
   }
   )mlir";
   DialectRegistry registry;
   registerAllDialects(registry);
TEST(MLIRExecutionEngine, SKIP_WITHOUT_JIT(SubtractFloat)) {
   // The result of the function must be passed as output argument.
   float result = -1;
   llvm::Error error =
       jit->invoke("foo", 43.0f, 1.0f, ExecutionEngine::result(result));
   ASSERT_TRUE(!error);
   ASSERT_EQ(result, 42.f);
 }
 
-TEST(NativeMemRefJit, ZeroRankMemref) {
+TEST(NativeMemRefJit, SKIP_WITHOUT_JIT(ZeroRankMemref)) {
   OwningMemRef<float, 0> a({});
   a[{}] = 42.;
   ASSERT_EQ(*a->data, 42);
   a[{}] = 0;
   std::string moduleStr = R"mlir(
   func.func @zero_ranked(%arg0 : memref<f32>) attributes { llvm.emit_c_interface } {
     %cst42 = arith.constant 42.0 : f32
     memref.store %cst42, %arg0[] : memref<f32>
TEST(NativeMemRefJit, SKIP_WITHOUT_JIT(ZeroRankMemref)) {
 
   llvm::Error error = jit->invoke("zero_ranked", &*a);
   ASSERT_TRUE(!error);
   EXPECT_EQ((a[{}]), 42.);
   for (float &elt : *a)
     EXPECT_EQ(&elt, &(a[{}]));
 }
 
-TEST(NativeMemRefJit, RankOneMemref) {
+TEST(NativeMemRefJit, SKIP_WITHOUT_JIT(RankOneMemref)) {
   int64_t shape[] = {9};
   OwningMemRef<float, 1> a(shape);
   int count = 1;
   for (float &elt : *a) {
     EXPECT_EQ(&elt, &(a[{count - 1}]));
     elt = count++;
   }
 
for (float &elt : *a) {
     if (count == 6)
       EXPECT_EQ(elt, 42.);
     else
       EXPECT_EQ(elt, count);
     count++;
   }
 }
 
-TEST(NativeMemRefJit, BasicMemref) {
+TEST(NativeMemRefJit, SKIP_WITHOUT_JIT(BasicMemref)) {
   constexpr int k = 3;
   constexpr int m = 7;
   // Prepare arguments beforehand.
   auto init = [=](float &elt, ArrayRef<int64_t> indices) {
     assert(indices.size() == 2);
     elt = m * indices[0] + indices[1];
   };
   int64_t shape[] = {k, m};
static void memrefMultiply(::StridedMemRefType<float, 2> *memref,
   for (float &elt : *memref)
     elt *= coefficient;
 }
 
 // MSAN does not work with JIT.
 #if __has_feature(memory_sanitizer)
 #define MAYBE_JITCallback DISABLED_JITCallback
 #else
-#define MAYBE_JITCallback JITCallback
+#define MAYBE_JITCallback SKIP_WITHOUT_JIT(JITCallback)
 #endif
 TEST(NativeMemRefJit, MAYBE_JITCallback) {
   constexpr int k = 2;
   constexpr int m = 2;
   int64_t shape[] = {k, m};
   int64_t shapeAlloc[] = {k + 1, m + 1};
   OwningMemRef<float, 2> a(shape, shapeAlloc);
   int count = 1;