[AArch64][GlobalISel] Enable GlobalISel at -O0 by default

[aemerson]

Summary

Tests updated to explicitly use fast-isel at -O0 instead of implicitly.

We should properly fall back to selection-dag in the cases where we don't support some feature.

This change also allows an explicit -fast-isel option to override an implicitly enabled global-isel. Otherwise -fast-isel would have no effect at -O0.

[qcolombet]

Hi Amara,

LGTM, though I would leave some time to others to reply to the review too.
I am clearly biased here :)

Cheers,
-Quentin

[efriedma]

Are you sure this is the right patch? If I try passing "-mllvm -aarch64-enable-global-isel-at-O=0" to clang on trunk, I get errors like "fatal error: error in backend: unable to lower arguments".

[qcolombet]

Sounds like this option doesn’t use the fallback path.
I thought it did.
Something to double check.

[efriedma]

Ran some tests internally with "-mllvm -aarch64-enable-global-isel-at-O=0 -mllvm -global-isel-abort=0". Just filed https://bugs.llvm.org/show_bug.cgi?id=35690 . Otherwise seems to work.

[aemerson]

I've updated the patch and introduced a new default behavior for -global-isel-abort so it can co-operate with targets which have GISel enabled by default. The bug Eli raised should also be fixed as well in another commit.

[qcolombet]

Sounds like this option doesn’t use the fallback path.
I thought it did.
Something to double check.

lib/CodeGen/TargetPassConfig.cpp

141 ↗

(On Diff #127775)

I would rather have another target hook for the abort case.
Same thing as what we have for GISel enabled (option takes precedence if set, otherwise we look at what the target wants)

[aemerson]

In D41362#961501, @qcolombet wrote:

Sounds like this option doesn’t use the fallback path.
I thought it did.
Something to double check.

I'm not exactly sure what you mean by this.

How about checking for EnableGlobalISelAbort.getNumOccurences() > 0 to check for an command line override, otherwise doing what the target wants via the isGlobalISelAbortEnabled() override?

[qcolombet]

How about checking for EnableGlobalISelAbort.getNumOccurences() > 0 to check for an command line override, otherwise doing what the target wants via the isGlobalISelAbortEnabled() override?

That's what I meant :). (Instead of adding a == 3 case for the option.)

Now, regarding the we should double check thing, I meant that we should double check that the -aarch64-enable-global-isel-at-O=0 properly set abort to false in the aarch64 override.

[dsanders]

In D41362#961783, @qcolombet wrote:

How about checking for EnableGlobalISelAbort.getNumOccurences() > 0 to check for an command line override, otherwise doing what the target wants via the isGlobalISelAbortEnabled() override?

That's what I meant :). (Instead of adding a == 3 case for the option.)

I'd be surprised if an API user want to force aborts to be enabled so it makes sense to me to check the number of occurrences of the command line option.

One possible spelling nit: If we want a target hook to control this, shouldn't the hook describe a property of the target rather than a property of the GlobalISel implementation? I'm thinking along the lines of CompleteModel in the scheduler to get something like isGlobalISelComplete() or useGlobalISelOnly()

[aemerson]

Patch updated. I've changed the abort method to no longer be an override. It doesn't really make sense to override it since only the TPC implementation can access the CLI options, so overriding it wouldn't work anyway.

We already have enough information from the isGlobalISelEnabled() hook to determine whether we should abort.

For aarch64-enable-global-isel-at-O the isGlobalISelEnabled() hook will check that value and so the abort will be implicitly disabled via that hook with this patch.

[Gerolf]

Hi,

I see all issues that came up in this thread covered by the last patch. Before this can be committed I still want to check that all paths/issues are tested/covered wrt to fast-isel:
a) how do we guarantee that there is no fallback from GISel to FastISel (when GISel is supported)? This is probably a nit since it is obvious to everyone deeper in the implementation then I am.
b) for all the tests where fast-isel was added shouldn't there equivalent tests for GISel, too? Even if the tests target fast-isel specific issues how do we make sure GISel does not expose similar/same bugs?

Thanks
Gerolf

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

1 ↗

(On Diff #127892)

Nit: -global-isel is no longer needed.

[aemerson]

In D41362#965646, @Gerolf wrote:

Hi,

I see all issues that came up in this thread covered by the last patch. Before this can be committed I still want to check that all paths/issues are tested/covered wrt to fast-isel:
a) how do we guarantee that there is no fallback from GISel to FastISel (when GISel is supported)? This is probably a nit since it is obvious to everyone deeper in the implementation then I am.
b) for all the tests where fast-isel was added shouldn't there equivalent tests for GISel, too? Even if the tests target fast-isel specific issues how do we make sure GISel does not expose similar/same bugs?

Thanks
Gerolf

Reading the code again I think we're covered for the fall back issue for the cases where TargetOptions::EnableFastISel is queried.

As for the tests, this change should maintain the existing test coverage of the code. You're correct in that there are cases where we're not testing the equivalent feature with global-isel, this should be follow on work after this to make GISel feature complete (rdar://35603467). I don't think adding -global-isel to the existing tests would help since we'd end up having to XFAIL them anyway. For the tests which are fast-isel specific they're mostly implementing support for something rather than bugs, and we should have tests for those as we add support over time in the GlobalISel specific tests.

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

1 ↗

(On Diff #127892)

Ack.

[Gerolf]

Thanks! This LGTM then.

-Gerolf

[aemerson]

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

1 ↗

(On Diff #127892)

Actually is still needed since we're no longer aborting by default, while this specific test is checking that we fail on a specific input MIR so we need to trigger the error instead of falling back to SDAG.

llvm/trunk/include/llvm/CodeGen/TargetPassConfig.h

public:
   /// verification is enabled.
   void addVerifyPass(const std::string &Banner);
 
   /// Check whether or not GlobalISel should be enabled by default.
   /// Fallback/abort behavior is controlled via other methods.
   virtual bool isGlobalISelEnabled() const;
 
   /// Check whether or not GlobalISel should abort on error.
-  /// When this is disable, GlobalISel will fall back on SDISel instead of
+  /// When this is disabled, GlobalISel will fall back on SDISel instead of
   /// erroring out.
-  virtual bool isGlobalISelAbortEnabled() const;
+  bool isGlobalISelAbortEnabled() const;
 
   /// Check whether or not a diagnostic should be emitted when GlobalISel
   /// uses the fallback path. In other words, it will emit a diagnostic
   /// when GlobalISel failed and isGlobalISelAbortEnabled is false.
   virtual bool reportDiagnosticWhenGlobalISelFallback() const;
 
 protected:
   // Helper to verify the analysis is really immutable.

llvm/trunk/lib/CodeGen/TargetPassConfig.cpp

bool TargetPassConfig::addCoreISelPasses() {
   // Enable FastISel with -fast, but allow that to be overridden.
   TM->setO0WantsFastISel(EnableFastISelOption != cl::BOU_FALSE);
   if (EnableFastISelOption == cl::BOU_TRUE ||
       (TM->getOptLevel() == CodeGenOpt::None && TM->getO0WantsFastISel()))
     TM->setFastISel(true);
 
   // Ask the target for an isel.
   // Enable GlobalISel if the target wants to, but allow that to be overriden.
+  // Explicitly enabling fast-isel should override implicitly enabled
+  // global-isel.
   if (EnableGlobalISel == cl::BOU_TRUE ||
-      (EnableGlobalISel == cl::BOU_UNSET && isGlobalISelEnabled())) {
+      (EnableGlobalISel == cl::BOU_UNSET && isGlobalISelEnabled() &&
+       EnableFastISelOption != cl::BOU_TRUE)) {
     if (addIRTranslator())
       return true;
 
     addPreLegalizeMachineIR();
 
     if (addLegalizeMachineIR())
       return true;
 

 /// GlobalISel Configuration
 //===---------------------------------------------------------------------===//
 
 bool TargetPassConfig::isGlobalISelEnabled() const {
   return false;
 }
 
 bool TargetPassConfig::isGlobalISelAbortEnabled() const {
+  if (EnableGlobalISelAbort.getNumOccurrences() > 0)
     return EnableGlobalISelAbort == 1;
+
+  // When no abort behaviour is specified, we don't abort if the target says
+  // that GISel is enabled.
+  return !isGlobalISelEnabled();
 }
 
 bool TargetPassConfig::reportDiagnosticWhenGlobalISelFallback() const {
   return EnableGlobalISelAbort == 2;
 }

llvm/trunk/lib/Target/AArch64/AArch64TargetMachine.cpp

 static cl::opt<bool>
     EnableLoopDataPrefetch("aarch64-enable-loop-data-prefetch", cl::Hidden,
                            cl::desc("Enable the loop data prefetch pass"),
                            cl::init(true));
 
 static cl::opt<int> EnableGlobalISelAtO(
     "aarch64-enable-global-isel-at-O", cl::Hidden,
     cl::desc("Enable GlobalISel at or below an opt level (-1 to disable)"),
-    cl::init(-1));
+    cl::init(0));
 
 static cl::opt<bool> EnableFalkorHWPFFix("aarch64-enable-falkor-hwpf-fix",
                                          cl::init(true), cl::Hidden);
 
 extern "C" void LLVMInitializeAArch64Target() {
   // Register the target.
   RegisterTargetMachine<AArch64leTargetMachine> X(getTheAArch64leTarget());
   RegisterTargetMachine<AArch64beTargetMachine> Y(getTheAArch64beTarget());

llvm/trunk/test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

-; RUN: not llc -O0 -global-isel -verify-machineinstrs %s -o - 2>&1 | FileCheck %s --check-prefix=ERROR
+; RUN: not llc -O0 -global-isel -global-isel-abort=1 -verify-machineinstrs %s -o - 2>&1 | FileCheck %s --check-prefix=ERROR
 ; RUN: llc -O0 -global-isel -global-isel-abort=0 -verify-machineinstrs %s -o - 2>&1 | FileCheck %s --check-prefix=FALLBACK
 ; RUN: llc -O0 -global-isel -global-isel-abort=2 -pass-remarks-missed='gisel*' -verify-machineinstrs %s -o %t.out 2> %t.err
 ; RUN: FileCheck %s --check-prefix=FALLBACK-WITH-REPORT-OUT < %t.out
 ; RUN: FileCheck %s --check-prefix=FALLBACK-WITH-REPORT-ERR < %t.err
 ; RUN: not llc -global-isel -mtriple aarch64_be %s -o - 2>&1 | FileCheck %s --check-prefix=BIG-ENDIAN
 ; This file checks that the fallback path to selection dag works.
 ; The test is fragile in the sense that it must be updated to expose
 ; something that fails with global-isel.

llvm/trunk/test/CodeGen/AArch64/GlobalISel/gisel-commandline-option.ll

 ; RUN: llc -mtriple=aarch64-- -debug-pass=Structure %s -o /dev/null 2>&1 \
-; RUN:   -O0 -aarch64-enable-global-isel-at-O=0 \
+; RUN:   -O0 | FileCheck %s --check-prefix ENABLED --check-prefix ENABLED-O0 --check-prefix FALLBACK
+
+; RUN: llc -mtriple=aarch64-- -debug-pass=Structure %s -o /dev/null 2>&1 \
+; RUN:   -O0 -aarch64-enable-global-isel-at-O=0 -global-isel-abort=1 \
 ; RUN:   | FileCheck %s --check-prefix ENABLED --check-prefix ENABLED-O0 --check-prefix NOFALLBACK
 
 ; RUN: llc -mtriple=aarch64-- -debug-pass=Structure %s -o /dev/null 2>&1 \
 ; RUN:   -O0 -aarch64-enable-global-isel-at-O=0 -global-isel-abort=2  \
 ; RUN:   | FileCheck %s --check-prefix ENABLED --check-prefix ENABLED-O0 --check-prefix FALLBACK
 
 ; RUN: llc -mtriple=aarch64-- -debug-pass=Structure %s -o /dev/null 2>&1 \
 ; RUN:   -global-isel \

llvm/trunk/test/CodeGen/AArch64/aarch64_f16_be.ll

-; RUN: llc -mtriple=aarch64-linux-gnuabi -O0 < %s | FileCheck %s
-; RUN: llc -mtriple=aarch64_be-linux-gnuabi -O0 < %s | FileCheck %s --check-prefix=CHECK-BE
+; RUN: llc -mtriple=aarch64-linux-gnuabi -O0 -fast-isel < %s | FileCheck %s
+; RUN: llc -mtriple=aarch64_be-linux-gnuabi -O0 -fast-isel < %s | FileCheck %s --check-prefix=CHECK-BE
 
 define void @test_bitcast_v8f16_to_v4f32(<8 x half> %a) {
 ; CHECK-LABEL: test_bitcast_v8f16_to_v4f32:
 ; CHECK-NOT: st1
 
 ; CHECK-BE-LABEL: test_bitcast_v8f16_to_v4f32:
 ; CHECK-BE: st1
 

llvm/trunk/test/CodeGen/AArch64/and-mask-removal.ll

-; RUN: llc -O0 -fast-isel=false -mtriple=arm64-apple-darwin  < %s  | FileCheck %s
+; RUN: llc -mtriple=arm64-apple-darwin  < %s  | FileCheck %s
 
 @board = common global [400 x i8] zeroinitializer, align 1
 @next_string = common global i32 0, align 4
 @string_number = common global [400 x i32] zeroinitializer, align 4
 
 ; Function Attrs: nounwind ssp
 define void @new_position(i32 %pos) {
 entry:

llvm/trunk/test/CodeGen/AArch64/arm64-EXT-undef-mask.ll

-; RUN: llc -O0 -mtriple=arm64-eabi -aarch64-neon-syntax=apple -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -mtriple=arm64-eabi -aarch64-neon-syntax=apple -verify-machineinstrs < %s | FileCheck %s
 
 ; The following 2 test cases test shufflevector with beginning UNDEF mask.
 define <8 x i16> @test_vext_undef_traverse(<8 x i16> %in) {
 ;CHECK-LABEL: test_vext_undef_traverse:
 ;CHECK: {{ext.16b.*v0, #4}}
   %vext = shufflevector <8 x i16> <i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 0, i16 0>, <8 x i16> %in, <8 x i32> <i32 undef, i32 undef, i32 undef, i32 undef, i32 6, i32 7, i32 8, i32 9>
   ret <8 x i16> %vext
 }

llvm/trunk/test/CodeGen/AArch64/arm64-abi.ll

 ; RUN: llc     -mtriple=arm64-apple-darwin -mcpu=cyclone -enable-misched=false < %s | FileCheck %s
-; RUN: llc -O0 -mtriple=arm64-apple-darwin                                     < %s | FileCheck --check-prefix=FAST %s
+; RUN: llc -O0 -fast-isel -mtriple=arm64-apple-darwin                          < %s | FileCheck --check-prefix=FAST %s
 
 ; rdar://9932559
 define i64 @i8i16callee(i64 %a1, i64 %a2, i64 %a3, i8 signext %a4, i16 signext %a5, i64 %a6, i64 %a7, i64 %a8, i8 signext %b1, i16 signext %b2, i8 signext %b3, i8 signext %b4) nounwind readnone noinline {
 entry:
 ; CHECK-LABEL: i8i16callee:
 ; The 8th, 9th, 10th and 11th arguments are passed at sp, sp+2, sp+4, sp+5.
 ; They are i8, i16, i8 and i8.
 ; CHECK-DAG: ldrsb {{w[0-9]+}}, [sp, #5]

llvm/trunk/test/CodeGen/AArch64/arm64-abi_align.ll

 ; RUN: llc < %s -mtriple=arm64-apple-darwin -mcpu=cyclone -enable-misched=false -disable-fp-elim | FileCheck %s
-; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 -disable-fp-elim | FileCheck -check-prefix=FAST %s
+; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 -disable-fp-elim -fast-isel | FileCheck -check-prefix=FAST %s
 
 ; rdar://12648441
 ; Generated from arm64-arguments.c with -O2.
 ; Test passing structs with size < 8, < 16 and > 16
 ; with alignment of 16 and without
 
 ; Structs with size < 8
 %struct.s38 = type { i32, i16 }

llvm/trunk/test/CodeGen/AArch64/arm64-elf-constpool.ll

 ; RUN: llc -mtriple=arm64-linux-gnu -o - %s | FileCheck %s
-; RUN: llc -mtriple=arm64-linux-gnu -O0 -o - %s | FileCheck %s
+; RUN: llc -mtriple=arm64-linux-gnu -O0 -fast-isel -o - %s | FileCheck %s
 
 ; O0 checked for fastisel purposes. It has a separate path which
 ; creates a constpool entry for floating values.
 
 define double @needs_const() {
   ret double 3.14159
 ; CHECK: .LCPI0_0:
 
 ; CHECK: adrp {{x[0-9]+}}, .LCPI0_0
 ; CHECK: ldr d0, [{{x[0-9]+}}, :lo12:.LCPI0_0]
 }

llvm/trunk/test/CodeGen/AArch64/arm64-elf-globals.ll

 ; RUN: llc -mtriple=arm64-linux-gnu -o - %s -mcpu=cyclone | FileCheck %s
-; RUN: llc -mtriple=arm64-linux-gnu -o - %s -O0 -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
+; RUN: llc -mtriple=arm64-linux-gnu -o - %s -O0 -fast-isel -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
 ; RUN: llc -mtriple=arm64-linux-gnu -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-PIC
-; RUN: llc -mtriple=arm64-linux-gnu -O0 -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST-PIC
+; RUN: llc -mtriple=arm64-linux-gnu -O0 -fast-isel -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST-PIC
 ; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -o - %s -mcpu=cyclone | FileCheck %s
-; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -o - %s -O0 -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
+; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -o - %s -O0 -fast-isel -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
 ; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-PIC
-; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -O0 -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST-PIC
+; RUN: llc -mtriple=aarch64-fuchsia -code-model=kernel -O0 -fast-isel -relocation-model=pic -o - %s -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST-PIC
 
 @var8 = external global i8, align 1
 @var16 = external global i16, align 2
 @var32 = external global i32, align 4
 @var64 = external global i64, align 8
 
 define i8 @test_i8(i8 %new) {
   %val = load i8, i8* @var8, align 1

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-alloca.ll

 ; This test should cause the TargetMaterializeAlloca to be invoked
-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin -disable-fp-elim < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin -disable-fp-elim < %s | FileCheck %s
 
 %struct.S1Ty = type { i64 }
 %struct.S2Ty = type { %struct.S1Ty, %struct.S1Ty }
 
 define void @takeS1(%struct.S1Ty* %V) nounwind {
 entry:
   %V.addr = alloca %struct.S1Ty*, align 8
   store %struct.S1Ty* %V, %struct.S1Ty** %V.addr, align 8

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-br.ll

-; RUN: llc -O0 -fast-isel-abort=1 -mtriple=arm64-apple-darwin -mcpu=cyclone -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -mtriple=arm64-apple-darwin -mcpu=cyclone -verify-machineinstrs < %s | FileCheck %s
 
 define void @branch1() nounwind uwtable ssp {
   %x = alloca i32, align 4
   store i32 0, i32* %x, align 4
   %1 = load i32, i32* %x, align 4
   %2 = icmp ne i32 %1, 0
   br i1 %2, label %3, label %4
 

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-call.ll

-; RUN: llc -O0 -fast-isel-abort=2 -code-model=small -verify-machineinstrs -disable-fp-elim -mtriple=arm64-apple-darwin   < %s | FileCheck %s
-; RUN: llc -O0 -fast-isel-abort=2 -code-model=large -verify-machineinstrs -disable-fp-elim -mtriple=arm64-apple-darwin   < %s | FileCheck %s --check-prefix=LARGE
-; RUN: llc -O0 -fast-isel-abort=2 -code-model=small -verify-machineinstrs -disable-fp-elim -mtriple=aarch64_be-linux-gnu < %s | FileCheck %s --check-prefix=CHECK-BE
+; RUN: llc -O0 -fast-isel -fast-isel-abort=2 -code-model=small -verify-machineinstrs -disable-fp-elim -mtriple=arm64-apple-darwin   < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=2 -code-model=large -verify-machineinstrs -disable-fp-elim -mtriple=arm64-apple-darwin   < %s | FileCheck %s --check-prefix=LARGE
+; RUN: llc -O0 -fast-isel -fast-isel-abort=2 -code-model=small -verify-machineinstrs -disable-fp-elim -mtriple=aarch64_be-linux-gnu < %s | FileCheck %s --check-prefix=CHECK-BE
 
 define void @call0() nounwind {
 entry:
   ret void
 }
 
 define void @foo0() nounwind {
 entry:

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-conversion-fallback.ll

-; RUN: llc -O0 -verify-machineinstrs -mtriple=arm64-eabi < %s | FileCheck --enable-var-scope %s
+; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=arm64-eabi < %s | FileCheck --enable-var-scope %s
 
 ; Test fptosi
 define i32 @fptosi_wh(half %a) nounwind ssp {
 entry:
 ; CHECK-LABEL: fptosi_wh
 ; CHECK: fcvt s1, h0
 ; CHECK: fcvtzs [[REG:w[0-9]+]], s1
 ; CHECK: mov w0, [[REG]]

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-conversion.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin -mcpu=cyclone < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin -mcpu=cyclone < %s | FileCheck %s
 
 ;; Test various conversions.
 define zeroext i32 @trunc_(i8 zeroext %a, i16 zeroext %b, i32 %c, i64 %d) nounwind ssp {
 entry:
 ; CHECK-LABEL: trunc_
 ; CHECK: sub sp, sp, #16
 ; CHECK: strb w0, [sp, #15]
 ; CHECK: strh w1, [sp, #12]

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-fcmp.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 define zeroext i1 @fcmp_float1(float %a) {
 ; CHECK-LABEL: fcmp_float1
 ; CHECK:       fcmp s0, #0.0
 ; CHECK-NEXT:  cset {{w[0-9]+}}, ne
   %1 = fcmp une float %a, 0.000000e+00
   ret i1 %1
 }

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-gv.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 ; Test load/store of global value from global offset table.
 @seed = common global i64 0, align 8
 
 define void @Initrand() nounwind {
 entry:
 ; CHECK: @Initrand
 ; CHECK: adrp [[REG:x[0-9]+]], _seed@GOTPAGE

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-icmp.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 define i32 @icmp_eq_imm(i32 %a) nounwind ssp {
 entry:
 ; CHECK-LABEL: icmp_eq_imm
 ; CHECK:       cmp w0, #31
 ; CHECK-NEXT:  cset w0, eq
   %cmp = icmp eq i32 %a, 31
   %conv = zext i1 %cmp to i32

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -relocation-model=dynamic-no-pic -mtriple=arm64-apple-ios < %s | FileCheck %s --check-prefix=ARM64
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -relocation-model=dynamic-no-pic -mtriple=arm64-apple-ios < %s | FileCheck %s --check-prefix=ARM64
 
 @message = global [80 x i8] c"The LLVM Compiler Infrastructure\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00", align 16
 @temp = common global [80 x i8] zeroinitializer, align 16
 
 define void @t1() {
 ; ARM64-LABEL: t1
 ; ARM64: adrp x8, _message@PAGE
 ; ARM64: add x0, x8, _message@PAGEOFF

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-materialize.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 ; Materialize using fmov
 define float @fmov_float1() {
 ; CHECK-LABEL: fmov_float1
 ; CHECK:       fmov s0, #1.25000000
   ret float 1.250000e+00
 }
 

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-noconvert.ll

-; RUN: llc -O0 -verify-machineinstrs -mtriple=aarch64-apple-ios < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=aarch64-apple-ios < %s | FileCheck %s
 
 ; Fast-isel can't do vector conversions yet, but it was emitting some highly
 ; suspect UCVTFUWDri MachineInstrs.
 define <4 x float> @test_uitofp(<4 x i32> %in) {
 ; CHECK-LABEL: test_uitofp:
 ; CHECK: ucvtf.4s v0, v0
 
   %res = uitofp <4 x i32> %in to <4 x float>

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-rem.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
-; RUN: llc %s -O0 -fast-isel-abort=1 -mtriple=arm64-apple-darwin -print-machineinstrs=expand-isel-pseudos -o /dev/null 2> %t
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc %s -O0 -fast-isel -fast-isel-abort=1 -mtriple=arm64-apple-darwin -print-machineinstrs=expand-isel-pseudos -o /dev/null 2> %t
 ; RUN: FileCheck %s < %t --check-prefix=CHECK-SSA
 
 ; CHECK-SSA-LABEL: Machine code for function t1
 
 ; CHECK-SSA: [[QUOTREG:%[0-9]+]]:gpr32 = SDIVWr
 ; CHECK-SSA-NOT: [[QUOTREG]] =
 ; CHECK-SSA: {{%[0-9]+}}:gpr32 = MSUBWrrr killed [[QUOTREG]]
 

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel-ret.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 ;; Test returns.
 define void @t0() nounwind ssp {
 entry:
 ; CHECK: t0
 ; CHECK: ret
   ret void
 }

llvm/trunk/test/CodeGen/AArch64/arm64-fast-isel.ll

-; RUN: llc -O0 -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -fast-isel-abort=1 -verify-machineinstrs -mtriple=arm64-apple-darwin < %s | FileCheck %s
 
 define void @t0(i32 %a) nounwind {
 entry:
 ; CHECK: t0
 ; CHECK: str {{w[0-9]+}}, [sp, #12]
 ; CHECK-NEXT: ldr [[REGISTER:w[0-9]+]], [sp, #12]
 ; CHECK-NEXT: str [[REGISTER]], [sp, #12]
 ; CHECK: ret

llvm/trunk/test/CodeGen/AArch64/arm64-simd-scalar-to-vector.ll

 ; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple -mcpu=cyclone | FileCheck %s
-; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple -O0 -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
+; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple -O0 -fast-isel -mcpu=cyclone | FileCheck %s --check-prefix=CHECK-FAST
 
 define <16 x i8> @foo(<16 x i8> %a) nounwind optsize readnone ssp {
 ; CHECK: uaddlv.16b h0, v0
 ; CHECK: rshrn.8b v0, v0, #4
 ; CHECK: dup.16b v0, v0[0]
 ; CHECK: ret
 
 ; CHECK-FAST: uaddlv.16b

llvm/trunk/test/CodeGen/AArch64/arm64-tls-dynamic-together.ll

-; RUN: llc -O0 -mtriple=arm64-none-linux-gnu -relocation-model=pic \
+; RUN: llc -O0 -fast-isel -mtriple=arm64-none-linux-gnu -relocation-model=pic \
 ; RUN:     -verify-machineinstrs < %s | FileCheck -check-prefix=CHECK -check-prefix=NOEMU %s
-; RUN: llc -emulated-tls -O0 -mtriple=arm64-none-linux-gnu -relocation-model=pic \
+; RUN: llc -emulated-tls -O0 -fast-isel -mtriple=arm64-none-linux-gnu -relocation-model=pic \
 ; RUN:     -verify-machineinstrs < %s | FileCheck -check-prefix=CHECK -check-prefix=EMU %s
 
 ; If the .tlsdesccall and blr parts are emitted completely separately (even with
 ; glue) then LLVM will separate them quite happily (with a spill at O0, hence
 ; the option). This is definitely wrong, so we make sure they are emitted
 ; together.
 
 @general_dynamic_var = external thread_local global i32

llvm/trunk/test/CodeGen/AArch64/arm64-vcvt_f.ll

 ; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple | FileCheck %s
-; RUN: llc < %s -O0 -mtriple=arm64-eabi -aarch64-neon-syntax=apple | FileCheck %s
+; RUN: llc < %s -O0 -fast-isel -mtriple=arm64-eabi -aarch64-neon-syntax=apple | FileCheck %s
 
 define <2 x double> @test_vcvt_f64_f32(<2 x float> %x) nounwind readnone ssp {
 ; CHECK-LABEL: test_vcvt_f64_f32:
   %vcvt1.i = fpext <2 x float> %x to <2 x double>
 ; CHECK: fcvtl  v0.2d, v0.2s
   ret <2 x double> %vcvt1.i
 ; CHECK: ret
 }

llvm/trunk/test/CodeGen/AArch64/br-cond-not-merge.ll

 ; RUN: llc -mtriple=aarch64 -verify-machineinstrs < %s | FileCheck --check-prefix=CHECK --check-prefix=OPT %s
-; RUN: llc -mtriple=aarch64 -verify-machineinstrs -O0 -fast-isel=0 < %s | FileCheck --check-prefix=CHECK --check-prefix=NOOPT %s
+; RUN: llc -mtriple=aarch64 -verify-machineinstrs -O0 -fast-isel=0 -global-isel=false < %s | FileCheck --check-prefix=CHECK --check-prefix=NOOPT %s
 
 declare void @foo()
 
 ; Check that the inverted or doesn't inhibit the splitting of the
 ; complex conditional into three branch instructions.
 ; CHECK-LABEL: test_and_not:
 ; CHECK:       cbz w0, [[L:\.LBB[0-9_]+]]
 ; OPT:         cmp w1, #2

llvm/trunk/test/CodeGen/AArch64/cmpxchg-O0.ll

-; RUN: llc -verify-machineinstrs -mtriple=aarch64-linux-gnu -O0 -fast-isel=0 %s -o - | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=aarch64-linux-gnu -O0 -fast-isel=0 -global-isel=false %s -o - | FileCheck %s
 
 define { i8, i1 } @test_cmpxchg_8(i8* %addr, i8 %desired, i8 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_8:
 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     mov [[STATUS:w[3-9]+]], #0
 ; CHECK:     ldaxrb [[OLD:w[0-9]+]], [x0]
 ; CHECK:     cmp [[OLD]], w1, uxtb
 ; CHECK:     b.ne [[DONE:.LBB[0-9]+_[0-9]+]]

llvm/trunk/test/CodeGen/AArch64/cxx-tlscc.ll

 ; RUN: llc < %s -mtriple=aarch64-apple-ios | FileCheck %s
 ; RUN: llc < %s -mtriple=aarch64-apple-ios -enable-shrink-wrap=true | FileCheck %s
 ; Shrink wrapping currently does not kick in because we have a TLS CALL
 ; in the entry block and it will clobber the link register.
 
-; RUN: llc < %s -mtriple=aarch64-apple-ios -O0 | FileCheck --check-prefix=CHECK-O0 %s
+; RUN: llc < %s -mtriple=aarch64-apple-ios -O0 -fast-isel | FileCheck --check-prefix=CHECK-O0 %s
 
 %struct.S = type { i8 }
 
 @sg = internal thread_local global %struct.S zeroinitializer, align 1
 @__dso_handle = external global i8
 @__tls_guard = internal thread_local unnamed_addr global i1 false
 @sum1 = internal thread_local global i32 0, align 4
 

llvm/trunk/test/CodeGen/AArch64/fast-isel-atomic.ll

 ; RUN: llc -mtriple=aarch64-- -O0 -fast-isel -fast-isel-abort=4 -verify-machineinstrs < %s | FileCheck %s
-; RUN: llc -mtriple=aarch64-- -O0 -fast-isel=0 -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -mtriple=aarch64-- -O0 -fast-isel=0 -global-isel=false -verify-machineinstrs < %s | FileCheck %s
 
 ; Note that checking SelectionDAG output isn't strictly necessary, but they
 ; currently match, so we might as well check both!  Feel free to remove SDAG.
 
 ; CHECK-LABEL: atomic_store_monotonic_8:
 ; CHECK-NEXT: // %bb.0:
 ; CHECK-NEXT:  strb  w1, [x0]
 ; CHECK-NEXT:  ret

llvm/trunk/test/CodeGen/AArch64/fast-isel-sp-adjust.ll

-; RUN: llc -O0 -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
-; RUN: not llc -O0 -mtriple=aarch64-apple-ios -o /dev/null -fast-isel-abort=3 %s 2> %t
+; RUN: llc -O0 -fast-isel -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
+; RUN: not llc -O0 -mtriple=aarch64-apple-ios -o /dev/null -fast-isel -fast-isel-abort=3 %s 2> %t
 ; RUN: FileCheck %s --check-prefix=CHECK-ERRORS < %t
 
 ; The issue here is that FastISel cannot emit an ADDrr where one of the inputs
 ; is SP. This only ever crops up with function calls, and then only if the
 ; argument is at an offset > 2^12 * size from SP.
 
 ; If FastISel ever starts coping with this and emits an "add xD, sp, xM" it's
 ; critical to check the encoding as well as the textual assembly. An ADDXrs with

llvm/trunk/test/CodeGen/AArch64/i128-fast-isel-fallback.ll

-; RUN: llc -O0 -mtriple=arm64-apple-ios7.0 -mcpu=generic < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -mtriple=arm64-apple-ios7.0 -mcpu=generic < %s | FileCheck %s
 
 ; Function Attrs: nounwind ssp
 define void @test1() {
   %1 = sext i32 0 to i128
   call void  @test2(i128 %1)
   ret void
 
 ; The i128 is 0 so the we can test to make sure it is propogated into the x

llvm/trunk/test/CodeGen/AArch64/preferred-alignment.ll

-; RUN: llc -mtriple=aarch64 -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=aarch64 -O0 -fast-isel < %s | FileCheck %s
 
 ; Function Attrs: nounwind
 define i32 @foo() #0 {
 entry:
   %c = alloca i8, align 1
 ; CHECK:  add x0, sp, #12
   %s = alloca i16, align 2
 ; CHECK-NEXT: add x1, sp, #8

llvm/trunk/test/CodeGen/AArch64/swift-return.ll

 ; RUN: llc -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
-; RUN: llc -O0 -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0
+; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0
 
 ; CHECK-LABEL: test1
 ; CHECK: bl      _gen
 ; CHECK: sxth    [[TMP:w.*]], w0
 ; CHECK: add     w0, [[TMP]], w1, sxtb
 ; CHECK-O0-LABEL: test1
 ; CHECK-O0: bl      _gen
 ; CHECK-O0: sxth    [[TMP:w.*]], w0

llvm/trunk/test/CodeGen/AArch64/swifterror.ll

 ; RUN: llc -verify-machineinstrs -disable-fp-elim -enable-shrink-wrap=false < %s -mtriple=aarch64-apple-ios -disable-post-ra | FileCheck --check-prefix=CHECK-APPLE %s
-; RUN: llc -verify-machineinstrs -disable-fp-elim -O0 < %s -mtriple=aarch64-apple-ios -disable-post-ra | FileCheck --check-prefix=CHECK-O0 %s
+; RUN: llc -verify-machineinstrs -disable-fp-elim -O0 -fast-isel < %s -mtriple=aarch64-apple-ios -disable-post-ra | FileCheck --check-prefix=CHECK-O0 %s
 
 declare i8* @malloc(i64)
 declare void @free(i8*)
 %swift_error = type {i64, i8}
 
 ; This tests the basic usage of a swifterror parameter. "foo" is the function
 ; that takes a swifterror parameter and "caller" is the caller of "foo".
 define float @foo(%swift_error** swifterror %error_ptr_ref) {

llvm/trunk/test/CodeGen/AArch64/swiftself.ll

 ; RUN: llc -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck --check-prefix=CHECK --check-prefix=OPT %s
-; RUN: llc -O0 -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
+; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=aarch64-unknown-linux-gnu -o - %s | FileCheck --check-prefix=CHECK --check-prefix=OPT %s
 
 ; Parameter with swiftself should be allocated to x20.
 ; CHECK-LABEL: swiftself_param:
 ; CHECK: mov x0, x20
 ; CHECK-NEXT: ret
 define i8* @swiftself_param(i8* swiftself %addr0) {
   ret i8 *%addr0

llvm/trunk/test/CodeGen/AArch64/tailcall-fastisel.ll

-; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 | FileCheck %s
+; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 -fast-isel | FileCheck %s
 
 ; CHECK: b _foo0
 
 define i32 @foo1() {
 entry:
   %call = tail call i32 @foo0()
   ret i32 %call
 }
 
 declare i32 @foo0()

llvm/trunk/test/DebugInfo/AArch64/asan-stack-vars.ll

-; RUN: llc -O0 -filetype=obj -o - %s | llvm-dwarfdump -v - | FileCheck %s
+; RUN: llc -O0 -fast-isel -filetype=obj -o - %s | llvm-dwarfdump -v - | FileCheck %s
 ;
 ; Derived from (clang -O0 -g -fsanitize=address -fobjc-arc)
 ;   @protocol NSObject
 ;   @end
 ;   @interface NSObject<NSObject>{}
 ;   + (instancetype)alloc;
 ;   @end
 ;   struct CGSize {

llvm/trunk/test/DebugInfo/AArch64/frameindices.ll

-; RUN: llc -disable-fp-elim -O0 -filetype=obj < %s | llvm-dwarfdump -v - | FileCheck %s
+; RUN: llc -disable-fp-elim -O0 -fast-isel -filetype=obj < %s | llvm-dwarfdump -v - | FileCheck %s
 ; Test that a variable with multiple entries in the MMI table makes it into the
 ; debug info.
 ;
 ; CHECK: DW_TAG_inlined_subroutine
 ; CHECK:    "_Z3f111A"
 ; CHECK: DW_TAG_formal_parameter
 ; CHECK: DW_AT_location [DW_FORM_block1]    (DW_OP_piece 0x1, DW_OP_fbreg -47, DW_OP_piece 0xf, DW_OP_piece 0x1, DW_OP_fbreg -54, DW_OP_piece 0x7)
 ; CHECK: DW_AT_abstract_origin {{.*}} "p1"

llvm/trunk/test/DebugInfo/AArch64/line-header.ll

-; RUN: llc -mtriple=aarch64-none-linux -O0 -filetype=obj - < %S/../Inputs/line.ll | llvm-dwarfdump -v - | FileCheck %s
-; RUN: llc -mtriple=aarch64_be-none-linux -O0 -filetype=obj - < %S/../Inputs/line.ll | llvm-dwarfdump -v - | FileCheck %s
+; RUN: llc -mtriple=aarch64-none-linux -O0 -fast-isel -filetype=obj - < %S/../Inputs/line.ll | llvm-dwarfdump -v - | FileCheck %s
+; RUN: llc -mtriple=aarch64_be-none-linux -O0 -fast-isel -filetype=obj - < %S/../Inputs/line.ll | llvm-dwarfdump -v - | FileCheck %s
 
 ; check line table length is correctly calculated for both big and little endian
 CHECK-LABEL: .debug_line contents:
 CHECK: total_length: 0x0000003f

llvm/trunk/test/DebugInfo/AArch64/prologue_end.ll

-; RUN: llc -disable-fp-elim -O0 %s -mtriple aarch64-apple-darwin -o - | FileCheck %s
+; RUN: llc -disable-fp-elim -O0 -fast-isel %s -mtriple aarch64-apple-darwin -o - | FileCheck %s
 
 ; int func(void);
 ; void prologue_end_test() {
 ;   func();
 ;   func();
 ; }
 
 define void @prologue_end_test() nounwind uwtable !dbg !4 {