[SelectionDAG] make INLINEASM_BR use MachineBasicBlocks instead of BlockAddresses

[nickdesaulniers]

Summary

As part of re-architecting callbr to no longer use blockaddresses
(https://reviews.llvm.org/D129288), we don't really need them in MIR.
They make comparing MachineBasicBlocks of indirect targets during
MachineVerifier a PITA.

Suggested by @efriedma from the discussion:
https://reviews.llvm.org/D130290#3669531

[nickdesaulniers]

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

3642–3646

I'm going to test these out, too. But RFC; comments welcome.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

Not sure this is quite right, but also not sure what it should be. Maybe the reviewers do?

llvm/test/CodeGen/X86/tail-dup-asm-goto.ll

10–11

I think I should pre-commit an initial run of update_mir_test_checks.py on this...

[nickdesaulniers]

• rebase on top of 0ccb6da725bad797f576713613f8368f7f52290c

llvm/test/CodeGen/X86/tail-dup-asm-goto.ll

10–11

0ccb6da725bad797f576713613f8368f7f52290c

[nickdesaulniers]

• SelectionDAGBuilder::visitCallBr() no longer needs to call MachineBasicBlock::setHasAddressTaken()

[nickdesaulniers]

We might also be able to remove some of the handling in
EmitGCCInlineAsmStr <llvm/lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp>

• commit 83d690a14980 ("Don't rely on 'l'(ell) modifiers to indicate a label reference")
• commit 1b104388752f ("[MC] Don't recreate a label if it's already used")

[arsenm]

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

3641–3644

Comments can be removed after D130323?

[barannikov88]

llvm/test/CodeGen/AArch64/callbr-asm-label.ll

10

I'm a bit out of context, so the question might be silly, sorry.
Shouldn't this temporary label no longer be emitted? It used to be referenced, but no longer is.

[nikic]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5293

This is a roundabout way of saying cast<CallBrInst>(&Call)->getIndirectDest(LabelNo) -- unless the fact that a BlockAddress on the IR level is created is still important for something?

5294–5295

BA has an i8* type, so can directly use that?

[nickdesaulniers]

llvm/test/CodeGen/AArch64/callbr-asm-label.ll

10

It's not a silly question; it's a great question.

I've been trying to track that down, as I don't know the answer myself.

IIRC, we did previously have an issue with labels disappearing out from under us. @void solved that in either

• commit 83d690a14980 ("Don't rely on 'l'(ell) modifiers to indicate a label reference")
• commit 1b104388752f ("[MC] Don't recreate a label if it's already used")

I've already found that the latter is no longer necessary, hence https://reviews.llvm.org/D130323, but that's not what was retaining these additional symbols.

(or maybe it was a third commit, maybe 79176a2542d03107b90613c84f18ccba41ad8fa8?)

So I would like to find the answer to this question, and perhaps remove the duplicate labels. But it would be a minor cosmetic fix; I'm not going to spend too much time on it since this patch is just a chain in a longer series that we're ultimately trying to fix. Fixing duplicate labels is nice to have, but slightly orthogonal.

[nickdesaulniers]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

Type::getInt8PtrTy seems to cause crashes...cant tell what type is actually being used; everything looks like opaque ptrs...

[nickdesaulniers]

• use CallBrInst::getIndirectDest rather than BlockAddress::getBasicBlock for OpInfo.CallOperandVal

[nickdesaulniers]

• prefer MachineBasicBlock::setLabelMustBeEmitted in SelectionDAGBuilder::visitCallBr over an additional check for MachineBasicBlock::isInlineAsmBrIndirectTarget in AsmPrinter::shouldEmitLabelForBasicBlock

[nickdesaulniers]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5293

Actually, it might be for some of the follow up reverts. I wonder if this is what's creating all of the .Ltmp labels that no longer seem necessary...

[nickdesaulniers]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

branch-folder is messing something up...

[nickdesaulniers]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

ahoy! Both TailDuplicator and BranchFolding were relying on whether an MBB has its address taken. After this change, they wont, so those cases additionally need to check for isInlineAsmBrIndirectTarget. With that added, we're working AND the .Ltmp labels are gone (good)! Now just to clean up the test churn after lunch, but I think this will all work and look nice. Stay tuned.

[nickdesaulniers]

• get rid of the stupid .Ltmp double labels

[nickdesaulniers]

Ok, this is ready for review/testing. It's part of a series:

1. https://reviews.llvm.org/D130316
2. https://reviews.llvm.org/D130323
3. https://reviews.llvm.org/D130391

Then 4 (https://reviews.llvm.org/D130290) which I will rebase once this lands.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

The last wart on an otherwise great simplification is getting the context for Type::getInt8PtrTy. Thoughts on how I can improve this?

[nikic]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

I think you can just use getProgramPointerTy(DL) to directly get the right MVT. No need to indirect via IR types.

You should also be able to drop the getBlockAddressForIndirectDest() method.

[barannikov88]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5297

Debugging leftover, I guess?

[barannikov88]

llvm/test/CodeGen/X86/inline-asm-pic.ll

34

The one here remains.

[nickdesaulniers]

• use getProgramPointerTy, del CallBrInst::getBlockAddressForIndirectDest

[nickdesaulniers]

llvm/test/CodeGen/X86/inline-asm-pic.ll

34

This is expected. You'll notice that it's address is taken by the blockaddress constant in @y (Line 57).

My change stops using blockaddress for callbr. It doesn't remove blockaddress from the language; test @y is testing the "address of a label" as an input to the inline asm, which is distinct from the label list used in asm goto.

That said, thanks for triple checking!

[barannikov88]

llvm/test/CodeGen/X86/inline-asm-pic.ll

34

Nope, line 57 is a different test.
Anyway, this was just an observation and not a suggestion to look into this specific case. Most parasitic labels are gone, so this is probably something X86-specific, which I believe is out of scope of this patch series.

[nikic]

I like the change, but I'm not familiar enough with the backend to fully understand the implications. The main thing I'd worry about is that more places that currently check hasAddressTaken() would also need to check isInlineAsmBrIndirectTarget(). E.g. I see more calls to the method in IfConversion. (Would it make sense to stop using blockaddresses, but still mark the MBBs are address-taken?)

[nickdesaulniers]

In D130316#3673721, @nikic wrote:

I like the change, but I'm not familiar enough with the backend to fully understand the implications. The main thing I'd worry about is that more places that currently check hasAddressTaken() would also need to check isInlineAsmBrIndirectTarget(). E.g. I see more calls to the method in IfConversion. (Would it make sense to stop using blockaddresses, but still mark the MBBs are address-taken?)

It does seem risky and orthogonal to the goal of this patch. Let me break out the s/setHasAddressTaken/setLabelMustBeEmitted/ change into a distinct child patch. BRB while I rechurn the test changes, again.

[nickdesaulniers]

• keep setHasAddressTaken, just add setLabelMustBeEmitted

[nickdesaulniers]

In D130316#3673721, @nikic wrote:

I like the change, but I'm not familiar enough with the backend to fully understand the implications. The main thing I'd worry about is that more places that currently check hasAddressTaken() would also need to check isInlineAsmBrIndirectTarget(). E.g. I see more calls to the method in IfConversion. (Would it make sense to stop using blockaddresses, but still mark the MBBs are address-taken?)

Done; PTAL. Sorry for the delay, was offline most of last week.

[arsenm]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

getProgramPointerTy is correct but I would expect taking the type directly from the IR would be better (in principle we could have individual functions with a different address space from the DL one, although I think that's currently unsupported)

[nickdesaulniers]

• rebase, remove setting OpInfo.ConstraintVT

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

5294–5295

FWICT, this assignment is no longer necessary. Removing it, as it doesn't affect tests.

[nickdesaulniers]

bumping for review

[void]

This patch seems pretty straight-forward, so I'll add my LGTM here. Please feel free to override that if other reviewers are less sure. :-)

llvm/test/CodeGen/AArch64/callbr-asm-label.ll

10

My memory is hazy on the details, but there were a number of issues with inline asm and labels when dealing with block addresses and basic blocks. The changes you mentioned are large hammers, but *seemed* necessary. There may be better ways to handle the label issue though.

[efriedma]

LGTM

I just merged D124697; I expect rebasing this on top of that should be trivial, but feel free to ask for another round of review if it isn't for some reason.

[nickdesaulniers]

• rebase, update mir tests to remove ir-block-address-taken MBB annotation

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

 
 bool AsmPrinter::shouldEmitLabelForBasicBlock(
     const MachineBasicBlock &MBB) const {
   // With `-fbasic-block-sections=`, a label is needed for every non-entry block
   // in the labels mode (option `=labels`) and every section beginning in the
   // sections mode (`=all` and `=list=`).
   if ((MF->hasBBLabels() || MBB.isBeginSection()) && !MBB.isEntryBlock())
     return true;
+  // Retain labels for INLINEASM_BR indirect targets.
+  // TODO: should callers of
+  // MachineBasicBlock::setIsInlineAsmBrIndirectTarget(true) also set
+  // MachineBasicBlock::setLabelMustBeEmitted()?
+  if (MBB.isInlineAsmBrIndirectTarget())
+    return true;
   // A label is needed for any block with at least one predecessor (when that
   // predecessor is not the fallthrough predecessor, or if it is an EH funclet
   // entry, or if a label is forced).
   return !MBB.pred_empty() &&
          (!isBlockOnlyReachableByFallthrough(&MBB) || MBB.isEHFuncletEntry() ||
           MBB.hasLabelMustBeEmitted());
 }
 

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

void SelectionDAGBuilder::visitCallBr(const CallBrInst &I) {
   MachineBasicBlock *Return = FuncInfo.MBBMap[I.getDefaultDest()];
 
   // Update successor info.
   addSuccessorWithProb(CallBrMBB, Return, BranchProbability::getOne());
   for (unsigned i = 0, e = I.getNumIndirectDests(); i < e; ++i) {
     BasicBlock *Dest = I.getIndirectDest(i);
     MachineBasicBlock *Target = FuncInfo.MBBMap[Dest];
     Target->setIsInlineAsmBrIndirectTarget();
-    Target->setHasAddressTaken();
     // Don't add duplicate machine successors.
     if (Dests.insert(Dest).second)
       addSuccessorWithProb(CallBrMBB, Target, BranchProbability::getZero());
   }
   CallBrMBB->normalizeSuccProbs();
 
   // Drop into default successor.
   DAG.setRoot(DAG.getNode(ISD::BR, getCurSDLoc(),

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

case InlineAsm::isOutput:
         OpInfo.ConstraintVT =
             getAsmOperandValueType(DL, Call.getType()).getSimpleVT();
       }
       ++ResNo;
       break;
     case InlineAsm::isInput:
       OpInfo.CallOperandVal = Call.getArgOperand(ArgNo);
       break;
-    case InlineAsm::isLabel:
-      OpInfo.CallOperandVal =
-          cast<CallBrInst>(&Call)->getBlockAddressForIndirectDest(LabelNo);
+    case InlineAsm::isLabel: {
+      const BlockAddress *BA =
+      OpInfo.CallOperandVal = BA->getBasicBlock();
       OpInfo.ConstraintVT =
-          getAsmOperandValueType(DL, OpInfo.CallOperandVal->getType())
-              .getSimpleVT();
+        getAsmOperandValueType(DL, BA->getType()).getSimpleVT();
       ++LabelNo;
       continue;
+    }
     case InlineAsm::isClobber:
       // Nothing to do.
       break;
     }
 
     if (OpInfo.CallOperandVal) {
       llvm::Type *OpTy = OpInfo.CallOperandVal->getType();
       if (OpInfo.isIndirect) {

llvm/lib/Target/X86/X86ISelLowering.cpp

       int64_t ExtVal = ExtOpc == ISD::ZERO_EXTEND ? CST->getZExtValue()
                                                   : CST->getSExtValue();
       Result = DAG.getTargetConstant(ExtVal, SDLoc(Op), MVT::i64);
       break;
     }
 
     // In any sort of PIC mode addresses need to be computed at runtime by
     // adding in a register or some sort of table lookup.  These can't
-    // be used as immediates. BlockAddresses are fine though.
+    // be used as immediates. BlockAddresses and BasicBlocks are fine though.
     if ((Subtarget.isPICStyleGOT() || Subtarget.isPICStyleStubPIC()) &&
-        !isa<BlockAddressSDNode>(Op))
+        !(isa<BlockAddressSDNode>(Op) || isa<BasicBlockSDNode>(Op)))
       return;
 
     // If we are in non-pic codegen mode, we allow the address of a global (with
     // an optional displacement) to be used with 'i'.
     if (auto *GA = dyn_cast<GlobalAddressSDNode>(Op))
       // If we require an extra load to get this address, as in PIC mode, we
       // can't accept it.
       if (isGlobalStubReference(

llvm/test/CodeGen/AArch64/callbr-asm-label.ll

 ; RUN: llc < %s -mtriple=aarch64-linux-gnu | FileCheck %s
 
 @X = common local_unnamed_addr global i32 0, align 4
 
 define i32 @test1() {
 ; CHECK-LABEL: test1:
 ; CHECK:         .word b
-; CHECK-NEXT:    .word .Ltmp0
+; CHECK-NEXT:    .word .LBB0_2
 ; CHECK: // %bb.1:
-; CHECK: .Ltmp0:
+; CHECK: .Ltmp1:
 ; CHECK: .LBB0_2: // %indirect
 entry:
   callbr void asm sideeffect "1:\0A\09.word b, ${0:l}\0A\09", "!i"()
           to label %cleanup [label %indirect]
 
 indirect:
   br label %cleanup
 
 cleanup:
   %retval.0 = phi i32 [ 1, %indirect ], [ 0, %entry ]
   ret i32 %retval.0
 }
 
 define void @test2() {
 ; CHECK-LABEL: test2:
 entry:
   %0 = load i32, i32* @X, align 4
   %and = and i32 %0, 1
   %tobool = icmp eq i32 %and, 0
   br i1 %tobool, label %if.end10, label %if.then
 
 if.then:
 ; CHECK:       .word b
-; CHECK-NEXT:  .word .Ltmp2
-; CHECK:       .Ltmp2:
+; CHECK-NEXT:  .word .LBB1_3
+; CHECK:       .Ltmp3:
 ; CHECK-NEXT:  .LBB1_3: // %if.end6
   callbr void asm sideeffect "1:\0A\09.word b, ${0:l}\0A\09", "!i"()
           to label %if.then4 [label %if.end6]
 
 if.then4:
   %call5 = tail call i32 bitcast (i32 (...)* @g to i32 ()*)()
   br label %if.end6
 

llvm/test/CodeGen/X86/callbr-asm-bb-exports.ll

 ; REQUIRES: asserts
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -o /dev/null -debug-only=isel 2>&1 | FileCheck %s
 
 ; Make sure we emit the basic block exports and the TokenFactor before the
 ; inlineasm_br. Not sure how to get a MachineIR change so this reads the debug
 ; output from SelectionDAG.
 
 ; CHECK: t0: ch = EntryToken
-; CHECK-NEXT: t16: i64 = BlockAddress<@test, %fail> 0
 ; CHECK-NEXT: t4: i32,ch = CopyFromReg t0, Register:i32 %3
 ; CHECK-NEXT: t10: i32 = add t4, Constant:i32<1>
 ; CHECK-NEXT: t12: ch = CopyToReg t0, Register:i32 %0, t10
 ; CHECK-NEXT: t6: i32,ch = CopyFromReg t0, Register:i32 %4
 ; CHECK-NEXT: t13: i32 = add t6, Constant:i32<1>
 ; CHECK-NEXT: t15: ch = CopyToReg t0, Register:i32 %1, t13
 ; CHECK-NEXT: t17: ch = TokenFactor t12, t15
 ; CHECK-NEXT: t2: i32,ch = CopyFromReg t0, Register:i32 %2
 ; CHECK-NEXT: t8: i32 = add t2, Constant:i32<4>
 ; CHECK-NEXT: t22: ch,glue = CopyToReg t17, Register:i32 %5, t8
-; CHECK-NEXT: t30: ch,glue = inlineasm_br t22, TargetExternalSymbol:i64'xorl $0, $0; jmp ${1:l}', MDNode:ch<null>, TargetConstant:i64<0>, TargetConstant:i32<2359305>, Register:i32 %5, TargetConstant:i64<13>, TargetBlockAddress:i64<@test, %fail> 0, TargetConstant:i32<12>, Register:i32 $df, TargetConstant:i32<12>, Register:i16 $fpsw, TargetConstant:i32<12>, Register:i32 $eflags, t22:1
+; CHECK-NEXT: t29: ch,glue = inlineasm_br t22, TargetExternalSymbol:i64'xorl $0, $0; jmp ${1:l}', MDNode:ch<null>, TargetConstant:i64<0>, TargetConstant:i32<2359305>, Register:i32 %5, TargetConstant:i64<13>, BasicBlock:ch<fail 0x{{[0-9a-f]+}}>, TargetConstant:i32<12>, Register:i32 $df, TargetConstant:i32<12>, Register:i16 $fpsw, TargetConstant:i32<12>, Register:i32 $eflags, t22:1
 
 define i32 @test(i32 %a, i32 %b, i32 %c) {
 entry:
   %0 = add i32 %a, 4
   %1 = add i32 %b, 1
   %2 = add i32 %c, 1
   callbr void asm "xorl $0, $0; jmp ${1:l}", "r,!i,~{dirflag},~{fpsr},~{flags}"(i32 %0) to label %normal [label %fail]
 
 normal:
   ret i32 %1
 
 fail:
   ret i32 %2
 }

llvm/test/CodeGen/X86/callbr-asm-branch-folding.ll

 ; CHECK-NEXT:    testl %ebp, %ebp
 ; CHECK-NEXT:    je .LBB0_8
 ; CHECK-NEXT:  # %bb.6: # %if.then14
 ; CHECK-NEXT:    movl {{[-0-9]+}}(%r{{[sb]}}p), %eax # 4-byte Reload
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    jmp .LBB0_9
 ; CHECK-NEXT:  .Ltmp0: # Block address taken
-; CHECK-NEXT:  # %bb.7: # %if.then20.critedge
+; CHECK-NEXT:  .LBB0_7: # %if.then20.critedge
 ; CHECK-NEXT:    movl j(%rip), %edi
 ; CHECK-NEXT:    movslq %eax, %rcx
 ; CHECK-NEXT:    movl $1, %esi
 ; CHECK-NEXT:    movq %r15, %rdx
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %r12
 ; CHECK-NEXT:    popq %r13

llvm/test/CodeGen/X86/callbr-asm-destinations.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=i686-- -verify-machineinstrs < %s | FileCheck %s
 
 define i32 @duplicate_normal_and_indirect_dest(i32 %a) {
 ; CHECK-LABEL: duplicate_normal_and_indirect_dest:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    addl $4, %eax
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmp .Ltmp0
+; CHECK-NEXT:    jmp .LBB0_1
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  .Ltmp0: # Block address taken
-; CHECK-NEXT:  # %bb.1: # %fail
+; CHECK-NEXT:  .LBB0_1: # %fail
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retl
 entry:
   %0 = add i32 %a, 4
   callbr void asm "xorl $0, $0; jmp ${1:l}", "r,!i,~{dirflag},~{fpsr},~{flags}"(i32 %0) to label %fail [label %fail]
 
 fail:
   ret i32 1
 }
 
 define i32 @duplicate_indirect_dest(i32 %a) {
 ; CHECK-LABEL: duplicate_indirect_dest:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    addl $4, %eax
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmp .Ltmp1
-; CHECK-NEXT:    jmp .Ltmp1
+; CHECK-NEXT:    jmp .LBB1_2
+; CHECK-NEXT:    jmp .LBB1_2
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %normal
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  .Ltmp1: # Block address taken
 ; CHECK-NEXT:  .LBB1_2: # %fail
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retl
 entry:

llvm/test/CodeGen/X86/callbr-asm-instr-scheduling.ll

 ; CHECK-NEXT:    andq %rax, %rdx
 ; CHECK-NEXT:    movzbl pgdir_shift(%rip), %eax
 ; CHECK-NEXT:    movq page_offset_base(%rip), %rcx
 ; CHECK-NEXT:    shrxq %rax, %rdi, %rax
 ; CHECK-NEXT:    addq %rcx, %rdx
 ; CHECK-NEXT:    andl $511, %eax # imm = 0x1FF
 ; CHECK-NEXT:    leaq (%rdx,%rax,8), %rax
 ; CHECK-NEXT:    #APP
+; CHECK-NEXT:  .Ltmp0:
+; CHECK-NEXT:    jmp .Ltmp1
 ; CHECK-NEXT:  .Ltmp2:
-; CHECK-NEXT:    jmp .Ltmp3
-; CHECK-NEXT:  .Ltmp4:
-; CHECK-NEXT:    .zero (-(((.Ltmp5-.Ltmp6)-(.Ltmp4-.Ltmp2))>0))*((.Ltmp5-.Ltmp6)-(.Ltmp4-.Ltmp2)),144
-; CHECK-NEXT:  .Ltmp7:
+; CHECK-NEXT:    .zero (-(((.Ltmp3-.Ltmp4)-(.Ltmp2-.Ltmp0))>0))*((.Ltmp3-.Ltmp4)-(.Ltmp2-.Ltmp0)),144
+; CHECK-NEXT:  .Ltmp5:
 entry:
   %0 = tail call i64 asm sideeffect "mov %cr3,$0\0A\09", "=r,=*m,~{dirflag},~{fpsr},~{flags}"(ptr elementtype(i64) nonnull @__force_order)
   %and.i = and i64 %0, 9223372036854771712
   %1 = load i64, ptr @page_offset_base, align 8
   %add = add i64 %and.i, %1
   %2 = inttoptr i64 %add to ptr
   %3 = load i32, ptr @pgdir_shift, align 4
   %sh_prom = zext i32 %3 to i64

llvm/test/CodeGen/X86/callbr-asm-label-addr.ll

 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
 
 define i32 @test1(i32 %x) {
 ; CHECK-LABEL: test1:
 ; CHECK:         .quad .Ltmp0
-; CHECK-NEXT:    .quad .Ltmp1
+; CHECK-NEXT:    .quad .LBB0_1
 ; CHECK: .Ltmp1:
-; CHECK-NEXT: # %bb.1: # %bar
+; CHECK-NEXT: .LBB0_1: # %bar
 ; CHECK-NEXT:    callq foo
 ; CHECK-NEXT: .Ltmp0:
 ; CHECK-NEXT:  # %bb.2: # %baz
 entry:
   callbr void asm sideeffect ".quad ${0:l}\0A\09.quad ${1:l}", "i,!i,~{dirflag},~{fpsr},~{flags}"(ptr blockaddress(@test1, %baz))
           to label %asm.fallthrough [label %bar]
 
 asm.fallthrough:

llvm/test/CodeGen/X86/callbr-asm-outputs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=i686-- -verify-machineinstrs < %s | FileCheck %s
 
 ; A test for asm-goto output
 
 define i32 @test1(i32 %x) {
 ; CHECK-LABEL: test1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    addl $4, %eax
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmp .Ltmp0
+; CHECK-NEXT:    jmp .LBB0_2
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %normal
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  .Ltmp0: # Block address taken
 ; CHECK-NEXT:  .LBB0_2: # %abnormal
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retl
 entry:

 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; CHECK-NEXT:    movl $-1, %eax
 ; CHECK-NEXT:    cmpl %edi, %esi
 ; CHECK-NEXT:    jge .LBB1_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    testl %esi, %esi
 ; CHECK-NEXT:    testl %edi, %esi
-; CHECK-NEXT:    jne .Ltmp1
+; CHECK-NEXT:    jne .LBB1_4
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    jmp .LBB1_3
 ; CHECK-NEXT:  .LBB1_2: # %if.else
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    testl %esi, %edi
 ; CHECK-NEXT:    testl %esi, %edi
-; CHECK-NEXT:    jne .Ltmp2
+; CHECK-NEXT:    jne .LBB1_5
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  .LBB1_3:
 ; CHECK-NEXT:    movl %esi, %eax
 ; CHECK-NEXT:    addl %edi, %eax
-; CHECK-NEXT:  .Ltmp2: # Block address taken
+; CHECK-NEXT:  .Ltmp1: # Block address taken
 ; CHECK-NEXT:  .LBB1_5: # %return
 ; CHECK-NEXT:    popl %esi
 ; CHECK-NEXT:    .cfi_def_cfa_offset 8
 ; CHECK-NEXT:    popl %edi
 ; CHECK-NEXT:    .cfi_def_cfa_offset 4
 ; CHECK-NEXT:    retl
-; CHECK-NEXT:  .Ltmp1: # Block address taken
+; CHECK-NEXT:  .Ltmp2: # Block address taken
 ; CHECK-NEXT:  .LBB1_4: # %label_true
 ; CHECK-NEXT:    .cfi_def_cfa_offset 12
 ; CHECK-NEXT:    movl $-2, %eax
 ; CHECK-NEXT:    jmp .LBB1_5
 entry:
   %cmp = icmp slt i32 %out1, %out2
   br i1 %cmp, label %if.then, label %if.else
 

 define i32 @test4(i32 %out1, i32 %out2) {
 ; CHECK-LABEL: test4:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl $-1, %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    testl %ecx, %ecx
 ; CHECK-NEXT:    testl %edx, %ecx
-; CHECK-NEXT:    jne .Ltmp4
+; CHECK-NEXT:    jne .LBB3_3
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %asm.fallthrough
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    testl %ecx, %edx
 ; CHECK-NEXT:    testl %ecx, %edx
-; CHECK-NEXT:    jne .Ltmp5
+; CHECK-NEXT:    jne .LBB3_4
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.2: # %asm.fallthrough2
 ; CHECK-NEXT:    addl %edx, %ecx
 ; CHECK-NEXT:    movl %ecx, %eax
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  .Ltmp4: # Block address taken
 ; CHECK-NEXT:  .LBB3_3: # %label_true
 ; CHECK-NEXT:    movl $-2, %eax

 ; into "={esp},{esp}". This previously caused an ICE; this test is more so
 ; about avoiding another ICE than what the actual output is.
 define dso_local void @test5() {
 ; CHECK-LABEL: test5:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  .Ltmp6: # Block address taken
-; CHECK-NEXT:  # %bb.1:
+; CHECK-NEXT:  .LBB4_1:
 ; CHECK-NEXT:    retl
   %1 = call i32 @llvm.read_register.i32(metadata !3)
   %2 = callbr i32 asm "", "={esp},!i,{esp},~{dirflag},~{fpsr},~{flags}"(i32 %1)
           to label %3 [label %4]
 
 3:
   call void @llvm.write_register.i32(metadata !3, i32 %2)
   br label %4
 
 4:
   ret void
 }
 
 declare i32 @llvm.read_register.i32(metadata)
 declare void @llvm.write_register.i32(metadata, i32)
 !3 = !{!"esp"}

llvm/test/CodeGen/X86/callbr-asm-sink.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
 
 ;; Verify that the machine instructions generated from the first
 ;; getelementptr don't get sunk below the callbr. (Reduced from a bug
 ;; report.)
 
 %struct1 = type { i8*, i32 }
 
 define void @klist_dec_and_del(%struct1*) {
 ; CHECK-LABEL: klist_dec_and_del:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    leaq 8(%rdi), %rax
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    # 8(%rdi) .Ltmp0
+; CHECK-NEXT:    # 8(%rdi) .LBB0_1
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.2:
 ; CHECK-NEXT:    retq
 ; CHECK-NEXT:  .Ltmp0: # Block address taken
 ; CHECK-NEXT:  .LBB0_1:
 ; CHECK-NEXT:    movq $0, -8(%rax)
 ; CHECK-NEXT:    retq
   %2 = getelementptr inbounds %struct1, %struct1* %0, i64 0, i32 1

llvm/test/CodeGen/X86/callbr-asm.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-- -O3 -verify-machineinstrs | FileCheck %s
 
 ; Tests for using callbr as an asm-goto wrapper
 
 ; Test 1 - fallthrough label gets removed, but the fallthrough code that is
 ; unreachable due to asm ending on a jmp is still left in.
 define i32 @test1(i32 %a) {
 ; CHECK-LABEL: test1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    addl $4, %eax
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmp .Ltmp0
+; CHECK-NEXT:    jmp .LBB0_2
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %normal
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  .Ltmp0: # Block address taken
 ; CHECK-NEXT:  .LBB0_2: # %fail
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retl

 define i32 @test1b(i32 %a) {
 ; CHECK-LABEL: test1b:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    addl $4, %eax
 ; CHECK-NEXT:    #APP
 ; CHECK-EMPTY:
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmp .Ltmp1
+; CHECK-NEXT:    jmp .LBB1_2
 ; CHECK-EMPTY:
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %normal
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  .Ltmp1: # Block address taken
 ; CHECK-NEXT:  .LBB1_2: # %fail
 ; CHECK-NEXT:    movl $1, %eax

 ; CHECK-NEXT:    # Child Loop BB3_4 Depth 4
 ; CHECK-NEXT:  .Ltmp5: # Block address taken
 ; CHECK-NEXT:  .LBB3_4: # %label04
 ; CHECK-NEXT:    # Parent Loop BB3_1 Depth=1
 ; CHECK-NEXT:    # Parent Loop BB3_2 Depth=2
 ; CHECK-NEXT:    # Parent Loop BB3_3 Depth=3
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=4
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    jmp .Ltmp2
-; CHECK-NEXT:    jmp .Ltmp3
-; CHECK-NEXT:    jmp .Ltmp4
+; CHECK-NEXT:    jmp .LBB3_1
+; CHECK-NEXT:    jmp .LBB3_2
+; CHECK-NEXT:    jmp .LBB3_3
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.5: # %normal0
 ; CHECK-NEXT:    # in Loop: Header=BB3_4 Depth=4
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    jmp .Ltmp2
-; CHECK-NEXT:    jmp .Ltmp3
-; CHECK-NEXT:    jmp .Ltmp4
-; CHECK-NEXT:    jmp .Ltmp5
+; CHECK-NEXT:    jmp .LBB3_1
+; CHECK-NEXT:    jmp .LBB3_2
+; CHECK-NEXT:    jmp .LBB3_3
+; CHECK-NEXT:    jmp .LBB3_4
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.6: # %normal1
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    retl
 entry:
   %a.addr = alloca i32, align 4
   store i32 %a, ptr %a.addr, align 4
   br label %label01
normal1:                                          ; preds = %normal0
   ret i32 %1
 }
 
 ; Test 4 - asm-goto referenced with the 'l' (ell) modifier and not.
 define void @test4() {
 ; CHECK-LABEL: test4:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    ja .Ltmp6
+; CHECK-NEXT:    ja .LBB4_3
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.1: # %asm.fallthrough
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    ja .Ltmp6
+; CHECK-NEXT:    ja .LBB4_3
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  .Ltmp6: # Block address taken
 ; CHECK-NEXT:  .LBB4_3: # %quux
 ; CHECK-NEXT:    retl
 entry:
   callbr void asm sideeffect "ja $0", "!i,~{dirflag},~{fpsr},~{flags}"()
           to label %asm.fallthrough [label %quux]
 

llvm/test/CodeGen/X86/inline-asm-pic.ll

 
 ; The intent of this test is to ensure that we handle blockaddress' correctly
 ; with "i" constraints for -m32 -fPIC.
 
 define void @x() {
 ; CHECK-LABEL: x:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    ## InlineAsm Start
-; CHECK-NEXT:    ## Ltmp0
+; CHECK-NEXT:    ## LBB1_1
 ; CHECK-EMPTY:
 ; CHECK-NEXT:    ## InlineAsm End
 ; CHECK-NEXT:  ## %bb.2: ## %return
 ; CHECK-NEXT:    retl
 ; CHECK-NEXT:  Ltmp0: ## Block address taken
 ; CHECK-NEXT:  LBB1_1: ## %overflow
 ; CHECK-NEXT:    retl
   callbr void asm "#  ${0:l}\0A", "!i"()
           to label %return [label %overflow]
 
 overflow:
   br label %return
 

llvm/test/CodeGen/X86/shrinkwrap-callbr.ll

 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    testl %edi, %edi
 ; CHECK-NEXT:    je .LBB0_3
 ; CHECK-NEXT:  # %bb.1: # %if.end
 ; CHECK-NEXT:    callq fn
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    # jump to .Ltmp0
+; CHECK-NEXT:    # jump to .LBB0_4
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:  # %bb.2: # %return
 ; CHECK-NEXT:    movl $4, %eax
 ; CHECK-NEXT:    popq %rcx
 ; CHECK-NEXT:    .cfi_def_cfa_offset 8
 ; CHECK-NEXT:    retq
 ; CHECK-NEXT:  .LBB0_3: # %ret0
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16

llvm/test/CodeGen/X86/tail-dup-asm-goto.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 ; RUN: llc -mtriple=x86_64-linux -stop-after=early-tailduplication < %s | FileCheck %s
 
 ; Ensure that we don't duplicate a block with an "INLINEASM_BR" instruction
 ; during code gen.
 declare dso_local void @foo()
 
 define ptr @test1(ptr %arg1, ptr %arg2) {
   ; CHECK-LABEL: name: test1
   ; CHECK: bb.0.bb:
   ; CHECK-NEXT:   successors: %bb.1(0x50000000), %bb.2(0x30000000)
   ; CHECK-NEXT:   liveins: $rdi, $rsi
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[COPY:%[0-9]+]]:gr64 = COPY $rsi
   ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:gr64 = COPY $rdi
   ; CHECK-NEXT:   [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm [[COPY1]], 1, $noreg, 0, $noreg :: (load (s64) from %ir.arg1)
   ; CHECK-NEXT:   [[SUB64rr:%[0-9]+]]:gr64 = SUB64rr [[MOV64rm]], [[COPY]], implicit-def $eflags
   ; CHECK-NEXT:   JCC_1 %bb.2, 4, implicit $eflags
   ; CHECK-NEXT:   JMP_1 %bb.1
define ptr @test1(ptr %arg1, ptr %arg2) {
   ; CHECK-NEXT:   ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
   ; CHECK-NEXT:   CALL64pcrel32 @foo, csr_64, implicit $rsp, implicit $ssp, implicit-def $rsp, implicit-def $ssp
   ; CHECK-NEXT:   ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.bb110:
   ; CHECK-NEXT:   successors: %bb.5(0x80000000), %bb.4(0x00000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[PHI:%[0-9]+]]:gr64 = PHI [[COPY]], %bb.2, [[MOV64rm]], %bb.1
-  ; CHECK-NEXT:   INLINEASM_BR &"#$0 $1 $2", 9 /* sideeffect mayload attdialect */, 13 /* imm */, 42, 13 /* imm */, 0, 13 /* imm */, blockaddress(@test1, %ir-block.bb17.i.i.i), 12 /* clobber */, implicit-def early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def early-clobber $eflags
+  ; CHECK-NEXT:   INLINEASM_BR &"#$0 $1 $2", 9 /* sideeffect mayload attdialect */, 13 /* imm */, 42, 13 /* imm */, 0, 13 /* imm */, %bb.4, 12 /* clobber */, implicit-def early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def early-clobber $eflags
   ; CHECK-NEXT:   JMP_1 %bb.5
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.4.bb17.i.i.i (address-taken, inlineasm-br-indirect-target):
   ; CHECK-NEXT:   successors: %bb.5(0x80000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.5.kmem_cache_has_cpu_partial.exit:
   ; CHECK-NEXT:   $rax = COPY [[PHI]]