[GlobalISel][InlineAsm] Add support for basic output operand constraints

[arsenm]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

90

Don't need to repeat function name

92

Start with lowercase

119–120

Picking types from classes is usually problematic for me. Do we really need this type, and if so, is it feasible to use LLT instead?

140

RC->contains()?

214

Start with lowercase

[kschwarz]

Address review comments

[kschwarz]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

119–120

Nope, I actually had a FIXME to remove it. We should only need the register size, and this we can always recompute.

[arsenm]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

105

This should probably propagate that this is an error with the function return value?

120

Specify physical register in comment?

128–131

Looking again this assert doesn't really make sense. You are finding the registers from the reg class iterator, so obviously it is in the class already

133

Picking multiple physical registers by scanning through the register class doesn't make any sense to me. I would expect NumRegs > 1 only with a virtual register

154

Move this into default case?

380

Reg.isPhysical()

[kschwarz]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

120

Which comment do you mean?

128–131

I've mostly copied this function from the SelectionDAG code, but I agree this assertion should never trigger. I couldn't find out from the history why this assertion was added, maybe because the target provides both the physical register and the register class, and they have to match?
I'll remove the assertion in the next round.

133

The LLVM IR reference manual contains the following paragraph:

There is also an “interesting” feature which deserves a bit of explanation: if a register class constraint allocates a register which is too small for the value type operand provided as input, the input value will be split into multiple registers, and all of them passed to the inline asm.

X86 uses this at least with the 'A' register constraint, but there are other targets, too (e.g. ARM uses it to pass 64-bit values in even-odd register pairs).

154

At least my Xcode warns about adding a default case for a switch which covers all enumeration values. I think there's also a similar notice in the development guide?

[kschwarz]

Address some review comments

[arsenm]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

133

With physical registers? Can you add a testcase for this?

[kschwarz]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

133

I can add a fictitious test for AArch64, but it doesn't have asm template argument modifiers to access the "higher" register.
ARM has test cases for this (for SelectionDAG though) here: https://github.com/llvm/llvm-project/blob/master/llvm/test/CodeGen/ARM/inlineasm-operand-implicit-cast.ll#L291

[paquette]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

118–123

Comments seem out of place here?

Move them closer to the code that corresponds to them?

128–131

Maybe just use std::find_if?

Also what should happen if you don't find the assigned register here?

179

Extra parens around if?

222–224

Remove braces on single-line if?

316

s/node/instruction/

355–359

Can this be a range-based for?

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-inline-asm.ll

3

-verify-machineinstrs?

[kschwarz]

Rebase & address more review comments

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

128–131

Previously, there was an assertion if the end of the iterator was reached without finding the register, but I removed it in another iteration.
Since the physical register and the register class are both provided by the target, there isn't really anything else we can do than abort the compilation if they don't match. I'm adding back the assertion for this case then.

133

Turns out I can't add a test for this at the moment, because I'm bailing out if an output constraint needs more than one register (see line 408).
Do you want me to add support for it in this patch?

[arsenm]

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

251

Add a TRI variable instead of calling getSubtarget().getRegisterInfo() repeatedly through the function

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

693–694

This part isn't tested, I would leave this for a separate patch. I'm not sure checking for inline asm explicitly is the right thing here

[kschwarz]

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

693–694

Sure I can remove this change from the patch. I temporarily removed the -stop-after argument from the irtranslator test to run down to assembly, which resulted in a crash in the register bank selector of AArch64.
Since we cannot have generic virtual registers on INLINEASM instructions, I figured not trying to select a register bank for these instructions would be fine.

[kschwarz]

Hoist call to getRegisterInfo() and create local variable. Fix a few clangTidy issues

[kschwarz]

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

693–694

After removing this line, another AArch64 test starts failing (stack_guard_remat.ll). It triggers the very same crash in the register bank selector of AArch64: llvm::LLT::getScalarSizeInBits() const: Assertion 'RawData != 0 && "Invalid Type"'.

What do you think would be an appropriate check during register bank selection?

[arsenm]

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

693–694

OK, can you add an explicit regbankselect for asm test then?

[kschwarz]

Add regbank-select test.

[kschwarz]

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

693–694

I've added a test and extracted the MI.isInlineAsm() check similar to the MI.isDebugInstr() below.

Note, however, that I had to specifically add the register class annotations to the input MIR (for the virtual register definitions of inline asm instructions), because it is not printed by the MIRPrinter by default. This makes the current output of MIRPrinter for inline asm instructions not round-trippable.

[kschwarz]

@arsenm, is this good to go now?

[arsenm]

LGTM

llvm/include/llvm/CodeGen/GlobalISel/InlineAsmLowering.h

 //===- llvm/CodeGen/GlobalISel/InlineAsmLowering.h --------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file describes how to lower LLVM inline asm to machine code INLINEASM.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CODEGEN_GLOBALISEL_INLINEASMLOWERING_H
 #define LLVM_CODEGEN_GLOBALISEL_INLINEASMLOWERING_H
 
+#include "llvm/ADT/ArrayRef.h"
+#include <functional>
+
 namespace llvm {
 class CallBase;
 class MachineIRBuilder;
+class Register;
 class TargetLowering;
+class Value;
 
 class InlineAsmLowering {
   const TargetLowering *TLI;
 
   virtual void anchor();
 
 public:
-  bool lowerInlineAsm(MachineIRBuilder &MIRBuilder, const CallBase &CB) const;
+  bool lowerInlineAsm(MachineIRBuilder &MIRBuilder, const CallBase &CB,
+                      std::function<ArrayRef<Register>(const Value &Val)>
+                          GetOrCreateVRegs) const;
 
 protected:
   /// Getter for generic TargetLowering class.
   const TargetLowering *getTLI() const { return TLI; }
 
   /// Getter for target specific TargetLowering class.
   template <class XXXTargetLowering> const XXXTargetLowering *getTLI() const {
     return static_cast<const XXXTargetLowering *>(TLI);

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

bool IRTranslator::translateInlineAsm(const CallBase &CB,
   const InlineAsmLowering *ALI = MF->getSubtarget().getInlineAsmLowering();
 
   if (!ALI) {
     LLVM_DEBUG(
         dbgs() << "Inline asm lowering is not supported for this target yet\n");
     return false;
   }
 
-  return ALI->lowerInlineAsm(MIRBuilder, CB);
+  return ALI->lowerInlineAsm(
+      MIRBuilder, CB, [&](const Value &Val) { return getOrCreateVRegs(Val); });
 }
 
 bool IRTranslator::translateCallBase(const CallBase &CB,
                                      MachineIRBuilder &MIRBuilder) {
   ArrayRef<Register> Res = getOrCreateVRegs(CB);
 
   SmallVector<ArrayRef<Register>, 8> Args;
   Register SwiftInVReg = 0;

llvm/lib/CodeGen/GlobalISel/InlineAsmLowering.cpp

 #include "llvm/IR/Module.h"
 
 #define DEBUG_TYPE "inline-asm-lowering"
 
 using namespace llvm;
 
 void InlineAsmLowering::anchor() {}
 
-bool InlineAsmLowering::lowerInlineAsm(MachineIRBuilder &MIRBuilder,
-                                       const CallBase &Call) const {
+namespace {
 
+/// GISelAsmOperandInfo - This contains information for each constraint that we
+/// are lowering.
+class GISelAsmOperandInfo : public TargetLowering::AsmOperandInfo {
+public:
+  /// Regs - If this is a register or register class operand, this
+  /// contains the set of assigned registers corresponding to the operand.
+  SmallVector<Register, 1> Regs;
+
+  explicit GISelAsmOperandInfo(const TargetLowering::AsmOperandInfo &Info)
+      : TargetLowering::AsmOperandInfo(Info) {}
+};
+
+using GISelAsmOperandInfoVector = SmallVector<GISelAsmOperandInfo, 16>;
+
+class ExtraFlags {
+  unsigned Flags = 0;
+
+public:
+  explicit ExtraFlags(const CallBase &CB) {
+    const InlineAsm *IA = cast<InlineAsm>(CB.getCalledOperand());
+    if (IA->hasSideEffects())
+      Flags |= InlineAsm::Extra_HasSideEffects;
+    if (IA->isAlignStack())
+      Flags |= InlineAsm::Extra_IsAlignStack;
+    if (CB.isConvergent())
+      Flags |= InlineAsm::Extra_IsConvergent;
+    Flags |= IA->getDialect() * InlineAsm::Extra_AsmDialect;
+  }
+
+  void update(const TargetLowering::AsmOperandInfo &OpInfo) {
+    // Ideally, we would only check against memory constraints.  However, the
+    // meaning of an Other constraint can be target-specific and we can't easily
+    // reason about it.  Therefore, be conservative and set MayLoad/MayStore
+    // for Other constraints as well.
+    if (OpInfo.ConstraintType == TargetLowering::C_Memory ||
+        OpInfo.ConstraintType == TargetLowering::C_Other) {
+      if (OpInfo.Type == InlineAsm::isInput)
+        Flags |= InlineAsm::Extra_MayLoad;
+      else if (OpInfo.Type == InlineAsm::isOutput)
+        Flags |= InlineAsm::Extra_MayStore;
+      else if (OpInfo.Type == InlineAsm::isClobber)
+        Flags |= (InlineAsm::Extra_MayLoad | InlineAsm::Extra_MayStore);
+    }
+  }
+
+  unsigned get() const { return Flags; }
+};
+
+} // namespace
+
+/// Assign virtual/physical registers for the specified register operand.
+static void getRegistersForValue(MachineFunction &MF,
+                                 MachineIRBuilder &MIRBuilder,
+                                 GISelAsmOperandInfo &OpInfo,
+                                 GISelAsmOperandInfo &RefOpInfo) {
+
+  const TargetLowering &TLI = *MF.getSubtarget().getTargetLowering();
+  const TargetRegisterInfo &TRI = *MF.getSubtarget().getRegisterInfo();
+
+  // No work to do for memory operations.
+  if (OpInfo.ConstraintType == TargetLowering::C_Memory)
+    return;
+
+  // If this is a constraint for a single physreg, or a constraint for a
+  // register class, find it.
+  Register AssignedReg;
+  const TargetRegisterClass *RC;
+  std::tie(AssignedReg, RC) = TLI.getRegForInlineAsmConstraint(
+      &TRI, RefOpInfo.ConstraintCode, RefOpInfo.ConstraintVT);
+  // RC is unset only on failure. Return immediately.
+  if (!RC)
+    return;
+
+  // No need to allocate a matching input constraint since the constraint it's
+  // matching to has already been allocated.
+  if (OpInfo.isMatchingInputConstraint())
+    return;
+
+  // Initialize NumRegs.
+  unsigned NumRegs = 1;
+  if (OpInfo.ConstraintVT != MVT::Other)
+    NumRegs =
+        TLI.getNumRegisters(MF.getFunction().getContext(), OpInfo.ConstraintVT);
+
+  // If this is a constraint for a specific physical register, but the type of
+  // the operand requires more than one register to be passed, we allocate the
+  // required amount of physical registers, starting from the selected physical
+  // register.
+  // For this, first retrieve a register iterator for the given register class
+  TargetRegisterClass::iterator I = RC->begin();
+  MachineRegisterInfo &RegInfo = MF.getRegInfo();
+
+  // Advance the iterator to the assigned register (if set)
+  if (AssignedReg) {
+    for (; *I != AssignedReg; ++I)
+      assert(I != RC->end() && "AssignedReg should be a member of provided RC");
+  }
+
+  // Finally, assign the registers. If the AssignedReg isn't set, create virtual
+  // registers with the provided register class
+  for (; NumRegs; --NumRegs, ++I) {
+    assert(I != RC->end() && "Ran out of registers to allocate!");
+    Register R = AssignedReg ? Register(*I) : RegInfo.createVirtualRegister(RC);
+    OpInfo.Regs.push_back(R);
+  }
+}
+
+/// Return an integer indicating how general CT is.
+static unsigned getConstraintGenerality(TargetLowering::ConstraintType CT) {
+  switch (CT) {
+  case TargetLowering::C_Immediate:
+  case TargetLowering::C_Other:
+  case TargetLowering::C_Unknown:
+    return 0;
+  case TargetLowering::C_Register:
+    return 1;
+  case TargetLowering::C_RegisterClass:
+    return 2;
+  case TargetLowering::C_Memory:
+    return 3;
+  }
+  llvm_unreachable("Invalid constraint type");
+}
+
+static void chooseConstraint(TargetLowering::AsmOperandInfo &OpInfo,
+                             const TargetLowering *TLI) {
+  assert(OpInfo.Codes.size() > 1 && "Doesn't have multiple constraint options");
+  unsigned BestIdx = 0;
+  TargetLowering::ConstraintType BestType = TargetLowering::C_Unknown;
+  int BestGenerality = -1;
+
+  // Loop over the options, keeping track of the most general one.
+  for (unsigned i = 0, e = OpInfo.Codes.size(); i != e; ++i) {
+    TargetLowering::ConstraintType CType =
+        TLI->getConstraintType(OpInfo.Codes[i]);
+
+    // Indirect 'other' or 'immediate' constraints are not allowed.
+    if (OpInfo.isIndirect && !(CType == TargetLowering::C_Memory ||
+                               CType == TargetLowering::C_Register ||
+                               CType == TargetLowering::C_RegisterClass))
+      continue;
+
+    // If this is an 'other' or 'immediate' constraint, see if the operand is
+    // valid for it. For example, on X86 we might have an 'rI' constraint. If
+    // the operand is an integer in the range [0..31] we want to use I (saving a
+    // load of a register), otherwise we must use 'r'.
+    if (CType == TargetLowering::C_Other ||
+        CType == TargetLowering::C_Immediate) {
+      assert(OpInfo.Codes[i].size() == 1 &&
+             "Unhandled multi-letter 'other' constraint");
+      // FIXME: prefer immediate constraints if the target allows it
+    }
+
+    // Things with matching constraints can only be registers, per gcc
+    // documentation.  This mainly affects "g" constraints.
+    if (CType == TargetLowering::C_Memory && OpInfo.hasMatchingInput())
+      continue;
+
+    // This constraint letter is more general than the previous one, use it.
+    int Generality = getConstraintGenerality(CType);
+    if (Generality > BestGenerality) {
+      BestType = CType;
+      BestIdx = i;
+      BestGenerality = Generality;
+    }
+  }
+
+  OpInfo.ConstraintCode = OpInfo.Codes[BestIdx];
+  OpInfo.ConstraintType = BestType;
+}
+
+static void computeConstraintToUse(const TargetLowering *TLI,
+                                   TargetLowering::AsmOperandInfo &OpInfo) {
+  assert(!OpInfo.Codes.empty() && "Must have at least one constraint");
+
+  // Single-letter constraints ('r') are very common.
+  if (OpInfo.Codes.size() == 1) {
+    OpInfo.ConstraintCode = OpInfo.Codes[0];
+    OpInfo.ConstraintType = TLI->getConstraintType(OpInfo.ConstraintCode);
+  } else {
+    chooseConstraint(OpInfo, TLI);
+  }
+
+  // 'X' matches anything.
+  if (OpInfo.ConstraintCode == "X" && OpInfo.CallOperandVal) {
+    // Labels and constants are handled elsewhere ('X' is the only thing
+    // that matches labels).  For Functions, the type here is the type of
+    // the result, which is not what we want to look at; leave them alone.
+    Value *Val = OpInfo.CallOperandVal;
+    if (isa<BasicBlock>(Val) || isa<ConstantInt>(Val) || isa<Function>(Val))
+      return;
+
+    // Otherwise, try to resolve it to something we know about by looking at
+    // the actual operand type.
+    if (const char *Repl = TLI->LowerXConstraint(OpInfo.ConstraintVT)) {
+      OpInfo.ConstraintCode = Repl;
+      OpInfo.ConstraintType = TLI->getConstraintType(OpInfo.ConstraintCode);
+    }
+  }
+}
+
+bool InlineAsmLowering::lowerInlineAsm(
+    MachineIRBuilder &MIRBuilder, const CallBase &Call,
+    std::function<ArrayRef<Register>(const Value &Val)> GetOrCreateVRegs)
+    const {
   const InlineAsm *IA = cast<InlineAsm>(Call.getCalledOperand());
-  StringRef ConstraintStr = IA->getConstraintString();
 
-  bool HasOnlyMemoryClobber = false;
-  if (!ConstraintStr.empty()) {
-    // Until we have full inline assembly support, we just try to handle the
-    // very simple case of just "~{memory}" to avoid falling back so often.
-    if (ConstraintStr != "~{memory}")
+  /// ConstraintOperands - Information about all of the constraints.
+  GISelAsmOperandInfoVector ConstraintOperands;
+
+  MachineFunction &MF = MIRBuilder.getMF();
+  const Function &F = MF.getFunction();
+  const DataLayout &DL = F.getParent()->getDataLayout();
+  const TargetRegisterInfo *TRI = MF.getSubtarget().getRegisterInfo();
+
+  MachineRegisterInfo *MRI = MIRBuilder.getMRI();
+
+  TargetLowering::AsmOperandInfoVector TargetConstraints =
+      TLI->ParseConstraints(DL, TRI, Call);
+
+  ExtraFlags ExtraInfo(Call);
+  unsigned ResNo = 0; // ResNo - The result number of the next output.
+  for (auto &T : TargetConstraints) {
+    ConstraintOperands.push_back(GISelAsmOperandInfo(T));
+    GISelAsmOperandInfo &OpInfo = ConstraintOperands.back();
+
+    // Compute the value type for each operand.
+    if (OpInfo.Type == InlineAsm::isInput ||
+        (OpInfo.Type == InlineAsm::isOutput && OpInfo.isIndirect)) {
+
+      LLVM_DEBUG(dbgs() << "Input operands and indirect output operands are "
+                           "not supported yet\n");
       return false;
-    HasOnlyMemoryClobber = true;
+
+    } else if (OpInfo.Type == InlineAsm::isOutput && !OpInfo.isIndirect) {
+      assert(!Call.getType()->isVoidTy() && "Bad inline asm!");
+      if (StructType *STy = dyn_cast<StructType>(Call.getType())) {
+        OpInfo.ConstraintVT =
+            TLI->getSimpleValueType(DL, STy->getElementType(ResNo));
+      } else {
+        assert(ResNo == 0 && "Asm only has one result!");
+        OpInfo.ConstraintVT = TLI->getSimpleValueType(DL, Call.getType());
+      }
+      ++ResNo;
+    } else {
+      OpInfo.ConstraintVT = MVT::Other;
     }
 
-  unsigned ExtraInfo = 0;
-  if (IA->hasSideEffects())
-    ExtraInfo |= InlineAsm::Extra_HasSideEffects;
-  if (IA->getDialect() == InlineAsm::AD_Intel)
-    ExtraInfo |= InlineAsm::Extra_AsmDialect;
-
-  // HACK: special casing for ~memory.
-  if (HasOnlyMemoryClobber)
-    ExtraInfo |= (InlineAsm::Extra_MayLoad | InlineAsm::Extra_MayStore);
+    // Compute the constraint code and ConstraintType to use.
+    computeConstraintToUse(TLI, OpInfo);
+
+    // The selected constraint type might expose new sideeffects
+    ExtraInfo.update(OpInfo);
+  }
 
-  auto Inst = MIRBuilder.buildInstr(TargetOpcode::INLINEASM)
+  // At this point, all operand types are decided.
+  // Create the MachineInstr, but don't insert it yet since input
+  // operands still need to insert instructions before this one
+  auto Inst = MIRBuilder.buildInstrNoInsert(TargetOpcode::INLINEASM)
                   .addExternalSymbol(IA->getAsmString().c_str())
-                  .addImm(ExtraInfo);
+                  .addImm(ExtraInfo.get());
+
+  // Collects the output operands for later processing
+  GISelAsmOperandInfoVector OutputOperands;
+
+  for (auto &OpInfo : ConstraintOperands) {
+    GISelAsmOperandInfo &RefOpInfo =
+        OpInfo.isMatchingInputConstraint()
+            ? ConstraintOperands[OpInfo.getMatchedOperand()]
+            : OpInfo;
+
+    // Assign registers for register operands
+    getRegistersForValue(MF, MIRBuilder, OpInfo, RefOpInfo);
+
+    switch (OpInfo.Type) {
+    case InlineAsm::isOutput:
+      if (OpInfo.ConstraintType == TargetLowering::C_Memory) {
+        unsigned ConstraintID =
+            TLI->getInlineAsmMemConstraint(OpInfo.ConstraintCode);
+        assert(ConstraintID != InlineAsm::Constraint_Unknown &&
+               "Failed to convert memory constraint code to constraint id.");
+
+        // Add information to the INLINEASM instruction to know about this
+        // output.
+        unsigned OpFlags = InlineAsm::getFlagWord(InlineAsm::Kind_Mem, 1);
+        OpFlags = InlineAsm::getFlagWordForMem(OpFlags, ConstraintID);
+        Inst.addImm(OpFlags);
+        ArrayRef<Register> SourceRegs =
+            GetOrCreateVRegs(*OpInfo.CallOperandVal);
+        assert(
+            SourceRegs.size() == 1 &&
+            "Expected the memory output to fit into a single virtual register");
+        Inst.addReg(SourceRegs[0]);
+      } else {
+        // Otherwise, this outputs to a register (directly for C_Register /
+        // C_RegisterClass. Find a register that we can use.
+        assert(OpInfo.ConstraintType == TargetLowering::C_Register ||
+               OpInfo.ConstraintType == TargetLowering::C_RegisterClass);
+
+        if (OpInfo.Regs.empty()) {
+          LLVM_DEBUG(dbgs()
+                     << "Couldn't allocate output register for constraint\n");
+          return false;
+        }
+
+        // Add information to the INLINEASM instruction to know that this
+        // register is set.
+        unsigned Flag = InlineAsm::getFlagWord(
+            OpInfo.isEarlyClobber ? InlineAsm::Kind_RegDefEarlyClobber
+                                  : InlineAsm::Kind_RegDef,
+            OpInfo.Regs.size());
+        if (OpInfo.Regs.front().isVirtual()) {
+          // Put the register class of the virtual registers in the flag word.
+          // That way, later passes can recompute register class constraints for
+          // inline assembly as well as normal instructions. Don't do this for
+          // tied operands that can use the regclass information from the def.
+          const TargetRegisterClass *RC = MRI->getRegClass(OpInfo.Regs.front());
+          Flag = InlineAsm::getFlagWordForRegClass(Flag, RC->getID());
+        }
+
+        Inst.addImm(Flag);
+
+        for (Register Reg : OpInfo.Regs) {
+          Inst.addReg(Reg,
+                      RegState::Define | getImplRegState(Reg.isPhysical()));
+        }
+
+        // Remember this output operand for later processing
+        OutputOperands.push_back(OpInfo);
+      }
+
+      break;
+    case InlineAsm::isInput:
+      return false;
+    case InlineAsm::isClobber: {
+
+      unsigned NumRegs = OpInfo.Regs.size();
+      if (NumRegs > 0) {
+        unsigned Flag =
+            InlineAsm::getFlagWord(InlineAsm::Kind_Clobber, NumRegs);
+        Inst.addImm(Flag);
+
+        for (Register Reg : OpInfo.Regs) {
+          Inst.addReg(Reg, RegState::Define | RegState::EarlyClobber |
+                               getImplRegState(Reg.isPhysical()));
+        }
+      }
+      break;
+    }
+    }
+  }
+
   if (const MDNode *SrcLoc = Call.getMetadata("srcloc"))
     Inst.addMetadata(SrcLoc);
 
+  // All inputs are handled, insert the instruction now
+  MIRBuilder.insertInstr(Inst);
+
+  // Finally, copy the output operands into the output registers
+  ArrayRef<Register> ResRegs = GetOrCreateVRegs(Call);
+  if (ResRegs.size() != OutputOperands.size()) {
+    LLVM_DEBUG(dbgs() << "Expected the number of output registers to match the "
+                         "number of destination registers\n");
+    return false;
+  }
+  for (unsigned int i = 0, e = ResRegs.size(); i < e; i++) {
+    GISelAsmOperandInfo &OpInfo = OutputOperands[i];
+
+    if (OpInfo.Regs.empty())
+      continue;
+
+    switch (OpInfo.ConstraintType) {
+    case TargetLowering::C_Register:
+    case TargetLowering::C_RegisterClass: {
+      if (OpInfo.Regs.size() > 1) {
+        LLVM_DEBUG(dbgs() << "Output operands with multiple defining "
+                             "registers are not supported yet\n");
+        return false;
+      }
+
+      Register SrcReg = OpInfo.Regs[0];
+      unsigned SrcSize = TRI->getRegSizeInBits(SrcReg, *MRI);
+      if (MRI->getType(ResRegs[i]).getSizeInBits() < SrcSize) {
+        // First copy the non-typed virtual register into a generic virtual
+        // register
+        Register Tmp1Reg =
+            MRI->createGenericVirtualRegister(LLT::scalar(SrcSize));
+        MIRBuilder.buildCopy(Tmp1Reg, SrcReg);
+        // Need to truncate the result of the register
+        MIRBuilder.buildTrunc(ResRegs[i], Tmp1Reg);
+      } else {
+        MIRBuilder.buildCopy(ResRegs[i], SrcReg);
+      }
+      break;
+    }
+    case TargetLowering::C_Immediate:
+    case TargetLowering::C_Other:
+      LLVM_DEBUG(
+          dbgs() << "Cannot lower target specific output constraints yet\n");
+      return false;
+    case TargetLowering::C_Memory:
+      break; // Already handled.
+    case TargetLowering::C_Unknown:
+      LLVM_DEBUG(dbgs() << "Unexpected unknown constraint\n");
+      return false;
+    }
+  }
+
   return true;
 }

llvm/lib/CodeGen/GlobalISel/RegBankSelect.cpp

for (MachineBasicBlock *MBB : RPOT) {
     for (MachineBasicBlock::iterator MII = MBB->begin(), End = MBB->end();
          MII != End;) {
       // MI might be invalidated by the assignment, so move the
       // iterator before hand.
       MachineInstr &MI = *MII++;
 
       // Ignore target-specific post-isel instructions: they should use proper
       // regclasses.
       if (isTargetSpecificOpcode(MI.getOpcode()) && !MI.isPreISelOpcode())
         continue;
 
+      // Ignore inline asm instructions: they should use physical
+      // registers/regclasses
+      if (MI.isInlineAsm())
+        continue;
+
       // Ignore debug info.
       if (MI.isDebugInstr())
         continue;
 
       if (!assignInstr(MI)) {
         reportGISelFailure(MF, *TPC, *MORE, "gisel-regbankselect",
                            "unable to map instruction", MI);
         return false;

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-inline-asm.ll

+; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+; RUN: llc -mtriple=aarch64-darwin-ios13 -O0 -global-isel -stop-after=irtranslator -verify-machineinstrs -o - %s | FileCheck %s
+
+define void @asm_simple_memory_clobber() {
+  ; CHECK-LABEL: name: asm_simple_memory_clobber
+  ; CHECK: bb.1 (%ir-block.0):
+  ; CHECK:   INLINEASM &"", 25 /* sideeffect mayload maystore attdialect */, !0
+  ; CHECK:   INLINEASM &"", 1 /* sideeffect attdialect */, !0
+  ; CHECK:   RET_ReallyLR
+  call void asm sideeffect "", "~{memory}"(), !srcloc !0
+  call void asm sideeffect "", ""(), !srcloc !0
+  ret void
+}
+
+!0 = !{i32 70}
+
+define void @asm_simple_register_clobber() {
+  ; CHECK-LABEL: name: asm_simple_register_clobber
+  ; CHECK: bb.1 (%ir-block.0):
+  ; CHECK:   INLINEASM &"mov x0, 7", 1 /* sideeffect attdialect */, 12 /* clobber */, implicit-def early-clobber $x0, !0
+  ; CHECK:   RET_ReallyLR
+  call void asm sideeffect "mov x0, 7", "~{x0}"(), !srcloc !0
+  ret void
+}
+
+define i32 @test_specific_register_output() nounwind ssp {
+  ; CHECK-LABEL: name: test_specific_register_output
+  ; CHECK: bb.1.entry:
+  ; CHECK:   INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 10 /* regdef */, implicit-def $w0
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK:   $w0 = COPY [[COPY]](s32)
+  ; CHECK:   RET_ReallyLR implicit $w0
+entry:
+  %0 = tail call i32 asm "mov ${0:w}, 7", "={w0}"() nounwind
+  ret i32 %0
+}
+
+define i32 @test_single_register_output() nounwind ssp {
+  ; CHECK-LABEL: name: test_single_register_output
+  ; CHECK: bb.1.entry:
+  ; CHECK:   INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY %0
+  ; CHECK:   $w0 = COPY [[COPY]](s32)
+  ; CHECK:   RET_ReallyLR implicit $w0
+entry:
+  %0 = tail call i32 asm "mov ${0:w}, 7", "=r"() nounwind
+  ret i32 %0
+}
+
+define i64 @test_single_register_output_s64() nounwind ssp {
+  ; CHECK-LABEL: name: test_single_register_output_s64
+  ; CHECK: bb.1.entry:
+  ; CHECK:   INLINEASM &"mov $0, 7", 0 /* attdialect */, 1441802 /* regdef:GPR64common */, def %0
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s64) = COPY %0
+  ; CHECK:   $x0 = COPY [[COPY]](s64)
+  ; CHECK:   RET_ReallyLR implicit $x0
+entry:
+  %0 = tail call i64 asm "mov $0, 7", "=r"() nounwind
+  ret i64 %0
+}
+
+; Check support for returning several floats
+define float @test_multiple_register_outputs_same() #0 {
+  ; CHECK-LABEL: name: test_multiple_register_outputs_same
+  ; CHECK: bb.1 (%ir-block.0):
+  ; CHECK:   INLINEASM &"mov $0, #0; mov $1, #0", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0, 655370 /* regdef:GPR32common */, def %1
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY %0
+  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY %1
+  ; CHECK:   [[FADD:%[0-9]+]]:_(s32) = G_FADD [[COPY]], [[COPY1]]
+  ; CHECK:   $s0 = COPY [[FADD]](s32)
+  ; CHECK:   RET_ReallyLR implicit $s0
+  %1 = call { float, float } asm "mov $0, #0; mov $1, #0", "=r,=r"()
+  %asmresult = extractvalue { float, float } %1, 0
+  %asmresult1 = extractvalue { float, float } %1, 1
+  %add = fadd float %asmresult, %asmresult1
+  ret float %add
+}
+
+; Check support for returning several floats
+define double @test_multiple_register_outputs_mixed() #0 {
+  ; CHECK-LABEL: name: test_multiple_register_outputs_mixed
+  ; CHECK: bb.1 (%ir-block.0):
+  ; CHECK:   INLINEASM &"mov $0, #0; mov $1, #0", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0, 1245194 /* regdef:FPR64 */, def %1
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY %0
+  ; CHECK:   [[COPY1:%[0-9]+]]:_(s64) = COPY %1
+  ; CHECK:   $d0 = COPY [[COPY1]](s64)
+  ; CHECK:   RET_ReallyLR implicit $d0
+  %1 = call { float, double } asm "mov $0, #0; mov $1, #0", "=r,=w"()
+  %asmresult = extractvalue { float, double } %1, 1
+  ret double %asmresult
+}
+
+define i32 @test_specific_register_output_trunc() nounwind ssp {
+  ; CHECK-LABEL: name: test_specific_register_output_trunc
+  ; CHECK: bb.1.entry:
+  ; CHECK:   INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 10 /* regdef */, implicit-def $x0
+  ; CHECK:   [[COPY:%[0-9]+]]:_(s64) = COPY $x0
+  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[COPY]](s64)
+  ; CHECK:   $w0 = COPY [[TRUNC]](s32)
+  ; CHECK:   RET_ReallyLR implicit $w0
+entry:
+  %0 = tail call i32 asm "mov ${0:w}, 7", "={x0}"() nounwind
+  ret i32 %0
+}
+
+define zeroext i8 @test_register_output_trunc(i8* %src) nounwind {
+  ; CHECK-LABEL: name: test_register_output_trunc
+  ; CHECK: bb.1.entry:
+  ; CHECK:   liveins: $x0
+  ; CHECK:   [[COPY:%[0-9]+]]:_(p0) = COPY $x0
+  ; CHECK:   INLINEASM &"mov ${0:w}, 32", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %1
+  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY %1
+  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY1]](s32)
+  ; CHECK:   [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[TRUNC]](s8)
+  ; CHECK:   $w0 = COPY [[ZEXT]](s32)
+  ; CHECK:   RET_ReallyLR implicit $w0
+entry:
+  %0 = tail call i8 asm "mov ${0:w}, 32", "=r"() nounwind
+  ret i8 %0
+}
+
+define float @test_vector_output() nounwind {
+  ; CHECK-LABEL: name: test_vector_output
+  ; CHECK: bb.1 (%ir-block.0):
+  ; CHECK:   [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+  ; CHECK:   INLINEASM &"fmov ${0}.2s, #1.0", 1 /* sideeffect attdialect */, 10 /* regdef */, implicit-def $d14
+  ; CHECK:   [[COPY:%[0-9]+]]:_(<2 x s32>) = COPY $d14
+  ; CHECK:   [[EVEC:%[0-9]+]]:_(s32) = G_EXTRACT_VECTOR_ELT [[COPY]](<2 x s32>), [[C]](s64)
+  ; CHECK:   $s0 = COPY [[EVEC]](s32)
+  ; CHECK:   RET_ReallyLR implicit $s0
+  %1 = tail call <2 x float> asm sideeffect "fmov ${0}.2s, #1.0", "={v14}"() nounwind
+  %2 = extractelement <2 x float> %1, i32 0
+  ret float %2
+}

llvm/test/CodeGen/AArch64/GlobalISel/regbank-inlineasm.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=aarch64-unknown-unknown -verify-machineinstrs -O0 -run-pass=regbankselect %s -o - | FileCheck %s
+
+---
+name:            inlineasm_memory_clobber
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: inlineasm_memory_clobber
+    ; CHECK: INLINEASM &"", 25 /* sideeffect mayload maystore attdialect */
+    ; CHECK: INLINEASM &"", 1 /* sideeffect attdialect */
+    ; CHECK: RET_ReallyLR
+    INLINEASM &"", 25
+    INLINEASM &"", 1
+    RET_ReallyLR
+...
+
+---
+name:            inlineasm_register_clobber
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: inlineasm_register_clobber
+    ; CHECK: INLINEASM &"", 25 /* sideeffect mayload maystore attdialect */, 12 /* clobber */, implicit-def early-clobber $d0
+    ; CHECK: RET_ReallyLR
+    INLINEASM &"", 25, 12, implicit-def early-clobber $d0
+    RET_ReallyLR
+...
+
+---
+name:            inlineasm_phys_reg_output
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: inlineasm_phys_reg_output
+    ; CHECK: INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 10 /* regdef */, implicit-def $w0
+    ; CHECK: [[COPY:%[0-9]+]]:gpr(s32) = COPY $w0
+    ; CHECK: $w0 = COPY [[COPY]](s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 10 /* regdef */, implicit-def $w0
+    %0:_(s32) = COPY $w0
+    $w0 = COPY %0(s32)
+    RET_ReallyLR implicit $w0
+...
+
+---
+name:            inlineasm_virt_reg_output
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: inlineasm_virt_reg_output
+    ; CHECK: INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0
+    ; CHECK: [[COPY:%[0-9]+]]:gpr(s32) = COPY %0
+    ; CHECK: $w0 = COPY [[COPY]](s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    INLINEASM &"mov ${0:w}, 7", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0:gpr32common
+    %1:_(s32) = COPY %0
+    $w0 = COPY %1(s32)
+    RET_ReallyLR implicit $w0
+...
+
+---
+name:            inlineasm_virt_mixed_types
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: inlineasm_virt_mixed_types
+    ; CHECK: INLINEASM &"mov $0, #0; mov $1, #0", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0, 1245194 /* regdef:FPR64 */, def %1
+    ; CHECK: [[COPY:%[0-9]+]]:gpr(s32) = COPY %0
+    ; CHECK: [[COPY1:%[0-9]+]]:fpr(s64) = COPY %1
+    ; CHECK: $d0 = COPY [[COPY1]](s64)
+    ; CHECK: RET_ReallyLR implicit $d0
+    INLINEASM &"mov $0, #0; mov $1, #0", 0 /* attdialect */, 655370 /* regdef:GPR32common */, def %0:gpr32common, 1245194 /* regdef:FPR64 */, def %1:fpr64
+    %3:_(s32) = COPY %0
+    %4:_(s64) = COPY %1
+    $d0 = COPY %4(s64)
+    RET_ReallyLR implicit $d0
+...

llvm/test/CodeGen/AArch64/GlobalISel/translate-inline-asm.ll

-; RUN: llc -mtriple=aarch64-darwin-ios13 -O0 -global-isel -stop-after=irtranslator -o - %s | FileCheck %s
-
-; The update_mir_test_checks script doesn't seem to handle INLINE_ASM well. Write this manually.
-
-define void @asm_simple_memory_clobber() {
-  ; CHECK-LABEL: name: asm_simple_memory_clobber
-  ; CHECK: INLINEASM &"", 25
-  ; CHECK:  INLINEASM &"", 1
-  call void asm sideeffect "", "~{memory}"(), !srcloc !0
-  call void asm sideeffect "", ""(), !srcloc !0
-  ret void
-}
-
-!0 = !{i32 70}