[SystemZ] Support symbolic displacements.

[jonpa]

Summary

This patch aims to add support for symbolic displacements, e.g. like 'lg %r0, sym(%r1)', which is done by means of relocations. This is needed to compile the kernel with clang without disabling the integrated assembler.

[jonpa]

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCAsmBackend.cpp

147

IIUC, this is the place where the bit-offset should be applied, but I see another reloc kind with a bit offset that was not applied previously, and I am not sure exactly why... I could not yet find a way to test this either...

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCCodeEmitter.cpp

190

Still looking for a good way to find the byte offset for a displacement.

So far I have found only two categories of byte offsets:

1. The majority of these instructions seem to have the displacement at 2 bytes + 4 bit (=> starting at 20th bit).
2. MVC (and others) has a second address displacement with a 4 byte offset.
3. ...?

• It would be nice to somehow deduce the byte/bit offset from the OpNum from the MCInstrDesc, but I am not sure that's possible...
• Using TSFlags in the instruction descriptor might be an alternative..

202

Should these displacement fixups also deal with the TLS marker like in SystemZMCCodeEmitter::getPCRelEncoding()?

[jonpa]

• Mapping of MI:OperandIndex to fixup byte offset: I added a TSFlags bit 'MemMem' which is checked for in getDispOpValue(). From this flag it is known that there are two memory operands, with displacements starting at bit 20 and 36. For the matter of choosing which one of these are in question in getDispOpValue(), based on the OpNum, I was not sure if this could be hard coded as well in the formats file, or if it would be simpler to check this at compile time the way that is currently done..?

• Instead of in the AsmParser allowing specifically an MCSymbolRefExpr as displacement (which does not work with 'b-a'), the patch now allows any MCExpr if a symbol allowed as displacement. It seems unrelocatable expressions are caught by evaluateAsRelocatable().

• Removed previous changes in applyFixup(). It seems the bit offset is actually not needed there during the insertion of reversed bytes... Test added for this as well (reloc-absolute.s). Is there a need for a range check here (for a too-big displacement)? Handle the value for a FK_390_20 displacement by putting the high byte first - I hope that holds true for all uses...?

[jonpa]

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCAsmBackend.cpp

145

Is it right to truncate a Disp20 (and not Disp12), or maybe it doesn't matter?

[uweigand]

In D113341#3126096, @jonpa wrote:

• Mapping of MI:OperandIndex to fixup byte offset: I added a TSFlags bit 'MemMem' which is checked for in getDispOpValue(). From this flag it is known that there are two memory operands, with displacements starting at bit 20 and 36. For the matter of choosing which one of these are in question in getDispOpValue(), based on the OpNum, I was not sure if this could be hard coded as well in the formats file, or if it would be simpler to check this at compile time the way that is currently done..?

There's a number of options I can see, each of which has their own drawbacks.

1. We know that the encoding callbacks are called by TableGen-generated code in operand order. So we could simply track at which memory operands we are by counting them (i.e. add a member MemOpsEmitted or so to the SystemZMCCodeEmitter class, reset it to zero at the beginning of each instruction in encodeInstruction and increment it whenever emitting any of the BD* type operands). Then we could simply assume that the first memory operand has a byte offset of 2, and the second a byte offset of 4. This wouldn't even require any TSFlags bit.
2. We could follow the existing precedent with the pc-relative operands that the operand type encodes the byte offset. This means that for the MemMem operands, we'd have to use two different operand types. So in addition to bdaddr12only we'd have another type like bdaddr12second that would work the same way everywhere, except that the first type uses a byte offset of 2 for relocations while the second type uses a byte offset of 4. (We'd likewise need variants of bdaddr12onlylen4 and bdaddr12onlylen8, I think.) Then we can explicitly encode in each insn pattern where to use bdaddr12only and where to use bdaddr12second.
3. Ideally, TableGen should be able to handle this for us. It actually already knows exactly which bits the results of the operand encoding will end up at, because it needs to place them there. If we could get TableGen to pass that information (e.g. the mask and shift count) into the Encoding callback as well, then we wouldn't need to duplicate that information in the first place. See the generated SystemZMCCodeEmitter::getBinaryCodeForInstr function:

// op: BDL1
op = getBDLAddr12Len8Encoding(MI, 0, Fixups, STI);
op &= UINT64_C(16777215);
op <<= 16;
Value |= op;
// op: BD2
op = getBDAddr12Encoding(MI, 3, Fixups, STI);
op &= UINT64_C(65535);
Value |= op;
break;

I guess I'd be OK with an implementation along any of these lines.

• Instead of in the AsmParser allowing specifically an MCSymbolRefExpr as displacement (which does not work with 'b-a'), the patch now allows any MCExpr if a symbol allowed as displacement. It seems unrelocatable expressions are caught by evaluateAsRelocatable().

That seems correct to me.

• Removed previous changes in applyFixup(). It seems the bit offset is actually not needed there during the insertion of reversed bytes... Test added for this as well (reloc-absolute.s). Is there a need for a range check here (for a too-big displacement)? Handle the value for a FK_390_20 displacement by putting the high byte first - I hope that holds true for all uses...?

As to the bit offset, I believe this counts from the LSB, and so it would actually be zero for all of our fields.

I do believe we need to have a range check. Doesn't common code already do that for fixups? Might be good to check how other targets handle overflow here.

The 20-bit displacements are always encoded the same way, that's why they can share the same relocation type.

Also, I think we should have more tests, e.g. testing both operands of the MVC-type instructions, testing at least one representative of each of the relevant insn formats (SSa-f, SSE, SSF). We also should have tests both for the case where the fixup can be resolved, and where the fixup results in a relocation.

[jonpa]

1. We know that the encoding callbacks are called by TableGen-generated code in operand order. So we could simply track at which memory operands we are by counting them (i.e. add a member MemOpsEmitted or so to the SystemZMCCodeEmitter class, reset it to zero at the beginning of each instruction in encodeInstruction and increment it whenever emitting any of the BD* type operands). Then we could simply assume that the first memory operand has a byte offset of 2, and the second a byte offset of 4. This wouldn't even require any TSFlags bit.

I tried this and it was simple enough - with just an increment in a single place in getDispOpValue(). However, encodeInstruction() is an overriden const function, so clearing MemOpsEmitted wass not permitted. Then it occured to me that if we really can make this broad assumption, we could just check the size of the Fixups vector and that way differentiate between the first and second fixup.

As to the bit offset, I believe this counts from the LSB, and so it would actually be zero for all of our fields.

I tried clearing the bit offsets, but that broke some tests. It looks like the commenting algorithm is using these:

With bit offset: la	%r1, b-a(%r1)                   # encoding: [0x41,0x10,0b0001AAAA,A]
Without        : la	%r1, b-a(%r1)                   # encoding: [0x41,0x10,0x10'A',0bAAAA0000]    ### WRONG

So in this context it seems that the bit offset makes sense - it shifts the fixup bits to the right where they should be in little-endian order...

I do believe we need to have a range check. Doesn't common code already do that for fixups? Might be good to check how other targets handle overflow here.

It seems the targets do that themselves in applyFixup() during handling of Value/FixupKind ("adjustFixupValue()"), so I added these checks as well for SystemZ. I suppose they should also be added for the PCXXDBL:s as well? Separately then I suppose might be best?

The 20-bit displacements are always encoded the same way, that's why they can share the same relocation type.

right..

Also, I think we should have more tests, e.g. testing both operands of the MVC-type instructions, testing at least one representative of each of the relevant insn formats (SSa-f, SSE, SSF). We also should have tests both for the case where the fixup can be resolved, and where the fixup results in a relocation.

More tests added.

[uweigand]

In D113341#3131566, @jonpa wrote:

I tried this and it was simple enough - with just an increment in a single place in getDispOpValue(). However, encodeInstruction() is an overriden const function, so clearing MemOpsEmitted wass not permitted. Then it occured to me that if we really can make this broad assumption, we could just check the size of the Fixups vector and that way differentiate between the first and second fixup.

Ah, OK - even simpler.

As to the bit offset, I believe this counts from the LSB, and so it would actually be zero for all of our fields.

I tried clearing the bit offsets, but that broke some tests. It looks like the commenting algorithm is using these:

With bit offset: la	%r1, b-a(%r1)                   # encoding: [0x41,0x10,0b0001AAAA,A]
Without        : la	%r1, b-a(%r1)                   # encoding: [0x41,0x10,0x10'A',0bAAAA0000]    ### WRONG

So in this context it seems that the bit offset makes sense - it shifts the fixup bits to the right where they should be in little-endian order...

OK, looks like this is correct.

I do believe we need to have a range check. Doesn't common code already do that for fixups? Might be good to check how other targets handle overflow here.

It seems the targets do that themselves in applyFixup() during handling of Value/FixupKind ("adjustFixupValue()"), so I added these checks as well for SystemZ.

Thanks!

I suppose they should also be added for the PCXXDBL:s as well? Separately then I suppose might be best?

Agreed.

Otherwise this now LGTM - one minor nit inline.

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCAsmBackend.cpp

149

I think it might be preferabe to do the bit-swapping in extractBitsForFixup - that routine is supposed to return the bits to be placed in the instruction slot.

[jonpa]

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCAsmBackend.cpp

149

ah, yes - makes sense...

llvm/lib/Target/SystemZ/AsmParser/SystemZAsmParser.cpp

 #include <cstddef>
 #include <cstdint>
 #include <iterator>
 #include <memory>
 #include <string>
 
 using namespace llvm;
 
-// Return true if Expr is in the range [MinValue, MaxValue].
-static bool inRange(const MCExpr *Expr, int64_t MinValue, int64_t MaxValue) {
+// Return true if Expr is in the range [MinValue, MaxValue]. If AllowSymbol
+// is true any MCExpr is accepted (address displacement).
+static bool inRange(const MCExpr *Expr, int64_t MinValue, int64_t MaxValue,
+                    bool AllowSymbol = false) {
   if (auto *CE = dyn_cast<MCConstantExpr>(Expr)) {
     int64_t Value = CE->getValue();
     return Value >= MinValue && Value <= MaxValue;
   }
-  return false;
+  return AllowSymbol;
 }
 
 namespace {
 
 enum RegisterKind {
   GR32Reg,
   GRH32Reg,
   GR64Reg,
return (Kind == KindMem &&
              // A BDMem can be treated as a BDXMem in which the index
              // register field is 0.
              (Mem.MemKind == BDMem && MemKind == BDXMem)));
   }
   bool isMem(MemoryKind MemKind, RegisterKind RegKind) const {
     return isMem(MemKind) && Mem.RegKind == RegKind;
   }
   bool isMemDisp12(MemoryKind MemKind, RegisterKind RegKind) const {
-    return isMem(MemKind, RegKind) && inRange(Mem.Disp, 0, 0xfff);
+    return isMem(MemKind, RegKind) && inRange(Mem.Disp, 0, 0xfff, true);
   }
   bool isMemDisp20(MemoryKind MemKind, RegisterKind RegKind) const {
-    return isMem(MemKind, RegKind) && inRange(Mem.Disp, -524288, 524287);
+    return isMem(MemKind, RegKind) && inRange(Mem.Disp, -524288, 524287, true);
   }
   bool isMemDisp12Len4(RegisterKind RegKind) const {
     return isMemDisp12(BDLMem, RegKind) && inRange(Mem.Length.Imm, 1, 0x10);
   }
   bool isMemDisp12Len8(RegisterKind RegKind) const {
     return isMemDisp12(BDLMem, RegKind) && inRange(Mem.Length.Imm, 1, 0x100);
   }
 

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZInstPrinter.h

SystemZInstPrinter(const MCAsmInfo &MAI, const MCInstrInfo &MII,
     : MCInstPrinter(MAI, MII, MRI) {}
 
   // Automatically generated by tblgen.
   std::pair<const char *, uint64_t> getMnemonic(const MCInst *MI) override;
   void printInstruction(const MCInst *MI, uint64_t Address, raw_ostream &O);
   static const char *getRegisterName(unsigned RegNo);
 
   // Print an address with the given base, displacement and index.
-  static void printAddress(const MCAsmInfo *MAI, unsigned Base, int64_t Disp,
-                           unsigned Index, raw_ostream &O);
+  static void printAddress(const MCAsmInfo *MAI, unsigned Base,
+                           const MCOperand &DispMO, unsigned Index,
+                           raw_ostream &O);
 
   // Print the given operand.
   static void printOperand(const MCOperand &MO, const MCAsmInfo *MAI,
                            raw_ostream &O);
 
   static void printFormattedRegName(const MCAsmInfo *MAI, unsigned RegNo,
                                     raw_ostream &O);
 

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZInstPrinter.cpp

 
 using namespace llvm;
 
 #define DEBUG_TYPE "asm-printer"
 
 #include "SystemZGenAsmWriter.inc"
 
 void SystemZInstPrinter::printAddress(const MCAsmInfo *MAI, unsigned Base,
-                                      int64_t Disp, unsigned Index,
+                                      const MCOperand &DispMO, unsigned Index,
                                       raw_ostream &O) {
-  O << Disp;
+  printOperand(DispMO, MAI, O);
   if (Base || Index) {
     O << '(';
     if (Index) {
       printFormattedRegName(MAI, Index, O);
       if (Base)
         O << ',';
     }
     if (Base)

 
 void SystemZInstPrinter::printOperand(const MCInst *MI, int OpNum,
                                       raw_ostream &O) {
   printOperand(MI->getOperand(OpNum), &MAI, O);
 }
 
 void SystemZInstPrinter::printBDAddrOperand(const MCInst *MI, int OpNum,
                                             raw_ostream &O) {
-  printAddress(&MAI, MI->getOperand(OpNum).getReg(),
-               MI->getOperand(OpNum + 1).getImm(), 0, O);
+  printAddress(&MAI, MI->getOperand(OpNum).getReg(), MI->getOperand(OpNum + 1),
+               0, O);
 }
 
 void SystemZInstPrinter::printBDXAddrOperand(const MCInst *MI, int OpNum,
                                              raw_ostream &O) {
-  printAddress(&MAI, MI->getOperand(OpNum).getReg(),
-               MI->getOperand(OpNum + 1).getImm(),
+  printAddress(&MAI, MI->getOperand(OpNum).getReg(), MI->getOperand(OpNum + 1),
                MI->getOperand(OpNum + 2).getReg(), O);
 }
 
 void SystemZInstPrinter::printBDLAddrOperand(const MCInst *MI, int OpNum,
                                              raw_ostream &O) {
   unsigned Base = MI->getOperand(OpNum).getReg();
-  uint64_t Disp = MI->getOperand(OpNum + 1).getImm();
+  const MCOperand &DispMO = MI->getOperand(OpNum + 1);
   uint64_t Length = MI->getOperand(OpNum + 2).getImm();
-  O << Disp << '(' << Length;
+  printOperand(DispMO, &MAI, O);
+  O << '(' << Length;
   if (Base) {
     O << ",";
     printRegName(O, Base);
   }
   O << ')';
 }
 
 void SystemZInstPrinter::printBDRAddrOperand(const MCInst *MI, int OpNum,
                                              raw_ostream &O) {
   unsigned Base = MI->getOperand(OpNum).getReg();
-  uint64_t Disp = MI->getOperand(OpNum + 1).getImm();
+  const MCOperand &DispMO = MI->getOperand(OpNum + 1);
   unsigned Length = MI->getOperand(OpNum + 2).getReg();
-  O << Disp << "(";
+  printOperand(DispMO, &MAI, O);
+  O << "(";
   printRegName(O, Length);
   if (Base) {
     O << ",";
     printRegName(O, Base);
   }
   O << ')';
 }
 
 void SystemZInstPrinter::printBDVAddrOperand(const MCInst *MI, int OpNum,
                                              raw_ostream &O) {
-  printAddress(&MAI, MI->getOperand(OpNum).getReg(),
-               MI->getOperand(OpNum + 1).getImm(),
+  printAddress(&MAI, MI->getOperand(OpNum).getReg(), MI->getOperand(OpNum + 1),
                MI->getOperand(OpNum + 2).getReg(), O);
 }
 
 void SystemZInstPrinter::printCond4Operand(const MCInst *MI, int OpNum,
                                            raw_ostream &O) {
   static const char *const CondNames[] = {
     "o", "h", "nle", "l", "nhe", "lh", "ne",
     "e", "nlh", "he", "nl", "le", "nh", "no"
   };
   uint64_t Imm = MI->getOperand(OpNum).getImm();
   assert(Imm > 0 && Imm < 15 && "Invalid condition");
   O << CondNames[Imm - 1];
 }

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCAsmBackend.cpp

static uint64_t extractBitsForFixup(MCFixupKind Kind, uint64_t Value) {
 
   switch (unsigned(Kind)) {
   case SystemZ::FK_390_PC12DBL:
   case SystemZ::FK_390_PC16DBL:
   case SystemZ::FK_390_PC24DBL:
   case SystemZ::FK_390_PC32DBL:
     return (int64_t)Value / 2;
 
+  case SystemZ::FK_390_12:
+  case SystemZ::FK_390_20:
+    return Value;
+
   case SystemZ::FK_390_TLS_CALL:
     return 0;
   }
 
   llvm_unreachable("Unknown fixup kind!");
 }
 
 namespace {
#undef ELF_RELOC
   if (Type != -1u)
     return static_cast<MCFixupKind>(FirstLiteralRelocationKind + Type);
   return None;
 }
 
 const MCFixupKindInfo &
 SystemZMCAsmBackend::getFixupKindInfo(MCFixupKind Kind) const {
   const static MCFixupKindInfo Infos[SystemZ::NumTargetFixupKinds] = {
     { "FK_390_PC12DBL",  4, 12, MCFixupKindInfo::FKF_IsPCRel },
     { "FK_390_PC16DBL",  0, 16, MCFixupKindInfo::FKF_IsPCRel },
     { "FK_390_PC24DBL",  0, 24, MCFixupKindInfo::FKF_IsPCRel },
     { "FK_390_PC32DBL",  0, 32, MCFixupKindInfo::FKF_IsPCRel },
-    { "FK_390_TLS_CALL", 0, 0, 0 }
+    { "FK_390_TLS_CALL", 0, 0, 0 },
+    { "FK_390_12",       4, 12, 0 },
+    { "FK_390_20",       4, 20, 0 }
   };
 
   // Fixup kinds from .reloc directive are like R_390_NONE. They
   // do not require any extra processing.
   if (Kind >= FirstLiteralRelocationKind)
     return MCAsmBackend::getFixupKindInfo(FK_NONE);
 
   if (Kind < FirstTargetFixupKind)
void SystemZMCAsmBackend::applyFixup(const MCAssembler &Asm,
   unsigned Size = (BitSize + 7) / 8;
 
   assert(Offset + Size <= Data.size() && "Invalid fixup offset!");
 
   // Big-endian insertion of Size bytes.
   Value = extractBitsForFixup(Kind, Value);
   if (BitSize < 64)
     Value &= ((uint64_t)1 << BitSize) - 1;
+  if (Kind == (MCFixupKind)SystemZ::FK_390_20) {
+    // The high byte of a 20 bit displacement value comes first.
+    uint64_t DLo = Value & 0xfff;
+    uint64_t DHi = (Value >> 12) & 0xff;
+    Value = (DLo << 8) | DHi;
+  }
   unsigned ShiftValue = (Size * 8) - 8;
   for (unsigned I = 0; I != Size; ++I) {
     Data[Offset + I] |= uint8_t(Value >> ShiftValue);
     ShiftValue -= 8;
   }
 }
 
 bool SystemZMCAsmBackend::writeNopData(raw_ostream &OS, uint64_t Count,

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCCodeEmitter.cpp

 //===-- SystemZMCCodeEmitter.cpp - Convert SystemZ code to machine code ---===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the SystemZMCCodeEmitter class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/SystemZMCFixups.h"
 #include "MCTargetDesc/SystemZMCTargetDesc.h"
+#include "SystemZInstrInfo.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/MC/MCCodeEmitter.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCFixup.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
uint64_t getBinaryCodeForInstr(const MCInst &MI,
                                  const MCSubtargetInfo &STI) const;
 
   // Called by the TableGen code to get the binary encoding of operand
   // MO in MI.  Fixups is the list of fixups against MI.
   uint64_t getMachineOpValue(const MCInst &MI, const MCOperand &MO,
                              SmallVectorImpl<MCFixup> &Fixups,
                              const MCSubtargetInfo &STI) const;
 
+  // Return the displacement value for the OpNum operand. If it is a symbol,
+  // add a fixup for it and return 0.
+  uint64_t getDispOpValue(const MCInst &MI, unsigned OpNum,
+                          SmallVectorImpl<MCFixup> &Fixups,
+                          SystemZ::FixupKind Kind) const;
+
   // Called by the TableGen code to get the binary encoding of an address.
   // The index or length, if any, is encoded first, followed by the base,
   // followed by the displacement.  In a 20-bit displacement,
   // the low 12 bits are encoded before the high 8 bits.
   uint64_t getBDAddr12Encoding(const MCInst &MI, unsigned OpNum,
                                SmallVectorImpl<MCFixup> &Fixups,
                                const MCSubtargetInfo &STI) const;
   uint64_t getBDAddr20Encoding(const MCInst &MI, unsigned OpNum,
getMachineOpValue(const MCInst &MI, const MCOperand &MO,
                   const MCSubtargetInfo &STI) const {
   if (MO.isReg())
     return Ctx.getRegisterInfo()->getEncodingValue(MO.getReg());
   if (MO.isImm())
     return static_cast<uint64_t>(MO.getImm());
   llvm_unreachable("Unexpected operand type!");
 }
 
 uint64_t SystemZMCCodeEmitter::
+getDispOpValue(const MCInst &MI, unsigned OpNum,
+               SmallVectorImpl<MCFixup> &Fixups,
+               SystemZ::FixupKind Kind) const {
+  const MCInstrDesc &MCID = MCII.get(MI.getOpcode());
+  auto isDispOp = [&MCID](unsigned OpIdx) -> bool {
+    return OpIdx > 0 &&
+      MCID.OpInfo[OpIdx].OperandType == MCOI::OPERAND_MEMORY &&
+      MCID.OpInfo[OpIdx].RegClass == -1 &&
+      MCID.OpInfo[OpIdx - 1].OperandType == MCOI::OPERAND_MEMORY &&
+      MCID.OpInfo[OpIdx - 1].RegClass != -1;
+  };
+  assert(isDispOp(OpNum) && "Expected a displacement op after base reg op.");
+
+  const MCOperand &MO = MI.getOperand(OpNum);
+  if (MO.isImm())
+    return static_cast<uint64_t>(MO.getImm());
+  if (MO.isExpr()) {
+    unsigned ByteOffs = 2;
+    if (MCID.TSFlags & SystemZII::MemMemOp) {
+      unsigned DisplacementOps[2];
+      unsigned D = 0;
+      for (unsigned Idx = 1, e = MCID.getNumOperands(); Idx != e; ++Idx)
+        if (isDispOp(Idx))
+          DisplacementOps[D++] = Idx;
+      assert(D == 2 && "Expected exactly two memory operands.");
+      // The second MemMem displacement starts with a byte offset of 4.
+      if (OpNum == DisplacementOps[1])
+        ByteOffs = 4;
+    }
+    Fixups.push_back(MCFixup::create(ByteOffs, MO.getExpr(), (MCFixupKind)Kind));
+    return 0;
+  }
+  llvm_unreachable("Unexpected operand type!");
+}
+
+uint64_t SystemZMCCodeEmitter::
 getBDAddr12Encoding(const MCInst &MI, unsigned OpNum,
                     SmallVectorImpl<MCFixup> &Fixups,
                     const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   assert(isUInt<4>(Base) && isUInt<12>(Disp));
   return (Base << 12) | Disp;
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDAddr20Encoding(const MCInst &MI, unsigned OpNum,
                     SmallVectorImpl<MCFixup> &Fixups,
                     const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_20);
   assert(isUInt<4>(Base) && isInt<20>(Disp));
   return (Base << 20) | ((Disp & 0xfff) << 8) | ((Disp & 0xff000) >> 12);
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDXAddr12Encoding(const MCInst &MI, unsigned OpNum,
                      SmallVectorImpl<MCFixup> &Fixups,
                      const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   uint64_t Index = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI);
   assert(isUInt<4>(Base) && isUInt<12>(Disp) && isUInt<4>(Index));
   return (Index << 16) | (Base << 12) | Disp;
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDXAddr20Encoding(const MCInst &MI, unsigned OpNum,
                      SmallVectorImpl<MCFixup> &Fixups,
                      const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_20);
   uint64_t Index = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI);
   assert(isUInt<4>(Base) && isInt<20>(Disp) && isUInt<4>(Index));
   return (Index << 24) | (Base << 20) | ((Disp & 0xfff) << 8)
     | ((Disp & 0xff000) >> 12);
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDLAddr12Len4Encoding(const MCInst &MI, unsigned OpNum,
                          SmallVectorImpl<MCFixup> &Fixups,
                          const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   uint64_t Len  = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI) - 1;
   assert(isUInt<4>(Base) && isUInt<12>(Disp) && isUInt<4>(Len));
   return (Len << 16) | (Base << 12) | Disp;
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDLAddr12Len8Encoding(const MCInst &MI, unsigned OpNum,
                          SmallVectorImpl<MCFixup> &Fixups,
                          const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   uint64_t Len  = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI) - 1;
   assert(isUInt<4>(Base) && isUInt<12>(Disp) && isUInt<8>(Len));
   return (Len << 16) | (Base << 12) | Disp;
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDRAddr12Encoding(const MCInst &MI, unsigned OpNum,
                      SmallVectorImpl<MCFixup> &Fixups,
                      const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   uint64_t Len  = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI);
   assert(isUInt<4>(Base) && isUInt<12>(Disp) && isUInt<4>(Len));
   return (Len << 16) | (Base << 12) | Disp;
 }
 
 uint64_t SystemZMCCodeEmitter::
 getBDVAddr12Encoding(const MCInst &MI, unsigned OpNum,
                      SmallVectorImpl<MCFixup> &Fixups,
                      const MCSubtargetInfo &STI) const {
   uint64_t Base = getMachineOpValue(MI, MI.getOperand(OpNum), Fixups, STI);
-  uint64_t Disp = getMachineOpValue(MI, MI.getOperand(OpNum + 1), Fixups, STI);
+  uint64_t Disp = getDispOpValue(MI, OpNum + 1, Fixups, SystemZ::FK_390_12);
   uint64_t Index = getMachineOpValue(MI, MI.getOperand(OpNum + 2), Fixups, STI);
   assert(isUInt<4>(Base) && isUInt<12>(Disp) && isUInt<5>(Index));
   return (Index << 16) | (Base << 12) | Disp;
 }
 
 uint64_t
 SystemZMCCodeEmitter::getPCRelEncoding(const MCInst &MI, unsigned OpNum,
                                        SmallVectorImpl<MCFixup> &Fixups,

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCFixups.h

 namespace SystemZ {
 enum FixupKind {
   // These correspond directly to R_390_* relocations.
   FK_390_PC12DBL = FirstTargetFixupKind,
   FK_390_PC16DBL,
   FK_390_PC24DBL,
   FK_390_PC32DBL,
   FK_390_TLS_CALL,
+  FK_390_12,
+  FK_390_20,
 
   // Marker
   LastTargetFixupKind,
   NumTargetFixupKinds = LastTargetFixupKind - FirstTargetFixupKind
 };
 } // end namespace SystemZ
 } // end namespace llvm
 
 #endif

llvm/lib/Target/SystemZ/MCTargetDesc/SystemZMCObjectWriter.cpp

 
 // Return the relocation type for an absolute value of MCFixupKind Kind.
 static unsigned getAbsoluteReloc(unsigned Kind) {
   switch (Kind) {
   case FK_Data_1: return ELF::R_390_8;
   case FK_Data_2: return ELF::R_390_16;
   case FK_Data_4: return ELF::R_390_32;
   case FK_Data_8: return ELF::R_390_64;
+  case SystemZ::FK_390_12: return ELF::R_390_12;
+  case SystemZ::FK_390_20: return ELF::R_390_20;
   }
   llvm_unreachable("Unsupported absolute address");
 }
 
 // Return the relocation type for a PC-relative value of MCFixupKind Kind.
 static unsigned getPCRelReloc(unsigned Kind) {
   switch (Kind) {
   case FK_Data_2:                return ELF::R_390_PC16;

llvm/lib/Target/SystemZ/SystemZAsmPrinter.cpp

bool SystemZAsmPrinter::PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
   SystemZInstPrinter::printOperand(MCOp, MAI, OS);
   return false;
 }
 
 bool SystemZAsmPrinter::PrintAsmMemoryOperand(const MachineInstr *MI,
                                               unsigned OpNo,
                                               const char *ExtraCode,
                                               raw_ostream &OS) {
-  SystemZInstPrinter::printAddress(MAI, MI->getOperand(OpNo).getReg(),
-                                   MI->getOperand(OpNo + 1).getImm(),
-                                   MI->getOperand(OpNo + 2).getReg(), OS);
+  SystemZInstPrinter::
+    printAddress(MAI, MI->getOperand(OpNo).getReg(),
+                 MCOperand::createImm(MI->getOperand(OpNo + 1).getImm()),
   return false;
 }
 
 void SystemZAsmPrinter::emitEndOfAsmFile(Module &M) {
   emitStackMaps(SM);
 }
 
 // Force static initialization.
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeSystemZAsmPrinter() {
   RegisterAsmPrinter<SystemZAsmPrinter> X(getTheSystemZTarget());
 }

llvm/lib/Target/SystemZ/SystemZInstrFormats.td

class InstSystemZ<int size, dag outs, dag ins, string asmstr,
   // assumed to set CC to either CCMASK_LOGICAL_ZERO or
   // CCMASK_LOGICAL_NONZERO.
   bit IsLogical = 0;
 
   // True if the (add or sub) instruction sets CC like a compare of the
   // result against zero, but only if the 'nsw' flag is set.
   bit CCIfNoSignedWrap = 0;
 
+  // True for a mem-mem operation like MVC that has a second memory operand.
+  bit MemMemOp = 0;
+
   let TSFlags{0}     = SimpleBDXLoad;
   let TSFlags{1}     = SimpleBDXStore;
   let TSFlags{2}     = Has20BitOffset;
   let TSFlags{3}     = HasIndex;
   let TSFlags{4}     = Is128Bit;
   let TSFlags{9-5}   = AccessBytes;
   let TSFlags{13-10} = CCValues;
   let TSFlags{17-14} = CompareZeroCCMask;
   let TSFlags{18}    = CCMaskFirst;
   let TSFlags{19}    = CCMaskLast;
   let TSFlags{20}    = IsLogical;
   let TSFlags{21}    = CCIfNoSignedWrap;
+  let TSFlags{22}    = MemMemOp;
 }
 
 //===----------------------------------------------------------------------===//
 // Mappings between instructions
 //===----------------------------------------------------------------------===//
 
 // Return the version of an instruction that has an unsigned 12-bit
 // displacement.
class InstSMI<bits<8> op, dag outs, dag ins, string asmstr, list<dag> pattern>
 
   let Inst{47-40} = op;
   let Inst{39-36} = M1;
   let Inst{35-32} = 0;
   let Inst{31-16} = BD3;
   let Inst{15-0}  = RI2;
 }
 
+let MemMemOp = 1 in {
 class InstSSa<bits<8> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   : InstSystemZ<6, outs, ins, asmstr, pattern> {
   field bits<48> Inst;
   field bits<48> SoftFail = 0;
 
   bits<24> BDL1;
   bits<16> BD2;
 
class InstSSF<bits<12> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   bits<4>  R3;
 
   let Inst{47-40} = op{11-4};
   let Inst{39-36} = R3;
   let Inst{35-32} = op{3-0};
   let Inst{31-16} = BD1;
   let Inst{15-0}  = BD2;
 }
+} // MemMemOp
 
 class InstS<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   : InstSystemZ<4, outs, ins, asmstr, pattern> {
   field bits<32> Inst;
   field bits<32> SoftFail = 0;
 
   bits<16> BD2;
 

llvm/lib/Target/SystemZ/SystemZInstrInfo.h

 namespace llvm {
 
 class SystemZSubtarget;
 
 namespace SystemZII {
 
 enum {
   // See comments in SystemZInstrFormats.td.
   SimpleBDXLoad          = (1 << 0),
   SimpleBDXStore         = (1 << 1),
   Has20BitOffset         = (1 << 2),
   HasIndex               = (1 << 3),
   Is128Bit               = (1 << 4),
   AccessSizeMask         = (31 << 5),
   AccessSizeShift        = 5,
   CCValuesMask           = (15 << 10),
   CCValuesShift          = 10,
   CompareZeroCCMaskMask  = (15 << 14),
   CompareZeroCCMaskShift = 14,
   CCMaskFirst            = (1 << 18),
   CCMaskLast             = (1 << 19),
   IsLogical              = (1 << 20),
-  CCIfNoSignedWrap       = (1 << 21)
+  CCIfNoSignedWrap       = (1 << 21),
+  MemMemOp               = (1 << 22)
 };
 
 static inline unsigned getAccessSize(unsigned int Flags) {
   return (Flags & AccessSizeMask) >> AccessSizeShift;
 }
 
 static inline unsigned getCCValues(unsigned int Flags) {
   return (Flags & CCValuesMask) >> CCValuesShift;

llvm/test/MC/SystemZ/fixups.s

 
-# RUN: llvm-mc -triple s390x-unknown-unknown --show-encoding %s | FileCheck %s
+# RUN: llvm-mc -triple s390x-unknown-unknown -mcpu=z13 --show-encoding %s | FileCheck %s
 
-# RUN: llvm-mc -triple s390x-unknown-unknown -filetype=obj %s | \
+# RUN: llvm-mc -triple s390x-unknown-unknown -mcpu=z13 -filetype=obj %s | \
 # RUN: llvm-readobj -r - | FileCheck %s -check-prefix=CHECK-REL
 
 # CHECK: larl %r14, target                      # encoding: [0xc0,0xe0,A,A,A,A]
 # CHECK-NEXT:                                   # fixup A - offset: 2, value: target+2, kind: FK_390_PC32DBL
 # CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_PC32DBL target 0x2
   .align 16
   larl %r14, target
 

 # CHECK-NEXT:                                   # fixup A - offset: 2, value: target@PLT+2, kind: FK_390_PC16DBL
 # CHECK-NEXT:                                   # fixup B - offset: 0, value: sym@TLSLDM, kind: FK_390_TLS_CALL
 # CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_PLT16DBL target 0x2
 # CHECK-REL:                                    0x{{[0-9A-F]*0}} R_390_TLS_LDCALL sym 0x0
   .align 16
   bras %r14, target@plt:tls_ldcall:sym
 
 
+# Symbolic displacements
+# CHECK: lg %r14, src                           # encoding: [0xe3,0xe0,0b0000AAAA,A,A,0x04]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: src, kind: FK_390_20
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_20 src 0x0
+  .align 16
+  lg %r14, src
+
+# CHECK: la %r14, src(%r1)                      # encoding: [0x41,0xe0,0b0001AAAA,A]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: src, kind: FK_390_12
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_12 src 0x0
+        .align 16
+        la %r14, src(%r1)
+
+# CHECK: lay %r14, src(%r1,%r2)                 # encoding: [0xe3,0xe1,0b0010AAAA,A,A,0x71]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: src, kind: FK_390_20
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_20 src 0x0
+        .align 16
+        lay %r14, src(%r1, %r2)
+
+# CHECK: mvc dst(1,%r1), src(%r1)               # encoding: [0xd2,0x00,0b0001AAAA,A,0b0001BBBB,B]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: dst, kind: FK_390_12
+# CHECK-NEXT:                                   # fixup B - offset: 4, value: src, kind: FK_390_12
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_12 dst 0x0
+# CHECK-REL:                                    0x{{[0-9A-F]*4}} R_390_12 src 0x0
+        .align 16
+        mvc dst(1,%r1), src(%r1)
+
+# CHECK: lmd %r2, %r4, src1(%r1), src2(%r1)     # encoding: [0xef,0x24,0b0001AAAA,A,0b0001BBBB,B]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: src1, kind: FK_390_12
+# CHECK-NEXT:                                   # fixup B - offset: 4, value: src2, kind: FK_390_12
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_12 src1 0x0
+# CHECK-REL:                                    0x{{[0-9A-F]*4}} R_390_12 src2 0x0
+        .align 16
+        lmd    %r2, %r4, src1(%r1), src2(%r1)
+
+# CHECK: vl %v0, src                            # encoding: [0xe7,0x00,0b0000AAAA,A,0x00,0x06]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: src, kind: FK_390_12
+# CHECK-REL:                                    0x{{[0-9A-F]*2}} R_390_12 src 0x0
+        .align 16
+        vl %v0, src
+
+
 # Data relocs
 # llvm-mc does not show any "encoding" string for data, so we just check the relocs
 
 # CHECK-REL: .rela.data
   .data
 
 # CHECK-REL: 0x{{[0-9A-F]*0}} R_390_TLS_LE64 target 0x0
   .align 16

llvm/test/MC/SystemZ/reloc-absolute.s

+
+# RUN: llvm-mc -triple s390x-unknown-unknown --show-encoding %s | FileCheck %s
+
+# RUN: llvm-mc -triple s390x-unknown-unknown -filetype=obj %s | \
+# RUN: llvm-objdump -d - | FileCheck %s -check-prefix=CHECK-REL
+
+  .text
+
+# CHECK: la %r1, b-a(%r1)                       # encoding: [0x41,0x10,0b0001AAAA,A]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: b-a, kind: FK_390_12
+# CHECK-REL:                                    0: 41 10 10 04    la %r1, 4(%r1)
+  la     %r1, b-a(%r1)
+
+# CHECK: lay %r1, b-a(%r1)                      # encoding: [0xe3,0x10,0b0001AAAA,A,A,0x71]
+# CHECK-NEXT:                                   # fixup A - offset: 2, value: b-a, kind: FK_390_20
+# CHECK-REL:                                    4: e3 10 10 04 00 71    lay %r1, 4(%r1)
+  lay    %r1, b-a(%r1)
+
+  .type a,@object
+  .local  a
+  .comm a,4,4
+  .type b,@object
+  .local  b
+  .comm b,4,4

llvm/test/MC/SystemZ/reloc-directive.s

 # RUN: llvm-mc -triple=s390x-linux-gnu %s | FileCheck --check-prefix=PRINT %s
 
 # RUN: llvm-mc -filetype=obj -triple=s390x-linux-gnu %s -o %t
 # RUN: llvm-readobj -r %t | FileCheck %s
 
 # PRINT:      .reloc 2, R_390_NONE, .data
 # PRINT-NEXT: .reloc 1, R_390_NONE, foo+4
 # PRINT-NEXT: .reloc 0, R_390_NONE, 8
 # PRINT-NEXT: .reloc 0, R_390_64, .data+2
 # PRINT-NEXT: .reloc 0, R_390_GOTENT, foo+3
 # PRINT-NEXT: .reloc 0, R_390_PC32DBL, 6
+# PRINT-NEXT: .reloc 4, R_390_12, foo
+# PRINT-NEXT: .reloc 2, R_390_20, foo
 # PRINT:      .reloc 0, BFD_RELOC_NONE, 9
 # PRINT-NEXT: .reloc 0, BFD_RELOC_8, 9
 # PRINT-NEXT: .reloc 0, BFD_RELOC_16, 9
 # PRINT-NEXT: .reloc 0, BFD_RELOC_32, 9
 # PRINT-NEXT: .reloc 0, BFD_RELOC_64, 9
 
 # CHECK:      0x2 R_390_NONE .data 0x0
 # CHECK-NEXT: 0x1 R_390_NONE foo 0x4
 # CHECK-NEXT: 0x0 R_390_NONE - 0x8
 # CHECK-NEXT: 0x0 R_390_64 .data 0x2
 # CHECK-NEXT: 0x0 R_390_GOTENT foo 0x3
 # CHECK-NEXT: 0x0 R_390_PC32DBL - 0x6
+# CHECK-NEXT: 0x4 R_390_12 foo 0x0
+# CHECK-NEXT: 0x2 R_390_20 foo 0x0
 # CHECK-NEXT: 0x0 R_390_NONE - 0x9
 # CHECK-NEXT: 0x0 R_390_8 - 0x9
 # CHECK-NEXT: 0x0 R_390_16 - 0x9
 # CHECK-NEXT: 0x0 R_390_32 - 0x9
 # CHECK-NEXT: 0x0 R_390_64 - 0x9
 
 .text
   br %r14
   nop
   nop
   .reloc 2, R_390_NONE, .data
   .reloc 1, R_390_NONE, foo+4
   .reloc 0, R_390_NONE, 8
   .reloc 0, R_390_64, .data+2
   .reloc 0, R_390_GOTENT, foo+3
   .reloc 0, R_390_PC32DBL, 6
+  .reloc 4, R_390_12, foo
+  .reloc 2, R_390_20, foo
 
   .reloc 0, BFD_RELOC_NONE, 9
   .reloc 0, BFD_RELOC_8, 9
   .reloc 0, BFD_RELOC_16, 9
   .reloc 0, BFD_RELOC_32, 9
   .reloc 0, BFD_RELOC_64, 9
 
 .data
 .globl foo
 foo:
   .word 0
   .word 0