Enable index register memory constraints for inline ASM operands on SystemZ

[colpell]

Summary

This enables use of the 'R' and 'T' memory constraints for inline ASM operands on SystemZ, which allow an index register as well as an immediate displacement. This patch includes corresponding documentation and test case updates.

As with the last patch of this kind, I moved the 'm' constraint to the most general case, which is now 'T' (base + 20-bit signed displacement + index register).

[uweigand]

There's a typo in the documentation change, see inline annotations. Otherwise, LGTM.

docs/LangRef.rst

3598

The 20-bit immediates are *signed*, not unsigned like the 12-bit ones.

3600

Likewise.

[colpell]

Fix documentation typo (unsigned -> signed).

docs/LangRef.rst

 
 SystemZ:
 
 - ``I``: An immediate unsigned 8-bit integer.
 - ``J``: An immediate unsigned 12-bit integer.
 - ``K``: An immediate signed 16-bit integer.
 - ``L``: An immediate signed 20-bit integer.
 - ``M``: An immediate integer 0x7fffffff.
-- ``Q``, ``R``: A memory address operand with a base address and a 12-bit
-  immediate unsigned displacement.
-- ``S``, ``T``: A memory address operand with a base address and a 20-bit
-  immediate signed displacement.
+- ``Q``: A memory address operand with a base address and a 12-bit immediate
+  unsigned displacement.
+- ``R``: A memory address operand with a base address, a 12-bit immediate
+  unsigned displacement, and an index register.
+- ``S``: A memory address operand with a base address and a 20-bit immediate
+  signed displacement.
+- ``T``: A memory address operand with a base address, a 20-bit immediate
+  signed displacement, and an index register.
 - ``r`` or ``d``: A 32, 64, or 128-bit integer register.
 - ``a``: A 32, 64, or 128-bit integer address register (excludes R0, which in an
   address context evaluates as zero).
 - ``h``: A 32-bit value in the high part of a 64bit data register
   (LLVM-specific)
 - ``f``: A 32, 64, or 128-bit floating point register.
 
 X86:

 
 The ``llvm.bitsets`` global metadata is used to implement
 :doc:`bitsets <BitSets>`.
 
 '``invariant.group``' Metadata
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The ``invariant.group`` metadata may be attached to ``load``/``store`` instructions.
-The existence of the ``invariant.group`` metadata on the instruction tells 
-the optimizer that every ``load`` and ``store`` to the same pointer operand 
-within the same invariant group can be assumed to load or store the same  
-value (but see the ``llvm.invariant.group.barrier`` intrinsic which affects 
+The existence of the ``invariant.group`` metadata on the instruction tells
+the optimizer that every ``load`` and ``store`` to the same pointer operand
+within the same invariant group can be assumed to load or store the same
+value (but see the ``llvm.invariant.group.barrier`` intrinsic which affects
 when two pointers are considered the same).
 
 Examples:
 
 .. code-block:: llvm
 
    @unknownPtr = external global i8
    ...
    %ptr = alloca i8
    store i8 42, i8* %ptr, !invariant.group !0
    call void @foo(i8* %ptr)
-   
    %a = load i8, i8* %ptr, !invariant.group !0 ; Can assume that value under %ptr didn't change
    call void @foo(i8* %ptr)
    %b = load i8, i8* %ptr, !invariant.group !1 ; Can't assume anything, because group changed
-  
-   %newPtr = call i8* @getPointer(i8* %ptr) 
+   %newPtr = call i8* @getPointer(i8* %ptr)
    %c = load i8, i8* %newPtr, !invariant.group !0 ; Can't assume anything, because we only have information about %ptr
-   
    %unknownValue = load i8, i8* @unknownPtr
    store i8 %unknownValue, i8* %ptr, !invariant.group !0 ; Can assume that %unknownValue == 42
-   
    call void @foo(i8* %ptr)
    %newPtr2 = call i8* @llvm.invariant.group.barrier(i8* %ptr)
    %d = load i8, i8* %newPtr2, !invariant.group !0  ; Can't step through invariant.group.barrier to get value of %ptr
-   
    ...
    declare void @foo(i8*)
    declare i8* @getPointer(i8*)
    declare i8* @llvm.invariant.group.barrier(i8*)
-   
    !0 = !{!"magic ptr"}
    !1 = !{!"other ptr"}
 
 
 
 Module Flags Metadata
 =====================
 

 The optional ``!nontemporal`` metadata must reference a single metadata
 name ``<index>`` corresponding to a metadata node with one ``i32`` entry of
 value 1. The existence of the ``!nontemporal`` metadata on the instruction
 tells the optimizer and code generator that this load is not expected to
 be reused in the cache. The code generator may select special
 instructions to save cache bandwidth, such as the ``MOVNT`` instruction on
 x86.
 
-The optional ``!invariant.group`` metadata must reference a 
+The optional ``!invariant.group`` metadata must reference a
 single metadata name ``<index>``. See ``invariant.group`` metadata.
 
 Semantics:
 """"""""""
 
 The contents of memory are updated to contain ``<value>`` at the
 location specified by the ``<pointer>`` operand. If ``<value>`` is
 of scalar type then the number of bytes written does not exceed the

 
 Arguments:
 """"""""""
 
 There are three arguments to the '``cmpxchg``' instruction: an address
 to operate on, a value to compare to the value currently be at that
 address, and a new value to place at that address if the compared values
 are equal. The type of '<cmp>' must be an integer or pointer type whose
-bit width is a power of two greater than or equal to eight and less 
+bit width is a power of two greater than or equal to eight and less
 than or equal to a target-specific size limit. '<cmp>' and '<new>' must
-have the same type, and the type of '<pointer>' must be a pointer to 
-that type. If the ``cmpxchg`` is marked as ``volatile``, then the 
+have the same type, and the type of '<pointer>' must be a pointer to
+that type. If the ``cmpxchg`` is marked as ``volatile``, then the
 optimizer is not allowed to modify the number or order of execution of
 this ``cmpxchg`` with other :ref:`volatile operations <volatile>`.
 
 The success and failure :ref:`ordering <ordering>` arguments specify how this
 ``cmpxchg`` synchronizes with other atomic operations. Both ordering parameters
 must be at least ``monotonic``, the ordering constraint on failure must be no
 stronger than that on success, and the failure ordering cannot be either
 ``release`` or ``acq_rel``.
-  Option ``-tailcallopt`` is enabled, or
       ``llvm::GuaranteedTailCallOpt`` is ``true``.
    -  `Platform-specific constraints are
       met. <CodeGenerator.html#tailcallopt>`_
 
 #. The optional ``notail`` marker indicates that the optimizers should not add
    ``tail`` or ``musttail`` markers to the call. It is used to prevent tail
    call optimization from being performed on the call.
 
-#. The optional ``fast-math flags`` marker indicates that the call has one or more 
+#. The optional ``fast-math flags`` marker indicates that the call has one or more
    :ref:`fast-math flags <fastmath>`, which are optimization hints to enable
    otherwise unsafe floating-point optimizations. Fast-math flags are only valid
    for calls that return a floating-point scalar or vector type.
 
 #. The optional "cconv" marker indicates which :ref:`calling
    convention <callingconv>` the call should use. If none is
    specified, the call defaults to using C calling conventions. The
    calling convention of the call must match the calling convention of

 
 ::
 
       declare i8* @llvm.invariant.group.barrier(i8* <ptr>)
 
 Overview:
 """""""""
 
-The '``llvm.invariant.group.barrier``' intrinsic can be used when an invariant 
+The '``llvm.invariant.group.barrier``' intrinsic can be used when an invariant
 established by invariant.group metadata no longer holds, to obtain a new pointer
 value that does not carry the invariant information.
 
 
 Arguments:
 """"""""""
 
 The ``llvm.invariant.group.barrier`` takes only one argument, which is
 the pointer to the memory for which the ``invariant.group`` no longer holds.
 
 Semantics:
 """"""""""
 
-Returns another pointer that aliases its argument but which is considered different 
+Returns another pointer that aliases its argument but which is considered different
 for the purposes of ``load``/``store`` ``invariant.group`` metadata.
 
 General Intrinsics
 ------------------
 
 This class of intrinsics is designed to be generic and has no specific
 purpose.
 

lib/Target/SystemZ/README.txt

 //===---------------------------------------------------------------------===//
 // Random notes about and ideas for the SystemZ backend.
 //===---------------------------------------------------------------------===//
 
 The initial backend is deliberately restricted to z10.  We should add support
 for later architectures at some point.
 
 --
 
-SystemZDAGToDAGISel::SelectInlineAsmMemoryOperand() treats the Q and R
-constraints the same, and the S and T constraints the same, because the optional
-index is not used.
-
---
-
 If an inline asm ties an i32 "r" result to an i64 input, the input
 will be treated as an i32, leaving the upper bits uninitialised.
 For example:
 
 define void @f4(i32 *%dst) {
   %val = call i32 asm "blah $0", "=r,0" (i64 103)
   store i32 %val, i32 *%dst
   ret void

lib/Target/SystemZ/SystemZISelDAGToDAG.cpp

void SystemZDAGToDAGISel::Select(SDNode *Node) {
 
   SelectCode(Node);
 }
 
 bool SystemZDAGToDAGISel::
 SelectInlineAsmMemoryOperand(const SDValue &Op,
                              unsigned ConstraintID,
                              std::vector<SDValue> &OutOps) {
+  SystemZAddressingMode::AddrForm Form;
+  SystemZAddressingMode::DispRange DispRange;
   SDValue Base, Disp, Index;
 
   switch(ConstraintID) {
   default:
     llvm_unreachable("Unexpected asm memory constraint");
   case InlineAsm::Constraint_i:
   case InlineAsm::Constraint_Q:
+    // Accept an address with a short displacement, but no index.
+    Form = SystemZAddressingMode::FormBD;
+    DispRange = SystemZAddressingMode::Disp12Only;
+    break;
   case InlineAsm::Constraint_R:
-    // Accept addresses with short displacements, which are compatible
-    // with Q and R. But keep the index operand for future expansion (e.g. the
-    // index for R).
-    if (selectBDXAddr(SystemZAddressingMode::FormBD,
-                      SystemZAddressingMode::Disp12Only,
-                      Op, Base, Disp, Index)) {
-      OutOps.push_back(Base);
-      OutOps.push_back(Disp);
-      OutOps.push_back(Index);
-      return false;
-    }
+    // Accept an address with a short displacement and an index.
+    Form = SystemZAddressingMode::FormBDXNormal;
+    DispRange = SystemZAddressingMode::Disp12Only;
     break;
   case InlineAsm::Constraint_S:
+    // Accept an address with a long displacement, but no index.
+    Form = SystemZAddressingMode::FormBD;
+    DispRange = SystemZAddressingMode::Disp20Only;
+    break;
   case InlineAsm::Constraint_T:
   case InlineAsm::Constraint_m:
-    // Accept addresses with long displacements. As above, keep the index for
-    // future implementation of index for the T constraint.
-    if (selectBDXAddr(SystemZAddressingMode::FormBD,
-                      SystemZAddressingMode::Disp20Only,
-                      Op, Base, Disp, Index)) {
-      OutOps.push_back(Base);
-      OutOps.push_back(Disp);
-      OutOps.push_back(Index);
-      return false;
-    }
-    break;
-  }
+    // Accept an address with a long displacement and an index.
+    // m works the same as T, as this is the most general case.
+    Form = SystemZAddressingMode::FormBDXNormal;
+    DispRange = SystemZAddressingMode::Disp20Only;
+    break;
+  }
+
+  if (selectBDXAddr(Form, DispRange, Op, Base, Disp, Index)) {
+
   return true;
 }

test/CodeGen/SystemZ/asm-02.ll

 ; CHECK: blah 0(%r2)
 ; CHECK: br %r14
   %add = add i64 %base, 4096
   %addr = inttoptr i64 %add to i64 *
   call void asm "blah $0", "=*R" (i64 *%addr)
   ret void
 }
 
-; FIXME: at the moment the precise constraint is not passed down to
-; target code, so we must conservatively treat "R" as "Q".
+; Check that indices are allowed
+define void @f5(i64 %base, i64 %index) {
+; CHECK-LABEL: f5:
+; CHECK: blah 0(%r3,%r2)
+; CHECK: br %r14
+  %add = add i64 %base, %index
+  %addr = inttoptr i64 %add to i64 *
+  call void asm "blah $0", "=*R" (i64 *%addr)
+  ret void
+}
+
+; Check that indices and displacements are allowed simultaneously
+define void @f6(i64 %base, i64 %index) {
+; CHECK-LABEL: f6:
+; CHECK: blah 4095(%r3,%r2)
+; CHECK: br %r14
+  %add = add i64 %base, 4095
+  %addi = add i64 %add, %index
+  %addr = inttoptr i64 %addi to i64 *
+  call void asm "blah $0", "=*R" (i64 *%addr)
+  ret void
+}
+
+; Check that LAY is used if there is an index but the displacement is too large
+define void @f7(i64 %base, i64 %index) {
+; CHECK-LABEL: f7:
+; CHECK: lay %r0, 4096(%r3,%r2)
+; CHECK: blah 0(%r0)
+; CHECK: br %r14
+  %add = add i64 %base, 4096
+  %addi = add i64 %add, %index
+  %addr = inttoptr i64 %addi to i64 *
+  call void asm "blah $0", "=*R" (i64 *%addr)
+  ret void
+}

test/CodeGen/SystemZ/asm-04.ll

 ; Test the "T" asm constraint, which accepts addresses that have a base,
 ; an index and a 20-bit displacement.
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -no-integrated-as | FileCheck %s
 
+; Check the lowest range.
 define void @f1(i64 %base) {
 ; CHECK-LABEL: f1:
+; CHECK: blah -524288(%r2)
+; CHECK: br %r14
+  %add = add i64 %base, -524288
+  %addr = inttoptr i64 %add to i64 *
+  call void asm "blah $0", "=*T" (i64 *%addr)
+  ret void
+}
+
+; Check the next lowest byte.
+define void @f2(i64 %base) {
+; CHECK-LABEL: f2:
+; CHECK: agfi %r2, -524289
 ; CHECK: blah 0(%r2)
 ; CHECK: br %r14
-  %addr = inttoptr i64 %base to i64 *
+  %add = add i64 %base, -524289
+  %addr = inttoptr i64 %add to i64 *
   call void asm "blah $0", "=*T" (i64 *%addr)
   ret void
 }
 
-; FIXME: at the moment the precise constraint is not passed down to
-; target code, so we must conservatively treat "T" as "S".
+; Check the highest range.
+define void @f3(i64 %base) {
+; CHECK-LABEL: f3:
+; CHECK: blah 524287(%r2)
+; CHECK: br %r14
+  %add = add i64 %base, 524287
+  %addr = inttoptr i64 %add to i64 *
+  call void asm "blah $0", "=*T" (i64 *%addr)
+  ret void
+}
+
+; Check the next highest byte.
+define void @f4(i64 %base) {
+; CHECK-LABEL: f4:
+; CHECK: agfi %r2, 524288
+; CHECK: blah 0(%r2)
+; CHECK: br %r14
+  %add = add i64 %base, 524288
+  %addr = inttoptr i64 %add to i64 *
+  call void asm "blah $0", "=*T" (i64 *%addr)
+  ret void
+}
+
+; Check that indices are allowed
+define void @f5(i64 %base, i64 %index) {
+; CHECK-LABEL: f5:
+; CHECK: blah 0(%r3,%r2)
+; CHECK: br %r14
+  %add = add i64 %base, %index
+  %addr = inttoptr i64 %add to i64 *
+  call void asm "blah $0", "=*T" (i64 *%addr)
+  ret void
+}
+
+; Check that indices and displacements are allowed simultaneously
+define void @f6(i64 %base, i64 %index) {
+; CHECK-LABEL: f6:
+; CHECK: blah 524287(%r3,%r2)
+; CHECK: br %r14
+  %add = add i64 %base, 524287
+  %addi = add i64 %add, %index
+  %addr = inttoptr i64 %addi to i64 *
+  call void asm "blah $0", "=*T" (i64 *%addr)
+  ret void
+}

test/CodeGen/SystemZ/asm-05.ll

 ; Test the "m" asm constraint, which is equivalent to "T".
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -no-integrated-as | FileCheck %s
 
 define void @f1(i64 %base) {
 ; CHECK-LABEL: f1:
 ; CHECK: blah 0(%r2)
 ; CHECK: br %r14
   %addr = inttoptr i64 %base to i64 *
   call void asm "blah $0", "=*m" (i64 *%addr)
   ret void
 }
-
-; FIXME: at the moment the precise constraint is not passed down to
-; target code, so we must conservatively treat "m" as "S".