[SystemZ] Emit .gnu_attribute for an externally visible vector abi.

[jonpa]

Summary

This is the SystemZ part originally part of https://reviews.llvm.org/D102894.

On SystemZ, the vector ABI changes depending on the presence of hardware vector support. Therefore, each binary compiled with a visible vector ABI (e.g. calling an external function with a vector argument) should be marked with a .gnu_attribute describing this.

This adds support on SystemZ to emit a gnu attribute for the vector abi both to asm and object files.

[jonpa]

This is missing some cases. We need to catch any type that contains a vector type somewhere, which includes in particular aggregate types like array and structs, since the different alignment of the member may in turn trigger a different alignment of the aggregate. E.g. a struct that contains a member of vector type should trigger the attribute.

right... added recursive handling for array/struct types.

We also possibly need to catch function pointer types where the pointed-to function has a vector (or derived) type as argument or return value. (However, that may be overkill: it really only "counts" if such a type is *used* either to perform a call via function pointer, or else if the address of a local function is passed as pointer to some external user. If this is too difficult to determine, then I guess it's OK to just count all function pointer types.)

Added handling for FunctionType. In both these cases the globally called function is found to have a user and then inspected for the function type.

[uweigand]

llvm/lib/Target/SystemZ/SystemZAsmPrinter.cpp

766

Now that isVectorTypeBased supports function types, I think the code below can be replaced by a single call to isVectorTypeBased(FTy).

776

I don't think this implementation of vararg support is quite complete. On the caller side, you check for variable arguments to any direct function with a prototype. However, this misses the case where an *indirect* function call (using a function pointer) has variable arguments.

This also completely misses the *callee* side: every function that takes variable arguments and uses va_arg with a vector type is also dependent on the vector ABI.

[jonpa]

I don't think this implementation of vararg support is quite complete. On the caller side, you check for variable arguments to any direct function with a prototype. However, this misses the case where an *indirect* function call (using a function pointer) has variable arguments.

This also completely misses the *callee* side: every function that takes variable arguments and uses va_arg with a vector type is also dependent on the vector ABI.

Patch updated to address these cases as well.

[uweigand]

In D105067#2857971, @jonpa wrote:

I don't think this implementation of vararg support is quite complete. On the caller side, you check for variable arguments to any direct function with a prototype. However, this misses the case where an *indirect* function call (using a function pointer) has variable arguments.

This also completely misses the *callee* side: every function that takes variable arguments and uses va_arg with a vector type is also dependent on the vector ABI.

Patch updated to address these cases as well.

Unfortunately I think this still catches only a subset of those cases. For the indirect calls, you now check those using a global function pointer variable. But calls via a local function pointer (e.g. a callback argument) have the same problem. For the callee side, you now check exported functions that use va_start - but we also need to handle functions that get a va_list as argument, as well as non-exported functions (you can pass va_list through to other routines).

GCC checks every call instruction in the whole compilation unit that uses variable argument lists, and it also checks every use of va_arg. I believe we may have to do the same. (This is a problem since va_arg is already expanded by the front-end. I'm starting to think this whole logic might be better placed in clang; we can just set a flag there when we expand a call or a va_arg, and don't have to scan everything after the fact ...)

[jonpa]

we also need to handle functions that get a va_list as argument, as well as non-exported functions (you can pass va_list through to other routines).

Patch updated per review.

Callee:

• if va_arg with vector type is lowered in a global function, VecABI is visible. Needs to be checked in FE (SystemZABIInfo::EmitVAArg).
• FunctionType of global function is checked in SystemZAsmPrinter.

Caller:

Since we need to inspect all calls (not just to globally declared/defined functions), I thought we either need to do a scan over all functions/instructions in AsmPrinter, or we could check all calls during isel in LowerCall() (Doing it in the FE seems worse as no optimizations at all have taken place yet). I tried now to do it in LowerCall(), but as I did not find any way to set a flag at Module scope each new function has still calls checked until the function attribute is added. Scanning everything in AsmPrinter would stop at the first occurrence on the other hand...

I hope all calls of interest have the CLI.CB pointer (Instruction*) set. The one case I have seen where this is null is for an intrinsic (patch-point)...

It seems that gcc emits attribute if va_start is used, even if va_list is not used. I have simplified this a bit to assume that va_start is always present in a var-arg function.

It seems that gcc is conservative about the va_list being passed on to other functions: As soon as a vector vararg argument is passed, it seems the vector ABI becomes visible, even if calling a static function, and I did the same.

isVectorTypeBased() now moved into SystemZ namespace so that it can be used also in LowerCall().

[uweigand]

I tried now to do it in LowerCall(), but as I did not find any way to set a flag at Module scope

You might want to look at the module flags metadata mechanism: https://llvm.org/docs/LangRef.html#module-flags-metadata
See the addModuleFlag / getModuleFlag routines.

[jonpa]

Updated per review to use a module flag instead of function attribute.

No other function changes, but some thoughts:

Not sure if SystemZMachineFunctionInfo.h is the right place for isVectorTypeBased()... It could simply be local to SystemZAsmPrinter.cpp if we scanned for all call instructions there instead of checking them in SystemZISelLowering::LowerCall(). I would feel just slightly more comfortable with that rather than relying on the presence of CLI.CB in detectVisibleVectorABI(), although I guess that should be ok... It might also be slightly more readable to not handle the call sites in a different place, but rather do all the back-end checking in AsmPrinter.

hasVisibleVectorABI() checks varargs also when calling a local function (see comment there). It would have been nice to instead identify va_list in isVectorTypeBased(), but I don't think that's possible. Another way might be to check in the front-end when a va_list is passed to a global function... Not sure how much it matters - I think GCC is being overconservative here.

[jonpa]

Patch rebased.

[jonpa]

Patch rebased with needed changes in order to build with passing tests:

• Use PTy->getPointerElementType() and not PTy->getElementType().

• Use CLI.CB->arg_size() instead of CLI.CB->getNumArgOperands().

• Check for existing module flag before adding "visible-vector-ABI" flag in SystemZABIInfo::EmitVAArg(). This seems to have been needed before as well, but the assertion for this did not trigger (now triggered by systemz-abi-vector.c, not changed by this patch).

[jonpa]

We also need to think about "opaque pointers" at some point...

[jonpa]

Tried to rebase today, but got some build error. I also think we probably should consider opaque pointers now as it has become the default, by going through the tests and also type handling of pointers in the patch, I would guess.

First question though is if this patch became so complex that it is more reasonable to abandon it, or is this still under review?

[jonpa]

Patch reworked to support opaque pointers which are now the default and soon mandatory. This was not just a syntactic change, but as far as I understand we have to respect that it is legal to use pointer in any arbitrary way. For example, this program now compiles without warning:

define void @fun(ptr %Src, ptr %Dst) {
  %L1 = load <4 x i32>, ptr %Src
  store <4 x i32> %L1, ptr %Dst
  %L2 = load i32, ptr %Src
  store i32 %L2, ptr %Dst

  call void %Src()

  ret void
}

I have updated the patch according to the conservative assumption that a pointer could always potentially expose the vector ABI. This seems necessary when handling this in the backend at least. This leads to a somewhat more simple handling, but I am not sure if this is acceptable. I guess the only alternative would be to handle everything in the front end instead, where hopefully these things are known. Then again, as I believe we have discussed before, that has the drawback of deciding on this before optimizations have run, which also could mean over-conservative results.

[jonpa]

Patch reworked to do the visible vector ABI detection in the Clang FE instead, which is necessary after the move to opaque pointers.

• The recursive search of the type is basically moved to the frontend. A set has to be used to avoid revisiting the same type again which is catastrophic in compile time.

• Tried to ignore unused pointers, but it seems that with C++ isUsed() always returns false, which means the patch is acting conservatively with pointers by assuming they always expose the ABI.

• Did not find a possible way to check vararg arguments when passed via funtion pointer: The check in CodeGenFunction::EmitCall() with checkFunctionCallABI() does not work for calls via function pointer. I think that either a similar method could be called here as well for the case where only a function prototype is available (and not a declaration), or it could be checked in CodeGenFunction::getVarArgType(). As long as it's only needed for varargs I guess its easier to do it in getVarArgType() (which the patch now does), but on the other hand it might make sense to have the check done in one place regardless of declaration/funptr.

• Would like to avoid looking up with getModuleFlag() extensively (only based on reasoning, not any compile time measurements), so added a mutable internal flag in SystemZTargetCodeGenInfo as setTargetAttributes() is const. Not sure if that makes a difference...

• Tried to get some suggestions on the name of the new module flag on Discourse a while ago, but without any replies.

• In the last minute I noticed that three regression tests are failing. They all fail at the same point when SystemZTargetCodeGenInfo::setTargetAttributes() calls GetGVALinkageForFunction() on a function with the 'inline' keyword without a function body, which triggers an assertion. Not sure if this is a bug in GetGVALinkageForFunction() or if this is a call that should never be made.

clang/test/CodeGen/PR2743-reference-missing-static.c
clang/test/CodeGen/unique-internal-linkage-names.c
clang/test/CoverageMapping/decl.c

[jonpa]

Avoid the problem with GetGVALinkageForFunction() by calling isExternallyVisible() instead, which seems to be the better alternative as well.

[jonpa]

I tried to move the handling of functions into computeInfo(), but found that it is not possible to know from a CGFunctionInfo if the function is externally visible or not. It does however seem like all vararg calls go through here (for normal functions only the declaration/definition is processed), which is fortunate as it happens that even a local vararg call should be regarded (the va_list could be passed to another global function potentially). I could therefore remove the new target hook I had added for the purpose of checking varargs.

[uweigand]

clang/lib/CodeGen/TargetInfo.cpp

7284

I'm wondering if it wouldn't be preferable to move those (and the functions operating on them) to SystemZTargetCodeGenInfo instead? There is precedent for mutable members in XCodeTargetCodeGenInfo.

7378

I think this will also have to look into C++ base classes, similar to what is done in isSingleElementStruct, doesn't it?

7490

Not sure about this. I think we need to do this unconditionally. Think e.g. of a global function that gets called with a va_list argument, passes that va_list to another *local* function, which invokes the va_arg on it. This would not be caught by your current code.

[jonpa]

Thanks for review - patch updated. Tests vec-abi-gnuattr-14.c and vec-abi-gnuattr-20.cpp strengthened accordingly.

[uweigand]

See one inline comment, otherwise LGTM now. Thanks!

clang/lib/CodeGen/TargetInfo.cpp

7490

Just one minor nit: why use CGF.CGM in some places and CGT.getCGM() in others? These should be the same, but it would be clearer to use the same expression everywhere.

clang/lib/CodeGen/TargetInfo.cpp

 namespace {
 
 class SystemZABIInfo : public SwiftABIInfo {
   bool HasVector;
   bool IsSoftFloatABI;
 
 public:
   SystemZABIInfo(CodeGenTypes &CGT, bool HV, bool SF)
     : SwiftABIInfo(CGT), HasVector(HV), IsSoftFloatABI(SF) {}
 
   bool isPromotableIntegerTypeForABI(QualType Ty) const;
   bool isCompoundType(QualType Ty) const;
   bool isVectorArgumentType(QualType Ty) const;
   bool isFPArgumentType(QualType Ty) const;
   QualType GetSingleElementType(QualType Ty) const;
 
   ABIArgInfo classifyReturnType(QualType RetTy) const;
bool SystemZABIInfo::isPromotableIntegerTypeForABI(QualType Ty) const {
   if (ABIInfo::isPromotableIntegerTypeForABI(Ty))
     return true;
 
   if (const auto *EIT = Ty->getAs<ExtIntType>())
     if (EIT->getNumBits() < 64)
       return true;
 
   // 32-bit values must also be promoted.
   if (const BuiltinType *BT = Ty->getAs<BuiltinType>())
     switch (BT->getKind()) {
     case BuiltinType::Int:
     case BuiltinType::UInt:
       return true;
     default:
       return false;
     }
   return false;
Address SystemZABIInfo::EmitVAArg(CodeGenFunction &CGF, Address VAListAddr,
   llvm::Type *ArgTy = CGF.ConvertTypeForMem(Ty);
   llvm::Type *DirectTy = ArgTy;
   ABIArgInfo AI = classifyArgumentType(Ty);
   bool IsIndirect = AI.isIndirect();
   bool InFPRs = false;
   bool IsVector = false;
   CharUnits UnpaddedSize;
   CharUnits DirectAlign;
+  if (!CGF.CurFn->hasLocalLinkage() && ArgTy->isVectorTy())
+    CGF.CGM.getModule().addModuleFlag(llvm::Module::Warning,
+                                      "visible-vector-ABI", 1);
   if (IsIndirect) {
     DirectTy = llvm::PointerType::getUnqual(DirectTy);
     UnpaddedSize = DirectAlign = CharUnits::fromQuantity(8);
   } else {
     if (AI.getCoerceToType())
       ArgTy = AI.getCoerceToType();
     InFPRs = (!IsSoftFloatABI && (ArgTy->isFloatTy() || ArgTy->isDoubleTy()));
     IsVector = ArgTy->isVectorTy();

clang/test/CodeGen/SystemZ/systemz-abi-vector2.c

+// RUN: %clang_cc1 -triple s390x-ibm-linux -target-cpu arch10 -emit-llvm \
+// RUN:   -fzvector -o - %s 2>&1 | FileCheck  %s --check-prefix=ARCH10-ATTR
+// RUN: %clang_cc1 -triple s390x-ibm-linux -target-cpu arch13 -emit-llvm \
+// RUN:   -fzvector -o - %s 2>&1 | FileCheck  %s --check-prefix=ARCH13-ATTR
+
+// RUN: %clang_cc1 -triple s390x-ibm-linux -target-cpu arch10 -S \
+// RUN:   -fzvector -o - %s 2>&1 | FileCheck  %s --check-prefix=ARCH10-ASM
+// RUN: %clang_cc1 -triple s390x-ibm-linux -target-cpu arch13 -S \
+// RUN:   -fzvector -o - %s 2>&1 | FileCheck  %s --check-prefix=ARCH13-ASM
+
+// Use of va_arg with a vector type exposes the vector ABI.
+
+#include <stdarg.h>
+
+int fun(va_list vl) {
+  return va_arg(vl, vector int)[0];
+}
+
+
+//ARCH10-ATTR: !llvm.module.flags = !{!0, !1}
+//ARCH10-ATTR: !0 = !{i32 2, !"visible-vector-ABI", i32 1}
+//ARCH13-ATTR: !llvm.module.flags = !{!0, !1}
+//ARCH13-ATTR: !0 = !{i32 2, !"visible-vector-ABI", i32 1}
+
+//ARCH10-ASM: .gnu_attribute 8, 1
+//ARCH13-ASM: .gnu_attribute 8, 2

llvm/lib/Target/SystemZ/AsmParser/SystemZAsmParser.cpp

private:
 
   bool parseAddress(bool &HaveReg1, Register &Reg1, bool &HaveReg2,
                     Register &Reg2, const MCExpr *&Disp, const MCExpr *&Length,
                     bool HasLength = false, bool HasVectorIndex = false);
   bool parseAddressRegister(Register &Reg);
 
   bool ParseDirectiveInsn(SMLoc L);
   bool ParseDirectiveMachine(SMLoc L);
+  bool ParseGNUAttribute(SMLoc L);
 
   OperandMatchResultTy parseAddress(OperandVector &Operands,
                                     MemoryKind MemKind,
                                     RegisterKind RegKind);
 
   OperandMatchResultTy parsePCRel(OperandVector &Operands, int64_t MinVal,
                                   int64_t MaxVal, bool AllowTLS);
 

 
 bool SystemZAsmParser::ParseDirective(AsmToken DirectiveID) {
   StringRef IDVal = DirectiveID.getIdentifier();
 
   if (IDVal == ".insn")
     return ParseDirectiveInsn(DirectiveID.getLoc());
   if (IDVal == ".machine")
     return ParseDirectiveMachine(DirectiveID.getLoc());
+  if (IDVal.startswith(".gnu_attribute"))
+    return ParseGNUAttribute(DirectiveID.getLoc());
 
   return true;
 }
 
 /// ParseDirectiveInsn
 /// ::= .insn [ format, encoding, (operands (, operands)*) ]
 bool SystemZAsmParser::ParseDirectiveInsn(SMLoc L) {
   MCAsmParser &Parser = getParser();
bool SystemZAsmParser::ParseDirectiveMachine(SMLoc L) {
   STI.setDefaultFeatures(CPU, /*TuneCPU*/ CPU, "");
   setAvailableFeatures(ComputeAvailableFeatures(STI.getFeatureBits()));
 
   getTargetStreamer().emitMachine(CPU);
 
   return false;
 }
 
+bool SystemZAsmParser::ParseGNUAttribute(SMLoc L) {
+  int64_t Tag;
+  int64_t IntegerValue;
+  if (!Parser.parseGNUAttribute(L, Tag, IntegerValue))
+    return false;
+
+  // Tag_GNU_S390_ABI_Vector tag is '8' and can be 0, 1, or 2.
+  if (Tag != 8 || (IntegerValue < 0 || IntegerValue > 2)) {
+    Error(Parser.getTok().getLoc(),
+          "Unrecognized .gnu_attribute tag/value pair.");
+    return false;
+  }
+
+  Parser.getStreamer().emitGNUAttribute(Tag, IntegerValue);
+
+  return true;
+}
+
 bool SystemZAsmParser::ParseRegister(unsigned &RegNo, SMLoc &StartLoc,
                                      SMLoc &EndLoc, bool RestoreOnFailure) {
   Register Reg;
   if (parseRegister(Reg, RestoreOnFailure))
     return true;
   if (Reg.Group == RegGR)
     RegNo = SystemZMC::GR64Regs[Reg.Num];
   else if (Reg.Group == RegFP)

llvm/lib/Target/SystemZ/SystemZAsmPrinter.h

bool doInitialization(Module &M) override {
     SM.reset();
     return AsmPrinter::doInitialization(M);
   }
 
 private:
   void LowerFENTRY_CALL(const MachineInstr &MI, SystemZMCInstLower &MCIL);
   void LowerSTACKMAP(const MachineInstr &MI);
   void LowerPATCHPOINT(const MachineInstr &MI, SystemZMCInstLower &Lower);
+  void emitAttributes(Module &M);
 };
 } // end namespace llvm
 
 #endif

llvm/lib/Target/SystemZ/SystemZAsmPrinter.cpp

 // MCInsts and MCExprs respectively.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SystemZAsmPrinter.h"
 #include "MCTargetDesc/SystemZInstPrinter.h"
 #include "SystemZConstantPoolValue.h"
 #include "SystemZMCInstLower.h"
+#include "SystemZMachineFunctionInfo.h"
 #include "TargetInfo/SystemZTargetInfo.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/CodeGen/MachineModuleInfoImpls.h"
 #include "llvm/CodeGen/TargetLoweringObjectFileImpl.h"
 #include "llvm/IR/Mangler.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInstBuilder.h"
 #include "llvm/MC/MCSectionELF.h"
assert(NumBytes >= EncodedBytes &&
          "Patchpoint can't request size less than the length of a call.");
   assert((NumBytes - EncodedBytes) % 2 == 0 &&
          "Invalid number of NOP bytes requested!");
   while (EncodedBytes < NumBytes)
     EncodedBytes += EmitNop(OutContext, *OutStreamer, NumBytes - EncodedBytes,
                             getSubtargetInfo());
 }
 
+// Returns true (the first time) when Ty is found to be or contain a vector
+// type.
+bool llvm::SystemZ::isVectorTypeBased(Type *Ty, std::set<Type *> &Visited) {
+  if (!Visited.insert(Ty).second) // Avoid infinite recursion via pointers.
+    return false;
+  if (Ty->isVectorTy())
+    return true;
+  if (auto *PTy = dyn_cast<PointerType>(Ty))
+    return isVectorTypeBased(PTy->getElementType(), Visited);
+  if (auto *ArrayTy = dyn_cast<ArrayType>(Ty))
+    return isVectorTypeBased(ArrayTy->getElementType(), Visited);
+  if (auto *STy = dyn_cast<StructType>(Ty)) {
+    for (StructType::element_iterator I = STy->element_begin(),
+                                      E = STy->element_end();
+         I != E; ++I)
+      if (isVectorTypeBased(*I, Visited))
+        return true;
+  }
+  if (auto *FTy = dyn_cast<FunctionType>(Ty)) {
+    if (isVectorTypeBased(FTy->getReturnType(), Visited))
+      return true;
+    for (auto I = FTy->param_begin(), E = FTy->param_end(); I != E; ++I)
+      if (isVectorTypeBased(*I, Visited))
+        return true;
+  }
+  return false;
+}
+
+// The *alignment* of 128-bit vector types is different between the software
+// and hardware vector ABIs. If the there is an externally visible use of a
+// vector type in the module it should be annotated with an attribute.
+void SystemZAsmPrinter::emitAttributes(Module &M) {
+  std::set<Type *> VisitedTypes;
+  bool HasVectorABI = M.getModuleFlag("visible-vector-ABI");
+
+  if (!HasVectorABI) {
+    for (const GlobalValue &GV : M.global_values()) {
+      // Externally visible function definitions.
+      if (const Function *F = dyn_cast<Function>(&GV))
+        if (!F->hasLocalLinkage() && !F->isDeclaration() &&
+            SystemZ::isVectorTypeBased(F->getFunctionType(), VisitedTypes)) {
+          HasVectorABI = true;
+          break;
+        }
+      // Externally visible variables.
+      if (isa<GlobalVariable>(GV) && !GV.hasLocalLinkage() &&
+          (!GV.isDeclaration() || !GV.use_empty()) &&
+          SystemZ::isVectorTypeBased(GV.getValueType(), VisitedTypes)) {
+        HasVectorABI = true;
+        break;
+      }
+    }
+  }
+
+  if (HasVectorABI) {
+    bool HasVectorFeature =
+      TM.getMCSubtargetInfo()->getFeatureBits()[SystemZ::FeatureVector];
+    OutStreamer->emitGNUAttribute(8, HasVectorFeature ? 2 : 1);
+  }
+}
+
 // Convert a SystemZ-specific constant pool modifier into the associated
 // MCSymbolRefExpr variant kind.
 static MCSymbolRefExpr::VariantKind
 getModifierVariantKind(SystemZCP::SystemZCPModifier Modifier) {
   switch (Modifier) {
   case SystemZCP::TLSGD: return MCSymbolRefExpr::VK_TLSGD;
   case SystemZCP::TLSLDM: return MCSymbolRefExpr::VK_TLSLDM;
   case SystemZCP::DTPOFF: return MCSymbolRefExpr::VK_DTPOFF;
bool SystemZAsmPrinter::PrintAsmMemoryOperand(const MachineInstr *MI,
                                               raw_ostream &OS) {
   SystemZInstPrinter::printAddress(MAI, MI->getOperand(OpNo).getReg(),
                                    MI->getOperand(OpNo + 1).getImm(),
                                    MI->getOperand(OpNo + 2).getReg(), OS);
   return false;
 }
 
 void SystemZAsmPrinter::emitEndOfAsmFile(Module &M) {
+  emitAttributes(M);
   emitStackMaps(SM);
 }
 
 // Force static initialization.
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeSystemZAsmPrinter() {
   RegisterAsmPrinter<SystemZAsmPrinter> X(getTheSystemZTarget());
 }

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

for (unsigned I = 0, E = ArgLocs.size(); I != E; ++I) {
     if (Reg == SystemZ::R6H || Reg == SystemZ::R6L || Reg == SystemZ::R6D)
       return false;
     if (Outs[I].Flags.isSwiftSelf() || Outs[I].Flags.isSwiftError())
       return false;
   }
   return true;
 }
 
+static bool hasVisibleVectorABI(SystemZTargetLowering::CallLoweringInfo &CLI) {
+  Value *Callee = CLI.CB->getCalledOperand();
+  bool LocalCallee = false;
+  if (const Function *F = dyn_cast<Function>(Callee))
+    LocalCallee = F->hasLocalLinkage();
+
+  std::set<Type *> VisitedTypes;
+  if (!LocalCallee &&
+      SystemZ::isVectorTypeBased(Callee->getType(), VisitedTypes))
+    // External or indirect function call.
+    return true;
+  else if (CLI.IsVarArg) {
+    // Check the passed varargs. Do this as well in case of a local callee as
+    // va_list may be passed on by it but cannot be identified as a parameter
+    // by isVectorTypeBased().
+    for (unsigned Idx = CLI.CB->getFunctionType()->getNumParams();
+         Idx < CLI.CB->getNumArgOperands(); Idx++) {
+      Type *ArgTy = CLI.CB->getArgOperand(Idx)->getType();
+      if (SystemZ::isVectorTypeBased(ArgTy, VisitedTypes))
+        return true;
+    }
+  }
+
+  return false;
+}
+
 SDValue
 SystemZTargetLowering::LowerCall(CallLoweringInfo &CLI,
                                  SmallVectorImpl<SDValue> &InVals) const {
   SelectionDAG &DAG = CLI.DAG;
   SDLoc &DL = CLI.DL;
   SmallVectorImpl<ISD::OutputArg> &Outs = CLI.Outs;
   SmallVectorImpl<SDValue> &OutVals = CLI.OutVals;
   SmallVectorImpl<ISD::InputArg> &Ins = CLI.Ins;
   SDValue Chain = CLI.Chain;
   SDValue Callee = CLI.Callee;
   bool &IsTailCall = CLI.IsTailCall;
   CallingConv::ID CallConv = CLI.CallConv;
   bool IsVarArg = CLI.IsVarArg;
   MachineFunction &MF = DAG.getMachineFunction();
+  Module &M = *MF.getFunction().getParent();
   EVT PtrVT = getPointerTy(MF.getDataLayout());
   LLVMContext &Ctx = *DAG.getContext();
 
   // Detect unsupported vector argument and return types.
   if (Subtarget.hasVector()) {
     VerifyVectorTypes(Outs);
     VerifyVectorTypes(Ins);
   }
 
+  if (!M.getModuleFlag("visible-vector-ABI") && CLI.CB &&
+      hasVisibleVectorABI(CLI))
+    M.addModuleFlag(llvm::Module::Warning, "visible-vector-ABI", 1);
+
   // Analyze the operands of the call, assigning locations to each operand.
   SmallVector<CCValAssign, 16> ArgLocs;
   SystemZCCState ArgCCInfo(CallConv, IsVarArg, MF, ArgLocs, Ctx);
   ArgCCInfo.AnalyzeCallOperands(Outs, CC_SystemZ);
 
   // We don't support GuaranteedTailCallOpt, only automatically-detected
   // sibling calls.
   if (IsTailCall && !canUseSiblingCall(ArgCCInfo, ArgLocs, Outs))

llvm/lib/Target/SystemZ/SystemZMachineFunctionInfo.h

 // A struct to hold the low and high GPR registers to be saved/restored as
 // well as the offset into the register save area of the low register.
 struct GPRRegs {
   unsigned LowGPR;
   unsigned HighGPR;
   unsigned GPROffset;
   GPRRegs() : LowGPR(0), HighGPR(0), GPROffset(0) {}
   };
+
+  bool isVectorTypeBased(Type *Ty, std::set<Type *> &Visited);
 }
 
 class SystemZMachineFunctionInfo : public MachineFunctionInfo {
   virtual void anchor();
 
   SystemZ::GPRRegs SpillGPRRegs;
   SystemZ::GPRRegs RestoreGPRRegs;
   Register VarArgsFirstGPR;

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-00.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to external function with vector argument.
+define internal void @fun() {
+entry:
+  tail call void @foo(<4 x i32> zeroinitializer)
+  ret void
+}
+
+declare void @foo(<4 x i32>)
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-01.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to external function with vector return value.
+define internal void @fun() {
+entry:
+  %indirect-arg-temp = alloca <4 x i32>, align 16
+  %C = call <4 x i32> @foo()
+  store <4 x i32> %C, <4 x i32>* %indirect-arg-temp
+  ret void
+}
+
+declare <4 x i32> @foo()
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-02.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to external function *without* vector arguments.
+define internal void @fun() {
+entry:
+  tail call void @foo(i64 0)
+  ret void
+}
+
+declare void @foo(i64)
+
+; CHECK-NOT: .gnu_attribute 8

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-03.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Calling *local* function with vector argument.
+define void @fun() {
+entry:
+  %indirect-arg-temp = alloca <4 x i32>, align 16
+  store <4 x i32> zeroinitializer, <4 x i32>* %indirect-arg-temp
+  call void @foo(<4 x i32>* nonnull %indirect-arg-temp)
+  ret void
+}
+
+define internal void @foo(<4 x i32>*) { ret void }
+
+; CHECK-LABEL: fun:
+; CHECK-NOT: .gnu_attribute 8

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-04.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call via global function pointer in internal function.
+
+@foo = global void (<4 x i32>*)* null, align 8
+
+define internal void @fun() {
+entry:
+  %V = alloca <4 x i32>, align 16
+  store <4 x i32> zeroinitializer, <4 x i32>* %V, align 16
+  %FuncPtr = load void (<4 x i32>*)*, void (<4 x i32>*)** @foo, align 8
+  call void %FuncPtr(<4 x i32>* nonnull %V)
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 1

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-05.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Passing address of local function with vector arg to global function.
+
+define void @fun() {
+entry:
+  tail call void @GlobFun(void (<4 x i32>*)* nonnull @InternalFun)
+  ret void
+}
+
+declare void @GlobFun(void (<4 x i32>*)*)
+
+define internal void @InternalFun(<4 x i32>* nocapture %V) {
+entry:
+  store <4 x i32> zeroinitializer, <4 x i32>* %V, align 8
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-06.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Passing address of an array of vectors.
+define internal void @fun() {
+entry:
+  %V = alloca [2 x <4 x i32>], align 16
+  call void ([2 x <4 x i32>]*) @foo([2 x <4 x i32>]* nonnull %V)
+  ret void
+}
+
+declare void @foo([2 x <4 x i32>]* %V)
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-07.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Passing a struct containing a vector element.
+%struct.S = type { <4 x i32>, i32, [12 x i8] }
+define internal void @fun() {
+entry:
+  %V = alloca %struct.S, align 16
+  call void (%struct.S*) @foo(%struct.S* nonnull %V)
+  ret void
+}
+
+declare void @foo(%struct.S*)
+
+; CHECK: .gnu_attribute 8, 1

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-08.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to vararg function with a vector argument.
+define internal void @fun() {
+entry:
+  call void (...) @foo(<2 x i64> zeroinitializer)
+  ret void
+}
+
+declare void @foo(...)
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-09.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to vararg function *without* any vector argument.
+define internal void @fun() {
+entry:
+  call void (...) @foo(i32 0)
+  ret void
+}
+
+declare void @foo(...)
+
+; CHECK-NOT: .gnu_attribute 8

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-10.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Call to varargs function via global pointer to pointer to function.
+
+@fptrp = common global void (i32, ...)** null, align 8
+
+define void @fun() {
+  %1 = load void (i32, ...)**, void (i32, ...)*** @fptrp, align 8
+  %2 = load void (i32, ...)*, void (i32, ...)** %1, align 8
+  tail call void (i32, ...) %2(i32 signext 0, <4 x i32> zeroinitializer)
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 1

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-11.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Passing vector argument to varargs function via local function pointer.
+
+define void @fun(void (i32, ...)* nocapture %fptr) {
+entry:
+  tail call void (i32, ...) %fptr(i32 signext 0, <4 x i32> zeroinitializer)
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-12.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Passing a vararg vector argument should always emit the gnu attribute as
+; the va_list may potentially be passed on to another (global) function.
+
+define internal void @foo(i32 %a, ...) {
+entry:
+  ret void
+}
+
+define internal void @fun() {
+  call void (i32, ...) @foo(i32 1, <4 x i32> zeroinitializer);
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-13.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Defining globally visible function with vector arguments.
+define <4 x i32> @fun(<4 x i32> %Arg) {
+entry:
+  ret <4 x i32> %Arg
+}
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-14.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch10 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Globally visible vector variable.
+@VisibleVector = global <4 x i32> zeroinitializer
+
+; CHECK: .gnu_attribute 8, 1

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-15.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Using external vector variable.
+@ExtVector = external global <4 x i32>
+
+define void @fun() {
+entry:
+  store <4 x i32> zeroinitializer, <4 x i32>* @ExtVector
+  ret void
+}
+
+; CHECK: .gnu_attribute 8, 2

llvm/test/CodeGen/SystemZ/vec-abi-gnuattr-16.ll

+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=arch13 < %s | FileCheck %s
+;
+; Test the emission of a .gnu_attribute describing the vector abi.
+
+; Using *local* vector variable.
+@LocVector = internal global <4 x i32> zeroinitializer
+
+define void @fun() {
+entry:
+  store volatile <4 x i32> zeroinitializer, <4 x i32>* @LocVector
+  ret void
+}
+
+; CHECK-NOT: .gnu_attribute 8

llvm/test/MC/SystemZ/gnu-attributes.s

+# RUN: llvm-mc -triple s390x-linux-gnu -filetype=asm %s | \
+# RUN: FileCheck %s --check-prefix=ASM
+# RUN: llvm-mc -triple s390x-linux-gnu -filetype=obj %s | \
+# RUN: llvm-objdump --mcpu=z14 -D - | FileCheck %s --check-prefix=OBJ
+
+#ASM:      .text
+#ASM:      .gnu_attribute 8, 2
+  
+#OBJ:  0000000000000000 <.gnu.attributes>:
+#OBJ:       0: 41 00 00 00
+#OBJ:       4: 0f 67
+#OBJ:       6: 6e 75 00 01
+#OBJ:       a: 00 00
+#OBJ:       c: 00 07
+#OBJ:       e: 08 02
+  
+  .gnu_attribute 8, 2