[SystemZ] Support -msoft-float

[jonpa]

Summary

First attempt of reusing code from other targets to recognize -msoft-float on SystemZ.

Seems to work, and tests are passing.

[jonpa]

Don't save fp-regs with varargs + soft-float

Not sure how to use -msoft-float (tried linking various libraries but did not work...)

Is this a priority for the next LLVM release?

[uweigand]

Just a couple of general comments for now, not yet going into implementation details.

The point of -msoft-float is to ensure that the compiler generates code that never touches any floating-point (or vector) register. To ensure this mainly requires:

• A change of the ABI to not use floating-point or vector registers to pass anything. In particular, this means -msoft-float implies -mno-vx so that nothing using vector registers; in addition, the ABI is changed by -msoft-float so that floating-point values are passed like integer values of the same size (in GPRs and/or on the stack).

• Changing the compiler to ensure nothing creates any implicit use of floating-point (or vector) registers, e.g. to temporarily hold values, or to implement auto-vectorization.

(Some targets, in particular ARM, allow a third mode that uses the soft-float ABI at call boundaries, but is still allowed to make internal use of floating-point registers otherwise. We've never supported this on SystemZ.)

To implement the second part in clang/LLVM, I believe we need to

• Have -msoft-float imply -mno-vx (either on the clang side, or the LLVM side, or both). This doesn't seem to be implemented in the patch yet.

• If -msoft-float is active, have all floating-point (and vector) types be marked as not legal in LLVM. This is done by the patch (at least for floating-point types -- vector types should be covered as soon as -msoft-float implies -mno-vx).

• Remove any other uses of the floating-point (or vector) register classes in SystemZ target code. In particular, I think we need to stop using FP classes to implement the 'f' constraint in SystemZTargetLowering::getRegForInlineAsmConstraint. Not sure if I missed anything else.

Now, to implement the first part (ABI), I think we need additional changes both on the clang and LLVM side:

• For arguments of type float or double, I think this should be handled on the LLVM side. In fact, this may be done automatically if the types are marked as not legal, since then they should be converted to i32 or i64 by the type legalizer (TypeSoftenFloat action). This needs to be verified (and test cases!) though.

• The special case handling of aggregates with a single float/double member needs to be deactivated. I think this should be done in clang, probably best by having SystemZABIInfo::isFPArgumentType return false if msoft-float is active.

• Variable argument handling needs to be updated to match. One part (on the LLVM side) is already in this patch. But there needs to be a corresponding change in clang, in SystemZABIInfo::EmitVAArg -- the InFPRs flag always must be false with -msoft-float. (Why isn't this using SystemZABIInfo::isFPArgumentType in the first place?)

Now as to the various interfaces:

• In LLVM, you now have both a feature (+soft-float, -soft-float) and a function attribute ("use-soft-float"). I'm not really sure we need both ... and in fact, X86 doesn't do it that way either. I think we don't need the explicit feature here.

• In clang-cc1, common code supports both the -mfloat-abi={soft,hard} flag (to define the FP ABI only) and the -msoft-float flag (to probihit internal use of FP registers). I guess it makes sense to just re-use those same flags on SystemZ (as your patch does), even if we don't actually support the "Soft-FP ABI but allow internal FP use" mode.

• However in the clang driver, your patch now also supports both the -msoft-float/-mhard-float and -mfloat-abi=... flags. Since GCC does not support -mfloat-abi=... on SystemZ, I believe we should not support it either in clang, for consistency.

As a general note on testing, since -msoft-float defines a new ABI, you'd need a set of system libraries build with -msoft-float in order to run any real tests. Given that we've never really supported -msoft-float for user-space, no current SystemZ distro actually provides those libraries. So there probably cannot be any better tests than just normal .ll unit tests.

The only user of -msoft-float is the Linux kernel -- which doesn't use libraries anyway. However, the kernel does rely on -msoft-float because kernel code must not ever access floating-point registers since those are not saved/restored when switching between user and kernel code. So if we want to enable building the kernel with clang, it is mandatory to correctly support -msoft-float.

[jonpa]

To implement the second part in clang/LLVM, I believe we need to

• Have -msoft-float imply -mno-vx (either on the clang side, or the LLVM side, or both). This doesn't seem to be implemented in the patch yet.

-msoft-float now gives "-vector" in addition to "use-soft-float"="true"
In the subtarget, HasSoftFloat clears HasVector, so that "soft-float"="true" attribute implies "-vector" on the LLVM function.

• If -msoft-float is active, have all floating-point (and vector) types be marked as not legal in LLVM. This is done by the patch (at least for floating-point types -- vector types should be covered as soon as -msoft-float implies -mno-vx).

Yes, covered implicitly by -mno-vx, but thought it was more readable to have it in the same if clause.

• Remove any other uses of the floating-point (or vector) register classes in SystemZ target code. In particular, I think we need to stop using FP classes to implement the 'f' constraint in SystemZTargetLowering::getRegForInlineAsmConstraint. Not sure if I missed anything else.

Done. I did not find anything else either. I tried making a test case to check if the machine verifier would help us to detect any use of fp/vector regs with -soft-float. It seems that it currently does not do this. I edited a .mir test case by inserting a vector instruction using vector regs while the "+soft-float" attribute was present. The verifier did not catch this as an error, but I think we could patch the verifier once we have the soft-float support in the backend (this patch).

• For arguments of type float or double, I think this should be handled on the LLVM side. In fact, this may be done automatically if the types are marked as not legal, since then they should be converted to i32 or i64 by the type legalizer (TypeSoftenFloat action). This needs to be verified (and test cases!) though.

Hope to have covered most of the calling convention for ingoing and outgoing arguments as well as return values in soft-float-args.ll. I reused the existing test cases for the multiple return values and inserted the generated code there, for which I can at least say is not using any fp/vector registers. I started by collecting simple new tests soft-float*.ll, but then realized it might be even better to rerun existing tests with SOFT-FLOAT checks instead, as in eg vec-args-06.ll. Not sure if that's needed (to redo the tests)?

• The special case handling of aggregates with a single float/double member needs to be deactivated. I think this should be done in clang, probably best by having SystemZABIInfo::isFPArgumentType return false if msoft-float is active.

Done, with added test matching output. I am a little curious as to why this is needed...

• Variable argument handling needs to be updated to match. One part (on the LLVM side) is already in this patch. But there needs to be a corresponding change in clang, in SystemZABIInfo::EmitVAArg -- the InFPRs flag always must be false with -msoft-float. (Why isn't this using SystemZABIInfo::isFPArgumentType in the first place?)

I implemented that and updated systemz-abi.c accordingly to pass (again, not sure about the details here). (Seems that isFPArgumentType() takes a QualType argument, and not Type...)

• In LLVM, you now have both a feature (+soft-float, -soft-float) and a function attribute ("use-soft-float"). I'm not really sure we need both ... and in fact, X86 doesn't do it that way either. > I think we don't need the explicit feature here.

Removed the adding of the feature. Also aemoved the explicit generation of "-mfloat-abi" "hard", in AddSystemZTargetArgs (like X86).

• In clang-cc1, common code supports both the -mfloat-abi={soft,hard} flag (to define the FP ABI only) and the -msoft-float flag (to probihit internal use of FP registers). I guess it makes sense to just re-use those same flags on SystemZ (as your patch does), even if we don't actually support the "Soft-FP ABI but allow internal FP use" mode.

• However in the clang driver, your patch now also supports both the -msoft-float/-mhard-float and -mfloat-abi=... flags. Since GCC does not support -mfloat-abi=... on SystemZ, I believe we should not support it either in clang, for consistency.

Added a check against using -mfloat-abi with the clang driver.

When building SPEC, I got one failure due to the generation of a s390.tdc intrinsic, which resulted in

SoftenFloatOperand Op #1: t6: i32 = llvm.s390.tdc TargetConstant:i64<6064>, t4, Constant:i64<2730>

Do not know how to soften this operator's operand!
UNREACHABLE executed at /home/ijonpan/llvm-project/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp:765!

It seems to me best to not do this optimization, so I disabled that in SystemZTDC.cpp with a check for the soft-float function attribute. I am however not quite happy with this, because if the function attribute in the test case is removed, and then -mattr=soft-float is passed to llc, this still fails. I wanted to get the subtarget and do the query to (SystemZ) Subtarget.hasSoftFloat(), but I haven't yet been able to figure out how to do that in SystemZTDC.cpp. SystemZTargetTransformInfo has the SystemZSubtarget subtarget pointer, but I don't know how to access it that way either... :-/

BTW, I see that hasVectorEnhancment1/2 returns true even with when hasVector() returns false. This is probably a separate topic, but to me it seems preferred to return the expected values. The subtarget features are built up nicely in an incremental way so one might be mislead to believe that this is always reflected when checking for subtarget features. Or maybe it's enough to just assume that since the types are not legal, this will never matter..?

[uweigand]

clang/lib/Basic/Targets/SystemZ.h

118

Given that you no longer provide the +soft-float target feature, this check is never true, right?

On the other hand, this is really now only used to define the SOFT_FLOAT predefined macro, which we probably shouldn't have anyway, given that GCC does not provide that predefine either.

So I think all this can just go away.

clang/lib/Driver/ToolChains/Arch/SystemZ.cpp

37

What's the point of the Invalid setting now? Just set the variable to Hard initially, and if there's no option, that's where it will stay ...

73

Having the override in LLVM itself should be OK, I don't think we need this here.

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1115

I don't think a fatal error is the correct action here. We should simply not return an unsupported regclass. Just like we already don't return a regclass for "v" unless Subtarget.hasVector, we should similarly add a useSoftFloat guard for the "f" constraint.

llvm/lib/Target/SystemZ/SystemZTDC.cpp

313 ↗

(On Diff #238345)

It seems it would be preferable to avoid the duplicate check by using the Subtarget,
e.g. something like F.getSubtarget().hasSoftFloat().

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

179

So this change actually enables another feature: per-function CPU / feature selection.

Now that is of course an interesting feature that we actually want to have as well, but it seems like this really ought to be a separate patch, and in any case there should be test cases to verify that feature.

[jonpa]

Updated per review - see inline comments.

[jonpa]

clang/lib/Driver/ToolChains/Arch/SystemZ.cpp

73

OK, removed, but now the driver produces (clang -msoft-float -mvx):

clang -cc1 ... "-target-feature" "+vector" "-msoft-float" "-mfloat-abi" "soft"

, and the generated function gets these attributes:

attributes #0 = { ... "target-features"="+transactional-execution,+vector,+vector-enhancements-1" ... "use-soft-float"="true" }

You are right however, this doesn't matter - the output is soft float.

Updated soft-float-03.ll test to check that these function attributes works as expected in the backend.

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1115

Ah, yes, that's better.

I found out that the check for '{v}' is not wrapped by a guard against hasVector() as 'v' has, but it fails during SelectionDAGBuilder. A separate patch might be to give a better message here to the user in case of compiling without vector support / soft-float.

Updated tests and removed the test for {v}, since it fails with an assert.

llvm/lib/Target/SystemZ/SystemZTDC.cpp

313 ↗

(On Diff #238345)

Yeah, right - as I wrote before I didn't find a way to get the Subtarget which was disapointing but still true... This is an IR pass, and I didn't see it being done anywhere.

This would really be preffered, since -mattr=soft-float passed to llc would not be handled as it is.

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

179

ok, removed it from this patch.

I had to change soft-float-02.ll to use -mattr=soft-float instead of a function attribute after removing this.

[uweigand]

174 ↗
(On Diff #238345)

ok, removed it from this patch.

I had to change soft-float-02.ll to use -mattr=soft-float instead of a function attribute after removing this.

Ah, I see. But note that you're now not supporting "use-soft-float" at all (which I think is fine at this step!), so you should update all tests to no longer use "use-soft-float".

clang/lib/Driver/ToolChains/Arch/SystemZ.cpp

12

Still needed?

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1109

Unneeded whitespace change?

1115

Please keep the test and just add the hasVector guard for the {v} constraints.

llvm/lib/Target/SystemZ/SystemZSubtarget.cpp

39

I just realized there is another place where this check is duplicated: UsesVectorABI in SystemZTargetMachine.cpp. This now also needs to check for the soft-float feature: with -msoft-float, GCC also falls back to the 16-byte vector alignment, so we must match that for ABI compatibility. Note that at that point, we only check the global features, not per-function features.

llvm/lib/Target/SystemZ/SystemZTDC.cpp

313 ↗

(On Diff #238345)

This is particularly annoying given that this patch now no longer supports "use-soft-float" anywhere else.

I believe you can get at the Subtarget via the TargetTransformInfo, which should be available to IR passes. Look at how other passes do it, something like:

const TargetTransformInfo &TTI =
    getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);

Then you'll have to register that you'll be using the TargetTransformInfoWrapperPass by providing a getAnalysisUsage override, something like:

void getAnalysisUsage(AnalysisUsage &AU) const override {
  AU.addRequired<TargetTransformInfoWrapperPass>();
}

And you may also have to add:

INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)

[jonpa]

Ah, I see. But note that you're now not supporting "use-soft-float" at all (which I think is fine at this step!), so you should update all tests to no longer use "use-soft-float".

Done.
All llc invocations use -mattr=soft-float instead of relying on the function attributes, as must be done.

Addded a new test systemz-float-02.c to check that -msoft-float actually works all the way to assembly output. The type v4si using vector_size() gives a different output compared to gcc, so I guess this is implementation defined?

See inline comments.

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1115

Sorry for being unclear: even with the hasVector guard for {v}, it fails with an assert, unlike the case with {f}.

Since this is currently not handled with -vector, I thought this should be a separate patch...?

llvm/lib/Target/SystemZ/SystemZSubtarget.cpp

39

In order to do this, I added back the SoftFloat flag in clang/lib/Basic/Targets/SystemZ.h in order to call resetDataLayout() when it is set. (There is a check in clang/lib/CodeGen/BackendUtil.cpp that indicates that it is necessary to set the v128 alignment also here).

"+soft-float" is pushed to Features in systemz::getSystemZTargetFeatures() so that SoftFloat will be set per above, and so that UsesVectorABI() can check for it and return true if found. Furthermore, this is actually needed I think, since we are not looking at the function attributes anymore.

Added a test for this in clang/test/CodeGen/target-data.c.

llvm/lib/Target/SystemZ/SystemZTDC.cpp

313 ↗

(On Diff #238345)

Thanks for helping me out :-)

It seemed to work so far as to get the TTI, but unfortunately it seems I would somehow need the TTIImpl pointer to get to the SystemZ implementation, but it is private and inaccessible. The TTI class hierarchy is very complex, so I chose to instead simply not add the SystemZTDCPass at all in SystemZPassConfig::addIRPasses(), since the TM is there available.

This should work fine as long as the user is passing an option to use soft-float. In other words, this will not work with function attributes, I think.

[uweigand]

In D72189#1850468, @jonpa wrote:

Ah, I see. But note that you're now not supporting "use-soft-float" at all (which I think is fine at this step!), so you should update all tests to no longer use "use-soft-float".

Done.
All llc invocations use -mattr=soft-float instead of relying on the function attributes, as must be done.

Ok, good.

Addded a new test systemz-float-02.c to check that -msoft-float actually works all the way to assembly output. The type v4si using vector_size() gives a different output compared to gcc, so I guess this is implementation defined?

What's the difference in assembler? Could this be because you've implemented the VectorABI check the wrong way around? (If soft-float, then we have *no* VectorABI!)

clang/lib/Basic/Targets/SystemZ.h

122

Eh, I guess that should be "HasVector && !SoftFloat".

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1176

This should have "&& Subtarget.hasVector()" here. Shouldn't that fix the problem with {v}?

If it does not, I agree this can wait for another patch.

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

62

No, VectorABI = false then.

Also, should you look for "soft-float" and "-soft-float", just as is done for "vector" above?

201–202

This can be just getTM<SystemZTargetMachine>() instead of the manual cast.

[jonpa]

If soft-float, then we have *no* VectorABI!

Oh! I misunderstood your previous comment "with -msoft-float, GCC also falls back to the 16-byte vector alignment, so we must match that for ABI compatibility" to mean that (source code) vectors should be aligned to 16 bytes in memory. I also think I saw GCC outputing an NLL to align a store in such a case, but I can't reproduce so I guess I was confused.

What's the difference in assembler? Could this be because you've implemented the VectorABI check the wrong way around? (If soft-float, then we have *no* VectorABI!)

Also with the latest updates of the patch, I see for this function:

typedef int v4si __attribute__ ((vector_size (16)));
v4si foo(v4si *Dst) {
  return *Dst;
}

gcc -S -O3 -march=z14 -msoft-float

foo:
  lmg     %r4,%r5,0(%r3)
  stmg    %r4,%r5,0(%r2)
  br      %r14

clang -S -O3 -march=z14 -msoft-float

foo:                                    # @foo
l       %r0, 0(%r2)
l       %r3, 4(%r2)
l       %r4, 8(%r2)
l       %r5, 12(%r2)
lr      %r2, %r0
br      %r14

It seems that gcc is loading from %r3, while clang is loading from %r2, and gcc is returning on the stack, while clang seems to be returning in registers.

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

1176

For some reason, the compilation then aborts with an assert if that constraint has been specified with either -vector or soft-float, so I think it should wait for a separate patch.

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

62

With the clang driver, it seems that
-mhard-float after -msoft-float works to cancel the first argument.
-mno-soft-float is not recognized here / handled (by any target) (warning: unused)
-mno-hard-float is not even allowed (error)

With the features arguments (-target-feature / -mattr), it however works the same as with the 'vector' feature, so it makes sense to handle it the same way.

[uweigand]

In D72189#1852408, @jonpa wrote:

If soft-float, then we have *no* VectorABI!

Oh! I misunderstood your previous comment "with -msoft-float, GCC also falls back to the 16-byte vector alignment, so we must match that for ABI compatibility" to mean that (source code) vectors should be aligned to 16 bytes in memory.

Yes, that's what I meant. Maybe what's confusing is that on our platform, the "vector ABI" only has an 8-byte alignment for vector types (which is more efficient), while the "old" (non-vector) ABI has a 16-byte alignment (for historical reasons).

What's the difference in assembler? Could this be because you've implemented the VectorABI check the wrong way around? (If soft-float, then we have *no* VectorABI!)

Also with the latest updates of the patch, I see for this function:

typedef int v4si __attribute__ ((vector_size (16)));
v4si foo(v4si *Dst) {
  return *Dst;
}

gcc -S -O3 -march=z14 -msoft-float

foo:
  lmg     %r4,%r5,0(%r3)
  stmg    %r4,%r5,0(%r2)
  br      %r14

clang -S -O3 -march=z14 -msoft-float

foo:                                    # @foo
l       %r0, 0(%r2)
l       %r3, 4(%r2)
l       %r4, 8(%r2)
l       %r5, 12(%r2)
lr      %r2, %r0
br      %r14

It seems that gcc is loading from %r3, while clang is loading from %r2, and gcc is returning on the stack, while clang seems to be returning in registers.

That looks completely broken. Note that current clang, when using the non-vector ABI, does generate code that is compatible to what GCC generates, so this must have been broken by this patch somehow. (Possibly the problem is when checking for the vector ABI in TargetInfo.cpp, see inline comment.

clang/lib/CodeGen/TargetInfo.cpp

9935

Hmm. In the SoftFloat case, we also should be setting HasVector to false here. Maybe that even explains the incorrect assembler code you're seeing.

(Maybe it would be best after all to reset the HasVector flag globally, then we wouldn't have the problem of forgetting all these cases ...)

[jonpa]

Patch updated per review to reset HasVector globally both in clang/lib/CodeGen/TargetInfo.cpp (constructor argument) and clang/lib/Basic/Targets/SystemZ.h (handleTargetFeatures()).

This seems to fix the problem with the test case - now the clang output seems to match gcc:

foo:                                    # @foo
# %bb.0:                                # %entry
mvc     8(8,%r2), 8(%r3)
mvc     0(8,%r2), 0(%r3)
br      %r14

[uweigand]

LGTM, thanks!

[thakis]

Hello, this breaks tests on all platforms (e.g.: http://45.33.8.238/mac/7170/step_11.txt) Please revert immediately and please run tests locally before committing next time.

[jonpa]

In D72189#1857098, @thakis wrote:

Hello, this breaks tests on all platforms (e.g.: http://45.33.8.238/mac/7170/step_11.txt) Please revert immediately and please run tests locally before committing next time.

I have added the missing 'REQUIRES', and -mtriple:s to the tests which was not needed when I tested locally. Sorry for the noise.

clang/lib/Basic/Targets/SystemZ.h

 class LLVM_LIBRARY_VISIBILITY SystemZTargetInfo : public TargetInfo {
 
   static const Builtin::Info BuiltinInfo[];
   static const char *const GCCRegNames[];
   std::string CPU;
   int ISARevision;
   bool HasTransactionalExecution;
   bool HasVector;
+  bool SoftFloat;
 
 public:
   SystemZTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple), CPU("z10"), ISARevision(8),
-        HasTransactionalExecution(false), HasVector(false) {
+        HasTransactionalExecution(false), HasVector(false), SoftFloat(false) {
     IntMaxType = SignedLong;
     Int64Type = SignedLong;
     TLSSupported = true;
     IntWidth = IntAlign = 32;
     LongWidth = LongLongWidth = LongAlign = LongLongAlign = 64;
     PointerWidth = PointerAlign = 64;
     LongDoubleWidth = 128;
     LongDoubleAlign = 64;
bool handleTargetFeatures(std::vector<std::string> &Features,
                             DiagnosticsEngine &Diags) override {
     HasTransactionalExecution = false;
     HasVector = false;
     for (const auto &Feature : Features) {
       if (Feature == "+transactional-execution")
         HasTransactionalExecution = true;
       else if (Feature == "+vector")
         HasVector = true;
+      else if (Feature == "+soft-float")
+        SoftFloat = true;
     }
     // If we use the vector ABI, vector types are 64-bit aligned.
-    if (HasVector) {
+    if (HasVector && !SoftFloat) {
       MaxVectorAlign = 64;
       resetDataLayout("E-m:e-i1:8:16-i8:8:16-i64:64-f128:64"
                       "-v128:64-a:8:16-n32:64");
     }
     return true;
   }
 
   bool hasFeature(StringRef Feature) const override;

clang/lib/CodeGen/TargetInfo.cpp

 //===----------------------------------------------------------------------===//
 // SystemZ ABI Implementation
 //===----------------------------------------------------------------------===//
 
 namespace {
 
 class SystemZABIInfo : public SwiftABIInfo {
   bool HasVector;
+  bool IsSoftFloatABI;
 
 public:
-  SystemZABIInfo(CodeGenTypes &CGT, bool HV)
-    : SwiftABIInfo(CGT), HasVector(HV) {}
+  SystemZABIInfo(CodeGenTypes &CGT, bool HV, bool SF)
+    : SwiftABIInfo(CGT), HasVector(HV), IsSoftFloatABI(SF) {}
 
   bool isPromotableIntegerType(QualType Ty) const;
   bool isCompoundType(QualType Ty) const;
   bool isVectorArgumentType(QualType Ty) const;
   bool isFPArgumentType(QualType Ty) const;
   QualType GetSingleElementType(QualType Ty) const;
 
   ABIArgInfo classifyReturnType(QualType RetTy) const;
public:
   }
   bool isSwiftErrorInRegister() const override {
     return false;
   }
 };
 
 class SystemZTargetCodeGenInfo : public TargetCodeGenInfo {
 public:
-  SystemZTargetCodeGenInfo(CodeGenTypes &CGT, bool HasVector)
-    : TargetCodeGenInfo(new SystemZABIInfo(CGT, HasVector)) {}
+  SystemZTargetCodeGenInfo(CodeGenTypes &CGT, bool HasVector, bool SoftFloatABI)
+    : TargetCodeGenInfo(new SystemZABIInfo(CGT, HasVector, SoftFloatABI)) {}
 };
 
 }
 
 bool SystemZABIInfo::isPromotableIntegerType(QualType Ty) const {
   // Treat an enum type as its underlying type.
   if (const EnumType *EnumTy = Ty->getAs<EnumType>())
     Ty = EnumTy->getDecl()->getIntegerType();

 
 bool SystemZABIInfo::isVectorArgumentType(QualType Ty) const {
   return (HasVector &&
           Ty->isVectorType() &&
           getContext().getTypeSize(Ty) <= 128);
 }
 
 bool SystemZABIInfo::isFPArgumentType(QualType Ty) const {
+  if (IsSoftFloatABI)
+    return false;
+
   if (const BuiltinType *BT = Ty->getAs<BuiltinType>())
     switch (BT->getKind()) {
     case BuiltinType::Float:
     case BuiltinType::Double:
       return true;
     default:
       return false;
     }
Address SystemZABIInfo::EmitVAArg(CodeGenFunction &CGF, Address VAListAddr,
   CharUnits UnpaddedSize;
   CharUnits DirectAlign;
   if (IsIndirect) {
     DirectTy = llvm::PointerType::getUnqual(DirectTy);
     UnpaddedSize = DirectAlign = CharUnits::fromQuantity(8);
   } else {
     if (AI.getCoerceToType())
       ArgTy = AI.getCoerceToType();
-    InFPRs = ArgTy->isFloatTy() || ArgTy->isDoubleTy();
+    InFPRs = (!IsSoftFloatABI && (ArgTy->isFloatTy() || ArgTy->isDoubleTy()));
     IsVector = ArgTy->isVectorTy();
     UnpaddedSize = TyInfo.first;
     DirectAlign = TyInfo.second;
   }
   CharUnits PaddedSize = CharUnits::fromQuantity(8);
   if (IsVector && UnpaddedSize > PaddedSize)
     PaddedSize = CharUnits::fromQuantity(16);
   assert((UnpaddedSize <= PaddedSize) && "Invalid argument size.");
if (ABIStr.endswith("f"))
       ABIFLen = 32;
     else if (ABIStr.endswith("d"))
       ABIFLen = 64;
     return SetCGInfo(new RISCVTargetCodeGenInfo(Types, XLen, ABIFLen));
   }
 
   case llvm::Triple::systemz: {
     bool HasVector = getTarget().getABI() == "vector";
-    return SetCGInfo(new SystemZTargetCodeGenInfo(Types, HasVector));
+    bool SoftFloat = CodeGenOpts.FloatABI == "soft";
+    return SetCGInfo(new SystemZTargetCodeGenInfo(Types, HasVector, SoftFloat));
   }
 
   case llvm::Triple::tce:
   case llvm::Triple::tcele:
     return SetCGInfo(new TCETargetCodeGenInfo(Types));
 
   case llvm::Triple::x86: {
     bool IsDarwinVectorABI = Triple.isOSDarwin();

clang/lib/Driver/ToolChains/Arch/SystemZ.h

 //===--- SystemZ.h - SystemZ-specific Tool Helpers --------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_ARCH_SYSTEMZ_H
 #define LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_ARCH_SYSTEMZ_H
 
+#include "clang/Driver/Driver.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Option/Option.h"
 #include <string>
 #include <vector>
 
 namespace clang {
 namespace driver {
 namespace tools {
 namespace systemz {
 
+enum class FloatABI {
+  Soft,
+  Hard,
+};
+
+FloatABI getSystemZFloatABI(const Driver &D, const llvm::opt::ArgList &Args);
+
 std::string getSystemZTargetCPU(const llvm::opt::ArgList &Args);
 
-void getSystemZTargetFeatures(const llvm::opt::ArgList &Args,
+void getSystemZTargetFeatures(const Driver &D, const llvm::opt::ArgList &Args,
                               std::vector<llvm::StringRef> &Features);
 
 } // end namespace systemz
 } // end namespace target
 } // end namespace driver
 } // end namespace clang
 
 #endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_ARCH_SYSTEMZ_H

clang/lib/Driver/ToolChains/Arch/SystemZ.cpp

 //===--- SystemZ.cpp - SystemZ Helpers for Tools ----------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "SystemZ.h"
+#include "clang/Driver/DriverDiagnostic.h"
 #include "clang/Driver/Options.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Support/Host.h"
 
 using namespace clang::driver;
 using namespace clang::driver::tools;
 using namespace clang;
 using namespace llvm::opt;
 
+systemz::FloatABI systemz::getSystemZFloatABI(const Driver &D,
+                                              const ArgList &Args) {
+  // Hard float is the default.
+  systemz::FloatABI ABI = systemz::FloatABI::Hard;
+  if (Args.hasArg(options::OPT_mfloat_abi_EQ))
+    D.Diag(diag::err_drv_unsupported_opt)
+      << Args.getLastArg(options::OPT_mfloat_abi_EQ)->getAsString(Args);
+
+  if (Arg *A = Args.getLastArg(clang::driver::options::OPT_msoft_float,
+                               options::OPT_mhard_float))
+    if (A->getOption().matches(clang::driver::options::OPT_msoft_float))
+      ABI = systemz::FloatABI::Soft;
+
+  return ABI;
+}
+
 std::string systemz::getSystemZTargetCPU(const ArgList &Args) {
   if (const Arg *A = Args.getLastArg(clang::driver::options::OPT_march_EQ)) {
     llvm::StringRef CPUName = A->getValue();
 
     if (CPUName == "native") {
       std::string CPU = llvm::sys::getHostCPUName();
       if (!CPU.empty() && CPU != "generic")
         return CPU;
       else
         return "";
     }
 
     return CPUName;
   }
   return "z10";
 }
 
-void systemz::getSystemZTargetFeatures(const ArgList &Args,
+void systemz::getSystemZTargetFeatures(const Driver &D, const ArgList &Args,
                                        std::vector<llvm::StringRef> &Features) {
   // -m(no-)htm overrides use of the transactional-execution facility.
   if (Arg *A = Args.getLastArg(options::OPT_mhtm, options::OPT_mno_htm)) {
     if (A->getOption().matches(options::OPT_mhtm))
       Features.push_back("+transactional-execution");
     else
       Features.push_back("-transactional-execution");
   }
   // -m(no-)vx overrides use of the vector facility.
   if (Arg *A = Args.getLastArg(options::OPT_mvx, options::OPT_mno_vx)) {
     if (A->getOption().matches(options::OPT_mvx))
       Features.push_back("+vector");
     else
       Features.push_back("-vector");
   }
+
+  systemz::FloatABI FloatABI = systemz::getSystemZFloatABI(D, Args);
+  if (FloatABI == systemz::FloatABI::Soft)
+    Features.push_back("+soft-float");
 }

clang/lib/Driver/ToolChains/Clang.cpp

static void getTargetFeatures(const ToolChain &TC, const llvm::Triple &Triple,
   case llvm::Triple::ppc64le:
     ppc::getPPCTargetFeatures(D, Triple, Args, Features);
     break;
   case llvm::Triple::riscv32:
   case llvm::Triple::riscv64:
     riscv::getRISCVTargetFeatures(D, Triple, Args, Features);
     break;
   case llvm::Triple::systemz:
-    systemz::getSystemZTargetFeatures(Args, Features);
+    systemz::getSystemZTargetFeatures(D, Args, Features);
     break;
   case llvm::Triple::aarch64:
   case llvm::Triple::aarch64_32:
   case llvm::Triple::aarch64_be:
     aarch64::getAArch64TargetFeatures(D, Triple, Args, Features);
     break;
   case llvm::Triple::x86:
   case llvm::Triple::x86_64:
if (HasBackchain && HasPackedStack) {
     D.Diag(diag::err_drv_unsupported_opt)
       << Args.getLastArg(options::OPT_mpacked_stack)->getAsString(Args) +
       " " + Args.getLastArg(options::OPT_mbackchain)->getAsString(Args);
   }
   if (HasBackchain)
     CmdArgs.push_back("-mbackchain");
   if (HasPackedStack)
     CmdArgs.push_back("-mpacked-stack");
+
+  systemz::FloatABI FloatABI =
+      systemz::getSystemZFloatABI(getToolChain().getDriver(), Args);
+
+  if (FloatABI == systemz::FloatABI::Soft) {
+    // Floating point operations and argument passing are soft.
+    CmdArgs.push_back("-msoft-float");
+    CmdArgs.push_back("-mfloat-abi");
+    CmdArgs.push_back("soft");
+  }
 }
 
 static void addX86AlignBranchArgs(const Driver &D, const ArgList &Args,
                                   ArgStringList &CmdArgs) {
   if (Args.hasArg(options::OPT_mbranches_within_32B_boundaries)) {
     CmdArgs.push_back("-mllvm");
     CmdArgs.push_back("-x86-branches-within-32B-boundaries");
   }

clang/test/CodeGen/systemz-abi.c

 // RUN: %clang_cc1 -triple s390x-linux-gnu \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-feature +vector \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu z13 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu arch11 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu z14 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu arch12 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu z15 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
 // RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu arch13 \
-// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN:   -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,HARD-FLOAT
+// RUN: %clang_cc1 -triple s390x-linux-gnu -target-cpu arch13 \
+// RUN:   -emit-llvm -o - %s -mfloat-abi soft | FileCheck %s \
+// RUN:   --check-prefixes=CHECK,SOFT-FLOAT
 
 // Scalar types
 
 char pass_char(char arg) { return arg; }
 // CHECK-LABEL: define signext i8 @pass_char(i8 signext %{{.*}})
 
 short pass_short(short arg) { return arg; }
 // CHECK-LABEL: define signext i16 @pass_short(i16 signext %{{.*}})

 struct agg_16byte pass_agg_16byte(struct agg_16byte arg) { return arg; }
 // CHECK-LABEL: define void @pass_agg_16byte(%struct.agg_16byte* noalias sret %{{.*}}, %struct.agg_16byte* %{{.*}})
 
 
 // Float-like aggregate types
 
 struct agg_float { float a; };
 struct agg_float pass_agg_float(struct agg_float arg) { return arg; }
-// CHECK-LABEL: define void @pass_agg_float(%struct.agg_float* noalias sret %{{.*}}, float %{{.*}})
+// HARD-FLOAT-LABEL: define void @pass_agg_float(%struct.agg_float* noalias sret %{{.*}}, float %{{.*}})
+// SOFT-FLOAT-LABEL: define void @pass_agg_float(%struct.agg_float* noalias sret %{{.*}}, i32 %{{.*}})
 
 struct agg_double { double a; };
 struct agg_double pass_agg_double(struct agg_double arg) { return arg; }
-// CHECK-LABEL: define void @pass_agg_double(%struct.agg_double* noalias sret %{{.*}}, double %{{.*}})
+// HARD-FLOAT-LABEL: define void @pass_agg_double(%struct.agg_double* noalias sret %{{.*}}, double %{{.*}})
+// SOFT-FLOAT-LABEL: define void @pass_agg_double(%struct.agg_double* noalias sret %{{.*}}, i64 %{{.*}})
 
 struct agg_longdouble { long double a; };
 struct agg_longdouble pass_agg_longdouble(struct agg_longdouble arg) { return arg; }
 // CHECK-LABEL: define void @pass_agg_longdouble(%struct.agg_longdouble* noalias sret %{{.*}}, %struct.agg_longdouble* %{{.*}})
 
 struct agg_float_a8 { float a __attribute__((aligned (8))); };
 struct agg_float_a8 pass_agg_float_a8(struct agg_float_a8 arg) { return arg; }
-// CHECK-LABEL: define void @pass_agg_float_a8(%struct.agg_float_a8* noalias sret %{{.*}}, double %{{.*}})
+// HARD-FLOAT-LABEL: define void @pass_agg_float_a8(%struct.agg_float_a8* noalias sret %{{.*}}, double %{{.*}})
+// SOFT-FLOAT-LABEL: define void @pass_agg_float_a8(%struct.agg_float_a8* noalias sret %{{.*}}, i64 %{{.*}})
 
 struct agg_float_a16 { float a __attribute__((aligned (16))); };
 struct agg_float_a16 pass_agg_float_a16(struct agg_float_a16 arg) { return arg; }
 // CHECK-LABEL: define void @pass_agg_float_a16(%struct.agg_float_a16* noalias sret %{{.*}}, %struct.agg_float_a16* %{{.*}})
 
 
 // Verify that the following are *not* float-like aggregate types
 

 // CHECK: [[OVERFLOW_ARG_AREA2:%[^ ]+]] = getelementptr i8, i8* [[OVERFLOW_ARG_AREA]], i64 8
 // CHECK: store i8* [[OVERFLOW_ARG_AREA2]], i8** [[OVERFLOW_ARG_AREA_PTR]]
 // CHECK: [[VA_ARG_ADDR:%[^ ]+]] = phi i64* [ [[REG_ADDR]], %{{.*}} ], [ [[MEM_ADDR]], %{{.*}} ]
 // CHECK: [[RET:%[^ ]+]] = load i64, i64* [[VA_ARG_ADDR]]
 // CHECK: ret i64 [[RET]]
 
 double va_double(__builtin_va_list l) { return __builtin_va_arg(l, double); }
 // CHECK-LABEL: define double @va_double(%struct.__va_list_tag* %{{.*}})
-// CHECK: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// HARD-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// SOFT-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 0
 // CHECK: [[REG_COUNT:%[^ ]+]] = load i64, i64* [[REG_COUNT_PTR]]
-// CHECK: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// HARD-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// SOFT-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 5
 // CHECK: br i1 [[FITS_IN_REGS]],
 // CHECK: [[SCALED_REG_COUNT:%[^ ]+]] = mul i64 [[REG_COUNT]], 8
-// CHECK: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// HARD-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// SOFT-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 16
 // CHECK: [[REG_SAVE_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 3
 // CHECK: [[REG_SAVE_AREA:%[^ ]+]] = load i8*, i8** [[REG_SAVE_AREA_PTR:[^ ]+]]
 // CHECK: [[RAW_REG_ADDR:%[^ ]+]] = getelementptr i8, i8* [[REG_SAVE_AREA]], i64 [[REG_OFFSET]]
 // CHECK: [[REG_ADDR:%[^ ]+]] = bitcast i8* [[RAW_REG_ADDR]] to double*
 // CHECK: [[REG_COUNT1:%[^ ]+]] = add i64 [[REG_COUNT]], 1
 // CHECK: store i64 [[REG_COUNT1]], i64* [[REG_COUNT_PTR]]
 // CHECK: [[OVERFLOW_ARG_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 2
 // CHECK: [[OVERFLOW_ARG_AREA:%[^ ]+]] = load i8*, i8** [[OVERFLOW_ARG_AREA_PTR]]

 // CHECK: [[MEM_ADDR:%[^ ]+]] = bitcast i8* [[RAW_MEM_ADDR]] to %struct.agg_8byte*
 // CHECK: [[OVERFLOW_ARG_AREA2:%[^ ]+]] = getelementptr i8, i8* [[OVERFLOW_ARG_AREA]], i64 8
 // CHECK: store i8* [[OVERFLOW_ARG_AREA2]], i8** [[OVERFLOW_ARG_AREA_PTR]]
 // CHECK: [[VA_ARG_ADDR:%[^ ]+]] = phi %struct.agg_8byte* [ [[REG_ADDR]], %{{.*}} ], [ [[MEM_ADDR]], %{{.*}} ]
 // CHECK: ret void
 
 struct agg_float va_agg_float(__builtin_va_list l) { return __builtin_va_arg(l, struct agg_float); }
 // CHECK-LABEL: define void @va_agg_float(%struct.agg_float* noalias sret %{{.*}}, %struct.__va_list_tag* %{{.*}}
-// CHECK: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// HARD-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// SOFT-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 0
 // CHECK: [[REG_COUNT:%[^ ]+]] = load i64, i64* [[REG_COUNT_PTR]]
-// CHECK: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// HARD-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// SOFT-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 5
 // CHECK: br i1 [[FITS_IN_REGS]],
 // CHECK: [[SCALED_REG_COUNT:%[^ ]+]] = mul i64 [[REG_COUNT]], 8
-// CHECK: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// HARD-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// SOFT-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 20
 // CHECK: [[REG_SAVE_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 3
 // CHECK: [[REG_SAVE_AREA:%[^ ]+]] = load i8*, i8** [[REG_SAVE_AREA_PTR:[^ ]+]]
 // CHECK: [[RAW_REG_ADDR:%[^ ]+]] = getelementptr i8, i8* [[REG_SAVE_AREA]], i64 [[REG_OFFSET]]
 // CHECK: [[REG_ADDR:%[^ ]+]] = bitcast i8* [[RAW_REG_ADDR]] to %struct.agg_float*
 // CHECK: [[REG_COUNT1:%[^ ]+]] = add i64 [[REG_COUNT]], 1
 // CHECK: store i64 [[REG_COUNT1]], i64* [[REG_COUNT_PTR]]
 // CHECK: [[OVERFLOW_ARG_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 2
 // CHECK: [[OVERFLOW_ARG_AREA:%[^ ]+]] = load i8*, i8** [[OVERFLOW_ARG_AREA_PTR]]
 // CHECK: [[RAW_MEM_ADDR:%[^ ]+]] = getelementptr i8, i8* [[OVERFLOW_ARG_AREA]], i64 4
 // CHECK: [[MEM_ADDR:%[^ ]+]] = bitcast i8* [[RAW_MEM_ADDR]] to %struct.agg_float*
 // CHECK: [[OVERFLOW_ARG_AREA2:%[^ ]+]] = getelementptr i8, i8* [[OVERFLOW_ARG_AREA]], i64 8
 // CHECK: store i8* [[OVERFLOW_ARG_AREA2]], i8** [[OVERFLOW_ARG_AREA_PTR]]
 // CHECK: [[VA_ARG_ADDR:%[^ ]+]] = phi %struct.agg_float* [ [[REG_ADDR]], %{{.*}} ], [ [[MEM_ADDR]], %{{.*}} ]
 // CHECK: ret void
 
 struct agg_double va_agg_double(__builtin_va_list l) { return __builtin_va_arg(l, struct agg_double); }
 // CHECK-LABEL: define void @va_agg_double(%struct.agg_double* noalias sret %{{.*}}, %struct.__va_list_tag* %{{.*}}
-// CHECK: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// HARD-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// SOFT-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 0
 // CHECK: [[REG_COUNT:%[^ ]+]] = load i64, i64* [[REG_COUNT_PTR]]
-// CHECK: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// HARD-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// SOFT-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 5
 // CHECK: br i1 [[FITS_IN_REGS]],
 // CHECK: [[SCALED_REG_COUNT:%[^ ]+]] = mul i64 [[REG_COUNT]], 8
-// CHECK: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// HARD-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// SOFT-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 16
 // CHECK: [[REG_SAVE_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 3
 // CHECK: [[REG_SAVE_AREA:%[^ ]+]] = load i8*, i8** [[REG_SAVE_AREA_PTR:[^ ]+]]
 // CHECK: [[RAW_REG_ADDR:%[^ ]+]] = getelementptr i8, i8* [[REG_SAVE_AREA]], i64 [[REG_OFFSET]]
 // CHECK: [[REG_ADDR:%[^ ]+]] = bitcast i8* [[RAW_REG_ADDR]] to %struct.agg_double*
 // CHECK: [[REG_COUNT1:%[^ ]+]] = add i64 [[REG_COUNT]], 1
 // CHECK: store i64 [[REG_COUNT1]], i64* [[REG_COUNT_PTR]]
 // CHECK: [[OVERFLOW_ARG_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 2
 // CHECK: [[OVERFLOW_ARG_AREA:%[^ ]+]] = load i8*, i8** [[OVERFLOW_ARG_AREA_PTR]]

 // CHECK: [[OVERFLOW_ARG_AREA2:%[^ ]+]] = getelementptr i8, i8* [[OVERFLOW_ARG_AREA]], i64 8
 // CHECK: store i8* [[OVERFLOW_ARG_AREA2]], i8** [[OVERFLOW_ARG_AREA_PTR]]
 // CHECK: [[VA_ARG_ADDR:%[^ ]+]] = phi %struct.agg_longdouble** [ [[REG_ADDR]], %{{.*}} ], [ [[MEM_ADDR]], %{{.*}} ]
 // CHECK: [[INDIRECT_ARG:%[^ ]+]] = load %struct.agg_longdouble*, %struct.agg_longdouble** [[VA_ARG_ADDR]]
 // CHECK: ret void
 
 struct agg_float_a8 va_agg_float_a8(__builtin_va_list l) { return __builtin_va_arg(l, struct agg_float_a8); }
 // CHECK-LABEL: define void @va_agg_float_a8(%struct.agg_float_a8* noalias sret %{{.*}}, %struct.__va_list_tag* %{{.*}}
-// CHECK: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// HARD-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 1
+// SOFT-FLOAT: [[REG_COUNT_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 0
 // CHECK: [[REG_COUNT:%[^ ]+]] = load i64, i64* [[REG_COUNT_PTR]]
-// CHECK: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// HARD-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 4
+// SOFT-FLOAT: [[FITS_IN_REGS:%[^ ]+]] = icmp ult i64 [[REG_COUNT]], 5
 // CHECK: br i1 [[FITS_IN_REGS]],
 // CHECK: [[SCALED_REG_COUNT:%[^ ]+]] = mul i64 [[REG_COUNT]], 8
-// CHECK: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// HARD-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 128
+// SOFT-FLOAT: [[REG_OFFSET:%[^ ]+]] = add i64 [[SCALED_REG_COUNT]], 16
 // CHECK: [[REG_SAVE_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 3
 // CHECK: [[REG_SAVE_AREA:%[^ ]+]] = load i8*, i8** [[REG_SAVE_AREA_PTR:[^ ]+]]
 // CHECK: [[RAW_REG_ADDR:%[^ ]+]] = getelementptr i8, i8* [[REG_SAVE_AREA]], i64 [[REG_OFFSET]]
 // CHECK: [[REG_ADDR:%[^ ]+]] = bitcast i8* [[RAW_REG_ADDR]] to %struct.agg_float_a8*
 // CHECK: [[REG_COUNT1:%[^ ]+]] = add i64 [[REG_COUNT]], 1
 // CHECK: store i64 [[REG_COUNT1]], i64* [[REG_COUNT_PTR]]
 // CHECK: [[OVERFLOW_ARG_AREA_PTR:%[^ ]+]] = getelementptr inbounds %struct.__va_list_tag, %struct.__va_list_tag* %{{.*}}, i32 0, i32 2
 // CHECK: [[OVERFLOW_ARG_AREA:%[^ ]+]] = load i8*, i8** [[OVERFLOW_ARG_AREA_PTR]]

clang/test/CodeGen/systemz-abi.cpp

 // RUN: %clang_cc1 -triple s390x-linux-gnu -emit-llvm -x c++ -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple s390x-linux-gnu -emit-llvm -x c++ -o - %s -mfloat-abi soft \
+// RUN:   | FileCheck %s --check-prefix=SOFT-FLOAT
 
 // For compatibility with GCC, this structure is passed in an FPR in C++,
 // but passed in a GPR in C (checked in systemz-abi.c).
 
 struct agg_float_cpp { float a; int : 0; };
 struct agg_float_cpp pass_agg_float_cpp(struct agg_float_cpp arg) { return arg; }
 // CHECK-LABEL: define void @_Z18pass_agg_float_cpp13agg_float_cpp(%struct.agg_float_cpp* noalias sret %{{.*}}, float %{{.*}})
-
+// SOFT-FLOAT:  define void @_Z18pass_agg_float_cpp13agg_float_cpp(%struct.agg_float_cpp* noalias sret %{{.*}}, i32 %{{.*}})

clang/test/CodeGen/target-data.c

 // ARC: target datalayout = "e-m:e-p:32:32-i1:8:32-i8:8:32-i16:16:32-i32:32:32-f32:32:32-i64:32-f64:32-a:0:32-n32"
 
 // RUN: %clang_cc1 -triple hexagon-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=HEXAGON
 // HEXAGON: target datalayout = "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048"
 
 // RUN: %clang_cc1 -triple s390x-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=SYSTEMZ
+// RUN: %clang_cc1 -triple s390x-unknown -target-cpu z13 -target-feature +soft-float -o - -emit-llvm %s | \
+// RUN: FileCheck %s -check-prefix=SYSTEMZ
 // SYSTEMZ: target datalayout = "E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-a:8:16-n32:64"
 
 // RUN: %clang_cc1 -triple s390x-unknown -target-cpu z13 -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=SYSTEMZ-VECTOR
 // RUN: %clang_cc1 -triple s390x-unknown -target-cpu arch11 -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=SYSTEMZ-VECTOR
 // RUN: %clang_cc1 -triple s390x-unknown -target-cpu z14 -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=SYSTEMZ-VECTOR

clang/test/Driver/systemz-float-01.c

+// Check handling -mhard-float / -msoft-float options
+// when build for SystemZ platforms.
+//
+// Default
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu \
+// RUN:   | FileCheck --check-prefix=CHECK-DEF %s
+// CHECK-DEF-NOT: "-msoft-float"
+// CHECK-DEF-NOT: "-mfloat-abi" "soft"
+//
+// -mhard-float
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu -mhard-float \
+// RUN:   | FileCheck --check-prefix=CHECK-HARD %s
+// CHECK-HARD-NOT: "-msoft-float"
+// CHECK-HARD-NOT: "-mfloat-abi" "soft"
+//
+// -msoft-float
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu -msoft-float \
+// RUN:   | FileCheck --check-prefix=CHECK-SOFT %s
+// CHECK-SOFT: "-msoft-float" "-mfloat-abi" "soft"
+//
+// -mfloat-abi=soft
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu -mfloat-abi=soft \
+// RUN:   | FileCheck --check-prefix=CHECK-FLOATABISOFT %s
+// CHECK-FLOATABISOFT: error: unsupported option '-mfloat-abi=soft'
+//
+// -mfloat-abi=hard
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu -mfloat-abi=hard \
+// RUN:   | FileCheck --check-prefix=CHECK-FLOATABIHARD %s
+// CHECK-FLOATABIHARD: error: unsupported option '-mfloat-abi=hard'
+//
+// check invalid -mfloat-abi
+// RUN: %clang -c %s -### -o %t.o 2>&1 \
+// RUN:     -target s390x-linux-gnu -mfloat-abi=x \
+// RUN:   | FileCheck --check-prefix=CHECK-ERRMSG %s
+// CHECK-ERRMSG: error: unsupported option '-mfloat-abi=x'
+
+int foo(void) {
+  return 0;
+}
+

clang/test/Driver/systemz-float-02.c

+// RUN: %clang -target s390x-linux-gnu -march=z13 -S %s -o - -msoft-float | FileCheck %s
+//
+// Check that -msoft-float works all the way to assembly output.
+
+double fun0(double *A) {
+// CHECK-LABEL: fun0
+// CHECK-NOT: {{%f[0-9]}}
+// CHECK: brasl %r14, __adddf3@PLT
+  return *A + 1.0;
+}
+
+typedef int v4si __attribute__ ((vector_size (16)));
+v4si fun1(v4si *A) {
+// CHECK-LABEL: fun1
+// CHECK-NOT: {{%[v][0-9]}}
+// CHECK: ark
+// CHECK-NEXT: ark
+// CHECK-NEXT: ark
+// CHECK-NEXT: ark
+  v4si B = {1, 1, 1, 1};
+  return *A + B;
+}

llvm/lib/Target/SystemZ/SystemZFeatures.td

 
 class SystemZFeatureList<list<SystemZFeature> x> {
   list<SystemZFeature> List = x;
 }
 
 class SystemZFeatureAdd<list<SystemZFeature> x, list<SystemZFeature> y>
   : SystemZFeatureList<!listconcat(x, y)>;
 
+// This feature is added as a subtarget feature whenever the function is
+// compiled to use soft-float.
+def FeatureSoftFloat : SystemZFeature<
+  "soft-float", "SoftFloat",
+  "Use software emulation for floating point"
+>;
+
 //===----------------------------------------------------------------------===//
 //
 // New features added in the Ninth Edition of the z/Architecture
 //
 //===----------------------------------------------------------------------===//
 
 def FeatureDistinctOps : SystemZFeature<
   "distinct-ops", "DistinctOps",

llvm/lib/Target/SystemZ/SystemZISelLowering.h

 class SystemZSubtarget;
 class SystemZTargetMachine;
 
 class SystemZTargetLowering : public TargetLowering {
 public:
   explicit SystemZTargetLowering(const TargetMachine &TM,
                                  const SystemZSubtarget &STI);
 
+  bool useSoftFloat() const override;
+
   // Override TargetLowering.
   MVT getScalarShiftAmountTy(const DataLayout &, EVT) const override {
     return MVT::i32;
   }
   MVT getVectorIdxTy(const DataLayout &DL) const override {
     // Only the lower 12 bits of an element index are used, so we don't
     // want to clobber the upper 32 bits of a GPR unnecessarily.
     return MVT::i32;

llvm/lib/Target/SystemZ/SystemZISelLowering.cpp

SystemZTargetLowering::SystemZTargetLowering(const TargetMachine &TM,
   MVT PtrVT = MVT::getIntegerVT(8 * TM.getPointerSize(0));
 
   // Set up the register classes.
   if (Subtarget.hasHighWord())
     addRegisterClass(MVT::i32, &SystemZ::GRX32BitRegClass);
   else
     addRegisterClass(MVT::i32, &SystemZ::GR32BitRegClass);
   addRegisterClass(MVT::i64, &SystemZ::GR64BitRegClass);
+  if (!useSoftFloat()) {
     if (Subtarget.hasVector()) {
       addRegisterClass(MVT::f32, &SystemZ::VR32BitRegClass);
       addRegisterClass(MVT::f64, &SystemZ::VR64BitRegClass);
     } else {
       addRegisterClass(MVT::f32, &SystemZ::FP32BitRegClass);
       addRegisterClass(MVT::f64, &SystemZ::FP64BitRegClass);
     }
     if (Subtarget.hasVectorEnhancements1())
       addRegisterClass(MVT::f128, &SystemZ::VR128BitRegClass);
     else
       addRegisterClass(MVT::f128, &SystemZ::FP128BitRegClass);
 
     if (Subtarget.hasVector()) {
       addRegisterClass(MVT::v16i8, &SystemZ::VR128BitRegClass);
       addRegisterClass(MVT::v8i16, &SystemZ::VR128BitRegClass);
       addRegisterClass(MVT::v4i32, &SystemZ::VR128BitRegClass);
       addRegisterClass(MVT::v2i64, &SystemZ::VR128BitRegClass);
       addRegisterClass(MVT::v4f32, &SystemZ::VR128BitRegClass);
       addRegisterClass(MVT::v2f64, &SystemZ::VR128BitRegClass);
     }
+  }
 
   // Compute derived properties from the register classes
   computeRegisterProperties(Subtarget.getRegisterInfo());
 
   // Set up special registers.
   setStackPointerRegisterToSaveRestore(SystemZ::R15D);
 
   // TODO: It may be better to default to latency-oriented scheduling, however
SystemZTargetLowering::SystemZTargetLowering(const TargetMachine &TM,
   // than "STC;MVC".  Handle the choice in target-specific code instead.
   MaxStoresPerMemset = 0;
   MaxStoresPerMemsetOptSize = 0;
 
   // Default to having -disable-strictnode-mutation on
   IsStrictFPEnabled = true;
 }
 
+bool SystemZTargetLowering::useSoftFloat() const {
+  return Subtarget.hasSoftFloat();
+}
+
 EVT SystemZTargetLowering::getSetCCResultType(const DataLayout &DL,
                                               LLVMContext &, EVT VT) const {
   if (!VT.isVector())
     return MVT::i32;
   return VT.changeVectorElementTypeToInteger();
 }
 
 bool SystemZTargetLowering::isFMAFasterThanFMulAndFAdd(
if (!Failed && Index < Size && Map[Index])
       return std::make_pair(Map[Index], RC);
   }
   return std::make_pair(0U, nullptr);
 }
 
 std::pair<unsigned, const TargetRegisterClass *>
 SystemZTargetLowering::getRegForInlineAsmConstraint(
     const TargetRegisterInfo *TRI, StringRef Constraint, MVT VT) const {
   if (Constraint.size() == 1) {
     // GCC Constraint Letters
     switch (Constraint[0]) {
     default: break;
     case 'd': // Data register (equivalent to 'r')
     case 'r': // General-purpose register
       if (VT == MVT::i64)
         return std::make_pair(0U, &SystemZ::GR64BitRegClass);
       else if (VT == MVT::i128)
         return std::make_pair(0U, &SystemZ::GR128BitRegClass);
       return std::make_pair(0U, &SystemZ::GR32BitRegClass);
 
     case 'a': // Address register
       if (VT == MVT::i64)
         return std::make_pair(0U, &SystemZ::ADDR64BitRegClass);
       else if (VT == MVT::i128)
         return std::make_pair(0U, &SystemZ::ADDR128BitRegClass);
       return std::make_pair(0U, &SystemZ::ADDR32BitRegClass);
 
     case 'h': // High-part register (an LLVM extension)
       return std::make_pair(0U, &SystemZ::GRH32BitRegClass);
 
     case 'f': // Floating-point register
+      if (!useSoftFloat()) {
         if (VT == MVT::f64)
           return std::make_pair(0U, &SystemZ::FP64BitRegClass);
         else if (VT == MVT::f128)
           return std::make_pair(0U, &SystemZ::FP128BitRegClass);
         return std::make_pair(0U, &SystemZ::FP32BitRegClass);
-
+      }
+      break;
     case 'v': // Vector register
       if (Subtarget.hasVector()) {
         if (VT == MVT::f32)
           return std::make_pair(0U, &SystemZ::VR32BitRegClass);
         if (VT == MVT::f64)
           return std::make_pair(0U, &SystemZ::VR64BitRegClass);
         return std::make_pair(0U, &SystemZ::VR128BitRegClass);
       }
if (Constraint[1] == 'r') {
         return parseRegisterNumber(Constraint, &SystemZ::GR32BitRegClass,
                                    SystemZMC::GR32Regs, 16);
       if (VT == MVT::i128)
         return parseRegisterNumber(Constraint, &SystemZ::GR128BitRegClass,
                                    SystemZMC::GR128Regs, 16);
       return parseRegisterNumber(Constraint, &SystemZ::GR64BitRegClass,
                                  SystemZMC::GR64Regs, 16);
     }
-    if (Constraint[1] == 'f') {
+    if (Constraint[1] == 'f' && !useSoftFloat()) {
       if (VT == MVT::f32)
         return parseRegisterNumber(Constraint, &SystemZ::FP32BitRegClass,
                                    SystemZMC::FP32Regs, 16);
       if (VT == MVT::f128)
         return parseRegisterNumber(Constraint, &SystemZ::FP128BitRegClass,
                                    SystemZMC::FP128Regs, 16);
       return parseRegisterNumber(Constraint, &SystemZ::FP64BitRegClass,
                                  SystemZMC::FP64Regs, 16);
     }
     if (Constraint[1] == 'v') {
       if (VT == MVT::f32)
         return parseRegisterNumber(Constraint, &SystemZ::VR32BitRegClass,
                                    SystemZMC::VR32Regs, 32);
       if (VT == MVT::f64)
         return parseRegisterNumber(Constraint, &SystemZ::VR64BitRegClass,
                                    SystemZMC::VR64Regs, 32);
       return parseRegisterNumber(Constraint, &SystemZ::VR128BitRegClass,
                                  SystemZMC::VR128Regs, 32);
if (IsVarArg) {
     // ...and a similar frame index for the caller-allocated save area
     // that will be used to store the incoming registers.
     int64_t RegSaveOffset = -SystemZMC::CallFrameSize;
     unsigned RegSaveIndex = MFI.CreateFixedObject(1, RegSaveOffset, true);
     FuncInfo->setRegSaveFrameIndex(RegSaveIndex);
 
     // Store the FPR varargs in the reserved frame slots.  (We store the
     // GPRs as part of the prologue.)
-    if (NumFixedFPRs < SystemZ::NumArgFPRs) {
+    if (NumFixedFPRs < SystemZ::NumArgFPRs && !useSoftFloat()) {
       SDValue MemOps[SystemZ::NumArgFPRs];
       for (unsigned I = NumFixedFPRs; I < SystemZ::NumArgFPRs; ++I) {
         unsigned Offset = TFL->getRegSpillOffset(SystemZ::ArgFPRs[I]);
         int FI = MFI.CreateFixedObject(8, RegSaveOffset + Offset, true);
         SDValue FIN = DAG.getFrameIndex(FI, getPointerTy(DAG.getDataLayout()));
         unsigned VReg = MF.addLiveIn(SystemZ::ArgFPRs[I],
                                      &SystemZ::FP64BitRegClass);
         SDValue ArgValue = DAG.getCopyFromReg(Chain, DL, VReg, MVT::f64);

llvm/lib/Target/SystemZ/SystemZSubtarget.h

protected:
   bool HasVectorPackedDecimal;
   bool HasInsertReferenceBitsMultiple;
   bool HasMiscellaneousExtensions3;
   bool HasMessageSecurityAssist9;
   bool HasVectorEnhancements2;
   bool HasVectorPackedDecimalEnhancement;
   bool HasEnhancedSort;
   bool HasDeflateConversion;
+  bool HasSoftFloat;
 
 private:
   Triple TargetTriple;
   SystemZInstrInfo InstrInfo;
   SystemZTargetLowering TLInfo;
   SystemZSelectionDAGInfo TSInfo;
   SystemZFrameLowering FrameLowering;
 
public:
   }
 
   // Return true if the target has the enhanced-sort facility.
   bool hasEnhancedSort() const { return HasEnhancedSort; }
 
   // Return true if the target has the deflate-conversion facility.
   bool hasDeflateConversion() const { return HasDeflateConversion; }
 
+  // Return true if soft float should be used.
+  bool hasSoftFloat() const { return HasSoftFloat; }
+
   // Return true if GV can be accessed using LARL for reloc model RM
   // and code model CM.
   bool isPC32DBLSymbol(const GlobalValue *GV, CodeModel::Model CM) const;
 
   bool isTargetELF() const { return TargetTriple.isOSBinFormatELF(); }
 };
 } // end namespace llvm
 
 #endif

llvm/lib/Target/SystemZ/SystemZSubtarget.cpp

 
 SystemZSubtarget &
 SystemZSubtarget::initializeSubtargetDependencies(StringRef CPU, StringRef FS) {
   std::string CPUName = CPU;
   if (CPUName.empty())
     CPUName = "generic";
   // Parse features string.
   ParseSubtargetFeatures(CPUName, FS);
+
+  // -msoft-float implies -mno-vx.
+  if (HasSoftFloat)
+    HasVector = false;
+
   return *this;
 }
 
 SystemZSubtarget::SystemZSubtarget(const Triple &TT, const std::string &CPU,
                                    const std::string &FS,
                                    const TargetMachine &TM)
     : SystemZGenSubtargetInfo(TT, CPU, FS), HasDistinctOps(false),
       HasLoadStoreOnCond(false), HasHighWord(false), HasFPExtension(false),
       HasPopulationCount(false), HasMessageSecurityAssist3(false),
       HasMessageSecurityAssist4(false), HasResetReferenceBitsMultiple(false),
       HasFastSerialization(false), HasInterlockedAccess1(false),
       HasMiscellaneousExtensions(false),
       HasExecutionHint(false), HasLoadAndTrap(false),
       HasTransactionalExecution(false), HasProcessorAssist(false),
       HasDFPZonedConversion(false), HasEnhancedDAT2(false),
       HasVector(false), HasLoadStoreOnCond2(false),
       HasLoadAndZeroRightmostByte(false), HasMessageSecurityAssist5(false),
       HasDFPPackedConversion(false),
       HasMiscellaneousExtensions2(false), HasGuardedStorage(false),
       HasMessageSecurityAssist7(false), HasMessageSecurityAssist8(false),
       HasVectorEnhancements1(false), HasVectorPackedDecimal(false),
       HasInsertReferenceBitsMultiple(false),
       HasMiscellaneousExtensions3(false), HasMessageSecurityAssist9(false),
       HasVectorEnhancements2(false), HasVectorPackedDecimalEnhancement(false),
-      HasEnhancedSort(false), HasDeflateConversion(false),
+      HasEnhancedSort(false), HasDeflateConversion(false), HasSoftFloat(false),
       TargetTriple(TT), InstrInfo(initializeSubtargetDependencies(CPU, FS)),
       TLInfo(TM, *this), TSInfo(), FrameLowering() {}
 
 
 bool SystemZSubtarget::enableSubRegLiveness() const {
   return UseSubRegLiveness;
 }
 

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

 }
 
 // Determine whether we use the vector ABI.
 static bool UsesVectorABI(StringRef CPU, StringRef FS) {
   // We use the vector ABI whenever the vector facility is avaiable.
   // This is the case by default if CPU is z13 or later, and can be
   // overridden via "[+-]vector" feature string elements.
   bool VectorABI = true;
+  bool SoftFloat = false;
   if (CPU.empty() || CPU == "generic" ||
       CPU == "z10" || CPU == "z196" || CPU == "zEC12")
     VectorABI = false;
 
   SmallVector<StringRef, 3> Features;
   FS.split(Features, ',', -1, false /* KeepEmpty */);
   for (auto &Feature : Features) {
     if (Feature == "vector" || Feature == "+vector")
       VectorABI = true;
     if (Feature == "-vector")
       VectorABI = false;
+    if (Feature == "soft-float" || Feature == "+soft-float")
+      SoftFloat = true;
+    if (Feature == "-soft-float")
+      SoftFloat = false;
   }
 
-  return VectorABI;
+  return VectorABI && !SoftFloat;
 }
 
 static std::string computeDataLayout(const Triple &TT, StringRef CPU,
                                      StringRef FS) {
   bool VectorABI = UsesVectorABI(CPU, FS);
   std::string Ret;
 
   // Big endian.
   Ret += "E";

 /// SystemZ Code Generator Pass Configuration Options.
 class SystemZPassConfig : public TargetPassConfig {
 public:
   SystemZPassConfig(SystemZTargetMachine &TM, PassManagerBase &PM)
     : TargetPassConfig(TM, PM) {}
 
   SystemZTargetMachine &getSystemZTargetMachine() const {
     return getTM<SystemZTargetMachine>();
   }
 
   ScheduleDAGInstrs *
   createPostMachineScheduler(MachineSchedContext *C) const override {
     return new ScheduleDAGMI(C,
                              std::make_unique<SystemZPostRASchedStrategy>(C),
                              /*RemoveKillFlags=*/true);
   }
 
   void addIRPasses() override;
   bool addInstSelector() override;
   bool addILPOpts() override;
   void addPostRewrite() override;
   void addPostRegAlloc() override;
   void addPreSched2() override;
   void addPreEmitPass() override;
 };
 
 } // end anonymous namespace
 
 void SystemZPassConfig::addIRPasses() {
   if (getOptLevel() != CodeGenOpt::None) {
+    if (!getTM<SystemZTargetMachine>().getSubtargetImpl()->hasSoftFloat())
       addPass(createSystemZTDCPass());
     addPass(createLoopDataPrefetchPass());
   }
 
   TargetPassConfig::addIRPasses();
 }
 
 bool SystemZPassConfig::addInstSelector() {
   addPass(createSystemZISelDag(getSystemZTargetMachine(), getOptLevel()));

llvm/test/CodeGen/SystemZ/args-07.ll

 ; Test multiple return values (LLVM ABI extension)
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -verify-machineinstrs| FileCheck %s
+; RUN: llc < %s -mtriple=s390x-linux-gnu -verify-machineinstrs \
+; RUN:   -mattr=soft-float | FileCheck %s --check-prefix=SOFT-FLOAT
 
 ; Up to four integer return values fit into GPRs.
 define { i64, i64, i64, i64 } @f1() {
 ; CHECK-LABEL: f1:
 ; CHECK: lghi %r2, 0
 ; CHECK: lghi %r3, 1
 ; CHECK: lghi %r4, 2
 ; CHECK: lghi %r5, 3

 ; CHECK: ldeb %f0, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: ldeb %f2, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: ldeb %f4, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: ldeb %f6, 0([[TMP]])
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f3:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: llihh   %r2, 16368
+; SOFT-FLOAT-NEXT: llihh   %r3, 16384
+; SOFT-FLOAT-NEXT: llihh   %r4, 16392
+; SOFT-FLOAT-NEXT: llihh   %r5, 16400
+; SOFT-FLOAT-NEXT: br      %r14
   ret { double, double, double, double }
       { double 1.0, double 2.0, double 3.0, double 4.0 }
 }
 
 ; More than four floating-point return values use sret.
 define { double, double, double, double, double } @f4() {
 ; CHECK-LABEL: f4:
 ; CHECK: llihh [[TMP:%r[0-5]]], 16404
 ; CHECK: stg [[TMP]], 32(%r2)
 ; CHECK: llihh [[TMP:%r[0-5]]], 16400
 ; CHECK: stg [[TMP]], 24(%r2)
 ; CHECK: llihh [[TMP:%r[0-5]]], 16392
 ; CHECK: stg [[TMP]], 16(%r2)
 ; CHECK: llihh [[TMP:%r[0-5]]], 16384
 ; CHECK: stg [[TMP]], 8(%r2)
 ; CHECK: llihh [[TMP:%r[0-5]]], 16368
 ; CHECK: stg [[TMP]], 0(%r2)
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f4:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT-NOT: %r2
+; SOFT-FLOAT: llihh   %r0, 16404
+; SOFT-FLOAT-NEXT: stg     %r0, 32(%r2)
+; SOFT-FLOAT-NEXT: llihh   %r0, 16400
+; SOFT-FLOAT-NEXT: stg     %r0, 24(%r2)
+; SOFT-FLOAT-NEXT: llihh   %r0, 16392
+; SOFT-FLOAT-NEXT: stg     %r0, 16(%r2)
+; SOFT-FLOAT-NEXT: llihh   %r0, 16384
+; SOFT-FLOAT-NEXT: stg     %r0, 8(%r2)
+; SOFT-FLOAT-NEXT: llihh   %r0, 16368
+; SOFT-FLOAT-NEXT: stg     %r0, 0(%r2)
+; SOFT-FLOAT-NEXT: br      %r14
   ret { double, double, double, double, double }
       { double 1.0, double 2.0, double 3.0, double 4.0, double 5.0 }
 }

llvm/test/CodeGen/SystemZ/soft-float-01.ll

+; RUN: llc -mcpu=z10 -mattr=soft-float -O0 < %s | FileCheck %s
+
+; Arithmetic functions
+
+define float @test_addsf3(float %a, float %b) {
+  ; CHECK-LABEL:  test_addsf3:
+  ; CHECK:        brasl %r14, __addsf3
+  %add = fadd float %a, %b
+  ret float %add
+}
+
+define double @test_adddf3(double %a, double %b) {
+  ; CHECK-LABEL:  test_adddf3:
+  ; CHECK:        brasl %r14, __adddf3
+  %add = fadd double %a, %b
+  ret double %add
+}
+
+define fp128 @test_addtf3(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_addtf3:
+  ; CHECK:        brasl %r14, __addtf3
+  %add = fadd fp128 %a, %b
+  ret fp128 %add
+}
+
+define float @test_mulsf3(float %a, float %b) {
+  ; CHECK-LABEL:  test_mulsf3:
+  ; CHECK:        brasl %r14, __mulsf3
+  %mul = fmul float %a, %b
+  ret float %mul
+}
+
+define double @test_muldf3(double %a, double %b) {
+  ; CHECK-LABEL:  test_muldf3:
+  ; CHECK:        brasl %r14, __muldf3
+  %mul = fmul double %a, %b
+  ret double %mul
+}
+
+define fp128 @test_multf3(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_multf3:
+  ; CHECK:        brasl %r14, __multf3
+  %mul = fmul fp128 %a, %b
+  ret fp128 %mul
+}
+
+define float @test_subsf3(float %a, float %b) {
+  ; CHECK-LABEL:  test_subsf3:
+  ; CHECK:        brasl %r14, __subsf3
+  %sub = fsub float %a, %b
+  ret float %sub
+}
+
+define double @test_subdf3(double %a, double %b) {
+  ; CHECK-LABEL:  test_subdf3:
+  ; CHECK:        brasl %r14, __subdf3
+  %sub = fsub double %a, %b
+  ret double %sub
+}
+
+define fp128 @test_subtf3(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_subtf3:
+  ; CHECK:        brasl %r14, __subtf3
+  %sub = fsub fp128 %a, %b
+  ret fp128 %sub
+}
+
+define float @test_divsf3(float %a, float %b) {
+  ; CHECK-LABEL:  test_divsf3:
+  ; CHECK:        brasl %r14, __divsf3
+  %div = fdiv float %a, %b
+  ret float %div
+}
+
+define double @test_divdf3(double %a, double %b) {
+  ; CHECK-LABEL:  test_divdf3:
+  ; CHECK:        brasl %r14, __divdf3
+  %div = fdiv double %a, %b
+  ret double %div
+}
+
+define fp128 @test_divtf3(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_divtf3:
+  ; CHECK:        brasl %r14, __divtf3
+  %div = fdiv fp128 %a, %b
+  ret fp128 %div
+}
+
+; Comparison functions
+define i1 @test_unordsf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_unordsf2:
+  ; CHECK:        brasl %r14, __unordsf2
+  %cmp = fcmp uno float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_unorddf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_unorddf2:
+  ; CHECK:        brasl %r14, __unorddf2
+  %cmp = fcmp uno double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_unordtf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_unordtf2:
+  ; CHECK:        brasl %r14, __unordtf2
+  %cmp = fcmp uno fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_eqsf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_eqsf2:
+  ; CHECK:        brasl %r14, __eqsf2
+  %cmp = fcmp oeq float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_eqdf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_eqdf2:
+  ; CHECK:        brasl %r14, __eqdf2
+  %cmp = fcmp oeq double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_eqtf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_eqtf2:
+  ; CHECK:        brasl %r14, __eqtf2
+  %cmp = fcmp oeq fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_nesf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_nesf2:
+  ; CHECK:        brasl %r14, __nesf2
+  %cmp = fcmp une float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_nedf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_nedf2:
+  ; CHECK:        brasl %r14, __nedf2
+  %cmp = fcmp une double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_netf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_netf2:
+  ; CHECK:        brasl %r14, __netf2
+  %cmp = fcmp une fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_gesf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_gesf2:
+  ; CHECK:        brasl %r14, __gesf2
+  %cmp = fcmp oge float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_gedf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_gedf2:
+  ; CHECK:        brasl %r14, __gedf2
+  %cmp = fcmp oge double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_getf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_getf2:
+  ; CHECK:        brasl %r14, __getf2
+  %cmp = fcmp oge fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_ltsf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_ltsf2:
+  ; CHECK:        brasl %r14, __ltsf2
+  %cmp = fcmp olt float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_ltdf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_ltdf2:
+  ; CHECK:        brasl %r14, __ltdf2
+  %cmp = fcmp olt double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_lttf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_lttf2:
+  ; CHECK:        brasl %r14, __lttf2
+  %cmp = fcmp olt fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_lesf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_lesf2:
+  ; CHECK:        brasl %r14, __lesf2
+  %cmp = fcmp ole float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_ledf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_ledf2:
+  ; CHECK:        brasl %r14, __ledf2
+  %cmp = fcmp ole double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_letf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_letf2:
+  ; CHECK:        brasl %r14, __letf2
+  %cmp = fcmp ole fp128 %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_gtsf2(float %a, float %b) {
+  ; CHECK-LABEL:  test_gtsf2:
+  ; CHECK:        brasl %r14, __gtsf2
+  %cmp = fcmp ogt float %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_gtdf2(double %a, double %b) {
+  ; CHECK-LABEL:  test_gtdf2:
+  ; CHECK:        brasl %r14, __gtdf2
+  %cmp = fcmp ogt double %a, %b
+  ret i1 %cmp
+}
+
+define i1 @test_gttf2(fp128 %a, fp128 %b) {
+  ; CHECK-LABEL:  test_gttf2:
+  ; CHECK:        brasl %r14, __gttf2
+  %cmp = fcmp ogt fp128 %a, %b
+  ret i1 %cmp
+}

llvm/test/CodeGen/SystemZ/soft-float-02.ll

+; RUN: llc < %s -mtriple=s390x-linux-gnu -mattr=soft-float | FileCheck %s
+;
+; Check that FP registers are not saved in a vararg function if soft-float is
+; used.
+
+define void @fun0(...) {
+; CHECK-LABEL: fun0
+; CHECK-NOT: std %f0
+; CHECK-NOT: std %f2
+; CHECK-NOT: std %f4
+; CHECK-NOT: std %f6
+  ret void
+}
+
+

llvm/test/CodeGen/SystemZ/soft-float-03.ll

+; RUN: llc -mcpu=z13 -mattr=soft-float -O3 < %s | FileCheck %s
+;
+; Check that soft-float implies "-vector".
+
+define <2 x i64> @f0(<2 x i64> %dummy, <2 x i64> %val1, <2 x i64> %val2) {
+; CHECK-LABEL: f0:
+; CHECK-NOT: vag
+; CHECK-NOT: %v
+  %res = add <2 x i64> %val1, %val2
+  ret <2 x i64> %res
+}

llvm/test/CodeGen/SystemZ/soft-float-04.ll

+; RUN: llc -mcpu=z14 -O3 -mattr=soft-float < %s  | FileCheck %s
+;
+; Check that this function with soft-float does not result in a s390.tdc
+; intrinsic (which cannot be handled by SoftenFloatOperand).
+
+define void @fun(float %arg) {
+; CHECK-LABEL: fun:
+; CHECK: cijl
+bb:
+  %tmp = bitcast float %arg to i32
+  br label %bb1
+
+bb1:                                              ; preds = %bb
+  %tmp2 = icmp sgt i32 %tmp, -1
+  br i1 %tmp2, label %bb3, label %bb4
+
+bb3:                                              ; preds = %bb1
+  unreachable
+
+bb4:                                              ; preds = %bb1
+  unreachable
+}

llvm/test/CodeGen/SystemZ/soft-float-args.ll

+; RUN: llc -mcpu=z13 -mattr=soft-float -O3 < %s | FileCheck %s
+;
+; Test that arguments and return values of fp/vector types are always handled
+; with gprs with soft-float.
+
+define double @f1(double %arg) {
+; CHECK-LABEL: f1:
+; CHECK-NOT: %r2
+; CHECK-NOT: %{{[fv]}}
+; CHECK: llihh %r3, 16368
+; CHECK-NEXT: brasl %r14, __adddf3@PLT
+; CHECK-NEXT: lmg   %r14, %r15, 272(%r15)
+; CHECK-NEXT: br    %r14
+  %res = fadd double %arg, 1.0
+  ret double %res
+}
+
+define float @f2(float %arg) {
+; CHECK-LABEL: f2:
+; CHECK-NOT: %r2
+; CHECK-NOT: %{{[fv]}}
+; CHECK: llgfr   %r2, %r2
+; CHECK-NEXT: llilh   %r3, 16256
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: # kill: def $r2l killed $r2l killed $r2d
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd float %arg, 1.0
+  ret float %res
+}
+
+define fp128 @f2_fp128(fp128 %arg) {
+; CHECK-LABEL: f2_fp128:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: aghi    %r15, -208
+; CHECK-NEXT: .cfi_def_cfa_offset 368
+; CHECK-NEXT: lg      %r0, 0(%r2)
+; CHECK-NEXT: lg      %r1, 8(%r2)
+; CHECK-NEXT: llihf   %r2, 1073823744
+; CHECK-NEXT: stg     %r2, 160(%r15)
+; CHECK-NEXT: la      %r2, 192(%r15)
+; CHECK-NEXT: la      %r3, 176(%r15)
+; CHECK-NEXT: la      %r4, 160(%r15)
+; CHECK-NEXT: stg     %r1, 184(%r15)
+; CHECK-NEXT: stg     %r0, 176(%r15)
+; CHECK-NEXT: mvghi   168(%r15), 0
+; CHECK-NEXT: brasl   %r14, __addtf3@PLT
+; CHECK-NEXT: lg      %r2, 192(%r15)
+; CHECK-NEXT: lg      %r3, 200(%r15)
+; CHECK-NEXT: lmg     %r14, %r15, 320(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd fp128 %arg, 0xL00000000000000004001400000000000
+  ret fp128 %res
+}
+
+define <2 x double> @f3(<2 x double> %arg) {
+; CHECK-LABEL: f3:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: lg      %r13, 8(%r2)
+; CHECK-NEXT: lg      %r2, 0(%r2)
+; CHECK-NEXT: llihh   %r3, 16368
+; CHECK-NEXT: brasl   %r14, __adddf3@PLT
+; CHECK-NEXT: lgr     %r12, %r2
+; CHECK-NEXT: lgr     %r2, %r13
+; CHECK-NEXT: llihh   %r3, 16368
+; CHECK-NEXT: brasl   %r14, __adddf3@PLT
+; CHECK-NEXT: lgr     %r3, %r2
+; CHECK-NEXT: lgr     %r2, %r12
+; CHECK-NEXT: lmg     %r12, %r15, 256(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd <2 x double> %arg, <double 1.000000e+00, double 1.000000e+00>
+  ret <2 x double> %res
+}
+
+define <2 x float> @f4(<2 x float> %arg) {
+; CHECK-LABEL: f4:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: lr      %r13, %r3
+; CHECK-NEXT: llgfr   %r2, %r2
+; CHECK-NEXT: llilh   %r3, 16256
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: lgr     %r12, %r2
+; CHECK-NEXT: llgfr   %r2, %r13
+; CHECK-NEXT: llilh   %r3, 16256
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: lgr     %r3, %r2
+; CHECK-NEXT: lr      %r2, %r12
+; CHECK-NEXT: # kill: def $r3l killed $r3l killed $r3d
+; CHECK-NEXT: lmg     %r12, %r15, 256(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd <2 x float> %arg, <float 1.000000e+00, float 1.000000e+00>
+  ret <2 x float> %res
+}
+
+define <2 x i64> @f5(<2 x i64> %arg) {
+; CHECK-LABEL: f5:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: lghi    %r0, 1
+; CHECK-NEXT: ag      %r0, 0(%r2)
+; CHECK-NEXT: lghi    %r3, 1
+; CHECK-NEXT: ag      %r3, 8(%r2)
+; CHECK-NEXT: lgr     %r2, %r0
+; CHECK-NEXT: br      %r14
+  %res = add <2 x i64> %arg, <i64 1, i64 1>
+  ret <2 x i64> %res
+}
+
+define <2 x i32> @f6(<2 x i32> %arg) {
+; CHECK-LABEL: f6:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: ahi     %r2, 1
+; CHECK-NEXT: ahi     %r3, 1
+; CHECK-NEXT: br      %r14
+  %res = add <2 x i32> %arg, <i32 1, i32 1>
+  ret <2 x i32> %res
+}
+
+;; Stack arguments
+
+define double @f7(double %A, double %B, double %C, double %D, double %E,
+                  double %F) {
+; CHECK-LABEL: f7:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: aghi    %r15, -160
+; CHECK-NEXT: .cfi_def_cfa_offset 320
+; CHECK-NEXT: lg      %r3, 320(%r15)
+; CHECK-NEXT: brasl   %r14, __adddf3@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+
+  %res = fadd double %A, %F
+  ret double %res
+}
+
+define float @f8(float %A, float %B, float %C, float %D, float %E,
+                 float %F) {
+; CHECK-LABEL: f8:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: aghi    %r15, -160
+; CHECK-NEXT: .cfi_def_cfa_offset 320
+; CHECK-NEXT: llgf    %r3, 324(%r15)
+; CHECK-NEXT: llgfr   %r2, %r2
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: # kill: def $r2l killed $r2l killed $r2d
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd float %A, %F
+  ret float %res
+}
+
+define <2 x double> @f9(<2 x double> %A, <2 x double> %B, <2 x double> %C,
+                        <2 x double> %D, <2 x double> %E, <2 x double> %F,
+                        <2 x double> %G, <2 x double> %H, <2 x double> %I) {
+; CHECK-LABEL: f9:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: aghi    %r15, -160
+; CHECK-NEXT: .cfi_def_cfa_offset 320
+; CHECK-NEXT: lg      %r1, 344(%r15)
+; CHECK-NEXT: lg      %r13, 8(%r2)
+; CHECK-NEXT: lg      %r2, 0(%r2)
+; CHECK-NEXT: lg      %r3, 0(%r1)
+; CHECK-NEXT: lg      %r12, 8(%r1)
+; CHECK-NEXT: brasl   %r14, __adddf3@PLT
+; CHECK-NEXT: lgr     %r11, %r2
+; CHECK-NEXT: lgr     %r2, %r13
+; CHECK-NEXT: lgr     %r3, %r12
+; CHECK-NEXT: brasl   %r14, __adddf3@PLT
+; CHECK-NEXT: lgr     %r3, %r2
+; CHECK-NEXT: lgr     %r2, %r11
+; CHECK-NEXT: lmg     %r11, %r15, 248(%r15)
+; CHECK-NEXT: br      %r14
+  %res = fadd <2 x double> %A, %I
+  ret <2 x double> %res
+}
+
+define <2 x float> @f10(<2 x float> %A, <2 x float> %B, <2 x float> %C,
+                        <2 x float> %D, <2 x float> %E, <2 x float> %F,
+                        <2 x float> %G, <2 x float> %H, <2 x float> %I) {
+; CHECK-LABEL: f10:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: aghi    %r15, -160
+; CHECK-NEXT: .cfi_def_cfa_offset 320
+; CHECK-NEXT: lr      %r13, %r3
+; CHECK-NEXT: llgf    %r3, 412(%r15)
+; CHECK-NEXT: llgf    %r12, 420(%r15)
+; CHECK-NEXT: llgfr   %r2, %r2
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: lgr     %r11, %r2
+; CHECK-NEXT: llgfr   %r2, %r13
+; CHECK-NEXT: lgr     %r3, %r12
+; CHECK-NEXT: brasl   %r14, __addsf3@PLT
+; CHECK-NEXT: lgr     %r3, %r2
+; CHECK-NEXT: lr      %r2, %r11
+; CHECK-NEXT: # kill: def $r3l killed $r3l killed $r3d
+; CHECK-NEXT: lmg     %r11, %r15, 248(%r15)
+; CHECK-NEXT: br      %r14
+
+  %res = fadd <2 x float> %A, %I
+  ret <2 x float> %res
+}
+
+define <2 x i64> @f11(<2 x i64> %A, <2 x i64> %B, <2 x i64> %C,
+                      <2 x i64> %D, <2 x i64> %E, <2 x i64> %F,
+                      <2 x i64> %G, <2 x i64> %H, <2 x i64> %I) {
+; CHECK-LABEL: f11:
+; CHECK-NOT: %{{[fv]}}
+; CHECK: lg      %r1, 184(%r15)
+; CHECK-NEXT: lg      %r3, 8(%r2)
+; CHECK-NEXT: lg      %r2, 0(%r2)
+; CHECK-NEXT: ag      %r2, 0(%r1)
+; CHECK-NEXT: ag      %r3, 8(%r1)
+; CHECK-NEXT: br      %r14
+  %res = add <2 x i64> %A, %I
+  ret <2 x i64> %res
+}
+
+;; calls
+
+declare double @bar_double(double %arg);
+define double @f12(double %arg, double %arg2) {
+; CHECK-LABEL: f12:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r{{[23]}}
+; CHECK: lgr     %r2, %r3
+; CHECK-NEXT: brasl   %r14, bar_double@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call double @bar_double(double %arg2)
+  ret double %res
+}
+
+declare float @bar_float(float %arg);
+define float @f13(float %arg, float %arg2) {
+; CHECK-LABEL: f13:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r{{[23]}}
+; CHECK: lr     %r2, %r3
+; CHECK-NEXT: brasl   %r14, bar_float@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call float @bar_float(float %arg2)
+  ret float %res
+}
+
+declare fp128 @bar_fp128(fp128 %arg);
+define fp128 @f14(fp128 %arg, fp128 %arg2) {
+; CHECK-LABEL: f14:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r3
+; CHECK: lg      %r0, 0(%r3)
+; CHECK-NEXT: lg      %r1, 8(%r3)
+; CHECK-NEXT: la      %r2, 160(%r15)
+; CHECK-NEXT: stg     %r1, 168(%r15)
+; CHECK-NEXT: stg     %r0, 160(%r15)
+; CHECK-NEXT: brasl   %r14, bar_fp128@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 288(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call fp128 @bar_fp128(fp128 %arg2)
+  ret fp128 %res
+}
+
+declare <2 x double> @bar_v2f64(<2 x double> %arg);
+define <2 x double> @f15(<2 x double> %arg, <2 x double> %arg2) {
+; CHECK-LABEL: f15:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r3
+; CHECK: lg      %r0, 0(%r3)
+; CHECK-NEXT: lg      %r1, 8(%r3)
+; CHECK-NEXT: la      %r2, 160(%r15)
+; CHECK-NEXT: stg     %r1, 168(%r15)
+; CHECK-NEXT: stg     %r0, 160(%r15)
+; CHECK-NEXT: brasl   %r14, bar_v2f64@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 288(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call <2 x double> @bar_v2f64(<2 x double> %arg2)
+  ret <2 x double> %res
+}
+
+declare <2 x float> @bar_v2f32(<2 x float> %arg);
+define <2 x float> @f16(<2 x float> %arg, <2 x float> %arg2) {
+; CHECK-LABEL: f16:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r{{[2345]}}
+; CHECK: lr      %r3, %r5
+; CHECK-NEXT: lr      %r2, %r4
+; CHECK-NEXT: brasl   %r14, bar_v2f32@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 272(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call <2 x float> @bar_v2f32(<2 x float> %arg2)
+  ret <2 x float> %res
+}
+
+declare <2 x i64> @bar_v2i64(<2 x i64> %arg);
+define <2 x i64> @f17(<2 x i64> %arg, <2 x i64> %arg2) {
+; CHECK-LABEL: f17:
+; CHECK-NOT: %{{[fv]}}
+; CHECK-NOT: %r3
+; CHECK: lg      %r0, 0(%r3)
+; CHECK-NEXT: lg      %r1, 8(%r3)
+; CHECK-NEXT: la      %r2, 160(%r15)
+; CHECK-NEXT: stg     %r1, 168(%r15)
+; CHECK-NEXT: stg     %r0, 160(%r15)
+; CHECK-NEXT: brasl   %r14, bar_v2i64@PLT
+; CHECK-NEXT: lmg     %r14, %r15, 288(%r15)
+; CHECK-NEXT: br      %r14
+  %res = call <2 x i64> @bar_v2i64(<2 x i64> %arg2)
+  ret <2 x i64> %res
+}

llvm/test/CodeGen/SystemZ/soft-float-inline-asm-01.ll

+; RUN: not llc < %s -mcpu=z13 -mattr=soft-float -O3 2>&1 | FileCheck %s
+;
+; Verify that inline asms cannot use fp/vector registers with soft-float.
+
+define float @f1() {
+  %ret = call float asm "", "=f" ()
+  ret float %ret
+}
+
+; CHECK: error: couldn't allocate output register for constraint 'f'

llvm/test/CodeGen/SystemZ/soft-float-inline-asm-02.ll

+; RUN: not llc < %s -mcpu=z13 -mattr=soft-float -O3 2>&1 | FileCheck %s
+;
+; Verify that inline asms cannot use fp/vector registers with soft-float.
+
+define float @f1() {
+  %ret = call float asm "", "={f0}" ()
+  ret float %ret
+}
+
+; CHECK: error: couldn't allocate output register for constraint '{f0}'

llvm/test/CodeGen/SystemZ/soft-float-inline-asm-03.ll

+; RUN: not llc < %s -mcpu=z13 -mattr=soft-float -O3 2>&1 | FileCheck %s
+;
+; Verify that inline asms cannot use fp/vector registers with soft-float.
+
+define <2 x i64> @f1() {
+  %ret = call <2 x i64> asm "", "=v" ()
+  ret <2 x i64> %ret
+}
+
+; CHECK: error: couldn't allocate output register for constraint 'v'

llvm/test/CodeGen/SystemZ/vec-abi-align.ll

 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mattr=vector,-vector | \
 ; RUN:   FileCheck -check-prefix=CHECK-NOVECTOR %s
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mattr=+vector,-vector | \
 ; RUN:   FileCheck -check-prefix=CHECK-NOVECTOR %s
 
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 -mattr=-vector | \
 ; RUN:   FileCheck -check-prefix=CHECK-NOVECTOR %s
 
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 -mattr=+soft-float | \
+; RUN:   FileCheck -check-prefix=CHECK-NOVECTOR %s
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 \
+; RUN:   -mattr=soft-float,-soft-float | \
+; RUN:   FileCheck -check-prefix=CHECK-VECTOR %s
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 \
+; RUN:   -mattr=-soft-float,soft-float | \
+; RUN:   FileCheck -check-prefix=CHECK-NOVECTOR %s
+
 %struct.S = type { i8, <2 x i64> }
 
 define void @test(%struct.S* %s) nounwind {
 ; CHECK-VECTOR-LABEL: @test
 ; CHECK-VECTOR: vl %v0, 8(%r2)
 ; CHECK-NOVECTOR-LABEL: @test
 ; CHECK-NOVECTOR-DAG: agsi 16(%r2), 1
 ; CHECK-NOVECTOR-DAG: agsi 24(%r2), 1
   %ptr = getelementptr %struct.S, %struct.S* %s, i64 0, i32 1
   %vec = load <2 x i64>, <2 x i64>* %ptr
   %add = add <2 x i64> %vec, <i64 1, i64 1>
   store <2 x i64> %add, <2 x i64>* %ptr
   ret void
 }
 

llvm/test/CodeGen/SystemZ/vec-args-06.ll

 ; Test multiple return values (LLVM ABI extension)
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 | FileCheck %s
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 -mattr=soft-float \
+; RUN:   | FileCheck %s --check-prefix=SOFT-FLOAT
 
 ; Up to eight vector return values fit into VRs.
 define { <2 x double>, <2 x double>, <2 x double>, <2 x double>,
          <2 x double>, <2 x double>, <2 x double>, <2 x double> } @f1() {
 ; CHECK-LABEL: f1:
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v24, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v26, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v28, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v30, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v25, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v27, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v29, 0([[TMP]])
 ; CHECK: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl %v31, 0([[TMP]])
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f1:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: llihf
+; SOFT-FLOAT-NEXT: oilf
+; SOFT-FLOAT-NEXT: stg
+; SOFT-FLOAT-NEXT: llihh
+; SOFT-FLOAT-NEXT: stg
+; SOFT-FLOAT-NEXT: llihf
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: br      %r14
   ret { <2 x double>, <2 x double>, <2 x double>, <2 x double>,
         <2 x double>, <2 x double>, <2 x double>, <2 x double> }
       { <2 x double> <double 1.0, double 1.1>,
         <2 x double> <double 2.0, double 2.1>,
         <2 x double> <double 3.0, double 3.1>,
         <2 x double> <double 4.0, double 4.1>,
         <2 x double> <double 5.0, double 5.1>,
         <2 x double> <double 6.0, double 6.1>,

 ; CHECK-DAG: vst [[VTMP]], 32(%r2)
 ; CHECK-DAG: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl [[VTMP:%v[0-9]+]], 0([[TMP]])
 ; CHECK-DAG: vst [[VTMP]], 16(%r2)
 ; CHECK-DAG: larl [[TMP:%r[0-5]]], .LCPI
 ; CHECK: vl [[VTMP:%v[0-9]+]], 0([[TMP]])
 ; CHECK: vst [[VTMP]], 0(%r2)
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f2:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: llihf
+; SOFT-FLOAT-NEXT: oilf
+; SOFT-FLOAT-NEXT: stg
+; SOFT-FLOAT-NEXT: llihh
+; SOFT-FLOAT-NEXT: stg
+; SOFT-FLOAT-NEXT: llihf
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: br      %r14
   ret { <2 x double>, <2 x double>, <2 x double>, <2 x double>,
         <2 x double>, <2 x double>, <2 x double>, <2 x double>,
         <2 x double> }
       { <2 x double> <double 1.0, double 1.1>,
         <2 x double> <double 2.0, double 2.1>,
         <2 x double> <double 3.0, double 3.1>,
         <2 x double> <double 4.0, double 4.1>,
         <2 x double> <double 5.0, double 5.1>,
         <2 x double> <double 6.0, double 6.1>,
         <2 x double> <double 7.0, double 7.1>,
         <2 x double> <double 8.0, double 8.1>,
         <2 x double> <double 9.0, double 9.1> }
 }

llvm/test/CodeGen/SystemZ/vec-args-07.ll

 ; Test calling functions with multiple return values (LLVM ABI extension)
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 | FileCheck %s
+; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 -mattr=soft-float \
+; RUN:   | FileCheck %s --check-prefix=SOFT-FLOAT
 
 ; Up to eight vector return values fit into VRs.
 declare { <2 x double>, <2 x double>, <2 x double>, <2 x double>,
           <2 x double>, <2 x double>, <2 x double>, <2 x double> } @bar1()
 
 define <2 x double> @f1() {
 ; CHECK-LABEL: f1:
 ; CHECK: brasl %r14, bar1
 ; CHECK: vlr %v24, %v31
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f1:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: brasl   %r14, bar1
+; SOFT-FLOAT-NEXT: lg      %r3, 280(%r15)
+; SOFT-FLOAT-NEXT: lg      %r2, 272(%r15)
+; SOFT-FLOAT-NEXT: lmg     %r14, %r15, 400(%r15)
+; SOFT-FLOAT-NEXT: br      %r14
   %mret = call { <2 x double>, <2 x double>,
                  <2 x double>, <2 x double>,
                  <2 x double>, <2 x double>,
                  <2 x double>, <2 x double> } @bar1()
   %ret = extractvalue { <2 x double>, <2 x double>,
                         <2 x double>, <2 x double>,
                         <2 x double>, <2 x double>,
                         <2 x double>, <2 x double> } %mret, 7
   ret <2 x double> %ret
 }
 
 ; More than eight vector return values use sret.
 declare { <2 x double>, <2 x double>, <2 x double>, <2 x double>,
           <2 x double>, <2 x double>, <2 x double>, <2 x double>,
           <2 x double> } @bar2()
 
 define <2 x double> @f2() {
 ; CHECK-LABEL: f2:
 ; CHECK: la %r2, 160(%r15)
 ; CHECK: brasl %r14, bar2
 ; CHECK: vl  %v24, 288(%r15)
 ; CHECK: br %r14
+
+; SOFT-FLOAT-LABEL: f2:
+; SOFT-FLOAT-NOT: %{{[fv]}}
+; SOFT-FLOAT: brasl   %r14, bar2
+; SOFT-FLOAT-NEXT: lg      %r3, 296(%r15)
+; SOFT-FLOAT-NEXT: lg      %r2, 288(%r15)
+; SOFT-FLOAT-NEXT: lmg     %r14, %r15, 416(%r15)
+; SOFT-FLOAT-NEXT: br      %r14
   %mret = call { <2 x double>, <2 x double>,
                  <2 x double>, <2 x double>,
                  <2 x double>, <2 x double>,
                  <2 x double>, <2 x double>,
                  <2 x double> } @bar2()
   %ret = extractvalue { <2 x double>, <2 x double>,
                         <2 x double>, <2 x double>,
                         <2 x double>, <2 x double>,
                         <2 x double>, <2 x double>,
                         <2 x double> } %mret, 8
   ret <2 x double> %ret
 }