This is an archive of the discontinued LLVM Phabricator instance.

Modify FPFeatures to use delta not absolute settings to solve PCH compatibility problems
ClosedPublic

Authored by mibintc on Jun 15 2020, 12:55 PM.

Details

Summary

To solve https://bugs.llvm.org/show_bug.cgi?id=46166 where the floating point settings in PCH files aren't compatible, rewrite FPFeatures to use a delta in the settings rather than absolute settings

Diff Detail

Event Timeline

mibintc created this revision.Jun 15 2020, 12:55 PM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptJun 15 2020, 12:55 PM
mibintc marked an inline comment as done.Jun 15 2020, 12:58 PM

@rjmccall You suggested that the FPFeatures could be delta instead of absolute settings, I think this is approximately what you mean. This is a rough patch employing that idea, would welcome your early comments. There are 2 clang tests that are failing with this patch, I need to dig into those.

Failed Tests (2):

Clang :: CodeGen/fp-floatcontrol-stack.cpp
Clang :: PCH/pragma-floatcontrol.c
llvm/include/llvm/ADT/FloatingPointMode.h
43 ↗(On Diff #270832)

I added "unset" enumerals to the 3 enumeration values. Note that I can't just use the "invalid" enumeration since the clang field is only 3 bits wide so Invalid would appear to equal Dynamic

mibintc updated this revision to Diff 271188.Jun 16 2020, 1:49 PM

This version passes all the lit tests, i believe it's functional tho' maybe not elegant.
I still need to add a test case that PCH behaves as desired, and that the floating point command line options from PCH create do not clobber command line options from compilations that use a pch

rjmccall added inline comments.Jun 16 2020, 1:52 PM
clang/include/clang/AST/Expr.h
2281

I would call this one getFPOptionOverrides or getOverriddenFPOptions. If you're going to have a getFPFeatures, it really ought to be the other one — although since we're changing pretty much all of the call sites, we might as well rename it to getFPOptions (or getFPOptionsInEffect, that seems fine, too).

clang/include/clang/Basic/LangOptions.h
769

As an alternative storage strategy, I would suggest:

  • in FPOptions, use an integer and masks instead of bit-fields
  • in FPOptionsOverride, store the same integer from FPOptions plus the active override mask

This will make the merge and extract operations really painless. It can get a little boilerplate, though, so I'd recommend also introducing an "x-macro" file:

// clang/Basic/FPOptions.def
// OPTION(name, type, width, previousName)
OPTION(FPContractMode, LangOptions::FPModeKind, 2, First)
OPTION(RoundingMode, RoundingMode, 3, FPContractMode)
...
OPTION(AllowReciprocal, bool, 1, NoSignedZeros)
...
#undef OPTION

...

class FPOptions {
public:
  // We start by defining the layout.
  using storage_type = uint16_t;

  // Define a fake option named "First" so that we have a PREVIOUS even for the real first option.
  static constexpr storage_type FirstShift = 0,  FirstWidth = 0;
#define OPTION(NAME, TYPE, WIDTH, PREVIOUS) \
  static constexpr storage_type NAME#Shift = PREVIOUS#Shift + PREVIOUS#Width; \
  static constexpr storage_type NAME#Width = width; \
  static constexpr storage_type NAME#Mask = ((1 << NAME#Width) - 1) << NAME#Shift;
#include "clang/Basic/FPOptions.def"

private:
  storage_type Value;

public:
  // Now you can define constructors and so on.
  FPOptions() : Value(0) {}

  bool operator==(FPOptions other) const {
    return Value == other.Value;
  }
  bool operator!=(FPOptionsOverride other) const {
    return Value != other.Value;
  }

  storage_type getAsOpaqueInt() const { return Value; }
  static FPOptions getFromOpaqueInt(storage_type value) {
     FPOptions result; 
     result.Value = value;
     return result;
  }

  // We can define most of the accessors automatically:
#define OPTION(NAME, TYPE, WIDTH, PREVIOUS) \
  TYPE get#NAME() const { \
    return TYPE((Value & NAME#Mask) >> NAME#Shift);  \
  } \
  void set#NAME(TYPE value) { \
    Value = (Value & ~NAME#Mask) | (storage_type(value) << NAME#Shift); \
  }
#include "clang/Basic/FPOptions.def"
};

// The override type is then basically just that, plus a mask about whether fields are actually set in it:
class FPOptionsOverride {
  FPOptions Options;
  FPOptions::storage_type OverrideMask = 0;

public:
  FPOptionsOverride() {}

  bool hasAnyOverrides() const {
    return OverrideMask != 0;
  }

  FPOptions applyOverrides(FPOptions base) {
    return FPOptions::getFromOpaqueInt(base.getAsOpaqueInt()
         | (OverrideMask & Options.getAsOpaqueInt()));
  }

  bool operator==(FPOptionsOverride other) const {
    return Options == other.Options && OverrideMask == other.OverrideMask;
  }
  bool operator!=(FPOptionsOverride other) const {
    return !(*this == other);
  }

#define OPTION(NAME, TYPE, WIDTH, PREVIOUS) \
  bool has#NAME#Override() const { \
    return OverrideMask & NAME#Mask; \
  } \
  TYPE get#NAME#Override() const { \
    assert(has#NAME#Override()); \
    return Options.get#NAME();
  } \
  void clear#NAME#Override() { \
    /* Clear the actual value so that we don't have spurious differences when testing equality. */ \
    Options.set#NAME(TYPE(0)); \
    OverrideMask &= ~NAME#Mask; \
  } \
  void set#NAME#Override(TYPE value) { \
    Options.set#NAME(value); \
    OverrideMask |= NAME#Mask; \
 }
#include "clang/Basic/FPOptions.def"
};
mibintc updated this revision to Diff 272536.Jun 22 2020, 1:39 PM

This revision rewrites FPOptionsOverride like @rjmccall suggests. There are a couple problems, i'll add inline comments in the trouble areas

mibintc marked 6 inline comments as done.Jun 22 2020, 2:08 PM
mibintc added a subscriber: yaxunl.

@rjmccall I added some inline questions and comments for you. Thanks a lot for your review.

clang/include/clang/AST/ExprCXX.h
170

It seems that this field is basically not used, if I eliminate all these member functions then the AST reader-writer fails but the information isn't being used in Sema or Codegen. I guess there's a bug somewhere if this information is actually needed when creating Float comparison codegen. Also,, from reading the comments in this file it sems that it is needed only when "rewrite == operator into OperatorCall as required by CXX20", the comments also say that the OperatorCall rewrite could equally have occurred as a rewrite into a BinaryOperator. If we choose to rewrite the == as BinaryOperator then it wouldn't be necessary to use TrailingStorage to hold FPOptionsOverride. The previously proposed revision didn't adding TrailingStorage on CXXOperatorCall. When I was working on a different patch I did study how to add TrailingStorage to a Call and I didn't know how I could accomplish that.

clang/include/clang/AST/Stmt.h
617

This is a temporary change, I know you don't want me to change the assert to allow a wider size than 8. So if this information is really needed (as I mentioned above it's not being used in Sema or Codegen) then alternatives are

  1. rewrite the == comparison using BinaryOperator
  2. put the FPFeatures Override into TrailingStorage (i think i will need implementation guidance if this is the path to go)
1140

this is a temporary change

clang/include/clang/Basic/LangOptions.def
196–201

Here's another problem, I added the test case from bug 46166 @yaxunl but that test case reports incompatibility because the pch is compiled with different settings than where it is used. I thought maybe based on the name, BENIGN_LANGOPT would allow different option values to not get the pch complaint but that didn't work, clang still complained about the different option values. Is there some existing way to get around the option complaint or do I need to invent a new field in the LANGOPT e.g. a boolean that says "Allow opton values to differ between pch-create and pch-use". I haven't deeply studied this to see if I'm missing something. I wanted to send this out to get comments on my other questions

clang/include/clang/Basic/LangOptions.h
423

Using #define OPTION trick would be preferrable except there is a compilation failure because something funny about RoundingMode--need the cast. doubtless there is fix for that

clang/lib/Parse/ParsePragma.cpp
657–658

Note: This patch is putting all the pragma's that modify floating point state into the FpPragmaStack, using the "Set" call to set the top value of the FP pragma stack. Of course the pragma's that affect the "fp contract" and "reassociate" etc. don't support stacking settings e.g. with PUSH and POP.

clang/test/SemaOpenCL/fp-options.cl
2

Here's the new test case from @yaxunl that I mentioned above. This test is failing. Besides that, check-all is passing all the lit tests

riccibruno added inline comments.
clang/include/clang/AST/Stmt.h
617

What do you mean by rewrite the == comparison using BinaryOperator? If you mean representing overloaded operator=='s by BinaryOperator, then that's certainly not right because BinaryOperators only represent binary builtin operators.

You will not be able to use TrailingObjects with CXXOperatorCallExpr because CallExpr stores its arguments after the object containing the CallExpr sub-object. You could manually stash it after CallExpr's arguments, but that would be pretty ugly.

Do you really need 32-bits worth of FP options? I see from FPOptions.def that you need 14 bits, so it should fit with room to spare.

1140

It better be :)

clang/include/clang/Basic/LangOptions.h
423

Yes it would be preferable and it should not be in a header too.

You need the cast because RoundingMode is a scoped enumeration, which does not have the implicit conversions of unscoped enumerations.

rjmccall added inline comments.Jun 22 2020, 8:14 PM
clang/include/clang/AST/Stmt.h
617

Bruno's close to right. The key question here is what node we use to represent dependent operators that might instantiate to a builtin floating-point operator, and the answer for binary operators (based on the code in Sema::CreateOverloadedBinOp) is that we use CXXOperatorCallExpr if lexical lookup found any matching operators (because we need some way to represent the operators, which BinaryOperator doesn't have) and otherwise we use BinaryOperator.

So, first, to create a test that depends on this, you need a dependent operator for which there is an overload in lexical scope, and then it needs to instantiate to just do a builtin operation, and this needs to be sensitive to the currently-active FP settings. So maybe something like:

struct Distance {};
Distance operator+(Distance, Distance);

template <class T> T add(T lhs, T rhs) {
#pragma STDC FENV_ACCESS on // or whatever pragma you like, doesn't have to be local, just has to override current settings and be different from the default
  return lhs + rhs;
}

float test() {
  return add(1.0f, 2.0f);
}

Second, how to represent this: while I think there would be some benefits from introducing a new Expr subclass that we always use to represent a dependent operator, I think it's fine (and certainly easier) to just stick with CXXOperatorCallExpr like we do now. Given that, you can either follow Bruno's advice and use fewer bits — which is probably a fairly short-term fix, since tracking almost any extra state would get us into trouble — or figure out some other way to store these.

Honestly, it's not the end of the world if CXXOperatorCallExpr just stores an FPOptionsOverride as an ordinary field — it just means we're not taking advantage of the fact that most contexts don't have overrides. Most instances of this operator are in dependent code where we can't know that we won't instantiate to a builtin operator, so there's no way to avoid storing the overrides if there are any.

If we do want to store FP features in trailing storage, then we're going to have to worry about the fact that CallExpr already has trailing storage. I think the right approach in this case is to have CallExpr generally account for the overrides in its trailing storage and then, dynamically, we know that that only actually happens with CXXOperatorCallExpr — all the other factories pass down empty overrides. One specific advantage of this is that it sets the stage for storing overrides on calls to builtins; we'd just need Sema to pass overrides down when it's building a builtin call. (We can rely on this even in templates because you can't invoke a builtin indirectly or by instantiation.)

clang/include/clang/Basic/LangOptions.h
423

Putting a cast to unsigned in the #define OPTION should be fine; if any of these fields outgrow that, we probably need to reconsider the representation.

mibintc updated this revision to Diff 272798.EditedJun 23 2020, 12:38 PM

I responded to review from @riccibruno and @rjmccall : I put the dump() routines into the .cpp file and changed them to use the x-macros. I moved CXXOperatorCall.FPOptionsOverride from the Expr bits into the OperatorCall itself. I added the operator call test case suggested by John. I must be mistaken about the bug, the operator call addition does account for the FPFeatures in both the unmodified compiler and the patched compiler.

The new test case is failing. Need to dig into that. I get this error:

error: no expected directives found: consider use of 'expected-no-diagnostics'
error: 'error' diagnostics seen but not expected:

(frontend): Include default header file for OpenCL was enabled in PCH file but is currently disabled

2 errors generated.

Clang :: SemaOpenCL/fp-options.cl

For the same test case, the unmodified compiler shows this,
error: no expected directives found: consider use of 'expected-no-diagnostics'
error: 'error' diagnostics seen but not expected:

(frontend): Permit Floating Point optimization without regard to signed zeros was disabled in PCH file but is currently enabled

2 errors generated

What else?

kpn added a subscriber: kpn.Jun 23 2020, 12:52 PM
mibintc updated this revision to Diff 272808.Jun 23 2020, 1:24 PM

The difference between this patch and the earlier one today is that I modified the new test case

I misunderstood why the test case was failing. The BENIGN_LANGOPT is working as desired and those pch diagnostics are no longer emitted. I didn't rerun but I believe all check-clang (and check-all) will now pass.

What else?

mibintc edited the summary of this revision. (Show Details)Jun 23 2020, 1:24 PM

I need to make another revision that makes a couple more of the fp options, like ffp-contract, "benign" and add a test case that demonstrates fp options don't carry over from pch-create to pch-use

Just a bunch of minor suggestions. LGTM if you get all the tests worked out and it actually works the way you want on the tests.

clang/include/clang/AST/Expr.h
2281

I don't think there's any reason for this to take a LangOptions, since the set of overrides should be independent of the global settings.

clang/include/clang/Basic/LangOptions.h
508

clang-format's suggestion is good here

531

Same here: clang-format's just asking you to right-justify the escapes, and it's right, that's the style we usually use with multi-line macros.

clang/include/clang/Sema/Sema.h
569

Spurious spaces (as pointed out by clang-format)

clang/include/clang/Serialization/ASTWriter.h
510

I think it's telling you that you're missing a forward-declaration of this type in this file.

cfang added a subscriber: cfang.Jun 24 2020, 9:09 AM
mibintc updated this revision to Diff 273133.Jun 24 2020, 1:01 PM

I decided that I shouldn't make float options that define a macro, like -ffast-math, as BENIGN_LANGOPT, I made ffp-contract= , fp-exception-behavior and rounding-mode BENIGN, I modified the pch test case to test that the benign command line floating options on the pch-create don't affect compilations that use the pch file.

@rjmccall thought this patch looked OK. I'll commit tomorrow unless I receive more change requests. Thanks a lot!

BTW I'll be away from my desk on Intel Sabbatical starting June 27 and returning Aug 31. I expect I'll be checking in here and there but I won't be able to tackle any major problems.

I decided that I shouldn't make float options that define a macro, like -ffast-math, as BENIGN_LANGOPT, I made ffp-contract= , fp-exception-behavior and rounding-mode BENIGN

That seems right to me. @yaxunl, is this going to be okay with you?

Would you please add the following lit test

If you change FastMath, FiniteMathOnly and UnsafeFPMath to COMPATIBLE_LANGOPT, the test should pass.

clang/include/clang/Basic/LangOptions.def
196–201

can you make FastMath, FiniteMathOnly and UnsafeFPMath COMPATIBLE_LANGOPT as before? This allows users to use -fno-validate-pch to relax the check on them. This is the old behavior and I do not see a reason to change that.

Would you please add the following lit test

If you change FastMath, FiniteMathOnly and UnsafeFPMath to COMPATIBLE_LANGOPT, the test should pass.

Yes I'll do that, thanks for your review!

mibintc updated this revision to Diff 273729.Jun 26 2020, 7:35 AM

with updates from @yaxunl , i'm planning to push this

This revision was not accepted when it landed; it landed in state Needs Review.Jun 26 2020, 8:11 AM
This revision was automatically updated to reflect the committed changes.