This is an archive of the discontinued LLVM Phabricator instance.

[ASTMatchers] Add support for floatLiterals
ClosedPublic

Authored by Lekensteyn on May 12 2017, 9:24 AM.

Details

Summary

Needed to support something like "floatLiteral(equals(1.0))". The
parser for floating point numbers is kept simple, so instead of ".1" you
have to use "0.1".

Diff Detail

Repository
rL LLVM

Event Timeline

Lekensteyn created this revision.May 12 2017, 9:24 AM
aaron.ballman added inline comments.May 12 2017, 10:08 AM
include/clang/ASTMatchers/Dynamic/Parser.h
25 ↗(On Diff #98784)

It'd be good to list the actual grammar rather than a few examples.

include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

This may or may not be a good idea, but do we want to put the values into an APFloat rather than a double? My concern with double is that (0) it may be subtly different if the user wants a 16- or 32-bit float explicitly, (1) it won't be able to represent long double values, or quad double.

I'm thinking this value could be passed directly from the C++ API as an APFloat, float, or double, or provided using a StringRef for the dynamic API.

lib/ASTMatchers/Dynamic/Parser.cpp
180 ↗(On Diff #98784)

This function should be renamed and the comment updated.

By the way, I think that long double is less common than long unsigned literals, so changing unsigned to uint64_t might be something more important?

include/clang/ASTMatchers/Dynamic/Parser.h
25 ↗(On Diff #98784)

I am concerned that listing a very precise grammar actually makes this less readable (see also the StringLiteral example).

If a grammar is used instead, how about this:

<Double> := [0-9]+.[0-9]* | [0-9]+.[0-9]*[eE][-+]?[0-9]+
include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

(32-bit) double values are a superset of (16-bit) float values, that should be OK.
Long doubles are possible in the AST (e.g. for 0.1L), but neither C11 nor C++14 seem to define a quad double literal type (so that should be of a lesser concern).

Reasons why I chose for double instead of APFloat:

  • strtod is readily available and does not abort the program. By contrast, APFloat(StringRef) trips on assertions if the input is invalid.
  • I was not sure if the APFloat class can be used in an union.
lib/ASTMatchers/Dynamic/Parser.cpp
180 ↗(On Diff #98784)

How does "consumeNumberLiteral" sound?

aaron.ballman edited edge metadata.May 21 2017, 7:48 AM

By the way, I think that long double is less common than long unsigned literals, so changing unsigned to uint64_t might be something more important?

I agree that it's likely a more common use case. There doesn't appear to be a suffix for __int128 (we talked about adding i128 once upon a time, but I don't believe it got in), so you may be okay using a uint64_t there rather than an APInt.

include/clang/ASTMatchers/Dynamic/Parser.h
25 ↗(On Diff #98784)

That's reasonable enough.

include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

The downside to using strtod() is that invalid input is silently accepted. However, assertions on invalid input is certainly not good either. It might be worth modifying APFloat::convertFromString() to accept invalid input and return an error.

I think instead of an APFloat, maybe using an APValue for both the Unsigned and Double fields might work. At the very least, it should give you implementation ideas.

There is a quad double literal suffix: q. It's only supported on some architectures, however. There are also imaginary numbers (i) and half (h).

lib/ASTMatchers/Dynamic/Parser.cpp
180 ↗(On Diff #98784)

Sounds good to me.

209 ↗(On Diff #98784)

You're failing to check errno here to ensure the value is actually valid.

Lekensteyn added inline comments.May 22 2017, 2:10 AM
include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

The strtod conversion was based on parseDouble in lib/Support/CommandLine.cpp, so any conversion issues also exist there.

Same question, can APFloat/APValue be used in a union?

float (or quad-double suffixes) are explicitly not supported now in this matcher, maybe they can be added later but for now I decided to keep the grammar simple (that is, do not express double/float data types via the literal).

lib/ASTMatchers/Dynamic/Parser.cpp
209 ↗(On Diff #98784)

will check this later, see also the previous comment

aaron.ballman added inline comments.May 22 2017, 5:15 AM
include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

The strtod conversion was based on parseDouble in lib/Support/CommandLine.cpp, so any conversion issues also exist there.

Good to know.

Same question, can APFloat/APValue be used in a union?

I believe so, but I've not tried it myself. Also, as I mentioned, APValue demonstrates another implementation strategy in case you cannot use a union directly.

float (or quad-double suffixes) are explicitly not supported now in this matcher, maybe they can be added later but for now I decided to keep the grammar simple (that is, do not express double/float data types via the literal).

That's reasonable for an initial implementation.

Lekensteyn updated this revision to Diff 101816.Jun 7 2017, 2:37 PM
Lekensteyn marked 8 inline comments as done.

diff from previous version:

diff --git a/include/clang/ASTMatchers/Dynamic/Parser.h b/include/clang/ASTMatchers/Dynamic/Parser.h
index 0d0c2ba540..5ec4a9abf4 100644
--- a/include/clang/ASTMatchers/Dynamic/Parser.h
+++ b/include/clang/ASTMatchers/Dynamic/Parser.h
@@ -22,7 +22,7 @@
 /// <Literal>           := <StringLiteral> | <Boolean> | <Double> | <Unsigned>
 /// <StringLiteral>     := "quoted string"
 /// <Boolean>           := true | false
-/// <Double>            := 1.0 | 2e-3 | 3.45e67
+/// <Double>            := [0-9]+.[0-9]* | [0-9]+.[0-9]*[eE][-+]?[0-9]+
 /// <Unsigned>          := [0-9]+
 /// <NamedValue>        := <Identifier>
 /// <MatcherExpression> := <Identifier>(<ArgumentList>) |
diff --git a/lib/ASTMatchers/Dynamic/Parser.cpp b/lib/ASTMatchers/Dynamic/Parser.cpp
index 669e5ca44f..ff5c5fb657 100644
--- a/lib/ASTMatchers/Dynamic/Parser.cpp
+++ b/lib/ASTMatchers/Dynamic/Parser.cpp
@@ -130,8 +130,8 @@ private:
 
     case '0': case '1': case '2': case '3': case '4':
     case '5': case '6': case '7': case '8': case '9':
-      // Parse an unsigned literal.
-      consumeUnsignedLiteral(&Result);
+      // Parse an unsigned and float literal.
+      consumeNumberLiteral(&Result);
       break;
 
     default:
@@ -176,8 +176,8 @@ private:
     return Result;
   }
 
-  /// \brief Consume an unsigned literal.
-  void consumeUnsignedLiteral(TokenInfo *Result) {
+  /// \brief Consume an unsigned and float literal.
+  void consumeNumberLiteral(TokenInfo *Result) {
     bool isFloatingLiteral = false;
     unsigned Length = 1;
     if (Code.size() > 1) {
@@ -205,8 +205,9 @@ private:
 
     if (isFloatingLiteral) {
       char *end;
+      errno = 0;
       double doubleValue = strtod(Result->Text.str().c_str(), &end);
-      if (*end == 0) {
+      if (*end == 0 && errno == 0) {
         Result->Kind = TokenInfo::TK_Literal;
         Result->Value = doubleValue;
         return;

Rebased patches on latest clang master (trunk), there were no changes in ASTMatchers.
boolean literal patch was unchanged, this floating literal patch was updated to address comments.

include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

I think I'll keep it like this for now and defer eventual conversion to APValue for a future patch that also makes uint64_t possible. Is that OK?

aaron.ballman accepted this revision.Jun 8 2017, 2:01 PM

LGTM!

include/clang/ASTMatchers/Dynamic/VariantValue.h
335 ↗(On Diff #98784)

Okay, that seems reasonable to me.

This revision is now accepted and ready to land.Jun 8 2017, 2:01 PM
This revision was automatically updated to reflect the committed changes.