[TableGen] Add true and false literals to represent booleans

[Paul-C-Anagnostopoulos]

Summary

This revision adds the 'true' and 'false' literals to TableGen, which represent boolean values.

This is primarily syntactic sugar aimed at improving the readability of TableGen files. Once parsed, these literals are converted to 1 and 0. Note that they can be used in bit sequences, too.

The documentation was updated. A test was added and a couple of other tests updated due to an improved error message.

[Paul-C-Anagnostopoulos]

llvm/lib/TableGen/TGLexer.cpp

344

I will remove this old code.

387

I will remove this old code.

[Paul-C-Anagnostopoulos]

I cleaned out the obsolete code lines.

I updated one more test to use the new literals.

[nhaehnle]

LGTM

llvm/lib/TableGen/TGParser.cpp

2247

Unrelated change.

[jrtc27]

Should these not be bits<1> instead? C/C++'s lax rules around integer promotion aren't generally something modern languages replicate.

[Paul-C-Anagnostopoulos]

Sorry, I'm not sure what you mean by "these." If you mean 'true' and 'false', they are simply named literals for 1 and 0. So they can be used in any context where an integer is allowed. TableGen already promotes bit and bit<n> to int and vice versa automatically.

A field type of bit is used for boolean fields. It is also used for single-bit fields where the individual bits feature of bits<n> is not required.

I contemplated inventing a boolean type, but quickly gave up on that idea.

[jrtc27]

In D90635#2377143, @Paul-C-Anagnostopoulos wrote:

Sorry, I'm not sure what you mean by "these." If you mean 'true' and 'false', they are simply named literals for 1 and 0. So they can be used in any context where an integer is allowed.

A field type of 'bit' is used for boolean fields. It is also used for single-bit fields where the individual bits feature of bits<n> is not required.

I contemplated inventing a 'boolean' type, but quickly gave up on that idea.

Yes, I meant the literals. It's less than ideal IMO that bits<3> = true is valid; I personally think that should be a type error.

[Paul-C-Anagnostopoulos]

I would have to invent a boolean type to prevent that assignment. That would be such a box of frogs that I quickly gave up on the idea. I'm not even sure we could do it compatibly. What would !eq(x, y) produce? And I don't think people would want !if(pred, ...) to require a boolean, since it doesn't in C++. [Much to my chagrin, and perhaps yours.]

Now, I might be able to make a special case out of field and let assignments. But do we want true/false suddenly to become "sort of a typed literal"?

My heart is with you, but I'm not sure we can do it.

[jrtc27]

In D90635#2377186, @Paul-C-Anagnostopoulos wrote:

I would have to invent a boolean type to prevent that assignment. That would be such a box of frogs that I quickly gave up on the idea. I'm not even sure we could do it compatibly. What would !eq(x, y) produce? And I don't think people would want !if(pred, ...) to require a boolean, since it doesn't in C++. [Much to my chagrin, and perhaps yours.]

Now, I might be able to make a special case out of field and let assignments. But do we want true/false suddenly to become "sort of a typed literal"?

My heart is with you, but I'm not sure we could do it.

Hm, if those already produce an integer rather than bit/bits<1> then yeah it seems our hands are tied here without disruptive changes.

[Paul-C-Anagnostopoulos]

Yes, I think it would be quite disruptive. But a strong stylistic suggestion in the documentation is in order.

Note that I have already modified one TableGen file to this:

bits<3> foo = {true, false, true};

Because the bits field is actually treated as three booleans. Consider status registers and predicate registers, too.

llvm/docs/TableGen/ProgRef.rst

.. productionlist::
    ualpha: "a"..."z" | "A"..."Z" | "_"
    TokIdentifier: ("0"..."9")* `ualpha` (`ualpha` | "0"..."9")*
    TokVarName: "$" `ualpha` (`ualpha` |  "0"..."9")*
 
 Note that, unlike most languages, TableGen allows :token:`TokIdentifier` to
 begin with an integer. In case of ambiguity, a token is interpreted as a
 numeric literal rather than an identifier.
 
-TableGen has the following reserved words, which cannot be used as
+TableGen has the following reserved keywords, which cannot be used as
 identifiers::
 
    bit        bits          class         code          dag
-   def        else          foreach       defm          defset
-   defvar     field         if            in            include
-   int        let           list          multiclass    string
-   then
+   def        else          false         foreach       defm
+   defset     defvar        field         if            in
+   include    int           let           list          multiclass
+   string     then          true
 
 .. warning::
   The ``field`` reserved word is deprecated.
 
 Bang operators
 --------------
 
 TableGen provides "bang operators" that have a wide variety of uses:
.. productionlist::
    SimpleValue: `TokInteger` | `TokString`+ | `TokCodeFragment`
 
 A value can be an integer literal, a string literal, or a code fragment
 literal. Multiple adjacent string literals are concatenated as in C/C++; the
 simple value is the concatenation of the strings. Code fragments become
 strings and then are indistinguishable from them.
 
 .. productionlist::
-   SimpleValue2: "?"
+   SimpleValue2: "true" | "false"
+
+The ``true`` and ``false`` literals are essentially syntactic sugar for the
+integer values 1 and 0. They improve the readability of TableGen files when
+boolean values are used in field values, bit sequences, ``if`` statements.
+etc. When parsed, these literals are converted to integers.
+
+.. productionlist::
+   SimpleValue3: "?"
 
 A question mark represents an uninitialized value.
 
 .. productionlist::
-   SimpleValue3: "{" [`ValueList`] "}"
+   SimpleValue4: "{" [`ValueList`] "}"
    ValueList: `ValueListNE`
    ValueListNE: `Value` ("," `Value`)*
 
 This value represents a sequence of bits, which can be used to initialize a
 ``bits<``\ *n*\ ``>`` field (note the braces). When doing so, the values
 must represent a total of *n* bits.
 
 .. productionlist::
-   SimpleValue4: "[" `ValueList` "]" ["<" `Type` ">"]
+   SimpleValue5: "[" `ValueList` "]" ["<" `Type` ">"]
 
 This value is a list initializer (note the brackets). The values in brackets
 are the elements of the list. The optional :token:`Type` can be used to
 indicate a specific element type; otherwise the element type is inferred
 from the given values. TableGen can usually infer the type, although
 sometimes not when the value is the empty list (``[]``).
 
 .. productionlist::
-   SimpleValue5: "(" `DagArg` [`DagArgList`] ")"
+   SimpleValue6: "(" `DagArg` [`DagArgList`] ")"
    DagArgList: `DagArg` ("," `DagArg`)*
    DagArg: `Value` [":" `TokVarName`] | `TokVarName`
 
 This represents a DAG initializer (note the parentheses).  The first
 :token:`DagArg` is called the "operator" of the DAG and must be a record.
 See `Directed acyclic graphs (DAGs)`_ for more details.
 
 .. productionlist::
-   SimpleValue6: `TokIdentifier`
+   SimpleValue7: `TokIdentifier`
 
 The resulting value is the value of the entity named by the identifier. The
 possible identifiers are described here, but the descriptions will make more
 sense after reading the remainder of this guide.
 
 .. The code for this is exceptionally abstruse. These examples are a
    best-effort attempt.
 

 * A variable defined with the ``defvar`` or ``defset`` statements.
 
 * The iteration variable of a ``foreach``, such as the use of ``i`` in::
 
      foreach i = 0...5 in
        def Foo#i;
 
 .. productionlist::
-   SimpleValue7: `ClassID` "<" `ValueListNE` ">"
+   SimpleValue8: `ClassID` "<" `ValueListNE` ">"
 
 This form creates a new anonymous record definition (as would be created by an
 unnamed ``def`` inheriting from the given class with the given template
 arguments; see `def`_) and the value is that record. A field of the record can be
 obtained using a suffix; see `Suffixed Values`_.
 
 Invoking a class in this manner can provide a simple subroutine facility.
 See `Using Classes as Subroutines`_ for more information.
 
 .. productionlist::
-   SimpleValue8: `BangOperator` ["<" `Type` ">"] "(" `ValueListNE` ")"
+   SimpleValue9: `BangOperator` ["<" `Type` ">"] "(" `ValueListNE` ")"
               :| `CondOperator` "(" `CondClause` ("," `CondClause`)* ")"
    CondClause: `Value` ":" `Value`
 
 The bang operators provide functions that are not available with the other
 simple values. Except in the case of ``!cond``, a bang
 operator takes a list of arguments enclosed in parentheses and performs some
 function on those arguments, producing a value for that
 bang operator. The ``!cond`` operator takes a list of pairs of arguments

llvm/lib/TableGen/TGLexer.h

 #include <vector>
 
 namespace llvm {
 template <typename T> class ArrayRef;
 class SourceMgr;
 class Twine;
 
 namespace tgtok {
   enum TokKind {
     // Markers
     Eof, Error,
 
     // Tokens with no info.
     minus, plus,        // - +
     l_square, r_square, // [ ]
     l_brace, r_brace,   // { }
     l_paren, r_paren,   // ( )
     less, greater,      // < >
     colon, semi,        // : ;
     comma, dot,         // , .
     equal, question,    // = ?
     paste,              // #
     dotdotdot,          // ...
 
-    // Keywords. ('ElseKW' is named to distinguish it from the existing 'Else'
-    // that means the preprocessor #else.)
-    Bit, Bits, Class, Code, Dag, Def, Foreach, Defm, Field, In, Int, Let, List,
-    MultiClass, String, Defset, Defvar, If, Then, ElseKW,
+    // Reserved keywords. ('ElseKW' is named to distinguish it from the
+    // existing 'Else' that means the preprocessor #else.)
+    Bit, Bits, Class, Code, Dag, Def, Defm, Defset, Defvar, ElseKW, FalseKW,
+    Field, Foreach, If, In, Include, Int, Let, List, MultiClass,
+    String, Then, TrueKW,
 
-    // !keywords.
+    // Bang operators.
     XConcat, XADD, XSUB, XMUL, XNOT, XAND, XOR, XXOR, XSRA, XSRL, XSHL,
     XListConcat, XListSplat, XStrConcat, XCast, XSubst, XForEach, XFoldl,
     XHead, XTail, XSize, XEmpty, XIf, XCond, XEq, XIsA, XDag, XNe, XLe,
     XLt, XGe, XGt, XSetDagOp, XGetDagOp,
 
+    // Boolean literals.
+    TrueVal, FalseVal,
+
     // Integer value.
     IntVal,
 
     // Binary constant.  Note that these are sized according to the number of
     // bits given.
     BinaryIntVal,
 
     // String valued tokens.

llvm/lib/TableGen/TGLexer.cpp

default:
     if (isalpha(CurChar) || CurChar == '_')
       return LexIdentifier();
 
     // Unknown character, emit an error.
     return ReturnError(TokStart, "Unexpected character");
   case EOF:
     // Lex next token, if we just left an include file.
     // Note that leaving an include file means that the next
-    // symbol is located at the end of 'include "..."'
+    // symbol is located at the end of the 'include "..."'
     // construct, so LexToken() is called with default
     // false parameter.
     if (processEOF())
       return LexToken();
 
     // Return EOF denoting the end of lexing.
     return tgtok::Eof;
 

 tgtok::TokKind TGLexer::LexIdentifier() {
   // The first letter is [a-zA-Z_].
   const char *IdentStart = TokStart;
 
   // Match the rest of the identifier regex: [0-9a-zA-Z_]*
   while (isalpha(*CurPtr) || isdigit(*CurPtr) || *CurPtr == '_')
     ++CurPtr;
 
-  // Check to see if this identifier is a keyword.
+  // Check to see if this identifier is a reserved keyword.
   StringRef Str(IdentStart, CurPtr-IdentStart);
 
-  if (Str == "include") {
-    if (LexInclude()) return tgtok::Error;
-    return Lex();
-  }
+////  if (Str == "include") {
+////    if (LexInclude()) return tgtok::Error;
+////    return Lex();
+////  }
 
   tgtok::TokKind Kind = StringSwitch<tgtok::TokKind>(Str)
     .Case("int", tgtok::Int)
     .Case("bit", tgtok::Bit)
     .Case("bits", tgtok::Bits)
     .Case("string", tgtok::String)
     .Case("list", tgtok::List)
     .Case("code", tgtok::Code)
     .Case("dag", tgtok::Dag)
     .Case("class", tgtok::Class)
     .Case("def", tgtok::Def)
+    .Case("true", tgtok::TrueVal)
+    .Case("false", tgtok::FalseVal)
     .Case("foreach", tgtok::Foreach)
     .Case("defm", tgtok::Defm)
     .Case("defset", tgtok::Defset)
     .Case("multiclass", tgtok::MultiClass)
     .Case("field", tgtok::Field)
     .Case("let", tgtok::Let)
     .Case("in", tgtok::In)
     .Case("defvar", tgtok::Defvar)
+    .Case("include", tgtok::Include)
     .Case("if", tgtok::If)
     .Case("then", tgtok::Then)
     .Case("else", tgtok::ElseKW)
     .Default(tgtok::Id);
 
-  if (Kind == tgtok::Id)
+  // A couple of tokens require special processing.
+  switch (Kind) {
+    case tgtok::Include:
+      if (LexInclude()) return tgtok::Error;
+      return Lex();
+    case tgtok::Id:
       CurStrVal.assign(Str.begin(), Str.end());
+      break;
+    default:
+      break;
+  }
+
+////  if (Kind == tgtok::Id)
+////    CurStrVal.assign(Str.begin(), Str.end());
   return Kind;
 }
 
 /// LexInclude - We just read the "include" token.  Get the string token that
 /// comes next and enter the include.
 bool TGLexer::LexInclude() {
   // The token after the include must be a string.
   tgtok::TokKind Tok = LexToken();

llvm/lib/TableGen/TGParser.cpp

 ///   SimpleValue ::= LISTSPLATTOK '(' Value ',' Value ')'
 ///   SimpleValue ::= STRCONCATTOK '(' Value ',' Value ')'
 ///   SimpleValue ::= COND '(' [Value ':' Value,]+ ')'
 ///
 Init *TGParser::ParseSimpleValue(Record *CurRec, RecTy *ItemType,
                                  IDParseMode Mode) {
   Init *R = nullptr;
   switch (Lex.getCode()) {
-  default: TokError("Unknown token when parsing a value"); break;
-  case tgtok::IntVal: R = IntInit::get(Lex.getCurIntVal()); Lex.Lex(); break;
+  default: TokError("Unknown or reserved token when parsing a value"); break;
+
+  case tgtok::TrueVal:
+    R = IntInit::get(1);
+    Lex.Lex();
+    break;
+  case tgtok::FalseVal:
+    R = IntInit::get(0);
+    Lex.Lex();
+    break;
+  case tgtok::IntVal:
+    R = IntInit::get(Lex.getCurIntVal());
+    Lex.Lex();
+    break;
   case tgtok::BinaryIntVal: {
     auto BinaryVal = Lex.getCurBinaryIntVal();
     SmallVector<Init*, 16> Bits(BinaryVal.second);
     for (unsigned i = 0, e = BinaryVal.second; i != e; ++i)
       Bits[i] = BitInit::get(BinaryVal.first & (1LL << i));
     R = BitsInit::get(Bits);
     Lex.Lex();
     break;
case tgtok::paste:
         case tgtok::l_brace:
           Result = LHS; // trailing paste, ignore.
           break;
         default:
           Init *RHSResult = ParseValue(CurRec, ItemType, ParseValueMode);
           if (!RHSResult)
             return nullptr;
           Result = BinOpInit::getListConcat(LHS, RHSResult);
+          break;
         }
         break;
       }
 
       // Create a !strconcat() operation, first casting each operand to
       // a string if necessary.
       if (LHS->getType() != StringRecTy::get()) {
         auto CastLHS = dyn_cast<TypedInit>(

llvm/test/TableGen/foreach-range-parse-errors0.td

 // RUN: not llvm-tblgen %s 2>&1 | FileCheck -DFILE=%s %s
 
 class ConstantsImpl {
   int Zero = 0;
   int One = 1;
   int Two = 2;
   int Three = 3;
   int Five = 5;
 }
 
 def Constants : ConstantsImpl;
 
-// CHECK-NOT:  error: Unknown token when parsing a value
-// CHECK: [[FILE]]:[[@LINE+3]]:22: error: Unknown token when parsing a value
+// CHECK-NOT:  error: Unknown or reserved token when parsing a value
+// CHECK: [[FILE]]:[[@LINE+3]]:22: error: Unknown or reserved token when parsing a value
 // CHECK: [[FILE]]:[[@LINE+2]]:22: error: expected integer value as end of range
 // CHECK: [[FILE]]:[[@LINE+1]]:22: error: expected declaration in for
 foreach Index = 0 -  in {
 
 }

llvm/test/TableGen/paste-reserved.td

 // RUN: not llvm-tblgen -DERROR1 %s 2>&1 | FileCheck --check-prefix=ERROR1 %s
 // RUN: not llvm-tblgen -DERROR2 %s 2>&1 | FileCheck --check-prefix=ERROR2 %s
 
 defvar list1 = ["foo", "bar", "snork"];
 
 // Pasting a list with a reserved word should produce an error.
 
 #ifdef ERROR1
 def list_paste {
   list<string> the_list = list1 # in;
 }
-// ERROR1: error: Unknown token when parsing a value
+// ERROR1: error: Unknown or reserved token when parsing a value
 #endif
 
 
 // Pasting an identifier with a reserved word should produce an error.
 
 #ifdef ERROR2
 def name_paste#in {
 }
-// ERROR2: error: Unknown token when parsing a value
+// ERROR2: error: Unknown or reserved token when parsing a value
 #endif

llvm/test/TableGen/true-false.td

+// RUN: llvm-tblgen %s | FileCheck %s
+// RUN: not llvm-tblgen -DERROR1 %s 2>&1 | FileCheck --check-prefix=ERROR1 %s
+
+// Tests for the true and false literals.
+
+defvar otherwise = true;
+defvar do_it = true;
+
+// CHECK: def rec1
+// CHECK:   bit flag1 = 1;
+// CHECK:   bit flag2 = 0;
+// CHECK:   int true_int = 1;
+
+def rec1 {
+  bit flag1 = true;
+  bit flag2 = false;
+  int true_int = true;
+}
+
+// CHECK: def rec2_true
+
+if true then
+  def rec2_true {}
+else
+  def rec2_bad {}
+
+// CHECK: def rec3_false
+
+if false then
+  def rec3_bad {}
+else
+  def rec3_false {}
+
+// CHECK: def rec4
+// CHECK:   int value = 52;
+
+def rec4 {
+  int value = !add(10, !if(!and(do_it, true), 42, 0));
+}
+
+// CHECK: def rec5
+// CHECK:    string name = "snork";
+
+def rec5 {
+  string name = !cond(false:       "foo",
+                      !not(do_it): "bar",
+                      otherwise:   "snork");
+}
+
+// CHECK: def rec6
+// CHECK:   bit xorFF = 0;
+// CHECK:   bit xorFT = 1;
+// CHECK:   bit xorTF = 1;
+// CHECK:   bit xorTT = 0;
+
+def rec6 {
+  bit xorFF = !xor(false, false);
+  bit xorFT = !xor(false, true);
+  bit xorTF = !xor(true, false);
+  bit xorTT = !xor(true, true);
+}
+
+// CHECK: def rec7
+// CHECK:   bits<3> flags = { 1, 0, 1 };
+
+def rec7 {
+  bits<3> flags = { true, false, true };
+}
+
+#ifdef ERROR1
+// ERROR1: Record name '1' is not a string
+
+def true {}
+#endif
+