[tablegen] Replace strconcat, listconcat by concat

[johannes]

Summary

Previously, the name concat was used by the operator !con. These
usages are renamed to just con, so that concat can be used by the new operation
which subsumes strconcat, listconcat and also works for code fragments. The old
!strconcat and !listconcat are just aliases for !concat.

[johannes]

adhere to the column limit

[stettberger]

I think there should be some documentation for this new TableGen command.

[johannes]

add documentation + test

[hfinkel]

lib/TableGen/Record.cpp

766

Maybe call this ConcatStringLikeInits or ConcatInits? It should be renamed now that it is not specific to StringInits.

[johannes]

remove codeconcat, make strconcat return code if all arguments are

[johannes]

I use a different approach now, WDYT?

lib/TableGen/Record.cpp

766

yeah, makes sense, done

[hfinkel]

In D40782#958027, @johannes wrote:

I use a different approach now, WDYT?

I'm not sure I like this. Before it was clear: !strconcat is for strings, !codeconcat was for code. Having !strconcat also work on code, and return something which isn't a string, seems unfortunate. And we also have !listconcat. If we want to unify things, I think it would make sense to add !concat (which works on all of those things). [If you do this, then also support !strconcat and !listconcat as aliases to be nice to out-of-tree backends].

[stettberger]

From a user's perspective, I got the impression that code [{}] is just a handy shortcut for multiline strings and I was quite surprised that they are internally actual different datatypes. So for me, changing the semantic of !strconcat would remain rather intuitive.

[johannes]

In D40782#960474, @hfinkel wrote:

In D40782#958027, @johannes wrote:

I use a different approach now, WDYT?

I'm not sure I like this. Before it was clear: !strconcat is for strings, !codeconcat was for code. Having !strconcat also work on code, and return something which isn't a string, seems unfortunate. And we also have !listconcat. If we want to unify things, I think it would make sense to add !concat (which works on all of those things). [If you do this, then also support !strconcat and !listconcat as aliases to be nice to out-of-tree backends].

Oh right, thanks, implementing !concat plus the aliases for the old functions seems to be the cleanest solution, assuming that this change won't cause too much confusion in existing code. I think it should be fine but I don't know how many people are using these operators.

[johannes]

use generic concat

If this will be approved I can also update existing usages of strconcat to concat

docs/TableGen/LangIntro.rst

``!listconcat(a, b, ...)``
     More than two arguments are accepted with the result being the concatenation
     of all the lists given.
 
 ``!strconcat(a, b, ...)``
     A string value that is the result of concatenating the 'a' and 'b' strings.
     More than two arguments are accepted with the result being the concatenation
     of all the strings given.
 
+``!codeconcat(a, b, ...)``
+    A code value that is the result of concatenating the 'a' and 'b' code
+    fragments. More than two arguments are accepted with the result being the
+    concatenation of all the code fragments given.
+
 ``str1#str2``
     "#" (paste) is a shorthand for !strconcat.  It may concatenate things that
     are not quoted strings, in which case an implicit !cast<string> is done on
     the operand of the paste.
 
 ``!cast<type>(a)``
     A symbol of type *type* obtained by looking up the string 'a' in the symbol
     table.  If the type of 'a' does not match *type*, TableGen aborts with an

include/llvm/TableGen/Record.h

public:
 
   std::string getAsString() const override;
 };
 
 /// !op (X, Y) - Combine two inits.
 class BinOpInit : public OpInit, public FoldingSetNode {
 public:
   enum BinaryOp : uint8_t { ADD, AND, OR, SHL, SRA, SRL, LISTCONCAT,
-                            STRCONCAT, CONCAT, EQ };
+                            STRCONCAT, CODECONCAT, CONCAT, EQ };
 
 private:
   Init *LHS, *RHS;
 
   BinOpInit(BinaryOp opc, Init *lhs, Init *rhs, RecTy *Type) :
       OpInit(IK_BinOpInit, Type, opc), LHS(lhs), RHS(rhs) {}
 
 public:

lib/TableGen/Record.cpp

BinOpInit *BinOpInit::get(BinaryOp Opc, Init *LHS,
   ThePool.InsertNode(I, IP);
   return I;
 }
 
 void BinOpInit::Profile(FoldingSetNodeID &ID) const {
   ProfileBinOpInit(ID, getOpcode(), getLHS(), getRHS(), getType());
 }
 
-static StringInit *ConcatStringInits(const StringInit *I0,
-                                     const StringInit *I1) {
+template <class T> static T *ConcatStringInits(const T *I0, const T *I1) {
   SmallString<80> Concat(I0->getValue());
   Concat.append(I1->getValue());
-  return StringInit::get(Concat);
+  return T::get(Concat);
 }
 
 Init *BinOpInit::Fold(Record *CurRec, MultiClass *CurMultiClass) const {
   switch (getOpcode()) {
   case CONCAT: {
     DagInit *LHSs = dyn_cast<DagInit>(LHS);
     DagInit *RHSs = dyn_cast<DagInit>(RHS);
     if (LHSs && RHSs) {
Init *BinOpInit::Fold(Record *CurRec, MultiClass *CurMultiClass) const {
   }
   case STRCONCAT: {
     StringInit *LHSs = dyn_cast<StringInit>(LHS);
     StringInit *RHSs = dyn_cast<StringInit>(RHS);
     if (LHSs && RHSs)
       return ConcatStringInits(LHSs, RHSs);
     break;
   }
+  case CODECONCAT: {
+    CodeInit *LHSs = dyn_cast<CodeInit>(LHS);
+    CodeInit *RHSs = dyn_cast<CodeInit>(RHS);
+    if (LHSs && RHSs)
+      return ConcatStringInits(LHSs, RHSs);
+    break;
+  }
   case EQ: {
     // try to fold eq comparison for 'bit' and 'int', otherwise fallback
     // to string objects.
     IntInit *L =
       dyn_cast_or_null<IntInit>(LHS->convertInitializerTo(IntRecTy::get()));
     IntInit *R =
       dyn_cast_or_null<IntInit>(RHS->convertInitializerTo(IntRecTy::get()));
 
std::string BinOpInit::getAsString() const {
   case AND: Result = "!and"; break;
   case OR: Result = "!or"; break;
   case SHL: Result = "!shl"; break;
   case SRA: Result = "!sra"; break;
   case SRL: Result = "!srl"; break;
   case EQ: Result = "!eq"; break;
   case LISTCONCAT: Result = "!listconcat"; break;
   case STRCONCAT: Result = "!strconcat"; break;
+  case CODECONCAT: Result = "!codeconcat"; break;
   }
   return Result + "(" + LHS->getAsString() + ", " + RHS->getAsString() + ")";
 }
 
 static void
 ProfileTernOpInit(FoldingSetNodeID &ID, unsigned Opcode, Init *LHS, Init *MHS,
                   Init *RHS, RecTy *Type) {
   ID.AddInteger(Opcode);

lib/TableGen/TGLexer.h

enum TokKind {
     equal, question,    // = ?
     paste,              // #
 
     // Keywords.
     Bit, Bits, Class, Code, Dag, Def, Foreach, Defm, Field, In, Int, Let, List,
     MultiClass, String,
 
     // !keywords.
-    XConcat, XADD, XAND, XOR, XSRA, XSRL, XSHL, XListConcat, XStrConcat, XCast,
-    XSubst, XForEach, XHead, XTail, XEmpty, XIf, XEq,
+    XConcat, XADD, XAND, XOR, XSRA, XSRL, XSHL, XListConcat, XStrConcat,
+    XCodeConcat, XCast, XSubst, XForEach, XHead, XTail, XEmpty, XIf, XEq,
 
     // Integer value.
     IntVal,
 
     // Binary constant.  Note that these are sized according to the number of
     // bits given.
     BinaryIntVal,
     

lib/TableGen/TGLexer.cpp

tgtok::TokKind Kind =
     .Case("sra", tgtok::XSRA)
     .Case("srl", tgtok::XSRL)
     .Case("cast", tgtok::XCast)
     .Case("empty", tgtok::XEmpty)
     .Case("subst", tgtok::XSubst)
     .Case("foreach", tgtok::XForEach)
     .Case("listconcat", tgtok::XListConcat)
     .Case("strconcat", tgtok::XStrConcat)
+    .Case("codeconcat", tgtok::XCodeConcat)
     .Default(tgtok::Error);
 
   return Kind != tgtok::Error ? Kind : ReturnError(Start-1, "Unknown operator");
 }

lib/TableGen/TGParser.cpp

Init *TGParser::ParseOperation(Record *CurRec, RecTy *ItemType) {
   case tgtok::XADD:
   case tgtok::XAND:
   case tgtok::XOR:
   case tgtok::XSRA:
   case tgtok::XSRL:
   case tgtok::XSHL:
   case tgtok::XEq:
   case tgtok::XListConcat:
-  case tgtok::XStrConcat: {  // Value ::= !binop '(' Value ',' Value ')'
+  case tgtok::XStrConcat:
+  case tgtok::XCodeConcat: {  // Value ::= !binop '(' Value ',' Value ')'
     tgtok::TokKind OpTok = Lex.getCode();
     SMLoc OpLoc = Lex.getLoc();
     Lex.Lex();  // eat the operation
 
     BinOpInit::BinaryOp Code;
     RecTy *Type = nullptr;
 
     switch (OpTok) {
case tgtok::XCodeConcat: {  // Value ::= !binop '(' Value ',' Value ')'
     case tgtok::XListConcat:
       Code = BinOpInit::LISTCONCAT;
       // We don't know the list type until we parse the first argument
       break;
     case tgtok::XStrConcat:
       Code = BinOpInit::STRCONCAT;
       Type = StringRecTy::get();
       break;
+    case tgtok::XCodeConcat:
+      Code = BinOpInit::CODECONCAT;
+      Type = CodeRecTy::get();
+      break;
     }
 
     if (Lex.getCode() != tgtok::l_paren) {
       TokError("expected '(' after binary operator");
       return nullptr;
     }
     Lex.Lex();  // eat the '('
 
if (OpTok == tgtok::XListConcat) {
         InitList[0]->print(errs());
         Error(OpLoc, "expected a list");
         return nullptr;
       }
     }
 
     // We allow multiple operands to associative operators like !strconcat as
     // shorthand for nesting them.
-    if (Code == BinOpInit::STRCONCAT || Code == BinOpInit::LISTCONCAT) {
+    if (Code == BinOpInit::STRCONCAT || Code == BinOpInit::CODECONCAT ||
+        Code == BinOpInit::LISTCONCAT) {
       while (InitList.size() > 2) {
         Init *RHS = InitList.pop_back_val();
         RHS = (BinOpInit::get(Code, InitList.back(), RHS, Type))
                            ->Fold(CurRec, CurMultiClass);
         InitList.back() = RHS;
       }
     }
 
Init *TGParser::ParseSimpleValue(Record *CurRec, RecTy *ItemType,
   case tgtok::XAND:
   case tgtok::XOR:
   case tgtok::XSRA:
   case tgtok::XSRL:
   case tgtok::XSHL:
   case tgtok::XEq:
   case tgtok::XListConcat:
   case tgtok::XStrConcat:   // Value ::= !binop '(' Value ',' Value ')'
+  case tgtok::XCodeConcat:
   case tgtok::XIf:
   case tgtok::XForEach:
   case tgtok::XSubst: {  // Value ::= !ternop '(' Value ',' Value ',' Value ')'
     return ParseOperation(CurRec, ItemType);
   }
   }
 
   return R;

test/TableGen/codeconcat.td

+// RUN: llvm-tblgen %s | FileCheck %s
+
+class Par<code Arg> {
+  code Code = !codeconcat([{ ( }], Arg, [{ ) }]);
+}
+
+// CHECK: def M {
+// CHECK:   code Code = [{ ( foo ) }];
+// CHECK: }
+def M : Par<[{foo}]> {}