[llvm-rc] Add .rc scripts parser.

[mnbvmar]

Summary

This adds the parsing ability to the llvm-rc tool. Currently, it can parse a limited amount of RC statements/resources, but the other ones can be easily added to the tool.

Note that this revision is build on top of D35957.

[mnbvmar]

Style fixes.

[zturner]

llvm/test/tools/llvm-rc/Inputs/parser1.rc

2

Can you choose some more descriptive file names for the rc files? Maybe a short 1-3 word filename that briefly explains what's interesting about this input file. For example, test-parse-error.rc or test-everything.rc or test-stringtable.rc, etc.

llvm/tools/llvm-rc/ResourceScriptParser.h

68

node is a strange name for a namespace. Is this used anywhere else? What about rc?

77

Should these be passed by reference?

102

I think this can just be passed by value. I don't think there's anything to be gained by moving StringRefs since they're trivially copyable.

105

Might as well just use a bool here?

128

Same here, I think this can be passed by const& and just assigned.

129–132

The semicolons aren't necessary after the function body.

152–153

I think the heavy use of templates and metaprogramming here is making it hard to understand the code, while not really adding much benefit. The rc language is probably not going to be extended much in the future, so we don't have to worry about being generic to adapt to future extensions, and there also are not very many statement types to handle. I feel like it would be much easier to understand the code if there less magic hidden behind the various template classes.

Maybe someone else will weigh in though.

166

Making an rvalue reference on the stack seems a little unusual. Is this correct?

297–301

I don't really follow this comment. Can't we use the stuff in llvm/Casting.h to maintain dynamic polymorphism?

llvm/tools/llvm-rc/ResourceScriptStmt.h

100

Why do these need to be rvalue references? Passing by value seems to be more appropriate.

115–116

Same here. Do we need rvalue references here? You could probably just accept ArrayRef<BasePtr> or ArrayRef<StringTableStmt>

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

The metaprogramming here was meant to make it easier to encode the language grammar - in case I do something in a wrong way, it should probably result in a compilation failure, not in a weird parsing error.

However, if the others also say this is really unnecessary, I'll be able rewrite the code.

297–301

Sorry for not noticing you. I locally rewrote Either as I found it counter-intuitive, too. However, new patch was not ready yet and I didn't submit it here.

[rnk]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

I'd prefer it if this were a normal, single parser class, recursive descent parser. I'm pretty sure the resource language is LL(1), so we can pretty much look at each token in sequence, switch on it, and decide how to proceed.

166

It's the universal reference pattern, it binds to anything without making copies.

297–301

+1, the usual thing of switching on the token kind and contents is probably more readable than TMP that tries to parse template arguments in sequence.

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

I think it's actually LL(2) as resource declarations usually follow the "[resource name] [resource type] [arguments]" convention, e.g. "1 ICON ...". (Of course, some resources don't have their names and then its type is the first token to be read.)

However, it doesn't hurt us very much and a single-class implementation is possible.

[mnbvmar]

Code rewritten a little bit.

Now, functions return objects instead of pointers to them. This will make the code more readable.

Either now takes also a base result class T and returns unique_ptr<T> (possibly upcasted from the created object). This way, we get dynamic polymorphism "for free."

[mnbvmar]

Parser rewritten to a single non-templated class. The new patch supports the same set of resources as before.

[mnbvmar]

Some comments stopped being relevant to the current patch.

[ecbeckmann]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

72

nit: Name could be clearer, such as "doesNextTokenMatchKind()" or "isNextTokenKind()".

[kuhar]

llvm/tools/llvm-rc/ResourceScriptStmt.h

38

nit: Is there a point moving StringRefs here? I think that internally StringRef is something like a pointer and length.

[mnbvmar]

Small style fixes according to the suggestions.

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

44

The pattern above:

auto Something = tryGetSomething(); // returns Expected<T>
if (!Something)
  return Something.takeError();
// Now we can use *Something to get the value of type T.

occurs in many parts of the parser. Maybe it is better to replace this pattern with a macro creating variable Something or fails, that is:

#define ASSIGN_OR_RETURN(var, expr)                                            \
  auto var = (expr);                                                           \
  if (!var)                                                                    \
    return var.takeError();

// ...
ASSIGN_OR_RETURN(Something, tryGetSomething());
// Now, use *Something.

Do you think it'd be more readable?

[zturner]

This looks MUCH better. Thanks

llvm/tools/llvm-rc/ResourceScriptParser.cpp

189–200

Do we need this to be a lambda? It will construct one of these on the stack every iteration of the loop. It seems like it should just be a global static function

llvm/tools/llvm-rc/ResourceScriptParser.h

88

Why is it returning an empty tuple? What's the point of the return value if it contains no information?

llvm/tools/llvm-rc/ResourceScriptStmt.h

74

I think it can just take this argument by value.

[ecbeckmann]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

44

I think a macro is valid here, I've done it in other parts of the codebase, see RETURN_IF_ERROR in COFFObjectFile.cpp

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

88

It returns nothing or an error message. Error might be better here.

[mnbvmar]

Style fixes, added macro-based error handling.

[zturner]

lgtm

[mnbvmar]

Tests failed locally on Windows; fixed their invocation.

llvm/test/tools/llvm-rc/Inputs/parser1.rc

+"meh" IcOn "hello.bmp"
+Icon Icon "Icon"
+
+LANGUAGE 5, 12
+
+STRINGTABLE
+LANGUAGE 1, 1
+CHARACTERISTICS 500
+LANGUAGE 3, 4
+VERSION 14
+{
+  1 "hello"
+  2 "world"
+}
+STRINGTABLE BEGIN END

llvm/test/tools/llvm-rc/Inputs/parser2.rc

+STRINGTABLE
+VERSION 8
+{
+  1 "hello"
+  2
+}

llvm/test/tools/llvm-rc/Inputs/parser3.rc

+LANGUAGE

llvm/test/tools/llvm-rc/Inputs/parser4.rc

+STRINGTABLE
+CHARACTERISTICS
+BEGIN
+  100 "No integer after CHARACTERISTICS."
+END

llvm/test/tools/llvm-rc/Inputs/parser5.rc

+& ICON "WeirdResourceName.ico"

llvm/test/tools/llvm-rc/Inputs/parser6.rc

+LANGUAGE 5 7

llvm/test/tools/llvm-rc/Inputs/parser7.rc

+LANGUAGE 5,, 7

llvm/test/tools/llvm-rc/parser.test

+; RUN: llvm-rc /V %p/Inputs/parser1.rc | FileCheck %s --check-prefix P1
+
+; P1:  Parsed file:
+; P1-NEXT:  Icon ("meh"): "hello.bmp"
+; P1-NEXT:  Icon (Icon): "Icon"
+; P1-NEXT:  Language: 5, Sublanguage: 12
+; P1-NEXT:  StringTable:
+; P1-NEXT:    Option: Language: 1, Sublanguage: 1
+; P1-NEXT:    Option: Characteristics: 500
+; P1-NEXT:    Option: Language: 3, Sublanguage: 4
+; P1-NEXT:    Option: Version: 14
+; P1-NEXT:    1 => "hello"
+; P1-NEXT:    2 => "world"
+; P1-NEXT:  StringTable:
+
+
+; RUN: llvm-rc /V %p/Inputs/parser2.rc 2> %t2 || true
+; RUN: FileCheck %s --check-prefix P2 --input-file %t2
+
+; P2:  llvm-rc: Error parsing file: expected string, got }
+
+
+; RUN: llvm-rc /V %p/Inputs/parser3.rc 2> %t3 || true
+; RUN: FileCheck %s --check-prefix P3 --input-file %t3
+
+; P3:  llvm-rc: Error parsing file: expected integer, got <EOF>
+
+
+; RUN: llvm-rc /V %p/Inputs/parser4.rc 2> %t4 || true
+; RUN: FileCheck %s --check-prefix P4 --input-file %t4
+
+; P4:  llvm-rc: Error parsing file: expected integer, got BEGIN
+
+
+; RUN: llvm-rc /V %p/Inputs/parser5.rc 2> %t5 || true
+; RUN: FileCheck %s --check-prefix P5 --input-file %t5
+
+; P5:  llvm-rc: Error parsing file: expected data/resource, got &
+
+
+; RUN: llvm-rc /V %p/Inputs/parser6.rc 2> %t6 || true
+; RUN: FileCheck %s --check-prefix P6 --input-file %t6
+
+; P6:  llvm-rc: Error parsing file: expected comma, got 7
+
+
+; RUN: llvm-rc /V %p/Inputs/parser7.rc 2> %t7 || true
+; RUN: FileCheck %s --check-prefix P7 --input-file %t7
+
+; P7:  llvm-rc: Error parsing file: expected integer, got ,

llvm/test/tools/llvm-rc/tokenizer.test

-; RUN: llvm-rc /V %p/Inputs/tokens.rc | FileCheck %s
+; RUN: llvm-rc /V %p/Inputs/tokens.rc > %t1 2> %t2 || echo "Errorcode $?" >> %t2
+; RUN: FileCheck %s --input-file %t1 --check-prefix CHECK
+; RUN: FileCheck %s --input-file %t2 --check-prefix PARSE
 
 ; CHECK:  Int: 1; int value = 1
 ; CHECK-NEXT:  Plus: +
 ; CHECK-NEXT:  Int: 2; int value = 2
 ; CHECK-NEXT:  Minus: -
 ; CHECK-NEXT:  Int: 3214L; int value = 3214
 ; CHECK-NEXT:  Amp: &
 ; CHECK-NEXT:  Int: 0x120894; int value = 1181844

 ; CHECK-NEXT:  String: "RC string test."
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  String: L"Another RC string test.'&{"
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  Int: 42; int value = 42
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  Int: 100; int value = 100
 ; CHECK-NEXT:  String: ":))"
+
+; PARSE:  llvm-rc: Error parsing file: expected data/resource, got 1
+; PARSE-NEXT:  Errorcode 1

llvm/tools/llvm-rc/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Option
   Support
   )
 
 set(LLVM_TARGET_DEFINITIONS Opts.td)
 
 tablegen(LLVM Opts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(RcTableGen)
 
 add_llvm_tool(llvm-rc
   llvm-rc.cpp
   ResourceScriptToken.cpp
+  ResourceScriptParser.cpp
+  ResourceScriptStmt.cpp
   )

llvm/tools/llvm-rc/ResourceScriptParser.h

+//===-- ResourceScriptParser.h ----------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This is an implementation of .rc scripts parser. It accepts the tokenized
+// input and produces single statement/resource definitions. This parser allows
+// to define the statements as the composition of simple elements. For example,
+// LANGUAGE statement has no assigned name; it starts by 'LANGUAGE' identifier,
+// and its parameters are: an integer and an integer followed by a comma:
+//
+//  ADD_IDENT_NAME(Language);  /* Introduces class 'LanguageName'. */
+//  class LanguageStmt : public Resource {
+//  public:
+//    LanguageStmt(uint32_t, uint32_t);
+//  };
+//
+//  using NodeLanguageStmt =
+//      UnnamedItem<LanguageName, ConcatArgs<LanguageStmt,
+//                                           NodeInt, AfterComma<NodeInt>>>;
+//
+// Each of the parser classes defines a static method parse(&Start, End),
+// where Start is an iterator to the beginning of the parsed token, and End
+// defines the end of the text. The method returns Expected<unique_ptr<Ret>>
+// which contains either an error description or a pointer to the read token.
+// Moreover, it advances the 'Start' iterator so that it points right after
+// the read token. Using pointers is necessary to maintain the polymorphism.
+//
+// All the parsers should define ParseType. This defines a type returned by the
+// parse() method. In fact, the returned type is llvm::Expected<ParseType>.
+//
+// Sometimes a parser class has multiple ways to parse a sequence of items
+// (e.g. NodeList might read any number of items and needs to know when to end;
+// NodeEither can choose one of ways of parsing the input etc.) Thus, we
+// assume the following algorithm:
+//   1. Pick a way of parsing a sequence of items (e.g. in case of NodeList,
+//        try parsing the next item.)
+//   2. If the chosen method consumes some tokens, commit to it. If it parses
+//        successfully, take the result and continue/return. If it fails,
+//        the whole class should fail.
+//   3. If the chosen method doesn't consume any tokens, try another method.
+//        Note that this method might look at the future tokens before trying
+//        to consume anything.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVMRC_RESOURCESCRIPTPARSER_H
+#define LLVM_TOOLS_LLVMRC_RESOURCESCRIPTPARSER_H
+
+#include "ResourceScriptToken.h"
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <cstdint>
+#include <string>
+#include <system_error>
+#include <vector>
+
+namespace llvm {
+namespace node {
+
+using Kind = RCToken::Kind;
+
+using LocIter = std::vector<llvm::RCToken>::iterator;
+
+// Class describing a single failure of parser.
+class ParserError : public ErrorInfo<ParserError> {
+public:
+  ParserError(Twine Expected, const LocIter CurLoc, const LocIter End);
+
+  void log(raw_ostream &OS) const override { OS << CurMessage; }
+  std::error_code convertToErrorCode() const override {
+    return std::make_error_code(std::errc::invalid_argument);
+  }
+  const std::string &getMessage() const { return CurMessage; }
+
+  static char ID; // Keep llvm::Error happy.
+
+private:
+  std::string CurMessage;
+  LocIter ErrorLoc, FileEnd;
+};
+
+Error getParserError(const Twine &Expected, const LocIter CurLoc,
+                     const LocIter End);
+
+// A class holding a name - either an integer or a reference to the string.
+class IntOrString {
+private:
+  union Data {
+    uint32_t Int;
+    StringRef String;
+    Data(uint32_t Value) : Int(Value) {}
+    Data(const StringRef Value) : String(Value) {}
+    Data(const RCToken &Token);
+  } Data;
+  bool IsInt;
+
+public:
+  IntOrString() : IntOrString(0) {}
+  IntOrString(uint32_t Value) : Data(Value), IsInt(1) {}
+  IntOrString(StringRef Value) : Data(std::move(Value)), IsInt(0) {}
+  IntOrString(const RCToken &Token)
+      : Data(Token), IsInt(Token.kind() == Kind::Int) {}
+
+  friend raw_ostream &operator<<(raw_ostream &, const IntOrString &);
+};
+
+// Describes a concatenated sequence of specified tokens. Templated by
+// NodeClass, a class deriving from ConcatArgs.
+// This class tries to parse all arguments (ArgTs...) sequentially. After it
+// succeeds, it constructs an object of type ConcatArgs, providing to it
+// all the parsed arguments.
+// If any of the parser sub-invocations fails, this parser itself fails and
+// forwards the error message to the caller.
+template <typename NodeClass, typename... ArgTs> class ConcatArgs {
+  // The tuple that stores the parsed arguments.
+  using ChildrenTuple = std::tuple<typename ArgTs::ParseType...>;
+
+  // Parses a single argument of type Type at position ParamId to tuple
+  // Children. If the result tuple is already in erroneous state, do nothing.
+  // If any error occurs when parsing, leave this error in Children variable.
+  template <size_t ParamId, typename Type>
+  static void parseOne(LocIter &Location, const LocIter End,
+                       Expected<ChildrenTuple> &Children) {
+    if (!Children)
+      return;
+
+    auto ParseResult = Type::parse(Location, End);
+    if (ParseResult)
+      std::get<ParamId>(*Children) = std::move(ParseResult.get());
+    else
+      Children = ParseResult.takeError();
+  }
+
+  // Main body of parsing function. Has a parameter pack describing the
+  // sequence of integers from 0 to sizeof...(ArgTs) - 1.
+  template <size_t... Ids>
+  static Expected<NodeClass> makePtr(LocIter &Location, const LocIter End,
+                                     integer_sequence<std::size_t, Ids...>) {
+    // Construct a tuple of results.
+    Expected<ChildrenTuple> Children{ChildrenTuple{}};
+    // Parse the sub-expressions one-by-one. If any of them fails, the error
+    // is unchanged in subsequent evaluations and returned in Children variable.
+    LLVM_ATTRIBUTE_UNUSED
+    int Temporary[] = {(parseOne<Ids, ArgTs>(Location, End, Children), 0)...};
+
+    if (!Children)
+      return Children.takeError();
+    return Expected<NodeClass>{std::move(NodeClass{
+        std::get<Ids>(std::forward<ChildrenTuple>(Children.get()))...})};
+  }
+
+public:
+  using ParseType = NodeClass;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    return makePtr(Location, End, index_sequence_for<ArgTs...>{});
+  }
+};
+
+// Describes a Node that comes right after the comma. The parser tries
+// to read the comma. If it succeeds, proceeds to parsing the templated token
+// and returns it.
+template <typename Node> class AfterComma {
+public:
+  using ParseType = typename Node::ParseType;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    if (Location == End || Location->kind() != Kind::Comma)
+      return getParserError("comma", Location, End);
+    ++Location;
+    return Node::parse(Location, End);
+  }
+};
+
+// Describes a Node that might possibly appear in the parsed sequence.
+// Parsing this returns llvm::Optional<(result of parsing of Node)>.
+// Note that consuming at least one token when parsing Node implies committing
+// to it. In such situaion, when an error occurs, Optional fails, too.
+template <typename Node> class NodeOptional {
+public:
+  using ParseType = Optional<typename Node::ParseType>;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    const LocIter OldLoc = Location;
+    auto Result = Node::parse(Location, End);
+    if (Result)
+      return Result.get();
+    else if (Location != OldLoc)
+      return Result.takeError();
+
+    // If parsing of Node failed without advancing Location, we just didn't
+    // read anything.
+    consumeError(Result.takeError());
+    return ParseType{};
+  }
+};
+
+// Describes a variable-length list of Nodes.
+// Returns std::vector<(result of parsing of Node)>.
+// The class reads Nodes until one of the parsing attempts fails. If any
+// token has been consumed during this single attempt, the whole List fails.
+// In the opposite case, already read Nodes (possibly none) are returned.
+template <typename Node> class NodeList {
+public:
+  using ParseType = std::vector<typename Node::ParseType>;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    LocIter CurLoc = Location;
+    ParseType ResultVec;
+    while (true) {
+      auto Result = Node::parse(Location, End);
+      if (!Result) {
+        auto Err = Result.takeError();
+        if (Location != CurLoc) {
+          return std::move(Err);
+        } else {
+          consumeError(std::move(Err));
+          break;
+        }
+      }
+      CurLoc = Location;
+      ResultVec.push_back(std::move(Result.get()));
+    }
+    Location = CurLoc;
+    return ResultVec;
+  }
+};
+
+// Describes a variable-length list of Nodes representing a single block
+// of script: BlockBegin [Node]* BlockEnd.
+// Returns std::vector<(result of parsing of Node)>.
+// The class is analoguous to NodeList in terms of its behavior.
+template <typename Node> class NodeBlockList {
+public:
+  using ParseType = typename NodeList<Node>::ParseType;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    if (Location == End || Location->kind() != Kind::BlockBegin)
+      return getParserError("{ or BEGIN", Location, End);
+    Location++;
+
+    auto Result = NodeList<Node>::parse(Location, End);
+    if (!Result)
+      return std::move(Result);
+
+    if (Location == End || Location->kind() != Kind::BlockEnd)
+      return getParserError("} or END", Location, End);
+    Location++;
+
+    return std::move(Result);
+  }
+};
+
+// Describes a node that might be one of EitherTypes.
+// Its ParseType is a pointer to ReturnType as we want to preserve the
+// polymorphism between ReturnType and each parsed type. That is, each
+// of the parsed types should derive, directly or indirectly, after
+// ReturnType.
+//
+// This tries matching the subsequent types one by one. If any of them
+// succeeds or consumes any token, the search is terminated and the
+// return value is forwarded. If no type matches, the search fails.
+template <typename...> class NodeEither;
+
+template <typename ReturnType, typename Head, typename... Tail>
+class NodeEither<ReturnType, Head, Tail...> {
+public:
+  using ParseType = std::unique_ptr<ReturnType>;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    const LocIter StartLoc = Location;
+    auto Result = Head::parse(Location, End);
+    if (Result)
+      return make_unique<typename Head::ParseType>(std::move(Result.get()));
+    if (Location != StartLoc)
+      return Result.takeError();
+
+    consumeError(Result.takeError());
+    return NodeEither<ReturnType, Tail...>::parse(Location, End);
+  }
+};
+
+template <typename ReturnType> class NodeEither<ReturnType> {
+public:
+  using ParseType = std::unique_ptr<ReturnType>;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    return getParserError("data/resource", Location, End);
+  }
+};
+
+// Describes an integer node. For now, it expects to read an integer.
+// In the future, it should read arithmetic expressions consisting of
+// tokens <int> ( ) + - | & ~.
+class NodeInt {
+public:
+  using ParseType = uint32_t;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End);
+};
+
+// Describes a node parsing string tokens.
+class NodeString {
+public:
+  using ParseType = StringRef;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End);
+};
+
+// Describes a node parsing tokens which might be either an integer,
+// string or identifier. This kind of nodes is used when naming resources:
+// msdn.microsoft.com/en-us/library/windows/desktop/aa380680(v=vs.85).aspx
+class NodeResourceName {
+public:
+  using ParseType = IntOrString;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End);
+};
+
+// Describes a resource or statement of the following format:
+//   FIXED_STRING Arguments...
+// where FIXED_STRING == ItemName::get() and arguments are parsed by ItemNode.
+// FIXED_STRING is compared in a case-insensitive manner.
+// An example of such item is LANGUAGE:
+//   msdn.microsoft.com/en-us/library/windows/desktop/aa381019(v=vs.85).aspx
+//
+// As soon as FIXED_STRING (name of resource) is matched, the token is
+// consumed in order to prevent other resource types from trying to parse
+// (that is, we commit to this type of item).
+template <typename ItemName, typename ItemNode> class UnnamedItem {
+protected:
+  static bool checkEq(LocIter Location) {
+    return Location->value().equals_lower(ItemName::get());
+  }
+
+public:
+  using ParseType = typename ItemNode::ParseType;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    if (Location == End || !checkEq(Location))
+      return getParserError(ItemName::get(), Location, End);
+    return ItemNode::parse(++Location, End);
+  }
+
+  static const char *itemType() { return ItemName::get(); }
+};
+
+// Describes a resource or statement of the following format:
+//   Name FIXED_STRING Arguments...
+// where Name is parsed by NodeResourceName,
+//       FIXED_STRING == ItemName::get() (case-insensitive comparison), and
+//       Arguments are parsed by ItemNode.
+//
+// As soon as Name and FIXED_STRING are matched, the two tokens are consumed
+// to disallow other resource types from trying to parse.
+template <typename ItemName, typename ItemNode>
+class NamedItem : public UnnamedItem<ItemName, ItemNode> {
+public:
+  using ParseType = typename ItemNode::ParseType;
+
+  static Expected<ParseType> parse(LocIter &Location, const LocIter End) {
+    if (Location == End)
+      return getParserError("resource name", Location, End);
+    if (Location->kind() != Kind::Int && Location->kind() != Kind::String &&
+        Location->kind() != Kind::Identifier)
+      return getParserError("resource name", Location, End);
+    if (!UnnamedItem<ItemName, ItemNode>::checkEq(Location + 1))
+      return getParserError(ItemName::get(), Location, End);
+    IntOrString Name{*Location};
+
+    Location += 2;
+    auto Result = ItemNode::parse(Location, End);
+
+    if (Result)
+      Result->setName(std::move(Name));
+
+    return std::move(Result);
+  }
+};
+
+// A macro emitting IdentName class declaring its name to be Ident. This allows
+// the classes to be used in UnnamedItem and NamedItem parsers.
+#define ADD_IDENT_NAME(Ident)                                                  \
+  class Ident##Name {                                                          \
+  public:                                                                      \
+    static const char *get() { return #Ident; }                                \
+  };
+
+} // namespace node
+} // namespace llvm
+
+#endif

llvm/tools/llvm-rc/ResourceScriptParser.cpp

+//===-- ResourceScriptParser.cpp --------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This implements non-templated functions from ResourceScriptParser.h.
+//
+//===---------------------------------------------------------------------===//
+
+#include "ResourceScriptParser.h"
+
+namespace llvm {
+namespace node {
+
+char ParserError::ID = 0;
+
+IntOrString::Data::Data(const RCToken &Token) {
+  if (Token.kind() == Kind::Int)
+    Int = Token.intValue();
+  else if (Token.kind() == Kind::String || Token.kind() == Kind::Identifier)
+    String = Token.value();
+  else
+    assert(false && "Tried to get IntOrString from neither int, string "
+                    "or identifier.");
+}
+
+ParserError::ParserError(const Twine Expected, const LocIter CurLoc,
+                         const LocIter End)
+    : ErrorLoc(CurLoc), FileEnd(End) {
+  CurMessage = "Error parsing file: expected " + Expected.str() + ", got " +
+               (CurLoc == End ? "<EOF>" : CurLoc->value()).str();
+}
+
+Error getParserError(const Twine &Message, const LocIter CurLoc,
+                     const LocIter End) {
+  return make_error<ParserError>(Message, CurLoc, End);
+}
+
+raw_ostream &operator<<(raw_ostream &OS, const IntOrString &Item) {
+  if (Item.IsInt)
+    return OS << Item.Data.Int;
+  else
+    return OS << Item.Data.String;
+}
+
+Expected<NodeInt::ParseType> NodeInt::parse(LocIter &Location,
+                                            const LocIter End) {
+  if (Location == End || Location->kind() != Kind::Int)
+    return getParserError("integer", Location, End);
+  return (Location++)->intValue();
+}
+
+Expected<NodeString::ParseType> NodeString::parse(LocIter &Location,
+                                                  const LocIter End) {
+  if (Location == End || Location->kind() != Kind::String)
+    return getParserError("string", Location, End);
+  return (Location++)->value();
+}
+
+Expected<NodeResourceName::ParseType>
+NodeResourceName::parse(LocIter &Location, const LocIter End) {
+  if (Location == End)
+    return getParserError("resource name", Location, End);
+  const Kind LocKind = Location->kind();
+  if (LocKind == Kind::Int)
+    return (Location++)->intValue();
+  if (LocKind == Kind::String || LocKind == Kind::Identifier)
+    return (Location++)->value();
+  return getParserError("resource name", Location, End);
+}
+
+} // namespace node
+} // namespace llvm

llvm/tools/llvm-rc/ResourceScriptStmt.h

+//===-- ResourceScriptStmt.h ------------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This contains the definition of types of statements and resources occurring
+// in RC scripts. For now, only a very limited set of resources is supported.
+// However, it will be extended in the future.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVMRC_RESOURCESCRIPTSTMT_H
+#define LLVM_TOOLS_LLVMRC_RESOURCESCRIPTSTMT_H
+
+#include "ResourceScriptParser.h"
+#include "ResourceScriptToken.h"
+
+namespace llvm {
+namespace node {
+
+// Base resource. All the resources should derive from this base.
+class Resource {
+protected:
+  IntOrString ResName;
+
+public:
+  Resource() = default;
+  Resource(Resource &&) = default;
+  void setName(const IntOrString &Name) { ResName = Name; }
+  virtual raw_ostream &log(raw_ostream &OS) const {
+    return OS << "Base statement\n";
+  };
+  virtual ~Resource() {}
+};
+
+// --- RC statements. --- //
+
+class OptionStatement {
+public:
+  virtual ~OptionStatement() {}
+  virtual raw_ostream &log(raw_ostream &X) const { return X; }
+};
+
+class BasicOptionStatement : public OptionStatement {};
+
+// LANGUAGE statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381019(v=vs.85).aspx
+ADD_IDENT_NAME(Language);
+
+class LanguageStmt : public BasicOptionStatement {
+  uint32_t Lang, SubLang;
+
+public:
+  LanguageStmt(uint32_t LangId, uint32_t SubLangId);
+  raw_ostream &log(raw_ostream &) const override;
+};
+
+using NodeLanguageStmt =
+    UnnamedItem<LanguageName,
+                ConcatArgs<LanguageStmt, NodeInt, AfterComma<NodeInt>>>;
+
+// CHARACTERISTICS statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380872(v=vs.85).aspx
+ADD_IDENT_NAME(Characteristics);
+
+class CharacteristicsStmt : public BasicOptionStatement {
+  int32_t Value;
+
+public:
+  CharacteristicsStmt(uint32_t ChValue);
+  raw_ostream &log(raw_ostream &) const override;
+};
+
+using NodeCharacteristicsStmt =
+    UnnamedItem<CharacteristicsName, ConcatArgs<CharacteristicsStmt, NodeInt>>;
+
+// VERSION statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381059(v=vs.85).aspx
+ADD_IDENT_NAME(Version);
+
+class VersionStmt : public BasicOptionStatement {
+  uint32_t Version;
+
+public:
+  VersionStmt(uint32_t VersionId);
+  raw_ostream &log(raw_ostream &) const override;
+};
+
+using NodeVersionStmt =
+    UnnamedItem<VersionName, ConcatArgs<VersionStmt, NodeInt>>;
+
+// A basic set of optional resource-specific statements.
+// Occurs e.g. in STRINGTABLE resource:
+//   msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+using NodeBasicOption =
+    NodeEither<BasicOptionStatement, NodeCharacteristicsStmt, NodeLanguageStmt,
+               NodeVersionStmt>;
+
+// --- RC resources and helper statements. --- //
+
+/*class Resource {
+  virtual raw_ostream &log(raw_ostream &) const;
+};*/
+
+// LANGUAGE resource.
+class LanguageRes : public Resource {
+  uint32_t Lang, SubLang;
+
+public:
+  LanguageRes(uint32_t LangId, uint32_t SubLangId);
+  raw_ostream &log(raw_ostream &) const override;
+};
+
+using NodeLanguageRes =
+    UnnamedItem<LanguageName,
+                ConcatArgs<LanguageRes, NodeInt, AfterComma<NodeInt>>>;
+
+// ICON resource.
+ADD_IDENT_NAME(Icon);
+
+class IconRes : public Resource {
+  StringRef Value;
+
+public:
+  IconRes(StringRef Name);
+  raw_ostream &log(raw_ostream &) const override;
+};
+
+using NodeIconRes = NamedItem<IconName, ConcatArgs<IconRes, NodeString>>;
+
+// Single statement in STRINGTABLE resource block.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+// It is a single "stringID string" statement.
+class StringTableStmt : public Resource {
+public:
+  uint32_t Id;
+  StringRef String;
+  StringTableStmt(uint32_t SId, StringRef SString);
+};
+
+using NodeStringTableStmt = ConcatArgs<StringTableStmt, NodeInt, NodeString>;
+
+// STRINGTABLE resource.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+ADD_IDENT_NAME(StringTable);
+
+class StringTableRes : public Resource {
+  std::vector<std::unique_ptr<BasicOptionStatement>> Options;
+  std::vector<StringTableStmt> Statements;
+
+public:
+  StringTableRes(std::vector<std::unique_ptr<BasicOptionStatement>> &&Opts,
+                 std::vector<StringTableStmt> &&Stmts);
+  raw_ostream &log(raw_ostream &OS) const override;
+};
+
+using NodeStringTableRes =
+    UnnamedItem<StringTableName,
+                ConcatArgs<StringTableRes, NodeList<NodeBasicOption>,
+                           NodeBlockList<NodeStringTableStmt>>>;
+
+// A top-level resource definition.
+using NodeResource =
+    NodeEither<Resource, NodeLanguageRes, NodeIconRes, NodeStringTableRes>;
+
+} // namespace node
+} // namespace llvm
+
+#endif

llvm/tools/llvm-rc/ResourceScriptStmt.cpp

+//===-- ResourceScriptStmt.cpp ----------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This implements methods defined in ResourceScriptStmt.h.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#include "ResourceScriptStmt.h"
+
+namespace llvm {
+namespace node {
+
+LanguageStmt::LanguageStmt(uint32_t LangId, uint32_t SubLangId)
+    : Lang(LangId), SubLang(SubLangId) {}
+
+raw_ostream &LanguageStmt::log(raw_ostream &OS) const {
+  return OS << "Language: " << Lang << ", Sublanguage: " << SubLang << "\n";
+}
+
+CharacteristicsStmt::CharacteristicsStmt(uint32_t ChValue) : Value(ChValue) {}
+
+raw_ostream &CharacteristicsStmt::log(raw_ostream &OS) const {
+  return OS << "Characteristics: " << Value << "\n";
+}
+
+VersionStmt::VersionStmt(uint32_t VersionId) : Version(VersionId) {}
+
+raw_ostream &VersionStmt::log(raw_ostream &OS) const {
+  return OS << "Version: " << Version << "\n";
+}
+
+LanguageRes::LanguageRes(uint32_t LangId, uint32_t SubLangId)
+    : Lang(LangId), SubLang(SubLangId) {}
+
+raw_ostream &LanguageRes::log(raw_ostream &OS) const {
+  return OS << "Language: " << Lang << ", Sublanguage: " << SubLang << "\n";
+}
+
+IconRes::IconRes(StringRef Name) : Value(Name) {}
+
+raw_ostream &IconRes::log(raw_ostream &OS) const {
+  return OS << "Icon (" << ResName << "): " << Value << "\n";
+}
+
+StringTableStmt::StringTableStmt(uint32_t SId, StringRef SString)
+    : Id(SId), String(SString) {}
+
+StringTableRes::StringTableRes(
+    std::vector<std::unique_ptr<BasicOptionStatement>> &&Opts,
+    std::vector<StringTableStmt> &&Stmts)
+    : Options(std::move(Opts)), Statements(std::move(Stmts)) {}
+
+raw_ostream &StringTableRes::log(raw_ostream &OS) const {
+  OS << "StringTable:\n";
+  for (const auto &Option : Options) {
+    OS << "  Option: ";
+    Option->log(OS);
+  }
+  for (const StringTableStmt &Stmt : Statements) {
+    OS << "  " << Stmt.Id << " => " << Stmt.String << "\n";
+  }
+  return OS;
+}
+
+} // namespace node
+} // namespace llvm

llvm/tools/llvm-rc/llvm-rc.cpp

 //===-- llvm-rc.cpp - Compile .rc scripts into .res -------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // Compile .rc scripts into .res files. This is intended to be a
 // platform-independent port of Microsoft's rc.exe tool.
 //
 //===----------------------------------------------------------------------===//
 
+#include "ResourceScriptStmt.h"
 #include "ResourceScriptToken.h"
 
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/Process.h"
for (const RCToken &Token : Tokens) {
              << Token.value();
       if (Token.kind() == RCToken::Kind::Int)
         outs() << "; int value = " << Token.intValue();
 
       outs() << "\n";
     }
   }
 
+  if (BeVerbose) {
+    outs() << "Parsed file:\n";
+    outs().flush();
+
+    auto TokenBegin = Tokens.begin();
+    while (TokenBegin != Tokens.end()) {
+      ExitOnErr(node::NodeResource::parse(TokenBegin, Tokens.end()))
+          ->log(outs());
+      outs().flush();
+    }
+  }
+
   return 0;
 }