[llvm-rc] Add .rc scripts parser.

[mnbvmar]

Summary

This adds the parsing ability to the llvm-rc tool. Currently, it can parse a limited amount of RC statements/resources, but the other ones can be easily added to the tool.

Note that this revision is build on top of D35957.

[mnbvmar]

Style fixes.

[zturner]

llvm/test/tools/llvm-rc/Inputs/parser1.rc

2

Can you choose some more descriptive file names for the rc files? Maybe a short 1-3 word filename that briefly explains what's interesting about this input file. For example, test-parse-error.rc or test-everything.rc or test-stringtable.rc, etc.

llvm/tools/llvm-rc/ResourceScriptParser.h

68

node is a strange name for a namespace. Is this used anywhere else? What about rc?

77

Should these be passed by reference?

102

I think this can just be passed by value. I don't think there's anything to be gained by moving StringRefs since they're trivially copyable.

105

Might as well just use a bool here?

128

Same here, I think this can be passed by const& and just assigned.

129–132

The semicolons aren't necessary after the function body.

152–153

I think the heavy use of templates and metaprogramming here is making it hard to understand the code, while not really adding much benefit. The rc language is probably not going to be extended much in the future, so we don't have to worry about being generic to adapt to future extensions, and there also are not very many statement types to handle. I feel like it would be much easier to understand the code if there less magic hidden behind the various template classes.

Maybe someone else will weigh in though.

166

Making an rvalue reference on the stack seems a little unusual. Is this correct?

297–301

I don't really follow this comment. Can't we use the stuff in llvm/Casting.h to maintain dynamic polymorphism?

llvm/tools/llvm-rc/ResourceScriptStmt.h

100

Why do these need to be rvalue references? Passing by value seems to be more appropriate.

115–116

Same here. Do we need rvalue references here? You could probably just accept ArrayRef<BasePtr> or ArrayRef<StringTableStmt>

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

The metaprogramming here was meant to make it easier to encode the language grammar - in case I do something in a wrong way, it should probably result in a compilation failure, not in a weird parsing error.

However, if the others also say this is really unnecessary, I'll be able rewrite the code.

297–301

Sorry for not noticing you. I locally rewrote Either as I found it counter-intuitive, too. However, new patch was not ready yet and I didn't submit it here.

[rnk]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

I'd prefer it if this were a normal, single parser class, recursive descent parser. I'm pretty sure the resource language is LL(1), so we can pretty much look at each token in sequence, switch on it, and decide how to proceed.

166

It's the universal reference pattern, it binds to anything without making copies.

297–301

+1, the usual thing of switching on the token kind and contents is probably more readable than TMP that tries to parse template arguments in sequence.

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

152–153

I think it's actually LL(2) as resource declarations usually follow the "[resource name] [resource type] [arguments]" convention, e.g. "1 ICON ...". (Of course, some resources don't have their names and then its type is the first token to be read.)

However, it doesn't hurt us very much and a single-class implementation is possible.

[mnbvmar]

Code rewritten a little bit.

Now, functions return objects instead of pointers to them. This will make the code more readable.

Either now takes also a base result class T and returns unique_ptr<T> (possibly upcasted from the created object). This way, we get dynamic polymorphism "for free."

[mnbvmar]

Parser rewritten to a single non-templated class. The new patch supports the same set of resources as before.

[mnbvmar]

Some comments stopped being relevant to the current patch.

[ecbeckmann]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

72

nit: Name could be clearer, such as "doesNextTokenMatchKind()" or "isNextTokenKind()".

[kuhar]

llvm/tools/llvm-rc/ResourceScriptStmt.h

38

nit: Is there a point moving StringRefs here? I think that internally StringRef is something like a pointer and length.

[mnbvmar]

Small style fixes according to the suggestions.

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

44

The pattern above:

auto Something = tryGetSomething(); // returns Expected<T>
if (!Something)
  return Something.takeError();
// Now we can use *Something to get the value of type T.

occurs in many parts of the parser. Maybe it is better to replace this pattern with a macro creating variable Something or fails, that is:

#define ASSIGN_OR_RETURN(var, expr)                                            \
  auto var = (expr);                                                           \
  if (!var)                                                                    \
    return var.takeError();

// ...
ASSIGN_OR_RETURN(Something, tryGetSomething());
// Now, use *Something.

Do you think it'd be more readable?

[zturner]

This looks MUCH better. Thanks

llvm/tools/llvm-rc/ResourceScriptParser.cpp

189–200

Do we need this to be a lambda? It will construct one of these on the stack every iteration of the loop. It seems like it should just be a global static function

llvm/tools/llvm-rc/ResourceScriptParser.h

88

Why is it returning an empty tuple? What's the point of the return value if it contains no information?

llvm/tools/llvm-rc/ResourceScriptStmt.h

74

I think it can just take this argument by value.

[ecbeckmann]

llvm/tools/llvm-rc/ResourceScriptParser.cpp

44

I think a macro is valid here, I've done it in other parts of the codebase, see RETURN_IF_ERROR in COFFObjectFile.cpp

[mnbvmar]

llvm/tools/llvm-rc/ResourceScriptParser.h

88

It returns nothing or an error message. Error might be better here.

[mnbvmar]

Style fixes, added macro-based error handling.

[zturner]

lgtm

[mnbvmar]

Tests failed locally on Windows; fixed their invocation.

llvm/test/tools/llvm-rc/Inputs/parser1.rc

+"meh" IcOn "hello.bmp"
+Icon Icon "Icon"
+
+LANGUAGE 5, 12
+
+STRINGTABLE
+LANGUAGE 1, 1
+CHARACTERISTICS 500
+LANGUAGE 3, 4
+VERSION 14
+{
+  1 "hello"
+  2 "world"
+}
+STRINGTABLE BEGIN END

llvm/test/tools/llvm-rc/Inputs/parser2.rc

+STRINGTABLE
+VERSION 8
+{
+  1 "hello"
+  2
+}

llvm/test/tools/llvm-rc/Inputs/parser3.rc

+LANGUAGE

llvm/test/tools/llvm-rc/Inputs/parser4.rc

+STRINGTABLE
+CHARACTERISTICS
+BEGIN
+  100 "No integer after CHARACTERISTICS."
+END

llvm/test/tools/llvm-rc/Inputs/parser5.rc

+& ICON "WeirdResourceName.ico"

llvm/test/tools/llvm-rc/Inputs/parser6.rc

+LANGUAGE 5 7

llvm/test/tools/llvm-rc/Inputs/parser7.rc

+LANGUAGE 5,, 7

llvm/test/tools/llvm-rc/parser.test

+; RUN: llvm-rc /V %p/Inputs/parser1.rc | FileCheck %s --check-prefix P1
+
+; P1:  Parsed file:
+; P1-NEXT:  Icon ("meh"): "hello.bmp"
+; P1-NEXT:  Icon (Icon): "Icon"
+; P1-NEXT:  Language: 5, Sublanguage: 12
+; P1-NEXT:  StringTable:
+; P1-NEXT:    Option: Language: 1, Sublanguage: 1
+; P1-NEXT:    Option: Characteristics: 500
+; P1-NEXT:    Option: Language: 3, Sublanguage: 4
+; P1-NEXT:    Option: Version: 14
+; P1-NEXT:    1 => "hello"
+; P1-NEXT:    2 => "world"
+; P1-NEXT:  StringTable:
+
+
+; RUN: llvm-rc /V %p/Inputs/parser2.rc 2> %t2 || true
+; RUN: FileCheck %s --check-prefix P2 --input-file %t2
+
+; P2:  llvm-rc: Error parsing file: expected string, got }
+
+
+; RUN: llvm-rc /V %p/Inputs/parser3.rc 2> %t3 || true
+; RUN: FileCheck %s --check-prefix P3 --input-file %t3
+
+; P3:  llvm-rc: Error parsing file: expected integer, got <EOF>
+
+
+; RUN: llvm-rc /V %p/Inputs/parser4.rc 2> %t4 || true
+; RUN: FileCheck %s --check-prefix P4 --input-file %t4
+
+; P4:  llvm-rc: Error parsing file: expected integer, got BEGIN
+
+
+; RUN: llvm-rc /V %p/Inputs/parser5.rc 2> %t5 || true
+; RUN: FileCheck %s --check-prefix P5 --input-file %t5
+
+; P5:  llvm-rc: Error parsing file: expected data/resource, got &
+
+
+; RUN: llvm-rc /V %p/Inputs/parser6.rc 2> %t6 || true
+; RUN: FileCheck %s --check-prefix P6 --input-file %t6
+
+; P6:  llvm-rc: Error parsing file: expected comma, got 7
+
+
+; RUN: llvm-rc /V %p/Inputs/parser7.rc 2> %t7 || true
+; RUN: FileCheck %s --check-prefix P7 --input-file %t7
+
+; P7:  llvm-rc: Error parsing file: expected integer, got ,

llvm/test/tools/llvm-rc/tokenizer.test

-; RUN: llvm-rc /V %p/Inputs/tokens.rc | FileCheck %s
+; RUN: llvm-rc /V %p/Inputs/tokens.rc > %t1 2> %t2 || echo "Errorcode $?" >> %t2
+; RUN: FileCheck %s --input-file %t1 --check-prefix CHECK
+; RUN: FileCheck %s --input-file %t2 --check-prefix PARSE
 
 ; CHECK:  Int: 1; int value = 1
 ; CHECK-NEXT:  Plus: +
 ; CHECK-NEXT:  Int: 2; int value = 2
 ; CHECK-NEXT:  Minus: -
 ; CHECK-NEXT:  Int: 3214L; int value = 3214
 ; CHECK-NEXT:  Amp: &
 ; CHECK-NEXT:  Int: 0x120894; int value = 1181844

 ; CHECK-NEXT:  String: "RC string test."
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  String: L"Another RC string test.'&{"
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  Int: 42; int value = 42
 ; CHECK-NEXT:  Comma: ,
 ; CHECK-NEXT:  Int: 100; int value = 100
 ; CHECK-NEXT:  String: ":))"
+
+; PARSE:  llvm-rc: Error parsing file: expected data/resource, got 1
+; PARSE-NEXT:  Errorcode 1

llvm/tools/llvm-rc/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Option
   Support
   )
 
 set(LLVM_TARGET_DEFINITIONS Opts.td)
 
 tablegen(LLVM Opts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(RcTableGen)
 
 add_llvm_tool(llvm-rc
   llvm-rc.cpp
   ResourceScriptToken.cpp
+  ResourceScriptParser.cpp
+  ResourceScriptStmt.cpp
   )

llvm/tools/llvm-rc/ResourceScriptParser.h

+//===-- ResourceScriptParser.h ----------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This is an implementation of .rc scripts parser. It accepts the tokenized
+// input and produces single statement/resource definitions. This parser allows
+// to define the statements as the composition of simple elements. For example,
+// LANGUAGE statement has no assigned name; it starts by 'LANGUAGE' identifier,
+// and its parameters are: an integer and an integer followed by a comma:
+//
+//  ADD_IDENT_NAME(Language);  /* Introduces class 'LanguageName'. */
+//  class LanguageStmt :
+//    public UnnamedItem<LanguageName, ConcatArgs<LanguageStmt,
+//                                                NodeInt,
+//                                                AfterComma<NodeInt>>> {
+//    LanguageStmt(uint32_t, uint32_t);
+//  }
+//
+// Each of the parser classes defines a static method parse(&Start, End),
+// where Start is an iterator to the beginning of the parsed token, and End
+// defines the end of the text. The method returns Expected<unique_ptr<Ret>>
+// which contains either an error description or a pointer to the read token.
+// Moreover, it advances the 'Start' iterator so that it points right after
+// the read token. Using pointers is necessary to maintain the polymorphism.
+//
+// Many of the classes defines ParseType. This defines a type returned by the
+// parse() method. In fact, the returned type is
+// llvm::Expected<std::unique_ptr<Class::ParseType>>.
+//
+// Sometimes a parser class has multiple ways to parse a sequence of items
+// (e.g. NodeList might read any number of items and needs to know when to end;
+// NodeEither can choose one of ways of parsing the input etc.) Thus, we
+// assume the following algorithm:
+//   1. Pick a way of parsing a sequence of items (e.g. in case of NodeList,
+//        try parsing the next item.)
+//   2. If the chosen method consumes some tokens, commit to it. If it parses
+//        successfully, take the result and continue/return. If it fails,
+//        the whole class should fail.
+//   3. If the chosen method doesn't consume any tokens, try another method.
+//        Note that this method might look at the future tokens before trying
+//        to consume anything.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVMRC_RESOURCESCRIPTPARSER_H
+#define LLVM_TOOLS_LLVMRC_RESOURCESCRIPTPARSER_H
+
+#include "ResourceScriptToken.h"
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <cstdint>
+#include <string>
+#include <system_error>
+#include <vector>
+
+namespace llvm {
+namespace node {
+
+using Kind = RCToken::Kind;
+
+using LocIter = std::vector<llvm::RCToken>::iterator;
+
+// Class describing a single failure of parser.
+class ParserError : public ErrorInfo<ParserError> {
+public:
+  ParserError(const Twine &Expected, const LocIter CurLoc,
+              const LocIter End);
+
+  void log(raw_ostream &OS) const override;
+  std::error_code convertToErrorCode() const override {
+    return std::make_error_code(std::errc::invalid_argument);
+  }
+  const std::string &getMessage() const { return CurMessage; }
+
+  static char ID;  // Keep llvm::Error happy.
+
+private:
+  std::string CurMessage;
+  LocIter ErrorLoc, FileEnd;
+};
+
+Error getParserError(const Twine &Expected, const LocIter CurLoc,
+                     const LocIter End);
+
+// A class holding a name - either an integer or a reference to the string.
+class IntOrString {
+private:
+  union Data {
+    uint32_t Int;
+    StringRef String;
+    Data(uint32_t Value) : Int(Value) {}
+    Data(StringRef &&Value) : String(std::move(Value)) {}
+    Data(const RCToken &Token);
+  } Data;
+  uint32_t IsInt : 1;
+
+public:
+  IntOrString() : Data(0) {}
+  IntOrString(uint32_t Value) : Data(Value), IsInt(1) {}
+  IntOrString(StringRef Value) : Data(std::move(Value)), IsInt(0) {}
+  IntOrString(const IntOrString &Other) = default;
+  IntOrString(IntOrString &&Other)
+      : Data(std::move(Other.Data)), IsInt(Other.IsInt) {}
+  IntOrString(const RCToken &Token)
+      : Data(Token), IsInt(Token.kind() == Kind::Int) {}
+
+  IntOrString &operator=(IntOrString &&Other) {
+    Data = std::move(Other.Data);
+    IsInt = Other.IsInt;
+    return *this;
+  }
+
+  friend raw_ostream &operator<<(raw_ostream &OS, const IntOrString &Item) {
+    if (Item.IsInt)
+      return OS << Item.Data.Int;
+    else
+      return OS << Item.Data.String;
+  }
+};
+
+// Base resource. All the parser classes and all statements/resources
+// should derive from this base.
+class ResourceStmtBase {
+protected:
+  IntOrString ResName;
+
+public:
+  void setName(IntOrString &&Name);
+  virtual raw_ostream &log(raw_ostream &OS) const;
+  virtual ~ResourceStmtBase();
+};
+
+// Base *. Return type of many parse() functions.
+using BasePtr = std::unique_ptr<ResourceStmtBase>;
+
+// Base **. Used as a return type in NodeEither. As return types are
+// dereferenced in many parser classes (e.g. in ConcatArgs), using single Base *
+// would upcast the returned value to Base and lose information. Therefore,
+// we need to return Base **.
+using BasePtrPtr = std::unique_ptr<BasePtr>;
+
+// Describes a concatenated sequence of specified tokens. Templated by
+// NodeClass, a class deriving from ConcatArgs (CRTP pattern).
+// This class tries to parse all arguments (ArgTs...) sequentially. After it
+// succeeds, it constructs an object of type ConcatArgs, providing to it
+// all the parsed arguments. Finally, it returns an encapsulated pointer to
+// this constructed object.
+// If any of the parser sub-invocations fails, this parser itself fails and
+// forwards the error message to the caller.
+template <typename NodeClass, typename... ArgTs>
+class ConcatArgs : public ResourceStmtBase {
+  // The tuple that stores the parsed arguments.
+  using ChildrenTuple = std::tuple<typename ArgTs::ParseType...>;
+
+  // Parses a single argument of type Type at position ParamId to tuple
+  // Children. If the result tuple is already in erroneous state, do nothing.
+  // If any error occurs when parsing, leave this error in Children variable.
+  template <size_t ParamId, typename Type>
+  static void parseOne(LocIter &Location, const LocIter End,
+                       Expected<ChildrenTuple> &Children) {
+    if (!Children)
+      return;
+
+    auto &&ParseResult = Type::parse(Location, End);
+    if (ParseResult)
+      std::get<ParamId>(*Children) = std::move(*ParseResult.get());
+    else
+      Children = ParseResult.takeError();
+  }
+
+  // Main body of parsing function. Has a parameter pack describing the
+  // sequence of integers from 0 to sizeof...(ArgTs) - 1.
+  template <size_t... Ids>
+  static Expected<std::unique_ptr<NodeClass>>
+  makePtr(LocIter &Location, const LocIter End,
+          integer_sequence<std::size_t, Ids...>) {
+    // Construct a tuple of results.
+    Expected<ChildrenTuple> Children((ChildrenTuple()));
+    // Parse the sub-expressions one-by-one. If any of them fails, the error
+    // is unchanged in subsequent evaluations and returned in Children variable.
+    LLVM_ATTRIBUTE_UNUSED
+    int Temporary[] = {(parseOne<Ids, ArgTs>(Location, End, Children), 0)...};
+
+    if (!Children)
+      return Children.takeError();
+    return make_unique<NodeClass>(
+        std::get<Ids>(std::forward<ChildrenTuple>(Children.get()))...);
+  }
+
+public:
+  static Expected<std::unique_ptr<NodeClass>> parse(LocIter &Location,
+                                                    const LocIter End) {
+    return makePtr(Location, End, index_sequence_for<ArgTs...>());
+  }
+};
+
+// Describes a Node that comes right after the comma. The parser tries
+// to read the comma. If it succeeds, proceeds to parsing the templated token
+// and returns it.
+template <typename Node> class AfterComma : public ResourceStmtBase {
+public:
+  using ParseType = typename Node::ParseType;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End) {
+    if (Location == End || Location->kind() != Kind::Comma)
+      return getParserError("comma", Location, End);
+    ++Location;
+    return Node::parse(Location, End);
+  }
+};
+
+// Describes a Node that might possibly appear in the parsed sequence.
+// Parsing this returns llvm::Optional<(result of parsing of Node)>.
+// Note that consuming at least one token when parsing Node implies committing
+// to it. In such situaion, when an error occurs, Optional fails, too.
+template <typename Node> class NodeOptional : public ResourceStmtBase {
+public:
+  using ParseType = Optional<typename Node::ParseType>;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End) {
+    const LocIter OldLoc = Location;
+    Expected<std::unique_ptr<typename Node::ParseType>> &&Result =
+        Node::parse(Location, End);
+    if (Result)
+      return make_unique<ParseType>(std::move(*Result.get()));
+    else if (Location != OldLoc)
+      return Expected<std::unique_ptr<ParseType>>(Result.takeError());
+
+    // If parsing of Node failed without advancing Location, we just didn't
+    // read anything.
+    consumeError(Result.takeError());
+    return make_unique<ParseType>();
+  }
+};
+
+// Describes a variable-length list of Nodes.
+// Returns std::vector<(result of parsing of Node)>.
+// The class reads Nodes until one of the parsing attempts fails. If any
+// token has been consumed during this single attempt, the whole List fails.
+// In the opposite case, already read Nodes (possibly none) are returned.
+template <typename Node> class NodeList : public ResourceStmtBase {
+public:
+  using ParseType = std::vector<typename Node::ParseType>;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End) {
+    LocIter CurLoc = Location;
+    std::unique_ptr<ParseType> ResultVec = make_unique<ParseType>();
+    while (true) {
+      Expected<std::unique_ptr<typename Node::ParseType>> Result =
+          Node::parse(Location, End);
+      if (!Result) {
+        auto &&Err = Result.takeError();
+        if (Location != CurLoc) {
+          return Expected<std::unique_ptr<ParseType>>(std::move(Err));
+        } else {
+          consumeError(std::move(Err));
+          break;
+        }
+      }
+      CurLoc = Location;
+      ResultVec->push_back(std::move(*Result.get()));
+    }
+    Location = CurLoc;
+    return ResultVec;
+  }
+};
+
+// Describes a variable-length list of Nodes representing a single block
+// of script: BlockBegin [Node]* BlockEnd.
+// Returns std::vector<(result of parsing of Node)>.
+// The class is analoguous to NodeList in terms of its behavior.
+template <typename Node> class NodeBlockList : public ResourceStmtBase {
+public:
+  using ParseType = typename NodeList<Node>::ParseType;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End) {
+    if (Location == End || Location->kind() != Kind::BlockBegin)
+      return getParserError("{ or BEGIN", Location, End);
+    Location++;
+
+    Expected<std::unique_ptr<ParseType>> &&Result =
+        NodeList<Node>::parse(Location, End);
+    if (!Result)
+      return std::move(Result);
+
+    if (Location == End || Location->kind() != Kind::BlockEnd)
+      return getParserError("} or END", Location, End);
+    Location++;
+
+    return std::move(Result);
+  }
+};
+
+// Describes a node that might be one of EitherTypes.
+// Returns BasePtrPtr; double pointer is necessary because other wrappers
+// (ConcatArgs, NodeList, ...) unpack a single pointer from return
+// values. To preserve the dynamic polymorphism, we need this unpacked
+// value to remain a pointer.
+//
+// This tries matching the subsequent types one by one. If any of them
+// succeeds or consumes any token, the search is terminated and the
+// return value is forwarded. If no type matches, the search fails.
+template <typename... EitherTypes> class NodeEither;
+
+template <typename Head, typename... Tail>
+class NodeEither<Head, Tail...> : public ResourceStmtBase {
+public:
+  using ParseType = BasePtr;
+
+  static Expected<BasePtrPtr> parse(LocIter &Location, const LocIter End) {
+    const LocIter StartLoc = Location;
+    auto &&Result = Head::parse(Location, End);
+    if (Result)
+      return make_unique<BasePtr>(std::move(Result.get()));
+    if (Location != StartLoc)
+      return Result.takeError();
+
+    consumeError(Result.takeError());
+    return NodeEither<Tail...>::parse(Location, End);
+  }
+};
+
+template <> class NodeEither<> : public ResourceStmtBase {
+public:
+  static Expected<BasePtrPtr> parse(LocIter &Location, const LocIter End) {
+    return getParserError("data/resource", Location, End);
+  }
+};
+
+// Describes an integer node. For now, it expects to read an integer.
+// In the future, it should read arithmetic expressions consisting of
+// tokens <int> ( ) + - | & ~.
+class NodeInt {
+public:
+  using ParseType = uint32_t;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End);
+};
+
+// Describes a node parsing string tokens.
+class NodeString {
+public:
+  using ParseType = StringRef;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End);
+};
+
+// Describes a node parsing tokens which might be either an integer,
+// string or identifier. This kind of nodes is used when naming resources:
+// msdn.microsoft.com/en-us/library/windows/desktop/aa380680(v=vs.85).aspx
+class NodeResourceName {
+public:
+  using ParseType = IntOrString;
+
+  static Expected<std::unique_ptr<ParseType>> parse(LocIter &Location,
+                                                    const LocIter End);
+};
+
+// Describes a resource or statement of the following format:
+//   FIXED_STRING Arguments...
+// where FIXED_STRING == ItemName::get() and arguments are parsed by ItemNode.
+// FIXED_STRING is compared in a case-insensitive manner.
+// An example of such item is LANGUAGE:
+//   msdn.microsoft.com/en-us/library/windows/desktop/aa381019(v=vs.85).aspx
+//
+// As soon as FIXED_STRING (name of resource) is matched, the token is
+// consumed in order to prevent other resource types from trying to parse
+// (that is, we commit to this type of item).
+template <typename ItemName, typename ItemNode>
+class UnnamedItem : public ResourceStmtBase {
+protected:
+  static bool checkEq(LocIter Location) {
+    return Location->value().equals_lower(ItemName::get());
+  }
+
+public:
+  static Expected<BasePtr> parse(LocIter &Location, const LocIter End) {
+    if (Location == End || !checkEq(Location))
+      return getParserError(ItemName::get(), Location, End);
+    return ItemNode::parse(++Location, End);
+  }
+
+  static const char *itemType() { return ItemName::get(); }
+};
+
+// Describes a resource or statement of the following format:
+//   Name FIXED_STRING Arguments...
+// where Name is parsed by NodeResourceName,
+//       FIXED_STRING == ItemName::get() (case-insensitive comparison), and
+//       Arguments are parsed by ItemNode.
+//
+// As soon as Name and FIXED_STRING are matched, the two tokens are consumed
+// to disallow other resource types from trying to parse.
+template <typename ItemName, typename ItemNode>
+class NamedItem : public UnnamedItem<ItemName, ItemNode> {
+public:
+  static Expected<BasePtr> parse(LocIter &Location, const LocIter End) {
+    if (Location == End)
+      return getParserError("resource name", Location, End);
+    if (Location->kind() != Kind::Int && Location->kind() != Kind::String &&
+        Location->kind() != Kind::Identifier)
+      return getParserError("resource name", Location, End);
+    if (!UnnamedItem<ItemName, ItemNode>::checkEq(Location + 1))
+      return getParserError(ItemName::get(), Location, End);
+    IntOrString Name(*Location);
+
+    Location += 2;
+    Expected<BasePtr> &&Result = ItemNode::parse(Location, End);
+
+    if (Result)
+      (*Result)->setName(std::move(Name));
+
+    return std::move(Result);
+  }
+};
+
+// A macro emitting IdentName class declaring its name to be Ident. This allows
+// the classes to be used in UnnamedItem and NamedItem parsers.
+#define ADD_IDENT_NAME(Ident)                                                  \
+  class Ident##Name {                                                          \
+  public:                                                                      \
+    static const char *get() { return #Ident; }                                \
+  };
+
+} // namespace node
+} // namespace llvm
+
+#endif

llvm/tools/llvm-rc/ResourceScriptParser.cpp

+//===-- ResourceScriptParser.cpp --------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This implements non-templated functions from ResourceScriptParser.h.
+//
+//===---------------------------------------------------------------------===//
+
+#include "ResourceScriptParser.h"
+
+namespace llvm {
+namespace node {
+
+char ParserError::ID = 0;
+
+ParserError::ParserError(const Twine &Expected, const LocIter CurLoc,
+                         const LocIter End) : ErrorLoc(CurLoc), FileEnd(End) {
+  CurMessage = "Error parsing file: expected " + Expected.str() + ", got " +
+      (CurLoc == End ? "<EOF>" : CurLoc->value()).str();
+}
+
+void ParserError::log(raw_ostream &OS) const {
+  OS << CurMessage;
+}
+
+void ResourceStmtBase::setName(IntOrString &&Name) {
+  ResName = std::move(Name);
+}
+
+raw_ostream &ResourceStmtBase::log(raw_ostream &OS) const {
+  return OS << "Base statement\n";
+}
+
+ResourceStmtBase::~ResourceStmtBase() {}
+
+Error getParserError(const Twine &Message, const LocIter CurLoc,
+                     const LocIter End) {
+  return make_error<ParserError>(Message, CurLoc, End);
+}
+
+IntOrString::Data::Data(const RCToken &Token) {
+  if (Token.kind() == Kind::Int)
+    Int = Token.intValue();
+  else if (Token.kind() == Kind::String || Token.kind() == Kind::Identifier)
+    String = Token.value();
+  else
+    assert(false && "Tried to get IntOrString from neither int, string "
+                    "or identifier.");
+}
+
+Expected<std::unique_ptr<NodeInt::ParseType>>
+NodeInt::parse(LocIter &Location, const LocIter End) {
+  if (Location == End || Location->kind() != Kind::Int)
+    return getParserError("integer", Location, End);
+  return make_unique<ParseType>((Location++)->intValue());
+}
+
+Expected<std::unique_ptr<NodeString::ParseType>>
+NodeString::parse(LocIter &Location, const LocIter End) {
+  if (Location == End || Location->kind() != Kind::String)
+    return getParserError("string", Location, End);
+  return make_unique<ParseType>((Location++)->value());
+}
+
+Expected<std::unique_ptr<NodeResourceName::ParseType>>
+NodeResourceName::parse(LocIter &Location, const LocIter End) {
+  if (Location == End)
+    return getParserError("resource name", Location, End);
+  const Kind LocKind = Location->kind();
+  if (LocKind == Kind::Int)
+    return make_unique<ParseType>((Location++)->intValue());
+  if (LocKind == Kind::String || LocKind == Kind::Identifier)
+    return make_unique<ParseType>((Location++)->value());
+  return getParserError("resource name", Location, End);
+}
+
+} // namespace node
+} // namespace llvm

llvm/tools/llvm-rc/ResourceScriptStmt.h

+//===-- ResourceScriptStmt.h ------------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This contains the definition of types of statements and resources occurring
+// in RC scripts. For now, only a very limited set of resources is supported.
+// However, it will be extended in the future.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVMRC_RESOURCESCRIPTSTMT_H
+#define LLVM_TOOLS_LLVMRC_RESOURCESCRIPTSTMT_H
+
+#include "ResourceScriptParser.h"
+#include "ResourceScriptToken.h"
+
+namespace llvm {
+namespace node {
+
+// --- RC statements. --- //
+
+// LANGUAGE statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381019(v=vs.85).aspx
+ADD_IDENT_NAME(Language);
+
+class LanguageStmt
+    : public UnnamedItem<LanguageName, ConcatArgs<LanguageStmt, NodeInt,
+                                                  AfterComma<NodeInt>>> {
+  uint32_t Lang, SubLang;
+
+public:
+  LanguageStmt(uint32_t LangId, uint32_t SubLangId);
+  virtual raw_ostream &log(raw_ostream &) const;
+};
+
+// CHARACTERISTICS statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380872(v=vs.85).aspx
+ADD_IDENT_NAME(Characteristics);
+
+class CharacteristicsStmt
+    : public UnnamedItem<CharacteristicsName,
+                         ConcatArgs<CharacteristicsStmt, NodeInt>> {
+  int32_t Value;
+
+public:
+  CharacteristicsStmt(uint32_t ChValue);
+  virtual raw_ostream &log(raw_ostream &) const;
+};
+
+// VERSION statement.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381059(v=vs.85).aspx
+ADD_IDENT_NAME(Version);
+
+class VersionStmt
+    : public UnnamedItem<VersionName, ConcatArgs<VersionStmt, NodeInt>> {
+  uint32_t Version;
+
+public:
+  VersionStmt(uint32_t VersionId);
+  virtual raw_ostream &log(raw_ostream &) const;
+};
+
+// A basic set of optional resource-specific statements.
+// Occurs e.g. in STRINGTABLE resource:
+//   msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+using StmtBasic = NodeEither<CharacteristicsStmt, LanguageStmt, VersionStmt>;
+
+// --- RC resources and helper statements. --- //
+
+// ICON resource.
+ADD_IDENT_NAME(Icon);
+
+class IconRes : public NamedItem<IconName, ConcatArgs<IconRes, NodeString>> {
+  StringRef Value;
+
+public:
+  IconRes(StringRef &&Name);
+  virtual raw_ostream &log(raw_ostream &) const;
+};
+
+// Single statement in STRINGTABLE resource block.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+// It is a single "stringID string" statement.
+class StringTableStmt
+    : public ConcatArgs<StringTableStmt, NodeInt, NodeString> {
+public:
+  using ParseType = StringTableStmt;
+
+  int32_t Id;
+  StringRef String;
+
+  StringTableStmt(int32_t &&SId, StringRef &&SString);
+};
+
+// STRINGTABLE resource.
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa381050(v=vs.85).aspx
+ADD_IDENT_NAME(StringTable);
+
+class StringTableRes
+    : public UnnamedItem<StringTableName,
+                         ConcatArgs<StringTableRes, NodeList<StmtBasic>,
+                                    NodeBlockList<StringTableStmt>>> {
+  std::vector<BasePtr> Options;
+  std::vector<StringTableStmt> Statements;
+
+public:
+  StringTableRes(std::vector<BasePtr> &&Opts,
+                 std::vector<StringTableStmt> &&Stmts);
+  virtual raw_ostream &log(raw_ostream &OS) const;
+};
+
+// A top-level resource definition.
+class ResourceStmt : public NodeEither<LanguageStmt, IconRes, StringTableRes> {
+};
+
+} // namespace node
+} // namespace llvm
+
+#endif

llvm/tools/llvm-rc/ResourceScriptStmt.cpp

+//===-- ResourceScriptStmt.cpp ----------------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This implements methods defined in ResourceScriptStmt.h.
+//
+// Ref: msdn.microsoft.com/en-us/library/windows/desktop/aa380599(v=vs.85).aspx
+//
+//===---------------------------------------------------------------------===//
+
+#include "ResourceScriptStmt.h"
+
+namespace llvm {
+namespace node {
+
+LanguageStmt::LanguageStmt(uint32_t LangId, uint32_t SubLangId)
+    : Lang(LangId), SubLang(SubLangId) {}
+
+raw_ostream &LanguageStmt::log(raw_ostream &OS) const {
+  return OS << itemType() << ": " << Lang << ", Sublanguage: " << SubLang
+            << "\n";
+}
+
+CharacteristicsStmt::CharacteristicsStmt(uint32_t ChValue) : Value(ChValue) {}
+
+raw_ostream &CharacteristicsStmt::log(raw_ostream &OS) const {
+  return OS << itemType() << ": " << Value << "\n";
+}
+
+VersionStmt::VersionStmt(uint32_t VersionId) : Version(VersionId) {}
+
+raw_ostream &VersionStmt::log(raw_ostream &OS) const {
+  return OS << itemType() << ": " << Version << "\n";
+}
+
+IconRes::IconRes(StringRef &&Name) : Value(std::move(Name)) {}
+
+raw_ostream &IconRes::log(raw_ostream &OS) const {
+  return OS << itemType() << " (" << ResName << "): " << Value << "\n";
+}
+
+StringTableStmt::StringTableStmt(int32_t &&SId, StringRef &&SString)
+    : Id(std::move(SId)), String(std::move(SString)) {}
+
+StringTableRes::StringTableRes(std::vector<BasePtr> &&Opts,
+                               std::vector<StringTableStmt> &&Stmts)
+    : Options(std::move(Opts)), Statements(std::move(Stmts)) {}
+
+raw_ostream &StringTableRes::log(raw_ostream &OS) const {
+  OS << itemType() << ":\n";
+  for (const BasePtr &Option : Options) {
+    OS << "  Option: ";
+    Option->log(OS);
+  }
+  for (const StringTableStmt &Stmt : Statements) {
+    OS << "  " << Stmt.Id << " => " << Stmt.String << "\n";
+  }
+  return OS;
+}
+
+} // namespace node
+} // namespace llvm

llvm/tools/llvm-rc/llvm-rc.cpp

 //===-- llvm-rc.cpp - Compile .rc scripts into .res -------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // Compile .rc scripts into .res files. This is intended to be a
 // platform-independent port of Microsoft's rc.exe tool.
 //
 //===----------------------------------------------------------------------===//
 
 #include "ResourceScriptToken.h"
+#include "ResourceScriptStmt.h"
 
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/Process.h"
 #include "llvm/Support/Signals.h"
for (const RCToken &Token : Tokens) {
              << Token.value();
       if (Token.kind() == RCToken::Kind::Int)
         outs() << "; int value = " << Token.intValue();
 
       outs() << "\n";
     }
   }
 
+  if (BeVerbose) {
+    outs() << "Parsed file:\n";
+    outs().flush();
+
+    auto TokenBegin = Tokens.begin();
+    while (TokenBegin != Tokens.end()) {
+      (*ExitOnErr(node::ResourceStmt::parse(TokenBegin, Tokens.end())))->log(outs());
+      outs().flush();
+    }
+  }
+
   return 0;
 }