[Symbolizer] Implement contextual symbolizer markup elements.

[mysterymath]

Summary

This change implements the contextual symbolizer markup elements: reset,
module, and mmap. These provide information about the runtime context of
the binary necessary to resolve addresses to symbolic values.

Summary information is printed to the output about this context.
Multiple mmap elements for the same module line are coalesced together.
The standard requires that such elements occur on their own lines to
allow for this; accordingly, anything after a contextual element on a
line is silently discarded.

Implementing this cleanly requires that the filter drive the parser;
this allows skipped sections to avoid being parsed. This also makes the
filter quite a bit easier to use, at the cost of some unused
flexibility.

[mysterymath]

Clean up diff.

[phosek]

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

130–131

I'd recommend using DenseMap which is more performant than std::map.

[mysterymath]

Use Densemap<unique_ptr> for Modules; fix flipped overlappingMMap condition.

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

130–131

For Modules, it's desirable that Module pointers should be stable; this allows module IDs to be looked up once during parsing, then the resulting pointers stored indefinitely. But this is better done via a DenseMap to unique_ptr, so I've done so.

For MMaps, the collection is ordered by address, and this is used to quickly identify overlapping MMap regions. This isn't obvious though, so I've added a comment.

[jhenderson]

llvm/docs/CommandGuide/llvm-symbolizer.rst

254–255

The term "only" implies there are many other elements that aren't supported (at least more than the number of supported elements). Is that still the case? If not, delete "only".

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

68

119

[mysterymath]

Add periods. Correct typo.

llvm/docs/CommandGuide/llvm-symbolizer.rst

254–255

Yep, still far more unimplemented than implemented. It's also not actually possble to use the markup for symbolization tasks yet, this just establishes the baseline.
SGTM on the language though; it should probably become just a list of supported elements once there's enough present to actually be useful, then a list of unsupported elements once it's past the halfway point.

[mysterymath]

preceeding -> preceding

[mysterymath]

Fixed use after free with MarginNodes. MarginNodes ended up being unnecessary,
since when a new module line is opened, the next thing to print will
necessarily be that module line. That means the next things to be output will
be the margin nodes, and these can be summarily printed.

This is also true for the prefix of the module info line, but the code seems a
bit cleaner if the info handling is centralized in flushModuleInfo().

[peter.smith]

Apologies for the delay in responding. Some small suggestions but otherwise looking good to me.

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

64

Would it be worth calling this ModuleLineInfo? or ModuleInfoLine? This would make it explicit from the name that this is a textual element and not information about Module that is permanently retained?

69

Would InlineMMaps be more descriptive? I think this is true when there is a module info line that has a module element immediately followed by mmaps for that module id. False if a new module info line is started for a previously seen module id?

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

89

Was thinking if there was a neat way of making sure that if true then all Deferred nodes had been output. Only thing I could think of was clearing Deferred in the FlushDeferred lambda and asserting that it was empty here. It would sadly mean making DeferredNodes non const in tryContextualElement so debateable whether it is worth doing.

302

Perhaps ASSIGN_OR_RETURN_NONE? Makes it a bit clearer at the call site. Not a strong opinion though.

llvm/test/DebugInfo/symbolize-filter-markup-module.test

13

Worth adding a newline to space the errors from the check as I think that would match the log below, which does have a newline in it.

[mysterymath]

Address review comments. Refactored out printing module info line prefix.

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

69

Refactored this away.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

89

We don't necessarily need to output the DeferredNodes if the function returns true. In case of errors, the line is still considered contextual, but it still might be foldable away. In this case, we wouldn't ever call FlushDeferred, but we would still return true.

[peter.smith]

Thanks for the updates. I've no more substantial comments. Will be worth waiting a bit to see if anyone else on the list have anything more to add.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

148

May be worth making it more explicit that mmap needs to be processed before the other tags due to the presence of endModuleInfoLine here. Without looking carefully it could seem like new tags can be added anywhere. Some possibilities:

• Comment in header or by mmap.
• if (Element.Tag == "mmap") { ... } else { endModuleInfoLine(); ... }
• if (Element.Tag != "mmap") { endModuleInfoLine(); } else if (ElementTag == "mmap") { ... } if (Element.Tag == "reset") { ... }

[phosek]

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

121

One potential improvement I've been thinking about would be to split the handling of different tags into separate methods. What I had in mind would be to have an enum for known tags, use llvm:StringSwitch to convert the tag to an enum and then use it for the dispatch. The only complication is endModuleInfoLine which needs a special handling. This could also be done in a future change.

[mysterymath]

Address review comments.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

121

I played around with this a bit; it seems like there's exactly one string comparison per element per known tag, so going through an enum woudn't really add anything from a performance standpoint without building a real lexer and trie-ing-up the tags.

The string-switch approach would be more readable though; along the same lines, I broke up out each tag in tryContextualElement into a try<TAG>() function. Each begins with a tag check, so the end result for tryContextualElement ends up looking more like a usual case analysis in a recursive-descent parser.

148

Ended up splitting these out into methods, so I've added a comment in the greatly-reduced tryContextualElement().

[peter.smith]

Just a couple of typos spotted in a comment.

One thing I find hard to reason about is the DeferredNodes. We add deferred nodes in tryContextualLine, but never in tryContextualElement, from that point onwards we can flush or clear only. As the latter two are read-write we can't use const to let readers know from the type signature what is and isn't expected. One possible way around this is to make deferred nodes its own type that implements more than one interface. An interface that permits adding, and one that does not (only supports flush and clear). The latter interface can be passed to tryContextualElement. Possibly overboard for this patch when there are only a few ContextualElements, but could be worth it if it gets more complicated.

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

122

typos:
conextual -> contextual
tha -> that

[mysterymath]

I did a bit more hammering on the semantics of contexutal line elision.
Previously, contextual elements would be output as text if they were preceded
by text, which wasn't intended. Fixing this means that a non-contextual line
will be entirely deferred. This means that tryContextualLine() doesn't really
make sense anymore; it always handles the whole line. I've thus made the
modified version of it into the new body of filter().

I also made the deferred nodes a const parameter of the try calls again; this
seems cleaner than moving it out into the body, since it's produced by filter()
and possibly consumed by the try() calls.

[mysterymath]

Fix indentation.

[mysterymath]

Remove unused declaration.

[mysterymath]

Remove unnecessary flag from test RUN: lines.

[phosek]

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

121

FYI the code generated by llvm::StringSwitch is more efficient than plain string comparison, the pattern used by llvm::StringSwitch implementation was chosen deliberately to be recognized by the optimizer. However, I don't think the string comparisons used here are performance critical so the difference is probably going to be in the noise.

[mysterymath]

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

122

Removed.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

121

Ah, that's good to know; I had thought it suspicious that StringSwitch even existed, given that it doesn't tend to make the surrounding code that much clearer than if/else. I didn't know that the C++ optimizer could do detailed looks into things like StringRef's, but it makes sense in retrospect.

[peter.smith]

Thanks for the update, I prefer the deferred nodes approach you have. I've only got a couple of optional suggestions.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

45

I think it would be useful to retain the information in the previous comment

// See if the line consists of whitespace and SGR nodes followed by a contextual
// element. In this case, anything after the contextual element can be elided,
// and the entire line might be folded away with the next contextual element.

Without this contextual line looks like it is a term that isn't really defined, other than by guessing from the context.

53

I had to look at the definition of endModuleLine() to check what happened if there was no module to end. Could be worth breaking the symmetry and changing the name to something like endAnyModuleInfoLine which implies that there may not be one? Alternatively // This was not a contextual line. Output any pending ModuleInfo.

[mysterymath]

Address review comments.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

45

Done; updated the text to match the new semantics.

53

I like endAnyModuleInfoLine; it captures the notion I was trying to convey better than endModuleLine. Done.

[peter.smith]

Thanks for the update. I've no more comments. I'm happy with the changes.

[mstorsjo]

This broke building with GCC 9:

In file included from ../tools/llvm-symbolizer/llvm-symbolizer.cpp:23:
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:55:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::MMap::Module’ changes meaning of ‘Module’ [-fpermissive]
   55 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:65:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::ModuleInfoLine::Module’ changes meaning of ‘Module’ [-fpermissive]
   65 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~

[mysterymath]

In D129519#3669649, @mstorsjo wrote:

This broke building with GCC 9:

In file included from ../tools/llvm-symbolizer/llvm-symbolizer.cpp:23:
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:55:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::MMap::Module’ changes meaning of ‘Module’ [-fpermissive]
   55 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:65:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::ModuleInfoLine::Module’ changes meaning of ‘Module’ [-fpermissive]
   65 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~

Sorry about that, see https://github.com/llvm/llvm-project/commit/6605187103a2369418d014a7f146fee4a04b11bf

[mstorsjo]

In D129519#3669684, @mysterymath wrote:

In D129519#3669649, @mstorsjo wrote:

This broke building with GCC 9:

In file included from ../tools/llvm-symbolizer/llvm-symbolizer.cpp:23:
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:55:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::MMap::Module’ changes meaning of ‘Module’ [-fpermissive]
   55 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:65:19: error: declaration of ‘const llvm::symbolize::MarkupFilter::Module* llvm::symbolize::MarkupFilter::ModuleInfoLine::Module’ changes meaning of ‘Module’ [-fpermissive]
   65 |     const Module *Module;
      |                   ^~~~~~
../include/llvm/DebugInfo/Symbolize/MarkupFilter.h:46:10: note: ‘Module’ declared here as ‘struct llvm::symbolize::MarkupFilter::Module’
   46 |   struct Module {
      |          ^~~~~~

Sorry about that, see https://github.com/llvm/llvm-project/commit/6605187103a2369418d014a7f146fee4a04b11bf

Thanks, that seems to have done the trick!

[dyung]

One of the tests you added symbolize-filter-markup-reset.test seems to be failing on the PS5 Windows bot, can you take a look?

https://lab.llvm.org/staging/#/builders/204/builds/1751

[mysterymath]

In D129519#3669837, @dyung wrote:

One of the tests you added symbolize-filter-markup-reset.test seems to be failing on the PS5 Windows bot, can you take a look?

https://lab.llvm.org/staging/#/builders/204/builds/1751

Looking on my Windows box; I'll revert if I can't get to the bottom of it in a few minutes.

[mysterymath]

In D129519#3669903, @mysterymath wrote:

In D129519#3669837, @dyung wrote:

One of the tests you added symbolize-filter-markup-reset.test seems to be failing on the PS5 Windows bot, can you take a look?

https://lab.llvm.org/staging/#/builders/204/builds/1751

Looking on my Windows box; I'll revert if I can't get to the bottom of it in a few minutes.

Seems to have been a use-after-free; this should be fixed by cc0a1078f5fb53638e6c125f3be2728dfe84e129.

llvm/docs/CommandGuide/llvm-symbolizer.rst

.. option:: --fallback-debug-path <path>
   When a separate file contains debug data, and is referenced by a GNU debug
   link section, use the specified path as a basis for locating the debug data if
   it cannot be found relative to the object.
 
 .. option:: --filter-markup
 
   Reads from standard input, converts contained
   :doc:`Symbolizer Markup </SymbolizerMarkupFormat>` into human-readable form,
   and prints the results to standard output. Presently, only the following
   markup elements are supported:
 
   * ``{{symbol}}``
+  * ``{{reset}}``
+  * ``{{module}}``
+  * ``{{mmap}}``
 
 .. _llvm-symbolizer-opt-f:
 
 .. option:: --functions [=<none|short|linkage>], -f
 
   Specify the way function names are printed (omit function name, print short
   function name, or print full linkage name, respectively). Defaults to
   ``linkage``.

llvm/docs/SymbolizerMarkupFormat.rst

 
 Contextual elements should appear in the logging stream before they are needed.
 That is, if some piece of context may affect how the symbolizing filter would
 interpret or present a later presentation element, the necessary contextual
 elements should have appeared somewhere earlier in the logging stream. It should
 always be possible for the symbolizing filter to be implemented as a single pass
 over the raw logging stream, accumulating context and massaging text as it goes.
 
-``{{{reset}}}`` [#not_yet_implemented]_
+``{{{reset}}}``
 
   This should be output before any other contextual element. The need for this
   contextual element is to support implementations that handle logs coming from
   multiple processes. Such implementations might not know when a new process
   starts or ends. Because some identifying information (like process IDs) might
   be the same between old and new processes, a way is needed to distinguish two
   processes with such identical identifying information. This element informs
   such implementations to reset the state of a filter so that information from a
   previous process's contextual elements is not assumed for new process that
   just happens have the same identifying information.
 
-``{{{module:%i:%s:%s:...}}}`` [#not_yet_implemented]_
+``{{{module:%i:%s:%s:...}}}``
 
   This element represents a so-called "module". A "module" is a single linked
   binary, such as a loaded ELF file. Usually each module occupies a contiguous
   range of memory.
 
   Here ``%i`` is the module ID which is used by other contextual elements to
   refer to this module. The first ``%s`` is a human-readable identifier for the
   module, such as an ELF ``DT_SONAME`` string or a file name; but it might be
``{{{module:%i:%s:%s:...}}}``
   Here ``%x`` encodes an ELF Build ID. The Build ID should refer to a single
   linked binary. The Build ID string is the sole way to identify the binary from
   which this module was loaded.
 
   Example::
 
     {{{module:1:libc.so:elf:83238ab56ba10497}}}
 
-``{{{mmap:%p:%i:...}}}`` [#not_yet_implemented]_
+``{{{mmap:%p:%i:...}}}``
 
   This contextual element is used to give information about a particular region
   in memory. ``%p`` is the starting address and ``%i`` gives the size in hex of the
   region of memory. The ``...`` part can take different forms to give different
   information about the specified region of memory. The allowed forms are the
   following:
 
   * ``load:%i:%s:%p``

llvm/include/llvm/DebugInfo/Symbolize/Markup.h

public:
   /// Returns the next node in the input sequence.
   ///
   /// Calling nextNode() may invalidate the contents of the node returned by the
   /// previous call.
   ///
   /// \returns the next markup node or None if none remain.
   Optional<MarkupNode> nextNode();
 
+  bool isSGR(const MarkupNode &Node) const {
+    return SGRSyntax.match(Node.Text);
+  }
+
 private:
   Optional<MarkupNode> parseElement(StringRef Line);
   void parseTextOutsideMarkup(StringRef Text);
   Optional<StringRef> parseMultiLineBegin(StringRef Line);
   Optional<StringRef> parseMultiLineEnd(StringRef Line);
 
   // Tags of elements that can span multiple lines.
   const StringSet<> MultilineTags;

llvm/include/llvm/DebugInfo/Symbolize/MarkupFilter.h

 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_SYMBOLIZE_MARKUPFILTER_H
 #define LLVM_DEBUGINFO_SYMBOLIZE_MARKUPFILTER_H
 
 #include "Markup.h"
 
+#include <map>
+
+#include "llvm/ADT/DenseMap.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace llvm {
 namespace symbolize {
 
 /// Filter to convert parsed log symbolizer markup elements into human-readable
 /// text.
 class MarkupFilter {
 public:
   MarkupFilter(raw_ostream &OS, Optional<bool> ColorsEnabled = llvm::None);
 
-  /// Begins a logical \p Line of markup.
-  ///
-  /// This must be called for each line of the input stream before calls to
-  /// filter() for elements of that line. The provided \p Line must be the same
-  /// one that was passed to parseLine() to produce the elements to be later
-  /// passed to filter().
+  /// Filters a line containing symbolizer markup and writes the human-readable
+  /// results to the output stream.
   ///
-  /// This informs the filter that a new line is beginning and establishes a
-  /// context for error location reporting.
-  void beginLine(StringRef Line);
+  /// Invalid or unimplemented markup elements are removed. Some output may be
+  /// deferred until future filter() or finish() call.
+  void filter(StringRef Line);
 
-  /// Handle a \p Node of symbolizer markup.
-  ///
-  /// If the node is a recognized, valid markup element, it is replaced with a
-  /// human-readable string. If the node isn't an element or the element isn't
-  /// recognized, it is output verbatim. If the element is recognized but isn't
-  /// valid, it is omitted from the output.
-  void filter(const MarkupNode &Node);
+  /// Records that the input stream has ended and writes any deferred output.
+  void finish();
 
 private:
+  struct Module {
+    uint64_t ID;
+    std::string Name;
+    SmallVector<uint8_t> BuildID;
+  };
+
+  struct MMap {
+    uint64_t Addr;
+    uint64_t Size;
+    const Module *Module;
+    std::string Mode; // Lowercase
+    uint64_t ModuleRelativeAddr;
+
+    bool contains(uint64_t Addr) const;
+  };
+
+  // An informational module line currently being constructed. As many mmap
+  // elements as possible are folded into one ModuleInfo line.
+  struct ModuleInfoLine {
+    const Module *Module;
+
+    SmallVector<const MMap *> MMaps = {};
+  };
+
+  bool tryContextualElement(const MarkupNode &Node,
+                            const SmallVector<MarkupNode> &DeferredNodes);
+  bool tryMMap(const MarkupNode &Element,
+               const SmallVector<MarkupNode> &DeferredNodes);
+  bool tryReset(const MarkupNode &Element,
+                const SmallVector<MarkupNode> &DeferredNodes);
+  bool tryModule(const MarkupNode &Element,
+                 const SmallVector<MarkupNode> &DeferredNodes);
+
+  void beginModuleInfoLine(const Module *M);
+  void endAnyModuleInfoLine();
+
+  void filterNode(const MarkupNode &Node);
+
+  bool tryPresentation(const MarkupNode &Node);
+  bool trySymbol(const MarkupNode &Node);
+
   bool trySGR(const MarkupNode &Node);
 
   void highlight();
+  void highlightValue();
   void restoreColor();
   void resetColor();
 
+  Optional<Module> parseModule(const MarkupNode &Element) const;
+  Optional<MMap> parseMMap(const MarkupNode &Element) const;
+
+  Optional<uint64_t> parseAddr(StringRef Str) const;
+  Optional<uint64_t> parseModuleID(StringRef Str) const;
+  Optional<uint64_t> parseSize(StringRef Str) const;
+  Optional<SmallVector<uint8_t>> parseBuildID(StringRef Str) const;
+  Optional<std::string> parseMode(StringRef Str) const;
+
   bool checkTag(const MarkupNode &Node) const;
-  bool checkNumFields(const MarkupNode &Node, size_t Size) const;
+  bool checkNumFields(const MarkupNode &Element, size_t Size) const;
+  bool checkNumFieldsAtLeast(const MarkupNode &Element, size_t Size) const;
 
   void reportTypeError(StringRef Str, StringRef TypeName) const;
   void reportLocation(StringRef::iterator Loc) const;
 
+  const MMap *overlappingMMap(const MMap &Map) const;
+
+  StringRef lineEnding() const;
+
   raw_ostream &OS;
   const bool ColorsEnabled;
 
+  MarkupParser Parser;
+
+  // Current line being filtered.
   StringRef Line;
 
+  // A module info line currently being built. This incorporates as much mmap
+  // information as possible before being emitted.
+  Optional<ModuleInfoLine> MIL;
+
+  // SGR state.
   Optional<raw_ostream::Colors> Color;
   bool Bold = false;
+
+  // Map from Module ID to Module.
+  DenseMap<uint64_t, std::unique_ptr<Module>> Modules;
+
+  // Ordered map from starting address to mmap.
+  std::map<uint64_t, MMap> MMaps;
 };
 
 } // end namespace symbolize
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_SYMBOLIZE_MARKUPFILTER_H

llvm/lib/DebugInfo/Symbolize/Markup.cpp

Optional<MarkupNode> MarkupParser::nextNode() {
 
   // The line doesn't contain any more markup elements, so emit it as text.
   parseTextOutsideMarkup(Line);
   Line = Line.drop_front(Line.size());
   return nextNode();
 }
 
 void MarkupParser::flush() {
+  Buffer.clear();
+  NextIdx = 0;
+  Line = {};
   if (InProgressMultiline.empty())
     return;
   FinishedMultiline.swap(InProgressMultiline);
   parseTextOutsideMarkup(FinishedMultiline);
 }
 
 // Finds and returns the next valid markup element in the given line. Returns
 // None if the line contains no valid elements.

llvm/lib/DebugInfo/Symbolize/MarkupFilter.cpp

 //===-- lib/DebugInfo/Symbolize/MarkupFilter.cpp -------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines the implementation of a filter that replaces symbolizer
 /// markup with human-readable expressions.
 ///
+/// See https://llvm.org/docs/SymbolizerMarkupFormat.html
+///
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/Symbolize/MarkupFilter.h"
 
 #include "llvm/ADT/None.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringSwitch.h"
+#include "llvm/DebugInfo/Symbolize/Markup.h"
+#include "llvm/Debuginfod/Debuginfod.h"
 #include "llvm/Demangle/Demangle.h"
+#include "llvm/Object/ObjectFile.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 using namespace llvm::symbolize;
 
 MarkupFilter::MarkupFilter(raw_ostream &OS, Optional<bool> ColorsEnabled)
     : OS(OS), ColorsEnabled(ColorsEnabled.value_or(
                   WithColor::defaultAutoDetectFunction()(OS))) {}
 
-void MarkupFilter::beginLine(StringRef Line) {
+void MarkupFilter::filter(StringRef Line) {
   this->Line = Line;
   resetColor();
+
+  Parser.parseLine(Line);
+  SmallVector<MarkupNode> DeferredNodes;
+  // See if the line is a contextual (i.e. contains a contextual element).
+  // In this case, anything after the contextual element is elided, or the whole
+  // line may be elided.
+  while (Optional<MarkupNode> Node = Parser.nextNode()) {
+    // If this was a contextual line, then summarily stop processing.
+    if (tryContextualElement(*Node, DeferredNodes))
+      return;
+    // This node may yet be part of an elided contextual line.
+    DeferredNodes.push_back(*Node);
   }
 
-void MarkupFilter::filter(const MarkupNode &Node) {
-  if (!checkTag(Node))
+  // This was not a contextual line, so nothing in it should be elided.
+  endAnyModuleInfoLine();
+  for (const MarkupNode &Node : DeferredNodes)
+    filterNode(Node);
+}
+
+void MarkupFilter::finish() {
+  Parser.flush();
+  while (Optional<MarkupNode> Node = Parser.nextNode())
+    filterNode(*Node);
+  endAnyModuleInfoLine();
+  resetColor();
+  Modules.clear();
+  MMaps.clear();
+}
+
+// See if the given node is a contextual element and handle it if so. This may
+// either output or defer the element; in the former case, it will first emit
+// any DeferredNodes.
+//
+// Returns true if the given element was a contextual element. In this case,
+// DeferredNodes should be considered handled and should not be emitted. The
+// rest of the containing line must also be ignored in case the element was
+// deferred to a following line.
+bool MarkupFilter::tryContextualElement(
+    const MarkupNode &Node, const SmallVector<MarkupNode> &DeferredNodes) {
+  if (tryMMap(Node, DeferredNodes))
+    return true;
+  if (tryReset(Node, DeferredNodes))
+    return true;
+  return tryModule(Node, DeferredNodes);
+}
+
+bool MarkupFilter::tryMMap(const MarkupNode &Node,
+                           const SmallVector<MarkupNode> &DeferredNodes) {
+  if (Node.Tag != "mmap")
+    return false;
+  Optional<MMap> ParsedMMap = parseMMap(Node);
+  if (!ParsedMMap)
+    return true;
+
+  if (const MMap *M = overlappingMMap(*ParsedMMap)) {
+    WithColor::error(errs())
+        << formatv("overlapping mmap: #{0:x} [{1:x},{2:x})\n", M->Module->ID,
+                   M->Addr, M->Addr + M->Size);
+    reportLocation(Node.Fields[0].begin());
+    return true;
+  }
+
+  auto Res = MMaps.emplace(ParsedMMap->Addr, std::move(*ParsedMMap));
+  assert(Res.second && "Overlap check should ensure emplace succeeds.");
+  MMap &MMap = Res.first->second;
+
+  if (!MIL || MIL->Module != MMap.Module) {
+    endAnyModuleInfoLine();
+    for (const MarkupNode &Node : DeferredNodes)
+      filterNode(Node);
+    beginModuleInfoLine(MMap.Module);
+    OS << "; adds";
+  }
+  MIL->MMaps.push_back(&MMap);
+  return true;
+}
+
+bool MarkupFilter::tryReset(const MarkupNode &Node,
+                            const SmallVector<MarkupNode> &DeferredNodes) {
+  if (Node.Tag != "reset")
+    return false;
+  if (!checkNumFields(Node, 0))
+    return true;
+
+  if (!Modules.empty() || !MMaps.empty()) {
+    Modules.clear();
+    MMaps.clear();
+
+    endAnyModuleInfoLine();
+    for (const MarkupNode &Node : DeferredNodes)
+      filterNode(Node);
+    highlight();
+    OS << "[[[reset]]]" << lineEnding();
+    restoreColor();
+  }
+  return true;
+}
+
+bool MarkupFilter::tryModule(const MarkupNode &Node,
+                             const SmallVector<MarkupNode> &DeferredNodes) {
+  if (Node.Tag != "module")
+    return false;
+  Optional<Module> ParsedModule = parseModule(Node);
+  if (!ParsedModule)
+    return true;
+
+  auto Res = Modules.try_emplace(
+      ParsedModule->ID, std::make_unique<Module>(std::move(*ParsedModule)));
+  if (!Res.second) {
+    WithColor::error(errs()) << "duplicate module ID\n";
+    reportLocation(Node.Fields[0].begin());
+    return true;
+  }
+  Module &Module = *Res.first->second;
+
+  endAnyModuleInfoLine();
+  for (const MarkupNode &Node : DeferredNodes)
+    filterNode(Node);
+  beginModuleInfoLine(&Module);
+  OS << "; BuildID=";
+  highlightValue();
+  OS << toHex(Module.BuildID, /*LowerCase=*/true);
+  highlight();
+  return true;
+}
+
+void MarkupFilter::beginModuleInfoLine(const Module *M) {
+  highlight();
+  OS << "[[[ELF module";
+  highlightValue();
+  OS << formatv(" #{0:x} \"{1}\"", M->ID, M->Name);
+  highlight();
+  MIL = ModuleInfoLine{M};
+}
+
+void MarkupFilter::endAnyModuleInfoLine() {
+  if (!MIL)
     return;
+  llvm::stable_sort(MIL->MMaps, [](const MMap *A, const MMap *B) {
+    return A->Addr < B->Addr;
+  });
+  for (const MMap *M : MIL->MMaps) {
+    OS << (M == MIL->MMaps.front() ? ' ' : '-');
+    highlightValue();
+    OS << formatv("{0:x}", M->Addr);
+    highlight();
+    OS << '(';
+    highlightValue();
+    OS << M->Mode;
+    highlight();
+    OS << ')';
+  }
+  OS << "]]]" << lineEnding();
+  restoreColor();
+  MIL.reset();
+}
 
+// Handle a node that is known not to be a contextual element.
+void MarkupFilter::filterNode(const MarkupNode &Node) {
+  if (!checkTag(Node))
+    return;
+  if (tryPresentation(Node))
+    return;
   if (trySGR(Node))
     return;
 
-  if (Node.Tag == "symbol") {
-    if (!checkNumFields(Node, 1))
-      return;
-    highlight();
-    OS << llvm::demangle(Node.Fields.front().str());
-    restoreColor();
-    return;
-  }
-
-  OS << Node.Text;
+  OS << Node.Text;
+}
+
+bool MarkupFilter::tryPresentation(const MarkupNode &Node) {
+  return trySymbol(Node);
+}
+
+bool MarkupFilter::trySymbol(const MarkupNode &Node) {
+  if (Node.Tag != "symbol")
+    return false;
+    return true;
+
+  return true;
 }
 
 bool MarkupFilter::trySGR(const MarkupNode &Node) {
   if (Node.Text == "\033[0m") {
     resetColor();
     return true;
   }
   if (Node.Text == "\033[1m") {

 void MarkupFilter::highlight() {
   if (!ColorsEnabled)
     return;
   OS.changeColor(Color == raw_ostream::Colors::BLUE ? raw_ostream::Colors::CYAN
                                                     : raw_ostream::Colors::BLUE,
                  Bold);
 }
 
+// Begin highlighting a field within a highlighted markup string.
+void MarkupFilter::highlightValue() {
+  if (!ColorsEnabled)
+    return;
+  OS.changeColor(raw_ostream::Colors::GREEN, Bold);
+}
+
 // Set the output stream's color to the current color and bold state of the SGR
 // abstract machine.
 void MarkupFilter::restoreColor() {
   if (!ColorsEnabled)
     return;
   if (Color) {
     OS.changeColor(*Color, Bold);
   } else {
     OS.resetColor();
     if (Bold)
       OS.changeColor(raw_ostream::Colors::SAVEDCOLOR, Bold);
   }
 }
 
 // Set the SGR and output stream's color and bold states back to the default.
 void MarkupFilter::resetColor() {
   if (!Color && !Bold)
     return;
   Color.reset();
   Bold = false;
   if (ColorsEnabled)
     OS.resetColor();
 }
 
+// This macro helps reduce the amount of indirection done through Optional
+// below, since the usual case upon returning a None Optional is to return None.
+#define ASSIGN_OR_RETURN_NONE(TYPE, NAME, EXPR)                                \
+  auto NAME##Opt = (EXPR);                                                     \
+  if (!NAME##Opt)                                                              \
+    return None;                                                               \
+  TYPE NAME = std::move(*NAME##Opt)
+
+Optional<MarkupFilter::Module>
+MarkupFilter::parseModule(const MarkupNode &Element) const {
+  if (!checkNumFieldsAtLeast(Element, 3))
+    return None;
+  ASSIGN_OR_RETURN_NONE(uint64_t, ID, parseModuleID(Element.Fields[0]));
+  StringRef Name = Element.Fields[1];
+  StringRef Type = Element.Fields[2];
+  if (Type != "elf") {
+    WithColor::error() << "unknown module type\n";
+    reportLocation(Type.begin());
+    return None;
+  }
+  if (!checkNumFields(Element, 4))
+    return None;
+  ASSIGN_OR_RETURN_NONE(SmallVector<uint8_t>, BuildID,
+                        parseBuildID(Element.Fields[3]));
+  return Module{ID, Name.str(), std::move(BuildID)};
+}
+
+Optional<MarkupFilter::MMap>
+MarkupFilter::parseMMap(const MarkupNode &Element) const {
+  if (!checkNumFieldsAtLeast(Element, 3))
+    return None;
+  ASSIGN_OR_RETURN_NONE(uint64_t, Addr, parseAddr(Element.Fields[0]));
+  ASSIGN_OR_RETURN_NONE(uint64_t, Size, parseSize(Element.Fields[1]));
+  StringRef Type = Element.Fields[2];
+  if (Type != "load") {
+    WithColor::error() << "unknown mmap type\n";
+    reportLocation(Type.begin());
+    return None;
+  }
+  if (!checkNumFields(Element, 6))
+    return None;
+  ASSIGN_OR_RETURN_NONE(uint64_t, ID, parseModuleID(Element.Fields[3]));
+  ASSIGN_OR_RETURN_NONE(std::string, Mode, parseMode(Element.Fields[4]));
+  auto It = Modules.find(ID);
+  if (It == Modules.end()) {
+    WithColor::error() << "unknown module ID\n";
+    reportLocation(Element.Fields[3].begin());
+    return None;
+  }
+  ASSIGN_OR_RETURN_NONE(uint64_t, ModuleRelativeAddr,
+                        parseAddr(Element.Fields[5]));
+  return MMap{Addr, Size, It->second.get(), std::move(Mode),
+              ModuleRelativeAddr};
+}
+
+// Parse an address (%p in the spec).
+Optional<uint64_t> MarkupFilter::parseAddr(StringRef Str) const {
+  if (Str.empty()) {
+    reportTypeError(Str, "address");
+    return None;
+  }
+  if (all_of(Str, [](char C) { return C == '0'; }))
+    return 0;
+  if (!Str.startswith("0x")) {
+    reportTypeError(Str, "address");
+    return None;
+  }
+  uint64_t Addr;
+  if (Str.drop_front(2).getAsInteger(16, Addr)) {
+    reportTypeError(Str, "address");
+    return None;
+  }
+  return Addr;
+}
+
+// Parse a module ID (%i in the spec).
+Optional<uint64_t> MarkupFilter::parseModuleID(StringRef Str) const {
+  uint64_t ID;
+  if (Str.getAsInteger(0, ID)) {
+    reportTypeError(Str, "module ID");
+    return None;
+  }
+  return ID;
+}
+
+// Parse a size (%i in the spec).
+Optional<uint64_t> MarkupFilter::parseSize(StringRef Str) const {
+  uint64_t ID;
+  if (Str.getAsInteger(0, ID)) {
+    reportTypeError(Str, "size");
+    return None;
+  }
+  return ID;
+}
+
+// Parse a build ID (%x in the spec).
+Optional<SmallVector<uint8_t>> MarkupFilter::parseBuildID(StringRef Str) const {
+  std::string Bytes;
+  if (Str.empty() || Str.size() % 2 || !tryGetFromHex(Str, Bytes)) {
+    reportTypeError(Str, "build ID");
+    return None;
+  }
+  ArrayRef<uint8_t> BuildID(reinterpret_cast<const uint8_t *>(Bytes.data()),
+                            Bytes.size());
+  return SmallVector<uint8_t>(BuildID.begin(), BuildID.end());
+}
+
+// Parses the mode string for an mmap element.
+Optional<std::string> MarkupFilter::parseMode(StringRef Str) const {
+  if (Str.empty()) {
+    reportTypeError(Str, "mode");
+    return None;
+  }
+
+  // Pop off each of r/R, w/W, and x/X from the front, in that order.
+  StringRef Remainder = Str;
+  if (!Remainder.empty() && tolower(Remainder.front()) == 'r')
+    Remainder = Remainder.drop_front();
+  if (!Remainder.empty() && tolower(Remainder.front()) == 'w')
+    Remainder = Remainder.drop_front();
+  if (!Remainder.empty() && tolower(Remainder.front()) == 'x')
+    Remainder = Remainder.drop_front();
+
+  // If anything remains, then the string wasn't a mode.
+  if (!Remainder.empty()) {
+    reportTypeError(Str, "mode");
+    return None;
+  }
+
+  // Normalize the mode.
+  return Str.lower();
+}
+
 bool MarkupFilter::checkTag(const MarkupNode &Node) const {
   if (any_of(Node.Tag, [](char C) { return C < 'a' || C > 'z'; })) {
     WithColor::error(errs()) << "tags must be all lowercase characters\n";
     reportLocation(Node.Tag.begin());
     return false;
   }
   return true;
 }
 
-bool MarkupFilter::checkNumFields(const MarkupNode &Node, size_t Size) const {
-  if (Node.Fields.size() != Size) {
+bool MarkupFilter::checkNumFields(const MarkupNode &Element,
+                                  size_t Size) const {
+  if (Element.Fields.size() != Size) {
     WithColor::error(errs()) << "expected " << Size << " fields; found "
-                             << Node.Fields.size() << "\n";
-    reportLocation(Node.Tag.end());
+                             << Element.Fields.size() << "\n";
+    reportLocation(Element.Tag.end());
+    return false;
+  }
+  return true;
+}
+
+bool MarkupFilter::checkNumFieldsAtLeast(const MarkupNode &Element,
+                                         size_t Size) const {
+  if (Element.Fields.size() < Size) {
+    WithColor::error(errs())
+        << "expected at least " << Size << " fields; found "
+        << Element.Fields.size() << "\n";
+    reportLocation(Element.Tag.end());
     return false;
   }
   return true;
 }
 
+void MarkupFilter::reportTypeError(StringRef Str, StringRef TypeName) const {
+  WithColor::error(errs()) << "expected " << TypeName << "; found '" << Str
+                           << "'\n";
+  reportLocation(Str.begin());
+}
+
+// Prints two lines that point out the given location in the current Line using
+// a caret. The iterator must be within the bounds of the most recent line
+// passed to beginLine().
 void MarkupFilter::reportLocation(StringRef::iterator Loc) const {
   errs() << Line;
   WithColor(errs().indent(Loc - Line.begin()), HighlightColor::String) << '^';
   errs() << '\n';
 }
+
+// Checks for an existing mmap that overlaps the given one and returns a
+// pointer to one of them.
+const MarkupFilter::MMap *MarkupFilter::overlappingMMap(const MMap &Map) const {
+  // If the given map contains the start of another mmap, they overlap.
+  auto I = MMaps.upper_bound(Map.Addr);
+  if (I != MMaps.end() && Map.contains(I->second.Addr))
+    return &I->second;
+
+  // If no element starts inside the given mmap, the only possible overlap would
+  // be if the preceding mmap contains the start point of the given mmap.
+  if (I != MMaps.begin()) {
+    --I;
+    if (I->second.contains(Map.Addr))
+      return &I->second;
+  }
+  return nullptr;
+}
+
+StringRef MarkupFilter::lineEnding() const {
+  return Line.endswith("\r\n") ? "\r\n" : "\n";
+}
+
+bool MarkupFilter::MMap::contains(uint64_t Addr) const {
+  return this->Addr <= Addr && Addr < this->Addr + Size;
+}

llvm/test/DebugInfo/symbolize-filter-markup-context-line-elision.test

+RUN: split-file %s %t
+RUN: llvm-symbolizer --filter-markup < %t/log | \
+RUN:   FileCheck --match-full-lines --implicit-check-not {{.}} \
+RUN:     --strict-whitespace %s
+
+CHECK:keep[[BEGIN:\[{3}]]ELF module #0x0 "a.o"; BuildID=ab 0x0(r)[[END:\]{3}]]
+CHECK:keep[[BEGIN]]ELF module #0x1 "b.o"; BuildID=cd[[END]]
+
+;--- log
+keep{{{module:0:a.o:elf:ab}}}skip
+skip{{{mmap:0:1:load:0:r:0}}}skip
+keep{{{module:1:b.o:elf:cd}}}skip

llvm/test/DebugInfo/symbolize-filter-markup-error-location.test

 RUN: split-file %s %t
-RUN: llvm-symbolizer --debug-file-directory=%p/Inputs --filter-markup < %t/log > /dev/null 2> %t.err
+RUN: llvm-symbolizer --filter-markup < %t/log > /dev/null 2> %t.err
 RUN: FileCheck %s -input-file=%t.err --match-full-lines --strict-whitespace
 
 CHECK:error: expected 1 fields; found 0
 CHECK:[[BEGIN:[{]{3}]]symbol[[END:[}]{3}]]
 CHECK:         ^
 CHECK:error: expected 1 fields; found 0
 CHECK:foo[[BEGIN]]symbol[[END]]bar[[BEGIN]]symbol[[END]]baz
 CHECK:            ^
 CHECK:error: expected 1 fields; found 0
 CHECK:foo[[BEGIN]]symbol[[END]]bar[[BEGIN]]symbol[[END]]baz
 CHECK:                           ^
 
 ;--- log
 {{{symbol}}}
 foo{{{symbol}}}bar{{{symbol}}}baz

llvm/test/DebugInfo/symbolize-filter-markup-mmap.test

+RUN: split-file %s %t
+RUN: llvm-symbolizer --filter-markup < %t/log > %t.out 2> %t.err
+RUN: FileCheck %s --input-file=%t.out --match-full-lines \
+RUN:   --implicit-check-not {{.}}
+RUN: FileCheck %s --check-prefix=ERR -input-file=%t.err --match-full-lines
+
+CHECK: [[BEGIN:\[{3}]]ELF module #0x0 "a.o"; BuildID=abb50d82b6bdc861 0x0(rwx)-0x1(r)-0x2(w)-0x3(x)-0x4(rwx)-0xa(r)[[END:\]{3}]]
+
+ERR: error: expected at least 3 fields; found 0
+ERR: error: unknown mmap type
+ERR: error: expected 6 fields; found 3
+ERR: error: expected address; found '1'
+ERR: error: expected size; found '-1'
+ERR: error: expected mode; found ''
+ERR: error: expected mode; found 'g'
+ERR: error: expected mode; found 'wr'
+ERR: error: overlapping mmap: #0x0 [0xa,0xc)
+ERR: error: overlapping mmap: #0x0 [0xa,0xc)
+ERR: error: overlapping mmap: #0x0 [0xa,0xc)
+
+;--- log
+{{{module:0:a.o:elf:abb50d82b6bdc861}}}
+{{{mmap:0x1:1:load:0:r:0}}}
+{{{mmap:0x2:1:load:0:w:0}}}
+{{{mmap:0x3:1:load:0:x:0}}}
+{{{mmap:0x4:1:load:0:rwx:0}}}
+{{{mmap:0x0:1:load:0:RWX:0}}}
+{{{mmap:0xa:2:load:0:r:0}}}
+
+{{{mmap}}}
+{{{mmap:0:1:unknown}}}
+{{{mmap:0:10000000:load}}}
+{{{mmap:1:10000000:load:0:r:0}}}
+{{{mmap:0:-1:load:0:r:0}}}
+{{{mmap:0:10000000:load:0::0}}}
+{{{mmap:0:10000000:load:0:g:0}}}
+{{{mmap:0:10000000:load:0:wr:0}}}
+{{{mmap:0xa:1:load:0:r:0}}}
+{{{mmap:0x9:2:load:0:r:0}}}
+{{{mmap:0x9:5:load:0:r:0}}}

llvm/test/DebugInfo/symbolize-filter-markup-module.test

+RUN: split-file %s %t
+RUN: llvm-symbolizer --filter-markup < %t/log > %t.out 2> %t.err
+RUN: FileCheck %s --input-file=%t.out --match-full-lines \
+RUN:   --implicit-check-not {{.}}
+RUN: FileCheck %s --check-prefix=ERR -input-file=%t.err --match-full-lines
+
+CHECK: [[BEGIN:\[{3}]]ELF module #0x0 "a.o"; BuildID=ab[[END:\]{3}]]
+CHECK: [[BEGIN]]ELF module #0x1 "b.o"; BuildID=abb50d82b6bdc861[[END]]
+CHECK: [[BEGIN]]ELF module #0x2 "c.o"; BuildID=cd[[END]]
+CHECK: [[BEGIN]]ELF module #0x1 "b.o"; adds 0x0(r)[[END]]
+
+ERR: error: expected at least 3 fields; found 0
+ERR: error: unknown module type
+ERR: error: duplicate module ID
+ERR: error: expected 4 fields; found 3
+
+;--- log
+{{{module:0:a.o:elf:ab}}}
+{{{module:1:b.o:elf:abb50d82b6bdc861}}}
+{{{module:2:c.o:elf:cd}}}
+{{{mmap:0:10000000:load:1:r:0}}}
+
+{{{module}}}
+{{{module:3:d.o:foo}}}
+{{{module:0:d.o:elf:ef}}}
+{{{module:4:d.o:elf}}}

llvm/test/DebugInfo/symbolize-filter-markup-parse-fields.test

+RUN: split-file %s %t
+RUN: llvm-symbolizer --filter-markup < %t/log 2> %t.err
+RUN: FileCheck %s -input-file=%t.err --match-full-lines
+
+CHECK-NOT: '0x4f'
+CHECK-NOT: '00'
+CHECK: error: expected address; found ''
+CHECK: error: expected address; found '42'
+CHECK: error: expected address; found '0xgg'
+
+CHECK-NOT: '0'
+CHECK: error: expected module ID; found ''
+CHECK: error: expected module ID; found '-1'
+CHECK-NOT: '077'
+CHECK: error: expected module ID; found '079'
+CHECK-NOT: '0xff'
+CHECK: error: expected module ID; found '0xfg'
+CHECK: error: expected module ID; found '0x'
+
+CHECK: error: expected build ID; found ''
+CHECK: error: expected build ID; found '0'
+CHECK-NOT: '0xff'
+CHECK: error: expected build ID; found 'fg'
+
+;--- log
+{{{mmap:0x4f:1:unknown}}}
+{{{mmap:00:1:unknown}}}
+{{{mmap::1:unknown}}}
+{{{mmap:42:1:unknown}}}
+{{{mmap:0xgg:1:unknown}}}
+
+{{{module:0::elf:00}}}
+{{{module:::elf:00}}}
+{{{module:-1::elf:00}}}
+{{{module:077::elf:00}}}
+{{{module:079::elf:00}}}
+{{{module:0xff::elf:00}}}
+{{{module:0xfg::elf:00}}}
+{{{module:0x::elf:00}}}
+
+{{{module:1::elf:}}}
+{{{module:2::elf:0}}}
+{{{module:3::elf:ff}}}
+{{{module:4::elf:fg}}}

llvm/test/DebugInfo/symbolize-filter-markup-reset.test

+RUN: split-file %s %t
+RUN: llvm-symbolizer --filter-markup < %t/log > %t.out 2> %t.err
+RUN: FileCheck %s --input-file=%t.out --match-full-lines \
+RUN:   --implicit-check-not {{.}}
+RUN: FileCheck %s --check-prefix=ERR -input-file=%t.err --match-full-lines
+
+CHECK: [[BEGIN:\[{3}]]ELF module #0x0 "a.o"; BuildID=ab 0x0(r)[[END:\]{3}]]
+CHECK: {{  }}[[BEGIN]]reset[[END]]
+CHECK: [[BEGIN:\[{3}]]ELF module #0x0 "b.o"; BuildID=cd 0x1(r)[[END:\]{3}]]
+
+ERR: error: expected 0 fields; found 1
+
+;--- log
+  {{{reset}}}
+{{{module:0:a.o:elf:ab}}}
+{{{mmap:0:1:load:0:r:0}}}
+  {{{reset}}}
+{{{module:0:b.o:elf:cd}}}
+{{{mmap:0x1:1:load:0:r:0}}}
+
+{{{reset:}}}

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

static SmallVector<uint8_t> parseBuildIDArg(const opt::InputArgList &Args,
   SmallVector<uint8_t> BuildID = parseBuildID(V);
   if (BuildID.empty()) {
     errs() << A->getSpelling() + ": expected a build ID, but got '" + V + "'\n";
     exit(1);
   }
   return BuildID;
 }
 
-// Symbolize the markup from stdin and write the result to stdout.
+// Symbolize markup from stdin and write the result to stdout.
 static void filterMarkup(const opt::InputArgList &Args) {
-  MarkupParser Parser;
   MarkupFilter Filter(outs(), parseColorArg(Args));
   for (std::string InputString; std::getline(std::cin, InputString);) {
     InputString += '\n';
-    Parser.parseLine(InputString);
-    Filter.beginLine(InputString);
-    while (Optional<MarkupNode> Element = Parser.nextNode())
-      Filter.filter(*Element);
-  }
-  Parser.flush();
-  while (Optional<MarkupNode> Element = Parser.nextNode())
-    Filter.filter(*Element);
+    Filter.filter(InputString);
+  }
+  Filter.finish();
 }
 
 ExitOnError ExitOnErr;
 
 int main(int argc, char **argv) {
   InitLLVM X(argc, argv);
   sys::InitializeCOMRAII COM(sys::COMThreadingMode::MultiThreaded);