[llvm-objdump] Implement syntax highlighting

[seiya]

Summary

This is a prototype implementation of syntax highlighting. I'll split into separate patches, but before that I'd like to discuss the
outline to make sure that I’m on the right track.

[jhenderson]

Something to be aware of is that people can change how the output is printed using switches like --x86-asm-syntax. If you're going down the route of parsing the text to identify how to highlight it, your parser needs to be robust enough to cope. Not only that but different architectures have different disassembly output, so there's a similar issue there.

[seiya]

• Hold the vector of marked up fragments instead of parsing the disassembly (suggested by Michael Spencer on llvm-dev).

[seiya]

It's still work-in-progress.

[seiya]

llvm/lib/Target/X86/MCTargetDesc/X86ATTInstPrinter.cpp

42 ↗

(On Diff #209420)

I think implementing operator<< would be better.

[seiya]

• Implement raw_ostream &operator<<(...).
• Support nested marked-up string (e.g., <mem:256(<reg:%rip>)>).

[seiya]

This comment was removed by seiya.

[seiya]

• Highlight symbol names.
• Fix some TODOs.

[seiya]

llvm/lib/Target/X86/MCTargetDesc/X86ATTInstPrinter.cpp

377 ↗

(On Diff #210031)

TODO: remove

[seiya]

I'll add tests and split into separate patches later but before that I'd like to discuss the outline to make sure that I’m on the right track.

[jhenderson]

Hi @seiya. As mentioned by email offline, I think this looks quite good. Please go ahead and add some reviewers, and write some tests for it. I'm happy to help out with reviewing it in more detail once it's ready.

[seiya]

I've uploaded a series of separate patches: https://reviews.llvm.org/D65191

llvm/tools/llvm-objdump/llvm-objdump.cpp

static cl::alias UnwindInfoShort("u", cl::desc("Alias for --unwind-info"),
                                  cl::NotHidden, cl::Grouping,
                                  cl::aliasopt(UnwindInfo));
 
 static cl::opt<bool>
     Wide("wide", cl::desc("Ignored for compatibility with GNU objdump"),
          cl::cat(ObjdumpCat));
 static cl::alias WideShort("w", cl::Grouping, cl::aliasopt(Wide));
 
+static cl::opt<bool> Highlight("highlight",
+                              cl::desc("Enable syntax highlighting"),
+                              cl::cat(ObjdumpCat));
+
 static cl::extrahelp
     HelpResponse("\nPass @FILE as argument to read options from FILE.\n");
 
 static StringSet<> DisasmFuncsSet;
 static StringSet<> FoundSectionSet;
 static StringRef ToolName;
 
 typedef std::vector<std::tuple<uint64_t, StringRef, uint8_t>> SectionSymbolsTy;
if (NumBytes == 8) {
       outs() << std::string(IndentOffset, ' ') << "         ";
       outs() << reinterpret_cast<char *>(AsciiData);
       outs() << '\n';
       NumBytes = 0;
     }
   }
 }
 
+// Parses given rich disassembly output (see https://llvm.org/docs/MarkedUpDisassembly.html)
+// and prints colored output.
+static void printMarkedUpInst(StringRef Text) {
+  if (!Highlight) {
+    outs() << Text;
+    return;
+  }
+
+  size_t Pos = 0;
+  while (Pos < Text.size()) {
+    size_t OpenBracket, Colon;
+    if ((OpenBracket = Text.find('<', Pos)) == StringRef::npos)
+      break;
+    if ((Colon = Text.find(':', OpenBracket)) == StringRef::npos)
+      break;
+
+    // Look for the corresponding closing bracket.
+    size_t NestedLevel = 1;
+    size_t CloseBracket = Colon + 1;
+    while (CloseBracket < Text.size())  {
+      if (Text[CloseBracket] == '<')
+        NestedLevel++;
+      if (Text[CloseBracket] == '>') {
+        NestedLevel--;
+        if (NestedLevel == 0)
+          break;
+      }
+
+      CloseBracket++;
+    }
+
+    if (CloseBracket == Text.size())
+      break;
+
+    // It looks a valid rich dissassembly format. Parse it as follows:
+    //
+    //        leaq  <mem:256(<reg:%rip>)>, <reg:%r8>
+    //        ^----- ^-- ^--------------
+    //    BeforeText  Tag   InnerText
+    //
+    StringRef BeforeText = Text.substr(Pos, OpenBracket - Pos);
+    StringRef Tag = Text.substr(OpenBracket + 1, Colon - OpenBracket - 1);
+    StringRef InnerText = Text.substr(Colon + 1, CloseBracket - Colon - 1);
+
+    outs() << BeforeText;
+
+    // WIP: Hghlight only register names for now.
+    // TODO: tag may contain multiple values (e.g. "<mem regoffset:...>").
+    if (Tag == "reg")
+      outs().changeColor(raw_ostream::RED);
+
+    // Print innter text recursively.
+    printMarkedUpInst(InnerText);
+
+    outs().resetColor();
+    Pos = CloseBracket + 1;
+  }
+
+  // Print the remaining part.
+  outs() << Text.substr(Pos);
+}
+
 static void disassembleObject(const Target *TheTarget, const ObjectFile *Obj,
                               MCContext &Ctx, MCDisassembler *PrimaryDisAsm,
                               MCDisassembler *SecondaryDisAsm,
                               const MCInstrAnalysis *MIA, MCInstPrinter *IP,
                               const MCSubtargetInfo *PrimarySTI,
                               const MCSubtargetInfo *SecondarySTI,
                               PrettyPrinter &PIP,
                               SourcePrinter &SP, bool InlineRelocs) {
#endif
         // provided
         MCInst Inst;
         bool Disassembled = DisAsm->getInstruction(
             Inst, Size, Bytes.slice(Index), SectionAddr + Index, DebugOut,
             CommentStream);
         if (Size == 0)
           Size = 1;
 
+        std::string InstText;
+        raw_string_ostream InstTextStream(InstText);
         PIP.printInst(
             *IP, Disassembled ? &Inst : nullptr, Bytes.slice(Index, Size),
-            {SectionAddr + Index + VMAAdjustment, Section.getIndex()}, outs(),
+            {SectionAddr + Index + VMAAdjustment, Section.getIndex()}, InstTextStream,
             "", *STI, &SP, &Rels);
+        printMarkedUpInst(StringRef(InstTextStream.str()));
         outs() << CommentStream.str();
         Comments.clear();
 
         // Try to resolve the target of a call, tail call, etc. to a specific
         // symbol.
         if (MIA && (MIA->isCall(Inst) || MIA->isUnconditionalBranch(Inst) ||
                     MIA->isConditionalBranch(Inst))) {
           uint64_t Target;
static void disassembleObject(const ObjectFile *Obj, bool InlineRelocs) {
 
   int AsmPrinterVariant = AsmInfo->getAssemblerDialect();
   std::unique_ptr<MCInstPrinter> IP(TheTarget->createMCInstPrinter(
       Triple(TripleName), AsmPrinterVariant, *AsmInfo, *MII, *MRI));
   if (!IP)
     report_error(Obj->getFileName(),
                  "no instruction printer for target " + TripleName);
   IP->setPrintImmHex(PrintImmHex);
+  IP->setUseMarkup(Highlight);
 
   PrettyPrinter &PIP = selectPrettyPrinter(Triple(TripleName));
   SourcePrinter SP(Obj, TheTarget->getName());
 
   for (StringRef Opt : DisassemblerOptions)
     if (!IP->applyTargetSpecificCLOption(Opt))
       error("Unrecognized disassembler option: " + Opt);