[llvm-symbolizer] Add options to disable printing source files & inlining

[danzimm]

Summary

Currently there is no way to disable source file output from llvm-symbolizer. Similarly there's no way to disable symbolicating inlined functions with llvm-symbolzer (this option is automatically disabled when llvm-addr2line is invoked instead of llvm-symbolizer).

This diff introduces flags to further customize llvm-symbolizer's output to support these two usecases:

• --no-inlining: This disables symbolicating inlined frames. When paired with --output-style=LLVM and --use-symbol-table this results in a list of functions that appear in the final binary
• --no-source-file: This disables printing source file information when symbolicating an address
• --source-file: This enables printing source file information (the default). This option was added to balance --no-source-file.

The last of --no-source-file and --source-file passed will determine whether source file information is printed or not. The behavior of llvm-symbolizer before this diff should be identical to the behavior after this diff when none of the new options are specified.

Together --functions=linkage --demangle --output-style=LLVM --no-source-file --no-inlining results in a list of symbol names which appear in the resulting binary. This is useful when symbolicating a list of addresses e.g. for link order files.

N.B. The same data can be extracted with a processor on top of --functions=linkage --demangle --output-style=LLVM, however with large lists of symbols I've found that this takes quite a long time (my processor(s) were in perl/python- in theory I could've written a C/++ one, but I figure best just add these as formatting options to llvm-symbolizer instead).

[danzimm]

If there are tests somewhere that I can add to, please point me to them! I'd love to add some tests, just couldn't quite find any (I'm guessing I'm just not looking in the right place... 😅)

[jhenderson]

Hi @danzimm,

In LLVM tools using the cl::opt interface, like llvm-symbolizer and llvm-addr2line, where you see a cl::opt<bool> (such as --inlining) you should be able to do --inlininng=0 to disable it. I'm not necessarily opposed to --no-inlining too mind you, but wanted to raise that before going further. If you wish to go ahead with adding --no-inlining, I'd recommend moving it to a separate patch, as it is independent of the --source-files stuff. Also, please add any new options to the llvm-symbolizer and llvm-addr2line docs (located in llvm/docs/CommandGuide).

In D83262#2134394, @danzimm wrote:

If there are tests somewhere that I can add to, please point me to them! I'd love to add some tests, just couldn't quite find any (I'm guessing I'm just not looking in the right place... 😅)

Most tests for llvm-symbolizer (and llvm-addr2line) are located in llvm/test/tools/llvm-symbolizer (there are a few in other scattered locations - grep for llvm-symbolizer). Any new front-end options like these should be tested there.

Together --functions=linkage --demangle --output-style=LLVM --no-source-file --no-inlining results in a list of symbol names which appear in the resulting binary. This is useful when symbolicating a list of addresses e.g. for link order files.

N.B. The same data can be extracted with a processor on top of --functions=linkage --demangle --output-style=LLVM, however with large lists of symbols I've found that this takes quite a long time (my processor(s) were in perl/python- in theory I could've written a C/++ one, but I figure best just add these as formatting options to llvm-symbolizer instead).

Are you specifically interested in symbols at specific addresses, or with a specific type? llvm-nm and llvm-readobj can both be used to dump symbols too. It doesn't feel to me like llvm-symbolizer is the right tool for the job if you want to dump all symbols (or all functions), though I could possibly see an argument if you are limiting it to the symbols with specific addresses. I personally would think it would make more sense to add any necessary options to llvm-nm or possibly llvm-readelf. Adding a few others with binutils knowledge for more visibility and to get their input.

[danzimm]

Wow, thanks for the quick and thoughtful response @jhenderson ! This was awesome to wake up to!

In LLVM tools using the cl::opt interface, like llvm-symbolizer and llvm-addr2line, where you see a cl::opt<bool> (such as --inlining) you should be able to do --inlininng=0 to disable it. I'm not necessarily opposed to --no-inlining too mind you, but wanted to raise that before going further

Whoa, I didn't know that! This is pretty cool! I'll go ahead and pull this change out of this diff. Do you have preference on introducing --no-inlining? I think I'll drop it altogether (as opposed to putting it in another diff) in order to try and reduce complexity.

Most tests for llvm-symbolizer (and llvm-addr2line) are located in llvm/test/tools/llvm-symbolizer (there are a few in other scattered locations - grep for llvm-symbolizer). Any new front-end options like these should be tested there.

Doh! It was right in front of me the whole time! Thanks for pointing that out. Next time (if there is a next time for this diff) I upload changes I'll add some tests!

Are you specifically interested in symbols at specific addresses, or with a specific type? llvm-nm and llvm-readobj can both be used to dump symbols too. It doesn't feel to me like llvm-symbolizer is the right tool for the job if you want to dump all symbols (or all functions), though I could possibly see an argument if you are limiting it to the symbols with specific addresses. I personally would think it would make more sense to add any necessary options to llvm-nm or possibly llvm-readelf. Adding a few others with binutils knowledge for more visibility and to get their input.

Originally I was interested in mapping a list of addresses to the names of the addresses of functions that appear in the binary from which the addresses came (these addresses are coming from instrumentation, e.g. -finstrument-function-entry-bare). After a bit of thought (and trial & error) I think I've concluded I actually do want source file information... it seems symbol names are duplicated across compilation units more often than I had originally expected.

So.... I think for my personal use cases we can go ahead and scrap this diff. I'm also ok with creating the tests and pushing through the --no-source-file change (it still probably has some utility in the case when there aren't duplicate symbols). What do you say @jhenderson ? Does scrapping this diff sound ok?

[jhenderson]

I'm personally fine with dropping this if it's not actually useful for you, as I don't have any use-case for it at the current time. Re. --no-inlining, I have a slight preference for not adding it, but I'm also okay with it being added, if you'd find it less confusing. I didn't know about the functionality of =0 to disable a flag in LLVM tools when I first came to the project myself, so it could be a little confusing. I actually added --no-demangle precisely for that reason.

[MaskRay]

In D83262#2135629, @jhenderson wrote:

I'm personally fine with dropping this if it's not actually useful for you, as I don't have any use-case for it at the current time. Re. --no-inlining, I have a slight preference for not adding it, but I'm also okay with it being added, if you'd find it less confusing. I didn't know about the functionality of =0 to disable a flag in LLVM tools when I first came to the project myself, so it could be a little confusing. I actually added --no-demangle precisely for that reason.

I have a slightly stronger opinion that we should not add it. We could improve help messages for cl::opt<bool> to mention the default value.

Originally I was interested in mapping a list of addresses to the names of the addresses of functions that appear in the binary from which the addresses came (these addresses are coming from instrumentation, e.g. -finstrument-function-entry-bare). After a bit of thought (and trial & error) I think I've concluded I actually do want source file information... it seems symbol names are duplicated across compilation units more often than I had originally expected.

Do you still have a need for output without source info? We add options if there is a reasonable use case, not that "we add it just to customize a behavior".

[MaskRay]

In D83262#2140642, @MaskRay wrote:

In D83262#2135629, @jhenderson wrote:

I'm personally fine with dropping this if it's not actually useful for you, as I don't have any use-case for it at the current time. Re. --no-inlining, I have a slight preference for not adding it, but I'm also okay with it being added, if you'd find it less confusing. I didn't know about the functionality of =0 to disable a flag in LLVM tools when I first came to the project myself, so it could be a little confusing. I actually added --no-demangle precisely for that reason.

I have a slightly stronger opinion that we should not add it. We could improve help messages for cl::opt<bool> to mention the default value.

Given it more thought, perhaps we should switch llvm-symbolizer to llvm-objcopy style OptTable. Many llvm::cl::opt based tools are not user facing (llc/opt). llvm::cl::opt is quick and easy. For user facing utilities (clang/lld/objcopy), OptTable may be more suitable as OptTable can be customized to be similar to the most common GNU-style getopt_long behavior. llvm-readobj/llvm-objdump are a bit special: they don't have defaulted-to-true llvm::cl::opt<bool>. If they do, we may face similar conumdrum like --no-demangle.

Originally I was interested in mapping a list of addresses to the names of the addresses of functions that appear in the binary from which the addresses came (these addresses are coming from instrumentation, e.g. -finstrument-function-entry-bare). After a bit of thought (and trial & error) I think I've concluded I actually do want source file information... it seems symbol names are duplicated across compilation units more often than I had originally expected.

Do you still have a need for output without source info? We add options if there is a reasonable use case, not that "we add it just to customize a behavior".

[MaskRay]

Created D83530 to switch to OptTable. The only unimplemented feature is grouped short options (POSIX.1-2017 12.2 Utility Syntax Guidelines, Guideline 5)

llvm/include/llvm/DebugInfo/Symbolize/DIPrinter.h

 
 private:
   raw_ostream &OS;
   bool PrintFunctionNames;
   bool PrintPretty;
   int PrintSourceContext;
   bool Verbose;
   OutputStyle Style;
+  bool PrintSourceFiles;
 
   void print(const DILineInfo &Info, bool Inlined);
   void printContext(const std::string &FileName, int64_t Line);
 
 public:
   DIPrinter(raw_ostream &OS, bool PrintFunctionNames = true,
             bool PrintPretty = false, int PrintSourceContext = 0,
-            bool Verbose = false, OutputStyle Style = OutputStyle::LLVM)
+            bool Verbose = false, OutputStyle Style = OutputStyle::LLVM,
+            bool PrintSourceFiles = true)
       : OS(OS), PrintFunctionNames(PrintFunctionNames),
         PrintPretty(PrintPretty), PrintSourceContext(PrintSourceContext),
-        Verbose(Verbose), Style(Style) {}
+        Verbose(Verbose), Style(Style), PrintSourceFiles(PrintSourceFiles) {}
 
   DIPrinter &operator<<(const DILineInfo &Info);
   DIPrinter &operator<<(const DIInliningInfo &Info);
   DIPrinter &operator<<(const DIGlobal &Global);
   DIPrinter &operator<<(const DILocal &Local);
 };
 }
 }
 
 #endif

llvm/lib/DebugInfo/Symbolize/DIPrinter.cpp

if (PrintFunctionNames) {
     std::string FunctionName = Info.FunctionName;
     if (FunctionName == DILineInfo::BadString)
       FunctionName = DILineInfo::Addr2LineBadString;
 
     StringRef Delimiter = PrintPretty ? " at " : "\n";
     StringRef Prefix = (PrintPretty && Inlined) ? " (inlined by) " : "";
     OS << Prefix << FunctionName << Delimiter;
   }
+  if (!PrintSourceFiles)
+    return;
   std::string Filename = Info.FileName;
   if (Filename == DILineInfo::BadString)
     Filename = DILineInfo::Addr2LineBadString;
   if (!Verbose) {
     OS << Filename << ":" << Info.Line;
     if (Style == OutputStyle::LLVM)
       OS << ":" << Info.Column;
     else if (Style == OutputStyle::GNU && Info.Discriminator != 0)

llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp

 
 static cl::opt<bool> ClUntagAddresses(
     "untag-addresses", cl::init(true),
     cl::desc("Remove memory tags from addresses before symbolization"));
 
 static cl::opt<bool>
     ClPrintInlining("inlining", cl::init(true),
                     cl::desc("Print all inlined frames for a given address"));
+
+static cl::opt<bool>
+    ClNoPrintInlining("no-inlining", cl::init(false),
+                      cl::desc("Don't print all inlined frames for a given"
+                               " address"));
+
 static cl::alias
     ClPrintInliningAliasI("i", cl::desc("Alias for -inlining"),
                           cl::NotHidden, cl::aliasopt(ClPrintInlining),
                           cl::Grouping);
 static cl::alias
     ClPrintInliningAliasInlines("inlines", cl::desc("Alias for -inlining"),
                                 cl::NotHidden, cl::aliasopt(ClPrintInlining));
 

 ClDemangle("demangle", cl::init(true), cl::desc("Demangle function names"));
 static cl::alias
 ClDemangleShort("C", cl::desc("Alias for -demangle"),
                 cl::NotHidden, cl::aliasopt(ClDemangle), cl::Grouping);
 static cl::opt<bool>
 ClNoDemangle("no-demangle", cl::init(false),
              cl::desc("Don't demangle function names"));
 
+static cl::opt<bool>
+    ClSourceFile("source-file", cl::init(true),
+                 cl::desc("Print source file info for a given address"));
+
+static cl::opt<bool> ClNoSourceFile(
+    "no-source-file", cl::init(false),
+    cl::desc("Don't print source file info for a given address"));
+
 static cl::opt<std::string> ClDefaultArch("default-arch", cl::init(""),
                                           cl::desc("Default architecture "
                                                    "(for multi-arch objects)"));
 
 static cl::opt<std::string>
 ClBinaryName("obj", cl::init(""),
              cl::desc("Path to object file to be symbolized (if not provided, "
                       "object file should be specified for each input line)"));
if (Cmd == Command::Data) {
     auto ResOrErr = Symbolizer.symbolizeInlinedCode(
         ModuleName, {Offset, object::SectionedAddress::UndefSection});
     Printer << (error(ResOrErr) ? DILineInfo() : ResOrErr.get().getFrame(0));
   } else {
     auto ResOrErr = Symbolizer.symbolizeCode(
         ModuleName, {Offset, object::SectionedAddress::UndefSection});
     Printer << (error(ResOrErr) ? DILineInfo() : ResOrErr.get());
   }
-  if (ClOutputStyle == DIPrinter::OutputStyle::LLVM)
+  if (ClOutputStyle == DIPrinter::OutputStyle::LLVM && ClPrintInlining)
     outs() << "\n";
 }
 
 int main(int argc, char **argv) {
   InitLLVM X(argc, argv);
 
   bool IsAddr2Line = sys::path::stem(argv[0]).contains("addr2line");
 
cl::ParseCommandLineOptions(
       argc, argv, IsAddr2Line ? "llvm-addr2line\n" : "llvm-symbolizer\n",
       /*Errs=*/nullptr,
       IsAddr2Line ? "LLVM_ADDR2LINE_OPTS" : "LLVM_SYMBOLIZER_OPTS");
 
   // If both --demangle and --no-demangle are specified then pick the last one.
   if (ClNoDemangle.getPosition() > ClDemangle.getPosition())
     ClDemangle = !ClNoDemangle;
 
+  if (ClNoSourceFile.getPosition() > ClSourceFile.getPosition())
+    ClSourceFile = !ClNoSourceFile;
+
+  if (ClNoPrintInlining.getPosition() > ClPrintInlining.getPosition())
+    ClPrintInlining = !ClNoPrintInlining;
+
   LLVMSymbolizer::Options Opts;
   Opts.PrintFunctions = ClPrintFunctions;
   Opts.UseSymbolTable = ClUseSymbolTable;
   Opts.Demangle = ClDemangle;
   Opts.RelativeAddresses = ClUseRelativeAddress;
   Opts.UntagAddresses = ClUntagAddresses;
   Opts.DefaultArch = ClDefaultArch;
   Opts.FallbackDebugPath = ClFallbackDebugPath;
   Opts.DWPName = ClDwpName;
   Opts.DebugFileDirectory = ClDebugFileDirectory;
   Opts.UseNativePDBReader = ClUseNativePDBReader;
   Opts.PathStyle = DILineInfoSpecifier::FileLineInfoKind::AbsoluteFilePath;
   // If both --basenames and --relativenames are specified then pick the last
   // one.
   if (ClBasenames.getPosition() > ClRelativenames.getPosition())
     Opts.PathStyle = DILineInfoSpecifier::FileLineInfoKind::BaseNameOnly;
   else if (ClRelativenames)
     Opts.PathStyle = DILineInfoSpecifier::FileLineInfoKind::RelativeFilePath;
 
+  if (!ClSourceFile)
+    Opts.PathStyle = DILineInfoSpecifier::FileLineInfoKind::None;
+
   for (const auto &hint : ClDsymHint) {
     if (sys::path::extension(hint) == ".dSYM") {
       Opts.DsymHints.push_back(hint);
     } else {
       errs() << "Warning: invalid dSYM hint: \"" << hint <<
                 "\" (must have the '.dSYM' extension).\n";
     }
   }
   LLVMSymbolizer Symbolizer(Opts);
 
   DIPrinter Printer(outs(), ClPrintFunctions != FunctionNameKind::None,
                     ClPrettyPrint, ClPrintSourceContextLines, ClVerbose,
-                    ClOutputStyle);
+                    ClOutputStyle, ClSourceFile);
 
   if (ClInputAddresses.empty()) {
     const int kMaxInputStringLength = 1024;
     char InputString[kMaxInputStringLength];
 
     while (fgets(InputString, sizeof(InputString), stdin)) {
       // Strip newline characters.
       std::string StrippedInputString(InputString);