[flang][driver] Add support for `-fdebug-dump-symbols-sources`

[awarzynski]

Summary

Note that the original spelling in f18 is -fget-symbols-sources. The
new driver will only support the new spelling, which is also added to
f18 as an alias for the existing one.

Tests are updated to use the new spelling and to use %flang_fc1
instead of %f18. This way both the new and the current driver are
tested.

As:

• both -fsyntax-only and -fdebug-dump-symbols-sources are action flags, and
• the new driver will only consider the right-most action flag,

RUN lines in tests are updated so that the tests work with both
f18 (requires both flags) and flang-new (only considers the last
action flag).

[arnamoy10]

LGTM

[tskeith]

-fget-symbols-sources is not a debug option, it's intended for integrating with IDEs like vscode. So I think the original name is better. Unlike the "dump" options it actually is an action and not something that is intended to produce debug output on the way to doing something else.

[awarzynski]

Switch back from -fdebug-dump-symbols-sources to -fget-symbols-sources

[awarzynski]

In D98191#2611694, @tskeith wrote:

-fget-symbols-sources is not a debug option, it's intended for integrating with IDEs like vscode. So I think the original name is better. Unlike the "dump" options it actually is an action and not something that is intended to produce debug output on the way to doing something else.

Makes sense, updated. Would this option be used to extract debug/code-navigation info? Is there an equivalent in clang or gfortran? Just curious.

On a separate note, all these debug options are _actions_. We may want to fine-tune it at some point.

[tskeith]

Would this option be used to extract debug/code-navigation info?

Yes, it's something related to mapping between symbols and source locations.

Is there an equivalent in clang or gfortran?

Not that I know of.

[kiranchandramohan]

In D98191#2611694, @tskeith wrote:

-fget-symbols-sources is not a debug option, it's intended for integrating with IDEs like vscode. So I think the original name is better. Unlike the "dump" options it actually is an action and not something that is intended to produce debug output on the way to doing something else.

A drive-through comment.
Do we plan to use a compiler option to integrate with IDEs? I was thinking the integration will be through another interface and not through the compiler driver and that the fget-symbols-sources is a temporary option to test the functionality. clangd which supports IDEs is a separate tool.
https://github.com/llvm/llvm-project/tree/main/clang-tools-extra/clangd

[kiranchandramohan]

flang/include/flang/Frontend/FrontendOptions.h

60

Nit: the -> then

clang/include/clang/Driver/Options.td

 def fdebug_dump_parse_tree : Flag<["-"], "fdebug-dump-parse-tree">, Group<Action_Group>,
   HelpText<"Dump the parse tree">;
 def fdebug_dump_provenance : Flag<["-"], "fdebug-dump-provenance">, Group<Action_Group>,
   HelpText<"Dump provenance">;
 def fdebug_measure_parse_tree : Flag<["-"], "fdebug-measure-parse-tree">, Group<Action_Group>,
   HelpText<"Measure the parse tree">;
 def fdebug_pre_fir_tree : Flag<["-"], "fdebug-pre-fir-tree">, Group<Action_Group>,
   HelpText<"Dump the pre-FIR tree">;
+def fget_symbols_sources : Flag<["-"], "fget-symbols-sources">, Group<Action_Group>,
+  HelpText<"Dump symbols and their source code locations">;
 
 }
 
 //===----------------------------------------------------------------------===//
 // CC1 Options
 //===----------------------------------------------------------------------===//
 
 let Flags = [CC1Option, NoDriverOption] in {

flang/include/flang/Frontend/FrontendActions.h

 //===- FrontendActions.h -----------------------------------------*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H
 #define LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H
 
 #include "flang/Frontend/FrontendAction.h"
 #include "flang/Semantics/semantics.h"
 #include <memory>
 
 namespace Fortran::frontend {
 
 // TODO: This is a copy from f18.cpp. It doesn't really belong here and should
 // be moved to a more suitable place in future.
 struct MeasurementVisitor {

 class DebugDumpParseTreeAction : public PrescanAndSemaAction {
   void ExecuteAction() override;
 };
 
 class DebugPreFIRTreeAction : public PrescanAndSemaAction {
   void ExecuteAction() override;
 };
 
+class GetSymbolsSourcesAction : public PrescanAndSemaAction {
+  void ExecuteAction() override;
+};
+
 class ParseSyntaxOnlyAction : public PrescanAndSemaAction {
   void ExecuteAction() override;
 };
 
 } // namespace Fortran::frontend
 
 #endif // LLVM_FLANG_FRONTEND_FRONTENDACTIONS_H

flang/include/flang/Frontend/FrontendOptions.h

 //===- FrontendOptions.h ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #ifndef LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H
 #define LLVM_FLANG_FRONTEND_FRONTENDOPTIONS_H
 
 #include "flang/Common/Fortran-features.h"
 #include "flang/Parser/characters.h"
 #include "flang/Parser/unparse.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/MemoryBuffer.h"
 
 #include <cstdint>
 #include <string>
 
enum ActionKind {
   /// Dump provenance
   DebugDumpProvenance,
 
   /// Parse then output the number of objects in the parse tree and the overall
   /// size
   DebugMeasureParseTree,
 
   /// Parse, run semantics and then output the pre-FIR tree
-  DebugPreFIRTree
+  DebugPreFIRTree,
+
+  /// Parse, run semantics and the dump symbol sources map
+  GetSymbolsSources
 
   /// TODO: RunPreprocessor, EmitLLVM, EmitLLVMOnly,
   /// EmitCodeGenOnly, EmitAssembly, (...)
 };
 
 /// \param suffix The file extension
 /// \return True if the file extension should be processed as fixed form
 bool isFixedFormSuffix(llvm::StringRef suffix);

 class FrontendOptions {
 public:
   /// Show the -help text.
   unsigned showHelp_ : 1;
 
   /// Show the -version text.
   unsigned showVersion_ : 1;
 
+  /// Instrument the parse to get a more verbose log
+  unsigned instrumentedParse_ : 1;
+
   /// The input files and their types.
   std::vector<FrontendInputFile> inputs_;
 
   /// The output file, if any.
   std::string outputFile_;
 
   /// The frontend action to perform.
   frontend::ActionKind programAction_;

flang/lib/Frontend/CompilerInvocation.cpp

 //===- CompilerInvocation.cpp ---------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/CompilerInvocation.h"
 #include "flang/Common/Fortran-features.h"
 #include "flang/Frontend/PreprocessorOptions.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Version.inc"
 #include "clang/Basic/AllDiagnostics.h"
 #include "clang/Basic/DiagnosticDriver.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Driver/DriverDiagnostic.h"
case clang::driver::options::OPT_fdebug_dump_provenance:
       opts.programAction_ = DebugDumpProvenance;
       break;
     case clang::driver::options::OPT_fdebug_measure_parse_tree:
       opts.programAction_ = DebugMeasureParseTree;
       break;
     case clang::driver::options::OPT_fdebug_pre_fir_tree:
       opts.programAction_ = DebugPreFIRTree;
       break;
+    case clang::driver::options::OPT_fget_symbols_sources:
+      opts.programAction_ = GetSymbolsSources;
+      break;
 
       // TODO:
       // case calng::driver::options::OPT_emit_llvm:
       // case clang::driver::options::OPT_emit_llvm_only:
       // case clang::driver::options::OPT_emit_codegen_only:
       // case clang::driver::options::OPT_emit_module:
       // (...)
     }

flang/lib/Frontend/FrontendActions.cpp

 //===--- FrontendActions.cpp ----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/FrontendActions.h"
 #include "flang/Common/default-kinds.h"
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/FrontendOptions.h"
 #include "flang/Lower/PFTBuilder.h"
 #include "flang/Parser/dump-parse-tree.h"
 #include "flang/Parser/parsing.h"
 #include "flang/Parser/provenance.h"
 #include "flang/Parser/source.h"
 #include "flang/Parser/unparse.h"
+#include "flang/Semantics/runtime-type-info.h"
 #include "flang/Semantics/semantics.h"
 #include "flang/Semantics/unparse-with-symbols.h"
 #include "llvm/ADT/StringRef.h"
 #include <clang/Basic/Diagnostic.h>
 #include <memory>
 
 using namespace Fortran::frontend;
 
if (auto ast{Fortran::lower::createPFT(
     Fortran::lower::dumpPFT(llvm::outs(), *ast);
   } else {
     unsigned diagID = ci.diagnostics().getCustomDiagID(
         clang::DiagnosticsEngine::Error, "Pre FIR Tree is NULL.");
     ci.diagnostics().Report(diagID);
   }
 }
 
+void GetSymbolsSourcesAction::ExecuteAction() {
+  // Report and exit if fatal semantic errors are present
+  if (reportFatalSemanticErrors(semantics(), this->instance().diagnostics(),
+          GetCurrentFileOrBufferName()))
+    return;
+
+  semantics().DumpSymbolsSources(llvm::outs());
+}
+
 void EmitObjAction::ExecuteAction() {
   CompilerInstance &ci = this->instance();
   unsigned DiagID = ci.diagnostics().getCustomDiagID(
       clang::DiagnosticsEngine::Error, "code-generation is not available yet");
   ci.diagnostics().Report(DiagID);
 }

flang/lib/FrontendTool/ExecuteCompilerInvocation.cpp

 //===--- ExecuteCompilerInvocation.cpp ------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file holds ExecuteCompilerInvocation(). It is split into its own file to
 // minimize the impact of pulling in essentially everything else in Flang.
 //
 //===----------------------------------------------------------------------===//
 
 #include "flang/Frontend/CompilerInstance.h"
 #include "flang/Frontend/FrontendActions.h"
 #include "clang/Driver/Options.h"
 #include "llvm/Option/OptTable.h"
 #include "llvm/Option/Option.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/CommandLine.h"
 
 namespace Fortran::frontend {
case DebugDumpProvenance:
     return std::make_unique<DebugDumpProvenanceAction>();
     break;
   case DebugMeasureParseTree:
     return std::make_unique<DebugMeasureParseTreeAction>();
     break;
   case DebugPreFIRTree:
     return std::make_unique<DebugPreFIRTreeAction>();
     break;
+  case GetSymbolsSources:
+    return std::make_unique<GetSymbolsSourcesAction>();
+    break;
   default:
     break;
     // TODO:
     // case RunPreprocessor:
     // case ParserSyntaxOnly:
     // case EmitLLVM:
     // case EmitLLVMOnly:
     // case EmitCodeGenOnly:

flang/test/Flang-Driver/driver-help.f90

 ! HELP-FC1-NEXT: -fdebug-unparse        Unparse and stop.
 ! HELP-FC1-NEXT: -fdefault-double-8  Set the default double precision kind to an 8 byte wide type
 ! HELP-FC1-NEXT: -fdefault-integer-8 Set the default integer kind to an 8 byte wide type
 ! HELP-FC1-NEXT: -fdefault-real-8    Set the default real kind to an 8 byte wide type
 ! HELP-FC1-NEXT: -ffixed-form           Process source files in fixed form
 ! HELP-FC1-NEXT: -ffixed-line-length=<value>
 ! HELP-FC1-NEXT: Use <value> as character line width in fixed mode
 ! HELP-FC1-NEXT: -ffree-form            Process source files in free form
+! HELP-FC1-NEXT: -fget-symbols-sources   Dump symbols and their source code locations
 ! HELP-FC1-NEXT: -fimplicit-none        No implicit typing allowed unless overridden by IMPLICIT statements
 ! HELP-FC1-NEXT: -finput-charset=<value> Specify the default character set for source files
 ! HELP-FC1-NEXT: -flarge-sizes          Use INTEGER(KIND=8) for the result type in size-related intrinsics
 ! HELP-FC1-NEXT: -flogical-abbreviations Enable logical abbreviations
 ! HELP-FC1-NEXT: -fopenacc              Enable OpenACC
 ! HELP-FC1-NEXT: -fopenmp               Parse OpenMP pragmas and generate parallel code.
 ! HELP-FC1-NEXT: -fxor-operator         Enable .XOR. as a synonym of .NEQV.
 ! HELP-FC1-NEXT: -help                  Display available options

flang/test/Semantics/getsymbols01.f90

contains
   end subroutine
  end subroutine
  recursive pure function f() result(x)
   real, allocatable :: x
   x = 1.0
  end function
 end module
 
-! RUN: %f18 -fget-symbols-sources -fsyntax-only %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -fsyntax-only -fget-symbols-sources %s 2>&1 | FileCheck %s
 ! CHECK-COUNT-1:f:{{.*}}getsymbols01.f90, 12, 26-27
 ! CHECK-COUNT-1:mm1:{{.*}}getsymbols01.f90, 2, 8-11
 ! CHECK-COUNT-1:s:{{.*}}getsymbols01.f90, 5, 18-19
 ! CHECK-COUNT-1:ss:{{.*}}getsymbols01.f90, 9, 19-21
 ! CHECK-COUNT-1:x:{{.*}}getsymbols01.f90, 5, 21-22
 ! CHECK-COUNT-1:x:{{.*}}getsymbols01.f90, 13, 24-25
 ! CHECK-COUNT-1:y:{{.*}}getsymbols01.f90, 5, 24-25

flang/test/Semantics/getsymbols02.f90

 ! Tests -fget-symbols-sources with modules.
 
 PROGRAM helloworld
     use mm2b
     implicit none
     integer::i
     i = callget5()
 ENDPROGRAM
 
-! RUN: %f18 -fsyntax-only %S/Inputs/getsymbols02-a.f90
-! RUN: %f18 -fsyntax-only %S/Inputs/getsymbols02-b.f90
-! RUN: %f18 -fget-symbols-sources -fsyntax-only %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -fsyntax-only %S/Inputs/getsymbols02-a.f90
+! RUN: %flang_fc1 -fsyntax-only %S/Inputs/getsymbols02-b.f90
+! RUN: %flang_fc1 -fsyntax-only -fget-symbols-sources %s 2>&1 | FileCheck %s
 ! CHECK: callget5: .{{[/\\]}}mm2b.mod,
 ! CHECK: get5: .{{[/\\]}}mm2a.mod,

flang/test/Semantics/getsymbols03-a.f90

 ! Tests -fget-symbols with INCLUDE
 INCLUDE "Inputs/getsymbols03-b.f90"
 
 program main
  use mm3
  integer :: x
  x = f
 end program
 
-! RUN: %f18 -fget-symbols-sources -fsyntax-only %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -fsyntax-only -fget-symbols-sources %s 2>&1 | FileCheck %s
 ! CHECK:f:{{.*}}getsymbols03-b.f90, 2, 12-13
 ! CHECK:main:{{.*}}getsymbols03-a.f90, 4, 9-13
 ! CHECK:mm3:{{.*}}getsymbols03-a.f90, 5, 6-9
 ! CHECK:x:{{.*}}getsymbols03-a.f90, 6, 13-14

flang/test/Semantics/getsymbols04.f90

 ! Tests -fget-symbols-sources with COMMON.
 program main
   integer :: x
   integer :: y
   common /x/ y
   x = y
 end program
 
-! RUN: %f18 -fget-symbols-sources -fsyntax-only %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -fsyntax-only -fget-symbols-sources %s 2>&1 | FileCheck %s
 ! CHECK:x:{{.*}}getsymbols04.f90, 3, 14-15
 ! CHECK:x:{{.*}}getsymbols04.f90, 5, 11-12
 ! CHECK:y:{{.*}}getsymbols04.f90, 4, 14-15

flang/test/Semantics/getsymbols05.f90

 ! Tests -fget-symbols-sources with COMMON.
 program main
   integer :: x
   integer :: y
   block
     integer :: x
     x = y
   end block
   x = y
 end program
 
-! RUN: %f18 -fget-symbols-sources -fsyntax-only %s 2>&1 | FileCheck %s
+! RUN: %flang_fc1 -fsyntax-only -fget-symbols-sources %s 2>&1 | FileCheck %s
 ! CHECK:x:{{.*}}getsymbols05.f90, 3, 14-15
 ! CHECK:x:{{.*}}getsymbols05.f90, 6, 16-17
 ! CHECK:y:{{.*}}getsymbols05.f90, 4, 14-15