Page MenuHomePhabricator

Please use GitHub pull requests for new patches. Phabricator shutdown timeline

[flang][driver] Add support for `-D`, `-U`

Authored by FarisRehman on Dec 16 2020, 8:31 AM.



Add support for options -D and -U in the Flang driver.

Summary of changes:

  • Create PreprocessorOptions, to be used by the driver then translated into Fortran::parser::Options
  • Create CompilerInvocation::setFortranOpts to pass preprocessor options into the parser options
  • Add a dedicated method, Flang::AddPreprocessingOptions, to extract preprocessing options from the driver arguments into the preprocessor command arguments

Macros specified like -DName will default to definition 1.

When defining macros, the new driver will drop anything after an end-of-line character. This is consistent with gfortran and clang, but different to what currently f18 does. However, flang (which is a bash wrapper for f18), also drops everything after an end-of-line character (at least in the bash on my Linux machine). So gfortran-like behaviour felt like the natural choice. Add a test to demonstrate this behaviour.

Diff Detail

Event Timeline

FarisRehman created this revision.Dec 16 2020, 8:31 AM
FarisRehman requested review of this revision.Dec 16 2020, 8:31 AM
Herald added a project: Restricted Project. · View Herald TranscriptDec 16 2020, 8:31 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript
FarisRehman edited projects, added Restricted Project; removed Restricted Project.
Herald added a project: Restricted Project. · View Herald TranscriptDec 16 2020, 8:34 AM
Herald added a subscriber: jdoerfert. · View Herald Transcript
FarisRehman edited the summary of this revision. (Show Details)Dec 17 2020, 3:10 AM

@FarisRehman, thank you for working on this! This looks really good and I think that it's almost ready. I did leave a few comments, but most are a matter of style and should be easy to address.


Clang deals with the preprocessor options in a dedicated method, AddPreprocessingOptions. See here:

I feel that it might be a good idea to follow that. The preprocessor options (and the corresponding logic) are bound to grow, so we may as well add a dedicated method sooner rather than later.


AFAIK, when you call AddAllArgs the corresponding options are eventually _claimed_:
So, IIUC, this line is not needed.


This change is not needed, is it?

  1. Could you add a comment explaining what this is for?
  1. Could you follow the current semantics used in the file and make it a std::shared_ptr? Long term we can expect PreprocessorOptions to grow big and at that point we wouldn't want to copy it too often (especially when compiling a lot of files).

IIUC, you are renaming this method because it is finally translating the user provided options (i.e. preprocessorOpts()) into Frontend options (i.e. fortranOptions). However, with your changes this method:

  • sets some _sane_ predefined defaults
  • adds some stuff from the user (the preprocessor defines)

So it's somewhere in between SetDefaultFortranOpts and SetFortranOpts (i.e. both names are _almost_ accurate). Perhaps splitting this into two methods would be better? E.g.:

  • SetDefaultFortranOpts (leave as is)
  • SetFortranOpts (implements new functionality, to replace SetDefaultFortranOpts eventually)

[nit] No need to repeat the name of the class.


[nit] IMO this would be preferred : /*isUndef=*/bool ( (currently you have bool /*isUndef*/)


I couldn't find any notes on naming static functions here: This means that we fallow this: So, parsePreprocessorArgs instead of ParsePreprocessorArgs.

Also, could you add a comment briefly explaining what the method is for? E.g. Parses all preprocessor input arguments and populates the preprocessor options accordingly? Ideally doxygen :) See example here:


Variable names should start with lower case.

[nit] Perhaps currentArg instead of A for the sake of being explicit? Or curArg?


This is not quite true, because you're collecting them into preprocessor options :)


static void instead (we don't need this method to be visible outside this translation unit).


All variables here have to be lower case:.

1 ↗(On Diff #312220)

There are no tests for e.g. -DSOME_VAR=SOME_VAL. It would be good to test that SOME_VAL was parsed correctly.

[nit[ Perhaps macro_def_undef.f90 would be more descriptive? (re filename).

tskeith added inline comments.Dec 17 2020, 8:25 AM

This is a change in behavior from f18, right? If so, the commit message should mention it and why it's changing.

FarisRehman edited the summary of this revision. (Show Details)

Address review comments

Address comments with the following changes:

  • Add a dedicated method to adding preprocessing options in Flang.cpp
  • Change preprocessorOpts to use std::shared_ptr
  • Separate SetDefaultFortranOpts from SetFortranOpts in CompilerInvocation
  • Fix method and variable naming in CompilerInvocation.cpp
  • Add comments to new methods and fields
  • Add missing file header to PreprocessorOptions.h
  • Remove unnecessary changes and method calls
  • Add a regression test to check end-of-line character behaviour in a macro definition
FarisRehman marked 11 inline comments as done.Dec 21 2020, 7:40 AM
FarisRehman edited the summary of this revision. (Show Details)Dec 21 2020, 7:53 AM
FarisRehman marked an inline comment as done.Dec 21 2020, 7:57 AM
FarisRehman added inline comments.

Updated the summary to address this, thanks

FarisRehman marked 2 inline comments as done.Dec 21 2020, 8:02 AM
FarisRehman added inline comments.

Added a comment to clarify the input and output arguments in the latest revision 👍

@FarisRehman , thank you for updating this! I have two high-level comments:

  1. In your summary:

Change the way the driver handles end-of-line characters in macro definitions.

That's a a bit misleading - which driver do you mean? You're not changing the behavior of the current driver, flang. Also, this functionality doesn't yet exist in the new driver, flang-new, so you're not changing it there either. You can probably remove this sentence and just make it clear that you meant the new driver.

  1. Could you doxygen style comments for input and output parameters in the new methods that you introduce? Ta!

More comments inline.


This is not used until line 73. Perhaps better to define it near to where it's used?

  1. Could you clarify the difference between Args and CmdArgs?
  2. Could you use doxygen's syntax to specify the parameter _direction_? E.g.:
/// \param [in] Args The list of input arguments
/// \param [out] CmdArgs The list of output arguments



Macros --> macros_ :)


[nit] Ideally this would use doxygen style: For example:

/// \param [in] ppOpts The preprocessor options
/// \param [out] opts The fortran options

Apologies for not making it clearer earlier.


I think that it would make sense to add a note in the commit message that -Dname predefines name as a macro with definition 1. Basically, a note that all macros default to 1.


[nit] Empty line not needed


[nit] Doesn't agree with the name of file. Also, I would just skip it.


[nit] Empty line not needed


[nit] Repetition of the RUN line


[nit] Doesn't agree with the name of file. Also, I would just skip it.


I think that it would be useful to make this a bit stronger:

! CHECK: {{^start a end$}}
! CHECK-NOT: this_should_not_exist_in_the_output
FarisRehman edited the summary of this revision. (Show Details)Dec 23 2020, 3:51 AM
FarisRehman marked 5 inline comments as done.

Address review comments

This revision addresses comments by @awarzynski

Summary of changes:

  • Update commit summary
  • Use Doxygen style for input/output parameters
  • Make method and variable names lowercase where required
  • Remove unnecessary lines in added tests
  • Update multiline macro test to have a stronger test
FarisRehman marked 7 inline comments as done.Dec 23 2020, 4:19 AM
awarzynski accepted this revision.Jan 4 2021, 8:03 AM

Thank you for working on this @FarisRehman , LGTM! From what I can tell you also addressed the point that @tskeith raised (updated commit msg and added a dedicated test). Ccould you wait another day or two before merging? Just in case people have some other comments/suggestions.


[nit] One minor suggestion:

/// \param Args The list of input driver arguments
/// \param CmdArgs The list of output command arguments
This revision is now accepted and ready to land.Jan 4 2021, 8:03 AM
This revision was landed with ongoing or failed builds.Jan 6 2021, 8:17 AM
This revision was automatically updated to reflect the committed changes.
sameeranjoshi added inline comments.Jan 9 2021, 1:53 AM

I see below crash report, when omitting the <value> but not omitting the = symbol.
Not sure if that's correct way of running hence instead of filing bug reporting here.

./bin/flang-new -E -DX= test.f90
PLEASE submit a bug report to and include the crash backtrace.
Stack dump:
0.      Program arguments: /home/amd/f18_git/final_test/driver_build/bin/flang-new -fc1 -E -D X= -o - test.f90
 #0 0x00007f26185d0bc1 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/amd/f18_git/final_test/driver_build/bin/../lib/
 #1 0x00007f26185ce9a4 llvm::sys::RunSignalHandlers() (/home/amd/f18_git/final_test/driver_build/bin/../lib/
 #2 0x00007f26185ceb10 SignalHandler(int) (/home/amd/f18_git/final_test/driver_build/bin/../lib/
 #3 0x00007f2617749470 (/lib/x86_64-linux-gnu/
 #4 0x00007f2617443498 Fortran::parser::OffsetToProvenanceMappings::Map(unsigned long) const (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
 #5 0x00007f261744adf2 Fortran::parser::TokenSequence::GetProvenanceRange() const (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
 #6 0x00007f26173dd334 Fortran::parser::Preprocessor::MacroReplacement(Fortran::parser::TokenSequence const&, Fortran::parser::Prescanner&) (.localalias) (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
 #7 0x00007f26173e9dc1 Fortran::parser::Prescanner::Statement() (.localalias) (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
 #8 0x00007f26173ea888 Fortran::parser::Prescanner::Prescan(Fortran::common::Interval<Fortran::parser::Provenance>) (.localalias) (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
 #9 0x00007f26173d480b Fortran::parser::Parsing::Prescan(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Fortran::parser::Options) (/home/amd/f18_git/final_test/driver_build/bin/../lib/../lib/
#10 0x00007f2618424ddd Fortran::frontend::FrontendAction::Execute() (/home/amd/f18_git/final_test/driver_build/bin/../lib/
#11 0x00007f261841fa5e Fortran::frontend::CompilerInstance::ExecuteAction(Fortran::frontend::FrontendAction&) (/home/amd/f18_git/final_test/driver_build/bin/../lib/
#12 0x00007f26184132fc Fortran::frontend::ExecuteCompilerInvocation(Fortran::frontend::CompilerInstance*) (/home/amd/f18_git/final_test/driver_build/bin/../lib/
#13 0x000055bad4081c20 fc1_main(llvm::ArrayRef<char const*>, char const*) (/home/amd/f18_git/final_test/driver_build/bin/flang-new+0x3c20)
#14 0x000055bad4080e59 main (/home/amd/f18_git/final_test/driver_build/bin/flang-new+0x2e59)
#15 0x00007f261772a1e3 __libc_start_main /build/glibc-5mDdLG/glibc-2.30/csu/../csu/libc-start.c:342:3
#16 0x000055bad4080eae _start (/home/amd/f18_git/final_test/driver_build/bin/flang-new+0x2eae)
flang-new: error: unable to execute command: Segmentation fault (core dumped)
flang-new: error: flang frontend command failed due to signal (use -v to see invocation)
flang-new version 12.0.0 ( 2f9cb090cc6db1be5bf524eb0a32537503b3e786)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/amd/f18_git/final_test/driver_build/bin
flang-new: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.
FarisRehman added inline comments.Jan 11 2021, 3:29 AM

Thanks for reporting this!
I was able to reproduce this error running:

  • flang-new -DX= file.f
  • flang-new -E -DX= file.f
  • flang-new -fc1 -E -DX= file.f
  • f18 -E -DX= file.f

However I did not receive an error when running: flang-new -fc1 -DX= file.f
I suspect this may be an issue in the frontend and I will look into this further.

FarisRehman marked an inline comment as done.Jan 15 2021, 1:51 AM
FarisRehman added inline comments.

The bug seems to occur in the frontend and a bug report has been filed:

clementval added inline comments.

@awarzynski I was looking at this file recently and I was wondering why we have different case style? Is there a particular reason?


You have a couple of these on your patch.

awarzynski added inline comments.Jan 27 2022, 8:50 AM

Thanks for pointing this out! See:

Is there a particular reason?

Tl;Dr This is a typo - I blame the reviewers :)

A bit longer discussion
The driver strives to follow Flang's C++ style. That was pretty much the only style used/available in LLVM Flang when the driver was introduced. Basically, it looked like the only available option. I didn't realise that in lowering and code-gen people decided to use the MLIR style. In hindsight, the driver should follow that too (to be better aligned with the rest of LLVM).

Flang's C++ style is very different to LLVM's, Clang's or MLIR's coding styles. Since the driver "glues" these projects together, it effectively needs to keep switching between the styles (i.e. LLVM and Clang's APIs fallow their style, MLIR's APIs fallow MLIR style, and Flang's parser/sema APIs follow Flang's style). To me personally, this has been very confusing and a major pain point.

I think that it would be good for the overall health of the project if the driver was refactored to follow the MLIR style. What are you thoughts?