This is an archive of the discontinued LLVM Phabricator instance.

Add support for #elifdef and #elifndef
ClosedPublic

Authored by aaron.ballman on Apr 23 2021, 12:19 PM.

Details

Summary

WG14 adopted N2645 and WG21 EWG has accepted P2334 in principle (still subject to full EWG vote + CWG review + plenary vote), which add support for #elifdef as shorthand for #elif defined and #elifndef as shorthand for #elif !defined. This patch adds support for the new preprocessor directives.

In this version of the patch, I am supporting the feature in all C and C++ modes. This seems like more useful functionality than diagnosing an unknown preprocessor directive in older language modes, and is a conforming extension in those modes anyway as use of an unknown directive is undefined behavior. However, it was not clear to me whether we want to issue a pedantic warning for this or not or whether we felt it was important to document this behavior explicitly.

Diff Detail

Event Timeline

aaron.ballman created this revision.Apr 23 2021, 12:19 PM
aaron.ballman requested review of this revision.Apr 23 2021, 12:19 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 23 2021, 12:19 PM
aaron.ballman edited the summary of this revision. (Show Details)Apr 23 2021, 12:20 PM
erichkeane added inline comments.Apr 23 2021, 12:30 PM
clang/lib/Format/UnwrappedLineParser.cpp
754

Just looking through this, so forgive me if I there is something I don't understand... Why is this not doing something like parsePPIf(/*IfDef=*/true) like above?

clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
924

Hrmph, not sure I understand this part either.

clang/lib/Lex/PPDirectives.cpp
651

Can you just pass 'Directive' here?

3288

Oh? What is the 'right' error here?

3297

Same here:)?

aaron.ballman marked 4 inline comments as done.Apr 23 2021, 12:39 PM
aaron.ballman added inline comments.
clang/lib/Format/UnwrappedLineParser.cpp
754

I had originally considered that, but it turns out that parsePPElif() just calls parsePPElse() and doesn't care about the condition whatsoever.

clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
924

I'm not 100% certain myself.

clang/lib/Lex/PPDirectives.cpp
651

I think I'd have to pass Directive.str().c_str() to ensure I get a null terminated string to pass (CheckEndOfDirective() takes a const char *), and that involves an allocation, so I think this form may be better? I don't have a strong opinion though.

3288

Good catch, I'll correct this.

3297

Same here.

erichkeane added inline comments.Apr 23 2021, 12:41 PM
clang/lib/Lex/PPDirectives.cpp
651

Ugh, gross. Didn't realize it took a const char*. This is the better of the two, ugly, but at least non-allocating.

aaron.ballman marked 4 inline comments as done.

Updated based on review feedback.

I have one 'nit of preference', otherwise I think I don't want to +1 this without giving people a few days to take a look. Based on my looks through the surrounding code, this _LOOKS_ right enough as far as I can tell, but I still want to give others a few days.

clang/lib/Lex/PPDirectives.cpp
629

As you know, I'm a giant fan of using enums for cases like this, so it would be my preference to have something like that. Sadly it looks like it would have to be file-level here instead of functionlevel like is most convenient.

clang/test/Preprocessor/elifdef.c
72

Are there any tests you could do to make sure this 'works'? That is:

#define BAR
// expected-error@+3 {{"AN ERROR!"}}
#ifdef FOO
#elifdef BAR
#error "AN ERROR!"
#endif

AND

// no error expected here!
#ifdef FOO
#elifndef BAR
#error "AN ERROR!"
#endif

And perhaps...

// expected-error@+4 {{"AN ERROR AGAIN!"}}
#ifdef FOO
#elifdef BAR
#else
#error "AN ERROR AGAIN!"
#endif

aaron.ballman marked 2 inline comments as done.

Updated based on review feedback.

clang/lib/Lex/PPDirectives.cpp
629

Given that all the diagnostics are in the same file, I suppose that's fine.

clang/test/Preprocessor/elifdef.c
72

I think the first test is what's on lines 63-69, the third is what's on lines 43-47, but I'll add the second test because I don't think that's covered yet. Thanks for the suggestion!

I reviewed the source minimizer sections, and the code looks correct.

It'd probably be good to have unit tests in clang/unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp for the incremental changes.

  • Do the new constructs stick around in the minimized source?
  • Do they get paired correctly for the ppranges stuff? (I think you can modify or create new versions of SkippedPPRangesBasic and SkippedPPRangesNested.)
clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
909

Please add the two new cases to this comment as well.

924

This part LGTM.

aaron.ballman marked 3 inline comments as done.

Updates based on review feedback.

Added tests for the source minimizer changes, thanks @dexonsmith!

clang/lib/Lex/DependencyDirectivesSourceMinimizer.cpp
909

Good catch, thanks!

Wow thanks for doing this! I worked on it a couple days a while ago but I abandoned the effort and went back to my day job. It seems like preprocessing ought to be something like a "state machine" but I couldn't figure out the mechanism. Would it make sense to add some kind of high level description of the components, now that you've gone to the [presumably massive] effort of understanding it? Just a couple small comments above.

clang/lib/Lex/PPDirectives.cpp
1059

please put a comment on 'false'

1063

please put a comment on true

aaron.ballman marked 3 inline comments as done.

Updating based on review comments.

Wow thanks for doing this! I worked on it a couple days a while ago but I abandoned the effort and went back to my day job.

Happy to help!

It seems like preprocessing ought to be something like a "state machine" but I couldn't figure out the mechanism. Would it make sense to add some kind of high level description of the components, now that you've gone to the [presumably massive] effort of understanding it? Just a couple small comments above.

Adding some developer documentation about the preprocessor may not be a bad idea in general, but I think that's orthogonal to this patch. We do have a place to add those kind of docs and it looks like the preprocessor is largely not mentioned: https://clang.llvm.org/docs/InternalsManual.html#the-lexer-and-preprocessor-library. However, I'd prefer not to sign up to write those docs at this time.

rsmith added inline comments.May 6 2021, 3:12 PM
clang/lib/Lex/PPDirectives.cpp
3265–3276

Hm, is the strict checking here appropriate? I'd expect skipping to start as soon as we hit the #elifdef directive, so the restriction on the form of the directive should be relaxed and we should just skip to the end of the line. ("When in a group that is skipped (15.2), the directive syntax is relaxed to allow any sequence of preprocessing tokens to occur between the directive name and the following new-line character."). For example, given:

#if 1
#elif
#endif

... we don't diagnose, even though the #elif line doesn't match the usual form for the directive (a constant-expression would require at least one token between elif and new-line), and I'd expect #elifdef to behave the same way.

With this fixed, the handling of #elif, #elifdef, and #elifndef (when not skipping) should be identical other than which callback is invoked; can the implementations be combined?

3302

This doesn't seem right to me: the macro's existence or non-existence does not affect preprocessing, so the macro was not used. But I assume this and the references to MD below will also be removed once this function stops parsing a macro name.

Updated based on review feedback.

aaron.ballman marked 2 inline comments as done.May 7 2021, 7:59 AM
aaron.ballman added inline comments.
clang/lib/Lex/PPDirectives.cpp
3265–3276

Good point! I think this means we'd need a second set of callbacks though -- we want the macro information when we process a #elifdef or a #elifndef that is taken (same as with #ifdef), so we'd need the interface that takes a MacroDefinition, but if we skip reading that because we're skipping the entire conditional block, we'd need a callback that takes the condition range.

I went that route with the updated patch. LMK if that works for you.

aaron.ballman marked an inline comment as done.

Now, with ever-so-slightly better formatting (the rest of the formatting issues are either matching existing style or a fight between my clang-format and CI's clang-format).

mibintc accepted this revision.May 17 2021, 7:19 AM

let 'er rip

This revision is now accepted and ready to land.May 17 2021, 7:19 AM

let 'er rip

Thanks @mibintc!

@rsmith -- I'll land this sometime later this week unless I hear further comments from you. If I land it and you still have feedback, I can address it post commit.

bjope added a subscriber: bjope.Jun 1 2021, 8:37 AM
bjope added inline comments.
clang/lib/Lex/PPDirectives.cpp
594

Hi @aaron.ballman

This change is missing from https://reviews.llvm.org/rG8edd3464afbff65d7d5945b3a8b20009d6ff5deb , while you instead got it when doing the pp_err_else_after_else diagnostic a couple of lines above this.

It causes asserts (in DIagnostic::getArgKind) for me in test cases verifying the "elif after else" scenario using a test case basically doing

#if 1
#else
#elif
#endif

So maybe such a test case should be added as well?

But it seems like the patch you committed doesn't match with the reviewed patch. So maybe you can look into this and fix it?
(let me know if you need some additional help)

aaron.ballman added inline comments.Jun 1 2021, 11:07 AM
clang/lib/Lex/PPDirectives.cpp
594

Great catch and thank you for the test case! It seems that when I applied the patch, it misapplied, but not in a way that caused a test failure. I have no idea how that happened, but I assume I screwed up something with git somewhere along the way and caused the failure.

I've pushed baa2b8d08502acfa91a8dfd699d25f7b4e25edbb to fix it. Thanks!

bjope added inline comments.Jun 1 2021, 11:32 AM
clang/lib/Lex/PPDirectives.cpp
594

Big thanks for the quick fix! While I managed to track down this problem I didn't have time to properly fix it right away. Now I'll have one thing less to worry about tomorrow :-)

aaron.ballman added inline comments.Jun 1 2021, 11:51 AM
clang/lib/Lex/PPDirectives.cpp
594

Happy to fix my own messes!

In this version of the patch, I am supporting the feature in all C and C++ modes. This seems like more useful functionality than diagnosing an unknown preprocessor directive in older language modes, and is a conforming extension in those modes anyway as use of an unknown directive is undefined behavior.

Is this really true? I think:

#define X
#if 0
#elifdef X
#error
#endif

... is a valid translation unit in C++ and is strictly conforming in C11 and earlier; any conditionally-supported-directive (using the C++ terminology) within a group is ignored when skipping that group.

That said... applying this to all language modes does seem like the right choice, even though it may be technically non-conforming.