Added support for FE Clang to create Debug Info entries (DIMacro and DIMacroFile) into generated module if "debug-info-kind" flag is set to "standalone".
Details
- Reviewers
bwyma dblaikie erichkeane probinson aprantl rsmith - Commits
- rG546bc1103b68: [DebugInfo] Added support to Clang FE for generating debug info for…
rC294637: [DebugInfo] Added support to Clang FE for generating debug info for…
rL294637: [DebugInfo] Added support to Clang FE for generating debug info for…
Diff Detail
- Repository
- rL LLVM
Event Timeline
include/clang/AST/ASTConsumer.h | ||
---|---|---|
163 ↗ | (On Diff #44715) | Richard, If you still think that this approach is not the right one, I would appreciate it if you can explain again how to implement the macro support differently. |
Added a few stylistic comments and request for more documentation.
lib/CodeGen/MacroPPCallbacks.cpp | ||
---|---|---|
1 ↗ | (On Diff #44715) | Please run your patch through clang-format. |
62 ↗ | (On Diff #44715) | Comment why SkipFiles is initialized to 2? |
74 ↗ | (On Diff #44715) | Comments? |
91 ↗ | (On Diff #44715) | Please add a comment explaining this condition. |
lib/CodeGen/MacroPPCallbacks.h | ||
34 ↗ | (On Diff #44715) | Please comment the purpose of these fields. |
45 ↗ | (On Diff #44715) | Since r237417 LLVM enabled autobrief, so it's no longer recommended to use \brief for one-liners. |
lib/CodeGen/CGDebugInfo.cpp | ||
---|---|---|
2099 ↗ | (On Diff #46034) | By using a bool for IsUndef, we'll end up with somewhat ugly call sites like DI->CreateMacro(Parent, /*IsUndef*/ true, ...); so we might as well just pass in the dwarf constant DI->CreateMacro(Parent, llvm::dwarf::DW_MACINFO_undef, ...); directly. |
Hi Amjad,
are you still planning on getting this patch and https://reviews.llvm.org/D16077 committed ? It looks like these two patches are final pieces in the puzzle to get macro information in the DWARF debug output.
I've downloaded the diffs and applied them myself on my local checkout and they seem to work fine. If you would like me to upload the rebased patches onto phabricator to save you the trouble of having to the fix conflicts downstream then let me know.
Thanks,
Ranjeet
Updated patch to top of trunk (r276746) - Thanks to Ranjeet Singh.
Please, provide your feedback.
include/clang/AST/ASTConsumer.h | ||
---|---|---|
163 ↗ | (On Diff #44715) | Don't add this to ASTConsumer; an AST consumer and a PP consumer are two largely separate ideas. Instead, you could create and register a PPCallbacks object directly from CodeGenAction::CreateASTConsumer. In fact, if you look there, a PPCallbacks subclass is already registered (for code coverage); you can do something similar to register your MacroPPCallbacks object. |
lib/CodeGen/CMakeLists.txt | ||
76 ↗ | (On Diff #65551) | This file is missing from the diff. |
lib/CodeGen/CodeGenModule.cpp | ||
28 ↗ | (On Diff #65551) | This file is missing from the diff. |
Thanks Richard for reviewing the patch and for the comments.
Please, see answers below.
Regards,
Amjad
include/clang/AST/ASTConsumer.h | ||
---|---|---|
163 ↗ | (On Diff #44715) | Thanks, I will try this approach, it is indeed cleaner. |
lib/CodeGen/CMakeLists.txt | ||
76 ↗ | (On Diff #65551) | Sorry about that, it seems when I updated the code these files were removed from the patch. By the way, you can find an old version of them in the previous diff. I will update them and upload them to the review together with the above suggestion of yours. |
Improved code based on Richard's comments.
Removed the change to ASTConsumer, now it does not need to know anything about PP consumer.
However, I still needed to change the CodeGenerator ASTConsumer to return the CGDebugInfo class.
Richard, please review the new patch.
This time I made sure to add the "MacroPPCallbacks.*" new files.
I admit it would be nice if someone reviewed this, but I'm really the wrong person for this code area. :-)
lib/CodeGen/MacroPPCallbacks.cpp | ||
---|---|---|
31 ↗ | (On Diff #82696) | std::next() ? |
44 ↗ | (On Diff #82696) | LLVM style prefers comments to be full sentences and above the code. |
83 ↗ | (On Diff #82696) | I think there is an LLVM_FALLTHROUGH macro (or something to that end) |
154 ↗ | (On Diff #82696) | #include as opposed to #import? If so, why? |
62 ↗ | (On Diff #44715) | Thanks. I think this comment should now go into the header and be promoted to the Docygen comment for the CommandIncludeFiles. Also consider doing CommandIncludeFiles = 0 in the class definition. |
lib/CodeGen/ModuleBuilder.cpp | ||
68 ↗ | (On Diff #82696) | What does Ref stand for? |
include/clang/CodeGen/ModuleBuilder.h | ||
---|---|---|
71 ↗ | (On Diff #82696) | Typo 'calss' |
72 ↗ | (On Diff #82696) | What is the returned value in that case? A reference to a null pointer? Generally, I'm not particularly happy with this approach of exposing a reference to an internal pointer as the way of propagating the CGDebugInfo to the macro generator as needed. Instead, how about giving the MacroPPCallbacks object a pointer to the CodeGenerator itself, and having it ask the CodeGenModule for the debug info object when it needs it? |
73 ↗ | (On Diff #82696) | caller -> the caller's |
74 ↗ | (On Diff #82696) | start using -> starting to use |
lib/CodeGen/MacroPPCallbacks.cpp | ||
1 ↗ | (On Diff #82696) | Wrong file header (should be .cpp, and you don't need the "-*- C++ -*-" in a .cpp file. |
Thanks Adrian and Richard for the comments, I appreciate that.
I am uploading a new patch that address most of the comments.
Adrian, I answered some of your comments below.
Thanks,
Amjad
lib/CodeGen/MacroPPCallbacks.cpp | ||
---|---|---|
31 ↗ | (On Diff #82696) | I am not that satisfied with this function, and I hope we can do it in a better way. /// PrintMacroDefinition - Print a macro definition in a form that will be /// properly accepted back as a definition. static void PrintMacroDefinition(const IdentifierInfo &II, const MacroInfo &MI, Preprocessor &PP, raw_ostream &OS) { |
44 ↗ | (On Diff #82696) | As I said above, this code is taken from "lib\Frontend\PrintPreprocessedOutput.cpp" But, I do not mind fixing this, if it is the preferred comment style. |
154 ↗ | (On Diff #82696) | I am not familiar with "#import" keyword.
The reason I have this switch below, is that I am not sure if we call this "InclusionDirective" callback for identifiers that are not "#include". However, I see no harm of updating the LastHashLoc every time we enter this function. The value will not be used unless it is set just before calling "FileChanged"callback. |
lib/CodeGen/ModuleBuilder.cpp | ||
68 ↗ | (On Diff #82696) | Ref - stands for reference, I think I do not need the "Ref" prefix for this member, only for the get function. |
Address Adrian and Richard comments.
Please, review the new patch and let me know if I need to change anything else.
Thanks,
Amjad
Thanks! Couple more inline comments.
lib/CodeGen/CGDebugInfo.h | ||
---|---|---|
415 ↗ | (On Diff #86669) | It would be good to be a bit more descriptive here. If the comment just repeats the name of the function it is not worth keeping, |
420 ↗ | (On Diff #86669) | E.g., |
lib/CodeGen/MacroPPCallbacks.cpp | ||
88 ↗ | (On Diff #86669) | I think the entire function can be replaced with if ((Status == MainFileScope) || (Status == CommandLineScopeIncludes && CommandIncludeFiles)) return Loc; return SourceLocation(); |
100 ↗ | (On Diff #86669) | Could you try splitting this function up into two functions? MacroPPCallbacks::FileEntered() and the common code factored out into a helper? The control flow here seems unnecessarily complex with all the flags. |
177 ↗ | (On Diff #86669) | calculat*e*MacroDefinition |
44 ↗ | (On Diff #82696) | I think it would be a good idea to clean this up. Feel free to either do this now or in a separate commit. |
154 ↗ | (On Diff #82696) | #import comes from Objective-C. It is equivalent to #include + #pragma once. |
Thanks Adrian for the fast response.
See some answers below.
I will update the code and upload a new patch soon.
lib/CodeGen/MacroPPCallbacks.cpp | ||
---|---|---|
88 ↗ | (On Diff #86669) | Sure, will do. |
100 ↗ | (On Diff #86669) | Will try restructure. I will see what I can do to make the code much clear. |
177 ↗ | (On Diff #86669) | How about: Do you think any of the above would be better choice? |
154 ↗ | (On Diff #82696) | I will remove the switch and update the LastHashLoc unconditionally. |
Addressed Adrian's comments.
Please, let me know if I missed any.
Also, I improved the code and added explanation about the file inclusion structure that is being handled in the macro callbacks.
Few more inline comments.
I think the code itself is looking good now. Why doesn't this patch include any test cases? I would have expected some that exercise the command line include handling, etc., ...
lib/CodeGen/MacroPPCallbacks.h | ||
---|---|---|
35 ↗ | (On Diff #87020) | s/was/were/ |
37 ↗ | (On Diff #87020) | What about EnteredCommandLineIncludes or EnteredCommandLineIncludeFiles? |
Addressed Adrian last comments.
Added a LIT tests that covers all the macro kinds:
- built-in (define)
- command-line (define, include, undef)
- main source (define, include, undef)
Checked the above with and without PCH include.
Notice, that current implementation does not support debug info for macro definition and inclusion generated during the PCH file creation.
To support that, we need to extend the PCH format to preserve this information.
If you believe this feature is important, I will open a bugzilla ticket and we can handle it separately.
That would be a good idea and important to fix. I don't think we want the the generated code (or the debug info for that matter) to change just because the user decides to switch to PCH. Ideally turning on PCH should be transparent.
I have one final question: What's the impact of this change on, e.g., building clang?
- How much does the build directory grow?
- Is there any noticeable compile time regression?
test/CodeGen/debug-info-macro.c | ||
---|---|---|
14 ↗ | (On Diff #87408) | Alternatively, you may be able to use #line to force an easily recognizable line number (unless that messes with the macro debug info, or you can use FileCheck's [[@LINE-offset]] feature. |
I will run self-build and come back with answers for these questions.
I am just wondering, if I build in debug mode, will it use the flag "-fstandalone-debug"?
This flag is set by default only for DarwinClang and FreeBSD, is there a way to assure it will be used with self-build?
Notice that without this flag, there will be no debug info macro generated.
How much does the build directory grow?
Is there any noticeable compile time regression?
I build clang in release mode using GCC, then used that build to build clang in debug mode with "-fstandalone-debug" flag, one time with my changes and another time without my changes.
Without my changes
Clang executable Size: 1,565MB
Build time: 17m57.504s
Withmy changes
Clang executable Size: 2,043MB
Build time: 17m58.008s
Do, build time is the same, but binary size was increased about 30%.
However, remember that we are emitting macro debug info in dwarf4 format, once we support emitting it in dwarf5 format the size should be reduced significantly.
I'm not sure it makes sense to motivate this feature with "-fstandalone-debug" flag.
Is "-fmacro-debug" flag sounds good?
Should we extend DebugInfoKind enumeration to support the debug info macro? Or we should add a new option to CodeGenOptions?
Added flag to control macro debug info generation:
- -fdebug-macro (-fno-debug-macro) for Clang driver
- -debug-info-macro for Clang compiler (-cc1)
Also, made sure FE will discard this flag when debug info is not required, i.e. with -g0.
Should we discard the macro debug info when "-gline-tables-only" is used? or it is up to the user not to use "-fdebug-macro" in this case?
I also extended the test to check all combinations of using -debug-info-macro flag with -debug-info-kind flag.
How are you measuring the build time? Total time for, say "ninja clang" with full parallelism? That'd be hard to measure the actual impact (since it could be the link time or other things are dominating, etc). If you have a reliable way to time (I'm assuming Intel has lots of tools for measuring compiler performance) the compilation, get a sense of the variance, etc (& link time, to get a sense of how much the larger inputs affect linker performance) would be useful.
I did a manual measurement, but I believe it is representative.
I used "time" tool to measure the whole time of the build (compile + link).
- time make -j8
Also, notice that I limited the targets to X86 only, so it was not a full build of LLVM.
- -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_CXX_FLAGS="-fstandalone-debug" -DCMAKE_C_FLAGS="-fstandalone-debug"
However, as I already said, it should answer the time change and size question fairly enough.
Does GCC have a command line option for this that we could mirror?
GCC emits macro debug info when when build with -g3.
Please also add driver testcase (e.g., to test/Driver/debug-options.c).
Technically clang does accept -g3 as an option, so we could wire it up for compatibility. I would still keep the -fmacro-debug driver option though.
docs/UsersManual.rst | ||
---|---|---|
1692 ↗ | (On Diff #87796) | Debug info for C preprocessor macros ... |
1697 ↗ | (On Diff #87796) | Generate debug info for preprocessor macros. ... |
Added test cases for driver debug info macro flags in test/Driver/debug-options.c
Applied some changes to the documentation suggested by Ardian.
Could you also add support for -g3? It looks like we are supporting -g0 and -g1, so we should probably also mirror -g3.
Thanks a lot for all your work!
cfe/trunk/lib/CodeGen/MacroPPCallbacks.cpp | ||
---|---|---|
125 | As a heads up... this fails under -Werror: llvm/tools/clang/lib/CodeGen/MacroPPCallbacks.cpp:125:3: error: default label in switch which covers all enumeration values [-Werror,-Wcovered-switch-default] default: ^ 1 error generated. |
As a heads up... this fails under -Werror:
llvm/tools/clang/lib/CodeGen/MacroPPCallbacks.cpp:125:3: error: default label in switch which covers all enumeration values [-Werror,-Wcovered-switch-default]
1 error generated.