Page MenuHomePhabricator
Feed Advanced Search

Mon, Jul 22

dblaikie added a comment to D65026: [Bugpoint redesign] Added pass to reduce Metadata.

The goal of this pass is to be able to remove as many intrinsic calls and metadata (debug info and non-debug info metadata alike) while still preserving the "interestingness". I would figure maybe some more testing would be appropriate - since this currently only tests (if I understand the test) that it can remove all of these things - not that it preserves any of them at any time? Perhaps some tests for certain interesting metadata and intrinsics to ensure some are preserved?

Mon, Jul 22, 4:45 PM · Restricted Project
dblaikie committed rG9ec6f9e07e6b: llvm-objcopy/test: add REQUIRES: shell for use of umask (authored by dblaikie).
llvm-objcopy/test: add REQUIRES: shell for use of umask
Mon, Jul 22, 3:26 PM
dblaikie committed rL366755: llvm-objcopy/test: add REQUIRES: shell for use of umask.
llvm-objcopy/test: add REQUIRES: shell for use of umask
Mon, Jul 22, 3:24 PM
dblaikie accepted D64544: [DWARF] Add more error handling to debug line parser..

Generally looks good (optional thoughT: maybe the .test file and the .s file could be merged (so the CHECK lines could sit near the assembly they're checking, would make it easier to eyeball whether the tests are testing the right thing, etc)

Mon, Jul 22, 3:15 PM · Restricted Project
dblaikie added inline comments to D65026: [Bugpoint redesign] Added pass to reduce Metadata.
Mon, Jul 22, 3:14 PM · Restricted Project
dblaikie added a comment to D65096: Add seek support into llvm::raw_ostream that can be conditionally enabled..

Would raw_pwrite_stream suffice?

It actually would as I only need to fixup a few things for FileWriter in GSYM. Keeping this patch does allow us to use raw_ostream more file a seekable file, where the pwrite forces you to just do minimal fixups here and there. I am fine abandoning this if no one else thinks it is useful. Comments?

Mon, Jul 22, 1:55 PM
dblaikie added a comment to D65096: Add seek support into llvm::raw_ostream that can be conditionally enabled..

Would raw_pwrite_stream suffice?

Mon, Jul 22, 1:41 PM

Wed, Jul 17

dblaikie added a comment to D64544: [DWARF] Add more error handling to debug line parser..

I was rather hoping for test coverage for all the new error messages this change introduced - is that unrealistic/impractical?

Oh, sure, I was focussed on the MD5 one because that's the only one that's "new". I'll make sure the other ones are covered too.

Yeah, the line table is especially tricky to hand-craft compared to checking in an object file. I think it technically can still be hand-crafted assembly (no line directives, etc, just a debug_line section with raw byte (etc) directives) - might be plausible & make it clearer what the input is? (checked in assembly, assembled with llvm-mc then run through llvm-dwarfdump to test the parsing)

Hear hear. It looks like this patch is all about diagnosing issues in the v5 prologue/file-table, and it should be relatively easy to construct bogus examples in assembler. You don't need a line-number program at all. There ought to be examples of *valid* v5 line-table headers lying around, I know I did all the dumper work with assembler tests before I ever generated any v5 headers.

How would you be able to emit a wrong form value for the MD5 hash from assembly? The directive (which I think you came up with?) is just md5 [data]? Deciding which form to emit is hard-coded in MCDwarf.

Wed, Jul 17, 11:41 AM · Restricted Project
dblaikie added inline comments to D63713: Add error handling to the DataExtractor class.
Wed, Jul 17, 11:01 AM · Restricted Project
dblaikie added a comment to D64544: [DWARF] Add more error handling to debug line parser..

I was rather hoping for test coverage for all the new error messages this change introduced - is that unrealistic/impractical?

Wed, Jul 17, 10:45 AM · Restricted Project

Tue, Jul 16

dblaikie committed rG40580d36c4de: DWARF: Skip zero column for inline call sites (authored by dblaikie).
DWARF: Skip zero column for inline call sites
Tue, Jul 16, 2:20 PM
dblaikie committed rL366264: DWARF: Skip zero column for inline call sites.
DWARF: Skip zero column for inline call sites
Tue, Jul 16, 2:15 PM
dblaikie closed D64784: Skip zero column for inline sites.
Tue, Jul 16, 2:15 PM · Restricted Project

Mon, Jul 15

dblaikie accepted D64407: [DWARF] Simplify DWARFAttribute. NFC..
Mon, Jul 15, 9:33 PM · Restricted Project
dblaikie accepted D64784: Skip zero column for inline sites.
Mon, Jul 15, 9:16 PM · Restricted Project
dblaikie added a comment to D64784: Skip zero column for inline sites.

Test coverage? (you could add another inlined call to the test case you added for the feature - but just leave the inlined-at location with a zero column for that particular call - so you can still test both positive/negatiev in the one file)

Mon, Jul 15, 8:23 PM · Restricted Project
dblaikie added a comment to D64033: Add column info for inline sites.

What's the motivation for this?

We need to differentiate inline sites on the same line for an internal (basically the tool needs to annotate each instruction with stack of inline site locations). And the missing column info for inline sites gets in the way. I thought the change will be helpful to others as well..

Mon, Jul 15, 8:14 PM · Restricted Project
dblaikie accepted D64176: [Bugpoint redesign] Added Pass to Remove Global Variables.
Mon, Jul 15, 3:53 PM · Restricted Project
dblaikie added a comment to D64033: Add column info for inline sites.

What's the motivation for this?
Should it be conditional on "-gcolumn-info"? (or skipping it in general if the column is zero? that'd make "-gno-column-info" fall out naturally)
(we don't put decl_column on subprogram DIEs, for instance - admittedly, it's more likely the same function could have multiple inline call sites on the same line than you'd have multiple functions defined on the same line)

Mon, Jul 15, 3:49 PM · Restricted Project
dblaikie accepted D64774: [DebugInfo] Move function from line table to the prologue (NFC).
Mon, Jul 15, 3:07 PM · Restricted Project, Restricted Project, debug-info
dblaikie added inline comments to D64176: [Bugpoint redesign] Added Pass to Remove Global Variables.
Mon, Jul 15, 2:43 PM · Restricted Project
dblaikie added inline comments to D64176: [Bugpoint redesign] Added Pass to Remove Global Variables.
Mon, Jul 15, 1:36 PM · Restricted Project
dblaikie added a comment to D64544: [DWARF] Add more error handling to debug line parser..

Looks like this code already had rudimentary error handling along this codepath where the assert is (the function returns empty, which returns false, which becomes an error further up, I think?) - and adding more descriptive/precise error handling along this path is probably good - but probably also means more testing is required to demonstrate all the new error results/messages/codepaths?

Mon, Jul 15, 1:29 PM · Restricted Project

Fri, Jul 12

dblaikie accepted D64601: [MemorySSA] Use SetVector to avoid nondeterminism..

Sounds good!

Fri, Jul 12, 3:13 PM · Restricted Project
dblaikie added a comment to D64601: [MemorySSA] Use SetVector to avoid nondeterminism..

I don't know if a bot exists with LLVM_ENABLE_REVERSE_ITERATION set to true, but if it does, then the check I proposed for the uselist order would likely fail, no?

Fri, Jul 12, 1:49 PM · Restricted Project
dblaikie added a comment to D64601: [MemorySSA] Use SetVector to avoid nondeterminism..

Hmm, AFAICT, there is no opt flag equivalent for LLVM_ENABLE_REVERSE_ITERATION. Did I miss it?

Fri, Jul 12, 12:38 PM · Restricted Project

Thu, Jul 11

dblaikie added inline comments to D64176: [Bugpoint redesign] Added Pass to Remove Global Variables.
Thu, Jul 11, 4:46 PM · Restricted Project
dblaikie added a comment to D64601: [MemorySSA] Use SetVector to avoid nondeterminism..

You can use SetVector's "getArrayRef" and pass it as "void removeBlocks(ArrayRef<BasicBlock *> DeadBlocks);"

removeBlocks is querying DeadBlocks with count(...). I can change the code to walk the Vector/ArrayRef, but it defeats the purpose of building a set.
It may be ok to change all callsites to just use vector too, assuming the normal expected size is small, but it may be costly for pathological cases (large number of deadblocks).
If folks have a preference either way, please let me know.

Thu, Jul 11, 3:45 PM · Restricted Project
dblaikie added a comment to D64601: [MemorySSA] Use SetVector to avoid nondeterminism..

I'd be interested if there's a better way to use a SmallSetVector, without needing the "8". I don't see an Impl version.
The only alternative I came up with was something like:

template <class OrderedSetT>
void removeBlocks(const OrderedSetT &DeadBlocks);
Thu, Jul 11, 3:27 PM · Restricted Project
dblaikie added a comment to D63031: DebugInfo: Render the canonical name of a class template specialization, even when nested in another class template specialization.

Ping

Thu, Jul 11, 11:12 AM · Restricted Project
dblaikie added a comment to D61584: [DebugInfo] Some fields do not need relocations even relax is enabled..

Does this solution extend/address other label deltas, like those that would appear in the other DWARF/non-frame sections? (eg: currently we use a length in the DWARF high_pc because it means not having to use a relocation for high_pc, which reduces object size - so, does this fix cause relocations to be used for that length expression to ensure they're correct if the linker changes the instructions in that range?)

After D45181 committed, LLVM will generate relocation pairs for symbol difference expressions if target enables relaxation. So, it will generate relocations for high_pc due to it is a symbol difference expression of end symbol and start symbol of the function range. Such as,

.word   .Lfunc_begin0           # DW_AT_low_pc
.word   .Lfunc_end4-.Lfunc_begin0 # DW_AT_high_pc

This commit modified the condition to generate relocations for symbol difference expressions. Not all symbol difference expressions are related to executable code. However, only executable code will be modified by linker for relaxation. So, we do not need to generate relocations for symbol differences not located in executable code sections.

Good to know - so has LLVM been creating incorrect DWARF in these cases so far - or has no target enabled linker relaxation yet?

Is there any way to make this more targeted - does linker relaxation allow arbitrary machine code changes? Or is there something more fine-grained - so we could compute compile-time constants for some label differences, but only use linker relocations for cases where we know the instruction sequence includes something relaxable?

AFAIK only RISC-V enables this feature. Currently, .debug_frame/.eh_frame is wrong after relaxation. I already fixed parts of the problem in D58335. In D58335, .debug_frame/.eh_frame will generate relocations if relaxation is enabled. Otherwise, it will not aware relaxation at all and it doesn't work to do frame inspection when debugging.

Linker relaxation processes some patterns of instruction sequences. For example, RISC-V could use auipc/jalr to jump to 32-bits pc-relative address. If linker knows the jump target is within +/-1 Mib, it could replace auipc/jalr to jal. The replacement will shorten the instruction sequence. So, the label differences between these instruction sequences may be changed after relaxation. That is why we need to annotate relocation types for these symbol differences in code sections.

Currently, there is no way to make it more targeted. When relaxation is enabled, it assumes all code sequences may be changed after relaxation. If we want to make placement of relocations more specific, we need to inspect every instruction between two symbols when we create symbol difference expressions for them. I think it will do a lot of work to gain few benefits. (Fewer relocations, smaller object code, less linker efforts, etc.)

Thu, Jul 11, 7:24 AM · Restricted Project

Wed, Jul 10

dblaikie added a comment to D61584: [DebugInfo] Some fields do not need relocations even relax is enabled..

Does this solution extend/address other label deltas, like those that would appear in the other DWARF/non-frame sections? (eg: currently we use a length in the DWARF high_pc because it means not having to use a relocation for high_pc, which reduces object size - so, does this fix cause relocations to be used for that length expression to ensure they're correct if the linker changes the instructions in that range?)

After D45181 committed, LLVM will generate relocation pairs for symbol difference expressions if target enables relaxation. So, it will generate relocations for high_pc due to it is a symbol difference expression of end symbol and start symbol of the function range. Such as,

.word   .Lfunc_begin0           # DW_AT_low_pc
.word   .Lfunc_end4-.Lfunc_begin0 # DW_AT_high_pc

This commit modified the condition to generate relocations for symbol difference expressions. Not all symbol difference expressions are related to executable code. However, only executable code will be modified by linker for relaxation. So, we do not need to generate relocations for symbol differences not located in executable code sections.

Wed, Jul 10, 9:31 PM · Restricted Project
dblaikie added inline comments to D64407: [DWARF] Simplify DWARFAttribute. NFC..
Wed, Jul 10, 10:42 AM · Restricted Project

Tue, Jul 9

dblaikie added inline comments to D64407: [DWARF] Simplify DWARFAttribute. NFC..
Tue, Jul 9, 4:54 PM · Restricted Project

Mon, Jul 8

dblaikie committed rG793231c319f0: [cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default… (authored by dblaikie).
[cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default…
Mon, Jul 8, 4:26 PM
dblaikie committed rL365406: [cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default….
[cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default…
Mon, Jul 8, 4:25 PM
dblaikie closed D64058: [cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default-constructible and assignable..
Mon, Jul 8, 4:24 PM · Restricted Project, Restricted Project
dblaikie accepted D64203: [OpaquePtr] add Type parameter to Loads.h analysis API..

Looks good :)

Mon, Jul 8, 4:09 PM · Restricted Project
dblaikie added a comment to D63894: [CXX] Exercise all paths through these tests.

I would've assumed these conditionals were added by Sony folks for their change in default dialect - that doesn't necessarily mean these tests are needed upstream (the functionality may be tested elsewhere)

Mon, Jul 8, 2:27 PM · Restricted Project, Restricted Project
dblaikie added a comment to D61584: [DebugInfo] Some fields do not need relocations even relax is enabled..

Does this solution extend/address other label deltas, like those that would appear in the other DWARF/non-frame sections? (eg: currently we use a length in the DWARF high_pc because it means not having to use a relocation for high_pc, which reduces object size - so, does this fix cause relocations to be used for that length expression to ensure they're correct if the linker changes the instructions in that range?)

Mon, Jul 8, 12:57 PM · Restricted Project
dblaikie accepted D64058: [cxx2a] P0624R2 fix: only lambdas with no lambda-capture are default-constructible and assignable..

Looks good to me. (possible the check (for no default capture and no captures) could be unified with the same check/language used for the implicit function pointer conversion operator ("if (Captures.empty() && CaptureDefault == LCD_None)" - around 1744 in SemaLambda.cpp) - maybe a common utility function to test this condition or the like. But it's hardly a big thing to have it written in two places, really)

Mon, Jul 8, 12:40 PM · Restricted Project, Restricted Project

Wed, Jul 3

dblaikie added a comment to D64006: [Support] Support 64-bit offsets in DataExtractor..

I think the main one is code duplication - write once/fix once/etc is valuable.

In that case, what do you think about templates?

Wed, Jul 3, 11:25 AM · Restricted Project
dblaikie added a comment to D63672: Added Delta IR Reduction Tool.

Did the broader design discussion about bugpoint lead to this design (a separate llvm-reduce tool)? I didn't quite follow that thread far enough to see the conclusion that lead to this direction (rather than improving/modifying bugpoint) - might be worth writing a bit more in the commit description about the conversation (link to the thread if that's helpful) and design choices - and long-term plan (to merge behavior, share things, remove one or the other - or should these two tools exist indefinitely with different goals/uses?)

Thanks for the comment, I've clarified it in the description. As well as a brief description of how the tool is structured.

The intent is to build this separately, then fold it into bugpoint? What's the tradeoff/motivation for that compared to building this into bugpoint to begin with? (seems like it presents a risk that that unification doesn't happen & we're left with a fractured solution ('one more competing standard' sort of situation)) Is the desire to get this reviewed/committed sooner while pursuing some bugpoint refactoring necessary before unification, without delaying implementation of this tool until after that refactoring?

The main reason I chose this route was to not interfere with bugpoint's current users, especially because there's low test coverage, these changes might end up doing more harm than good. Additionally, some community members also voiced their concerns about how the redesign might affect their current workflow.

I also agree the risks you mentioned are very real indeed, and I'm not still 100% sure this might be the best approach so other routes might be worth exploring.

Wed, Jul 3, 11:11 AM · Restricted Project
dblaikie added a comment to D63672: Added Delta IR Reduction Tool.

Did the broader design discussion about bugpoint lead to this design (a separate llvm-reduce tool)? I didn't quite follow that thread far enough to see the conclusion that lead to this direction (rather than improving/modifying bugpoint) - might be worth writing a bit more in the commit description about the conversation (link to the thread if that's helpful) and design choices - and long-term plan (to merge behavior, share things, remove one or the other - or should these two tools exist indefinitely with different goals/uses?)

Thanks for the comment, I've clarified it in the description. As well as a brief description of how the tool is structured.

Wed, Jul 3, 9:27 AM · Restricted Project
dblaikie added inline comments to D63672: Added Delta IR Reduction Tool.
Wed, Jul 3, 9:27 AM · Restricted Project
dblaikie added a comment to D64006: [Support] Support 64-bit offsets in DataExtractor..

...in places where it likely won't be needed any time soon, such as 64-bit DWARF parsing support.

In fact, I am going to implement 64-bit DWARF support. I started from DataExtractor because changing it seems inevitable for that anyway.

But it sounds like @ikudrin is suggesting maybe it's not their goal to migrate from 32 to 64 entirely, and that maybe it's OK to have both APIs?

In the beginning, I also thought that having only one possible type for offsets, uint64_t, is preferable. But thinking of that for some time, I cannot find any strong reason not to keep both variants, apart from some kind of aesthetic sense, which is debatable. I really do not see any new harm which may result from having overloads for both 32- and 64-bit offsets. The only issue I can guess is a potential risk of wrapping a 32-bit offset value, but the current implementation is also affected by that, so nothing new is here.

Wed, Jul 3, 8:50 AM · Restricted Project

Tue, Jul 2

dblaikie added inline comments to D63672: Added Delta IR Reduction Tool.
Tue, Jul 2, 4:00 PM · Restricted Project
dblaikie added a comment to D64006: [Support] Support 64-bit offsets in DataExtractor..

Thank you for demonstrating this! I agree that the patch also introduces lots of opportunities for subtle bugs. It also increases the memory footprint in places where it likely won't be needed any time soon, such as 64-bit DWARF parsing support. This patch looks much more palatable to me now and I'd be okay with taking it if @dblaikie is okay with it. Is there danger in using the same function name or should we call the 64-bit variants something more explicit?

Tue, Jul 2, 3:36 PM · Restricted Project
dblaikie added a comment to D63713: Add error handling to the DataExtractor class.

Ok, I got some numbers now. I went for a micro-benchmark-like setup to show some kind of a worst case scenario. The test setup is as follows:

I took the largest single .o file I had around (Registry.cpp.o in clangDynamicASTMatchers library). I then linked it to remove any relocations (otherwise, most of the time is spend applying those). Then I modified llvm-dwarfdump to parse the debug_loc section without dumping anything (to avoid measuring the time spend in printfs). Both llvm-dwarfdump and Registry.cpp.o I was dumping were built with -O3 -g, with asserts disabled (no FDO, LTO or other fancy stuff). This resulted in about 4.5 megabytes of debug_loc for parsing in Registry.cpp.o. Then I used the linux perf command to run llvm-dwarfdump -debug-loc 1000 times and dump the stats.

The baseline stats are:

  27.285986      task-clock:u (msec)       #    0.986 CPUs utilized            ( +-  0.11% )
          0      context-switches:u        #    0.000 K/sec                  
          0      cpu-migrations:u          #    0.000 K/sec                  
      2,813      page-faults:u             #    0.103 M/sec                    ( +-  0.24% )
 58,831,163      cycles:u                  #    2.156 GHz                      ( +-  0.07% )
    606,986      stalled-cycles-frontend:u #    1.03% frontend cycles idle     ( +-  3.76% )
  7,924,778      stalled-cycles-backend:u  #   13.47% backend cycles idle      ( +-  0.33% )
146,588,727      instructions:u            #    2.49  insn per cycle         
                                           #    0.05  stalled cycles per insn  ( +-  0.00% )
 29,545,620      branches:u                # 1082.813 M/sec                    ( +-  0.00% )
    222,276      branch-misses:u           #    0.75% of all branches          ( +-  0.15% )
 
0.027663381 seconds time elapsed                                          ( +-  0.11% )

The stats with this patch applied are:

  27.397390      task-clock:u (msec)       #    0.987 CPUs utilized            ( +-  0.10% )
          0      context-switches:u        #    0.000 K/sec                  
          0      cpu-migrations:u          #    0.000 K/sec                  
      2,833      page-faults:u             #    0.103 M/sec                    ( +-  0.24% )
 60,160,571      cycles:u                  #    2.196 GHz                      ( +-  0.07% )
    584,825      stalled-cycles-frontend:u #    0.97% frontend cycles idle     ( +-  3.37% )
 10,729,974      stalled-cycles-backend:u  #   17.84% backend cycles idle      ( +-  0.26% )
156,141,836      instructions:u            #    2.60  insn per cycle         
                                           #    0.07  stalled cycles per insn  ( +-  0.00% )
 31,599,940      branches:u                # 1153.392 M/sec                    ( +-  0.00% )
    221,247      branch-misses:u           #    0.70% of all branches          ( +-  0.06% )
 
0.027771865 seconds time elapsed                                          ( +-  0.10% )

The stats for a version of this patch which additionally checks for the error flag before attempting the parse (as discussed in the inline comment) are:

  27.808349      task-clock:u (msec)       #    0.986 CPUs utilized            ( +-  0.10% )
          0      context-switches:u        #    0.000 K/sec                  
          0      cpu-migrations:u          #    0.000 K/sec                  
      2,839      page-faults:u             #    0.102 M/sec                    ( +-  0.24% )
 62,887,388      cycles:u                  #    2.261 GHz                      ( +-  0.06% )
    575,264      stalled-cycles-frontend:u #    0.91% frontend cycles idle     ( +-  3.18% )
 14,757,888      stalled-cycles-backend:u  #   23.47% backend cycles idle      ( +-  0.23% )
167,562,307      instructions:u            #    2.66  insn per cycle         
                                           #    0.09  stalled cycles per insn  ( +-  0.00% )
 33,414,152      branches:u                # 1201.587 M/sec                    ( +-  0.00% )
    221,454      branch-misses:u           #    0.66% of all branches          ( +-  0.12% )
 
0.028201319 seconds time elapsed                                          ( +-  0.10% )

As can be seen, this patch increases the parsing time by about 0.4%. This is enough to be statistically significant in a benchmark like this, but probably not world-shattering (some slowdown is unavoidable with a change like this).

If we additionally enable the early returns we get an additional 1.5% slowdown (or 1.9% above baseline). Still not bad for a "micro-benchmark", but it does make one wonder, whether it is really worth it. My feeling would be that it isn't...

Tue, Jul 2, 3:32 PM · Restricted Project

Mon, Jul 1

dblaikie added a comment to D63672: Added Delta IR Reduction Tool.

Did the broader design discussion about bugpoint lead to this design (a separate llvm-reduce tool)? I didn't quite follow that thread far enough to see the conclusion that lead to this direction (rather than improving/modifying bugpoint) - might be worth writing a bit more in the commit description about the conversation (link to the thread if that's helpful) and design choices - and long-term plan (to merge behavior, share things, remove one or the other - or should these two tools exist indefinitely with different goals/uses?)

Mon, Jul 1, 4:03 PM · Restricted Project
dblaikie added inline comments to D63662: Changing CodeView Debug info Type record output.
Mon, Jul 1, 3:24 PM · Restricted Project
dblaikie added inline comments to D64006: [Support] Support 64-bit offsets in DataExtractor..
Mon, Jul 1, 1:56 PM · Restricted Project
dblaikie accepted D64020: [DWARF] Simplify dumping of a .debug_addr section..

Yeah, that looks incorrect/meaningless in this context - thanks for the catch!

Mon, Jul 1, 12:22 PM · Restricted Project

Fri, Jun 28

dblaikie added inline comments to D63662: Changing CodeView Debug info Type record output.
Fri, Jun 28, 3:14 PM · Restricted Project
dblaikie added a comment to D63905: Fix ASAN error caused by commit r364512.

Looks good to me, please commit

(@rovka - this is at least a short-term fix, I'm approving this to unbreak LLVM's mainline - feel free to refix with other ideas if you have any (also happy to discuss this further with you here, IRC, or elsewhere on the mailing lists)

Thanks for looking into it and sorry about the breakage. I think one InVReg is sufficient, there's no need for a whole SmallVector there. What do you think about something like this?

Fri, Jun 28, 8:08 AM · Restricted Project

Thu, Jun 27

dblaikie accepted D63905: Fix ASAN error caused by commit r364512.

Looks good to me, please commit

Thu, Jun 27, 4:29 PM · Restricted Project

Wed, Jun 26

dblaikie committed rG730a95c88aff: Fix some undefined behavior (excessive shift of signed value) in r364253… (authored by dblaikie).
Fix some undefined behavior (excessive shift of signed value) in r364253…
Wed, Jun 26, 12:20 PM
dblaikie committed rL364461: Fix some undefined behavior (excessive shift of signed value) in r364253….
Fix some undefined behavior (excessive shift of signed value) in r364253…
Wed, Jun 26, 12:18 PM
dblaikie accepted D63167: [Clang] Remove unused -split-dwarf and obsolete -enable-split-dwarf.

Sounds alright :)

Wed, Jun 26, 8:47 AM · Restricted Project, Restricted Project
dblaikie added a comment to D46628: [ELF] Add --strip-debug-non-line option.

I found that clang 9 (and previous versions) also generates non CU info in .debug_info with -gmlt:

$ echo 'struct A { A() {} int a = 0; }; struct B { B(); A a; }; B::B() {}' | clang++ -x c++ -O2 -gmlt -o clang-02-gmlt.o -c - && llvm-dwarfdump --debug-info clang-02-gmlt.o
clang-02-gmlt.o:	file format ELF64-x86-64

.debug_info contents:
0x00000000: Compile Unit: length = 0x00000051 version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x00000055)

0x0000000b: DW_TAG_compile_unit
              DW_AT_producer	("clang version 9.0.0 (https://github.com/llvm/llvm-project 01a99c0aa5ae5be47ea62bd6c87ca6bb63f5a454)")
              DW_AT_language	(DW_LANG_C_plus_plus)
              DW_AT_name	("-")
              DW_AT_stmt_list	(0x00000000)
              DW_AT_comp_dir	("/tmp/test")
              DW_AT_low_pc	(0x0000000000000000)
              DW_AT_high_pc	(0x0000000000000007)

0x0000002a:   DW_TAG_subprogram
                DW_AT_name	("A")

0x0000002f:   DW_TAG_subprogram
                DW_AT_low_pc	(0x0000000000000000)
                DW_AT_high_pc	(0x0000000000000007)
                DW_AT_name	("B")

0x00000040:     DW_TAG_inlined_subroutine
                  DW_AT_abstract_origin	(0x0000002a "A")
                  DW_AT_low_pc	(0x0000000000000000)
                  DW_AT_high_pc	(0x0000000000000006)
                  DW_AT_call_file	("/tmp/test/<stdin>")
                  DW_AT_call_line	(1)

0x00000053:     NULL

0x00000054:   NULL

Note that dropping with -O0 or -O1 .debug_info the DW_TAG_subprogram and DW_TAG_inlined_subroutine DIEs are not generated. -Os, -O2 and -O3 do.

$ echo 'struct A { A() {} int a = 0; }; struct B { B(); A a; }; B::B() {}' | clang++ -x c++ -O0 -gmlt -o clang-02-gmlt.o -c - && llvm-dwarfdump --debug-info clang-02-gmlt.o
clang-02-gmlt.o:	file format ELF64-x86-64

.debug_info contents:
0x00000000: Compile Unit: length = 0x00000026 version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x0000002a)

0x0000000b: DW_TAG_compile_unit
              DW_AT_producer	("clang version 9.0.0 (https://github.com/llvm/llvm-project 01a99c0aa5ae5be47ea62bd6c87ca6bb63f5a454)")
              DW_AT_language	(DW_LANG_C_plus_plus)
              DW_AT_name	("-")
              DW_AT_stmt_list	(0x00000000)
              DW_AT_comp_dir	("/tmp/test")
              DW_AT_low_pc	(0x0000000000000000)
              DW_AT_ranges	(0x00000000
                 [0x0000000000000000, 0x000000000000001b)
                 [0x0000000000000000, 0x0000000000000014))

The proposed --strip-debug-non-line is an improvement on top of what current gcc/clang produce with -g1 or -gmlt:

$ echo 'struct A { A() {} int a = 0; }; struct B { B(); A a; }; B::B() {}; int main(){ return 0;}' | clang++ -x c++ -O2 -gmlt -o clang-02-gmlt-strip -fuse-ld=lld -Wl,--strip-debug-non-line - && llvm-dwarfdump --debug-info clang-02-gmlt-strip
clang-02-gmlt-strip:	file format ELF64-x86-64

.debug_info contents:
0x00000000: Compile Unit: length = 0x00000026 version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x0000002a)

0x0000000b: DW_TAG_compile_unit
              DW_AT_producer	("clang version 9.0.0 (https://github.com/llvm/llvm-project 01a99c0aa5ae5be47ea62bd6c87ca6bb63f5a454)")
              DW_AT_language	(DW_LANG_C_plus_plus)
              DW_AT_name	("-")
              DW_AT_stmt_list	(0x00000000)
              DW_AT_comp_dir	("/tmp/test")
              DW_AT_low_pc	(0x0000000000201100)
              DW_AT_high_pc	(0x0000000000201113)

vs

$ echo 'struct A { A() {} int a = 0; }; struct B { B(); A a; }; B::B() {}; int main(){ return 0;}' | clang++ -x c++ -O2 -gmlt -o clang-02-gmlt -fuse-ld=lld  - && llvm-dwarfdump --debug-info clang-02-gmlt
clang-02-gmlt:	file format ELF64-x86-64

.debug_info contents:
0x00000000: Compile Unit: length = 0x00000051 version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08 (next unit at 0x00000055)

0x0000000b: DW_TAG_compile_unit
              DW_AT_producer	("clang version 9.0.0 (https://github.com/llvm/llvm-project 01a99c0aa5ae5be47ea62bd6c87ca6bb63f5a454)")
              DW_AT_language	(DW_LANG_C_plus_plus)
              DW_AT_name	("-")
              DW_AT_stmt_list	(0x00000000)
              DW_AT_comp_dir	("/tmp/test")
              DW_AT_low_pc	(0x0000000000201100)
              DW_AT_high_pc	(0x0000000000201113)

0x0000002a:   DW_TAG_subprogram
                DW_AT_name	("A")

0x0000002f:   DW_TAG_subprogram
                DW_AT_low_pc	(0x0000000000201100)
                DW_AT_high_pc	(0x0000000000201107)
                DW_AT_name	("B")

0x00000040:     DW_TAG_inlined_subroutine
                  DW_AT_abstract_origin	(0x0000002a "A")
                  DW_AT_low_pc	(0x0000000000201100)
                  DW_AT_high_pc	(0x0000000000201106)
                  DW_AT_call_file	("/tmp/test/<stdin>")
                  DW_AT_call_line	(1)

0x00000053:     NULL

0x00000054:   NULL

(I'll look into what triggers -gmlt + -O2 to generate the extra debug sections separately).

Wed, Jun 26, 8:43 AM · Restricted Project

Tue, Jun 25

dblaikie added inline comments to D63713: Add error handling to the DataExtractor class.
Tue, Jun 25, 4:31 PM · Restricted Project
dblaikie added a comment to D63537: [llvm-dwarfdump] --gdb-index: fix uninitialized TuListOffset.

@dblaikie save gdb-index generates index files of version 8. gold/lld generate version 7, llvm-dwarfdump recognize version 7 but not 8...

mkdir dir
gdb a -batch -q -ex 'save gdb-index dir'
objcopy --add-section .gdb_index=dir/a.gdb-index --set-section-flags .gdb_index=readonly a b
% myllvm-dwarfdump -gdb-index b
b:      file format ELF64-x86-64

.gdb_index contents:

<error parsing>
Tue, Jun 25, 11:57 AM · Restricted Project

Mon, Jun 24

dblaikie added a comment to D63632: Update the llvm::enumerate utility class result_pair to use the 'iterator_traits<R>::reference' instead of 'ValueOfIter<R> &'..

My understanding is that input_iterators do not have to return a reference, and may return a value type or some other proxy object.

(Sorry if there is a specific website that should be linked)
See specification of LegacyInputIterator: https://en.cppreference.com/w/cpp/named_req/InputIterator

Mon, Jun 24, 5:16 PM · Restricted Project
dblaikie committed rGf895e1bded03: DataExtractor: use decodeSLEB128 to implement getSLEB128 (authored by dblaikie).
DataExtractor: use decodeSLEB128 to implement getSLEB128
Mon, Jun 24, 4:46 PM
dblaikie committed rL364253: DataExtractor: use decodeSLEB128 to implement getSLEB128.
DataExtractor: use decodeSLEB128 to implement getSLEB128
Mon, Jun 24, 4:46 PM
dblaikie added a comment to D63632: Update the llvm::enumerate utility class result_pair to use the 'iterator_traits<R>::reference' instead of 'ValueOfIter<R> &'..

My understanding was/is that iterators can't return values instead of references (that such a thing doesn't meet the standard requirements for iterators) and that iterators basically have to have internal storage for things like this.

Mon, Jun 24, 3:17 PM · Restricted Project
dblaikie committed rG8242f35d5072: NFC: DataExtractor: use decodeULEB128 to implement getULEB128 (authored by dblaikie).
NFC: DataExtractor: use decodeULEB128 to implement getULEB128
Mon, Jun 24, 1:45 PM
dblaikie committed rL364230: NFC: DataExtractor: use decodeULEB128 to implement getULEB128.
NFC: DataExtractor: use decodeULEB128 to implement getULEB128
Mon, Jun 24, 1:43 PM
dblaikie added a comment to D63713: Add error handling to the DataExtractor class.

I'm kind of thinking a change to the cursor as Greg suggested might be better than wrapping the stream further - and probably using llvm::Error.

Mon, Jun 24, 11:27 AM · Restricted Project

Jun 21 2019

dblaikie added a comment to D63591: DWARFDebugLoc: Make parsing and error reporting more robust.

Given figuring out error handling for DataExtractor is perhaps a wider issue - if you want to go ahead with this change (continue with the review & defer error handling improvements for later, leave a FIXME, etc) that seems fine.

Jun 21 2019, 8:50 AM · Restricted Project
dblaikie accepted D63645: [Support] Fix error handling in DataExtractor::get[US]LEB128.

Awesome - thanks!

Jun 21 2019, 7:28 AM · Restricted Project

Jun 20 2019

dblaikie accepted D63573: Fix a crash with assembler source and -g..

Sounds alright to me

Jun 20 2019, 3:05 PM · Restricted Project, debug-info
dblaikie added inline comments to D63591: DWARFDebugLoc: Make parsing and error reporting more robust.
Jun 20 2019, 12:48 PM · Restricted Project

Jun 19 2019

dblaikie added inline comments to D63537: [llvm-dwarfdump] --gdb-index: fix uninitialized TuListOffset.
Jun 19 2019, 9:40 AM · Restricted Project

Jun 18 2019

dblaikie added a comment to D62635: Add enums as global variables in the IR metadata..
In D62635#1548721, @rnk wrote:
In D62635#1548157, @rnk wrote:

We did things this way to track which enumerators were used, not which enums were used. Size data showed it was worth doing (6%). The existing format doesn't support tracking usage of individual enumerators, so we pretended they were const integers to avoid changing the schema.

Ah - describing all the enumerators in any emitted enum would be too many bits/too much size in output?

Yes, that's what @akhuang tried and measured against to come up with the 6% number.

Jun 18 2019, 10:13 AM · Restricted Project, Restricted Project
dblaikie added a comment to D62635: Add enums as global variables in the IR metadata..
In D62635#1548157, @rnk wrote:

We did things this way to track which enumerators were used, not which enums were used. Size data showed it was worth doing (6%). The existing format doesn't support tracking usage of individual enumerators, so we pretended they were const integers to avoid changing the schema.

Jun 18 2019, 9:33 AM · Restricted Project, Restricted Project

Jun 17 2019

dblaikie added a comment to D62635: Add enums as global variables in the IR metadata..

I think the main issue was keeping track of which enums are used?

Jun 17 2019, 4:35 PM · Restricted Project, Restricted Project
dblaikie added a comment to D62635: Add enums as global variables in the IR metadata..

Is it practical/possible to do this in LLVM rather than in Clang? I'd have thought it'd be best to keep the IR metadata as output-format agnostic as practical/possible to reduce divergent codepaths, etc?

Jun 17 2019, 3:53 PM · Restricted Project, Restricted Project
dblaikie added a comment to D63031: DebugInfo: Render the canonical name of a class template specialization, even when nested in another class template specialization.

Ping (just curious about the change to the canonicalization printing policy - making sure I don't break something important)

Jun 17 2019, 2:39 PM · Restricted Project
dblaikie added a comment to D63377: [ORC] Avoid Race in Assertions .

Does the original bug show up with TSan on any existing/checked in tests, or could TSan catch it in some new test that isn't checked in yet/should be checked in?

Jun 17 2019, 2:33 PM · Restricted Project
dblaikie committed rG4f3b7364a459: PR42205: DebugInfio: Do not attempt to emit debug info metadata for static… (authored by dblaikie).
PR42205: DebugInfio: Do not attempt to emit debug info metadata for static…
Jun 17 2019, 12:39 PM
dblaikie committed rL363606: PR42205: DebugInfio: Do not attempt to emit debug info metadata for static….
PR42205: DebugInfio: Do not attempt to emit debug info metadata for static…
Jun 17 2019, 12:37 PM

Jun 13 2019

dblaikie committed rG4129e3e0f8e9: DebugInfo: Include enumerators in pubnames (authored by dblaikie).
DebugInfo: Include enumerators in pubnames
Jun 13 2019, 6:57 PM
dblaikie committed rL363349: DebugInfo: Include enumerators in pubnames.
DebugInfo: Include enumerators in pubnames
Jun 13 2019, 6:57 PM

Jun 12 2019

dblaikie accepted D62742: [OpaquePtr] BitcodeReader: don't rely on Types derived from a Value to provide pointer structure.

I don't fully have the direction you've in mind/that this is a part of in my head - but roughly enough to give a thumbs-up here :)

Jun 12 2019, 12:43 PM · Restricted Project

Jun 11 2019

dblaikie accepted D59673: [Clang] Harmonize Split DWARF options with llc.

Looks good!

Jun 11 2019, 5:57 PM · Restricted Project, Restricted Project
dblaikie added inline comments to D63167: [Clang] Remove unused -split-dwarf and obsolete -enable-split-dwarf.
Jun 11 2019, 5:57 PM · Restricted Project, Restricted Project
dblaikie accepted D63130: [Clang] Rename -split-dwarf-file to -split-dwarf-output.

Looks good - thanks!

Jun 11 2019, 5:54 PM · Restricted Project, Restricted Project

Jun 10 2019

dblaikie added a comment to D63013: [llvm-dwarfdump] Add -o to help text and remove --out-file from documentation.

dwarfdump(-classic) on macOS also supported the --out-file option and on macOS we strive for compatibility with the older tool.

Jun 10 2019, 5:20 PM · Restricted Project
dblaikie added a comment to D63013: [llvm-dwarfdump] Add -o to help text and remove --out-file from documentation.

I'm mildly against this change. IIRC, the idea was that --out-file is a long-form option that nobody uses in practice and we wanted people to prefer -o instead, which is what people are most familiar with working with compilers. So if -o shows up in the --help output, I'd leave it at that.

Could we just delete --out-file, if we don't want people to use it?

Not any more, I'm afraid :-)
Once an option is out there we will no doubt get complaints that we broke somebody's scripts if we remove it.

Jun 10 2019, 4:26 PM · Restricted Project

Jun 7 2019

dblaikie committed rG8472fa6c54c9: DebugInfo: Add support for 'nodebug' attribute on typedefs and alias templates (authored by dblaikie).
DebugInfo: Add support for 'nodebug' attribute on typedefs and alias templates
Jun 7 2019, 4:59 PM
dblaikie committed rL362856: DebugInfo: Add support for 'nodebug' attribute on typedefs and alias templates.
DebugInfo: Add support for 'nodebug' attribute on typedefs and alias templates
Jun 7 2019, 4:59 PM
dblaikie created D63031: DebugInfo: Render the canonical name of a class template specialization, even when nested in another class template specialization.
Jun 7 2019, 3:44 PM · Restricted Project
dblaikie accepted D63000: [ADT] Fix asan-detected stack-buffer-overflow in StringSetTest.cpp.

Sounds good - thanks

Jun 7 2019, 11:25 AM · Restricted Project

Jun 3 2019

dblaikie added a comment to D62634: Improve DWARF parsing and accessing by 1% to 2%.

This is intentional behavior in clang (controllable under the -f[no-]split-dwarf-inlining flag, and modified by the -f[no-]debug-info-for-profiling flag).

This extra debug info is used for online symbolication (in the absence of .dwo files) - such as for the sanitizers (accurate symbolication is necessary for correctness in msan, due to msan's necessary use of blacklisting to avoid certain false positives) and some forms of sample based profiling collection.

In the default mode, clang includes, as you noted, just the subprograms that have inlined subroutines in them in this split-dwarf-inlining data.
In -fdebug-info-for-profiling all subprograms are described in the skeleton CU with some minimal attributes (they don't need parameters, local variables/scopes, etc) necessary to do certain profile things I don't know lots about.

I think we have to tolerate the msan part. However, the profile feedback workflow could (should) surely be taught to read a .dwo file. The point of -fdebug-info-for-profiling was so you don't have to emit limited/full debug info in the .o file just to make profiling work; but if you're using split DWARF then you're doing limited/full anyway, and the feedback loop happens in the context of a build environment so the .dwo can be asserted to be available. So in split DWARF mode, we should not be emitting debug-info-for-profiling into the skeleton.

Jun 3 2019, 5:36 PM
dblaikie added a comment to D62369: [ADT] Enable set_difference() to be used on StringSet.

@dblaikie, did this get committed? I think @mmpozulp you may now have access so could commit if it hasn't happened yet.

Jun 3 2019, 11:21 AM · Restricted Project
dblaikie added a comment to D62634: Improve DWARF parsing and accessing by 1% to 2%.

Actually, there was a subtle change of behavior here. Before this change, if we tried to parse a DIE using an abbreviation table from a different file, we would most likely fail immediately because of the fail-safe in GetAbbreviationDeclarationPtr. Now, we just do a blind read and return bogus data.

Now, normally this shouldn't matter, but in case of split-dwarf, things start to get interesting. Lldb has this assumption that when we are reading a debug info entry (which is not the CU DIE), we are actually reading it from the dwo compile unit and not the skeleton unit in the main file. This means it uses the dwo abbreviation list and everything. Now, as far as I can tell, LLDB is kind of right here. The DWARF5 (in v4 split dwarf was a non-standard extension) spec says "A skeleton compilation unit has no children." (3.1.2 Skeleton Compilation Unit Entries). And indeed, gcc produces compilation units with no children. Clang, on the other hand, seems to be putting some DIEs into the main CU, if the compilation results in functions being inlined. And it seems clang has been doing this since at least 7.0 (I haven't tried older versions).

So it seems we have two problems here:

  1. We need to figure out whether there's a bug in clang and fix it. + @dblaikie for help with that
Jun 3 2019, 8:45 AM

May 29 2019

dblaikie added a comment to D62369: [ADT] Enable set_difference() to be used on StringSet.

Can you commit this yourself - or would you like me to do it for you?

May 29 2019, 2:42 PM · Restricted Project
dblaikie accepted D62319: IR: add 'byval(<ty>)' variant to 'byval' function parameters.

Seems good to me - thanks a bunch (:

May 29 2019, 11:22 AM · Restricted Project
dblaikie added inline comments to D62319: IR: add 'byval(<ty>)' variant to 'byval' function parameters.
May 29 2019, 10:13 AM · Restricted Project