This is an archive of the discontinued LLVM Phabricator instance.

Differential D44810

[PDB] Make LLD PDBs look a little more like Microsoft PDBs
ClosedPublic

Authored by zturner on Mar 22 2018, 3:45 PM.

Details

Reviewers
rnk
Commits
rGa6fb536e5b5c: [PDB] Make our PDBs look more like MS PDBs.
rL328348: [PDB] Make our PDBs look more like MS PDBs.
rLLD328348: [PDB] Make our PDBs look more like MS PDBs.
Summary

This effort came about when trying to diagnose some problems with LLD PDBs that would cause certain debugging features to not work. It was fairly difficult to investigate why our PDB was behaving differently because there were a lot of differences in the bytes we emitted, most of which were meaningless.

For example, we would output stream A to stream 7 then stream B to stream 12, then construct the name stream map which says Stream A is at 7 and B is at 12.

They would output stream A to stream 15 and stream B to stream 3, then construct the name stream map which says Stream B is at 3 and stream A is at 15.

Since there is a hash table, and buckets, and values in the buckets, and the order of insertion matters, you could end up looking at 2 completely different sequences of bytes that were both ultimately correct.

Issues like this make looking for *important* byte-level differences almost impossible, and it wasn't until I spent considerable time eliminating these superfluous byte level differences that I was ultimately able to find the real problem.

So, the idea is that these changes should aid future investigations while being NFC. Note that none of these changes are necessary for correctness, they are just good to have anyway.

Diff Detail

Event Timeline

zturner created this revision.Mar 22 2018, 3:45 PM
Herald added subscribers: mgrang, hiraditya. · View Herald TranscriptMar 22 2018, 3:45 PM
rnk accepted this revision.Mar 22 2018, 3:53 PM

lgtm

llvm/lib/DebugInfo/PDB/Native/DbiStreamBuilder.cpp
66

Are you sure you don't want to check the optional value and do the old error path? Honestly, this is an internal error, we could assert.

This revision is now accepted and ready to land.Mar 22 2018, 3:53 PM
amccarth added a subscriber: amccarth.Mar 22 2018, 4:00 PM
amccarth added inline comments.
llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp
54

This is (mildly) terrifying given how common it has been for compilers to get rounding of floating point literals wrong.

http://www.exploringbinary.com/visual-c-plus-plus-strtod-still-broken/
http://www.exploringbinary.com/real-c-rounding-is-perfect-gcc-now-converts-correctly/
http://www.exploringbinary.com/gays-strtod-returns-zero-for-inputs-just-above-2-1075/
http://www.exploringbinary.com/how-gcc-converts-decimal-literals-to-floating-point/
etc.

majnemer added a subscriber: majnemer.Mar 22 2018, 7:03 PM
majnemer added inline comments.
llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp
41–47

This pseudo-code infinite loops. What's the termination condition?

zturner added inline comments.Mar 22 2018, 7:24 PM
llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp
41–47

The body of the loop is basically the "insert" function of the hash table, so it's trying to illustrate that on each insert, they increment the count, and if the load factor goes over 75% they multiply the number of buckets by 1.5 and rehash.

If there's a way to write the pseudocode that's more clear I'm happy to change it.

What about this:

// Initial conditions
int BucketCount = 1;
int StringCount = 0;

// Called each time a new string is inserted.
void onStringInserted() {
  ++StringCount;
  if (BucketCount * 3 / 4 < StringCount)
    BucketCount = BucketCount * 3 / 2 + 1;
}
majnemer added inline comments.Mar 22 2018, 8:05 PM
llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp
41–47

SGTM.

However, I think the sign has some interesting implications.

If StringCount was 49 (with BucketCount 1034394550), this code will overflow:
1034394550 * 3 is -1191783646 which will be less than StringCount (now 50).

Your version will end up computing the mathematically correct result of 1551591826.

48–49

a006999 53 is 3491081611
a006999 54 is 5236622417, this number is too big to fit in a uint32_t. This would end up computing (uint32_t)5236622417 which is 941655121.

zturner added inline comments.Mar 22 2018, 9:01 PM
llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp
41–47

I'll have to check with the MS source code tomorrow and see if theirs overflows. Perhaps they're using a uint64 and I missed that.

Closed by commit rLLD328348: [PDB] Make our PDBs look more like MS PDBs. (authored by zturner). · Explain WhyMar 23 2018, 11:46 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
lld/
COFF/
7 lines
test/
COFF/
6 lines
2 lines
4 lines
6 lines
llvm/
include/
llvm/
DebugInfo/
CodeView/
2 lines
PDB/
Native/
2 lines
lib/
DebugInfo/
CodeView/
9 lines
PDB/
Native/
38 lines
8 lines
31 lines
26 lines
tools/
llvm-pdbutil/
1 line
108 lines
2 lines
9 lines

Diff 139526

lld/COFF/PDB.cpp

Show First 20 Lines • Show All 79 Lines • ▼ Show 20 Linesstruct CVIndexMap {
SmallVector<TypeIndex, 0> IPIMap; SmallVector<TypeIndex, 0> IPIMap;
bool IsTypeServerMap = false; bool IsTypeServerMap = false;
}; };
class PDBLinker { class PDBLinker {
public: public:
PDBLinker(SymbolTable *Symtab) PDBLinker(SymbolTable *Symtab)
: Alloc(), Symtab(Symtab), Builder(Alloc), TypeTable(Alloc), : Alloc(), Symtab(Symtab), Builder(Alloc), TypeTable(Alloc),
IDTable(Alloc), GlobalTypeTable(Alloc), GlobalIDTable(Alloc) {} IDTable(Alloc), GlobalTypeTable(Alloc), GlobalIDTable(Alloc) {
// This isn't strictly necessary, but link.exe usually puts an empty string
// as the first "valid" string in the string table, so we do the same in
// order to maintain as much byte-for-byte compatibility as possible.
PDBStrTab.insert("");
}
/// Emit the basic PDB structure: initial streams, headers, etc. /// Emit the basic PDB structure: initial streams, headers, etc.
void initialize(const llvm::codeview::DebugInfo &BuildId); void initialize(const llvm::codeview::DebugInfo &BuildId);
/// Link CodeView from each object file in the symbol table into the PDB. /// Link CodeView from each object file in the symbol table into the PDB.
void addObjectsToPDB(); void addObjectsToPDB();
/// Link CodeView from a single object file into the PDB. /// Link CodeView from a single object file into the PDB.
▲ Show 20 Lines • Show All 1,034 Lines • Show Last 20 Lines

lld/test/COFF/pdb-diff.test

  • This file was deleted.
This test verifies that we produce PDBs compatible with MSVC in various ways.
We check in a cl-generated object file, PDB, and original source which serve
as the "baseline" for us to measure against. Then we link the same object
file with LLD and compare the two PDBs. Since the baseline object file and
PDB are already checked in, we just run LLD on the object file.
RUN: rm -f %T/pdb-diff-lld.pdb %T/pdb-diff-lld.exe
RUN: lld-link /debug /pdb:%T/pdb-diff-lld.pdb /out:%T/pdb-diff-lld.exe /nodefaultlib \
RUN: /entry:main %S/Inputs/pdb-diff.obj
RUN: llvm-pdbutil diff -result -values=false -left-bin-root=%S -right-bin-root=D:/src/llvm-mono/lld/test/COFF/ \
RUN: %T/pdb-diff-lld.pdb %S/Inputs/pdb-diff-cl.pdb | FileCheck %s
CHECK: ----------------------
CHECK-NEXT: | MSF Super Block |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: | File | |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: | Block Size | I |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: | Block Count |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: | Unknown 1 | I |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: | Directory Size |
CHECK-NEXT: |----------------+---|
CHECK-NEXT: ------------------------------------
CHECK-NEXT: | Stream Directory |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | File | |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Stream Count | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Old MSF Directory | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | PDB Stream | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | TPI Stream | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | DBI Stream | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | IPI Stream | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | New FPO Data | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Section Header Data | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Named Stream "/names" | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Named Stream "/LinkInfo" | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Module "Inputs\pdb-diff.obj" | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Module "* Linker *" | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | TPI Hash | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | IPI Hash | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Public Symbol Hash | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Global Symbol Hash | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Symbol Records | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: ------------------------------------
CHECK-NEXT: | String Table |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | File | |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Number of Strings | D |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Hash Version | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Byte Size |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Signature | I |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | Empty Strings |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | {{.*}}pdb-diff.cpp | {{[EI]}} |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | $T0 $ebp = $...p $T0 8 + = | D |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: | d:\src\llvm-...er internal) | D |
CHECK-NEXT: |------------------------------+---|
CHECK-NEXT: ----------------------------
CHECK-NEXT: | PDB Stream |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | File | |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Stream Size |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Age | I |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Guid | D |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Signature | D |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Version | I |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Features (set) | I |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Feature | I |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Named Stream Size |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | Named Streams (map) | {{[EI]}} |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | /names | {{[EI]}} |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: | /LinkInfo | {{[EI]}} |
CHECK-NEXT: |----------------------+---|
CHECK-NEXT: ----------------------------------------------
CHECK-NEXT: | DBI Stream |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | File | |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Dbi Version | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Age | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Machine | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Flags | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Build Major | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Build Minor | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Build Number | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | PDB DLL Version | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | PDB DLL RBLD | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (FPO) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (Exception) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (Fixup) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (OmapToSrc) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (OmapFromSrc) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (SectionHdr) | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (TokenRidMap) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (Xdata) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (Pdata) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (NewFPO) | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | DBG (SectionHdrOrig) | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Globals Stream | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Publics Stream | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Symbol Records | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Has CTypes | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Is Incrementally Linked | D |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Is Stripped | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Module Count | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Source File Count | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Module "Inputs\pdb-diff.obj" |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Modi | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Obj File Name | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Debug Stream | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - C11 Byte Size | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - C13 Byte Size | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - # of files | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Pdb File Path Index | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Source File Name Index | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Symbol Byte Size |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | Module "* Linker *" |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Modi | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Obj File Name | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Debug Stream | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - C11 Byte Size | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - C13 Byte Size | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - # of files | I |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Pdb File Path Index | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Source File Name Index | {{[EI]}} |
CHECK-NEXT: |----------------------------------------+---|
CHECK-NEXT: | - Symbol Byte Size |
CHECK-NEXT: |----------------------------------------+---|

lld/test/COFF/pdb-lib.s

# REQUIRES: x86 # REQUIRES: x86
# RUN: rm -rf %t && mkdir -p %t && cd %t # RUN: rm -rf %t && mkdir -p %t && cd %t
# RUN: llvm-mc -filetype=obj -triple=i686-windows-msvc %s -o foo.obj # RUN: llvm-mc -filetype=obj -triple=i686-windows-msvc %s -o foo.obj
# RUN: llc %S/Inputs/bar.ll -filetype=obj -mtriple=i686-windows-msvc -o bar.obj # RUN: llc %S/Inputs/bar.ll -filetype=obj -mtriple=i686-windows-msvc -o bar.obj
# RUN: llvm-lib bar.obj -out:bar.lib # RUN: llvm-lib bar.obj -out:bar.lib
# RUN: lld-link -debug -pdb:foo.pdb foo.obj bar.lib -out:foo.exe -entry:main # RUN: lld-link -debug -pdb:foo.pdb foo.obj bar.lib -out:foo.exe -entry:main
# RUN: llvm-pdbutil dump -modules %t/foo.pdb | FileCheck %s # RUN: llvm-pdbutil dump -modules %t/foo.pdb | FileCheck %s
# Make sure that the PDB has module descriptors. foo.obj and bar.lib should be # Make sure that the PDB has module descriptors. foo.obj and bar.lib should be
# absolute paths, and bar.obj should be the relative path passed to llvm-lib. # absolute paths, and bar.obj should be the relative path passed to llvm-lib.
# CHECK: Modules # CHECK: Modules
# CHECK-NEXT: ============================================================ # CHECK-NEXT: ============================================================
# CHECK-NEXT: Mod 0000 | `{{.*pdb-lib.s.tmp[/\\]foo.obj}}`: # CHECK-NEXT: Mod 0000 | `{{.*pdb-lib.s.tmp[/\\]foo.obj}}`:
# CHECK-NEXT: Obj: `{{.*pdb-lib.s.tmp[/\\]foo.obj}}`: # CHECK-NEXT: Obj: `{{.*pdb-lib.s.tmp[/\\]foo.obj}}`:
# CHECK-NEXT: debug stream: 9, # files: 0, has ec info: false # CHECK-NEXT: debug stream: 10, # files: 0, has ec info: false
# CHECK-NEXT: pdb file ni: 0 ``, src file ni: 0 `` # CHECK-NEXT: pdb file ni: 0 ``, src file ni: 0 ``
# CHECK-NEXT: Mod 0001 | `bar.obj`: # CHECK-NEXT: Mod 0001 | `bar.obj`:
# CHECK-NEXT: Obj: `{{.*pdb-lib.s.tmp[/\\]bar.lib}}`: # CHECK-NEXT: Obj: `{{.*pdb-lib.s.tmp[/\\]bar.lib}}`:
# CHECK-NEXT: debug stream: 10, # files: 0, has ec info: false # CHECK-NEXT: debug stream: 11, # files: 0, has ec info: false
# CHECK-NEXT: pdb file ni: 0 ``, src file ni: 0 `` # CHECK-NEXT: pdb file ni: 0 ``, src file ni: 0 ``
# CHECK-NEXT: Mod 0002 | `* Linker *`: # CHECK-NEXT: Mod 0002 | `* Linker *`:
# CHECK-NEXT: Obj: ``: # CHECK-NEXT: Obj: ``:
# CHECK-NEXT: debug stream: 11, # files: 0, has ec info: false # CHECK-NEXT: debug stream: 12, # files: 0, has ec info: false
# CHECK-NEXT: pdb file ni: 1 `{{.*foo.pdb}}`, src file ni: 0 `` # CHECK-NEXT: pdb file ni: 1 `{{.*foo.pdb}}`, src file ni: 0 ``
.def _main; .def _main;
.scl 2; .scl 2;
.type 32; .type 32;
.endef .endef
.globl _main .globl _main
_main: _main:
calll _bar calll _bar
xor %eax, %eax xor %eax, %eax
retl retl

lld/test/COFF/pdb-linker-module.test

RUN: lld-link /debug /pdb:%t.pdb /nodefaultlib /entry:main %S/Inputs/pdb-diff.obj RUN: lld-link /debug /pdb:%t.pdb /nodefaultlib /entry:main %S/Inputs/pdb-diff.obj
RUN: llvm-pdbutil dump -modules %t.pdb | FileCheck --check-prefix=MODS %s RUN: llvm-pdbutil dump -modules %t.pdb | FileCheck --check-prefix=MODS %s
RUN: llvm-pdbutil dump -symbols %t.pdb | FileCheck --check-prefix=SYMS %s RUN: llvm-pdbutil dump -symbols %t.pdb | FileCheck --check-prefix=SYMS %s
MODS: Mod 0001 | `* Linker *` MODS: Mod 0001 | `* Linker *`
MODS-NEXT: Obj: ``: MODS-NEXT: Obj: ``:
MODS-NEXT: debug stream: 10, # files: 0, has ec info: false MODS-NEXT: debug stream: 12, # files: 0, has ec info: false
MODS-NEXT: pdb file ni: 1 `{{.*}}pdb-linker-module.test.tmp.pdb`, src file ni: 0 `` MODS-NEXT: pdb file ni: 1 `{{.*}}pdb-linker-module.test.tmp.pdb`, src file ni: 0 ``
SYMS: Mod 0001 | `* Linker *` SYMS: Mod 0001 | `* Linker *`
SYMS-NEXT: 4 | S_OBJNAME [size = 20] sig=0, `* Linker *` SYMS-NEXT: 4 | S_OBJNAME [size = 20] sig=0, `* Linker *`
SYMS-NEXT: 24 | S_COMPILE3 [size = 40] SYMS-NEXT: 24 | S_COMPILE3 [size = 40]
SYMS-NEXT: machine = intel 80386, Ver = LLVM Linker, language = link SYMS-NEXT: machine = intel 80386, Ver = LLVM Linker, language = link
SYMS-NEXT: frontend = 0.0.0.0, backend = 14.10.25019.0 SYMS-NEXT: frontend = 0.0.0.0, backend = 14.10.25019.0
SYMS-NEXT: flags = none SYMS-NEXT: flags = none
Show All 9 Lines

lld/test/COFF/pdb-same-name.test

Show All 9 Lines
# RUN: %t1/foo.lib %t2/foo.lib # RUN: %t1/foo.lib %t2/foo.lib
# RUN: llvm-pdbutil dump -modules %t.pdb | FileCheck -check-prefix RAW %s # RUN: llvm-pdbutil dump -modules %t.pdb | FileCheck -check-prefix RAW %s
RAW: Modules RAW: Modules
RAW-NEXT: ============================================================ RAW-NEXT: ============================================================
RAW-NEXT: Mod 0000 | `foo.obj`: RAW-NEXT: Mod 0000 | `foo.obj`:
RAW-NEXT: Obj: `{{.*}}1\foo.lib`: RAW-NEXT: Obj: `{{.*}}1\foo.lib`:
RAW-NEXT: debug stream: 9, # files: 1, has ec info: false RAW-NEXT: debug stream: 11, # files: 1, has ec info: false
RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 `` RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 ``
RAW-NEXT: Mod 0001 | `foo.obj`: RAW-NEXT: Mod 0001 | `foo.obj`:
RAW-NEXT: Obj: `{{.*}}2\foo.lib`: RAW-NEXT: Obj: `{{.*}}2\foo.lib`:
RAW-NEXT: debug stream: 10, # files: 1, has ec info: false RAW-NEXT: debug stream: 12, # files: 1, has ec info: false
RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 `` RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 ``

lld/test/COFF/pdb.test

Show First 20 Lines • Show All 115 Lines • ▼ Show 20 Lines
# CHECK-NEXT: - Kind: LF_BUILDINFO # CHECK-NEXT: - Kind: LF_BUILDINFO
# CHECK-NEXT: BuildInfo: # CHECK-NEXT: BuildInfo:
# CHECK-NEXT: ArgIndices: [ 4098, 4099, 4106, 4104, 4102 ] # CHECK-NEXT: ArgIndices: [ 4098, 4099, 4106, 4104, 4102 ]
RAW: Modules RAW: Modules
RAW-NEXT: ============================================================ RAW-NEXT: ============================================================
RAW-NEXT: Mod 0000 | `{{.*}}pdb.test.tmp1.obj`: RAW-NEXT: Mod 0000 | `{{.*}}pdb.test.tmp1.obj`:
RAW-NEXT: Obj: `{{.*}}pdb.test.tmp1.obj`: RAW-NEXT: Obj: `{{.*}}pdb.test.tmp1.obj`:
RAW-NEXT: debug stream: 9, # files: 1, has ec info: false RAW-NEXT: debug stream: 11, # files: 1, has ec info: false
RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 `` RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 ``
RAW-NEXT: Mod 0001 | `{{.*}}pdb.test.tmp2.obj`: RAW-NEXT: Mod 0001 | `{{.*}}pdb.test.tmp2.obj`:
RAW-NEXT: Obj: `{{.*}}pdb.test.tmp2.obj`: RAW-NEXT: Obj: `{{.*}}pdb.test.tmp2.obj`:
RAW-NEXT: debug stream: 10, # files: 1, has ec info: false RAW-NEXT: debug stream: 12, # files: 1, has ec info: false
RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 `` RAW-NEXT: pdb file ni: 0 ``, src file ni: 0 ``
RAW-NEXT: Mod 0002 | `* Linker *`: RAW-NEXT: Mod 0002 | `* Linker *`:
RAW-NEXT: Obj: ``: RAW-NEXT: Obj: ``:
RAW-NEXT: debug stream: 11, # files: 0, has ec info: false RAW-NEXT: debug stream: 13, # files: 0, has ec info: false
RAW-NEXT: pdb file ni: 1 `{{.*pdb.test.tmp.pdb}}`, src file ni: 0 `` RAW-NEXT: pdb file ni: 1 `{{.*pdb.test.tmp.pdb}}`, src file ni: 0 ``
RAW: Types (TPI Stream) RAW: Types (TPI Stream)
RAW-NEXT: ============================================================ RAW-NEXT: ============================================================
RAW-NEXT: Showing 5 records RAW-NEXT: Showing 5 records
RAW-NEXT: 0x1000 | LF_ARGLIST [size = 8, hash = 0xEC0] RAW-NEXT: 0x1000 | LF_ARGLIST [size = 8, hash = 0xEC0]
RAW-NEXT: 0x1001 | LF_PROCEDURE [size = 16, hash = 0x7BC] RAW-NEXT: 0x1001 | LF_PROCEDURE [size = 16, hash = 0x7BC]
RAW-NEXT: return type = 0x0074 (int), # args = 0, param list = 0x1000 RAW-NEXT: return type = 0x0074 (int), # args = 0, param list = 0x1000
RAW-NEXT: calling conv = cdecl, options = None RAW-NEXT: calling conv = cdecl, options = None
▲ Show 20 Lines • Show All 148 Lines • Show Last 20 Lines

llvm/include/llvm/DebugInfo/CodeView/DebugStringTableSubsection.h

Show First 20 Lines • Show All 76 Lines • ▼ Show 20 Linespublic:
uint32_t size() const; uint32_t size() const;
StringMap<uint32_t>::const_iterator begin() const { StringMap<uint32_t>::const_iterator begin() const {
return StringToId.begin(); return StringToId.begin();
} }
StringMap<uint32_t>::const_iterator end() const { return StringToId.end(); } StringMap<uint32_t>::const_iterator end() const { return StringToId.end(); }
std::vector<uint32_t> sortedIds() const;
private: private:
DenseMap<uint32_t, StringRef> IdToString; DenseMap<uint32_t, StringRef> IdToString;
StringMap<uint32_t> StringToId; StringMap<uint32_t> StringToId;
uint32_t StringSize = 1; uint32_t StringSize = 1;
}; };
} // end namespace codeview } // end namespace codeview
} // end namespace llvm } // end namespace llvm
#endif // LLVM_DEBUGINFO_CODEVIEW_DEBUGSTRINGTABLESUBSECTION_H #endif // LLVM_DEBUGINFO_CODEVIEW_DEBUGSTRINGTABLESUBSECTION_H

llvm/include/llvm/DebugInfo/PDB/Native/DbiStreamBuilder.h

Show First 20 Lines • Show All 115 Lines • ▼ Show 20 Linesprivate:
StringMap<uint32_t> SourceFileNames; StringMap<uint32_t> SourceFileNames;
PDBStringTableBuilder ECNamesBuilder; PDBStringTableBuilder ECNamesBuilder;
WritableBinaryStreamRef NamesBuffer; WritableBinaryStreamRef NamesBuffer;
MutableBinaryByteStream FileInfoBuffer; MutableBinaryByteStream FileInfoBuffer;
std::vector<SectionContrib> SectionContribs; std::vector<SectionContrib> SectionContribs;
ArrayRef<SecMapEntry> SectionMap; ArrayRef<SecMapEntry> SectionMap;
llvm::SmallVector<DebugStream, (int)DbgHeaderType::Max> DbgStreams; std::array<Optional<DebugStream>, (int)DbgHeaderType::Max> DbgStreams;
}; };
} }
} }
#endif #endif

llvm/lib/DebugInfo/CodeView/DebugStringTableSubsection.cpp

Show First 20 Lines • Show All 80 Lines • ▼ Show 20 LinesError DebugStringTableSubsection::commit(BinaryStreamWriter &Writer) const {
Writer.setOffset(End); Writer.setOffset(End);
assert((End - Begin) == StringSize); assert((End - Begin) == StringSize);
return Error::success(); return Error::success();
} }
uint32_t DebugStringTableSubsection::size() const { return StringToId.size(); } uint32_t DebugStringTableSubsection::size() const { return StringToId.size(); }
std::vector<uint32_t> DebugStringTableSubsection::sortedIds() const {
std::vector<uint32_t> Result;
Result.reserve(IdToString.size());
for (const auto &Entry : IdToString)
Result.push_back(Entry.first);
std::sort(Result.begin(), Result.end());
return Result;
}
uint32_t DebugStringTableSubsection::getIdForString(StringRef S) const { uint32_t DebugStringTableSubsection::getIdForString(StringRef S) const {
auto Iter = StringToId.find(S); auto Iter = StringToId.find(S);
assert(Iter != StringToId.end()); assert(Iter != StringToId.end());
return Iter->second; return Iter->second;
} }
StringRef DebugStringTableSubsection::getStringForId(uint32_t Id) const { StringRef DebugStringTableSubsection::getStringForId(uint32_t Id) const {
auto Iter = IdToString.find(Id); auto Iter = IdToString.find(Id);
assert(Iter != IdToString.end()); assert(Iter != IdToString.end());
return Iter->second; return Iter->second;
} }

llvm/lib/DebugInfo/PDB/Native/DbiStreamBuilder.cpp

Show All 21 Lines
using namespace llvm; using namespace llvm;
using namespace llvm::codeview; using namespace llvm::codeview;
using namespace llvm::msf; using namespace llvm::msf;
using namespace llvm::pdb; using namespace llvm::pdb;
DbiStreamBuilder::DbiStreamBuilder(msf::MSFBuilder &Msf) DbiStreamBuilder::DbiStreamBuilder(msf::MSFBuilder &Msf)
: Msf(Msf), Allocator(Msf.getAllocator()), Age(1), BuildNumber(0), : Msf(Msf), Allocator(Msf.getAllocator()), Age(1), BuildNumber(0),
PdbDllVersion(0), PdbDllRbld(0), Flags(0), MachineType(PDB_Machine::x86), PdbDllVersion(0), PdbDllRbld(0), Flags(0), MachineType(PDB_Machine::x86),
Header(nullptr), DbgStreams((int)DbgHeaderType::Max) {} Header(nullptr) {}
DbiStreamBuilder::~DbiStreamBuilder() {} DbiStreamBuilder::~DbiStreamBuilder() {}
void DbiStreamBuilder::setVersionHeader(PdbRaw_DbiVer V) { VerHeader = V; } void DbiStreamBuilder::setVersionHeader(PdbRaw_DbiVer V) { VerHeader = V; }
void DbiStreamBuilder::setAge(uint32_t A) { Age = A; } void DbiStreamBuilder::setAge(uint32_t A) { Age = A; }
void DbiStreamBuilder::setBuildNumber(uint16_t B) { BuildNumber = B; } void DbiStreamBuilder::setBuildNumber(uint16_t B) { BuildNumber = B; }
Show All 19 Lines
} }
void DbiStreamBuilder::setPublicsStreamIndex(uint32_t Index) { void DbiStreamBuilder::setPublicsStreamIndex(uint32_t Index) {
PublicsStreamIndex = Index; PublicsStreamIndex = Index;
} }
Error DbiStreamBuilder::addDbgStream(pdb::DbgHeaderType Type, Error DbiStreamBuilder::addDbgStream(pdb::DbgHeaderType Type,
ArrayRef<uint8_t> Data) { ArrayRef<uint8_t> Data) {
if (DbgStreams[(int)Type].StreamNumber != kInvalidStreamIndex) DbgStreams[(int)Type].emplace();
rnkUnsubmitted
Not Done
ReplyInline Actions

Are you sure you don't want to check the optional value and do the old error path? Honestly, this is an internal error, we could assert.

rnk: Are you sure you don't want to check the optional value and do the old error path? Honestly…
return make_error<RawError>(raw_error_code::duplicate_entry, DbgStreams[(int)Type]->Data = Data;
"The specified stream type already exists");
auto ExpectedIndex = Msf.addStream(Data.size());
if (!ExpectedIndex)
return ExpectedIndex.takeError();
uint32_t Index = std::move(*ExpectedIndex);
DbgStreams[(int)Type].Data = Data;
DbgStreams[(int)Type].StreamNumber = Index;
return Error::success(); return Error::success();
} }
uint32_t DbiStreamBuilder::addECName(StringRef Name) { uint32_t DbiStreamBuilder::addECName(StringRef Name) {
return ECNamesBuilder.insert(Name); return ECNamesBuilder.insert(Name);
} }
uint32_t DbiStreamBuilder::calculateSerializedLength() const { uint32_t DbiStreamBuilder::calculateSerializedLength() const {
▲ Show 20 Lines • Show All 178 Lines • ▼ Show 20 LinesError DbiStreamBuilder::finalize() {
H->MFCTypeServerIndex = kInvalidStreamIndex; H->MFCTypeServerIndex = kInvalidStreamIndex;
H->GlobalSymbolStreamIndex = GlobalsStreamIndex; H->GlobalSymbolStreamIndex = GlobalsStreamIndex;
Header = H; Header = H;
return Error::success(); return Error::success();
} }
Error DbiStreamBuilder::finalizeMsfLayout() { Error DbiStreamBuilder::finalizeMsfLayout() {
for (auto &S : DbgStreams) {
if (!S.hasValue())
continue;
auto ExpectedIndex = Msf.addStream(S->Data.size());
if (!ExpectedIndex)
return ExpectedIndex.takeError();
S->StreamNumber = *ExpectedIndex;
}
for (auto &MI : ModiList) { for (auto &MI : ModiList) {
if (auto EC = MI->finalizeMsfLayout()) if (auto EC = MI->finalizeMsfLayout())
return EC; return EC;
} }
uint32_t Length = calculateSerializedLength(); uint32_t Length = calculateSerializedLength();
if (auto EC = Msf.setStreamSize(StreamDBI, Length)) if (auto EC = Msf.setStreamSize(StreamDBI, Length))
return EC; return EC;
▲ Show 20 Lines • Show All 93 Lines • ▼ Show 20 LinesError DbiStreamBuilder::commit(const msf::MSFLayout &Layout,
} }
if (auto EC = Writer.writeStreamRef(FileInfoBuffer)) if (auto EC = Writer.writeStreamRef(FileInfoBuffer))
return EC; return EC;
if (auto EC = ECNamesBuilder.commit(Writer)) if (auto EC = ECNamesBuilder.commit(Writer))
return EC; return EC;
for (auto &Stream : DbgStreams) for (auto &Stream : DbgStreams) {
if (auto EC = Writer.writeInteger(Stream.StreamNumber)) uint16_t StreamNumber = kInvalidStreamIndex;
if (Stream.hasValue())
StreamNumber = Stream->StreamNumber;
if (auto EC = Writer.writeInteger(StreamNumber))
return EC; return EC;
}
for (auto &Stream : DbgStreams) { for (auto &Stream : DbgStreams) {
if (Stream.StreamNumber == kInvalidStreamIndex) if (!Stream.hasValue())
continue; continue;
assert(Stream->StreamNumber != kInvalidStreamIndex);
auto WritableStream = WritableMappedBlockStream::createIndexedStream( auto WritableStream = WritableMappedBlockStream::createIndexedStream(
Layout, MsfBuffer, Stream.StreamNumber, Allocator); Layout, MsfBuffer, Stream->StreamNumber, Allocator);
BinaryStreamWriter DbgStreamWriter(*WritableStream); BinaryStreamWriter DbgStreamWriter(*WritableStream);
if (auto EC = DbgStreamWriter.writeArray(Stream.Data)) if (auto EC = DbgStreamWriter.writeArray(Stream->Data))
return EC; return EC;
} }
if (Writer.bytesRemaining() > 0) if (Writer.bytesRemaining() > 0)
return make_error<RawError>(raw_error_code::invalid_format, return make_error<RawError>(raw_error_code::invalid_format,
"Unexpected bytes found in DBI Stream"); "Unexpected bytes found in DBI Stream");
return Error::success(); return Error::success();
} }

llvm/lib/DebugInfo/PDB/Native/GSIStreamBuilder.cpp

Show First 20 Lines • Show All 144 Lines • ▼ Show 20 Lines
Error GSIStreamBuilder::finalizeMsfLayout() { Error GSIStreamBuilder::finalizeMsfLayout() {
// First we write public symbol records, then we write global symbol records. // First we write public symbol records, then we write global symbol records.
uint32_t PSHZero = 0; uint32_t PSHZero = 0;
uint32_t GSHZero = PSH->calculateRecordByteSize(); uint32_t GSHZero = PSH->calculateRecordByteSize();
PSH->finalizeBuckets(PSHZero); PSH->finalizeBuckets(PSHZero);
GSH->finalizeBuckets(GSHZero); GSH->finalizeBuckets(GSHZero);
Expected<uint32_t> Idx = Msf.addStream(calculatePublicsHashStreamSize()); Expected<uint32_t> Idx = Msf.addStream(calculateGlobalsHashStreamSize());
if (!Idx) if (!Idx)
return Idx.takeError(); return Idx.takeError();
PSH->StreamIndex = *Idx; GSH->StreamIndex = *Idx;
Idx = Msf.addStream(calculateGlobalsHashStreamSize()); Idx = Msf.addStream(calculatePublicsHashStreamSize());
if (!Idx) if (!Idx)
return Idx.takeError(); return Idx.takeError();
GSH->StreamIndex = *Idx; PSH->StreamIndex = *Idx;
uint32_t RecordBytes = uint32_t RecordBytes =
GSH->calculateRecordByteSize() + PSH->calculateRecordByteSize(); GSH->calculateRecordByteSize() + PSH->calculateRecordByteSize();
Idx = Msf.addStream(RecordBytes); Idx = Msf.addStream(RecordBytes);
if (!Idx) if (!Idx)
return Idx.takeError(); return Idx.takeError();
RecordStreamIdx = *Idx; RecordStreamIdx = *Idx;
▲ Show 20 Lines • Show All 154 Lines • Show Last 20 Lines

llvm/lib/DebugInfo/PDB/Native/PDBFileBuilder.cpp

Show First 20 Lines • Show All 104 Lines • ▼ Show 20 Lines if (Ipi && Ipi->getRecordCount() > 0) {
// record, we leave open the opportunity to test older PDBs such as those // record, we leave open the opportunity to test older PDBs such as those
// that don't have an ID stream. // that don't have an ID stream.
auto &Info = getInfoBuilder(); auto &Info = getInfoBuilder();
Info.addFeature(PdbRaw_FeatureSig::VC140); Info.addFeature(PdbRaw_FeatureSig::VC140);
} }
uint32_t StringsLen = Strings.calculateSerializedSize(); uint32_t StringsLen = Strings.calculateSerializedSize();
Expected<uint32_t> SN = allocateNamedStream("/names", StringsLen); Expected<uint32_t> SN = allocateNamedStream("/LinkInfo", 0);
if (!SN)
return SN.takeError();
SN = allocateNamedStream("/LinkInfo", 0);
if (!SN) if (!SN)
return SN.takeError(); return SN.takeError();
if (Dbi) { if (Gsi) {
if (auto EC = Dbi->finalizeMsfLayout()) if (auto EC = Gsi->finalizeMsfLayout())
return std::move(EC); return std::move(EC);
if (Dbi) {
Dbi->setPublicsStreamIndex(Gsi->getPublicsStreamIndex());
Dbi->setGlobalsStreamIndex(Gsi->getGlobalsStreamIndex());
Dbi->setSymbolRecordStreamIndex(Gsi->getRecordStreamIdx());
}
} }
if (Tpi) { if (Tpi) {
if (auto EC = Tpi->finalizeMsfLayout()) if (auto EC = Tpi->finalizeMsfLayout())
return std::move(EC); return std::move(EC);
} }
if (Ipi) { if (Dbi) {
if (auto EC = Ipi->finalizeMsfLayout()) if (auto EC = Dbi->finalizeMsfLayout())
return std::move(EC); return std::move(EC);
} }
if (Gsi) { SN = allocateNamedStream("/names", StringsLen);
if (auto EC = Gsi->finalizeMsfLayout()) if (!SN)
return SN.takeError();
if (Ipi) {
if (auto EC = Ipi->finalizeMsfLayout())
return std::move(EC); return std::move(EC);
if (Dbi) {
Dbi->setPublicsStreamIndex(Gsi->getPublicsStreamIndex());
Dbi->setGlobalsStreamIndex(Gsi->getGlobalsStreamIndex());
Dbi->setSymbolRecordStreamIndex(Gsi->getRecordStreamIdx());
}
} }
// Do this last, since it relies on the named stream map being complete, and // Do this last, since it relies on the named stream map being complete, and
// that can be updated by previous steps in the finalization. // that can be updated by previous steps in the finalization.
if (Info) { if (Info) {
if (auto EC = Info->finalizeMsfLayout()) if (auto EC = Info->finalizeMsfLayout())
return std::move(EC); return std::move(EC);
} }
▲ Show 20 Lines • Show All 141 Lines • Show Last 20 Lines

llvm/lib/DebugInfo/PDB/Native/PDBStringTableBuilder.cpp

Show All 9 Lines
#include "llvm/DebugInfo/PDB/Native/PDBStringTableBuilder.h" #include "llvm/DebugInfo/PDB/Native/PDBStringTableBuilder.h"
#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/ArrayRef.h"
#include "llvm/DebugInfo/PDB/Native/Hash.h" #include "llvm/DebugInfo/PDB/Native/Hash.h"
#include "llvm/DebugInfo/PDB/Native/RawTypes.h" #include "llvm/DebugInfo/PDB/Native/RawTypes.h"
#include "llvm/Support/BinaryStreamWriter.h" #include "llvm/Support/BinaryStreamWriter.h"
#include "llvm/Support/Endian.h" #include "llvm/Support/Endian.h"
#include <cmath>
using namespace llvm; using namespace llvm;
using namespace llvm::msf; using namespace llvm::msf;
using namespace llvm::support; using namespace llvm::support;
using namespace llvm::support::endian; using namespace llvm::support::endian;
using namespace llvm::pdb; using namespace llvm::pdb;
uint32_t PDBStringTableBuilder::insert(StringRef S) { uint32_t PDBStringTableBuilder::insert(StringRef S) {
return Strings.insert(S); return Strings.insert(S);
} }
uint32_t PDBStringTableBuilder::getIdForString(StringRef S) const { uint32_t PDBStringTableBuilder::getIdForString(StringRef S) const {
return Strings.getIdForString(S); return Strings.getIdForString(S);
} }
StringRef PDBStringTableBuilder::getStringForId(uint32_t Id) const { StringRef PDBStringTableBuilder::getStringForId(uint32_t Id) const {
return Strings.getStringForId(Id); return Strings.getStringForId(Id);
} }
static uint32_t computeBucketCount(uint32_t NumStrings) { static uint32_t computeBucketCount(uint32_t NumStrings) {
// The /names stream is basically an on-disk open-addressing hash table. // This computes a closed form of BucketCount for the following recursive
// Hash collisions are resolved by linear probing. We cannot make // algorithm which is present in the reference implementation:
// utilization 100% because it will make the linear probing extremely // int BucketCount = 1;
// slow. But lower utilization wastes disk space. As a reasonable // int StringCount = 0;
// load factor, we choose 80%. We need +1 because slot 0 is reserved. // while (true) {
return (NumStrings + 1) * 1.25; // ++StringCount;
// if (BucketCount * 3 / 4 < StringCount)
// BucketCount = BucketCount * 3 / 2 + 1;
// }
majnemerUnsubmitted
Not Done
ReplyInline Actions

This pseudo-code infinite loops. What's the termination condition?

majnemer: This pseudo-code infinite loops. What's the termination condition?
zturnerAuthorUnsubmitted
Not Done
ReplyInline Actions

The body of the loop is basically the "insert" function of the hash table, so it's trying to illustrate that on each insert, they increment the count, and if the load factor goes over 75% they multiply the number of buckets by 1.5 and rehash.

If there's a way to write the pseudocode that's more clear I'm happy to change it.

What about this:

// Initial conditions
int BucketCount = 1;
int StringCount = 0;

// Called each time a new string is inserted.
void onStringInserted() {
  ++StringCount;
  if (BucketCount * 3 / 4 < StringCount)
    BucketCount = BucketCount * 3 / 2 + 1;
}
zturner: The body of the loop is basically the "insert" function of the hash table, so it's trying to…
majnemerUnsubmitted
Not Done
ReplyInline Actions

SGTM.

However, I think the sign has some interesting implications.

If StringCount was 49 (with BucketCount 1034394550), this code will overflow:
1034394550 * 3 is -1191783646 which will be less than StringCount (now 50).

Your version will end up computing the mathematically correct result of 1551591826.

majnemer: SGTM. However, I think the sign has some interesting implications. If StringCount was 49…
zturnerAuthorUnsubmitted
Not Done
ReplyInline Actions

I'll have to check with the MS source code tomorrow and see if theirs overflows. Perhaps they're using a uint64 and I missed that.

zturner: I'll have to check with the MS source code tomorrow and see if theirs overflows. Perhaps…
// This was determined emperically to be identical to https://oeis.org/A006999
// which offers the closed form.
majnemerUnsubmitted
Not Done
ReplyInline Actions

a006999 53 is 3491081611
a006999 54 is 5236622417, this number is too big to fit in a uint32_t. This would end up computing (uint32_t)5236622417 which is 941655121.

majnemer: a006999 53 is 3491081611 a006999 54 is 5236622417, this number is too big to fit in a uint32_t.
//
// Matching the algorithm exactly is not strictly necessary for correctness,
// but it helps when comparing LLD's PDBs with Microsoft's PDBs so as to
// eliminate superfluous differences.
constexpr double C = 1.0815136685898448773;
amccarthUnsubmitted
Not Done
ReplyInline Actions

This is (mildly) terrifying given how common it has been for compilers to get rounding of floating point literals wrong.

http://www.exploringbinary.com/visual-c-plus-plus-strtod-still-broken/
http://www.exploringbinary.com/real-c-rounding-is-perfect-gcc-now-converts-correctly/
http://www.exploringbinary.com/gays-strtod-returns-zero-for-inputs-just-above-2-1075/
http://www.exploringbinary.com/how-gcc-converts-decimal-literals-to-floating-point/
etc.

amccarth: This is (mildly) terrifying given how common it has been for compilers to get rounding of…
return -1 + (uint32_t)std::floor(
C * std::pow((3.0 / 2.0), (double)(NumStrings + 2)));
} }
uint32_t PDBStringTableBuilder::calculateHashTableSize() const { uint32_t PDBStringTableBuilder::calculateHashTableSize() const {
uint32_t Size = sizeof(uint32_t); // Hash table begins with 4-byte size field. uint32_t Size = sizeof(uint32_t); // Hash table begins with 4-byte size field.
Size += sizeof(uint32_t) * computeBucketCount(Strings.size()); Size += sizeof(uint32_t) * computeBucketCount(Strings.size());
return Size; return Size;
} }
▲ Show 20 Lines • Show All 92 Lines • Show Last 20 Lines

llvm/tools/llvm-pdbutil/DumpOutputStyle.h

Show First 20 Lines • Show All 68 Lines • ▼ Show 20 Lines
private: private:
PDBFile &getPdb(); PDBFile &getPdb();
object::COFFObjectFile &getObj(); object::COFFObjectFile &getObj();
Error dumpFileSummary(); Error dumpFileSummary();
Error dumpStreamSummary(); Error dumpStreamSummary();
Error dumpSymbolStats(); Error dumpSymbolStats();
Error dumpUdtStats(); Error dumpUdtStats();
Error dumpNamedStreams();
Error dumpStringTable(); Error dumpStringTable();
Error dumpStringTableFromPdb(); Error dumpStringTableFromPdb();
Error dumpStringTableFromObj(); Error dumpStringTableFromObj();
Error dumpLines(); Error dumpLines();
Error dumpInlineeLines(); Error dumpInlineeLines();
Error dumpXmi(); Error dumpXmi();
Error dumpXme(); Error dumpXme();
Error dumpTpiStream(uint32_t StreamIdx); Error dumpTpiStream(uint32_t StreamIdx);
Show All 22 Lines

llvm/tools/llvm-pdbutil/DumpOutputStyle.cpp

Show First 20 Lines • Show All 84 Lines • ▼ Show 20 LinesError DumpOutputStyle::dump() {
} }
if (opts::dump::DumpUdtStats) { if (opts::dump::DumpUdtStats) {
if (auto EC = dumpUdtStats()) if (auto EC = dumpUdtStats())
return EC; return EC;
P.NewLine(); P.NewLine();
} }
if (opts::dump::DumpStringTable) { if (opts::dump::DumpNamedStreams) {
if (auto EC = dumpNamedStreams())
return EC;
P.NewLine();
}
if (opts::dump::DumpStringTable || opts::dump::DumpStringTableDetails) {
if (auto EC = dumpStringTable()) if (auto EC = dumpStringTable())
return EC; return EC;
P.NewLine(); P.NewLine();
} }
if (opts::dump::DumpModules) { if (opts::dump::DumpModules) {
if (auto EC = dumpModules()) if (auto EC = dumpModules())
return EC; return EC;
▲ Show 20 Lines • Show All 750 Lines • ▼ Show 20 LinesError DumpOutputStyle::dumpStringTableFromPdb() {
AutoIndent Indent(P); AutoIndent Indent(P);
auto IS = getPdb().getStringTable(); auto IS = getPdb().getStringTable();
if (!IS) { if (!IS) {
P.formatLine("Not present in file"); P.formatLine("Not present in file");
consumeError(IS.takeError()); consumeError(IS.takeError());
return Error::success(); return Error::success();
} }
if (IS->name_ids().empty()) { if (opts::dump::DumpStringTable) {
if (IS->name_ids().empty())
P.formatLine("Empty"); P.formatLine("Empty");
return Error::success(); else {
} auto MaxID =
std::max_element(IS->name_ids().begin(), IS->name_ids().end());
auto MaxID = std::max_element(IS->name_ids().begin(), IS->name_ids().end());
uint32_t Digits = NumDigits(*MaxID); uint32_t Digits = NumDigits(*MaxID);
P.formatLine("{0} | {1}", fmt_align("ID", AlignStyle::Right, Digits), P.formatLine("{0} | {1}", fmt_align("ID", AlignStyle::Right, Digits),
"String"); "String");
std::vector<uint32_t> SortedIDs(IS->name_ids().begin(), IS->name_ids().end()); std::vector<uint32_t> SortedIDs(IS->name_ids().begin(),
IS->name_ids().end());
std::sort(SortedIDs.begin(), SortedIDs.end()); std::sort(SortedIDs.begin(), SortedIDs.end());
for (uint32_t I : SortedIDs) { for (uint32_t I : SortedIDs) {
auto ES = IS->getStringForID(I); auto ES = IS->getStringForID(I);
llvm::SmallString<32> Str; llvm::SmallString<32> Str;
if (!ES) { if (!ES) {
consumeError(ES.takeError()); consumeError(ES.takeError());
Str = "Error reading string"; Str = "Error reading string";
} else if (!ES->empty()) { } else if (!ES->empty()) {
Str.append("'"); Str.append("'");
Str.append(*ES); Str.append(*ES);
Str.append("'"); Str.append("'");
} }
if (!Str.empty()) if (!Str.empty())
P.formatLine("{0} | {1}", fmt_align(I, AlignStyle::Right, Digits), Str); P.formatLine("{0} | {1}", fmt_align(I, AlignStyle::Right, Digits),
Str);
}
}
}
if (opts::dump::DumpStringTableDetails) {
P.NewLine();
{
P.printLine("String Table Header:");
AutoIndent Indent(P);
P.formatLine("Signature: {0}", IS->getSignature());
P.formatLine("Hash Version: {0}", IS->getHashVersion());
P.formatLine("Name Buffer Size: {0}", IS->getByteSize());
P.NewLine();
}
BinaryStreamRef NameBuffer = IS->getStringTable().getBuffer();
ArrayRef<uint8_t> Contents;
cantFail(NameBuffer.readBytes(0, NameBuffer.getLength(), Contents));
P.formatBinary("Name Buffer", Contents, 0);
P.NewLine();
{
P.printLine("Hash Table:");
AutoIndent Indent(P);
P.formatLine("Bucket Count: {0}", IS->name_ids().size());
for (const auto &Entry : enumerate(IS->name_ids()))
P.formatLine("Bucket[{0}] : {1}", Entry.index(),
uint32_t(Entry.value()));
P.formatLine("Name Count: {0}", IS->getNameCount());
}
} }
return Error::success(); return Error::success();
} }
Error DumpOutputStyle::dumpStringTableFromObj() { Error DumpOutputStyle::dumpStringTableFromObj() {
iterateModuleSubsections<DebugStringTableSubsectionRef>( iterateModuleSubsections<DebugStringTableSubsectionRef>(
File, PrintScope{P, 4}, File, PrintScope{P, 4},
[&](uint32_t Modi, const SymbolGroup &Strings, [&](uint32_t Modi, const SymbolGroup &Strings,
Show All 9 Lines iterateModuleSubsections<DebugStringTableSubsectionRef>(
P.formatLine("{0} | {1}", fmt_align(Offset, AlignStyle::Right, 4), P.formatLine("{0} | {1}", fmt_align(Offset, AlignStyle::Right, 4),
Str); Str);
} }
}); });
return Error::success(); return Error::success();
} }
Error DumpOutputStyle::dumpNamedStreams() {
printHeader(P, "Named Streams");
AutoIndent Indent(P, 2);
if (File.isObj()) {
P.formatLine("Dumping Named Streams is only supported for PDB files.");
return Error::success();
}
ExitOnError Err("Invalid PDB File: ");
auto &IS = Err(File.pdb().getPDBInfoStream());
const NamedStreamMap &NS = IS.getNamedStreams();
for (const auto &Entry : NS.entries()) {
P.printLine(Entry.getKey());
AutoIndent Indent2(P, 2);
P.formatLine("Index: {0}", Entry.getValue());
P.formatLine("Size in bytes: {0}",
File.pdb().getStreamByteSize(Entry.getValue()));
}
return Error::success();
}
Error DumpOutputStyle::dumpStringTable() { Error DumpOutputStyle::dumpStringTable() {
printHeader(P, "String Table"); printHeader(P, "String Table");
if (File.isPdb()) if (File.isPdb())
return dumpStringTableFromPdb(); return dumpStringTableFromPdb();
return dumpStringTableFromObj(); return dumpStringTableFromObj();
} }
▲ Show 20 Lines • Show All 647 Lines • Show Last 20 Lines

llvm/tools/llvm-pdbutil/llvm-pdbutil.h

Show First 20 Lines • Show All 136 Lines • ▼ Show 20 Lines
extern llvm::cl::opt<bool> DumpSymbolStats; extern llvm::cl::opt<bool> DumpSymbolStats;
extern llvm::cl::opt<bool> DumpUdtStats; extern llvm::cl::opt<bool> DumpUdtStats;
extern llvm::cl::opt<bool> DumpStreamBlocks; extern llvm::cl::opt<bool> DumpStreamBlocks;
extern llvm::cl::opt<bool> DumpLines; extern llvm::cl::opt<bool> DumpLines;
extern llvm::cl::opt<bool> DumpInlineeLines; extern llvm::cl::opt<bool> DumpInlineeLines;
extern llvm::cl::opt<bool> DumpXmi; extern llvm::cl::opt<bool> DumpXmi;
extern llvm::cl::opt<bool> DumpXme; extern llvm::cl::opt<bool> DumpXme;
extern llvm::cl::opt<bool> DumpNamedStreams;
extern llvm::cl::opt<bool> DumpStringTable; extern llvm::cl::opt<bool> DumpStringTable;
extern llvm::cl::opt<bool> DumpStringTableDetails;
extern llvm::cl::opt<bool> DumpTypes; extern llvm::cl::opt<bool> DumpTypes;
extern llvm::cl::opt<bool> DumpTypeData; extern llvm::cl::opt<bool> DumpTypeData;
extern llvm::cl::opt<bool> DumpTypeExtras; extern llvm::cl::opt<bool> DumpTypeExtras;
extern llvm::cl::list<uint32_t> DumpTypeIndex; extern llvm::cl::list<uint32_t> DumpTypeIndex;
extern llvm::cl::opt<bool> DumpTypeDependents; extern llvm::cl::opt<bool> DumpTypeDependents;
extern llvm::cl::opt<bool> DumpSectionHeaders; extern llvm::cl::opt<bool> DumpSectionHeaders;
extern llvm::cl::opt<bool> DumpIds; extern llvm::cl::opt<bool> DumpIds;
▲ Show 20 Lines • Show All 46 Lines • Show Last 20 Lines

llvm/tools/llvm-pdbutil/llvm-pdbutil.cpp

Show First 20 Lines • Show All 528 Lines • ▼ Show 20 Linescl::opt<uint32_t> DumpModi("modi", cl::Optional,
"modules, limit to the specified module"), "modules, limit to the specified module"),
cl::cat(FileOptions), cl::sub(DumpSubcommand)); cl::cat(FileOptions), cl::sub(DumpSubcommand));
cl::opt<bool> JustMyCode("jmc", cl::Optional, cl::opt<bool> JustMyCode("jmc", cl::Optional,
cl::desc("For all options that iterate over modules, " cl::desc("For all options that iterate over modules, "
"ignore modules from system libraries"), "ignore modules from system libraries"),
cl::cat(FileOptions), cl::sub(DumpSubcommand)); cl::cat(FileOptions), cl::sub(DumpSubcommand));
// MISCELLANEOUS OPTIONS // MISCELLANEOUS OPTIONS
cl::opt<bool> DumpNamedStreams("named-streams",
cl::desc("dump PDB named stream table"),
cl::cat(MiscOptions), cl::sub(DumpSubcommand));
cl::opt<bool> DumpStringTable("string-table", cl::desc("dump PDB String Table"), cl::opt<bool> DumpStringTable("string-table", cl::desc("dump PDB String Table"),
cl::cat(MiscOptions), cl::sub(DumpSubcommand)); cl::cat(MiscOptions), cl::sub(DumpSubcommand));
cl::opt<bool> DumpStringTableDetails("string-table-details",
cl::desc("dump PDB String Table Details"),
cl::cat(MiscOptions),
cl::sub(DumpSubcommand));
cl::opt<bool> DumpSectionContribs("section-contribs", cl::opt<bool> DumpSectionContribs("section-contribs",
cl::desc("dump section contributions"), cl::desc("dump section contributions"),
cl::cat(MiscOptions), cl::cat(MiscOptions),
cl::sub(DumpSubcommand)); cl::sub(DumpSubcommand));
cl::opt<bool> DumpSectionMap("section-map", cl::desc("dump section map"), cl::opt<bool> DumpSectionMap("section-map", cl::desc("dump section map"),
cl::cat(MiscOptions), cl::sub(DumpSubcommand)); cl::cat(MiscOptions), cl::sub(DumpSubcommand));
cl::opt<bool> DumpSectionHeaders("section-headers", cl::opt<bool> DumpSectionHeaders("section-headers",
▲ Show 20 Lines • Show All 647 Lines • ▼ Show 20 Lines if (opts::dump::RawAll) {
opts::dump::DumpModuleFiles = true; opts::dump::DumpModuleFiles = true;
opts::dump::DumpPublics = true; opts::dump::DumpPublics = true;
opts::dump::DumpSectionContribs = true; opts::dump::DumpSectionContribs = true;
opts::dump::DumpSectionHeaders = true; opts::dump::DumpSectionHeaders = true;
opts::dump::DumpSectionMap = true; opts::dump::DumpSectionMap = true;
opts::dump::DumpStreams = true; opts::dump::DumpStreams = true;
opts::dump::DumpStreamBlocks = true; opts::dump::DumpStreamBlocks = true;
opts::dump::DumpStringTable = true; opts::dump::DumpStringTable = true;
opts::dump::DumpStringTableDetails = true;
opts::dump::DumpSummary = true; opts::dump::DumpSummary = true;
opts::dump::DumpSymbols = true; opts::dump::DumpSymbols = true;
opts::dump::DumpSymbolStats = true; opts::dump::DumpSymbolStats = true;
opts::dump::DumpTypes = true; opts::dump::DumpTypes = true;
opts::dump::DumpTypeExtras = true; opts::dump::DumpTypeExtras = true;
opts::dump::DumpUdtStats = true; opts::dump::DumpUdtStats = true;
opts::dump::DumpXme = true; opts::dump::DumpXme = true;
opts::dump::DumpXmi = true; opts::dump::DumpXmi = true;
▲ Show 20 Lines • Show All 110 Lines • Show Last 20 Lines
lld/COFF/PDB.cpp