It doesn't have a section header string table so add a vector to have
the strings and create name based on the program header type and the
It doesn't have a section header string table so add a vector to have
|60,080 ms||x64 debian > AddressSanitizer-x86_64-linux.TestCases::scariness_score_test.cpp|
Script: -- : 'RUN: at line 4'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O0 /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/asan/TestCases/scariness_score_test.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Output/scariness_score_test.cpp.tmp
|80 ms||x64 debian > LLVM.Object::objdump-no-sectionheaders.test|
Script: -- : 'RUN: at line 4'; /var/lib/buildkite-agent/builds/llvm-project/build/bin/yaml2obj /var/lib/buildkite-agent/builds/llvm-project/llvm/test/Object/objdump-no-sectionheaders.test -o /var/lib/buildkite-agent/builds/llvm-project/build/test/Object/Output/objdump-no-sectionheaders.test.tmp
|50 ms||x64 debian > LLVM.tools/llvm-objdump/X86::disassemble-no-section.test|
Script: -- : 'RUN: at line 6'; /var/lib/buildkite-agent/builds/llvm-project/build/bin/yaml2obj /var/lib/buildkite-agent/builds/llvm-project/llvm/test/tools/llvm-objdump/X86/disassemble-no-section.test -o /var/lib/buildkite-agent/builds/llvm-project/build/test/tools/llvm-objdump/X86/Output/disassemble-no-section.test.tmp
|60,050 ms||x64 debian > MLIR.Examples/standalone::test.toy|
Script: -- : 'RUN: at line 1'; /usr/bin/cmake /var/lib/buildkite-agent/builds/llvm-project/mlir/examples/standalone -G "Ninja" -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_C_COMPILER=/usr/bin/clang -DLLVM_ENABLE_LIBCXX=OFF -DMLIR_DIR=/var/lib/buildkite-agent/builds/llvm-project/build/lib/cmake/mlir -DLLVM_USE_LINKER=lld
- Have you looked into using StringTableBuilder at all? I think you should be able to use that instead of hand-rolling your own code to achieve what looks to be the same thing.
- I'm not convinced by the naming scheme for the fake section names. Are you following any existing precedent for it? If not, I think we need it to be more obviously different to regular section names, as it doesn't seem unreasonable for people to have custom section names like "load0" for their sections. My immediate thought was something like PT_LOAD#0 for e.g. the PT_LOAD at index 0.
@MaskRay, any thoughts on the naming scheme?
Does GNU objdump use load0?
% objdump -h llvm/test/Object/Inputs/no-sections.elf-x86-64 llvm/test/Object/Inputs/no-sections.elf-x86-64: file format elf64-x86-64 Sections: Idx Name Size VMA LMA File off Algn
I think PT_LOAD#0 looks good to me, but I don't object to matching GNU objdump for the commonly used PT_LOAD.
There is a small issue that we have to invent names for PT_* values, for values that GNU objdump doesn't hard code.
This is probably an area whether 100% compatibility doesn't matter.
Use SmallString<0> FakeSectionStrings.
Prefer SmallString because std::string has a small string optimization which isn't useful for this case.
Not sure what happened with the input data. But at least it worked with my test case and linux perf kcore image.
$ bin/yaml2obj ../llvm/test/tools/llvm-objdump/X86/disassemble-no-section.test -o nosec.elf $ objdump -h nosec.elf nosec.elf: file format elf64-x86-64 Sections: Idx Name Size VMA LMA File off Algn 0 load0 0000000a ffffffff00000000 ffffffff00000000 00001000 2**0 CONTENTS, ALLOC, LOAD, READONLY, CODE $ objdump -d nosec.elf nosec.elf: file format elf64-x86-64 Disassembly of section load0: ffffffff00000000 <load0>: ffffffff00000000: 55 push %rbp ffffffff00000001: 48 89 e5 mov %rsp,%rbp ffffffff00000004: 0f 1f 40 00 nopl 0x0(%rax) ffffffff00000008: 5d pop %rbp ffffffff00000009: c3 ret
I'm not sure we need to worry about inventing names for other PT_* values at this time, because fake sections are only created for executable PT_LOAD, if I'm not mistaken. That being said, I agree that it's unlikely it matters whether this is 100% compatible.
I was just experimenting with an admittedly old version of GNU objdump on my machine and that just tells me that there are no sections in the file and doesn't do any disassembly. It's possible it's just because it's that old version (I don't have a more recent version available currently). If this is available in GNU objdump, could you post the version it was added to, please?
That said, I think there is benefit from this regardless of the parity with GNU.
This probably wants to be a size_t since it's an index.
Not sure you need the llvm:: prefixes?
I'd strongly support replacing the pre-canned ELF with a YAML input in this test. At the moment, I have no idea if the behaviour is correct without digging out a binary inspection tool to look at the ELF object.
Some key characteristics I think that need testing:
To achieve these points, I think it would be sufficient to have an ELF with the following program headers:
If this is available in GNU objdump, could you post the version it was added to, please?
I don't know when it's added, it's hard to find.. But my version is 2.38.
namhyung@youngsil:llvm-project$ objdump --version GNU objdump (GNU Binutils for Debian) 2.38 Copyright (C) 2022 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) any later version. This program has absolutely no warranty.
It seems not. Will remove.
I think StringTableBuilder just adds complexity/overhead in this case. StringTableBuilder cannot deduplicate any string here (and the code does not need to optimize for size).
// There is no section name string table. Return FakeSectionStrings which is non-empty if we have created fake sections.
Just return FakeSectionStrings; and delete return ""
The convention does not place parens around !=
I disagree that it adds complexity. It seems more intuitive to do StrTab.add(someString) than std::copy(someString.begin(), someString.end(), std::back_inserter(StrTab)); followed by a null terminator addition. I do agree that the optimizations are unnecessary. Maybe there's room for a "simple" string builder class that does none of the deduplication/optimization?
- remove unnecessary llvm:: prefix
- convert disassemble-no-section.test to use yaml2obj
Note that the test fails due to a mismatch on the first section name.
It should be PT_LOAD#1 but it shows PT_LOAD#3. I don't know why. :(
Having the "symmetry" has the potential to actually cause confusion, since the LHS isn't negated unlike the RHS.
I think I see a potential problem here, which could explain the problem you're seeing: you haven't specified any contents or explicit offsets of any of these program headers, which may mean that even though they have addresses specified, they'll start at the same offset within the file. This might result in something bad such as only the one section appearing in the output, although I don't know the llvm-objdump code well enough to be certain.
I misread this sentence. Here's a better improvement.
The Offset will physically impact the size of the file on disk, meaning this generated YAML file will be huge. Instead, use offset values around 0x100 - 0x200, which are significantly smaller (and adjust the addresses to line up too, I recommend) (you can't use 0 because of the ELF header and Program Header table). Also, the sizes of the individual blobs of data can be only a couple of bytes for the test purposes to be satisfied.
I suspect the StringTableBuilder does something wrong, or my understanding is incorrect. I checked with a debugger that the sh_name offset set correctly, but the first string was "PT_LOAD#3" instead of "PT_LOAD#1". Investigating...
Sorry for the delay. I think I found the reason - it passed a stack-allocate string to the string table builder which keeps the cached string ref. So each section name pointed to the same place in stack and got overwritten. Will add a vector to hold the local strings.
I'm pretty sure this isn't going to work as intended in many cases: you create a StringRef to a std::string temporary, and stick that in SecNames. It's likely that the StringRef will be pointing at invalid memory immediately after its construction (it's likely just working because nothing has scribbled over the data in memory yet). Could you not just populate SecNames with the result of the "PT_LOAD#" + std::to_string(Idx) expression directly?
If the StringTableBuilder contains StringRefs to the strings, using a std::vector will potentially invalidate them, if it needs to increase its capacity, right? We probably want a different strategy here. Probably simplest is simply to give the vector an initial capacity equal to the total number of program headers. Given that there are rarely that many, this shouldn't have any significant impact on performance.
Thanks for catching this. It is a code smell but it actually works: "PT_LOAD#" + std::to_string(Idx) returns a std::string. It works as the the lifetime of the temporary object std::string applies to the full-expression.
The following should be better:
SmallVector<std::string, 0> SecNames; ... SecNames.push_back(("PT_LOAD#" + Twine(Idx)).str()); // An alternative is ("PT_LOAD#" + Twine(Idx)).toVector(Name); but a local `SmallString` variable is needed
Let me try one more time before going back. I think this failure came from the std::string implementation. From a quick glance it seems to store small strings inside the object, not in the external allocation. Then a later move due to vector resize will invalidate earlier references. But SmallString seems to store the data always in the external allocation so it'd be fine after move. Actually it passes the test when I change it like below
- SmallVector<std::string, 0> SecNames; + SmallVector<SmallString<10>, 0> SecNames;
Let me know if I missed something.
I don't think you can rely on the behaviour of what happens to the strings when vectors are resized - I don't believe there's anything in the C++ standard or the existing type's definition that will guarantee the behaviour will remain the same. Also, your explanation sounds reversed to me, although I haven't looked at the implementation: I'd expecting SmallString to have a stack-allocated array for storing strings when the size is small, but std::string to always use the heap. In fact, on some implementations, I wouldn't be surprised if std::string also uses stack-allocation for short strings (I recall running into a problem very similar to yours along the same lines recently due to this sort of thing). Either way, it cannot be relied upon to keep your references valid.
I don't think so. The StringRef was needed because the constructor of SmallString takes it only. I wanted to pass std::string directly but it's not supported IIUC. After it creates a SmallString, I don't use the StringRef for a temp string anymore. Instead it uses the SmallString data in the vector, that's why I passed SecNames.back(). I believe it's safe to refer the data even after vector is resized.
LGTM, with @MaskRay's nits addressed.
Having come back to this a day later, I think it's wise to keep it, as we don't want to risk there being code that assumes sh_name == '\0' means no name.
Concretely: "Check llvm-objdump -h ..."