[XCOFF][AIX] Enable tooling support for 64 bit symbol table parsing

[jasonliu]

Summary

Add in the ability of parsing symbol table for 64 bit object.

[jasonliu]

Address clang-tidy comment and added binary file.

[jasonliu]

Ping.

[jhenderson]

llvm/lib/Object/XCOFFObjectFile.cpp

831

Is this error user-facing (I'm assuming so)? Assuming it is, you should record here which symbol is causing the problem. Otherwise the user will be faced with an error along the lines of this:

error: this csec symbol contains no auxiliary entry

which is not really actionable (imagine the input had 100000 symbols in - the user can't realistically go through each to find the offending one).

848

Similar to my above comment - which entry was not found? Give the user more context so that they can act on the problem.

llvm/tools/llvm-readobj/XCOFFDumper.cpp

416

This will write the error inline, rather than to stderr. Are you sure that's what you want? it isn't what most dumping tools do on failure.

[jasonliu]

Address comments.

[jhenderson]

llvm/lib/Object/XCOFFObjectFile.cpp

831

I believe this requires std::move(Error);, as you're returning an Expected, not an Error.

835–836

I think it would make more sense to insert the name in the middle of the message to make it a bit more concise. Something like:
"csect symbol name contains no auxiliary entry"

863

I'd suggest quoting somehow the symbol name, so that any whitespace or similar that happens to end up in the name (rare, but possible to write using assembly, at least for other platforms) is easily understood to be part of the name.

llvm/tools/llvm-readobj/XCOFFDumper.cpp

414–415

Use reportError or reportWarning so that the error is reported in a clean manner and consistent with other llvm-readobj varieties, and not report_fatal_error which looks like a crash. General rule of thumb: try to avoid using report_fatal_error, especially in tool code where it is easy to report errors properly.

In llvm-readobj for ELF, we try to avoid even using reportError where possible, as that stops the tool from continuing dumping, which can be problematic occasionally. We prefer reportWarning (or more specifically the local reportUniqueWarning which avoids reporting the same warning multiple times) and bailing out of the current routine. Take a look at ELFDumper.cpp for examples.

[jasonliu]

Address comments.

llvm/tools/llvm-readobj/XCOFFDumper.cpp

414–415

Thanks for the elaboration. That clears up things a lot for me. I will use reportUniqueWarning here.

[jhenderson]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

414–415

reportUniqueWarning can take an Error directly, so you can just do:

if (!ErrOrCsectAuxRef)
  reportUniqueWarning(ErrOrCsectAuxRef.takeError());

Also, be careful, as the program continues, so referencing ErrOrCsectAuxRef after this may result in things going wrong...

[jasonliu]

Address comments.

[jasonliu]

@jhenderson @DiggerLin @Esme
Any more comments?

[jhenderson]

In D85774#2738962, @jasonliu wrote:

@jhenderson @DiggerLin @Esme
Any more comments?

I'll try to take another look in the next day or two. The pre-merge bots are failing. Is that an issue with this patch?

[jasonliu]

In D85774#2738969, @jhenderson wrote:

In D85774#2738962, @jasonliu wrote:

@jhenderson @DiggerLin @Esme
Any more comments?

I'll try to take another look in the next day or two. The pre-merge bots are failing. Is that an issue with this patch?

Thanks a lot.
I think it's more of an issue that the binaries did not get into the phabricator probably. Not really an issue with the patch itself.

[jhenderson]

Only some minor nits, otherwise LGTM. I haven't attempted to review the test coverage for all the new code, as I don't feel like I'm in a good position to do that, not being an XCOFF developer. I assume someone else has though.

llvm/include/llvm/Object/XCOFFObjectFile.h

483

485

llvm/lib/Object/XCOFFObjectFile.cpp

797

808

Basically any time you use consumeError, add a comment explaining why it's justified that we don't report the error to the user.

[DiggerLin]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

390–391

this is only for the 32bits .
" By convention, the csect auxiliary entry in an XCOFF32 file must be the last auxiliary entry for any external symbol that has more than one auxiliary entry"

for 64bit, it maybe look for the x_auxtype ==AUX_CSECT

[DiggerLin]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

187

if (AuxEntPtr->AuxType != XCOFF::AUX_FILE ) , it should not be parsed as
XCOFF::AUX_FILE
it may better to print out raw data in the printSymbol()

236

if (AuxEntPtr->AuxType != XCOFF::AUX_CSECT) , it should not be parsed as
XCOFF::AUX_CSECT above
it may better to print out raw data in the printSymbol()

[jasonliu]

Address comments.

[jasonliu]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

187

I think it's the caller's responsibility to make sure they are passing in the right auxiliary type. So we should assume when we enter this function, we have the right auxiliary type here. So I modified it and made it an assert instead.

236

Same above. I modified it to be an assertion instead.

390–391

Good point. Updated the code.

[DiggerLin]

llvm/lib/Object/XCOFFObjectFile.cpp

94

SymbolAuxType is only for the 64 bits.
add assert(is64Bit() ) before return ?

llvm/tools/llvm-readobj/XCOFFDumper.cpp

378

as you mention "I think it's the caller's responsibility to make sure they are passing in the right auxiliary type. "

I think we need to check the AuxType == XCOFF::AUX_FILE for 64 bits. if not , print out the raw data as AUX_CSECT did ?

[DiggerLin]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

416

we have iterated auxiliary entries from line 382~396 and reiterated again in the printCsectAuxEnt(). I think we can improve on it.

And in 64bits, The auxiliary entries maybe be reordered in above implement.
it will print out all no AUX_CSECT auxiliary entries first and then AUX_CSECT entry, even if the AUX_CSECT is in the middle of the auxiliary entries
and if there are two AUX_CSECT on auxiliary entries, we only print out one.

[jasonliu]

llvm/tools/llvm-readobj/XCOFFDumper.cpp

416

I don't think we reiterated again in printCsectAuxEnt() for other auxiliary entries.
We did similar things for 382~396 in getXCOFFCsectAuxRef, but that function is specifically designed to only get XCOFFCsectAuxRef. So it's really not intended to pass information out there.
I don't really think the re-iteration should be a big concern because in theory 382~396 won't be executed all that much. If it does, I would rather have it implement properly (i.e. actually recognize the missing auxiliary entry) instead of just printing out raw datas.

[DiggerLin]

LGTM with address comment.

llvm/tools/llvm-readobj/XCOFFDumper.cpp

366

it need "continue;"
here

[DiggerLin]

and please run clang-format

[jasonliu]

Address comments.

[jhenderson]

I took a quick look at the pre-merge output. The message is that the object isn't recognised as a valid object file, not that it wasn't found, which suggests either the object or code is broken in some manner, or you're missing a dependent patch. Does this patch depend on another patch that isn't in main yet?

llvm/tools/llvm-readobj/XCOFFDumper.cpp

357–372

i -> I

(you're changing most of the loop body - you might as well fix this whilst you're here)

363

Test case?

392–394

My English language ping went off at the word "till" in these two sentences. I'd probably change it to "to". Also, use "first" rather than "1st", I suggest in both places.

Also "skips" -> "skip" for grammatical consistency.

396

i -> I

[jasonliu]

Address comments.

[jasonliu]

In D85774#2758956, @jhenderson wrote:

I took a quick look at the pre-merge output. The message is that the object isn't recognised as a valid object file, not that it wasn't found, which suggests either the object or code is broken in some manner, or you're missing a dependent patch. Does this patch depend on another patch that isn't in main yet?

No this patch does not depend on other patches. I think this is caused by git generated binary for the patch doesn't really assemble to the same one?

[jhenderson]

In D85774#2760205, @jasonliu wrote:

In D85774#2758956, @jhenderson wrote:

I took a quick look at the pre-merge output. The message is that the object isn't recognised as a valid object file, not that it wasn't found, which suggests either the object or code is broken in some manner, or you're missing a dependent patch. Does this patch depend on another patch that isn't in main yet?

No this patch does not depend on other patches. I think this is caused by git generated binary for the patch doesn't really assemble to the same one?

Sounds vaguely plausible, I guess.

llvm/test/tools/llvm-readobj/XCOFF/file-aux-wrong64.test

2–3

Fix a couple of grammar issues and a premature line-wrap.

5–6

20

The output on this line looks incorrect to me. Possibly a bug in the code resulting from a missing newe line?

[jasonliu]

Address comments.

[jasonliu]

llvm/test/tools/llvm-readobj/XCOFF/file-aux-wrong64.test

20

I added an newline after all the raw bytes. Other than that, I think the output is good. We have 18 bytes per symbol table entry, and we are printing 18 raw bytes here.

[jhenderson]

llvm/test/tools/llvm-readobj/XCOFF/file-aux-wrong64.test

20

I don't know if it is, but you probably want 00fb indented to line up nicely with the previous line. You can then confirm that this indentation is maintained by enabling --match-full-lines and --strict-whitespace in FileCheck. (If you do that, you'll need to remove the space after CHECK-NEXT:)

[jasonliu]

Adjustment the print out.

[jasonliu]

llvm/test/tools/llvm-readobj/XCOFF/file-aux-wrong64.test

20

Hi James, I adjusted the code to print 00fb in the same line, I think that actually works better because we could have multiply auxiliary entry data to print out. Putting each in the same line is easier to parse.

[jhenderson]

Looks good again. Sorry for the delay - I was off work last week.

[jasonliu]

In D85774#2801934, @jhenderson wrote:

Looks good again. Sorry for the delay - I was off work last week.

No worries. Thanks for all the reviews.