This is an archive of the discontinued LLVM Phabricator instance.

TargetLibraryInfo checker tool
ClosedPublic

Authored by probinson on Oct 7 2021, 2:49 PM.

Details

Summary

A new tool that compares TargetLibraryInfo's opinion of the availability of library function calls against the functions actually exported by a specified set of libraries. Can be helpful in verifying the correctness of TLI for a given target, and avoid mishaps such as had to be addressed in D107509 and https://github.com/llvm/llvm-project/commit/94b4598d77fe0585a8a3bd2a798fc7ce15a6aa56

Diff Detail

Event Timeline

probinson created this revision.Oct 7 2021, 2:49 PM
probinson requested review of this revision.Oct 7 2021, 2:49 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 7 2021, 2:49 PM

+folks who participated in D107509

+ @jhenderson for object-file-reading expertise.

Hi Paul, thanks for submitting this, it looks super cool.

I've got a few comments on the syntax and options, but nothing too controversial.

The actual code looks super clean and easy to read, except for the SeparateMode part (to me, totally personal).

Perhaps it would be simpler if you always use an array of reports, and in separate mode, you populate the array one by one, but on joint mode you append to the first item, then report the array, however many items are there?

But none of that is really important, they're all just comments.

cheers,
--renato

llvm/docs/CommandGuide/llvm-tli-checker.rst
51

So, the syntax:

llvm-tli-checker ... --libdir=foo/bar a.o b.o c.o

will check against:

  • foo/bar/a.o
  • foo/bar/b.o
  • foo/bar/c.o

If so, couldn't this just be foo/bar/* and let the shell expand it?

66

This feels like more of a separate functionality, for example, --list, then just being verbose.

I usually interpret verbose as showing the steps taken, not showing more information.

Instead, if I just want to list the symbols (for ex. for grepping), then do I need to create an empty object file and use --verbose?

75

llvm-tli-checker?

Sorry for the long delay, between CPPcon and putting together a dev-meeting talk I've been pretty distracted.

The actual code looks super clean and easy to read, except for the SeparateMode part (to me, totally personal).

Perhaps it would be simpler if you always use an array of reports, and in separate mode, you populate the array one by one, but on joint mode you append to the first item, then report the array, however many items are there?

Yeah, that part is the bit I'm least happy about, it kind of just grew and probably could use a refactor now.
I invented separate mode so I could tell which library files had any libcalls in them at all, and reduce the fileset that I needed to look at. Maybe instead of separate mode, there could be a summary mode, that gave just the total number of matches per file. That would provide the functionality I actually needed.

llvm/docs/CommandGuide/llvm-tli-checker.rst
51

That could work. In my use case, the list of files to check is fixed, but I wanted to check against various releases, so it was easiest to put the list of files in a response file and then do --libdir=foo/bar @myliblist.txt for each release directory.
Also each foo/bar has many more files than just the libc/libm equivalents, and it's faster to look only at the specific files I want, so wildcarding would cost time and generate noise in the output.
I'd prefer to keep the option; I'll add a note about the response-file use case.

66

My inspiration was llvm-dwarfdump, where the default display shows you (for example) that an entity has a name attribute, and what that name is. In verbose mode, it also shows you details of the encoding, how it found the name. In that case, verbose does mean more information, so I did the same here.
Perhaps "print information" doesn't describe the behavior well, maybe "print results" is better? As it prints the results of all checks, not just the ones that don't match.
I could also see adding a --list option that didn't look at object files, but just dumped the info from TLI, would that be useful?

75

Oops

probinson updated this revision to Diff 384776.Nov 4 2021, 9:18 AM

Updated the UI:
Removed --verbose, replaced with --report=[summary,discrepancy,full]
Added --dump-tli to get info about TLI without reference to a library file

Reworked the reporting code, to make it more compact and hopefully easier to follow.

rengolin accepted this revision.Nov 4 2021, 9:40 AM

It's much clearer now, thanks for the updates!

This now looks good to me, but maybe wait a few days to make sure no one else has any further comments.

Thanks!

llvm/docs/CommandGuide/llvm-tli-checker.rst
51

I see, makes sense.

This revision is now accepted and ready to land.Nov 4 2021, 9:40 AM

Adding some Android folks to the subscriber list.

The motivating issue happened with Android (D107509). Maybe this test could be a useful way to ensure that LLVM doesn't assume a function is available that isn't actually in the NDK. To work, I think we'd want to test the LLVM toolchain with an NDK sysroot. For each arch-API combination, we'd want to run something like:

llvm-tli-checker --triple=${triple}${api} --libdir=${sysroot}/usr/lib/${triple}/${api} libc.so libm.so ../libc++_shared.so

I assume we need libc++_shared.so to provide the new/delete functions like LibFunc_Znwm.

The motivating issue happened with Android (D107509).

(Well, the Android bug was one of the motivating bugs. The other bug the author mentioned happened with the PS4, https://github.com/llvm/llvm-project/commit/94b4598d77fe0585a8a3bd2a798fc7ce15a6aa56)

This revision was landed with ongoing or failed builds.Nov 8 2021, 3:02 PM
This revision was automatically updated to reflect the committed changes.

What this tool does fit a bit into the realm of llvm-ifs (it had a previous name "llvm-elfabi" D100139. I like "elfabi" better as "ifs" seems confusing). Would it be suitable to move the code there?

llvm/test/tools/llvm-tli-checker/ps4-tli-check.s
8

The relanded version (38be8f4057c1bf19fd02d08d6116e28983a49d8d) uses prebuilt shared objects Inputs/ps4-tli-check.right.so and Inputs/ps4-tli-check.wrong.so. For llvm/tools/* tools working on object files, we try very hard to avoid checking in new prebuilt files: they are opaque, difficult to update, and waste space.

It's recommended to use yaml2obj, like many tests in test/tools/llvm-objcopy and test/tools/llvm-readobj.

@jhenderson

llvm/tools/llvm-tli-checker/llvm-tli-checker.cpp
61
70

static

106

What does ?? do? It's not regular external name prefix (_Z). If it is organization-specific, there should be a comment.

116

This can just use free functions instead of inheriting from std::vector.

122

static

155

SDK may be an organization-specific term. The more widely used term is library.

162

static

168

I know that the file has settled on capitalized messages, but non-capitalized message are more common:
https://llvm.org/docs/CodingStandards.html#error-and-warning-messages

171

More common to omit const for cast and change auto * to const auto *

173

range-based for

189

llvm::enumerate

221

It is more common to define a warning function and avoid referencing WithColor in the main program.

jhenderson added inline comments.Nov 22 2021, 1:16 AM
llvm/docs/CommandGuide/llvm-tli-checker.rst
75

Copy-paste error :)

llvm/test/tools/llvm-tli-checker/ps4-tli-check.s
8

+1 to what MaskRay said. If you need guidance on what the YAML might look like, feel free to chat with me outside this review.

You also can't use lld in a RUN command outside the lld and cross-project-tests projects, since there's no guarantee it would have been built (llvm/tools tests only have access to the core llvm project components).

Apologies for not noticing these points earlier: this slipped off my radar, and I ended up looking at other stuff.

probinson marked 5 inline comments as done.

What this tool does fit a bit into the realm of llvm-ifs (it had a previous name "llvm-elfabi" D100139. I like "elfabi" better as "ifs" seems confusing). Would it be suitable to move the code there?

I don't see any entry in https://llvm.org/docs/CommandGuide/index.html for llvm-ifs, and there is no description in llvm-ifs.cpp so it is hard to know what it does. I do see comments about "emitting stubs" which is not at all what this tool does, so I'm inclined to say No, combining them is not suitable.

See D114478 for code and test changes in response to this review.

llvm/test/tools/llvm-tli-checker/ps4-tli-check.s
8

I'll point out that the assembler source, with instructions for regenerating the prebuilt objects, is in the relanded patch, so there's no trouble to recreate them if necessary.

But I did get yaml2obj to do what I need, and the net file size is a small decrease, so I've replaced the test.

llvm/tools/llvm-tli-checker/llvm-tli-checker.cpp
61

Fixed comment

106

All names in the TLI set are external. _Z indicates an Itanium-style mangled name; ?? is the prefix for Windows mangling.
I've added a comment to that effect.

116

I think this is better OO design.

155

SDK is a common industry term. I didn't use "library" because the APIs may be implemented across several libraries within an SDK.

171

Okay.

173

I thought I had tried that and it didn't work, but it does now; changed.

189

I tried that, but the iterator doesn't have a reference member that llvm::enumerate appears to need.

221

This is not in the main program.