This is an archive of the discontinued LLVM Phabricator instance.

[JITLink][COFF][x86_64] Reimplement ADDR32NB/REL32.
ClosedPublic

Authored by sunho on Jul 16 2022, 11:22 AM.

Details

Summary

Reimplements ADDR32NB/REL32 relocations properly, out-of-reach targets will be dealt in the separate patch that will generate the stub for dllimport symbols.

Diff Detail

Event Timeline

sunho created this revision.Jul 16 2022, 11:22 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 16 2022, 11:22 AM
sunho requested review of this revision.Jul 16 2022, 11:22 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 16 2022, 11:22 AM

You said for both relocations that the value should not exceed 32bit. Is there a place for asserts to check/prove the fact?

I prefer enum class for new code.

sunho added a comment.Jul 16 2022, 1:50 PM

@tschuett It's checked in this patch using isInt<32> and also checked another time in x86_64.h when generic relocation edge is used. I also prefer to enum class but all existing code has been using enum. I also don't see an easy way to "extend" enum class. Is there a clever way? (for more context, if we were to swtich to enum class, we must have a way to extend enum as x86_64 edge kind was extended from generic jitlink edge kind in the first place)

That's a really creative solution for a tricky issue and it looks pretty good already! Kudos! Here's first set of questions and remarks.

llvm/lib/ExecutionEngine/JITLink/COFF_x86_64.cpp
214

Detail: We shouldn't use std::move here explicitly. Might prevent copy-elision.

221

Detail: I think we have to std::move(Err) here into the implicit ctor of Expected<JITTargetAddress>

227

This function appears to be the trickiest part of the patch. It's breaking out back into ORC to query the image base address. That's wild! And it might actually work!

The only thing that seems odd is that all COFF tests (and in general every user of the backend?) will need to define the absolute __ImageBase symbol now -- even though it's only used in two cases. Instead of calling it upfront here in the optimize function, could we call it on-demand in the specific cases?

I guess you could turn the variable here into a lambda, which would call getImageBaseAddress() only once and capture a stack variable to cache the result.. Does that make sense?

234

What else could it be?

248

Detail: I am a fan of writing out non-obvious types, but if not let's at least add the * please

sunho added inline comments.Jul 18 2022, 7:49 PM
llvm/lib/ExecutionEngine/JITLink/COFF_x86_64.cpp
234

I'm very uncertain about this part too. The tricky part is that makeTirple doesn't set COFF object format type for some reason. I tried chaning maketriple to set coff forat but it breaks several tests outside Orc. We can't just check with !isObjectFormatTypeELF() too as ELF is the default object format -- even for coff object format they are unchagned. I have to look into why maketriple doesn't set COFF object format.

sunho updated this revision to Diff 445731.Jul 19 2022, 1:47 AM

Address comments

sunho marked 4 inline comments as done.Jul 19 2022, 1:48 AM
sunho marked an inline comment as done.Jul 19 2022, 6:31 AM
sunho added inline comments.
llvm/lib/ExecutionEngine/JITLink/COFF_x86_64.cpp
234

Made it better by setting format inside coffgraphbuilder.

sunho updated this revision to Diff 446034.Jul 19 2022, 8:59 PM
sunho edited the summary of this revision. (Show Details)
sunho updated this revision to Diff 446038.Jul 19 2022, 9:35 PM

add neg addend

Nice, thanks for the updates. Are you planning to replace the COFF_external_func test or is it fully covered by the new COFF_addr32nb_reloc tests?

llvm/lib/ExecutionEngine/JITLink/COFFLinkGraphBuilder.cpp
24

Yes, I think it's a lot better to have it explicit like this. Can we still add a short note that summarizes our knowledge about missing COFF object format settings? Thanks!

327

How does your patch affect symbol sets so that they might be empty now?

llvm/test/ExecutionEngine/JITLink/X86/COFF_addr32nb_reloc_neg_addend.test
10

Can we name a good example here that involves negative addend?

sunho added a comment.Jul 20 2022, 6:01 AM

I'm planning to submit the patch that deals with dllimport stubs that cleanly solve the out of reach issue, since I think we agreed on requiring clients to manually set dllimport attribute for external (external to jitdylib) functions.

llvm/lib/ExecutionEngine/JITLink/COFFLinkGraphBuilder.cpp
327

It's actually the side effect of the last patch (not dead stripping asso

sunho added inline comments.Jul 20 2022, 6:05 AM
llvm/lib/ExecutionEngine/JITLink/COFFLinkGraphBuilder.cpp
327

The truncated comment I submitted is wrong comment. It's because of the change in this patch (ignoring IsDiscardable) This was needed to prevent a crash that happens in a hello world obj file. If you want me to, I can separate out that change as an another patch with a testcase.

sgraenitz accepted this revision.Jul 20 2022, 3:17 PM

Alright! Thanks, I didn't see that.

Form my side this looks all really good. I am sure you know best whether it's ready to land or if you want Lang to review as well.

llvm/lib/ExecutionEngine/JITLink/COFFLinkGraphBuilder.cpp
140

And then maybe:

// FIXME: Revisit crash when dropping IMAGE_SCN_MEM_DISCARDABLE sections
327

No worries, you don't need to split that out. However, keeping empty symbol sets around might be surprising for the reader (and in fact that might be solved at some point?). Do we always expect them to be IMAGE_SCN_MEM_DISCARDABLE? Then why not:

assert((cantFail(Obj.getSection(SecIndex))->Characteristics & 
        COFF::IMAGE_SCN_MEM_DISCARDABLE) != 0 && "Revisit after fixing crash")
This revision is now accepted and ready to land.Jul 20 2022, 3:17 PM
sunho updated this revision to Diff 446479.Jul 21 2022, 6:59 AM

Address comments & refactor

sunho updated this revision to Diff 446482.Jul 21 2022, 7:01 AM
sunho updated this revision to Diff 446485.Jul 21 2022, 7:04 AM
sunho updated this revision to Diff 446487.
sunho updated this revision to Diff 446494.Jul 21 2022, 7:23 AM
sunho updated this revision to Diff 447334.Jul 25 2022, 7:29 AM
This revision was landed with ongoing or failed builds.Jul 25 2022, 7:43 AM
This revision was automatically updated to reflect the committed changes.
llvm/test/ExecutionEngine/JITLink/X86/COFF_comdat_weak_duplicate.s