Page MenuHomePhabricator

[windows-itanium] Add Windows Itanium How-To Guide

Authored by bd1976llvm on Oct 15 2020, 5:33 PM.


Diff Detail

Event Timeline

bd1976llvm created this revision.Oct 15 2020, 5:33 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 15 2020, 5:33 PM
bd1976llvm requested review of this revision.Oct 15 2020, 5:33 PM
mstorsjo added inline comments.Oct 15 2020, 11:23 PM

Did you look into why it produces such comdats? Because for msvc triple targets, it should work with link.exe.


Actually - the code for patching it isn't emitted by either compiler or linker, it's done by the _pei386_runtime_relocator function in the mingw runtime. The linker just generates a list of fixups that the runtime later will need to process. If the runtime that processes this list isn't linked in, it won't work. Unfortunately, this can happen silently...

Have you checked this aspect, that it actually works as intended?

See for a patch (not recently rebased, unfortunately) to lld that makes it force a reference to this function, if runtime fixups actually are needed. That would make it clear if it's needed, but missing.

Finally, the _pei386_runtime_relocator function also needs to be called. Currently in the mingw-w64 runtime, it's always called unconditionally by other startup code, but with a change like, that object file would have a ctor that forces it to be called autonomously, if the object file is linked in. (Unfortunately, that change doesn't work with ld.bfd as it is right now, so it isn't upstreamed.)


Fwiw, since release 11, you can enable autoimporting alone without enabling the rest of the mingw specific quirks, with the -autoimport flag. Also see D89006 for a recent related fixup.


Typo, libc++abi


Why is that needed? If it's the same thing I've run into at some point, you can add -DLIBCXXABI_ENABLE_NEW_DELETE_DEFINITIONS=OFF to avoid the redundant symbols.

Hi @mstorsjo - I have been a bit short of time recently; but, I am going to look into everything that you have pointed out here. I should be able to make a reply by Monday.

Anyway, the point of this comment is just to say that I very much appreciate the helpful comments you have made - thanks!

mstorsjo added inline comments.Oct 30 2020, 2:28 AM

For my builds, as long as I'm using LIBUNWIND_USE_COMPILER_RT, it links clang-rt.builtins instead of libgcc, but I guess there's a need for more options there.


I hadn't noticed this wording here before, but this editing of cmake files to tweak visibility macros pretty much amounts to I guess? Hopefully that one (in whichever form I tweak it to) works for your case as well, to avoid any need for manual modifications.

bd1976llvm marked 7 inline comments as done.Nov 11 2020, 5:53 AM
bd1976llvm added inline comments.

I think I conflated several issues here, as I was trying to get the build set up, and got myself confused. Originally, I think I was somehow generating the "leaderless comdats" that you mentioned in another thread. I had thought that the leaderless comdats were related to multiply defined symbols being present. Happily, I no longer see the leaderless comdats and the remaining multiply defined symbol problems have now been solved and were unrelated. I can now link with MS's link.exe and I have adjusted the doc accordingly.

There is a remaining comdat issue that I see when building libc++ when not using the internal assembler which occurs if I try to use the ' -save-temps=obj' option for debugging. Errors are emitted such as:

  1. It seems that source -> assembly -> obj may report comdat
  2. errors (when assembling) that are not reported doing source-> obj.
  3. Example:
  4. src/CMakeFiles/cxx_shared.dir\locale.s:107958:2: error: section '.xdata$' is already linkonce
  5. .linkonce discard

I doubt this is related to Windows Itanium and I haven't looked into it more as it isn't currently blocking me.


It does not work as intended. Thanks for providing details here. I have adjusted the doc to add a more detailed explanation. As I understand it one of the goal of Windows Itanium is to use the Itanium C++ ABI on windows without relying on extra dependencies such as a custom runtime. Therefore, this must remain as a limitation for now. I spoke to @compnerd briefly about this problem and he though that the solution would be in the compiler emitting code that is called by the MS runtime at start-up and initialises these fields correct. I haven't yet investigated whether this is feasible or not.


Thanks. This is a nice improvement!


I'll look into upstreaming something for this. Currently, not adding these libraries is an easy edit to make tothe CMake files so I'm not worried about it.


Great - thanks for putting that up!


Thanks - it's not needed. There was a simple fix to the build that I had missed.

bd1976llvm marked 6 inline comments as done.

Updated the diff following helpful review comments.

mstorsjo added inline comments.Nov 11 2020, 2:07 PM

Ok, great to hear that those bits are sorted out.

Regarding roundtripping through assembly, it's possible that it ends up misgenerated for these cases, but it doesn't seem like I can reproduce it for a trivial case at least. For the gnu assembly syntax, GCC (which doesn't properly use all the aspects of COFF comdats) uses one syntax, while llvm has extended the syntax to support indicating the leader symbol.

A COFF comdat generated by GCC looks like this:

.section        .text$_Z6myfuncv,"x"
.linkonce discard

while generated by clang it looks like this:

.section        .text$_Z6myfuncv,"xr",discard,_Z6myfuncv

(For comdats without a leader symbol, LLVM also generates the GCC-like syntax.)

It sounds to me like there's some mixup between these two happening?

With a test file like this:

inline void myfunc(void) {}
void (*ptr)(void) = myfunc;

I get the expected syntax for a x86_64-windows-itanium target, but maybe it happens with cases that are a bit harder to trigger, like jump tables or something like that...


Oh, ok, that explains it.

Yeah it might be possible to make clang generate something like that, but IIRC I tried to do that at some point, and it ended up rather messy - see for an old attempt.

You can check how it works if you build this test file:

__declspec(dllimport) extern int dllVar;
int *myptr = &dllVar;

If you build this as C, it errors out (initializer element is not a compile-time constant), but if you build it as C++, it generates a constructor that initializes myptr at runtime. (And if you make it int *const myptr, the whole symbol vanishes!)


It seems nicer, yes, but it also gives the false impression that you can enable and use it without the corresponding mingw runtime, so that's not ideal either. I'm pondering if I should spend more effort on making it harder to fall into that trap.

bd1976llvm marked an inline comment as done.Nov 17 2020, 3:59 AM
bd1976llvm added inline comments.

Interestingly, If I try to use -auto-import in LLD it does diagnose the problem. I see errors like:

"lld-link: error: automatic dllimport of _ZTVN10cxxabiv117class_type_infoE in plugin-f2d5e0.o requires pseudo relocations"

This doesn't seem to happen with -lldmingw.

mstorsjo added inline comments.Nov 17 2020, 4:37 AM

Ah, that's right.

Yes, the feature is split up into two separate feature flags. -autoimport handles the aspect of looking for __imp_<symbol> if <symbol> is needed but missing. An autoimported variable may or may not need a pseudo relocation.

If the reference to the autoimported variable is in a pointer-sized section chunk, with the relocation being storing the absolute address of the variable, then lld can skip doing the pseudo relocation, and redirect all accesses from that section chunk towards the __imp_<symbol> symbol instead (which is a plain pointer that will contain the real address of the symbol at runtime). When compiling code for a mingw target, llvm generates indirection via such stubs (if you read code, COFFSTUB is used to refer to this), e.g. like this:

$ cat test.c
extern int maybeImported;
int getter(void) {
  return maybeImported;
$ clang -target x86_64-windows-gnu -S -o - test.c
        movq    .refptr.maybeImported(%rip), %rax
        movl    (%rax), %eax

        .section        .rdata$.refptr.maybeImported,"dr",discard,.refptr.maybeImported
        .globl  .refptr.maybeImported
        .quad   maybeImported

This is done for all accesses to global variables that aren't defined in the same translation unit. If the variable turns out to exist in the same DLL, it produces one pointer sized data block that contains a pointer to the function - if the variable turns out to need to be imported, the whole section ends up thrown away, and all accesses go to the __imp_<symbol> pointer instead, which behaves exactly like that one.

That works fine for cases where code accesses variables, but if you have a pointer in a structure next to other things, lld can't do that:

$ cat test2.c
extern int maybeImported;
struct {
    int foo;
    int *ptr;
    int bar;
} mystruct = {
$ clang -target x86_64-windows-gnu -S -o - test2.c
        .globl  mystruct
        .long   42
        .zero   4
        .quad   maybeImported
        .long   43
        .zero   4

So the cases with pointers in vtables, you really need psuedo relocations for that.

bd1976llvm marked an inline comment as done.Nov 18 2020, 5:12 AM
bd1976llvm added inline comments.

Thanks! It would be great to add a test-case to show this limitation with the current implementation. I suppose that anything that relies on a type comparison is broken. I will have a play around and see if I can come up with some testcases.

bd1976llvm added inline comments.Nov 25 2020, 9:08 AM

I gave this a go and couldn't make a testcase that fails. I think that this is because the type_info class only has one virtual method - the destructor - and the destructors of type_infos are not called (the standard does seem to permit the destructors not to be called). Therefore, the fact that the vtable references are not fixed up correct in type_info objects is not functionally important.

I think that the "correct" Windows Itanium fix for this is to adopt a change very much like your However, I wonder if, as a work-around, we couldn't implement a constructor (in the sense of some code that runs as program initialisation time) for Windows Itanium that patches the mingw "pseudo relocations". This might be preferable as I would like to keep the patchset for Windows Itanium as minimal as possible.

@compnerd @smeenai are you basically happy with this doc now?

Looks mostly ok to me now in general, just one minor note.


The doc still talks about generating invalid comdats that MS link.exe rejects - that's no longer the case right?

bd1976llvm updated this revision to Diff 310507.Dec 9 2020, 5:46 PM

Removed COMDAT limitation text.

bd1976llvm updated this revision to Diff 320652.Feb 1 2021, 6:06 PM
bd1976llvm marked an inline comment as done.

Added the cmake defines from for the libcxxabi build.

@mstorsjo are you happy with this now?


Correct. Thanks!

I think this one looks fine to me now - do @compnerd or @smeenai want to comment on it still?

smeenai accepted this revision.Mar 2 2021, 5:30 PM

Sorry for the delay here. This looks good; thank you!


Why this instead of directly using -target x86_64-unknown-windows-itanium?

This revision is now accepted and ready to land.Mar 2 2021, 5:30 PM
bd1976llvm marked an inline comment as done.Mar 8 2021, 3:52 PM

Sorry for the delay here. This looks good; thank you!

Thanks! I also have open - would be great if you could have a look at it.


There isn't a useable driver for Windows Itanium on Windows OS yet AFAIU. -target x86_64-unknown-windows-itanium uses the CrossWindowsToolChain driver - which is somewhat out of date. Currently, it makes sense (made my build script smaller) to use the windows-msvc driver - which knows how to locate the MS sdk components - and then force that driver to invoke the Windows Itanium code with "-Xclang -triple -Xclang x86_64-unknown-windows-itanium". If there is a better way please let me know :)

mstorsjo added inline comments.Mar 8 2021, 10:44 PM

--target=x86_64-windows-itanium should work; there's two syntaxes for that option in the regular clang driver, -target <triple> and --target=<triple>, the latter also works in clang-cl.