128-bit ABI for x86_64-w64-mingw32 incorrectly generated

[vtjnash]

Summary

Bug #16168

Summary: The calling convention of 128-bit integers generated for Windows x86-64 appears to be incorrectly using an approximation of the Linux ABI instead of the Windows ABI.

The llvm math intrinsics need to be updated to use call-by-reference for i128 parameters, and need to expect the return value in the XMM0 register, to be compatible with GCC and Microsoft on the Win64 platform.

[asl]

This is very invasive patch which definitely cannot be accepted as-is now, simply because it brings target-specific stuff inside the target-independent code.

Also, the patch has not tests at all, so it's hard to see, which problem the patch wants to solve (and thus it's not possible to suggest the alternative solution).

Start by adding the tests and we'll see how it will need to be reworked.

[majnemer]

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1872

Remove this commented out printfstatement.

1883

What is the point of this? It seems to always set isTailCall to false.

1895

No braces needed.

1945

Remove this commented out printfstatement.

2004

Remove this commented out printfstatement.

2026

No need for braces around a single statement.

2156

Remove this commented out printfstatement.

2184

No need for braces around a single statement.

lib/CodeGen/SelectionDAG/LegalizeTypes.cpp

1089

Remove this commented out printfstatement.

1112

No need for braces around a single statement.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

88

Remove this commented out printfstatement.

109

No need for braces around a single statement.

[vtjnash]

I am addressing the segfault in the following code snippet caused by the srem instruction using the wrong calling convention (GCC fixed this bug in 4.7). I have not been able to run the test suite on Windows (because my choices of python either don't support unix paths (native install), or don't support import threading (msys2 install)), but this is presumably already one of the tests.

#include <stdint.h>

int main() {
  volatile __int128 x = 1;
  return x % 3;
}

; ModuleID = 'i128.c'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-w64-mingw32"

define i32 @main() nounwind uwtable {
  %x = alloca i128, align 16
  store volatile i128 1, i128* %x, align 16
  %x.0.load1 = load volatile i128* %x, align 16
  %1 = srem i128 %x.0.load1, 3
  %2 = trunc i128 %1 to i32
  ret i32 %2
}

Note that there are 6 identical copies of the same patch here because there were 6 nearly identical functions in the code makeLibCall/ExpandLibCall/etc. . It seems all of these functions should be rewritten as 1 function. I mention that because I find it easier to refer to this as a change to a single function, while assuming the reader understands that that function exists in 6 locations.

@asl I agree that it is unfortunate that this is encoding target specific information in this file. Let me explain why I put it here: After this function is called, the function has been translated to the "llvm calling convention" and the information necessary to map that information to the calling convention of the target platform has been destroyed. However, before this function is called the information on what function is getting called and it's arguments is somewhat scattered. I would like to see the LLVM calling convention actually match the platform C ABI, so that it could be trivially used from other front-ends -- but I have seen mentioned that the LLVM calling convention is necessary for efficient optimization and won't be changed. Perhaps this information (ByRef vs. ByVal) could be part of RTLIB::Libcall, but I'm not sufficiently familiar with LLVM to make that modification.
To put this another way, makeLibCall is translating an LLVM opcode into a call to a C library function, and thus needs to know how to properly translate the C calling convention into LLVM.

@majnemer thanks for the feedback. I will be sure to incorporate those fixes along the way.

[loladiro]

Is there any possibility of this moving forward? This patch is crucial to get Julia to function properly on Windows.

[tkelman]

Seconded. How can we help improve the patch and get this problem fixed?

[asl]

Just start from simple things: address the review comments.

[yaron.keren]

I'm running the test suite on Windows 7 without problems, using standard Python 2.7.6 32 bit from python.org and running the tests from regular command line, not from MSYS.

Assuming llvm in c:\llvm was built RelWithDebInfo and Pyhton in c:\Python27, the commands to run all tests would be

cd c:\llvm
c:\Python27\python msvc/RelWithDebInfo/bin/llvm-lit.py test

To run just the CodeGen tests

cd c:\llvm
c:\Python27\python msvc/RelWithDebInfo/bin/llvm-lit.py test\CodeGen

Can you modify one of the tests (or create a new one) to show the problem?
possibly mul128.ll or i128-ret.ll would do.

[vtjnash]

thanks yaron.keren, it sounds like I need to rebuild using MSVC. when that finishes, I will add a test case here

@asl since the comments were mostly formatting, I was planning on addressing them when updating the patch for a more recent version of llvm. do you have a preference for what version (or svn checkout) that I retarget this for?

do you have some thoughts on where to locate this patch in the llvm source code? and how I might be able to modify it to address your previous comment that "it brings target-specific stuff inside the target-independent code"? I would like to also consolidate the various copies makeLibCall, but was initially trying to minimize the amount of code affected to help with merging

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1883

I was avoiding writing the memory location for the second i128 argument. I guess that's not actually necessary. I wasn't sure if this function could still be tail-called when it had arguments on the stack, so I disabled it to be safe.

[asl]

@vtjnash obviously, everything should be moved into X86 target. Any check for "win64" and similar stuff inside the common codegen is not acceptable at all. However, first you need to show that there is indeed a problem somewhere - I believe that the calling convention bits for 128 bit stuff is correct there.

[vtjnash]

I'm having trouble finding a button to update this diff. However, here's my new test, showing the code generated before/after the patch:

; RUN: llc < %s -mtriple=x86_64-linux | FileCheck %s -check-prefix=X86-64
; RUN: llc < %s -mtriple=x86_64-win32 | FileCheck %s -check-prefix=WIN64

define i128 @mod128(i128 %x) nounwind {
; X86-64: movl	$3, %edx
; X86-64: xorl	%ecx, %ecx
; X86-64: callq	__modti3
    
; WIN64-NOT: movl	$3, %r8d
; WIN64-NOT: xorl	%r9d, %r9d
; WIN64: leaq    -72(%rsp), %rsp
; WIN64: movq    %rdx, 56(%rsp)
; WIN64: leaq    32(%rsp), %rdx
; WIN64: movq    %rcx, 48(%rsp)
; WIN64: leaq    48(%rsp), %rcx
; WIN64: movq    $0, 40(%rsp)
; WIN64: movq    $3, 32(%rsp)
; WIN64: callq   __modti3
; WIN64: movd    %xmm0, %rax
; WIN64: punpckhqdq      %xmm0, %xmm0
; WIN64: movd    %xmm0, %rdx
; WIN64: leaq    72(%rsp), %rsp

  %1 = srem i128 %x, 3
  ret i128 %1
}

(this is based on my previous test code posted above, which segfaults without the patch and returns the correct answer with the patch)

[asl]

@vtjnash: I'm confused. Twice. In particular:

1. It looks like you're trying to match VCPP here, since you're using "-win32" target triple. However, you're calling __modti3 here, which is definitely not provided by VCPP...
2. Win64 ABI does not define the calling convention for __int128 (see http://msdn.microsoft.com/en-us/library/zthk2dkh.aspx). So, will you please indicate the document describing the desired behavior?

[vtjnash]

@vtjnash obviously, everything should be moved into X86 target. Any check for "win64" and similar stuff inside the common codegen is not acceptable at all. However, first you need to show that there is indeed a problem somewhere - I believe that the calling convention bits for 128 bit stuff is correct there.

I understand and agree. However, CodeGen here is implicitly assuming a calling convention for i128 types without acknowledging that it is doing target specific work -- in particular, it assumes that the library intrinsics functions (e.g. __modti3, provided by gcc-4.8) operate on registers, whereas WIN64 i128 uses a different calling convention (passed as pointers but returned in xmm0).

It looks like you're trying to match VCPP here, since you're using "-win32" target triple. However, you're calling __modti3 here, which is definitely not provided by VCPP...

True, I was running the tests on MinGW64 and OSX while I am waiting for the MSVC build to finish. LLVM is creating the call to __modti3, based on the llvm srem intrinsic.

Win64 ABI does not define the calling convention for __int128 (see http://msdn.microsoft.com/en-us/library/zthk2dkh.aspx). So, will you please indicate the document describing the desired behavior?

it does not explicitly call out i128, but it does state that anything over 8 bytes is unconditionally passed as a pointer. also relevant:
https://groups.google.com/forum/#!topic/llvm-dev/wZg9CnTevFI

[yaron.keren]

Jameson, to update the diff: click the Create diff button at the top right, paste the complete diff (not just the new test) and in the next screen instead of create new revision select atttach to revision D1998.

[vtjnash]

It looks like you're trying to match VCPP here, since you're using "-win32" target triple. However, you're calling __modti3 here, which is definitely not provided by VCPP...

It appears that MSVC-compiled LLVM generates identical code (including callq __modti3), although it moves one of the instructions, so my initial tests do not succeed, without reordering the assembly instructions in the test to match:

; WIN64-NOT: movl       $3, %r8d
; WIN64-NOT: xorl       %r9d, %r9d
; WIN64: leaq    -72(%rsp), %rsp
; WIN64: movq    %rdx, 56(%rsp)
; WIN64: movq    %rcx, 48(%rsp)
; WIN64: leaq    48(%rsp), %rcx
; WIN64: leaq    32(%rsp), %rdx # <= this instruction moved
; WIN64: movq    $0, 40(%rsp)
; WIN64: movq    $3, 32(%rsp)
; WIN64: callq   __modti3
; WIN64: movd    %xmm0, %rax
; WIN64: punpckhqdq      %xmm0, %xmm0
; WIN64: movd    %xmm0, %rdx
; WIN64: leaq    72(%rsp), %rsp

note that __modti3 can be optained as part of llvm (http://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/builtins/modti3.c) for compilers where it is not present, although my initial impression is that it would need to be ported to win32/pe-coff

[vtjnash]

thank you @yaron.keren

this updates the diff to address @majnemer's comments and add the test file

[vtjnash]

fix diff view to show all context (-U100000)

[vtjnash]

move logic to TargetLowering::LowerCallTo

this significantly simplifies the change, while moving it to a more sensible location

[asl]

@vtjnash: Again, this is still a layering violation. Move the logic to X86 target.

[asl]

test/CodeGen/X86/mod128.ll

11 ↗

(On Diff #8673)

This is incorrect. You need to make sure that the args are indeed passed where necessary, not only that the 'just' call is emitted.

[asl]

@vtjnash: I looked over the code and the proper solution would be:

1. You need to extend the calling convention bits to pass 128-bit stuff by value (see the existing code which already does this for vectors, i.e. __m128)
2. You need to make sure that libcalls are using the proper calling convention

This way no target-specific hacks in the target independent code would be required as well as much more clear implementation will be added.

[vtjnash]

Again, this is still a layering violation. Move the logic to X86 target.

I know, but hoped this would make it easier to identify the correct solution, by moving the logic closer to the invocation of the target-dependent code, X86TargetLowering::LowerCall

You need to extend the calling convention bits to pass 128-bit stuff by value (see the existing code which already does this for vectors, i.e. __m128)

Is this the patch you were expecting:

diff --git a/lib/Target/X86/X86CallingConv.td b/lib/Target/X86/X86CallingConv.td
index a78b5c0..7bc179d 100644
--- a/lib/Target/X86/X86CallingConv.td
+++ b/lib/Target/X86/X86CallingConv.td
@@ -137,6 +137,8 @@ def RetCC_X86_64_C : CallingConv<[
 def RetCC_X86_Win64_C : CallingConv<[
   // The X86-Win64 calling convention always returns __m64 values in RAX.
   CCIfType<[x86mmx], CCBitConvertToType<i64>>,
+  // The X86-Win64 calling convention always returns __m128i values in XMM.
+  CCIfType<[i128], CCAssignToReg<[XMM0,XMM1,XMM2,XMM3]>>,
 
   // Otherwise, everything is the same as 'normal' X86-64 C CC.
   CCDelegateTo<RetCC_X86_64_C>
@@ -324,7 +326,10 @@ def CC_X86_Win64_C : CallingConv<[
 
   // Long doubles get stack slots whose size and alignment depends on the
   // subtarget.
-  CCIfType<[f80], CCAssignToStack<0, 0>>
+  CCIfType<[f80], CCAssignToStack<0, 0>>,
+
+  // 128 bit numbers are passed by pointer
+  CCIfType<[i128], CCPassIndirect<i64>>
 ]>;
 
 def CC_X86_64_GHC : CallingConv<[

This was how I first attempted this patch. However, it does not work because line 6998 of the original file in the diff above ensures that the target-specific code will never see a type larger than a register. It seems like either more of the LowerCallTo code should be moved to the target dependent sections, or it should provide some hook to allow target-dependent register allocation

test/CodeGen/X86/mod128.ll

11 ↗

(On Diff #8673)

each version of llvm emits the arguments in a different order. looking at the documentation for FileCheck, it looks like WIN64-DAG may be the syntax I was looking for. I'll fix this in my next diff.

[asl]

Ideally you should custom lower the necessary libcalls. See e.g. in ARM backend how this is handled for SDIVREM nodes (ARMTargetLowering::LowerDivRem and around).

[vtjnash]

ok. i was not aware that could be done.

i think the following RTLIB functions would need this correction:
SHL_I128, SRL_I128, SRA_I128, MUL_I128, MULO_I128, SDIV_I128, UDIV_I128, SREM_I128, UREM_I128, SDIVREM_I128, UDIVREM_I128,

of these, I was able to generate calls to the following for MVT::i128
ISD::SDIV, ISD::UDIV, ISD::SREM, ISD::UREM
(I couldn't cause ISD::SDIVREM or ISD::UDIVREM, but I assumed they are possible)

[vtjnash]

move calling convention patch to X86 target

[vtjnash]

cygwin-64 has the same calling convention as win64

[loladiro]

Is this patch acceptable now?

[rnk]

lib/Target/X86/X86ISelLowering.cpp

1504 ↗

(On Diff #8678)

Please use two space indentation here.

12528–12534 ↗

(On Diff #8678)

I'd rather hoist the check for win64 + [if]128 out of the function that lowers them and turn these into assertions.

It might be better to strengthen these to assertions, if the result of failing to replace something is a miscompile.

12571 ↗

(On Diff #8678)

This is copy-pasted from other examples, but I'd whack it with clang-format if you have it easily available. Otherwise, it's fine.

13616 ↗

(On Diff #8678)

Indentation. Alternatively, move the check for win64 + [fi]128 over the lowering call and drop this check.

[vtjnash]

reformat per comments

[vtjnash]

@rnk I have updated the diff based on your comments (although I did not manage to locate a copy of clang-format). As suggested, I strengthened all of the conditional checks to assertions. The original code to retest the conditional was copied from the ARM code example and was not needed here.

[majnemer]

lib/Target/X86/X86ISelLowering.cpp

12551 ↗

(On Diff #9090)

Please keep columns under 80 characters.

[rnk]

Thanks, that looks good. I committed this with minor changes in r208029.

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

 }
 
 // ExpandLibCall - Expand a node into a call to a libcall.  If the result value
 // does not fit into a register, return the lo part and set the hi part to the
 // by-reg argument.  If it does fit into a single register, return the result
 // and leave the Hi part unset.
 SDValue SelectionDAGLegalize::ExpandLibCall(RTLIB::Libcall LC, SDNode *Node,
                                             bool isSigned) {
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+
+  // By default, the input chain to this libcall is the entry node of the
+  // function. If the libcall is going to be emitted as a tail call then
+  // TLI.isUsedByReturnOnly will change it to the right chain if the return
+  // node which is being folded has a non-entry input chain.
+  SDValue InChain = DAG.getEntryNode();
+  bool isTailCall = true;
+  DebugLoc dl = Node->getDebugLoc();
+
   TargetLowering::ArgListTy Args;
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 0, e = Node->getNumOperands(); i != e; ++i) {
     EVT ArgVT = Node->getOperand(i).getValueType();
     Type *ArgTy = ArgVT.getTypeForEVT(*DAG.getContext());
     Entry.Node = Node->getOperand(i); Entry.Ty = ArgTy;
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16); 
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16); 
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+      if (isTailCall) isTailCall = false;
+    }
     Args.push_back(Entry);
   }
   SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
                                          TLI.getPointerTy());
 
-  Type *RetTy = Node->getValueType(0).getTypeForEVT(*DAG.getContext());
-
-  // By default, the input chain to this libcall is the entry node of the
-  // function. If the libcall is going to be emitted as a tail call then
-  // TLI.isUsedByReturnOnly will change it to the right chain if the return
-  // node which is being folded has a non-entry input chain.
-  SDValue InChain = DAG.getEntryNode();
+  EVT RetVT = Node->getValueType(0);
+  Type *RetTy;
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+      isTailCall = false; // necessary because of the calling convention mismatch between LLVM and native Win64
+  } else {
+      RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  }
 
+  if (isTailCall) {
     // isTailCall may be true since the callee does not reference caller stack
     // frame. Check if it's in the right position.
     SDValue TCChain = InChain;
-  bool isTailCall = TLI.isInTailCallPosition(DAG, Node, TCChain);
+    isTailCall = TLI.isInTailCallPosition(DAG, Node, TCChain);
     if (isTailCall)
       InChain = TCChain;
+  }
 
   TargetLowering::
   CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false, false,
                     0, TLI.getLibcallCallingConv(LC), isTailCall,
                     /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
-                    Callee, Args, DAG, Node->getDebugLoc());
+                    Callee, Args, DAG, dl);
   std::pair<SDValue, SDValue> CallInfo = TLI.LowerCallTo(CLI);
 
 
   if (!CallInfo.second.getNode())
     // It's a tailcall, return the chain (which is the DAG root).
     return DAG.getRoot();
 
-  return CallInfo.first;
+  SDValue ret = CallInfo.first;
+  if (isa<VectorType>(RetTy))
+      ret = DAG.getNode(ISD::BITCAST, dl, RetVT, ret);
+  return ret;
 }
 
 /// ExpandLibCall - Generate a libcall taking the given operands as arguments
 /// and returning a result of type RetVT.
 SDValue SelectionDAGLegalize::ExpandLibCall(RTLIB::Libcall LC, EVT RetVT,
                                             const SDValue *Ops, unsigned NumOps,
                                             bool isSigned, DebugLoc dl) {
   TargetLowering::ArgListTy Args;
   Args.reserve(NumOps);
 
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+
+  SDValue InChain = DAG.getEntryNode();
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 0; i != NumOps; ++i) {
     Entry.Node = Ops[i];
     Entry.Ty = Entry.Node.getValueType().getTypeForEVT(*DAG.getContext());
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16); 
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16); 
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+    }
     Args.push_back(Entry);
   }
   SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
                                          TLI.getPointerTy());
 
-  Type *RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  Type *RetTy;
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+  } else {
+      RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  }
   TargetLowering::
-  CallLoweringInfo CLI(DAG.getEntryNode(), RetTy, isSigned, !isSigned, false,
+  CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false,
                        false, 0, TLI.getLibcallCallingConv(LC),
                        /*isTailCall=*/false,
                   /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
                   Callee, Args, DAG, dl);
   std::pair<SDValue,SDValue> CallInfo = TLI.LowerCallTo(CLI);
 
-  return CallInfo.first;
+  SDValue ret = CallInfo.first;
+  if (isa<VectorType>(RetTy))
+      ret = DAG.getNode(ISD::BITCAST, dl, RetVT, ret);
+  return ret; 
 }
 
 // ExpandChainLibCall - Expand a node into a call to a libcall. Similar to
 // ExpandLibCall except that the first operand is the in-chain.
 std::pair<SDValue, SDValue>
 SelectionDAGLegalize::ExpandChainLibCall(RTLIB::Libcall LC,
                                          SDNode *Node,
                                          bool isSigned) {
   SDValue InChain = Node->getOperand(0);
 
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+  DebugLoc dl = Node->getDebugLoc();
+
   TargetLowering::ArgListTy Args;
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 1, e = Node->getNumOperands(); i != e; ++i) {
     EVT ArgVT = Node->getOperand(i).getValueType();
     Type *ArgTy = ArgVT.getTypeForEVT(*DAG.getContext());
     Entry.Node = Node->getOperand(i);
     Entry.Ty = ArgTy;
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16);
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16);
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+    }
     Args.push_back(Entry);
   }
   SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
                                          TLI.getPointerTy());
-
-  Type *RetTy = Node->getValueType(0).getTypeForEVT(*DAG.getContext());
+  EVT RetVT = Node->getValueType(0);
+  Type *RetTy;
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+  } else {
+      RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  }
   TargetLowering::
   CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false, false,
                     0, TLI.getLibcallCallingConv(LC), /*isTailCall=*/false,
                     /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
-                    Callee, Args, DAG, Node->getDebugLoc());
+                    Callee, Args, DAG, dl);
   std::pair<SDValue, SDValue> CallInfo = TLI.LowerCallTo(CLI);
   
+  if (isa<VectorType>(RetTy))
+            CallInfo.first = DAG.getNode(ISD::BITCAST, dl, RetVT, CallInfo.first);
   return CallInfo;
 }
 
 SDValue SelectionDAGLegalize::ExpandFPLibCall(SDNode* Node,
                                               RTLIB::Libcall Call_F32,
                                               RTLIB::Libcall Call_F64,
                                               RTLIB::Libcall Call_F80,
                                               RTLIB::Libcall Call_F128,
SelectionDAGLegalize::ExpandDivRemLibCall(SDNode *Node,
   case MVT::i64:  LC= isSigned ? RTLIB::SDIVREM_I64 : RTLIB::UDIVREM_I64; break;
   case MVT::i128: LC= isSigned ? RTLIB::SDIVREM_I128:RTLIB::UDIVREM_I128; break;
   }
 
   // The input chain to this libcall is the entry node of the function.
   // Legalizing the call will automatically add the previous call to the
   // dependence.
   SDValue InChain = DAG.getEntryNode();
+  DebugLoc dl = Node->getDebugLoc();
 
   EVT RetVT = Node->getValueType(0);
   Type *RetTy = RetVT.getTypeForEVT(*DAG.getContext());
 
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+
   TargetLowering::ArgListTy Args;
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 0, e = Node->getNumOperands(); i != e; ++i) {
     EVT ArgVT = Node->getOperand(i).getValueType();
     Type *ArgTy = ArgVT.getTypeForEVT(*DAG.getContext());
     Entry.Node = Node->getOperand(i); Entry.Ty = ArgTy;
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16);
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16);
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+    }
     Args.push_back(Entry);
   }
 
   // Also pass the return address of the remainder.
   SDValue FIPtr = DAG.CreateStackTemporary(RetVT);
   Entry.Node = FIPtr;
   Entry.Ty = RetTy->getPointerTo();
   Entry.isSExt = isSigned;
   Entry.isZExt = !isSigned;
   Args.push_back(Entry);
 
   SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
                                          TLI.getPointerTy());
   
-  DebugLoc dl = Node->getDebugLoc();
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+  }
   TargetLowering::
   CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false, false,
                     0, TLI.getLibcallCallingConv(LC), /*isTailCall=*/false,
                     /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
                     Callee, Args, DAG, dl);
   std::pair<SDValue, SDValue> CallInfo = TLI.LowerCallTo(CLI);
 
   // Remainder is loaded back from the stack frame.
   SDValue Rem = DAG.getLoad(RetVT, dl, CallInfo.second, FIPtr,
                             MachinePointerInfo(), false, false, false, 0);
+  if (isa<VectorType>(RetTy))
+      Results.push_back(DAG.getNode(ISD::BITCAST, dl, RetVT, CallInfo.first));
+  else
       Results.push_back(CallInfo.first);
   Results.push_back(Rem);
 }
 
 /// isSinCosLibcallAvailable - Return true if sincos libcall is available.
 static bool isSinCosLibcallAvailable(SDNode *Node, const TargetLowering &TLI) {
   RTLIB::Libcall LC;
   switch (Node->getValueType(0).getSimpleVT().SimpleTy) {
   default: llvm_unreachable("Unexpected request for libcall!");

lib/CodeGen/SelectionDAG/LegalizeTypes.cpp

 // This file implements the SelectionDAG::LegalizeTypes method.  It transforms
 // an arbitrary well-formed SelectionDAG to only consist of legal types.  This
 // is common code shared among the LegalizeTypes*.cpp files.
 //
 //===----------------------------------------------------------------------===//
 
 #include "LegalizeTypes.h"
 #include "llvm/ADT/SetVector.h"
+#include "llvm/ADT/Triple.h"
 #include "llvm/IR/CallingConv.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 static cl::opt<bool>

 }
 
 // ExpandChainLibCall - Expand a node into a call to a libcall. Similar to
 // ExpandLibCall except that the first operand is the in-chain.
 std::pair<SDValue, SDValue>
 DAGTypeLegalizer::ExpandChainLibCall(RTLIB::Libcall LC,
                                          SDNode *Node,
                                          bool isSigned) {
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+  DebugLoc dl = Node->getDebugLoc();
+
   SDValue InChain = Node->getOperand(0);
 
   TargetLowering::ArgListTy Args;
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 1, e = Node->getNumOperands(); i != e; ++i) {
     EVT ArgVT = Node->getOperand(i).getValueType();
     Type *ArgTy = ArgVT.getTypeForEVT(*DAG.getContext());
     Entry.Node = Node->getOperand(i);
     Entry.Ty = ArgTy;
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16);
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16);
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+    }
     Args.push_back(Entry);
   }
   SDValue Callee = DAG.getExternalSymbol(TLI.getLibcallName(LC),
                                          TLI.getPointerTy());
 
-  Type *RetTy = Node->getValueType(0).getTypeForEVT(*DAG.getContext());
+  EVT RetVT = Node->getValueType(0);
+  Type *RetTy;
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+  } else {
+      RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  }
+
   TargetLowering::
   CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false, false,
                     0, TLI.getLibcallCallingConv(LC), /*isTailCall=*/false,
                     /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
                     Callee, Args, DAG, Node->getDebugLoc());
   std::pair<SDValue, SDValue> CallInfo = TLI.LowerCallTo(CLI);
 
+  if (isa<VectorType>(RetTy))
+      CallInfo.first = DAG.getNode(ISD::BITCAST, dl, RetVT, CallInfo.first);
   return CallInfo;
 }
 
 /// PromoteTargetBoolean - Promote the given target boolean to a target boolean
 /// of the given type.  A target boolean is an integer value, not necessarily of
 /// type i1, the bits of which conform to getBooleanContents.
 SDValue DAGTypeLegalizer::PromoteTargetBoolean(SDValue Bool, EVT VT) {
   DebugLoc dl = Bool.getDebugLoc();

lib/CodeGen/SelectionDAG/TargetLowering.cpp

 //===-- TargetLowering.cpp - Implement the TargetLowering class -----------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This implements the TargetLowering class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Target/TargetLowering.h"
 #include "llvm/ADT/BitVector.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/Triple.h"
 #include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineJumpTableInfo.h"
 #include "llvm/CodeGen/SelectionDAG.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/GlobalVariable.h"

 /// result of type RetVT.
 SDValue TargetLowering::makeLibCall(SelectionDAG &DAG,
                                     RTLIB::Libcall LC, EVT RetVT,
                                     const SDValue *Ops, unsigned NumOps,
                                     bool isSigned, DebugLoc dl) const {
   TargetLowering::ArgListTy Args;
   Args.reserve(NumOps);
 
+  Triple Trip(DAG.getTarget().getTargetTriple());
+  int Win64 = Trip.isArch64Bit() && Trip.isOSWindows();
+  SDValue InChain = DAG.getEntryNode();
   TargetLowering::ArgListEntry Entry;
   for (unsigned i = 0; i != NumOps; ++i) {
     Entry.Node = Ops[i];
     Entry.Ty = Entry.Node.getValueType().getTypeForEVT(*DAG.getContext());
     Entry.isSExt = isSigned;
     Entry.isZExt = !isSigned;
+    if (Win64 && Entry.Node.getValueType().getSizeInBits() > 64) {
+      //printf("win64 libcall arg %d i128*\n", i);
+      SDValue StackPtr = DAG.CreateStackTemporary(Entry.Node.getValueType(), 16); 
+      int SPFI = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
+      MachinePointerInfo PtrInfo = MachinePointerInfo::getFixedStack(SPFI);
+      // Emit a store to the stack slot.
+      InChain = DAG.getStore(InChain, dl, Entry.Node, StackPtr, PtrInfo,
+                                   false, false, 16); 
+      Entry.Node = StackPtr;
+      Entry.Ty = PointerType::get(Entry.Ty,0);
+      Entry.isSExt = false;
+      Entry.isZExt = false;
+    }
     Args.push_back(Entry);
   }
   SDValue Callee = DAG.getExternalSymbol(getLibcallName(LC), getPointerTy());
 
-  Type *RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  Type *RetTy;
+  if (Win64 && RetVT.getSizeInBits() == 128) {
+      RetTy = static_cast<EVT>(MVT::v2i64).getTypeForEVT(*DAG.getContext());
+  } else {
+      RetTy = RetVT.getTypeForEVT(*DAG.getContext());
+  }
   TargetLowering::
-  CallLoweringInfo CLI(DAG.getEntryNode(), RetTy, isSigned, !isSigned, false,
+  CallLoweringInfo CLI(InChain, RetTy, isSigned, !isSigned, false,
                     false, 0, getLibcallCallingConv(LC),
                     /*isTailCall=*/false,
                     /*doesNotReturn=*/false, /*isReturnValueUsed=*/true,
                     Callee, Args, DAG, dl);
   std::pair<SDValue,SDValue> CallInfo = LowerCallTo(CLI);
 
-  return CallInfo.first;
+  SDValue ret = CallInfo.first;
+  if (isa<VectorType>(RetTy))
+      ret = DAG.getNode(ISD::BITCAST, dl, RetVT, ret);
+  return ret;
 }
 
 
 /// SoftenSetCCOperands - Soften the operands of a comparison.  This code is
 /// shared among BR_CC, SELECT_CC, and SETCC handlers.
 void TargetLowering::softenSetCCOperands(SelectionDAG &DAG, EVT VT,
                                          SDValue &NewLHS, SDValue &NewRHS,
                                          ISD::CondCode &CCCode,