Extend or truncate __ptr32/__ptr64 pointers when dereferenced.

[akhuang]

Summary

A while ago I implemented the functionality to lower Microsoft ptr32
and ptr64 pointers, which are basically stored as 32-bit and 64-bit
pointers and are extended/truncated to the appropriate pointer size when
dereferenced. Apparently I never implemented this extend/truncate when
dereferencing part, so that is what this patch attempts to do.

For 32-bit pointers in 64-bit builds I put the sign/zero extend in
matchAddress in X86ISelDAGToDAG, but loading from 64-bit pointers in
32-bit builds fails in the legalizer before that point, which is why
this code is in two different places. Not sure if this is a good way of doing
this.

Bug: https://bugs.llvm.org/show_bug.cgi?id=42359

[hans]

which is why this code is in two different places

Could both cases be handled in the place that runs before the legalizer?

Hopefully Craig can take a look too, he's the expert here.

llvm/test/CodeGen/X86/mixed-ptr-sizes.ll

192

This just shows the sign extension. I would add a check line for the actual load too, to make sure that's correct and also to make the test easier to understand. Same for the tests below.

[JohnReagan]

Thanks for tackling this. Again, comparing this to what I have on my proprietary system with dual sized pointers, I have some observations (and some confusions on my part I'm sure)

For 32-bit targets, can you even talk about ptr64? My old 32-bit platforms don't allow it at all. However, your custom lowering for 32-bit targets seems to imply that you want to support ptr64 containers on 32-bit platforms by always truncating them. Or is the model that a 32-bit target would simply not use those two address spaces and therefore not generate the size/zero extension?

I'm not sure what "custom" means but you only set that for 32-bit targets. Does that mean the zero/sign-extension on the load doesn't occur for 64-bit targets? You still want to zero/sign-extend a ptr32 in a 64-bit target, yes? From what I think I read, the truncation of ptr32 for the store will only be for on 32-bit targets.

My memory was that you'd be able to mix-and-match ptr32 and ptr64 on 64-bit targets since you still need ptr32 to deal with legacy OS interfaces and such.

[hans]

In D81517#2084827, @JohnReagan wrote:

Thanks for tackling this. Again, comparing this to what I have on my proprietary system with dual sized pointers, I have some observations (and some confusions on my part I'm sure)

For 32-bit targets, can you even talk about ptr64? My old 32-bit platforms don't allow it at all. However, your custom lowering for 32-bit targets seems to imply that you want to support ptr64 containers on 32-bit platforms by always truncating them. Or is the model that a 32-bit target would simply not use those two address spaces and therefore not generate the size/zero extension?

It's following MSVC, whose behaviour can be observed here: https://godbolt.org/z/uNwjMW

[craig.topper]

I'm worried that load/store aren't the only things that can use these pointers. Or that we will have transformed the load/store into a target specific load/store before type legalizaton kicks in. I'm not an expert on address spaces, but it feels to me like we should have an address space cast in IR before the pointers are used by load/store. And that the address space cast is what should be turned into trunc or extend. @arsenm what do you think?

[arsenm]

In D81517#2085442, @craig.topper wrote:

I'm worried that load/store aren't the only things that can use these pointers. Or that we will have transformed the load/store into a target specific load/store before type legalizaton kicks in. I'm not an expert on address spaces, but it feels to me like we should have an address space cast in IR before the pointers are used by load/store. And that the address space cast is what should be turned into trunc or extend. @arsenm what do you think?

I don't think the addressing mode matching should be responsible for inserting this (it's also suspicious that this would need to look at the memory address space)

[craig.topper]

Part of me thinks we should just rewrite the loads/stores as early as possible using combineLoad/combineStore to use a ADDRSPACECAST. Then let the existing legalization of ADDRSPACECAST take care of it. And not change the setOperationAction for loads/store. I get a little nervous that something somewhere will care if an i32 load is Custom instead of Legal.

[akhuang]

Put addr space casting in combineLoad/combineStore.

[akhuang]

Thanks for the comments! I moved the addr space stuff to combineLoad/combineStore, which ends up being a lot simpler.

[craig.topper]

Can you rebase this patch? I added a 32-bit version of mixed-ptr-sizes.ll as mixed-ptr-sizes-i686.ll the other day. And the X86ISelDAGToDAG.cpp changes should already be done in trunk as well.

[akhuang]

Rebase and add more check lines to the tests.

[RKSimon]

@akhuang If you rebase the tests can now be regenerated with the update script

[akhuang]

@RKSimon oh, cool. thanks!

[akhuang]

Rebase tests.

[RKSimon]

please can you check the patch and rebase again? It looks like some of the code is now in the wrong function?

[akhuang]

Rebase, move code to the correct place

[akhuang]

sorry about that! It now should be the intended change.

[craig.topper]

LGTM

llvm/lib/Target/X86/X86ISelLowering.cpp

         ResElts = 128 / SVT.getSizeInBits();
       EVT TruncVT = EVT::getVectorVT(*DAG.getContext(), SVT, ResElts);
       SDValue Res = DAG.getNode(TruncOpc, DL, TruncVT, SatVal);
       return DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Res,
                          DAG.getIntPtrConstant(0, DL));
     }
   }
 
+  // Cast ptr32 and ptr64 pointers to the default address space before a load.
+  unsigned AddrSpace = Ld->getAddressSpace();
+  if (AddrSpace == X86AS::PTR64 || AddrSpace == X86AS::PTR32_SPTR ||
+      AddrSpace == X86AS::PTR32_UPTR) {
+    MVT PtrVT = TLI.getPointerTy(DAG.getDataLayout());
+    if (PtrVT != Ld->getBasePtr().getSimpleValueType()) {
+      SDValue Cast =
+          DAG.getAddrSpaceCast(dl, PtrVT, Ld->getBasePtr(), AddrSpace, 0);
+      return DAG.getLoad(RegVT, dl, Ld->getChain(), Cast, Ld->getPointerInfo(),
+                         Ld->getOriginalAlign(),
+                         Ld->getMemOperand()->getFlags());
+    }
+  }
+
   return SDValue();
 }
 
 /// This function detects the AVG pattern between vectors of unsigned i8/i16,
 /// which is c = (a + b + 1) / 2, and replace this operation with the efficient
 /// X86ISD::AVG instruction.
 static SDValue detectAVGPattern(SDValue In, EVT VT, SelectionDAG &DAG,
                                 const X86Subtarget &Subtarget,

         return EmitTruncSStore(false /* Unsigned saturation */, St->getChain(),
                                dl, Val, St->getBasePtr(),
                                St->getMemoryVT(), St->getMemOperand(), DAG);
     }
 
     return SDValue();
   }
 
+  // Cast ptr32 and ptr64 pointers to the default address space before a store.
+  unsigned AddrSpace = St->getAddressSpace();
+  if (AddrSpace == X86AS::PTR64 || AddrSpace == X86AS::PTR32_SPTR ||
+      AddrSpace == X86AS::PTR32_UPTR) {
+    MVT PtrVT = TLI.getPointerTy(DAG.getDataLayout());
+    if (PtrVT != St->getBasePtr().getSimpleValueType()) {
+      SDValue Cast =
+          DAG.getAddrSpaceCast(dl, PtrVT, St->getBasePtr(), AddrSpace, 0);
+      return DAG.getStore(St->getChain(), dl, StoredVal, Cast,
+                          St->getPointerInfo(), St->getOriginalAlign(),
+                          St->getMemOperand()->getFlags(), St->getAAInfo());
+    }
+  }
+
   // Turn load->store of MMX types into GPR load/stores.  This avoids clobbering
   // the FP state in cases where an emms may be missing.
   // A preferable solution to the general problem is to figure out the right
   // places to insert EMMS.  This qualifies as a quick hack.
 
   // Similarly, turn load->store of i64 into double load/stores in 32-bit mode.
   if (VT.getSizeInBits() != 64)
     return SDValue();

llvm/test/CodeGen/X86/mixed-ptr-sizes-i686.ll

 ; CHECK-O0-NEXT:    jmp _use_foo # TAILCALL
 entry:
   %0 = addrspacecast i32 addrspace(272)* %i to i32 addrspace(9)*
   %p_other = getelementptr inbounds %struct.Foo, %struct.Foo* %f, i32 0, i32 2
   store i32 addrspace(9)* %0, i32 addrspace(9)** %p_other, align 8
   tail call void @use_foo(%struct.Foo* %f)
   ret void
 }
+
+define i32 @test_load_sptr32(i32 addrspace(270)* %i) {
+; CHECK-LABEL: test_load_sptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl (%eax), %eax
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_load_sptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl (%eax), %eax
+; CHECK-O0-NEXT:    retl
+entry:
+  %0 = load i32, i32 addrspace(270)* %i, align 4
+  ret i32 %0
+}
+
+define i32 @test_load_uptr32(i32 addrspace(271)* %i) {
+; CHECK-LABEL: test_load_uptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl (%eax), %eax
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_load_uptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl (%eax), %eax
+; CHECK-O0-NEXT:    retl
+entry:
+  %0 = load i32, i32 addrspace(271)* %i, align 4
+  ret i32 %0
+}
+
+define i32 @test_load_ptr64(i32 addrspace(272)* %i) {
+; CHECK-LABEL: test_load_ptr64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl (%eax), %eax
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_load_ptr64:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    pushl %eax
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-O0-NEXT:    movl (%ecx), %ecx
+; CHECK-O0-NEXT:    movl %eax, (%esp)
+; CHECK-O0-NEXT:    movl %ecx, %eax
+; CHECK-O0-NEXT:    popl %ecx
+; CHECK-O0-NEXT:    retl
+entry:
+  %0 = load i32, i32 addrspace(272)* %i, align 8
+  ret i32 %0
+}
+
+define void @test_store_sptr32(i32 addrspace(270)* %s, i32 %i) {
+; CHECK-LABEL: test_store_sptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl %eax, (%ecx)
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_store_sptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-O0-NEXT:    movl %eax, (%ecx)
+; CHECK-O0-NEXT:    retl
+entry:
+  store i32 %i, i32 addrspace(270)* %s, align 4
+  ret void
+}
+
+define void @test_store_uptr32(i32 addrspace(271)* %s, i32 %i) {
+; CHECK-LABEL: test_store_uptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl %eax, (%ecx)
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_store_uptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-O0-NEXT:    movl %eax, (%ecx)
+; CHECK-O0-NEXT:    retl
+entry:
+  store i32 %i, i32 addrspace(271)* %s, align 4
+  ret void
+}
+
+define void @test_store_ptr64(i32 addrspace(272)* %s, i32 %i) {
+; CHECK-LABEL: test_store_ptr64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl %eax, (%ecx)
+; CHECK-NEXT:    retl
+; CHECK-O0-LABEL: test_store_ptr64:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-O0-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; CHECK-O0-NEXT:    movl %edx, (%ecx)
+; CHECK-O0-NEXT:    retl
+entry:
+  store i32 %i, i32 addrspace(272)* %s, align 8
+  ret void
+}

llvm/test/CodeGen/X86/mixed-ptr-sizes.ll

entry:
   %p64 = getelementptr inbounds %struct.Foo, %struct.Foo* %f, i64 0, i32 1
   store i32* %i, i32** %p64, align 8
   tail call void @use_foo(%struct.Foo* %f)
   ret void
 }
 
 ; Test that null can be passed as a 32-bit pointer.
 define dso_local void @test_null_arg(%struct.Foo* %f) {
+; CHECK-LABEL: test_null_arg:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    subq $40, %rsp
+; CHECK:         xorl %edx, %edx
+; CHECK-NEXT:    callq test_noop1
+; CHECK-NEXT:    nop
+; CHECK-NEXT:    addq $40, %rsp
+; CHECK-NEXT:    retq
+;
+; CHECK-O0-LABEL: test_null_arg:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    subq $40, %rsp
+; CHECK-O0:         xorl %edx, %edx
+; CHECK-O0-NEXT:    callq test_noop1
+; CHECK-O0-NEXT:    nop
+; CHECK-O0-NEXT:    addq $40, %rsp
+; CHECK-O0-NEXT:    retq
 entry:
   call void @test_noop1(%struct.Foo* %f, i32 addrspace(270)* null)
   ret void
 }
 
 ; Test casts between unrecognized address spaces.
 define void @test_unrecognized(%struct.Foo* %f, i32 addrspace(14)* %i) {
 ; CHECK-LABEL: test_unrecognized:

 ; CHECK-O0-NEXT:    jmp use_foo # TAILCALL
 entry:
   %0 = addrspacecast i32 addrspace(271)* %i to i32 addrspace(9)*
   %p32 = getelementptr inbounds %struct.Foo, %struct.Foo* %f, i64 0, i32 2
   store i32 addrspace(9)* %0, i32 addrspace(9)** %p32, align 8
   tail call void @use_foo(%struct.Foo* %f)
   ret void
 }
+
+define i32 @test_load_sptr32(i32 addrspace(270)* %i) {
+; CHECK-LABEL: test_load_sptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movslq  %ecx, %rax
+; CHECK-NEXT:    movl (%rax), %eax
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_load_sptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movslq  %ecx, %rax
+; CHECK-O0-NEXT:    movl (%rax), %eax
+; CHECK-O0-NEXT:    retq
+entry:
+  %0 = load i32, i32 addrspace(270)* %i, align 4
+  ret i32 %0
+}
+
+define i32 @test_load_uptr32(i32 addrspace(271)* %i) {
+; CHECK-LABEL: test_load_uptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl %ecx, %eax
+; CHECK-NEXT:    movl (%rax), %eax
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_load_uptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl %ecx, %eax
+; CHECK-O0-NEXT:    # kill: def $rax killed $eax
+; CHECK-O0-NEXT:    movl (%rax), %eax
+; CHECK-O0-NEXT:    retq
+entry:
+  %0 = load i32, i32 addrspace(271)* %i, align 4
+  ret i32 %0
+}
+
+define i32 @test_load_ptr64(i32 addrspace(272)* %i) {
+; CHECK-LABEL: test_load_ptr64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl (%rcx), %eax
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_load_ptr64:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl (%rcx), %eax
+; CHECK-O0-NEXT:    retq
+entry:
+  %0 = load i32, i32 addrspace(272)* %i, align 8
+  ret i32 %0
+}
+
+define void @test_store_sptr32(i32 addrspace(270)* %s, i32 %i) {
+; CHECK-LABEL: test_store_sptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movslq %ecx, %rax
+; CHECK-NEXT:    movl %edx, (%rax)
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_store_sptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movslq %ecx, %rax
+; CHECK-O0-NEXT:    movl %edx, (%rax)
+; CHECK-O0-NEXT:    retq
+entry:
+  store i32 %i, i32 addrspace(270)* %s, align 4
+  ret void
+}
+
+define void @test_store_uptr32(i32 addrspace(271)* %s, i32 %i) {
+; CHECK-LABEL: test_store_uptr32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl %ecx, %eax
+; CHECK-NEXT:    movl %edx, (%rax)
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_store_uptr32:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl %ecx, %eax
+; CHECK-O0-NEXT:    # kill: def $rax killed $eax
+; CHECK-O0-NEXT:    movl %edx, (%rax)
+; CHECK-O0-NEXT:    retq
+entry:
+  store i32 %i, i32 addrspace(271)* %s, align 4
+  ret void
+}
+
+define void @test_store_ptr64(i32 addrspace(272)* %s, i32 %i) {
+; CHECK-LABEL: test_store_ptr64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movl %edx, (%rcx)
+; CHECK-NEXT:    retq
+; CHECK-O0-LABEL: test_store_ptr64:
+; CHECK-O0:       # %bb.0: # %entry
+; CHECK-O0-NEXT:    movl %edx, (%rcx)
+; CHECK-O0-NEXT:    retq
+entry:
+  store i32 %i, i32 addrspace(272)* %s, align 8
+  ret void
+}