[X86] Avoid %fs:(%eax) references in x32 mode

[hvdijk]

Summary

The ABI explains that %fs:(%eax) zero-extends %eax to 64 bits, and adds
that the TLS base address, but that the TLS base address need not be
at the start of the TLS block, TLS references may use negative offsets.

[RKSimon]

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

1618

Is there a better name that we can use than NoRegisters?

[hvdijk]

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

1618

I was thinking of something like AllowSegmentReg initially, but that would be a poor name, since except for X32, segment registers are always allowed. I picked NoRegisters to reflect that when NoRegisters is true, it is known that AM does not and will not refer to any registers other than what matchLoadInAddress does. I now see that that is not entirely true either, as AM may in theory already reference another segment register. NoBaseReg would be slightly more accurate, but still not very clear. Perhaps I should go back to what I originally had and change it to the slightly verbose but pretty clear AllowSegmentRegForX32?

[RKSimon]

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

1618

Something X32 specific would be better - so AllowSegmentRegForX32 would be fine.

llvm/test/CodeGen/X86/pic.ll

339–340

it doesn't look great to allow add or lea like this - maybe add an additional CHECK-X32-ISEL/CHECK-X32-FAST prefix and check them properly?

[hvdijk]

Rename parameter, split addl/leal check in two.

[hvdijk]

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

1618

Alright, done.

llvm/test/CodeGen/X86/pic.ll

339–340

My thinking was that the leal is just used as a three-operand add. This test does not check which registers get used, so it seemed odd to me to check that the add's output register is the same as one of its inputs, it seemed like either addl or leal would be fine for either I686 or X32. However, thinking about it some more, there is a difference, and it does make sense to restrict which instructions get accepted by the test: I686 gets addl, because that's the shorter instruction. X32 gets leal, because this prevents clobbering its inputs: for x32, both inputs are reused later on. So, done.

[hvdijk]

llvm/test/CodeGen/X86/pic.ll

339–340

Looking at the new diff: for consistency with the other checks, I should have put ({{%.*,%.*}}) here rather than ({{.*,.*}}). I missed this until after I had already submitted the updated version. It's a trivial change that does not affect anything, but will update nonetheless.

[hvdijk]

Be consistent in CHECK lines.

[RKSimon]

LGTM

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

 
 // Include the pieces autogenerated from the target description.
 #include "X86GenDAGISel.inc"
 
   private:
     void Select(SDNode *N) override;
 
     bool foldOffsetIntoAddress(uint64_t Offset, X86ISelAddressMode &AM);
-    bool matchLoadInAddress(LoadSDNode *N, X86ISelAddressMode &AM);
+    bool matchLoadInAddress(LoadSDNode *N, X86ISelAddressMode &AM,
+                            bool AllowSegmentRegForX32 = false);
     bool matchWrapper(SDValue N, X86ISelAddressMode &AM);
     bool matchAddress(SDValue N, X86ISelAddressMode &AM);
     bool matchVectorAddress(SDValue N, X86ISelAddressMode &AM);
     bool matchAdd(SDValue &N, X86ISelAddressMode &AM, unsigned Depth);
     bool matchAddressRecursively(SDValue N, X86ISelAddressMode &AM,
                                  unsigned Depth);
     bool matchAddressBase(SDValue N, X86ISelAddressMode &AM);
     bool selectAddr(SDNode *Parent, SDValue N, SDValue &Base,
if (AM.BaseType == X86ISelAddressMode::FrameIndexBase &&
         !isDispSafeForFrameIndex(Val))
       return true;
   }
   AM.Disp = Val;
   return false;
 
 }
 
-bool X86DAGToDAGISel::matchLoadInAddress(LoadSDNode *N, X86ISelAddressMode &AM){
+bool X86DAGToDAGISel::matchLoadInAddress(LoadSDNode *N, X86ISelAddressMode &AM,
+                                         bool AllowSegmentRegForX32) {
   SDValue Address = N->getOperand(1);
 
   // load gs:0 -> GS segment register.
   // load fs:0 -> FS segment register.
   //
-  // This optimization is valid because the GNU TLS model defines that
-  // gs:0 (or fs:0 on X86-64) contains its own address.
+  // This optimization is generally valid because the GNU TLS model defines that
+  // gs:0 (or fs:0 on X86-64) contains its own address. However, for X86-64 mode
+  // with 32-bit registers, as we get in ILP32 mode, those registers are first
+  // zero-extended to 64 bits and then added it to the base address, which gives
+  // unwanted results when the register holds a negative value.
   // For more information see http://people.redhat.com/drepper/tls.pdf
-  if (ConstantSDNode *C = dyn_cast<ConstantSDNode>(Address))
+  if (ConstantSDNode *C = dyn_cast<ConstantSDNode>(Address)) {
     if (C->getSExtValue() == 0 && AM.Segment.getNode() == nullptr &&
         !IndirectTlsSegRefs &&
         (Subtarget->isTargetGlibc() || Subtarget->isTargetAndroid() ||
-         Subtarget->isTargetFuchsia()))
+         Subtarget->isTargetFuchsia())) {
+      if (Subtarget->isTarget64BitILP32() && !AllowSegmentRegForX32)
+        return true;
       switch (N->getPointerInfo().getAddrSpace()) {
       case X86AS::GS:
         AM.Segment = CurDAG->getRegister(X86::GS, MVT::i16);
         return false;
       case X86AS::FS:
         AM.Segment = CurDAG->getRegister(X86::FS, MVT::i16);
         return false;
       // Address space X86AS::SS is not handled here, because it is not used to
       // address TLS areas.
       }
+    }
+  }
 
   return true;
 }
 
 /// Try to match X86ISD::Wrapper and X86ISD::WrapperRIP nodes into an addressing
 /// mode. These wrap things that will resolve down into a symbol reference.
 /// If no match is possible, this returns true, otherwise it returns false.
 bool X86DAGToDAGISel::matchWrapper(SDValue N, X86ISelAddressMode &AM) {

 }
 
 /// Add the specified node to the specified addressing mode, returning true if
 /// it cannot be done. This just pattern matches for the addressing mode.
 bool X86DAGToDAGISel::matchAddress(SDValue N, X86ISelAddressMode &AM) {
   if (matchAddressRecursively(N, AM, 0))
     return true;
 
+  // Post-processing: Make a second attempt to fold a load, if we now know
+  // that there will not be any other register. This is only performed for
+  // 64-bit ILP32 mode since 32-bit mode and 64-bit LP64 mode will have folded
+  // any foldable load the first time.
+  if (Subtarget->isTarget64BitILP32() &&
+      AM.BaseType == X86ISelAddressMode::RegBase &&
+      AM.Base_Reg.getNode() != nullptr && AM.IndexReg.getNode() == nullptr) {
+    SDValue Save_Base_Reg = AM.Base_Reg;
+    if (auto *LoadN = dyn_cast<LoadSDNode>(Save_Base_Reg)) {
+      AM.Base_Reg = SDValue();
+      if (matchLoadInAddress(LoadN, AM, /*AllowSegmentRegForX32=*/true))
+        AM.Base_Reg = Save_Base_Reg;
+    }
+  }
+
   // Post-processing: Convert lea(,%reg,2) to lea(%reg,%reg), which has
   // a smaller encoding and avoids a scaled-index.
   if (AM.Scale == 2 &&
       AM.BaseType == X86ISelAddressMode::RegBase &&
       AM.Base_Reg.getNode() == nullptr) {
     AM.Base_Reg = AM.IndexReg;
     AM.Scale = 1;
   }

llvm/test/CodeGen/X86/pic.ll

 ; CHECK-I686: calll .L10$pb
 ; CHECK-I686-NEXT:  .L10$pb:
 ; CHECK-I686-NEXT:  popl
 ; CHECK-I686: addl  $_GLOBAL_OFFSET_TABLE_+(.L{{.*}}-.L10$pb),
 ; CHECK-I686-DAG: movl  tlsdstie@GOTNTPOFF(
 ; CHECK-I686-DAG: movl  %gs:0,
 ; CHECK-X32-DAG:  movl  tlsdstie@GOTTPOFF(%rip),
 ; CHECK-X32-DAG:  movl  %fs:0,
-; CHECK:  addl
+; CHECK-I686: addl
+; CHECK-X32:  leal  ({{%.*,%.*}}),
 ; CHECK-I686: movl  tlsptrie@GOTNTPOFF(
 ; CHECK-X32:  movl  tlsptrie@GOTTPOFF(%rip),
 ; CHECK-I686: movl  {{%.*}}, %gs:(
-; CHECK-X32:  movl  {{%.*}}, %fs:(
+; CHECK-X32:  movl  {{%.*}}, ({{%.*,%.*}})
 ; CHECK-I686: movl  tlssrcie@GOTNTPOFF(
 ; CHECK-X32:  movl  tlssrcie@GOTTPOFF(%rip),
 ; CHECK-I686: movl  %gs:(
-; CHECK-X32:  movl  %fs:(
+; CHECK-X32:  movl  ({{%.*,%.*}}),
 ; CHECK-I686: movl  {{%.*}}, %gs:(
-; CHECK-X32:  movl  {{%.*}}, %fs:(
+; CHECK-X32:  movl  {{%.*}}, ({{%.*,%.*}})
 ; CHECK-I686: ret
 ; CHECK-X32:  retq
 }
 
 define void @test11() nounwind {
 entry:
     store i32* @tlsdstle, i32** @tlsptrle
     %tmp.s = load i32, i32* @tlssrcle

llvm/test/CodeGen/X86/tls-pie.ll

 ; X86-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp0-.L2$pb), %eax
 ; X86-NEXT:    movl i2@GOTNTPOFF(%eax), %eax
 ; X86-NEXT:    movl %gs:(%eax), %eax
 ; X86-NEXT:    retl
 ;
 ; X32-LABEL: f3:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movl i2@{{.*}}(%rip), %eax
-; X32-NEXT:    movl %fs:(%eax), %eax
+; X32-NEXT:    movl %fs:0, %ecx
+; X32-NEXT:    movl (%ecx,%eax), %eax
 ; X32-NEXT:    retq
 ;
 ; X64-LABEL: f3:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movq i2@{{.*}}(%rip), %rax
 ; X64-NEXT:    movl %fs:(%rax), %eax
 ; X64-NEXT:    retq
 entry: