[X86][FPEnv] Lowering of {get,set,reset}_fpmode

[sepavloff]

Rebased patch

[RKSimon]

llvm/test/CodeGen/X86/fpenv.ll

4

regenerate with the update_llc_test_checks.py script?

llvm/test/CodeGen/X86/fpenv32.ll

2

regenerate with the update_llc_test_checks.py script?

[sepavloff]

Rebased and used update_llc_test_checks.py script

[sepavloff]

Rebased patch

[sepavloff]

Rebased patch

[craig.topper]

llvm/test/CodeGen/X86/fpenv32.ll

17

I'm still not clear who the intended user of these intrinsics are? How would they know when to use i32 and when to use i64?

[sepavloff]

llvm/test/CodeGen/X86/fpenv32.ll

17

who the intended user of these intrinsics are?

These intrinsics can be useful at least in these cases:

• native implementation of fegetmode/fesetmode,
• internally by compiler when it implements pragmas changing control modes. Such pragma acts only within a compound statement and control modes should be restored upon exit.

How would they know when to use i32 and when to use i64?

This is a platform-dependent choice, just as the size of pointers. We could encode them in DataLayout (see D71741).

[craig.topper]

llvm/test/CodeGen/X86/fpenv32.ll

17

who the intended user of these intrinsics are?

These intrinsics can be useful at least in these cases:

• native implementation of fegetmode/fesetmode,
• internally by compiler when it implements pragmas changing control modes. Such pragma acts only within a compound statement and control modes should be restored upon exit.

For the pragma case, would that mean the frontend would have to pass different values to the intrinsic for each target to match the target dependent behavior? Is the pragma also target specific? Or is there some target independent representation a frontend should be giving to llvm.

The ABI code in clang is a place where we have a ton of target specific rules in the frontend. It has come up many times on the mailing lists that this is bad design as every frontend needs to reimplement it. So I want to make sure we're being careful about the interface here and not creating work for frontends.

How would they know when to use i32 and when to use i64?

This is a platform-dependent choice, just as the size of pointers. We could encode them in DataLayout (see D71741).

Based on the tests here it appears the size to use is different on 32-bit mode depending on whether sse is enabled. DataLayout can't be dependent on subtarget features. It's derived from the target triple.

[kpn]

llvm/test/CodeGen/X86/fpenv32.ll

17

What pragmas are there that change control modes or the floating point environment? I don't think we have any, or at least none are required by C standards document.

[sepavloff]

llvm/test/CodeGen/X86/fpenv32.ll

17

who the intended user of these intrinsics are?

These intrinsics can be useful at least in these cases:

• native implementation of fegetmode/fesetmode,
• internally by compiler when it implements pragmas changing control modes. Such pragma acts only within a compound statement and control modes should be restored upon exit.

For the pragma case, would that mean the frontend would have to pass different values to the intrinsic for each target to match the target dependent behavior?

Yes. The frontend now does the same things for long and some other types, it takes IR representation from DataLayout , and it is different for different architectures.

Is the pragma also target specific?

pragma STDC FENV_ROUND is not target specific. Target-specific pragmas are of course possible, for example it could be a pragma that sets rounding mode for a particular type, if the target supports different modes for different types.

Or is there some target independent representation a frontend should be giving to llvm.

For llvm properly generated variables are enough. It is frontend that need information, which can be supplied by DataLayout or TatgetInfo.

The ABI code in clang is a place where we have a ton of target specific rules in the frontend. It has come up many times on the mailing lists that this is bad design as every frontend needs to reimplement it. So I want to make sure we're being careful about the interface here and not creating work for frontends.

The ABI is not touched here, these are intrinsics, they are not called as regular functions.

How would they know when to use i32 and when to use i64?

This is a platform-dependent choice, just as the size of pointers. We could encode them in DataLayout (see D71741).

Based on the tests here it appears the size to use is different on 32-bit mode depending on whether sse is enabled. DataLayout can't be dependent on subtarget features. It's derived from the target triple.

We can use the widest type for the mode set.

It is possible to use i64 for all targets, it would be enough for all targets supported by glibc. There are however functions fesetenv and fegetenv and in that case we cannot do similar thing as the size of FP state on X86 is 256 bits.

[craig.topper]

llvm/test/CodeGen/X86/fpenv32.ll

17

The ABI code in clang is a place where we have a ton of target specific rules in the frontend. It has come up many times on the mailing lists that this is bad design as every frontend needs to reimplement it. So I want to make sure we're being careful about the interface here and not creating work for frontends.

The ABI is not touched here, these are intrinsics, they are not called as regular functions.

I only mentioned ABI because it is an example of a place where the interface between llvm and frontends is considered the wrong design. I wasn't trying to say that this affects ABI.

It sounds like for this intrinsic to be usable by a frontend, that any frontend that wants to use it needs to know exactly which bits represent rounding mode, what encoding they use, the fact that x86 has rounding mode bits in two places, etc. That's a lot of information to try to pack into datalayout so likely it will have to just be hardcoded into custom C++ code in the frontend. Then when another frontend wants to use it they will have to write the same code into their frontend. And they'll need to update it every time they want to add support in their frontend for a new target that llvm already supports. This doesn't seem like a clean division between frontend and backend responsibilities.

[sepavloff]

llvm/test/CodeGen/X86/fpenv32.ll

17

What pragmas are there that change control modes or the floating point environment? I don't think we have any, or at least none are required by C standards document.

#pragma STDC FENV_ROUND can change the floating point environment.

17

It sounds like for this intrinsic to be usable by a frontend, that any frontend that wants to use it needs to know exactly which bits represent rounding mode, what encoding they use, the fact that x86 has rounding mode bits in two places, etc.

There is no attempt to interpret the control modes value. The main expected use cases are:

• Use get_fpmode to read current modes, tweak it, for example by setting new rounding mode using set_rounding, then restore the modes using set_fpmode. In this case the value returned by get_fpmode is opaque.

• Some code can call set_fpmode (actually fesetmode) and provide it with an expression formatted in a target-specific way to set required control modes. This usage can solve the problem described in https://reviews.llvm.org/D74729#2006017. In this case the frontend does not need to know what bits of control modes value mean.

The only thing that DataLayout need to keep is the size of control modes (and floating point environment in fegetenv).

[kpn]

llvm/test/CodeGen/X86/fpenv32.ll

17

I'm not convinced that it is _required_ to change the floating point environment. The compiler inserting calls to fesetround() would have a performance cost, and we shouldn't pay that cost unless we are required to by the standard.

I've started a thread on cfe-dev named "C2x FP_ROUND pragma meaning ambiguous?"

[kpn]

llvm/test/CodeGen/X86/fpenv32.ll

17

From asking on cfe-dev, it sounds like this FENV_ROUND pragma is a perfect use case for the constrained FP intrinsics. After all, they do have that rounding mode argument, and I'm pretty sure there are issues here with instructions getting reordered and possibly moved above or below the call to change the hardware rounding mode.

It would be up to the target backend to decide when its instructions can't or won't embed a rounding mode in them and instead issue the instructions to change the rounding mode.

[RKSimon]

Reverse ping

[sepavloff]

Rebased patch. Fixed issue of getConstantPool.

[sepavloff]

Changed mode type in test fpenv32.ll from i32 to i64 to match glibc types

[sepavloff]

Update patch because dependency has been updated

llvm/lib/Target/X86/X86ISelLowering.h

enum NodeType : unsigned {
     VZEXT_LOAD,
 
     // extract_vector_elt, store.
     VEXTRACT_STORE,
 
     // scalar broadcast from memory
     VBROADCAST_LOAD,
 
-    // Store FP control world into i16 memory.
+    // Store FP control word into i16 memory.
     FNSTCW16m,
 
+    // Load FP control word from i16 memory.
+    FLDCW16m,
+
     /// This instruction implements FP_TO_SINT with the
     /// integer destination in memory and a FP reg source.  This corresponds
     /// to the X86::FIST*m instructions and the rounding mode change stuff. It
     /// has two inputs (token chain and address) and two outputs (int value
     /// and token chain). Memory VT specifies the type to store to.
     FP_TO_INT_IN_MEM,
 
     /// This instruction implements SINT_TO_FP with the
private:
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAME_TO_ARGS_OFFSET(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerEH_RETURN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_LONGJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETUP_DISPATCH(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINIT_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerGET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerSET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerRESET_FPMODE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerWin64_i128OP(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerGC_TRANSITION(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerFaddFsub(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFP_EXTEND(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFP_ROUND(SDValue Op, SelectionDAG &DAG) const;
 
     SDValue LowerF128Call(SDValue Op, SelectionDAG &DAG,

llvm/lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::X86TargetLowering(const X86TargetMachine &TM,
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i8   , Legal);
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1   , Expand);
 
   setOperationAction(ISD::FREM             , MVT::f32  , Expand);
   setOperationAction(ISD::FREM             , MVT::f64  , Expand);
   setOperationAction(ISD::FREM             , MVT::f80  , Expand);
   setOperationAction(ISD::FREM             , MVT::f128 , Expand);
   setOperationAction(ISD::FLT_ROUNDS_      , MVT::i32  , Custom);
+  setOperationAction(ISD::GET_FPMODE       , MVT::i64  , Custom);
+  setOperationAction(ISD::GET_FPMODE       , MVT::i32  , Custom);
+  setOperationAction(ISD::SET_FPMODE       , MVT::i64  , Custom);
+  setOperationAction(ISD::SET_FPMODE       , MVT::i32  , Custom);
+  setOperationAction(ISD::RESET_FPMODE     , MVT::Other, Custom);
 
   // Promote the i8 variants and force them on up to i32 which has a shorter
   // encoding.
   setOperationPromotedToType(ISD::CTTZ           , MVT::i8   , MVT::i32);
   setOperationPromotedToType(ISD::CTTZ_ZERO_UNDEF, MVT::i8   , MVT::i32);
   if (!Subtarget.hasBMI()) {
     setOperationAction(ISD::CTTZ           , MVT::i16  , Custom);
     setOperationAction(ISD::CTTZ           , MVT::i32  , Custom);
DAG.getNode(ISD::AND, DL, MVT::i32,
                 DAG.getNode(ISD::SRL, DL, MVT::i32, LUT, Shift),
                 DAG.getConstant(3, DL, MVT::i32));
 
   RetVal = DAG.getZExtOrTrunc(RetVal, DL, VT);
 
   return DAG.getMergeValues({RetVal, Chain}, DL);
 }
 
+SDValue X86TargetLowering::LowerGET_FPMODE(SDValue Op,
+                                           SelectionDAG &DAG) const {
+  MachineFunction &MF = DAG.getMachineFunction();
+  SDLoc DL(Op);
+  SDValue Chain = Op.getOperand(0);
+  unsigned StackSize = Subtarget.hasSSE1() ? 8 : 2;
+  Align StackAlign(StackSize);
+
+  // Save FP Control Word to stack slot
+  int SSFI = MF.getFrameInfo().CreateStackObject(StackSize, StackAlign, false);
+  SDValue StackSlot =
+      DAG.getFrameIndex(SSFI, getPointerTy(DAG.getDataLayout()));
+  MachinePointerInfo MPI = MachinePointerInfo::getFixedStack(MF, SSFI);
+
+  SDValue Ops[] = {Chain, StackSlot};
+  Chain = DAG.getMemIntrinsicNode(X86ISD::FNSTCW16m, DL,
+                                  DAG.getVTList(MVT::Other), Ops, MVT::i16, MPI,
+                                  StackAlign, MachineMemOperand::MOStore);
+
+  // Load FP Control Word from stack slot
+  SDValue CWD = DAG.getLoad(MVT::i16, DL, Chain, StackSlot, MPI, StackAlign);
+  Chain = CWD.getValue(1);
+  CWD = DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i32, CWD);
+
+  if (!Subtarget.hasSSE1()) {
+    if (Op.getValueType() == MVT::i64)
+      CWD = DAG.getNode(ISD::BUILD_PAIR, DL, MVT::i64, CWD,
+                        DAG.getUNDEF(MVT::i32));
+    return DAG.getMergeValues({CWD, Chain}, DL);
+  }
+
+  // If target supports SSE, read MXCSR as well.
+  assert(Op.getValueType() == MVT::i64);
+  SDValue MXCSRAddr =
+      DAG.getNode(ISD::ADD, DL, StackSlot.getValueType(), StackSlot,
+                  DAG.getConstant(4, DL, StackSlot.getValueType()));
+  // Store MXCSR into memory.
+  Chain = DAG.getNode(
+      ISD::INTRINSIC_VOID, DL, DAG.getVTList(MVT::Other), Chain,
+      DAG.getTargetConstant(Intrinsic::x86_sse_stmxcsr, DL, MVT::i32),
+      MXCSRAddr);
+
+  if (Subtarget.is64Bit())
+    return DAG.getLoad(MVT::i64, DL, Chain, StackSlot, MPI);
+
+  // Load MXCSR from stack slot and put it into upper 32 bits of result.
+  SDValue MXCSR = DAG.getLoad(MVT::i32, DL, Chain, StackSlot, MPI);
+  Chain = MXCSR.getValue(1);
+  SDValue Result = DAG.getNode(ISD::BUILD_PAIR, DL, MVT::i64, CWD, MXCSR);
+  return DAG.getMergeValues({Result, Chain}, DL);
+}
+
+SDValue X86TargetLowering::LowerSET_FPMODE(SDValue Op,
+                                           SelectionDAG &DAG) const {
+  MachineFunction &MF = DAG.getMachineFunction();
+  SDLoc DL(Op);
+  SDValue Chain = Op.getOperand(0);
+  unsigned StackSize = Subtarget.hasSSE1() ? 8 : 2;
+  Align StackAlign(StackSize);
+
+  // Store FP modes into stack slot.
+  int SSFI = MF.getFrameInfo().CreateStackObject(StackSize, StackAlign, false);
+  SDValue StackSlot =
+      DAG.getFrameIndex(SSFI, getPointerTy(DAG.getDataLayout()));
+  MachinePointerInfo MPI = MachinePointerInfo::getFixedStack(MF, SSFI);
+  SDValue Modes = Op.getOperand(1);
+  if (!Subtarget.hasSSE1() && Modes.getValueType() == MVT::i64)
+    Modes = DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, Modes);
+  Chain = DAG.getStore(Chain, DL, Modes, StackSlot, MPI);
+
+  // Set X87 state.
+  SDValue Ops[] = {Chain, StackSlot};
+  Chain = DAG.getMemIntrinsicNode(X86ISD::FLDCW16m, DL,
+                                  DAG.getVTList(MVT::Other), Ops, MVT::i16, MPI,
+                                  StackAlign, MachineMemOperand::MOLoad);
+  if (!Subtarget.hasSSE1())
+    return Chain;
+
+  // If target supports SSE, set MXCSR as well.
+  assert(Modes.getValueType() == MVT::i64);
+  SDValue MXCSRAddr =
+      DAG.getNode(ISD::ADD, DL, StackSlot.getValueType(), StackSlot,
+                  DAG.getConstant(4, DL, StackSlot.getValueType()));
+  Chain = DAG.getNode(
+      ISD::INTRINSIC_VOID, DL, DAG.getVTList(MVT::Other), Chain,
+      DAG.getTargetConstant(Intrinsic::x86_sse_ldmxcsr, DL, MVT::i32),
+      MXCSRAddr);
+  return Chain;
+}
+
+SDValue X86TargetLowering::LowerRESET_FPMODE(SDValue Op,
+                                             SelectionDAG &DAG) const {
+  SDLoc DL(Op);
+  SDValue Chain = Op.getOperand(0);
+
+  // x87 FPU Control Word: mask all floating-point exceptions, sets rounding to
+  // nearest. FPU precision is set to 53 bits on Windows and 64 bits otherwise
+  // for compatibility with glibc.
+  const auto &Subtarget = static_cast<const X86Subtarget &>(DAG.getSubtarget());
+  unsigned X87CW = Subtarget.isTargetWindowsMSVC() ? 0x27F : 0x37F;
+  auto X87Mode = ConstantInt::get(Type::getInt32Ty(*DAG.getContext()), X87CW);
+  SDValue Mode = DAG.getConstantPool(X87Mode, MVT::i32);
+  SDValue Ops[] = {Chain, Mode};
+  MachinePointerInfo MPI =
+      MachinePointerInfo::getConstantPool(DAG.getMachineFunction());
+  Chain = DAG.getMemIntrinsicNode(X86ISD::FLDCW16m, DL,
+                                  DAG.getVTList(MVT::Other), Ops, MVT::i16, MPI,
+                                  Align(2), MachineMemOperand::MOLoad);
+  if (!Subtarget.hasSSE1())
+    return Chain;
+
+  // MXCSR: mask all floating-point exceptions, sets rounding to nearest,
+  // clear all exceptions, sets DAZ and FTZ to 0.
+  auto MXCSR = ConstantInt::get(Type::getInt32Ty(*DAG.getContext()), 0x1F80);
+  Mode = DAG.getConstantPool(MXCSR, MVT::i32);
+  Chain = DAG.getNode(
+      ISD::INTRINSIC_VOID, DL, MVT::Other, Chain,
+      DAG.getTargetConstant(Intrinsic::x86_sse_ldmxcsr, DL, MVT::i32), Mode);
+  return Chain;
+}
+
 /// Lower a vector CTLZ using native supported vector CTLZ instruction.
 //
 // i8/i16 vector implemented using dword LZCNT vector instruction
 // ( sub(trunc(lzcnt(zext32(x)))) ). In case zext32(x) is illegal,
 // split the vector, perform operation on it's Lo a Hi part and
 // concatenate the results.
 static SDValue LowerVectorCTLZ_AVX512CDI(SDValue Op, SelectionDAG &DAG,
                                          const X86Subtarget &Subtarget) {
SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::FSINCOS:            return LowerFSINCOS(Op, Subtarget, DAG);
   case ISD::MLOAD:              return LowerMLOAD(Op, Subtarget, DAG);
   case ISD::MSTORE:             return LowerMSTORE(Op, Subtarget, DAG);
   case ISD::MGATHER:            return LowerMGATHER(Op, Subtarget, DAG);
   case ISD::MSCATTER:           return LowerMSCATTER(Op, Subtarget, DAG);
   case ISD::GC_TRANSITION_START:
   case ISD::GC_TRANSITION_END:  return LowerGC_TRANSITION(Op, DAG);
   case ISD::ADDRSPACECAST:      return LowerADDRSPACECAST(Op, DAG);
+  case ISD::GET_FPMODE:         return LowerGET_FPMODE(Op, DAG);
+  case ISD::SET_FPMODE:         return LowerSET_FPMODE(Op, DAG);
+  case ISD::RESET_FPMODE:       return LowerRESET_FPMODE(Op, DAG);
   case X86ISD::CVTPS2PH:        return LowerCVTPS2PH(Op, DAG);
   }
 }
 
 /// Places new result values for the node in Results (their number
 /// and types must exactly match those of the original return values of
 /// the node), or leaves Results empty, which indicates that the node is not
 /// to be custom lowered after all.
case ISD::LOAD: {
     Results.push_back(Res.getValue(1));
     return;
   }
   case ISD::ADDRSPACECAST: {
     SDValue V = LowerADDRSPACECAST(SDValue(N,0), DAG);
     Results.push_back(V);
     return;
   }
+  case ISD::GET_FPMODE: {
+    SDValue V = LowerGET_FPMODE(SDValue(N, 0), DAG);
+    Results.push_back(V);
+    Results.push_back(V.getValue(1));
+    return;
+  }
   }
 }
 
 const char *X86TargetLowering::getTargetNodeName(unsigned Opcode) const {
   switch ((X86ISD::NodeType)Opcode) {
   case X86ISD::FIRST_NUMBER:       break;
 #define NODE_NAME_CASE(NODE) case X86ISD::NODE: return "X86ISD::" #NODE;
   NODE_NAME_CASE(BSF)
#define NODE_NAME_CASE(NODE) case X86ISD::NODE: return "X86ISD::" #NODE;
   NODE_NAME_CASE(TLSBASEADDR)
   NODE_NAME_CASE(TLSCALL)
   NODE_NAME_CASE(EH_SJLJ_SETJMP)
   NODE_NAME_CASE(EH_SJLJ_LONGJMP)
   NODE_NAME_CASE(EH_SJLJ_SETUP_DISPATCH)
   NODE_NAME_CASE(EH_RETURN)
   NODE_NAME_CASE(TC_RETURN)
   NODE_NAME_CASE(FNSTCW16m)
+  NODE_NAME_CASE(FLDCW16m)
   NODE_NAME_CASE(LCMPXCHG_DAG)
   NODE_NAME_CASE(LCMPXCHG8_DAG)
   NODE_NAME_CASE(LCMPXCHG16_DAG)
   NODE_NAME_CASE(LCMPXCHG8_SAVE_EBX_DAG)
   NODE_NAME_CASE(LCMPXCHG16_SAVE_RBX_DAG)
   NODE_NAME_CASE(LADD)
   NODE_NAME_CASE(LSUB)
   NODE_NAME_CASE(LOR)

llvm/lib/Target/X86/X86InstrFPStack.td

 
 def SDTX86Fld       : SDTypeProfile<1, 1, [SDTCisFP<0>,
                                            SDTCisPtrTy<1>]>;
 def SDTX86Fst       : SDTypeProfile<0, 2, [SDTCisFP<0>,
                                            SDTCisPtrTy<1>]>;
 def SDTX86Fild      : SDTypeProfile<1, 1, [SDTCisFP<0>, SDTCisPtrTy<1>]>;
 def SDTX86Fist      : SDTypeProfile<0, 2, [SDTCisFP<0>, SDTCisPtrTy<1>]>;
 
-def SDTX86CwdStore  : SDTypeProfile<0, 1, [SDTCisPtrTy<0>]>;
+def SDTX86CwdAccess : SDTypeProfile<0, 1, [SDTCisPtrTy<0>]>;
 
 def X86fld          : SDNode<"X86ISD::FLD", SDTX86Fld,
                              [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def X86fst          : SDNode<"X86ISD::FST", SDTX86Fst,
                              [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 def X86fild         : SDNode<"X86ISD::FILD", SDTX86Fild,
                              [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def X86fist         : SDNode<"X86ISD::FIST", SDTX86Fist,
                              [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 def X86fp_to_mem : SDNode<"X86ISD::FP_TO_INT_IN_MEM", SDTX86Fst,
                           [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
-def X86fp_cwd_get16 : SDNode<"X86ISD::FNSTCW16m",          SDTX86CwdStore,
+def X86fp_cwd_get16 : SDNode<"X86ISD::FNSTCW16m", SDTX86CwdAccess,
                              [SDNPHasChain, SDNPMayStore, SDNPSideEffect,
                               SDNPMemOperand]>;
+def X86fp_cwd_set16 : SDNode<"X86ISD::FLDCW16m", SDTX86CwdAccess,
+                             [SDNPHasChain, SDNPMayLoad, SDNPSideEffect,
+                              SDNPMemOperand]>;
 
 def X86fstf32 : PatFrag<(ops node:$val, node:$ptr),
                         (X86fst node:$val, node:$ptr), [{
   return cast<MemIntrinsicSDNode>(N)->getMemoryVT() == MVT::f32;
 }]>;
 def X86fstf64 : PatFrag<(ops node:$val, node:$ptr),
                         (X86fst node:$val, node:$ptr), [{
   return cast<MemIntrinsicSDNode>(N)->getMemoryVT() == MVT::f64;
def FNSTSW16r : I<0xDF, MRM_E0,                  // AX = fp flags
                   (outs), (ins), "fnstsw\t{%ax|ax}", []>;
 let Defs = [FPSW], Uses = [FPCW] in
 def FNSTCW16m : I<0xD9, MRM7m,                   // [mem16] = X87 control world
                   (outs), (ins i16mem:$dst), "fnstcw\t$dst",
                   [(X86fp_cwd_get16 addr:$dst)]>;
 } // SchedRW
 let Defs = [FPSW,FPCW], mayLoad = 1 in
 def FLDCW16m  : I<0xD9, MRM5m,                   // X87 control world = [mem16]
-                  (outs), (ins i16mem:$dst), "fldcw\t$dst", []>,
+                  (outs), (ins i16mem:$dst), "fldcw\t$dst",
+                  [(X86fp_cwd_set16 addr:$dst)]>,
                 Sched<[WriteLoad]>;
 
 // FPU control instructions
 let SchedRW = [WriteMicrocoded] in {
 def FFREE : FPI<0xDD, MRM0r, (outs), (ins RSTi:$reg), "ffree\t$reg">;
 def FFREEP : FPI<0xDF, MRM0r, (outs), (ins RSTi:$reg), "ffreep\t$reg">;
 
 let Defs = [FPSW, FPCW] in

llvm/test/CodeGen/X86/fpenv.ll

+; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=-sse -verify-machineinstrs < %s | FileCheck %s -check-prefixes=CHECK,X86-NOSSE
+; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+sse -verify-machineinstrs < %s | FileCheck %s -check-prefixes=CHECK,X86-SSE
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -verify-machineinstrs < %s | FileCheck %s -check-prefixes=CHECK,X64
+
+
+define i64 @func_01() {
+entry:
+  %v = call i64 @llvm.get.fpmode.i64()
+  ret i64 %v
+}
+; CHECK-LABEL: func_01:
+
+; X86-NOSSE:      fnstcw  (%esp)
+; X86-NOSSE-NEXT: movzwl  (%esp), %eax
+
+; X86-SSE:      fnstcw  (%esp)
+; X86-SSE-NEXT: movl    (%esp), %eax
+; X86-SSE-NEXT: stmxcsr 4(%esp)
+; X86-SSE-NEXT: movl    (%esp), %edx
+
+; X64:      fnstcw  -8(%rsp)
+; X64-NEXT: stmxcsr -4(%rsp)
+; X64-NEXT: movq    -8(%rsp), %rax
+; X64-NEXT: retq
+
+
+define void @func_02(i64 %fpe) {
+entry:
+  call void @llvm.set.fpmode.i64(i64 %fpe)
+  ret void
+}
+; CHECK-LABEL: func_02:
+
+; X86-NOSSE:      movl  8(%esp), %eax
+; X86-NOSSE-NEXT: movl  %eax, 2(%esp)
+; X86-NOSSE-NEXT: fldcw 2(%esp)
+
+; X86-SSE:      movl    %ecx, 4(%esp)
+; X86-SSE-NEXT: movl    %eax, (%esp)
+; X86-SSE-NEXT: fldcw   (%esp)
+; X86-SSE-NEXT: ldmxcsr 4(%esp)
+
+; X64:      movq    %rdi, -8(%rsp)
+; X64-NEXT: fldcw   -8(%rsp)
+; X64-NEXT: ldmxcsr -4(%rsp)
+; X64-NEXT: retq
+
+
+define void @func_03() {
+entry:
+  call void @llvm.reset.fpmode()
+  ret void
+}
+; CHECK-LABEL: func_03:
+
+; X86-NOSSE:      fldcw   .LCPI
+; X86-NOSSE-NEXT: retl
+
+; X86-SSE:      fldcw   .LCPI
+; X86-SSE-NEXT: ldmxcsr .LCPI
+; X86-SSE-NEXT: retl
+
+; X64:      fldcw   .LCPI{{.*}}(%rip)
+; X64-NEXT: ldmxcsr .LCPI{{.*}}(%rip)
+; X64-NEXT: retq
+
+
+declare i64 @llvm.get.fpmode.i64()
+declare void @llvm.set.fpmode.i64(i64 %fpenv)
+declare void @llvm.reset.fpmode()

llvm/test/CodeGen/X86/fpenv32.ll

+; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=-sse -verify-machineinstrs < %s | FileCheck %s
+
+
+define i32 @func_01() {
+entry:
+  %v = call i32 @llvm.get.fpmode.i32()
+  ret i32 %v
+}
+; CHECK-LABEL: func_01:
+; CHECK:       fnstcw  (%esp)
+; CHECK-NEXT:  movzwl  (%esp), %eax
+
+
+define void @func_02(i32 %fpe) {
+entry:
+  call void @llvm.set.fpmode.i32(i32 %fpe)
+  ret void
+}
+; CHECK-LABEL: func_02:
+; CHECK:       movl  8(%esp), %eax
+; CHECK-NEXT:  movl  %eax, 2(%esp)
+; CHECK-NEXT:  fldcw 2(%esp)
+
+
+declare i32 @llvm.get.fpmode.i32()
+declare void @llvm.set.fpmode.i32(i32 %fpenv)