[FPEnv][X86] Implement lowering of llvm.set.rounding

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

26994

Why do we need an APInt?

27063

Why do we need an APInt?

27077

Removed commented out code.

31336

Where did FNSTSW16r come from? I deleted that recently.

llvm/lib/Target/X86/X86ISelLowering.h

29

STMXCSR/LDMXCSR aren't used.

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

26978

I have no idea why FLT_ROUNDS calls TFI.getStackAlignment() to get the second parameter here. It should just be 4.

[sepavloff]

Modified patch according to reviewer's notes

[sepavloff]

llvm/lib/Target/X86/X86ISelLowering.cpp

26978

Changed here but implementation of LowerFLT_ROUNDS_ was not changed.

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

31336

FNSTSW16r was removed at D73863, but FNSTCW16m stayed

[RKSimon]

@sepavloff - what's happening with this?

[sepavloff]

Rebased patch

[sepavloff]

In D74730#2004240, @RKSimon wrote:

@sepavloff - what's happening with this?

It is still needed. This patch depends on D74729. Next steps, like support in frontend (as fesetround) and lowering on other platforms make sense only after destiny of these patches will be determined.

The motivation for this work is implementation of a pragma that would set/restore rounding mode without need to call fesetround.

[sepavloff]

Rebased patch

[RKSimon]

llvm/test/CodeGen/X86/fpenv.ll

5

regenerate with the update_llc_test_checks.py script?

[sepavloff]

Rebased and used update_llc_test_checks.py script

[sepavloff]

Clang-formatted the patch

[sepavloff]

Rebased patch

[sepavloff]

Rebased patch

[lebedev.ri]

llvm/lib/Target/X86/X86ISelLowering.cpp

27009–27010

Is this enforced elsewhere? (verifier, etc)
This doesn't seem like the right way do to error handling to me, we shouldn't crash.

llvm/lib/Target/X86/X86ISelLowering.h

855–856

I suspect this is backwards, and should say that glibc choose to use values compatible with the ones stored in FPSR.

[sepavloff]

Updated comment

[sepavloff]

llvm/lib/Target/X86/X86ISelLowering.cpp

27009–27010

It is not an error handling, it is an assertion check. This intrinsic is a low-level entity and is actually only a wrap over an instruction that writes to control register. It is responsibility of a caller to ensure that argument of this intrinsic is valid. A check in verifier is possible but is not much helpful, because the argument may be a runtime expression.

llvm/lib/Target/X86/X86ISelLowering.h

855–856

Updated the comment.

[sepavloff]

Ping.

[sepavloff]

Is there anything I can do to facilitate the review process?

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

26987

I think we cannot assume x87 is always available https://godbolt.org/z/4MbsarMcP. GCC supports nox87 feature, @LiuChen3 is doing the same thing in LLVM in D100091.
Furthermore, do we need to add a function call in case softfloat/libcall is used? Can we wisely generate one of them based on situation?

[sepavloff]

Lower set_rounding only if soft-float is off

[sepavloff]

llvm/lib/Target/X86/X86ISelLowering.cpp

26987

I think we cannot assume x87 is always available https://godbolt.org/z/4MbsarMcP. GCC supports nox87 feature, @LiuChen3 is doing the same thing in LLVM in D100091.

Is there an x86 core that does not support x87 instructions? If no, then -mno-80387, similar to -msoft-float and -mgeneral-regs-only are proposed for the cases like Linux kernel, where access to FP hardware is undesirable because the modification of FP state requires saving/restoring it in process context. As set_rounding by essence modifies FP state, -mno-80387 is identical to -msoft-float. By the way, GCC documentation list them as synonyms (https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html).

Furthermore, do we need to add a function call in case softfloat/libcall is used? Can we wisely generate one of them based on situation?

I added checks for soft-float in constructor of X86TargetLowering. If this option is effect, this function is not used for lowering of set_rounding.

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

26987

I think Core and most Atom support x87 instructions. Someone mentioned there're some targets for IoT don't. https://retrocomputing.stackexchange.com/a/9662

[sepavloff]

Added check for Subtarget.hasX87() to support -mno-80387

[sepavloff]

llvm/lib/Target/X86/X86ISelLowering.cpp

26987

Interesting reference, thank you.

It is still true that if hardware supports FP operations, it supports x87 also. There is no a processor that supports FP in hardware but do not support x87. So we do not need to check for x87 presence here, it should be enough to enable or not custom lowering in the constructor of X86TargetLowering. I added the check for Subtarget.hasX87() to support -mno-80387 also.

[pengfei]

This patch LGTM, but I'd like other sign off.

llvm/lib/Target/X86/X86ISelLowering.cpp

26987

Makes sense to me, thanks.

[sepavloff]

In D74730#2708712, @pengfei wrote:

This patch LGTM, but I'd like other sign off.

@pengfei Thank you!

Could anybody sign off this patch?

[sepavloff]

Rebased

[RKSimon]

I have no objections to this going in - @pengfei @craig.topper @andrew.w.kaylor ?

[RKSimon]

LGTM - I don't see any reason to delay this any longer

llvm/lib/Target/X86/X86ISelLowering.h

 //===-- X86ISelLowering.h - X86 DAG Lowering Interface ----------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines the interfaces that X86 uses to lower LLVM code into a
namespace llvm {
   class X86TargetMachine;
 
   namespace X86ISD {
     // X86 Specific DAG Nodes
   enum NodeType : unsigned {
     // Start the numbering where the builtin ops leave off.
     FIRST_NUMBER = ISD::BUILTIN_OP_END,
 
     /// Bit scan forward.
     BSF,
     /// Bit scan reverse.
     BSR,
 
     /// X86 funnel/double shift i16 instructions. These correspond to
     /// X86::SHLDW and X86::SHRDW instructions which have different amt
     /// modulo rules to generic funnel shifts.
     /// NOTE: The operand order matches ISD::FSHL/FSHR not SHLD/SHRD.
enum NodeType : unsigned {
     VEXTRACT_STORE,
 
     // scalar broadcast from memory.
     VBROADCAST_LOAD,
 
     // subvector broadcast from memory.
     SUBV_BROADCAST_LOAD,
 
-    // Store FP control world into i16 memory.
+    // Store FP control word into i16 memory.
     FNSTCW16m,
 
+    // Load FP control word from i16 memory.
+    FLDCW16m,
+
     /// This instruction implements FP_TO_SINT with the
     /// integer destination in memory and a FP reg source.  This corresponds
     /// to the X86::FIST*m instructions and the rounding mode change stuff. It
     /// has two inputs (token chain and address) and two outputs (int value
     /// and token chain). Memory VT specifies the type to store to.
     FP_TO_INT_IN_MEM,
 
     /// This instruction implements SINT_TO_FP with the
enum NodeType : unsigned {
     AESDECWIDE256KL,
 
     // WARNING: Do not add anything in the end unless you want the node to
     // have memop! In fact, starting from FIRST_TARGET_MEMORY_OPCODE all
     // opcodes will be thought as target memory ops!
   };
   } // end namespace X86ISD
 
+  namespace X86 {
+    /// Current rounding mode is represented in bits 11:10 of FPSR. These
+    /// values are same as corresponding constants for rounding mode used
+    /// in glibc.
+    enum RoundingMode {
+      rmToNearest   = 0,        // FE_TONEAREST
+      rmDownward    = 1 << 10,  // FE_DOWNWARD
+      rmUpward      = 2 << 10,  // FE_UPWARD
+      rmTowardZero  = 3 << 10,  // FE_TOWARDZERO
+      rmMask        = 3 << 10   // Bit mask selecting rounding mode
+    };
+  }
+
   /// Define some predicates that are used for node matching.
   namespace X86 {
     /// Returns true if Elt is a constant zero or floating point constant +0.0.
     bool isZeroNode(SDValue Elt);
 
     /// Returns true of the given offset can be
     /// fit into displacement field of the instruction.
     bool isOffsetSuitableForCodeModel(int64_t Offset, CodeModel::Model M,
private:
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAME_TO_ARGS_OFFSET(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerEH_RETURN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_LONGJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETUP_DISPATCH(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINIT_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerSET_ROUNDING(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerWin64_i128OP(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerGC_TRANSITION(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerFaddFsub(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFP_EXTEND(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFP_ROUND(SDValue Op, SelectionDAG &DAG) const;
 
     SDValue

llvm/lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::X86TargetLowering(const X86TargetMachine &TM,
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i16  , Legal);
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i8   , Legal);
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1   , Expand);
 
   setOperationAction(ISD::FREM             , MVT::f32  , Expand);
   setOperationAction(ISD::FREM             , MVT::f64  , Expand);
   setOperationAction(ISD::FREM             , MVT::f80  , Expand);
   setOperationAction(ISD::FREM             , MVT::f128 , Expand);
-  setOperationAction(ISD::FLT_ROUNDS_      , MVT::i32  , Custom);
+
+  if (!Subtarget.useSoftFloat() && Subtarget.hasX87()) {
+    setOperationAction(ISD::FLT_ROUNDS_    , MVT::i32  , Custom);
+    setOperationAction(ISD::SET_ROUNDING   , MVT::Other, Custom);
+  }
 
   // Promote the i8 variants and force them on up to i32 which has a shorter
   // encoding.
   setOperationPromotedToType(ISD::CTTZ           , MVT::i8   , MVT::i32);
   setOperationPromotedToType(ISD::CTTZ_ZERO_UNDEF, MVT::i8   , MVT::i32);
   if (!Subtarget.hasBMI()) {
     setOperationAction(ISD::CTTZ           , MVT::i16  , Custom);
     setOperationAction(ISD::CTTZ           , MVT::i32  , Custom);
DAG.getNode(ISD::AND, DL, MVT::i32,
                 DAG.getNode(ISD::SRL, DL, MVT::i32, LUT, Shift),
                 DAG.getConstant(3, DL, MVT::i32));
 
   RetVal = DAG.getZExtOrTrunc(RetVal, DL, VT);
 
   return DAG.getMergeValues({RetVal, Chain}, DL);
 }
 
+SDValue X86TargetLowering::LowerSET_ROUNDING(SDValue Op,
+                                             SelectionDAG &DAG) const {
+  MachineFunction &MF = DAG.getMachineFunction();
+  SDLoc DL(Op);
+  SDValue Chain = Op.getNode()->getOperand(0);
+
+  // FP control word may be set only from data in memory. So we need to allocate
+  // stack space to save/load FP control word.
+  int OldCWFrameIdx = MF.getFrameInfo().CreateStackObject(4, Align(4), false);
+  SDValue StackSlot =
+      DAG.getFrameIndex(OldCWFrameIdx, getPointerTy(DAG.getDataLayout()));
+  MachinePointerInfo MPI = MachinePointerInfo::getFixedStack(MF, OldCWFrameIdx);
+  MachineMemOperand *MMO =
+      MF.getMachineMemOperand(MPI, MachineMemOperand::MOStore, 2, Align(2));
+
+  // Store FP control word into memory.
+  SDValue Ops[] = {Chain, StackSlot};
+  Chain = DAG.getMemIntrinsicNode(
+      X86ISD::FNSTCW16m, DL, DAG.getVTList(MVT::Other), Ops, MVT::i16, MMO);
+
+  // Load FP Control Word from stack slot and clear RM field (bits 11:10).
+  SDValue CWD = DAG.getLoad(MVT::i16, DL, Chain, StackSlot, MPI);
+  Chain = CWD.getValue(1);
+  CWD = DAG.getNode(ISD::AND, DL, MVT::i16, CWD.getValue(0),
+                    DAG.getConstant(0xf3ff, DL, MVT::i16));
+
+  // Calculate new rounding mode.
+  SDValue NewRM = Op.getNode()->getOperand(1);
+  SDValue RMBits;
+  if (auto *CVal = dyn_cast<ConstantSDNode>(NewRM)) {
+    uint64_t RM = CVal->getZExtValue();
+    int FieldVal;
+    switch (static_cast<RoundingMode>(RM)) {
+    case RoundingMode::NearestTiesToEven: FieldVal = X86::rmToNearest; break;
+    case RoundingMode::TowardNegative:    FieldVal = X86::rmDownward; break;
+    case RoundingMode::TowardPositive:    FieldVal = X86::rmUpward; break;
+    case RoundingMode::TowardZero:        FieldVal = X86::rmTowardZero; break;
+    default:
+      llvm_unreachable("rounding mode is not supported by X86 hardware");
+    }
+    RMBits = DAG.getConstant(FieldVal, DL, MVT::i16);
+  } else {
+    // Need to convert argument into bits of control word:
+    //    0 Round to 0       -> 11
+    //    1 Round to nearest -> 00
+    //    2 Round to +inf    -> 10
+    //    3 Round to -inf    -> 01
+    // The 2-bit value needs then to be shifted so that it occupies bits 11:10.
+    // To make the conversion, put all these values into a value 0xc9 and shift
+    // it left depending on the rounding mode:
+    //    (0xc9 << 4) & 0xc00 = X86::rmTowardZero
+    //    (0xc9 << 6) & 0xc00 = X86::rmToNearest
+    //    ...
+    // (0xc9 << (2 * NewRM + 4)) & 0xc00
+    SDValue ShiftValue =
+        DAG.getNode(ISD::TRUNCATE, DL, MVT::i8,
+                    DAG.getNode(ISD::ADD, DL, MVT::i32,
+                                DAG.getNode(ISD::SHL, DL, MVT::i32, NewRM,
+                                            DAG.getConstant(1, DL, MVT::i8)),
+                                DAG.getConstant(4, DL, MVT::i32)));
+    SDValue Shifted =
+        DAG.getNode(ISD::SHL, DL, MVT::i16, DAG.getConstant(0xc9, DL, MVT::i16),
+                    ShiftValue);
+    RMBits = DAG.getNode(ISD::AND, DL, MVT::i16, Shifted,
+                         DAG.getConstant(0xc00, DL, MVT::i16));
+  }
+
+  // Update rounding mode bits and store the new FP Control Word into stack.
+  CWD = DAG.getNode(ISD::OR, DL, MVT::i16, CWD, RMBits);
+  Chain = DAG.getStore(Chain, DL, CWD, StackSlot, MPI, /* Alignment = */ 2);
+
+  // Load FP control word from the slot.
+  SDValue OpsLD[] = {Chain, StackSlot};
+  MachineMemOperand *MMOL =
+      MF.getMachineMemOperand(MPI, MachineMemOperand::MOLoad, 2, Align(2));
+  Chain = DAG.getMemIntrinsicNode(
+      X86ISD::FLDCW16m, DL, DAG.getVTList(MVT::Other), OpsLD, MVT::i16, MMOL);
+
+  // If target supports SSE, set MXCSR as well. Rounding mode is encoded in the
+  // same way but in bits 14:13.
+  if (Subtarget.hasSSE1()) {
+    // Store MXCSR into memory.
+    Chain = DAG.getNode(
+        ISD::INTRINSIC_VOID, DL, DAG.getVTList(MVT::Other), Chain,
+        DAG.getTargetConstant(Intrinsic::x86_sse_stmxcsr, DL, MVT::i32),
+        StackSlot);
+
+    // Load MXCSR from stack slot and clear RM field (bits 14:13).
+    SDValue CWD = DAG.getLoad(MVT::i32, DL, Chain, StackSlot, MPI);
+    Chain = CWD.getValue(1);
+    CWD = DAG.getNode(ISD::AND, DL, MVT::i32, CWD.getValue(0),
+                      DAG.getConstant(0xffff9fff, DL, MVT::i32));
+
+    // Shift X87 RM bits from 11:10 to 14:13.
+    RMBits = DAG.getNode(ISD::ZERO_EXTEND, DL, MVT::i32, RMBits);
+    RMBits = DAG.getNode(ISD::SHL, DL, MVT::i32, RMBits,
+                         DAG.getConstant(3, DL, MVT::i8));
+
+    // Update rounding mode bits and store the new FP Control Word into stack.
+    CWD = DAG.getNode(ISD::OR, DL, MVT::i32, CWD, RMBits);
+    Chain = DAG.getStore(Chain, DL, CWD, StackSlot, MPI, /* Alignment = */ 4);
+
+    // Load MXCSR from the slot.
+    Chain = DAG.getNode(
+        ISD::INTRINSIC_VOID, DL, DAG.getVTList(MVT::Other), Chain,
+        DAG.getTargetConstant(Intrinsic::x86_sse_ldmxcsr, DL, MVT::i32),
+        StackSlot);
+  }
+
+  return Chain;
+}
+
 /// Lower a vector CTLZ using native supported vector CTLZ instruction.
 //
 // i8/i16 vector implemented using dword LZCNT vector instruction
 // ( sub(trunc(lzcnt(zext32(x)))) ). In case zext32(x) is illegal,
 // split the vector, perform operation on it's Lo a Hi part and
 // concatenate the results.
 static SDValue LowerVectorCTLZ_AVX512CDI(SDValue Op, SelectionDAG &DAG,
                                          const X86Subtarget &Subtarget) {
SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::EH_RETURN:          return LowerEH_RETURN(Op, DAG);
   case ISD::EH_SJLJ_SETJMP:     return lowerEH_SJLJ_SETJMP(Op, DAG);
   case ISD::EH_SJLJ_LONGJMP:    return lowerEH_SJLJ_LONGJMP(Op, DAG);
   case ISD::EH_SJLJ_SETUP_DISPATCH:
     return lowerEH_SJLJ_SETUP_DISPATCH(Op, DAG);
   case ISD::INIT_TRAMPOLINE:    return LowerINIT_TRAMPOLINE(Op, DAG);
   case ISD::ADJUST_TRAMPOLINE:  return LowerADJUST_TRAMPOLINE(Op, DAG);
   case ISD::FLT_ROUNDS_:        return LowerFLT_ROUNDS_(Op, DAG);
+  case ISD::SET_ROUNDING:       return LowerSET_ROUNDING(Op, DAG);
   case ISD::CTLZ:
   case ISD::CTLZ_ZERO_UNDEF:    return LowerCTLZ(Op, Subtarget, DAG);
   case ISD::CTTZ:
   case ISD::CTTZ_ZERO_UNDEF:    return LowerCTTZ(Op, Subtarget, DAG);
   case ISD::MUL:                return LowerMUL(Op, Subtarget, DAG);
   case ISD::MULHS:
   case ISD::MULHU:              return LowerMULH(Op, Subtarget, DAG);
   case ISD::ROTL:
#define NODE_NAME_CASE(NODE) case X86ISD::NODE: return "X86ISD::" #NODE;
   NODE_NAME_CASE(TLSBASEADDR)
   NODE_NAME_CASE(TLSCALL)
   NODE_NAME_CASE(EH_SJLJ_SETJMP)
   NODE_NAME_CASE(EH_SJLJ_LONGJMP)
   NODE_NAME_CASE(EH_SJLJ_SETUP_DISPATCH)
   NODE_NAME_CASE(EH_RETURN)
   NODE_NAME_CASE(TC_RETURN)
   NODE_NAME_CASE(FNSTCW16m)
+  NODE_NAME_CASE(FLDCW16m)
   NODE_NAME_CASE(LCMPXCHG_DAG)
   NODE_NAME_CASE(LCMPXCHG8_DAG)
   NODE_NAME_CASE(LCMPXCHG16_DAG)
   NODE_NAME_CASE(LCMPXCHG16_SAVE_RBX_DAG)
   NODE_NAME_CASE(LADD)
   NODE_NAME_CASE(LSUB)
   NODE_NAME_CASE(LOR)
   NODE_NAME_CASE(LXOR)
   NODE_NAME_CASE(LAND)

llvm/lib/Target/X86/X86InstrFPStack.td

 def SDTX86Fld       : SDTypeProfile<1, 1, [SDTCisFP<0>,
                                            SDTCisPtrTy<1>]>;
 def SDTX86Fst       : SDTypeProfile<0, 2, [SDTCisFP<0>,
                                            SDTCisPtrTy<1>]>;
 def SDTX86Fild      : SDTypeProfile<1, 1, [SDTCisFP<0>, SDTCisPtrTy<1>]>;
 def SDTX86Fist      : SDTypeProfile<0, 2, [SDTCisFP<0>, SDTCisPtrTy<1>]>;
 
 def SDTX86CwdStore  : SDTypeProfile<0, 1, [SDTCisPtrTy<0>]>;
+def SDTX86CwdLoad   : SDTypeProfile<0, 1, [SDTCisPtrTy<0>]>;
 
 def X86fld          : SDNode<"X86ISD::FLD", SDTX86Fld,
                              [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def X86fst          : SDNode<"X86ISD::FST", SDTX86Fst,
                              [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 def X86fild         : SDNode<"X86ISD::FILD", SDTX86Fild,
                              [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def X86fist         : SDNode<"X86ISD::FIST", SDTX86Fist,
                              [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 def X86fp_to_mem : SDNode<"X86ISD::FP_TO_INT_IN_MEM", SDTX86Fst,
                           [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 def X86fp_cwd_get16 : SDNode<"X86ISD::FNSTCW16m",          SDTX86CwdStore,
                              [SDNPHasChain, SDNPMayStore, SDNPSideEffect,
                               SDNPMemOperand]>;
+def X86fp_cwd_set16 : SDNode<"X86ISD::FLDCW16m",           SDTX86CwdLoad,
+                             [SDNPHasChain, SDNPMayLoad, SDNPSideEffect,
+                              SDNPMemOperand]>;
 
 def X86fstf32 : PatFrag<(ops node:$val, node:$ptr),
                         (X86fst node:$val, node:$ptr), [{
   return cast<MemIntrinsicSDNode>(N)->getMemoryVT() == MVT::f32;
 }]>;
 def X86fstf64 : PatFrag<(ops node:$val, node:$ptr),
                         (X86fst node:$val, node:$ptr), [{
   return cast<MemIntrinsicSDNode>(N)->getMemoryVT() == MVT::f64;
def FNSTSW16r : I<0xDF, MRM_E0,                  // AX = fp flags
                   (outs), (ins), "fnstsw\t{%ax|ax}", []>;
 let Defs = [FPSW], Uses = [FPCW] in
 def FNSTCW16m : I<0xD9, MRM7m,                   // [mem16] = X87 control world
                   (outs), (ins i16mem:$dst), "fnstcw\t$dst",
                   [(X86fp_cwd_get16 addr:$dst)]>;
 } // SchedRW
 let Defs = [FPSW,FPCW], mayLoad = 1 in
 def FLDCW16m  : I<0xD9, MRM5m,                   // X87 control world = [mem16]
-                  (outs), (ins i16mem:$dst), "fldcw\t$dst", []>,
+                  (outs), (ins i16mem:$dst), "fldcw\t$dst",
+                  [(X86fp_cwd_set16 addr:$dst)]>,
                 Sched<[WriteLoad]>;
 
 // FPU control instructions
 let SchedRW = [WriteMicrocoded] in {
 def FFREE : FPI<0xDD, MRM0r, (outs), (ins RSTi:$reg), "ffree\t$reg">;
 def FFREEP : FPI<0xDF, MRM0r, (outs), (ins RSTi:$reg), "ffreep\t$reg">;
 
 let Defs = [FPSW, FPCW] in

llvm/test/CodeGen/X86/fpenv.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=-sse -verify-machineinstrs < %s | FileCheck %s -check-prefix=X86-NOSSE
+; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+sse -verify-machineinstrs < %s | FileCheck %s -check-prefix=X86-SSE
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -verify-machineinstrs < %s | FileCheck %s -check-prefix=X64
+
+declare void @llvm.set.rounding(i32 %x)
+
+define void @func_01() nounwind {
+; X86-NOSSE-LABEL: func_01:
+; X86-NOSSE:       # %bb.0:
+; X86-NOSSE-NEXT:    pushl %eax
+; X86-NOSSE-NEXT:    fnstcw (%esp)
+; X86-NOSSE-NEXT:    orb $12, {{[0-9]+}}(%esp)
+; X86-NOSSE-NEXT:    fldcw (%esp)
+; X86-NOSSE-NEXT:    popl %eax
+; X86-NOSSE-NEXT:    retl
+;
+; X86-SSE-LABEL: func_01:
+; X86-SSE:       # %bb.0:
+; X86-SSE-NEXT:    pushl %eax
+; X86-SSE-NEXT:    fnstcw (%esp)
+; X86-SSE-NEXT:    orb $12, {{[0-9]+}}(%esp)
+; X86-SSE-NEXT:    fldcw (%esp)
+; X86-SSE-NEXT:    stmxcsr (%esp)
+; X86-SSE-NEXT:    orb $96, {{[0-9]+}}(%esp)
+; X86-SSE-NEXT:    ldmxcsr (%esp)
+; X86-SSE-NEXT:    popl %eax
+; X86-SSE-NEXT:    retl
+;
+; X64-LABEL: func_01:
+; X64:       # %bb.0:
+; X64-NEXT:    fnstcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    orb $12, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    fldcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    stmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    orb $96, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    ldmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    retq
+  call void @llvm.set.rounding(i32 0)  ; TowardZero (CW[11-10] = 11)
+  ret void
+}
+
+define void @func_02() nounwind {
+; X86-NOSSE-LABEL: func_02:
+; X86-NOSSE:       # %bb.0:
+; X86-NOSSE-NEXT:    pushl %eax
+; X86-NOSSE-NEXT:    fnstcw (%esp)
+; X86-NOSSE-NEXT:    andb $-13, {{[0-9]+}}(%esp)
+; X86-NOSSE-NEXT:    fldcw (%esp)
+; X86-NOSSE-NEXT:    popl %eax
+; X86-NOSSE-NEXT:    retl
+;
+; X86-SSE-LABEL: func_02:
+; X86-SSE:       # %bb.0:
+; X86-SSE-NEXT:    pushl %eax
+; X86-SSE-NEXT:    fnstcw (%esp)
+; X86-SSE-NEXT:    andb $-13, {{[0-9]+}}(%esp)
+; X86-SSE-NEXT:    fldcw (%esp)
+; X86-SSE-NEXT:    stmxcsr (%esp)
+; X86-SSE-NEXT:    andb $-97, {{[0-9]+}}(%esp)
+; X86-SSE-NEXT:    ldmxcsr (%esp)
+; X86-SSE-NEXT:    popl %eax
+; X86-SSE-NEXT:    retl
+;
+; X64-LABEL: func_02:
+; X64:       # %bb.0:
+; X64-NEXT:    fnstcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    andb $-13, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    fldcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    stmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    andb $-97, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    ldmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    retq
+  call void @llvm.set.rounding(i32 1)  ; ToNearestTiesToEven (CW[11-10] = 00)
+  ret void
+}
+
+define void @func_03() nounwind {
+; X86-NOSSE-LABEL: func_03:
+; X86-NOSSE:       # %bb.0:
+; X86-NOSSE-NEXT:    pushl %eax
+; X86-NOSSE-NEXT:    fnstcw (%esp)
+; X86-NOSSE-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X86-NOSSE-NEXT:    andl (%esp), %eax
+; X86-NOSSE-NEXT:    orl $2048, %eax # imm = 0x800
+; X86-NOSSE-NEXT:    movw %ax, (%esp)
+; X86-NOSSE-NEXT:    fldcw (%esp)
+; X86-NOSSE-NEXT:    popl %eax
+; X86-NOSSE-NEXT:    retl
+;
+; X86-SSE-LABEL: func_03:
+; X86-SSE:       # %bb.0:
+; X86-SSE-NEXT:    pushl %eax
+; X86-SSE-NEXT:    fnstcw (%esp)
+; X86-SSE-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X86-SSE-NEXT:    andl (%esp), %eax
+; X86-SSE-NEXT:    orl $2048, %eax # imm = 0x800
+; X86-SSE-NEXT:    movw %ax, (%esp)
+; X86-SSE-NEXT:    fldcw (%esp)
+; X86-SSE-NEXT:    stmxcsr (%esp)
+; X86-SSE-NEXT:    movl $-24577, %eax # imm = 0x9FFF
+; X86-SSE-NEXT:    andl (%esp), %eax
+; X86-SSE-NEXT:    orl $16384, %eax # imm = 0x4000
+; X86-SSE-NEXT:    movl %eax, (%esp)
+; X86-SSE-NEXT:    ldmxcsr (%esp)
+; X86-SSE-NEXT:    popl %eax
+; X86-SSE-NEXT:    retl
+;
+; X64-LABEL: func_03:
+; X64:       # %bb.0:
+; X64-NEXT:    fnstcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %eax
+; X64-NEXT:    orl $2048, %eax # imm = 0x800
+; X64-NEXT:    movw %ax, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    fldcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    stmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-24577, %eax # imm = 0x9FFF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %eax
+; X64-NEXT:    orl $16384, %eax # imm = 0x4000
+; X64-NEXT:    movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    ldmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    retq
+  call void @llvm.set.rounding(i32 2)  ; Upward (CW[11-10] = 10)
+  ret void
+}
+
+define void @func_04() nounwind {
+; X86-NOSSE-LABEL: func_04:
+; X86-NOSSE:       # %bb.0:
+; X86-NOSSE-NEXT:    pushl %eax
+; X86-NOSSE-NEXT:    fnstcw (%esp)
+; X86-NOSSE-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X86-NOSSE-NEXT:    andl (%esp), %eax
+; X86-NOSSE-NEXT:    orl $1024, %eax # imm = 0x400
+; X86-NOSSE-NEXT:    movw %ax, (%esp)
+; X86-NOSSE-NEXT:    fldcw (%esp)
+; X86-NOSSE-NEXT:    popl %eax
+; X86-NOSSE-NEXT:    retl
+;
+; X86-SSE-LABEL: func_04:
+; X86-SSE:       # %bb.0:
+; X86-SSE-NEXT:    pushl %eax
+; X86-SSE-NEXT:    fnstcw (%esp)
+; X86-SSE-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X86-SSE-NEXT:    andl (%esp), %eax
+; X86-SSE-NEXT:    orl $1024, %eax # imm = 0x400
+; X86-SSE-NEXT:    movw %ax, (%esp)
+; X86-SSE-NEXT:    fldcw (%esp)
+; X86-SSE-NEXT:    stmxcsr (%esp)
+; X86-SSE-NEXT:    movl $-24577, %eax # imm = 0x9FFF
+; X86-SSE-NEXT:    andl (%esp), %eax
+; X86-SSE-NEXT:    orl $8192, %eax # imm = 0x2000
+; X86-SSE-NEXT:    movl %eax, (%esp)
+; X86-SSE-NEXT:    ldmxcsr (%esp)
+; X86-SSE-NEXT:    popl %eax
+; X86-SSE-NEXT:    retl
+;
+; X64-LABEL: func_04:
+; X64:       # %bb.0:
+; X64-NEXT:    fnstcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-3073, %eax # imm = 0xF3FF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %eax
+; X64-NEXT:    orl $1024, %eax # imm = 0x400
+; X64-NEXT:    movw %ax, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    fldcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    stmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-24577, %eax # imm = 0x9FFF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %eax
+; X64-NEXT:    orl $8192, %eax # imm = 0x2000
+; X64-NEXT:    movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    ldmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    retq
+  call void @llvm.set.rounding(i32 3)  ; Downward (CW[11-10] = 01)
+  ret void
+}
+
+define void @func_05(i32 %x) nounwind {
+; X86-NOSSE-LABEL: func_05:
+; X86-NOSSE:       # %bb.0:
+; X86-NOSSE-NEXT:    pushl %eax
+; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NOSSE-NEXT:    leal 4(%eax,%eax), %ecx
+; X86-NOSSE-NEXT:    movl $201, %eax
+; X86-NOSSE-NEXT:    # kill: def $cl killed $cl killed $ecx
+; X86-NOSSE-NEXT:    shll %cl, %eax
+; X86-NOSSE-NEXT:    andl $3072, %eax # imm = 0xC00
+; X86-NOSSE-NEXT:    fnstcw (%esp)
+; X86-NOSSE-NEXT:    movl $-3073, %ecx # imm = 0xF3FF
+; X86-NOSSE-NEXT:    andl (%esp), %ecx
+; X86-NOSSE-NEXT:    orl %eax, %ecx
+; X86-NOSSE-NEXT:    movw %cx, (%esp)
+; X86-NOSSE-NEXT:    fldcw (%esp)
+; X86-NOSSE-NEXT:    popl %eax
+; X86-NOSSE-NEXT:    retl
+;
+; X86-SSE-LABEL: func_05:
+; X86-SSE:       # %bb.0:
+; X86-SSE-NEXT:    pushl %eax
+; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-SSE-NEXT:    leal 4(%eax,%eax), %ecx
+; X86-SSE-NEXT:    movl $201, %eax
+; X86-SSE-NEXT:    # kill: def $cl killed $cl killed $ecx
+; X86-SSE-NEXT:    shll %cl, %eax
+; X86-SSE-NEXT:    andl $3072, %eax # imm = 0xC00
+; X86-SSE-NEXT:    fnstcw (%esp)
+; X86-SSE-NEXT:    movl $-3073, %ecx # imm = 0xF3FF
+; X86-SSE-NEXT:    andl (%esp), %ecx
+; X86-SSE-NEXT:    orl %eax, %ecx
+; X86-SSE-NEXT:    movw %cx, (%esp)
+; X86-SSE-NEXT:    fldcw (%esp)
+; X86-SSE-NEXT:    stmxcsr (%esp)
+; X86-SSE-NEXT:    movl $-24577, %ecx # imm = 0x9FFF
+; X86-SSE-NEXT:    andl (%esp), %ecx
+; X86-SSE-NEXT:    leal (%ecx,%eax,8), %eax
+; X86-SSE-NEXT:    movl %eax, (%esp)
+; X86-SSE-NEXT:    ldmxcsr (%esp)
+; X86-SSE-NEXT:    popl %eax
+; X86-SSE-NEXT:    retl
+;
+; X64-LABEL: func_05:
+; X64:       # %bb.0:
+; X64-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-NEXT:    leal 4(%rdi,%rdi), %ecx
+; X64-NEXT:    movl $201, %eax
+; X64-NEXT:    # kill: def $cl killed $cl killed $ecx
+; X64-NEXT:    shll %cl, %eax
+; X64-NEXT:    andl $3072, %eax # imm = 0xC00
+; X64-NEXT:    fnstcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-3073, %ecx # imm = 0xF3FF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %ecx
+; X64-NEXT:    orl %eax, %ecx
+; X64-NEXT:    movw %cx, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    fldcw -{{[0-9]+}}(%rsp)
+; X64-NEXT:    stmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    movl $-24577, %ecx # imm = 0x9FFF
+; X64-NEXT:    andl -{{[0-9]+}}(%rsp), %ecx
+; X64-NEXT:    leal (%rcx,%rax,8), %eax
+; X64-NEXT:    movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NEXT:    ldmxcsr -{{[0-9]+}}(%rsp)
+; X64-NEXT:    retq
+  call void @llvm.set.rounding(i32 %x)  ; Downward
+  ret void
+}