[CUDA] Add section to docs about controlling fp optimizations.

[jlebar]

Clarify that -fp-contract='fast' default is just for CUDA device code.

[tra]

docs/CompileCudaWithLLVM.rst

160

It's actually for both host and device. The idea was to keep host & device behavior as close to each other as we can.

[jlebar]

Update per tra's comment that fp-contract=fast is set on both the host and device.

[jlebar]

Remove unrelated changes accidentally included in last revision.

[jlebar]

Thanks, Art!

[rnk]

lgtm

docs/CompileCudaWithLLVM.rst

166–168

Do we actually know how to honor the C11 semantics?

[jlebar]

docs/CompileCudaWithLLVM.rst

166–168

My understanding from talking to Chandler was that this was the intent of "on". But there is a good chance I misunderstood.

docs/CompileCudaWithLLVM.rst

.. code-block:: c++
 
   #if defined(__clang__) && defined(__CUDA__) && defined(__CUDA_ARCH__)
     // clang compiling CUDA code, device mode.
   #endif
 
 Both clang and nvcc define ``__CUDACC__`` during CUDA compilation.  You can
 detect NVCC specifically by looking for ``__NVCC__``.
 
+Flags that control numerical code
+=================================
+
+If you're using GPUs, you probably care about making numerical code run fast.
+GPU hardware allows for more control over numerical operations than most CPUs,
+but this results in more compiler options for you to juggle.
+
+Flags you may wish to tweak include:
+
+* ``-ffp-contract={on,off,fast}`` (defaults to ``fast`` on host and device when
+  compiling CUDA) Controls whether the compiler emits fused multiply-add
+  operations.
+
+  * ``off``: never emit fma operations, and prevent ptxas from fusing multiply
+    and add instructions.
+  * ``on``: fuse multiplies and adds within a single statement, but never
+    across statements (C11 semantics).  Prevent ptxas from fusing other
+    multiplies and adds.
+  * ``fast``: fuse multiplies and adds wherever profitable, even across
+    statements.  Doesn't prevent ptxas from fusing additional multiplies and
+    adds.
+
+  Fused multiply-add instructions can be much faster than the unfused
+  equivalents, but because the intermediate result in an fma is not rounded,
+  this flag can affect numerical code.
+
+* ``-fcuda-flush-denormals-to-zero`` (default: off) When this is enabled,
+  floating point operations may flush `denormal
+  <https://en.wikipedia.org/wiki/Denormal_number>`_ inputs and/or outputs to 0.
+  Operations on denormal numbers are often much slower than the same operations
+  on normal numbers.
+
+* ``-fcuda-approx-transcendentals`` (default: off) When this is enabled, the
+  compiler may emit calls to faster, approximate versions of transcendental
+  functions, instead of using the slower, fully IEEE-compliant versions.  For
+  example, this flag allows clang to emit the ptx ``sin.approx.f32``
+  instruction.
+
+  This is implied by ``-ffast-math``.
+
 Optimizations
 =============
 
 CPU and GPU have different design philosophies and architectures. For example, a
 typical CPU has branch prediction, out-of-order execution, and is superscalar,
 whereas a typical GPU has none of these. Due to such differences, an
 optimization pipeline well-tuned for CPUs may be not suitable for GPUs.
 

include/llvm/IR/IntrinsicsNVVM.td

// Accessing special registers
   def int_nvvm_read_ptx_sreg_warpsize :
       Intrinsic<[llvm_i32_ty], [], [IntrNoMem]>,
       GCCBuiltin<"__nvvm_read_ptx_sreg_warpsize">;
 
 
 // Generated within nvvm. Use for ldu on sm_20 or later.  Second arg is the
 // pointer's alignment.
 def int_nvvm_ldu_global_i : Intrinsic<[llvm_anyint_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldu.global.i">;
 def int_nvvm_ldu_global_f : Intrinsic<[llvm_anyfloat_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldu.global.f">;
 def int_nvvm_ldu_global_p : Intrinsic<[llvm_anyptr_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldu.global.p">;
 
 // Generated within nvvm. Use for ldg on sm_35 or later.  Second arg is the
 // pointer's alignment.
 def int_nvvm_ldg_global_i : Intrinsic<[llvm_anyint_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldg.global.i">;
 def int_nvvm_ldg_global_f : Intrinsic<[llvm_anyfloat_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldg.global.f">;
 def int_nvvm_ldg_global_p : Intrinsic<[llvm_anyptr_ty],
-  [LLVMAnyPointerType<LLVMMatchType<0>>, llvm_i32_ty],
+  [LLVMQualPointerType<LLVMMatchType<0>, /* addrspace = */ 1>, llvm_i32_ty],
   [IntrReadMem, IntrArgMemOnly, NoCapture<0>],
   "llvm.nvvm.ldg.global.p">;
 
 // Use for generic pointers
 // - These intrinsics are used to convert address spaces.
 // - The input pointer and output pointer must have the same type, except for
 //   the address-space. (This restriction is not enforced here as there is
 //   currently no way to describe it).

lib/CodeGen/MachineLICM.cpp

for (MachineBasicBlock *CurrentLoopExitingBlock : CurrentLoopExitingBlocks)
       }
   }
 
   SpeculationState = SpeculateFalse;
   return true;
 }
 
 void MachineLICM::EnterScope(MachineBasicBlock *MBB) {
-  DEBUG(dbgs() << "Entering: " << MBB->getName() << '\n');
+  DEBUG(dbgs() << "Entering BB#" << MBB->getNumber() << '\n');
 
   // Remember livein register pressure.
   BackTrace.push_back(RegPressure);
 }
 
 void MachineLICM::ExitScope(MachineBasicBlock *MBB) {
-  DEBUG(dbgs() << "Exiting: " << MBB->getName() << '\n');
+  DEBUG(dbgs() << "Exiting BB#" << MBB->getNumber() << '\n');
   BackTrace.pop_back();
 }
 
 /// Destroy scope for the MBB that corresponds to the given dominator tree node
 /// if its a leaf or all of its children are done. Walk up the dominator tree to
 /// destroy ancestors which are now done.
 void MachineLICM::ExitScopeIfDone(MachineDomTreeNode *Node,
                 DenseMap<MachineDomTreeNode*, unsigned> &OpenChildren,
if (!IsLoopInvariantInst(*MI) || !IsProfitableToHoist(*MI)) {
     MI = ExtractHoistableLoad(MI);
     if (!MI) return false;
   }
 
   // Now move the instructions to the predecessor, inserting it before any
   // terminator instructions.
   DEBUG({
       dbgs() << "Hoisting " << *MI;
-      if (Preheader->getBasicBlock())
-        dbgs() << " to MachineBasicBlock "
-               << Preheader->getName();
       if (MI->getParent()->getBasicBlock())
-        dbgs() << " from MachineBasicBlock "
-               << MI->getParent()->getName();
+        dbgs() << " from BB#" << MI->getParent()->getNumber();
+      if (Preheader->getBasicBlock())
+        dbgs() << " to BB#" << Preheader->getNumber();
       dbgs() << "\n";
     });
 
   // If this is the first instruction being hoisted to the preheader,
   // initialize the CSE map with potential common expressions.
   if (FirstInLoop) {
     InitCSEMap(Preheader);
     FirstInLoop = false;