[llvm][APFloat] Add NaN-in-negative-zero formats by AMD and GraphCore

[krzysz00]

Summary

AMD, GraphCore, and Qualcom have published a standard for 8-bit floats that
differs from the 8-bit floats defined by Nvidia, Intel, and ARM. This
commit adds support for these alternate 8-bit floats to APFloat in
order to enable their usage in MLIR. These formats are presented in
the paper at https://arxiv.org/abs/2206.02915 and are implemented in
GRaphCore hardware whose ISA is available at
https://docs.graphcore.ai/projects/isa-mk2-with-fp8/en/latest/_static/TileVertexISA-IPU21-1.3.1.pdf .

In these formats, like the existing Float8E4M3FN, there are no
infinity values and there is only one NaN. Unlike in that format,
however, the NaN values is 0x80, which would be negative 0 in IEEE
formats. This means that these formats also make 0 unsigned.

To allow for these new variant semantics, this commit adds
fltNanEncoding, which can be IEEE (the default), AllOnes (used by
Fleat8E4M3FN), or NegativeZero (used by the new formats,
Float8E5M2FNUZ and Float8E4M3FNUZ). Normalization, arithmetic, and
other such routines have been updated to account for the potential
variant semantics.

The two new formats are Float8E5M2FNUZ (5 bits exponent, 2 bits
mantissa, finite, unsigned zero) and Float8E4M3FNUZ (4 bits exponent,
3 bits mantissa, finite, unsigned zero).

[krzysz00]

Implements similar functionality to https://reviews.llvm.org/D141432 - needs reconciliation.

[krzysz00]

Copy over missed tests from GraphCore's diff

[jakeh-gc]

llvm/unittests/ADT/APFloatTest.cpp

1877

There are now 20 test cases, not 16.
12 wasn't a correct value for this before.

My preference is to use std::size here to avoid these issues in future.

[krzysz00]

Address bug in tests

[jakeh-gc]

Looks good 👍

[krzysz00]

Adding people who've recently touched APFloat as reviewers.

[krzysz00]

Add tests for getSmallestNormalized(), changeSign()

[mehdi_amini]

llvm/lib/Support/APFloat.cpp

76

Typo: point

77

typo: represented

80

typo: behavior

[lattner]

llvm/include/llvm/ADT/APFloat.h

161

Typo follwing

171

Typo follwing

[lattner]

Looks fine with the requested changes.

[reedwm]

Looks good, just have some minor comments.

llvm/include/llvm/ADT/APFloat.h

165

You should document that the exponent bias is 16, since it differs from the IEEE standard exponent bias of 2**(num_exponent_bits - 1) - 1. And same for S_Float8E4M3FNUZ.

llvm/lib/Support/APFloat.cpp

62

This comment should be updated to mention the new dtypes, and also the NaN description no longer is accurate for the new dtypes.

Also, "Float8E5M2" should be replaced with "Float8E4M3FN". I accidentally put the wrong dtype in the comment when adding Float8E4M3FN.

77

The sign bit can still be zero. I would clarify that for NaNs the exponent and mantissa are all 1s.

81

Also mention that for NaNs, the mantissa is also all 0s (you only mentioned the exponent is all 0s).

840–841

This comment should be moved to the "else" branch you added (or just delete the comment).

2506

Don't we only lose info if the source of the conversion is -0?

llvm/unittests/ADT/APFloatTest.cpp

1855

What does this TODO mean?

2078

Typo: castss -> casts

5280

Add the word "negative". And similarly in Float8E4M3FNUZNext

5342–5343

This is the same as the next test case

5376

typo: hives -> gives

5570

This looks pretty similar to Float8E5M2FNUZExhaustive. Maybe merge them.

[krzysz00]

Fix typos, merge exhaustive tests, get hit by the clang-format stick

[reedwm]

Just some more minor nits about comments.

llvm/lib/Support/APFloat.cpp

78–79

fltNanEncoding::AllOnes technically has two NaN representations: a NaN with the sign bit set and a NaN with the sign bit unset. I would change this setence to "The details of the NaN representation(s) are determined by the fltNanEncoding enum

100

typo: hhaving -> having

842

typo: signallling -> signalling

llvm/unittests/ADT/APFloatTest.cpp

5414

Typo: smalest -> smallest

[krzysz00]

Fix a few more typos and an incorrect comment

llvm/lib/Support/APFloat.cpp

2506

I don't think so? If I understand losesInfo correctly, it's for (non-rounding-related?) cases where, for x : T, convert(T, convert(U, x)) != x is possible.

So since both +0 and -0 map to +0, we've lost information.

llvm/unittests/ADT/APFloatTest.cpp

1855

Stray comment that was a note about adding what's currently const bool signedZero; above.

[reedwm]

llvm/lib/Support/APFloat.cpp

2506

I don't think so? If I understand losesInfo correctly, it's for (non-rounding-related?) cases where, for x : T, convert(T, convert(U, x)) != x is possible.

By that logic, losesInfo should be false if the source of the conversion is +0.

E.g., in your equation convert(T, convert(U, x)) != x, if x is the FP32 value +0, T is FP32, and U is FP8, then the equation is false, since converting +0 to FP8 and back results in the FP32 value +0. So losesInfo should be false in that case. On the other hand, if x is the FP32 value -0, the equation is true, since converting -0 to FP8 and back results in the FP32 value +0, and so losesInfo should be true.

[krzysz00]

llvm/lib/Support/APFloat.cpp

2506

Ok, to rephrase: we know that converting f32 Inf or NaN to any of the FN APFloat formats loses info, because there's two distinct concepts that get collapsed down to one value. Similarly, in our formats, putting in a 0 from a format that isn't unsigned-zero-only also loses info, as now +0 and -0 have been collapsed down to one value and you can't tell which one you had on the way back out.

[krzysz00]

llvm/lib/Support/APFloat.cpp

2506

Ok, having looked at the comment, on the function again you were right, +0 should not lose info but -0 should.

[krzysz00]

Fix correctness of losesInfo for +0 and -0

llvm/include/llvm/ADT/APFloat.h

enum Semantics {
     S_BFloat,
     S_IEEEsingle,
     S_IEEEdouble,
     S_IEEEquad,
     S_PPCDoubleDouble,
     // 8-bit floating point number following IEEE-754 conventions with bit
     // layout S1E5M2 as described in https://arxiv.org/abs/2209.05433.
     S_Float8E5M2,
+    // 8-bit floating point number mostly follwing IEEE-754 conventions
+    // and bit layout S1E5M2 described in [[TODO where did we denclibe this]]
+    // with expanded range and with no infinity or signed zero.
+    // NaN is represnted as negative zero. (FN -> Finite, UZ -> unsigned zero)
+    S_Float8E5M2FNUZ,
     // 8-bit floating point number mostly following IEEE-754 conventions with
     // bit layout S1E4M3 as described in https://arxiv.org/abs/2209.05433.
     // Unlike IEEE-754 types, there are no infinity values, and NaN is
     // represented with the exponent and mantissa bits set to all 1s.
     S_Float8E4M3FN,
+    // 8-bit floating point number mostly follwing IEEE-754 conventions
+    // and bit layout S1E4M3 described in [[TODO where did we denclibe this]]
+    // with expanded range and with no infinity or signed zero.
+    // NaN is represnted as negative zero. (FN -> Finite, UZ -> unsigned zero)
+    S_Float8E4M3FNUZ,
+
     S_x87DoubleExtended,
     S_MaxSemantics = S_x87DoubleExtended,
   };
 
   static const llvm::fltSemantics &EnumToSemantics(Semantics S);
   static Semantics SemanticsToEnum(const llvm::fltSemantics &Sem);
 
   static const fltSemantics &IEEEhalf() LLVM_READNONE;
   static const fltSemantics &BFloat() LLVM_READNONE;
   static const fltSemantics &IEEEsingle() LLVM_READNONE;
   static const fltSemantics &IEEEdouble() LLVM_READNONE;
   static const fltSemantics &IEEEquad() LLVM_READNONE;
   static const fltSemantics &PPCDoubleDouble() LLVM_READNONE;
   static const fltSemantics &Float8E5M2() LLVM_READNONE;
+  static const fltSemantics &Float8E5M2FNUZ() LLVM_READNONE;
   static const fltSemantics &Float8E4M3FN() LLVM_READNONE;
+  static const fltSemantics &Float8E4M3FNUZ() LLVM_READNONE;
   static const fltSemantics &x87DoubleExtended() LLVM_READNONE;
 
   /// A Pseudo fltsemantic used to construct APFloats that cannot conflict with
   /// anything real.
   static const fltSemantics &Bogus() LLVM_READNONE;
 
   /// @}
 
private:
   APInt convertHalfAPFloatToAPInt() const;
   APInt convertBFloatAPFloatToAPInt() const;
   APInt convertFloatAPFloatToAPInt() const;
   APInt convertDoubleAPFloatToAPInt() const;
   APInt convertQuadrupleAPFloatToAPInt() const;
   APInt convertF80LongDoubleAPFloatToAPInt() const;
   APInt convertPPCDoubleDoubleAPFloatToAPInt() const;
   APInt convertFloat8E5M2APFloatToAPInt() const;
+  APInt convertFloat8E5M2FNUZAPFloatToAPInt() const;
   APInt convertFloat8E4M3FNAPFloatToAPInt() const;
+  APInt convertFloat8E4M3FNUZAPFloatToAPInt() const;
   void initFromAPInt(const fltSemantics *Sem, const APInt &api);
   void initFromHalfAPInt(const APInt &api);
   void initFromBFloatAPInt(const APInt &api);
   void initFromFloatAPInt(const APInt &api);
   void initFromDoubleAPInt(const APInt &api);
   void initFromQuadrupleAPInt(const APInt &api);
   void initFromF80LongDoubleAPInt(const APInt &api);
   void initFromPPCDoubleDoubleAPInt(const APInt &api);
   void initFromFloat8E5M2APInt(const APInt &api);
+  void initFromFloat8E5M2FNUZAPInt(const APInt &api);
   void initFromFloat8E4M3FNAPInt(const APInt &api);
+  void initFromFloat8E4M3FNUZAPInt(const APInt &api);
 
   void assign(const IEEEFloat &);
   void copySignificand(const IEEEFloat &);
   void freeSignificand();
 
   /// Note: this must be the first data member.
   /// The semantics that this value obeys.
   const fltSemantics *semantics;

llvm/lib/Support/APFloat.cpp

 //===-- APFloat.cpp - Implement APFloat class -----------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements a class to represent arbitrary precision floating
 // point values and provide a variety of arithmetic operations on them.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/FloatingPointMode.h"
 #include "llvm/ADT/FoldingSet.h"
 #include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/MathExtras.h"
namespace llvm {
 
   // How the nonfinite values Inf and NaN are represented.
   enum class fltNonfiniteBehavior {
     // Represents standard IEEE 754 behavior. A value is nonfinite if the
     // exponent field is all 1s. In such cases, a value is Inf if the
     // significand bits are all zero, and NaN otherwise
     IEEE754,
 
     // Only the Float8E5M2 has this behavior. There is no Inf representation. A
     // value is NaN if the exponent field and the mantissa field are all 1s.
     // This behavior matches the FP8 E4M3 type described in
     // https://arxiv.org/abs/2209.05433. We treat both signed and unsigned NaNs
     // as non-signalling, although the paper does not state whether the NaN
     // values are signalling or not.
     NanOnly,
   };
 
+  enum class fltNanEncoding {
+    // Represents the standard IEEE behavior where a value is NaN if its
+    // exponent is all 1s and the significand is non-zero.
+    IEEE,
+
+    // Represents the behavior in some 8-bit floating ponit types where NaN is
+    // represnted by the all-1s value
+    AllOnes,
+
+    // Represents the behaviro in some 8-bit floating point types where NaN is
+    // represented by a sign bit of 1 and all 0s in the exponent (i.e. the
+    // negative zero encoding in a IEEE float).
+    NegativeZero,
+  };
+
   /* Represents floating point arithmetic semantics.  */
   struct fltSemantics {
     /* The largest E such that 2^E is representable; this matches the
        definition of IEEE 754.  */
     APFloatBase::ExponentType maxExponent;
 
     /* The smallest E such that 2^E is a normalized number; this
        matches the definition of IEEE 754.  */
     APFloatBase::ExponentType minExponent;
 
     /* Number of bits in the significand.  This includes the integer
        bit.  */
     unsigned int precision;
 
     /* Number of bits actually used in the semantics. */
     unsigned int sizeInBits;
 
     fltNonfiniteBehavior nonFiniteBehavior = fltNonfiniteBehavior::IEEE754;
 
+    fltNanEncoding nanEncoding = fltNanEncoding::IEEE;
     // Returns true if any number described by this semantics can be precisely
     // represented by the specified semantics. Does not take into account
     // the value of fltNonfiniteBehavior.
     bool isRepresentableBy(const fltSemantics &S) const {
       return maxExponent <= S.maxExponent && minExponent >= S.minExponent &&
              precision <= S.precision;
     }
   };
 
   static const fltSemantics semIEEEhalf = {15, -14, 11, 16};
   static const fltSemantics semBFloat = {127, -126, 8, 16};
   static const fltSemantics semIEEEsingle = {127, -126, 24, 32};
   static const fltSemantics semIEEEdouble = {1023, -1022, 53, 64};
   static const fltSemantics semIEEEquad = {16383, -16382, 113, 128};
   static const fltSemantics semFloat8E5M2 = {15, -14, 3, 8};
-  static const fltSemantics semFloat8E4M3FN = {8, -6, 4, 8,
-                                               fltNonfiniteBehavior::NanOnly};
+  static const fltSemantics semFloat8E5M2FNUZ = {15,
+                                                 -15,
+                                                 3,
+                                                 8,
+                                                 fltNonfiniteBehavior::NanOnly,
+                                                 fltNanEncoding::NegativeZero};
+  static const fltSemantics semFloat8E4M3FN = {
+      8, -6, 4, 8, fltNonfiniteBehavior::NanOnly, fltNanEncoding::AllOnes};
+  static const fltSemantics semFloat8E4M3FNUZ = {
+      7, -7, 4, 8, fltNonfiniteBehavior::NanOnly, fltNanEncoding::NegativeZero};
   static const fltSemantics semX87DoubleExtended = {16383, -16382, 64, 80};
   static const fltSemantics semBogus = {0, 0, 0, 0};
 
   /* The IBM double-double semantics. Such a number consists of a pair of IEEE
      64-bit doubles (Hi, Lo), where |Hi| > |Lo|, and if normal,
      (double)(Hi + Lo) == Hi. The numeric value it's modeling is Hi + Lo.
      Therefore it has two 53-bit mantissa parts that aren't necessarily adjacent
      to each other, and two 11-bit exponents.
const llvm::fltSemantics &APFloatBase::EnumToSemantics(Semantics S) {
     case S_IEEEdouble:
       return IEEEdouble();
     case S_IEEEquad:
       return IEEEquad();
     case S_PPCDoubleDouble:
       return PPCDoubleDouble();
     case S_Float8E5M2:
       return Float8E5M2();
+    case S_Float8E5M2FNUZ:
+      return Float8E5M2FNUZ();
     case S_Float8E4M3FN:
       return Float8E4M3FN();
+    case S_Float8E4M3FNUZ:
+      return Float8E4M3FNUZ();
     case S_x87DoubleExtended:
       return x87DoubleExtended();
     }
     llvm_unreachable("Unrecognised floating semantics");
   }
 
   APFloatBase::Semantics
   APFloatBase::SemanticsToEnum(const llvm::fltSemantics &Sem) {
     if (&Sem == &llvm::APFloat::IEEEhalf())
       return S_IEEEhalf;
     else if (&Sem == &llvm::APFloat::BFloat())
       return S_BFloat;
     else if (&Sem == &llvm::APFloat::IEEEsingle())
       return S_IEEEsingle;
     else if (&Sem == &llvm::APFloat::IEEEdouble())
       return S_IEEEdouble;
     else if (&Sem == &llvm::APFloat::IEEEquad())
       return S_IEEEquad;
     else if (&Sem == &llvm::APFloat::PPCDoubleDouble())
       return S_PPCDoubleDouble;
     else if (&Sem == &llvm::APFloat::Float8E5M2())
       return S_Float8E5M2;
+    else if (&Sem == &llvm::APFloat::Float8E5M2FNUZ())
+      return S_Float8E5M2FNUZ;
     else if (&Sem == &llvm::APFloat::Float8E4M3FN())
       return S_Float8E4M3FN;
+    else if (&Sem == &llvm::APFloat::Float8E4M3FNUZ())
+      return S_Float8E4M3FNUZ;
     else if (&Sem == &llvm::APFloat::x87DoubleExtended())
       return S_x87DoubleExtended;
     else
       llvm_unreachable("Unknown floating semantics");
   }
 
   const fltSemantics &APFloatBase::IEEEhalf() {
     return semIEEEhalf;
   }
   const fltSemantics &APFloatBase::BFloat() {
     return semBFloat;
   }
   const fltSemantics &APFloatBase::IEEEsingle() {
     return semIEEEsingle;
   }
   const fltSemantics &APFloatBase::IEEEdouble() {
     return semIEEEdouble;
   }
   const fltSemantics &APFloatBase::IEEEquad() { return semIEEEquad; }
   const fltSemantics &APFloatBase::PPCDoubleDouble() {
     return semPPCDoubleDouble;
   }
   const fltSemantics &APFloatBase::Float8E5M2() { return semFloat8E5M2; }
+  const fltSemantics &APFloatBase::Float8E5M2FNUZ() {
+    return semFloat8E5M2FNUZ;
+  }
   const fltSemantics &APFloatBase::Float8E4M3FN() { return semFloat8E4M3FN; }
+  const fltSemantics &APFloatBase::Float8E4M3FNUZ() {
+    return semFloat8E4M3FNUZ;
+  }
   const fltSemantics &APFloatBase::x87DoubleExtended() {
     return semX87DoubleExtended;
   }
   const fltSemantics &APFloatBase::Bogus() { return semBogus; }
 
   constexpr RoundingMode APFloatBase::rmNearestTiesToEven;
   constexpr RoundingMode APFloatBase::rmTowardPositive;
   constexpr RoundingMode APFloatBase::rmTowardNegative;
void IEEEFloat::makeNaN(bool SNaN, bool Negative, const APInt *fill) {
   sign = Negative;
   exponent = exponentNaN();
 
   integerPart *significand = significandParts();
   unsigned numParts = partCount();
 
   APInt fill_storage;
   if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
     // The only NaN representation is where the mantissa is all 1s, which is
     // non-signalling.
     SNaN = false;
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero) {
+      sign = true;
+      fill_storage = APInt::getZero(semantics->precision - 1);
+    } else {
       fill_storage = APInt::getAllOnes(semantics->precision - 1);
+    }
     fill = &fill_storage;
   }
 
   // Set the significand bits to the fill.
   if (!fill || fill->getNumWords() < numParts)
     APInt::tcSet(significand, 0, numParts);
   if (fill) {
     APInt::tcAssign(significand, fill->getRawData(),
if (SNaN) {
     // We always have to clear the QNaN bit to make it an SNaN.
     APInt::tcClearBit(significand, QNaNBit);
 
     // If there are no bits set in the payload, we have to set
     // *something* to make it a NaN instead of an infinity;
     // conventionally, this is the next bit down from the QNaN bit.
     if (APInt::tcIsZero(significand, numParts))
       APInt::tcSetBit(significand, QNaNBit - 1);
+  } else if (semantics->nanEncoding == fltNanEncoding::NegativeZero) {
+    // The only NaN is a quiet NaN, and it has no bits sets in the significand.
+    // Do nothing.
   } else {
     // We always have to set the QNaN bit to make it a QNaN.
     APInt::tcSetBit(significand, QNaNBit);
   }
 
   // For x87 extended precision, we want to make a NaN, not a
   // pseudo-NaN.  Maybe we should expose the ability to make
   // pseudo-NaNs?
bool IEEEFloat::isSignificandAllZerosExceptMSB() const {
 
   const unsigned NumHighBits =
       PartCount * integerPartWidth - semantics->precision + 1;
   return Parts[PartCount - 1] == integerPart(1)
                                      << (integerPartWidth - NumHighBits);
 }
 
 bool IEEEFloat::isLargest() const {
-  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
+  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly &&
+      semantics->nanEncoding == fltNanEncoding::AllOnes) {
     // The largest number by magnitude in our format will be the floating point
     // number with maximum exponent and with significand that is all ones except
     // the LSB.
     return isFiniteNonZero() && exponent == semantics->maxExponent &&
            isSignificandAllOnesExceptLSB();
   } else {
     // The largest number by magnitude in our format will be the floating point
     // number with maximum exponent and with significand that is all ones.
if (rounding_mode == rmNearestTiesToEven ||
     return (opStatus) (opOverflow | opInexact);
   }
 
   /* Otherwise we become the largest finite number.  */
   category = fcNormal;
   exponent = semantics->maxExponent;
   tcSetLeastSignificantBits(significandParts(), partCount(),
                             semantics->precision);
-  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly)
+  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly &&
+      semantics->nanEncoding == fltNanEncoding::AllOnes)
     APInt::tcClearBit(significandParts(), 0);
 
   return opInexact;
 }
 
 /* Returns TRUE if, when truncating the current number, with BIT the
    new LSB, with the given lost fraction and rounding mode, the result
    would need to be rounded away from zero (i.e., by increasing the
if (exponentChange > 0) {
       /* Keep OMSB up-to-date.  */
       if (omsb > (unsigned) exponentChange)
         omsb -= exponentChange;
       else
         omsb = 0;
     }
   }
 
+  // The all-ones values is an overflow if NaN is all ones. If NaN is
+  // represented by negative zero, then it is a valid finite value.
   if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly &&
+      semantics->nanEncoding == fltNanEncoding::AllOnes &&
       exponent == semantics->maxExponent && isSignificandAllOnes())
     return handleOverflow(rounding_mode);
 
   /* Now round the number according to rounding_mode given the lost
      fraction.  */
 
   /* As specified in IEEE 754, since we do not trap we do not report
      underflow for exact results.  */
   if (lost_fraction == lfExactlyZero) {
     /* Canonicalize zeroes.  */
-    if (omsb == 0)
+    if (omsb == 0) {
       category = fcZero;
+      if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+        sign = false;
+    }
 
     return opOK;
   }
 
   /* Increment the significand if we're rounding away from zero.  */
   if (roundAwayFromZero(rounding_mode, lost_fraction, 0)) {
     if (omsb == 0)
       exponent = semantics->minExponent;
 
     incrementSignificand();
     omsb = significandMSB() + 1;
 
     /* Did the significand increment overflow?  */
     if (omsb == (unsigned) semantics->precision + 1) {
       /* Renormalize by incrementing the exponent and shifting our
          significand right one.  However if we already have the
          maximum exponent we overflow to infinity.  */
-      if (exponent == semantics->maxExponent) {
-        category = fcInfinity;
-
-        return (opStatus) (opOverflow | opInexact);
-      }
+      if (exponent == semantics->maxExponent)
+        // Invoke overflow handling with a rounding mode that will guarantee
+        // that the result gets turned into the correct infinity representation.
+        // This is needed instead of just setting the category to infinity to
+        // account for 8-bit floating point types that have no inf, only NaN.
+        return handleOverflow(sign ? rmTowardNegative : rmTowardPositive);
 
       shiftSignificandRight(1);
 
       return opInexact;
     }
 
+    // The all-ones values is an overflow if NaN is all ones. If NaN is
+    // represented by negative zero, then it is a valid finite value.
     if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly &&
+        semantics->nanEncoding == fltNanEncoding::AllOnes &&
         exponent == semantics->maxExponent && isSignificandAllOnes())
       return handleOverflow(rounding_mode);
   }
 
   /* The normal case - we were and are not denormal, and any
      significand increment above didn't overflow.  */
   if (omsb == semantics->precision)
     return opInexact;
 
   /* We have a non-zero denormal.  */
   assert(omsb < semantics->precision);
 
   /* Canonicalize zeroes.  */
-  if (omsb == 0)
+  if (omsb == 0) {
     category = fcZero;
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      sign = false;
+  }
 
   /* The fcZero case is a denormal that underflowed to zero.  */
   return (opStatus) (opUnderflow | opInexact);
 }
 
 IEEEFloat::opStatus IEEEFloat::addOrSubtractSpecials(const IEEEFloat &rhs,
                                                      bool subtract) {
   switch (PackCategoriesIntoKey(category, rhs.category)) {
IEEEFloat::opStatus IEEEFloat::remainderSpecials(const IEEEFloat &rhs) {
 
   case PackCategoriesIntoKey(fcNormal, fcNormal):
     return opDivByZero; // fake status, indicating this is not a special case
   }
 }
 
 /* Change sign.  */
 void IEEEFloat::changeSign() {
+  // With NaN-as-negative-zero, neither NaN or negative zero can change
+  // their signs.
+  if (semantics->nanEncoding == fltNanEncoding::NegativeZero &&
+      (isZero() || isNaN()))
+    return;
   /* Look mummy, this one's easy.  */
   sign = !sign;
 }
 
 /* Normalized addition or subtraction.  */
 IEEEFloat::opStatus IEEEFloat::addOrSubtract(const IEEEFloat &rhs,
                                              roundingMode rounding_mode,
                                              bool subtract) {
IEEEFloat::opStatus IEEEFloat::addOrSubtract(const IEEEFloat &rhs,
   }
 
   /* If two numbers add (exactly) to zero, IEEE 754 decrees it is a
      positive zero unless rounding to minus infinity, except that
      adding two like-signed zeroes gives that zero.  */
   if (category == fcZero) {
     if (rhs.category != fcZero || (sign == rhs.sign) == subtract)
       sign = (rounding_mode == rmTowardNegative);
+    // NaN-in-negative-zero means zeros need to be normalized to +0.
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      sign = false;
   }
 
   return fs;
 }
 
 /* Normalized addition.  */
 IEEEFloat::opStatus IEEEFloat::add(const IEEEFloat &rhs,
                                    roundingMode rounding_mode) {

 /* Normalized multiply.  */
 IEEEFloat::opStatus IEEEFloat::multiply(const IEEEFloat &rhs,
                                         roundingMode rounding_mode) {
   opStatus fs;
 
   sign ^= rhs.sign;
   fs = multiplySpecials(rhs);
 
+  if (isZero() && semantics->nanEncoding == fltNanEncoding::NegativeZero)
+    sign = false;
   if (isFiniteNonZero()) {
     lostFraction lost_fraction = multiplySignificand(rhs);
     fs = normalize(rounding_mode, lost_fraction);
     if (lost_fraction != lfExactlyZero)
       fs = (opStatus) (fs | opInexact);
   }
 
   return fs;
 }
 
 /* Normalized divide.  */
 IEEEFloat::opStatus IEEEFloat::divide(const IEEEFloat &rhs,
                                       roundingMode rounding_mode) {
   opStatus fs;
 
   sign ^= rhs.sign;
   fs = divideSpecials(rhs);
 
+  if (isZero() && semantics->nanEncoding == fltNanEncoding::NegativeZero)
+    sign = false;
   if (isFiniteNonZero()) {
     lostFraction lost_fraction = divideSignificand(rhs);
     fs = normalize(rounding_mode, lost_fraction);
     if (lost_fraction != lfExactlyZero)
       fs = (opStatus) (fs | opInexact);
   }
 
   return fs;
if (VEx.compare(PEx) == cmpGreaterThan) {
 
     cmpResult result = VEx.compare(PEx);
     if (result == cmpGreaterThan || result == cmpEqual) {
       fs = subtract(P, rmNearestTiesToEven);
       assert(fs == opOK);
     }
   }
 
-  if (isZero())
+  if (isZero()) {
     sign = origSign;    // IEEE754 requires this
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      // But some 8-bit floats only have positive 0.
+      sign = false;
+  }
+
   else
     sign ^= origSign;
   return fs;
 }
 
 /* Normalized llvm frem (C fmod). */
 IEEEFloat::opStatus IEEEFloat::mod(const IEEEFloat &rhs) {
   opStatus fs;
   fs = modSpecials(rhs);
   unsigned int origSign = sign;
 
   while (isFiniteNonZero() && rhs.isFiniteNonZero() &&
          compareAbsoluteValue(rhs) != cmpLessThan) {
     int Exp = ilogb(*this) - ilogb(rhs);
     IEEEFloat V = scalbn(rhs, Exp, rmNearestTiesToEven);
     // V can overflow to NaN with fltNonfiniteBehavior::NanOnly, so explicitly
     // check for it.
     if (V.isNaN() || compareAbsoluteValue(V) == cmpLessThan)
       V = scalbn(rhs, Exp - 1, rmNearestTiesToEven);
     V.sign = sign;
 
     fs = subtract(V, rmNearestTiesToEven);
     assert(fs==opOK);
   }
-  if (isZero())
+  if (isZero()) {
     sign = origSign; // fmod requires this
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      sign = false;
+  }
   return fs;
 }
 
 /* Normalized fused-multiply-add.  */
 IEEEFloat::opStatus IEEEFloat::fusedMultiplyAdd(const IEEEFloat &multiplicand,
                                                 const IEEEFloat &addend,
                                                 roundingMode rounding_mode) {
   opStatus fs;
if (isFiniteNonZero() &&
     lost_fraction = multiplySignificand(multiplicand, addend);
     fs = normalize(rounding_mode, lost_fraction);
     if (lost_fraction != lfExactlyZero)
       fs = (opStatus) (fs | opInexact);
 
     /* If two numbers add (exactly) to zero, IEEE 754 decrees it is a
        positive zero unless rounding to minus infinity, except that
        adding two like-signed zeroes gives that zero.  */
-    if (category == fcZero && !(fs & opUnderflow) && sign != addend.sign)
+    if (category == fcZero && !(fs & opUnderflow) && sign != addend.sign) {
       sign = (rounding_mode == rmTowardNegative);
+      if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+        sign = false;
+    }
   } else {
     fs = multiplySpecials(multiplicand);
 
     /* FS can only be opOK or opInvalidOp.  There is no more work
        to do in the latter case.  The IEEE-754R standard says it is
        implementation-defined in this case whether, if ADDEND is a
        quiet NaN, we raise invalid op; this implementation does so.
 
IEEEFloat::opStatus IEEEFloat::convert(const fltSemantics &toSemantics,
   } else if (category == fcNaN) {
     if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
       *losesInfo =
           fromSemantics.nonFiniteBehavior != fltNonfiniteBehavior::NanOnly;
       makeNaN(false, sign);
       return is_signaling ? opInvalidOp : opOK;
     }
 
+    // If NaN is negative zero, we need to create a new NaN to avoid converting
+    // NaN to -Inf.
+    if (fromSemantics.nanEncoding == fltNanEncoding::NegativeZero &&
+        semantics->nanEncoding != fltNanEncoding::NegativeZero)
+      makeNaN(false, false);
+
     *losesInfo = lostFraction != lfExactlyZero || X86SpecialNan;
 
     // For x87 extended precision, we want to make a NaN, not a special NaN if
     // the input wasn't special either.
     if (!X86SpecialNan && semantics == &semX87DoubleExtended)
       APInt::tcSetBit(significandParts(), semantics->precision - 1);
 
     // Convert of sNaN creates qNaN and raises an exception (invalid op).
     // This also guarantees that a sNaN does not become Inf on a truncation
     // that loses all payload bits.
     if (is_signaling) {
       makeQuiet();
       fs = opInvalidOp;
     } else {
       fs = opOK;
     }
   } else if (category == fcInfinity &&
              semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
     makeNaN(false, sign);
     *losesInfo = true;
     fs = opInexact;
+  } else if (category == fcZero &&
+             semantics->nanEncoding == fltNanEncoding::NegativeZero) {
+    *losesInfo = fromSemantics.nanEncoding != fltNanEncoding::NegativeZero;
+    fs = *losesInfo ? opInexact : opOK;
+    // NaN is negative zero means -0 -> +0, which can lose information
+    sign = false;
   } else {
     *losesInfo = false;
     fs = opOK;
   }
 
   return fs;
 }
 
IEEEFloat::convertFromDecimalString(StringRef str, roundingMode rounding_mode) {
   //
   // We computed firstSigDigit by ignoring all zeros and dots. Thus if
   // D->firstSigDigit equals str.end(), every digit must be a zero and there can
   // be at most one dot. On the other hand, if we have a zero with a non-zero
   // exponent, then we know that D.firstSigDigit will be non-numeric.
   if (D.firstSigDigit == str.end() || decDigitValue(*D.firstSigDigit) >= 10U) {
     category = fcZero;
     fs = opOK;
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      sign = false;
 
     /* Check whether the normalized exponent is high enough to overflow
        max during the log-rebasing in the max-exponent check below. */
   } else if (D.normalizedExponent - 1 > INT_MAX / 42039) {
     fs = handleOverflow(rounding_mode);
 
   /* If it wasn't, then it also wasn't high enough to overflow max
      during the log-rebasing in the min-exponent check.  Check that it
      won't overflow min in either check, then perform the min-exponent
      check. */
   } else if (D.normalizedExponent - 1 < INT_MIN / 42039 ||
if (isFiniteNonZero()) {
     myexponent = 0x1f;
     mysignificand = (uint32_t)*significandParts();
   }
 
   return APInt(8, (((sign & 1) << 7) | ((myexponent & 0x1f) << 2) |
                    (mysignificand & 0x3)));
 }
 
+APInt IEEEFloat::convertFloat8E5M2FNUZAPFloatToAPInt() const {
+  assert(semantics == (const llvm::fltSemantics *)&semFloat8E5M2FNUZ);
+  assert(partCount() == 1);
+
+  uint32_t myexponent, mysignificand;
+
+  if (isFiniteNonZero()) {
+    myexponent = exponent + 16; // bias
+    mysignificand = (uint32_t)*significandParts();
+    if (myexponent == 1 && !(mysignificand & 0x4))
+      myexponent = 0; // denormal
+  } else if (category == fcZero) {
+    myexponent = 0;
+    mysignificand = 0;
+  } else if (category == fcInfinity) {
+    myexponent = 0;
+    mysignificand = 0;
+  } else {
+    assert(category == fcNaN && "Unknown category!");
+    myexponent = 0;
+    mysignificand = (uint32_t)*significandParts();
+  }
+
+  return APInt(8, (((sign & 1) << 7) | ((myexponent & 0x1f) << 2) |
+                   (mysignificand & 0x3)));
+}
+
 APInt IEEEFloat::convertFloat8E4M3FNAPFloatToAPInt() const {
   assert(semantics == (const llvm::fltSemantics *)&semFloat8E4M3FN);
   assert(partCount() == 1);
 
   uint32_t myexponent, mysignificand;
 
   if (isFiniteNonZero()) {
     myexponent = exponent + 7; // bias
if (isFiniteNonZero()) {
     myexponent = 0xf;
     mysignificand = (uint32_t)*significandParts();
   }
 
   return APInt(8, (((sign & 1) << 7) | ((myexponent & 0xf) << 3) |
                    (mysignificand & 0x7)));
 }
 
+APInt IEEEFloat::convertFloat8E4M3FNUZAPFloatToAPInt() const {
+  assert(semantics == (const llvm::fltSemantics *)&semFloat8E4M3FNUZ);
+  assert(partCount() == 1);
+
+  uint32_t myexponent, mysignificand;
+
+  if (isFiniteNonZero()) {
+    myexponent = exponent + 8; // bias
+    mysignificand = (uint32_t)*significandParts();
+    if (myexponent == 1 && !(mysignificand & 0x8))
+      myexponent = 0; // denormal
+  } else if (category == fcZero) {
+    myexponent = 0;
+    mysignificand = 0;
+  } else if (category == fcInfinity) {
+    myexponent = 0;
+    mysignificand = 0;
+  } else {
+    assert(category == fcNaN && "Unknown category!");
+    myexponent = 0;
+    mysignificand = (uint32_t)*significandParts();
+  }
+
+  return APInt(8, (((sign & 1) << 7) | ((myexponent & 0xf) << 3) |
+                   (mysignificand & 0x7)));
+}
+
 // This function creates an APInt that is just a bit map of the floating
 // point constant as it would appear in memory.  It is not a conversion,
 // and treating the result as a normal integer is unlikely to be useful.
 
 APInt IEEEFloat::bitcastToAPInt() const {
   if (semantics == (const llvm::fltSemantics*)&semIEEEhalf)
     return convertHalfAPFloatToAPInt();
 
if (semantics == (const llvm::fltSemantics*)&semIEEEquad)
     return convertQuadrupleAPFloatToAPInt();
 
   if (semantics == (const llvm::fltSemantics *)&semPPCDoubleDoubleLegacy)
     return convertPPCDoubleDoubleAPFloatToAPInt();
 
   if (semantics == (const llvm::fltSemantics *)&semFloat8E5M2)
     return convertFloat8E5M2APFloatToAPInt();
 
+  if (semantics == (const llvm::fltSemantics *)&semFloat8E5M2FNUZ)
+    return convertFloat8E5M2FNUZAPFloatToAPInt();
+
   if (semantics == (const llvm::fltSemantics *)&semFloat8E4M3FN)
     return convertFloat8E4M3FNAPFloatToAPInt();
 
+  if (semantics == (const llvm::fltSemantics *)&semFloat8E4M3FNUZ)
+    return convertFloat8E4M3FNUZAPFloatToAPInt();
+
   assert(semantics == (const llvm::fltSemantics*)&semX87DoubleExtended &&
          "unknown format!");
   return convertF80LongDoubleAPFloatToAPInt();
 }
 
 float IEEEFloat::convertToFloat() const {
   assert(semantics == (const llvm::fltSemantics*)&semIEEEsingle &&
          "Float semantics are not IEEEsingle");
if (myexponent == 0 && mysignificand == 0) {
     *significandParts() = mysignificand;
     if (myexponent == 0) // denormal
       exponent = -14;
     else
       *significandParts() |= 0x4; // integer bit
   }
 }
 
+void IEEEFloat::initFromFloat8E5M2FNUZAPInt(const APInt &api) {
+  uint32_t i = (uint32_t)*api.getRawData();
+  uint32_t myexponent = (i >> 2) & 0x1f;
+  uint32_t mysignificand = i & 0x3;
+
+  initialize(&semFloat8E5M2FNUZ);
+  assert(partCount() == 1);
+
+  sign = i >> 7;
+  if (myexponent == 0 && mysignificand == 0 && sign == 0) {
+    makeZero(sign);
+  } else if (myexponent == 0 && mysignificand == 0 && sign == 1) {
+    category = fcNaN;
+    exponent = exponentNaN();
+    *significandParts() = mysignificand;
+  } else {
+    category = fcNormal;
+    exponent = myexponent - 16; // bias
+    *significandParts() = mysignificand;
+    if (myexponent == 0) // denormal
+      exponent = -15;
+    else
+      *significandParts() |= 0x4; // integer bit
+  }
+}
+
 void IEEEFloat::initFromFloat8E4M3FNAPInt(const APInt &api) {
   uint32_t i = (uint32_t)*api.getRawData();
   uint32_t myexponent = (i >> 3) & 0xf;
   uint32_t mysignificand = i & 0x7;
 
   initialize(&semFloat8E4M3FN);
   assert(partCount() == 1);
 
if (myexponent == 0 && mysignificand == 0) {
     *significandParts() = mysignificand;
     if (myexponent == 0) // denormal
       exponent = -6;
     else
       *significandParts() |= 0x8; // integer bit
   }
 }
 
+void IEEEFloat::initFromFloat8E4M3FNUZAPInt(const APInt &api) {
+  uint32_t i = (uint32_t)*api.getRawData();
+  uint32_t myexponent = (i >> 3) & 0xf;
+  uint32_t mysignificand = i & 0x7;
+
+  initialize(&semFloat8E4M3FNUZ);
+  assert(partCount() == 1);
+
+  sign = i >> 7;
+  if (myexponent == 0 && mysignificand == 0 && sign == 0) {
+    makeZero(sign);
+  } else if (myexponent == 0 && mysignificand == 0 && sign == 1) {
+    category = fcNaN;
+    exponent = exponentNaN();
+    *significandParts() = mysignificand;
+  } else {
+    category = fcNormal;
+    exponent = myexponent - 8; // bias
+    *significandParts() = mysignificand;
+    if (myexponent == 0) // denormal
+      exponent = -7;
+    else
+      *significandParts() |= 0x8; // integer bit
+  }
+}
+
 /// Treat api as containing the bits of a floating point number.
 void IEEEFloat::initFromAPInt(const fltSemantics *Sem, const APInt &api) {
   assert(api.getBitWidth() == Sem->sizeInBits);
   if (Sem == &semIEEEhalf)
     return initFromHalfAPInt(api);
   if (Sem == &semBFloat)
     return initFromBFloatAPInt(api);
   if (Sem == &semIEEEsingle)
     return initFromFloatAPInt(api);
   if (Sem == &semIEEEdouble)
     return initFromDoubleAPInt(api);
   if (Sem == &semX87DoubleExtended)
     return initFromF80LongDoubleAPInt(api);
   if (Sem == &semIEEEquad)
     return initFromQuadrupleAPInt(api);
   if (Sem == &semPPCDoubleDoubleLegacy)
     return initFromPPCDoubleDoubleAPInt(api);
   if (Sem == &semFloat8E5M2)
     return initFromFloat8E5M2APInt(api);
+  if (Sem == &semFloat8E5M2FNUZ)
+    return initFromFloat8E5M2FNUZAPInt(api);
   if (Sem == &semFloat8E4M3FN)
     return initFromFloat8E4M3FNAPInt(api);
+  if (Sem == &semFloat8E4M3FNUZ)
+    return initFromFloat8E4M3FNUZAPInt(api);
 
   llvm_unreachable(nullptr);
 }
 
 /// Make this number the largest magnitude normal number in the given
 /// semantics.
 void IEEEFloat::makeLargest(bool Negative) {
   // We want (in interchange format):
void IEEEFloat::makeLargest(bool Negative) {
   // Set the high integerPart especially setting all unused top bits for
   // internal consistency.
   const unsigned NumUnusedHighBits =
     PartCount*integerPartWidth - semantics->precision;
   significand[PartCount - 1] = (NumUnusedHighBits < integerPartWidth)
                                    ? (~integerPart(0) >> NumUnusedHighBits)
                                    : 0;
 
-  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly)
+  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly &&
+      semantics->nanEncoding == fltNanEncoding::AllOnes)
     significand[0] &= ~integerPart(1);
 }
 
 /// Make this number the smallest magnitude denormal number in the given
 /// semantics.
 void IEEEFloat::makeSmallest(bool Negative) {
   // We want (in interchange format):
   //   sign = {Negative}
case fcZero:
     makeSmallest(false);
     break;
   case fcNormal:
     // nextUp(-getSmallest()) = -0
     if (isSmallest() && isNegative()) {
       APInt::tcSet(significandParts(), 0, partCount());
       category = fcZero;
       exponent = 0;
+      if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+        sign = false;
       break;
     }
 
     if (isLargest() && !isNegative()) {
       if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
         // nextUp(getLargest()) == NAN
         makeNaN();
         break;
IEEEFloat::opStatus IEEEFloat::next(bool nextDown) {
   // If we are performing nextDown, swap sign so we have -nextUp(-x)
   if (nextDown)
     changeSign();
 
   return result;
 }
 
 APFloatBase::ExponentType IEEEFloat::exponentNaN() const {
-  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly)
+  if (semantics->nonFiniteBehavior == fltNonfiniteBehavior::NanOnly) {
+    if (semantics->nanEncoding == fltNanEncoding::NegativeZero)
+      return semantics->minExponent;
     return semantics->maxExponent;
+  }
   return semantics->maxExponent + 1;
 }
 
 APFloatBase::ExponentType IEEEFloat::exponentInf() const {
   return semantics->maxExponent + 1;
 }
 
 APFloatBase::ExponentType IEEEFloat::exponentZero() const {
void IEEEFloat::makeInf(bool Negative) {
   sign = Negative;
   exponent = exponentInf();
   APInt::tcSet(significandParts(), 0, partCount());
 }
 
 void IEEEFloat::makeZero(bool Negative) {
   category = fcZero;
   sign = Negative;
+  if (semantics->nanEncoding == fltNanEncoding::NegativeZero) {
+    // Merge negative zero to positive because 0b10000...000 is used for NaN
+    sign = false;
+  }
   exponent = exponentZero();
   APInt::tcSet(significandParts(), 0, partCount());
 }
 
 void IEEEFloat::makeQuiet() {
   assert(isNaN());
   if (semantics->nonFiniteBehavior != fltNonfiniteBehavior::NanOnly)
     APInt::tcSetBit(significandParts(), semantics->precision - 2);

llvm/unittests/ADT/APFloatTest.cpp

 //===- llvm/unittest/ADT/APFloat.cpp - APFloat unit tests ---------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/Hashing.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "gtest/gtest.h"
 #include <cmath>
 #include <ostream>
 #include <string>
 #include <tuple>

 TEST(APFloatTest, makeNaN) {
   const struct {
     uint64_t expected;
     const fltSemantics &semantics;
     bool SNaN;
     bool Negative;
     uint64_t payload;
   } tests[] = {
+      // clang-format off
     /*             expected              semantics   SNaN    Neg                payload */
     {         0x7fc00000ULL, APFloat::IEEEsingle(), false, false,         0x00000000ULL },
     {         0xffc00000ULL, APFloat::IEEEsingle(), false,  true,         0x00000000ULL },
     {         0x7fc0ae72ULL, APFloat::IEEEsingle(), false, false,         0x0000ae72ULL },
     {         0x7fffae72ULL, APFloat::IEEEsingle(), false, false,         0xffffae72ULL },
     {         0x7fdaae72ULL, APFloat::IEEEsingle(), false, false,         0x00daae72ULL },
     {         0x7fa00000ULL, APFloat::IEEEsingle(),  true, false,         0x00000000ULL },
     {         0xffa00000ULL, APFloat::IEEEsingle(),  true,  true,         0x00000000ULL },
     {         0x7f80ae72ULL, APFloat::IEEEsingle(),  true, false,         0x0000ae72ULL },
     {         0x7fbfae72ULL, APFloat::IEEEsingle(),  true, false,         0xffffae72ULL },
     {         0x7f9aae72ULL, APFloat::IEEEsingle(),  true, false,         0x001aae72ULL },
     { 0x7ff8000000000000ULL, APFloat::IEEEdouble(), false, false, 0x0000000000000000ULL },
     { 0xfff8000000000000ULL, APFloat::IEEEdouble(), false,  true, 0x0000000000000000ULL },
     { 0x7ff800000000ae72ULL, APFloat::IEEEdouble(), false, false, 0x000000000000ae72ULL },
     { 0x7fffffffffffae72ULL, APFloat::IEEEdouble(), false, false, 0xffffffffffffae72ULL },
     { 0x7ffdaaaaaaaaae72ULL, APFloat::IEEEdouble(), false, false, 0x000daaaaaaaaae72ULL },
     { 0x7ff4000000000000ULL, APFloat::IEEEdouble(),  true, false, 0x0000000000000000ULL },
     { 0xfff4000000000000ULL, APFloat::IEEEdouble(),  true,  true, 0x0000000000000000ULL },
     { 0x7ff000000000ae72ULL, APFloat::IEEEdouble(),  true, false, 0x000000000000ae72ULL },
     { 0x7ff7ffffffffae72ULL, APFloat::IEEEdouble(),  true, false, 0xffffffffffffae72ULL },
     { 0x7ff1aaaaaaaaae72ULL, APFloat::IEEEdouble(),  true, false, 0x0001aaaaaaaaae72ULL },
+    {               0x80ULL, APFloat::Float8E5M2FNUZ(), false, false,           0xaaULL },
+    {               0x80ULL, APFloat::Float8E5M2FNUZ(), false, true,            0xaaULL },
+    {               0x80ULL, APFloat::Float8E5M2FNUZ(), true, false,            0xaaULL },
+    {               0x80ULL, APFloat::Float8E5M2FNUZ(), true, true,             0xaaULL },
+    {               0x80ULL, APFloat::Float8E4M3FNUZ(), false, false,           0xaaULL },
+    {               0x80ULL, APFloat::Float8E4M3FNUZ(), false, true,            0xaaULL },
+    {               0x80ULL, APFloat::Float8E4M3FNUZ(), true, false,            0xaaULL },
+    {               0x80ULL, APFloat::Float8E4M3FNUZ(), true, true,             0xaaULL },
+      // clang-format on
   };
 
   for (const auto &t : tests) {
     ASSERT_EQ(t.expected, nanbitsFromAPInt(t.semantics, t.SNaN, t.Negative, t.payload));
   }
 }
 
 #ifdef GTEST_HAS_DEATH_TEST
llvm::detail::DoubleAPFloat T3(APFloat::PPCDoubleDouble(), std::move(F5),
                                 std::move(F6));
   EXPECT_FALSE(T3.isInteger());
 }
 
 TEST(APFloatTest, getLargest) {
   EXPECT_EQ(3.402823466e+38f, APFloat::getLargest(APFloat::IEEEsingle()).convertToFloat());
   EXPECT_EQ(1.7976931348623158e+308, APFloat::getLargest(APFloat::IEEEdouble()).convertToDouble());
   EXPECT_EQ(448, APFloat::getLargest(APFloat::Float8E4M3FN()).convertToDouble());
+  EXPECT_EQ(240,
+            APFloat::getLargest(APFloat::Float8E4M3FNUZ()).convertToDouble());
+  EXPECT_EQ(57344,
+            APFloat::getLargest(APFloat::Float8E5M2FNUZ()).convertToDouble());
 }
 
 TEST(APFloatTest, getSmallest) {
   APFloat test = APFloat::getSmallest(APFloat::IEEEsingle(), false);
   APFloat expected = APFloat(APFloat::IEEEsingle(), "0x0.000002p-126");
   EXPECT_FALSE(test.isNegative());
   EXPECT_TRUE(test.isFiniteNonZero());
   EXPECT_TRUE(test.isDenormal());
TEST(APFloatTest, getSmallest) {
   EXPECT_TRUE(test.bitwiseIsEqual(expected));
 
   test = APFloat::getSmallest(APFloat::IEEEquad(), true);
   expected = APFloat(APFloat::IEEEquad(), "-0x0.0000000000000000000000000001p-16382");
   EXPECT_TRUE(test.isNegative());
   EXPECT_TRUE(test.isFiniteNonZero());
   EXPECT_TRUE(test.isDenormal());
   EXPECT_TRUE(test.bitwiseIsEqual(expected));
+
+  test = APFloat::getSmallest(APFloat::Float8E5M2FNUZ(), false);
+  expected = APFloat(APFloat::Float8E5M2FNUZ(), "0x0.4p-15");
+  EXPECT_FALSE(test.isNegative());
+  EXPECT_TRUE(test.isFiniteNonZero());
+  EXPECT_TRUE(test.isDenormal());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
+
+  test = APFloat::getSmallest(APFloat::Float8E4M3FNUZ(), false);
+  expected = APFloat(APFloat::Float8E4M3FNUZ(), "0x0.2p-7");
+  EXPECT_FALSE(test.isNegative());
+  EXPECT_TRUE(test.isFiniteNonZero());
+  EXPECT_TRUE(test.isDenormal());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
 }
 
 TEST(APFloatTest, getSmallestNormalized) {
   APFloat test = APFloat::getSmallestNormalized(APFloat::IEEEsingle(), false);
   APFloat expected = APFloat(APFloat::IEEEsingle(), "0x1p-126");
   EXPECT_FALSE(test.isNegative());
   EXPECT_TRUE(test.isFiniteNonZero());
   EXPECT_FALSE(test.isDenormal());
TEST(APFloatTest, getSmallestNormalized) {
   EXPECT_TRUE(test.bitwiseIsEqual(expected));
   EXPECT_TRUE(test.isSmallestNormalized());
 }
 
 TEST(APFloatTest, getZero) {
   struct {
     const fltSemantics *semantics;
     const bool sign;
+    const bool signedZero;
     const unsigned long long bitPattern[2];
-    const unsigned bitPatternLength;
+    const unsigned bitPatternLength; // todo preserve sign flag
   } const GetZeroTest[] = {
-      {&APFloat::IEEEhalf(), false, {0, 0}, 1},
-      {&APFloat::IEEEhalf(), true, {0x8000ULL, 0}, 1},
-      {&APFloat::IEEEsingle(), false, {0, 0}, 1},
-      {&APFloat::IEEEsingle(), true, {0x80000000ULL, 0}, 1},
-      {&APFloat::IEEEdouble(), false, {0, 0}, 1},
-      {&APFloat::IEEEdouble(), true, {0x8000000000000000ULL, 0}, 1},
-      {&APFloat::IEEEquad(), false, {0, 0}, 2},
-      {&APFloat::IEEEquad(), true, {0, 0x8000000000000000ULL}, 2},
-      {&APFloat::PPCDoubleDouble(), false, {0, 0}, 2},
-      {&APFloat::PPCDoubleDouble(), true, {0x8000000000000000ULL, 0}, 2},
-      {&APFloat::x87DoubleExtended(), false, {0, 0}, 2},
-      {&APFloat::x87DoubleExtended(), true, {0, 0x8000ULL}, 2},
-      {&APFloat::Float8E5M2(), false, {0, 0}, 1},
-      {&APFloat::Float8E5M2(), true, {0x80ULL, 0}, 1},
-      {&APFloat::Float8E4M3FN(), false, {0, 0}, 1},
-      {&APFloat::Float8E4M3FN(), true, {0x80ULL, 0}, 1},
-  };
-  const unsigned NumGetZeroTests = 12;
+      {&APFloat::IEEEhalf(), false, true, {0, 0}, 1},
+      {&APFloat::IEEEhalf(), true, true, {0x8000ULL, 0}, 1},
+      {&APFloat::IEEEsingle(), false, true, {0, 0}, 1},
+      {&APFloat::IEEEsingle(), true, true, {0x80000000ULL, 0}, 1},
+      {&APFloat::IEEEdouble(), false, true, {0, 0}, 1},
+      {&APFloat::IEEEdouble(), true, true, {0x8000000000000000ULL, 0}, 1},
+      {&APFloat::IEEEquad(), false, true, {0, 0}, 2},
+      {&APFloat::IEEEquad(), true, true, {0, 0x8000000000000000ULL}, 2},
+      {&APFloat::PPCDoubleDouble(), false, true, {0, 0}, 2},
+      {&APFloat::PPCDoubleDouble(), true, true, {0x8000000000000000ULL, 0}, 2},
+      {&APFloat::x87DoubleExtended(), false, true, {0, 0}, 2},
+      {&APFloat::x87DoubleExtended(), true, true, {0, 0x8000ULL}, 2},
+      {&APFloat::Float8E5M2(), false, true, {0, 0}, 1},
+      {&APFloat::Float8E5M2(), true, true, {0x80ULL, 0}, 1},
+      {&APFloat::Float8E5M2FNUZ(), false, false, {0, 0}, 1},
+      {&APFloat::Float8E5M2FNUZ(), true, false, {0, 0}, 1},
+      {&APFloat::Float8E4M3FN(), false, true, {0, 0}, 1},
+      {&APFloat::Float8E4M3FN(), true, true, {0x80ULL, 0}, 1},
+      {&APFloat::Float8E4M3FNUZ(), false, false, {0, 0}, 1},
+      {&APFloat::Float8E4M3FNUZ(), true, false, {0, 0}, 1}};
+  const unsigned NumGetZeroTests = 16;
   for (unsigned i = 0; i < NumGetZeroTests; ++i) {
     APFloat test = APFloat::getZero(*GetZeroTest[i].semantics,
                                     GetZeroTest[i].sign);
     const char *pattern = GetZeroTest[i].sign? "-0x0p+0" : "0x0p+0";
     APFloat expected = APFloat(*GetZeroTest[i].semantics,
                                pattern);
     EXPECT_TRUE(test.isZero());
-    EXPECT_TRUE(GetZeroTest[i].sign? test.isNegative() : !test.isNegative());
+    if (GetZeroTest[i].signedZero)
+      EXPECT_TRUE(GetZeroTest[i].sign ? test.isNegative() : !test.isNegative());
+    else
+      EXPECT_TRUE(!test.isNegative());
     EXPECT_TRUE(test.bitwiseIsEqual(expected));
     for (unsigned j = 0, je = GetZeroTest[i].bitPatternLength; j < je; ++j) {
       EXPECT_EQ(GetZeroTest[i].bitPattern[j],
                 test.bitcastToAPInt().getRawData()[j]);
     }
   }
 }
 
 TEST(APFloatTest, copySign) {
   EXPECT_TRUE(APFloat(-42.0).bitwiseIsEqual(
       APFloat::copySign(APFloat(42.0), APFloat(-1.0))));
   EXPECT_TRUE(APFloat(42.0).bitwiseIsEqual(
       APFloat::copySign(APFloat(-42.0), APFloat(1.0))));
   EXPECT_TRUE(APFloat(-42.0).bitwiseIsEqual(
       APFloat::copySign(APFloat(-42.0), APFloat(-1.0))));
   EXPECT_TRUE(APFloat(42.0).bitwiseIsEqual(
       APFloat::copySign(APFloat(42.0), APFloat(1.0))));
+  // For floating-point formats with unsigned 0, copySign() to a zero is a noop
+  EXPECT_TRUE(
+      APFloat::getZero(APFloat::Float8E4M3FNUZ())
+          .bitwiseIsEqual(APFloat::copySign(
+              APFloat::getZero(APFloat::Float8E4M3FNUZ()), APFloat(-1.0))));
+  EXPECT_TRUE(
+      APFloat::getNaN(APFloat::Float8E4M3FNUZ(), true)
+          .bitwiseIsEqual(APFloat::copySign(
+              APFloat::getNaN(APFloat::Float8E4M3FNUZ(), true), APFloat(1.0))));
 }
 
 TEST(APFloatTest, convert) {
   bool losesInfo;
   APFloat test(APFloat::IEEEdouble(), "1.0");
   test.convert(APFloat::IEEEsingle(), APFloat::rmNearestTiesToEven, &losesInfo);
   EXPECT_EQ(1.0f, test.convertToFloat());
   EXPECT_FALSE(losesInfo);
TEST(APFloatTest, convert) {
   EXPECT_FALSE(losesInfo);
 
   test = APFloat(APFloat::IEEEsingle(), "0x0.01p-126");
   test.convert(APFloat::BFloat(), APFloat::rmNearestTiesToAway, &losesInfo);
   EXPECT_EQ(0x01, test.bitcastToAPInt());
   EXPECT_TRUE(losesInfo);
 }
 
+TEST(APFloatTest, Float8UZConvert) {
+  bool losesInfo = false;
+  std::pair<APFloat, APFloat::opStatus> toNaNTests[] = {
+      {APFloat::getQNaN(APFloat::IEEEsingle(), false), APFloat::opOK},
+      {APFloat::getQNaN(APFloat::IEEEsingle(), true), APFloat::opOK},
+      {APFloat::getSNaN(APFloat::IEEEsingle(), false), APFloat::opInvalidOp},
+      {APFloat::getSNaN(APFloat::IEEEsingle(), true), APFloat::opInvalidOp},
+      {APFloat::getInf(APFloat::IEEEsingle(), false), APFloat::opInexact},
+      {APFloat::getInf(APFloat::IEEEsingle(), true), APFloat::opInexact}};
+  for (auto [toTest, expectedRes] : toNaNTests) {
+    llvm::SmallString<16> value;
+    toTest.toString(value);
+    SCOPED_TRACE("toTest = " + value);
+    for (const fltSemantics *sem :
+         {&APFloat::Float8E4M3FNUZ(), &APFloat::Float8E5M2FNUZ()}) {
+      SCOPED_TRACE("Semantics = " +
+                   std::to_string(APFloat::SemanticsToEnum(*sem)));
+      losesInfo = false;
+      APFloat test = toTest;
+      EXPECT_EQ(test.convert(*sem, APFloat::rmNearestTiesToAway, &losesInfo),
+                expectedRes);
+      EXPECT_TRUE(test.isNaN());
+      EXPECT_TRUE(test.isNegative());
+      EXPECT_FALSE(test.isSignaling());
+      EXPECT_FALSE(test.isInfinity());
+      EXPECT_EQ(0x80, test.bitcastToAPInt());
+      EXPECT_TRUE(losesInfo);
+    }
+  }
+
+  // Zero conversions are information losing.
+  losesInfo = false;
+  APFloat test = APFloat::getZero(APFloat::IEEEsingle(), true);
+  EXPECT_EQ(test.convert(APFloat::Float8E5M2FNUZ(),
+                         APFloat::rmNearestTiesToAway, &losesInfo),
+            APFloat::opInexact);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(0x0, test.bitcastToAPInt());
+
+  losesInfo = false;
+  test = APFloat::getZero(APFloat::IEEEsingle(), false);
+  EXPECT_EQ(test.convert(APFloat::Float8E5M2FNUZ(),
+                         APFloat::rmNearestTiesToAway, &losesInfo),
+            APFloat::opInexact);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(0x0, test.bitcastToAPInt());
+
+  // Except in castss between ourselves
+  losesInfo = true;
+  test = APFloat::getZero(APFloat::Float8E5M2FNUZ());
+  EXPECT_EQ(test.convert(APFloat::Float8E4M3FNUZ(),
+                         APFloat::rmNearestTiesToAway, &losesInfo),
+            APFloat::opOK);
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(0x0, test.bitcastToAPInt());
+}
+
 TEST(APFloatTest, PPCDoubleDouble) {
   APFloat test(APFloat::PPCDoubleDouble(), "1.0");
   EXPECT_EQ(0x3ff0000000000000ull, test.bitcastToAPInt().getRawData()[0]);
   EXPECT_EQ(0x0000000000000000ull, test.bitcastToAPInt().getRawData()[1]);
 
   // LDBL_MAX
   test = APFloat(APFloat::PPCDoubleDouble(), "1.79769313486231580793728971405301e+308");
   EXPECT_EQ(0x7fefffffffffffffull, test.bitcastToAPInt().getRawData()[0]);
if (i == 1 || i == 129) {
       EXPECT_TRUE(test.isSmallest());
       EXPECT_EQ(abs(test).convertToDouble(), 0x1p-9);
     } else {
       EXPECT_FALSE(test.isSmallest());
     }
 
     // convert to BFloat
     APFloat test2 = test;
-    bool loses_info;
+    bool losesInfo;
     APFloat::opStatus status = test2.convert(
-        APFloat::BFloat(), APFloat::rmNearestTiesToEven, &loses_info);
+        APFloat::BFloat(), APFloat::rmNearestTiesToEven, &losesInfo);
     EXPECT_EQ(status, APFloat::opOK);
-    EXPECT_FALSE(loses_info);
+    EXPECT_FALSE(losesInfo);
     if (i == 127 || i == 255)
       EXPECT_TRUE(test2.isNaN());
     else
       EXPECT_EQ(test.convertToFloat(), test2.convertToFloat());
 
     // bitcastToAPInt
     EXPECT_EQ(i, test.bitcastToAPInt());
   }
 }
 
-TEST(APFloatTest, Float8E4M3FNExhaustivePair) {
-  // Test each pair of Float8E4M3FN values.
+TEST(APFloatTest, Float8E5M2FNUZNext) {
+  APFloat test(APFloat::Float8E5M2FNUZ(), APFloat::uninitialized);
+  APFloat expected(APFloat::Float8E5M2FNUZ(), APFloat::uninitialized);
+
+  // 1. NextUp of largest bit pattern is nan
+  test = APFloat::getLargest(APFloat::Float8E5M2FNUZ());
+  expected = APFloat::getNaN(APFloat::Float8E5M2FNUZ());
+  EXPECT_EQ(test.next(false), APFloat::opOK);
+  EXPECT_FALSE(test.isInfinity());
+  EXPECT_FALSE(test.isZero());
+  EXPECT_TRUE(test.isNaN());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
+
+  // 2. NextUp of smallest denormal is +0
+  test = APFloat::getSmallest(APFloat::Float8E5M2FNUZ(), true);
+  expected = APFloat::getZero(APFloat::Float8E5M2FNUZ(), false);
+  EXPECT_EQ(test.next(false), APFloat::opOK);
+  EXPECT_FALSE(test.isNegZero());
+  EXPECT_TRUE(test.isPosZero());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
+
+  // 3. nextDown of negative of largest value is NaN
+  test = APFloat::getLargest(APFloat::Float8E5M2FNUZ(), true);
+  expected = APFloat::getNaN(APFloat::Float8E5M2FNUZ());
+  EXPECT_EQ(test.next(true), APFloat::opOK);
+  EXPECT_FALSE(test.isInfinity());
+  EXPECT_FALSE(test.isZero());
+  EXPECT_TRUE(test.isNaN());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
+
+  // 4. nextDown of +0 is smallest negative denormal
+  test = APFloat::getZero(APFloat::Float8E5M2FNUZ(), false);
+  expected = APFloat::getSmallest(APFloat::Float8E5M2FNUZ(), true);
+  EXPECT_EQ(test.next(true), APFloat::opOK);
+  EXPECT_FALSE(test.isZero());
+  EXPECT_TRUE(test.isDenormal());
+  EXPECT_TRUE(test.bitwiseIsEqual(expected));
+}
+
+TEST(APFloatTest, UnsignedZeroArithmeticSpecial) {
+  // Float semantics with only unsigned zero (ex. Float8E4M3FNUZ) violate the
+  // IEEE rules about signs in arithmetic operations when producing zeros,
+  // because they only have one zero. Most of the rest of the complexities of
+  // arithmetic on these values are covered by the other Float8 types' test
+  // cases and so are not repeated here.
+
+  // The IEEE round towards negative rule doesn't apply
+  APFloat test = APFloat::getSmallest(APFloat::Float8E4M3FNUZ());
+  APFloat rhs = test;
+  EXPECT_EQ(test.subtract(rhs, APFloat::rmTowardNegative), APFloat::opOK);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+
+  // Multiplication of (small) * (-small) is +0
+  test = APFloat::getSmallestNormalized(APFloat::Float8E4M3FNUZ());
+  rhs = -test;
+  EXPECT_EQ(test.multiply(rhs, APFloat::rmNearestTiesToAway),
+            APFloat::opInexact | APFloat::opUnderflow);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+
+  // Dividing the negatize float_min by anything hives +0
+  test = APFloat::getSmallest(APFloat::Float8E4M3FNUZ(), true);
+  rhs = APFloat(APFloat::Float8E4M3FNUZ(), "2.0");
+  EXPECT_EQ(test.divide(rhs, APFloat::rmNearestTiesToEven),
+            APFloat::opInexact | APFloat::opUnderflow);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+
+  // Remainder can't copy sign because there's only one zero
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "-4.0");
+  rhs = APFloat(APFloat::Float8E4M3FNUZ(), "2.0");
+  EXPECT_EQ(test.remainder(rhs), APFloat::opOK);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+
+  // And same for mod
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "-4.0");
+  rhs = APFloat(APFloat::Float8E4M3FNUZ(), "2.0");
+  EXPECT_EQ(test.mod(rhs), APFloat::opOK);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+
+  // FMA correctly handles both the multiply and add parts of all this
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "2.0");
+  rhs = test;
+  APFloat addend = APFloat(APFloat::Float8E4M3FNUZ(), "-4.0");
+  EXPECT_EQ(test.fusedMultiplyAdd(rhs, addend, APFloat::rmTowardNegative),
+            APFloat::opOK);
+  EXPECT_TRUE(test.isZero());
+  EXPECT_FALSE(test.isNegative());
+}
+
+TEST(APFloatTest, Float8E5M2FNUZExhaustive) {
+  // Test each of the 256 Float8E5M2FNUZ values.
+  for (int i = 0; i < 256; i++) {
+    APFloat test(APFloat::Float8E5M2FNUZ(), APInt(8, i));
+    SCOPED_TRACE("i=" + std::to_string(i));
+
+    // isLargest
+    if (i == 127 || i == 255) {
+      EXPECT_TRUE(test.isLargest());
+      EXPECT_EQ(abs(test).convertToDouble(), 57344.);
+    } else {
+      EXPECT_FALSE(test.isLargest());
+    }
+
+    // isSmallest
+    if (i == 1 || i == 129) {
+      EXPECT_TRUE(test.isSmallest());
+      EXPECT_EQ(abs(test).convertToDouble(), 0x1p-17);
+    } else {
+      EXPECT_FALSE(test.isSmallest());
+    }
+
+    // convert to BFloat
+    APFloat test2 = test;
+    bool losesInfo;
+    APFloat::opStatus status = test2.convert(
+        APFloat::BFloat(), APFloat::rmNearestTiesToEven, &losesInfo);
+    EXPECT_EQ(status, APFloat::opOK);
+    EXPECT_FALSE(losesInfo);
+    if (i == 128)
+      EXPECT_TRUE(test2.isNaN());
+    else
+      EXPECT_EQ(test.convertToFloat(), test2.convertToFloat());
+
+    // bitcastToAPInt
+    EXPECT_EQ(i, test.bitcastToAPInt());
+  }
+}
+
+TEST(APFloatTest, Float8E4M3FNUZExhaustive) {
+  // Test each of the 256 Float8E4M3FNUZ values.
+  for (int i = 0; i < 256; i++) {
+    APFloat test(APFloat::Float8E4M3FNUZ(), APInt(8, i));
+    SCOPED_TRACE("i=" + std::to_string(i));
+
+    // isLargest
+    if (i == 127 || i == 255) {
+      EXPECT_TRUE(test.isLargest());
+      EXPECT_EQ(abs(test).convertToDouble(), 240.);
+    } else {
+      EXPECT_FALSE(test.isLargest());
+    }
+
+    // isSmallest
+    if (i == 1 || i == 129) {
+      EXPECT_TRUE(test.isSmallest());
+      EXPECT_EQ(abs(test).convertToDouble(), 0x1p-10);
+    } else {
+      EXPECT_FALSE(test.isSmallest());
+    }
+
+    // convert to BFloat
+    APFloat test2 = test;
+    bool losesInfo;
+    APFloat::opStatus status = test2.convert(
+        APFloat::BFloat(), APFloat::rmNearestTiesToEven, &losesInfo);
+    EXPECT_EQ(status, APFloat::opOK);
+    EXPECT_FALSE(losesInfo);
+    if (i == 128)
+      EXPECT_TRUE(test2.isNaN());
+    else
+      EXPECT_EQ(test.convertToFloat(), test2.convertToFloat());
+
+    // bitcastToAPInt
+    EXPECT_EQ(i, test.bitcastToAPInt());
+  }
+}
+
+TEST(APFloatTest, Float8ExhaustivePair) {
+  // Test each pair of 8-bit floats with non-standard semantics
+  for (APFloat::Semantics Sem :
+       {APFloat::S_Float8E4M3FN, APFloat::S_Float8E5M2FNUZ,
+        APFloat::S_Float8E4M3FNUZ}) {
+    const llvm::fltSemantics &S = APFloat::EnumToSemantics(Sem);
     for (int i = 0; i < 256; i++) {
       for (int j = 0; j < 256; j++) {
-      SCOPED_TRACE("i=" + std::to_string(i) + ",j=" + std::to_string(j));
-      APFloat x(APFloat::Float8E4M3FN(), APInt(8, i));
-      APFloat y(APFloat::Float8E4M3FN(), APInt(8, j));
+        SCOPED_TRACE("sem=" + std::to_string(Sem) + ",i=" + std::to_string(i) +
+                     ",j=" + std::to_string(j));
+        APFloat x(S, APInt(8, i));
+        APFloat y(S, APInt(8, j));
 
         bool losesInfo;
         APFloat x16 = x;
         x16.convert(APFloat::IEEEhalf(), APFloat::rmNearestTiesToEven,
                     &losesInfo);
         EXPECT_FALSE(losesInfo);
         APFloat y16 = y;
         y16.convert(APFloat::IEEEhalf(), APFloat::rmNearestTiesToEven,
                     &losesInfo);
         EXPECT_FALSE(losesInfo);
 
         // Add
         APFloat z = x;
         z.add(y, APFloat::rmNearestTiesToEven);
         APFloat z16 = x16;
         z16.add(y16, APFloat::rmNearestTiesToEven);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
-                  &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16));
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
 
         // Subtract
         z = x;
         z.subtract(y, APFloat::rmNearestTiesToEven);
         z16 = x16;
         z16.subtract(y16, APFloat::rmNearestTiesToEven);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
-                  &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16));
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
 
         // Multiply
         z = x;
         z.multiply(y, APFloat::rmNearestTiesToEven);
         z16 = x16;
         z16.multiply(y16, APFloat::rmNearestTiesToEven);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
-                  &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16)) << "i=" << i << ", j=" << j;
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
 
         // Divide
         z = x;
         z.divide(y, APFloat::rmNearestTiesToEven);
         z16 = x16;
         z16.divide(y16, APFloat::rmNearestTiesToEven);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
-                  &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16)) << "i=" << i << ", j=" << j;
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
 
         // Mod
         z = x;
         z.mod(y);
         z16 = x16;
         z16.mod(y16);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
-                  &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16)) << "i=" << i << ", j=" << j;
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
 
         // Remainder
         z = x;
         z.remainder(y);
         z16 = x16;
         z16.remainder(y16);
-      z16.convert(APFloat::Float8E4M3FN(), APFloat::rmNearestTiesToEven,
+        z16.convert(S, APFloat::rmNearestTiesToEven, &losesInfo);
+        EXPECT_TRUE(z.bitwiseIsEqual(z16))
+            << "sem=" << Sem << ", i=" << i << ", j=" << j;
+      }
+    }
+  }
+}
+
+TEST(APFloatTest, ConvertE5M2FNUZToE4M3FNUZ) {
+  bool losesInfo;
+  APFloat test(APFloat::Float8E5M2FNUZ(), "1.0");
+  APFloat::opStatus status = test.convert(
+      APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven, &losesInfo);
+  EXPECT_EQ(1.0f, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+  losesInfo = true;
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0.0");
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0.0f, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+  losesInfo = true;
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0x1.Cp7"); // 224
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0x1.Cp7 /* 224 */, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+  // Test overflow
+  losesInfo = false;
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0x1.0p8"); // 256
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_TRUE(std::isnan(test.convertToFloat()));
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOverflow | APFloat::opInexact);
+
+  // Test underflow
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0x1.0p-11");
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0., test.convertToFloat());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opUnderflow | APFloat::opInexact);
+
+  // Test rounding up to smallest denormal number
+  losesInfo = false;
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0x1.8p-11");
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0x1.0p-10, test.convertToFloat());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opUnderflow | APFloat::opInexact);
+
+  // Testing inexact rounding to denormal number
+  losesInfo = false;
+  test = APFloat(APFloat::Float8E5M2FNUZ(), "0x1.8p-10");
+  status = test.convert(APFloat::Float8E4M3FNUZ(), APFloat::rmNearestTiesToEven,
                         &losesInfo);
-      EXPECT_TRUE(z.bitwiseIsEqual(z16)) << "i=" << i << ", j=" << j;
-    }
-  }
+  EXPECT_EQ(0x1.0p-9, test.convertToFloat());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opUnderflow | APFloat::opInexact);
+
+TEST(APFloatTest, ConvertE4M3FNUZToE5M2FNUZ) {
+  bool losesInfo;
+  APFloat test(APFloat::Float8E4M3FNUZ(), "1.0");
+  APFloat::opStatus status = test.convert(
+      APFloat::Float8E5M2FNUZ(), APFloat::rmNearestTiesToEven, &losesInfo);
+  EXPECT_EQ(1.0f, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+  losesInfo = true;
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "0.0");
+  status = test.convert(APFloat::Float8E5M2FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0.0f, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+  losesInfo = false;
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "0x1.2p0"); // 1.125
+  status = test.convert(APFloat::Float8E5M2FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0x1.0p0 /* 1.0 */, test.convertToFloat());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opInexact);
+
+  losesInfo = false;
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "0x1.6p0"); // 1.375
+  status = test.convert(APFloat::Float8E5M2FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0x1.8p0 /* 1.5 */, test.convertToFloat());
+  EXPECT_TRUE(losesInfo);
+  EXPECT_EQ(status, APFloat::opInexact);
+
+  // Convert E4M3 denormal to E5M2 normal. Should not be truncated, despite the
+  // destination format having one fewer significand bit
+  losesInfo = true;
+  test = APFloat(APFloat::Float8E4M3FNUZ(), "0x1.Cp-8");
+  status = test.convert(APFloat::Float8E5M2FNUZ(), APFloat::rmNearestTiesToEven,
+                        &losesInfo);
+  EXPECT_EQ(0x1.Cp-8, test.convertToFloat());
+  EXPECT_FALSE(losesInfo);
+  EXPECT_EQ(status, APFloat::opOK);
+
+TEST(APFloatTest, Float8E4M3FNUZFromString) {
+  // Exactly representable
+  EXPECT_EQ(240, APFloat(APFloat::Float8E4M3FNUZ(), "240").convertToDouble());
+  // Round down to maximum value
+  EXPECT_EQ(240, APFloat(APFloat::Float8E4M3FNUZ(), "247").convertToDouble());
+  // Round up, causing overflow to NaN
+  EXPECT_TRUE(APFloat(APFloat::Float8E4M3FNUZ(), "248").isNaN());
+  // Overflow without rounding
+  EXPECT_TRUE(APFloat(APFloat::Float8E4M3FNUZ(), "256").isNaN());
+  // Inf converted to NaN
+  EXPECT_TRUE(APFloat(APFloat::Float8E4M3FNUZ(), "inf").isNaN());
+  // NaN converted to NaN
+  EXPECT_TRUE(APFloat(APFloat::Float8E4M3FNUZ(), "nan").isNaN());
+  // Negative zero converted to positive zero
+  EXPECT_TRUE(APFloat(APFloat::Float8E4M3FNUZ(), "-0").isPosZero());
 }
 
 TEST(APFloatTest, F8ToString) {
   for (APFloat::Semantics S :
-       {APFloat::S_Float8E5M2, APFloat::S_Float8E4M3FN}) {
+       {APFloat::S_Float8E5M2, APFloat::S_Float8E4M3FN,
+        APFloat::S_Float8E5M2FNUZ, APFloat::S_Float8E4M3FNUZ}) {
     SCOPED_TRACE("Semantics=" + std::to_string(S));
     for (int i = 0; i < 256; i++) {
       SCOPED_TRACE("i=" + std::to_string(i));
-      APFloat test(APFloat::Float8E5M2(), APInt(8, i));
+      APFloat test(APFloat::EnumToSemantics(S), APInt(8, i));
       llvm::SmallString<128> str;
       test.toString(str);
 
       if (test.isNaN()) {
         EXPECT_EQ(str, "NaN");
       } else {
-        APFloat test2(APFloat::Float8E5M2(), str);
+        APFloat test2(APFloat::EnumToSemantics(S), str);
         EXPECT_TRUE(test.bitwiseIsEqual(test2));
       }
     }
   }
 }
 
 TEST(APFloatTest, IEEEdoubleToDouble) {
   APFloat DPosZero(0.0);
TEST(APFloatTest, Float8E4M3FNToDouble) {
   APFloat SmallestDenorm = APFloat::getSmallest(APFloat::Float8E4M3FN(), false);
   EXPECT_TRUE(SmallestDenorm.isDenormal());
   EXPECT_EQ(0x1p-9, SmallestDenorm.convertToDouble());
 
   APFloat QNaN = APFloat::getQNaN(APFloat::Float8E4M3FN());
   EXPECT_TRUE(std::isnan(QNaN.convertToDouble()));
 }
 
+TEST(APFloatTest, Float8E5M2FNUZToDouble) {
+  APFloat One(APFloat::Float8E5M2FNUZ(), "1.0");
+  EXPECT_EQ(1.0, One.convertToDouble());
+  APFloat Two(APFloat::Float8E5M2FNUZ(), "2.0");
+  EXPECT_EQ(2.0, Two.convertToDouble());
+  APFloat PosLargest = APFloat::getLargest(APFloat::Float8E5M2FNUZ(), false);
+  EXPECT_EQ(57344., PosLargest.convertToDouble());
+  APFloat NegLargest = APFloat::getLargest(APFloat::Float8E5M2FNUZ(), true);
+  EXPECT_EQ(-57344., NegLargest.convertToDouble());
+  APFloat PosSmallest =
+      APFloat::getSmallestNormalized(APFloat::Float8E5M2FNUZ(), false);
+  EXPECT_EQ(0x1.p-15, PosSmallest.convertToDouble());
+  APFloat NegSmallest =
+      APFloat::getSmallestNormalized(APFloat::Float8E5M2FNUZ(), true);
+  EXPECT_EQ(-0x1.p-15, NegSmallest.convertToDouble());
+
+  APFloat SmallestDenorm =
+      APFloat::getSmallest(APFloat::Float8E5M2FNUZ(), false);
+  EXPECT_TRUE(SmallestDenorm.isDenormal());
+  EXPECT_EQ(0x1p-17, SmallestDenorm.convertToDouble());
+
+  APFloat QNaN = APFloat::getQNaN(APFloat::Float8E5M2FNUZ());
+  EXPECT_TRUE(std::isnan(QNaN.convertToDouble()));
+}
+
+TEST(APFloatTest, Float8E4M3FNUZToDouble) {
+  APFloat One(APFloat::Float8E4M3FNUZ(), "1.0");
+  EXPECT_EQ(1.0, One.convertToDouble());
+  APFloat Two(APFloat::Float8E4M3FNUZ(), "2.0");
+  EXPECT_EQ(2.0, Two.convertToDouble());
+  APFloat PosLargest = APFloat::getLargest(APFloat::Float8E4M3FNUZ(), false);
+  EXPECT_EQ(240., PosLargest.convertToDouble());
+  APFloat NegLargest = APFloat::getLargest(APFloat::Float8E4M3FNUZ(), true);
+  EXPECT_EQ(-240., NegLargest.convertToDouble());
+  APFloat PosSmallest =
+      APFloat::getSmallestNormalized(APFloat::Float8E4M3FNUZ(), false);
+  EXPECT_EQ(0x1.p-7, PosSmallest.convertToDouble());
+  APFloat NegSmallest =
+      APFloat::getSmallestNormalized(APFloat::Float8E4M3FNUZ(), true);
+  EXPECT_EQ(-0x1.p-7, NegSmallest.convertToDouble());
+
+  APFloat SmallestDenorm =
+      APFloat::getSmallest(APFloat::Float8E4M3FNUZ(), false);
+  EXPECT_TRUE(SmallestDenorm.isDenormal());
+  EXPECT_EQ(0x1p-10, SmallestDenorm.convertToDouble());
+
+  APFloat QNaN = APFloat::getQNaN(APFloat::Float8E4M3FNUZ());
+  EXPECT_TRUE(std::isnan(QNaN.convertToDouble()));
+}
+
 TEST(APFloatTest, IEEEsingleToFloat) {
   APFloat FPosZero(0.0F);
   APFloat FPosZeroToFloat(FPosZero.convertToFloat());
   EXPECT_TRUE(FPosZeroToFloat.isPosZero());
   APFloat FNegZero(-0.0F);
   APFloat FNegZeroToFloat(FNegZero.convertToFloat());
   EXPECT_TRUE(FNegZeroToFloat.isNegZero());