Implement some constexpr vector unary operators, fix boolean-ops

[erichkeane]

Summary

As requested in the review, this implements unary +,-,~, and ! for
vector types.

All of our boolean operations on vector types should be using something
like vcmpeqd, which results in a mask of '-1' for the 'truth' type. We are
currently instead using '1', which results in some incorrect
calculations when used later (note that it does NOT result in a boolean
vector, as that is not really a thing).

This patch corrects that 1 to be a -1, and updates the affected tests.

[erichkeane]

Note I am adding the folks who were added as reviewers the last time I did vector constexpr work: https://reviews.llvm.org/D79755

I again looked into operator[] to simplify things, but the 'LValueBase' stuff (seemingly required to do something like: VecTy[3] = 1;) is a little convoluted/tough to figure out and might be a fairly sizable 'touch'.

[efriedma]

clang/lib/AST/ExprConstant.cpp

2915

Using "operator=" to assign an int to an APInt is going to lead to weird/confusing results. For example, I think this produces the wrong result if you have an int128_t vector.

[erichkeane]

clang/lib/AST/ExprConstant.cpp

2915

Can you clarify your comment? The result type of one of these compare-ops is either an i8 or an i32 I think, so I would expect it to 'just work', right?

What problem of operator= on an APInt gives the wrong result? I guess I've never seen/don't know how to repro what you mean.

I DID try to get a test wtih a 'compare op' on an __int128 vector, but that ends up asserting in GetSignedVectorType

[erichkeane]

To add:
I DID just try to fix that thing:

[ekeane1@scsel-clx-24 clang]$ git diff
diff --git a/clang/lib/Sema/SemaExpr.cpp b/clang/lib/Sema/SemaExpr.cpp
index 2b69c3727852..330772d2b10a 100644
--- a/clang/lib/Sema/SemaExpr.cpp
+++ b/clang/lib/Sema/SemaExpr.cpp
@@ -12265,6 +12265,8 @@ QualType Sema::GetSignedVectorType(QualType V) {
       return Context.getExtVectorType(Context.ShortTy, VTy->getNumElements());
     else if (TypeSize == Context.getTypeSize(Context.IntTy))
       return Context.getExtVectorType(Context.IntTy, VTy->getNumElements());
+    else if (TypeSize == Context.getTypeSize(Context.Int128Ty))
+      return Context.getExtVectorType(Context.Int128Ty, VTy->getNumElements());
     else if (TypeSize == Context.getTypeSize(Context.LongTy))
       return Context.getExtVectorType(Context.LongTy, VTy->getNumElements());
     assert(TypeSize == Context.getTypeSize(Context.LongLongTy) &&
@@ -12272,7 +12274,10 @@ QualType Sema::GetSignedVectorType(QualType V) {
     return Context.getExtVectorType(Context.LongLongTy, VTy->getNumElements());
   }

-  if (TypeSize == Context.getTypeSize(Context.LongLongTy))
+  if (TypeSize == Context.getTypeSize(Context.Int128Ty))
+    return Context.getVectorType(Context.Int128Ty, VTy->getNumElements(),
+                                 VectorType::GenericVector);
+  else if (TypeSize == Context.getTypeSize(Context.LongLongTy))
     return Context.getVectorType(Context.LongLongTy, VTy->getNumElements(),
                                  VectorType::GenericVector);
   else if (TypeSize == Context.getTypeSize(Context.LongTy))
[ekeane1@scsel-clx-24 clang]$

I then compiled:

using VecTy = __int128 __attribute__((vector_size(32)));
constexpr auto F() {
  VecTy V{1,2};

  return V <= 2;
}
auto bar() {
  constexpr auto Var = F();
  return Var;
}

And got:

; Function Attrs: mustprogress noinline nounwind optnone
define dso_local <2 x i128> @_Z3barv() #0 {
entry:
  %Var = alloca <2 x i128>, align 32
  store <2 x i128> <i128 18446744073709551615, i128 18446744073709551615>, <2 x i128>* %Var, align 32
  ret <2 x i128> <i128 18446744073709551615, i128 18446744073709551615>
}

I believe that is the uint64 'max value', right? So would that would be what you mean?

[craig.topper]

clang/lib/AST/ExprConstant.cpp

2915

Don't compare ops produce a result with the same number of bits as the input? If the compare is a vector of int128 the result should also be a vector of int128. APInt::operator= takes a uint64_t as input and will zero extend it to 128 bits. I think we want to sign extend here.

I assume the APInt for result already been given its bit width before this call?

[craig.topper]

clang/lib/AST/ExprConstant.cpp

2915

The easy fix is probably just to do the assignment like it was before and then do Result.negate().

[erichkeane]

clang/lib/AST/ExprConstant.cpp

2915

Yes, the APInt should already be the right size. And you do seem correct about the width being the size of the elements, so you were right about that.

THanks for the tip on the .negate, I can just do that!

[erichkeane]

Enable __int128 vectors, and fix the issue of the >64 bit vectors for negation. Also add a test for these.

[erichkeane]

Note, my pre-merge check failure seems completely unrelated. It seems to have some problem with the some 'go' bindings, but I don't believe that has anything to do with this change.

[aaron.ballman]

clang/lib/AST/ExprConstant.cpp

10389

Would it be worth handling ~ and ! as well given that they seem pretty trivial to support? I don't insist, but I'm trying to reason whether the for loop should be hoisted out of the switch because it seems like it'll be needed for all of the cases.

clang/lib/Sema/SemaExpr.cpp

12262–12287

NFC nit: a whole pile of else after return that can be removed someday.

[erichkeane]

clang/lib/AST/ExprConstant.cpp

10389

Yep, I can take a look at that. Note the rest of the unary operators are all either already handled (real/imag), or require switching this over to an LValueBase/etc as they are modifying, which I don't understand. But those two should be pretty easy to do.

[craig.topper]

clang/lib/AST/ExprConstant.cpp

10393

Can this be Elt.getInt().negate()?

10397

Can this be Elt.getFloat().changeSign()?

[erichkeane]

@craig.topper : Yep, probably so in both those cases. I'm working on a patch that includes the 2 'nots' as well, which complicates this whole section a bunch though.

[erichkeane]

Implement the other two easily doable operators, do Craigs suggestions.

[aaron.ballman]

clang/lib/AST/ExprConstant.cpp

10188

Comment can be updated as we're not missing ~ any longer.

10399

operator!, right?

10405–10406

-1 for true and 0 for false (just to be super clear).

[craig.topper]

clang/lib/AST/ExprConstant.cpp

10188

Drop "unary ~" since you handled it?

10373

Lower case per coding standards? Put "handle" first since it is the verb. We seem to be inconcistent about where we put "Vector", we have handleVectorVectorBinOp and handleCompareOpForVector.

10395

Would APInt.getInt().flipAllbits() be better than re-creating an APValue?

clang/lib/Sema/SemaExpr.cpp

12264

Thanks for fixing the else after returns here. I almost commented on that earlier.

[erichkeane]

Fix Aaron+ Craig's comments.

[aaron.ballman]

LGTM, but I leave the final sign-off about the behavior to others.

[erichkeane]

Note: I think I've done everything requested, so I'm hoping to commit this tomorrow 1st thing in order to have it in time for everyone's vacations (and so my downstream can get it before the new year), unless I hear objections from the other reviewers.

[qiongsiwu1]

Hi everyone! We are encountering crashes in some of our altivec test cases due to this change. clang crashes when we have an instance of a struct or union type, and we assign the result of a unary operator ++ or -- with a vector operand to a field of the instance. If we are assigning the result to just a vector type, the compiler does not crash.

Here is a reproducer. To reproduce the crash, use command clang -target powerpc-unknown-unknown -maltivec small.c.

// small.c
#include <stdio.h>

// struct type
typedef struct result
{
    vector signed short result_vec;
} RESULT;

vector signed short op_nine = { 9, 9, 9, 9, 9, 9, 9, 9 };

int main() {
    // Instance of the struct type
    RESULT r;

    // This line below crashes - we are assigning the result of the unary operator to a struct field. 
    r.result_vec = ++op_nine;
    for (int i = 0; i < 8; ++i) {
        printf("%d\t", r.result_vec[i]);
    }
    printf("\n");

    // This case works fine. The compiler does not crash. 
    /*vector signed short op_10 = ++op_nine;
    for (int i = 0; i < 8; ++i) {
        printf("%d\t", op_10[i]);
    }
    printf("\n"); */
}

We discovered this on AIX and we can reproduce this on Linux on Power. The issue is reproducible on Intel based Macs as well. Here is a stack dump from MacOS.

Assertion failed: (isVector() && "Invalid accessor"), function getVectorLength, file /Users/qiongsiwu/workspace/community/llvm-project/clang/include/clang/AST/APValue.h, line 498.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /Users/qiongsiwu/workspace/community/llvm-project/build/bin/clang-14 -cc1 -triple powerpc-unknown-unknown -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -main-file-name small.c -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu ppc -target-feature +altivec -mfloat-abi hard -debugger-tuning=gdb -target-linker-version 650.9 -fcoverage-compilation-dir=/Users/qiongsiwu/workspace/13359 -resource-dir /Users/qiongsiwu/workspace/community/llvm-project/build/lib/clang/14.0.0 -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/ -fdebug-compilation-dir=/Users/qiongsiwu/workspace/13359 -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/gh/5hw84m4x2hv4csmhpp8y02680000gn/T/small-e9e02b.o -x c small.c
1.	small.c:23:5: current parser token 'return'
2.	small.c:12:12: parsing function body 'main'
3.	small.c:12:12: in compound statement ('{}')
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  clang-14                 0x0000000107d8d1f7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 39
1  clang-14                 0x0000000107d8bfb8 llvm::sys::RunSignalHandlers() + 248
2  clang-14                 0x0000000107d8d840 SignalHandler(int) + 272
3  libsystem_platform.dylib 0x00007ff80662fe2d _sigtramp + 29
4  libsystem_platform.dylib 0x00007ff7ba7632d0 _sigtramp + 18446744072435741888
5  libsystem_c.dylib        0x00007ff806566d10 abort + 123
6  libsystem_c.dylib        0x00007ff8065660be err + 0
7  clang-14                 0x000000010c208443 (anonymous namespace)::VectorExprEvaluator::VisitUnaryOperator(clang::UnaryOperator const*) (.cold.13) + 35
8  clang-14                 0x000000010a40b1d7 (anonymous namespace)::VectorExprEvaluator::VisitUnaryOperator(clang::UnaryOperator const*) + 2375

On AIX, the compiler can produce the following AST during EvaluateVector before the crash.

[EvaluateVector]
BinaryOperator 0x11173e148 '__vector short' '='
|-MemberExpr 0x11173e0e0 '__vector short' lvalue .result_vec 0x111739f40
| `-DeclRefExpr 0x11173a5a0 'RESULT':'struct result' lvalue Var 0x11173a520 'r' 'RESULT':'struct result'
`-UnaryOperator 0x11173e130 '__vector short' prefix '++'
  `-DeclRefExpr 0x11173e110 '__vector short' lvalue Var 0x11173a070 'op_nine' '__vector short'
[EvaluateVector]
UnaryOperator 0x11173e130 '__vector short' prefix '++'
`-DeclRefExpr 0x11173e110 '__vector short' lvalue Var 0x11173a070 'op_nine' '__vector short'

The compiler crashes when trying to deal with 0x11173a070. It is expecting a vector kind (APValue::ValueKind::Vector), but the actual kind is APValue::ValueKind::LValue, and the compiler crashes inside SubExprValue.getVectorLength().

Could you help us take a look? Thanks a lot! FYI @bmahjour @anhtuyen

[erichkeane]

Hi!
Sorry for the delay, I just got back from my christmas break. I'll take a look later today/this week.

[erichkeane]

Should be fixed here: 2edc21e8566be8fa9b20e0bb71a83af90ec9aa97

Thanks!

[qiongsiwu1]

In D115670#3220052, @erichkeane wrote:

Should be fixed here: 2edc21e8566be8fa9b20e0bb71a83af90ec9aa97

Thanks!

Thanks so much for the fix Erich! I can confirm the altivec test cases are all passing now.

clang/lib/AST/ExprConstant.cpp

 template <typename APTy>
 static bool
 handleCompareOpForVectorHelper(const APTy &LHSValue, BinaryOperatorKind Opcode,
                                const APTy &RHSValue, APInt &Result) {
   switch (Opcode) {
   default:
     llvm_unreachable("unsupported binary operator");
   case BO_EQ:
     Result = (LHSValue == RHSValue);
     break;
   case BO_NE:
     Result = (LHSValue != RHSValue);
     break;
   case BO_LT:
     Result = (LHSValue < RHSValue);
     break;
   case BO_GT:
     Result = (LHSValue > RHSValue);
     break;
   case BO_LE:
     Result = (LHSValue <= RHSValue);
     break;
   case BO_GE:
     Result = (LHSValue >= RHSValue);
     break;
   }
 
+  // The boolean operations on these vector types use an instruction that
+  // results in a mask of '-1' for the 'truth' value.  Ensure that we negate 1
+  // to -1 to make sure that we produce the correct value.
+  Result.negate();
+
   return true;
 }
 
 static bool handleCompareOpForVector(const APValue &LHSValue,
                                      BinaryOperatorKind Opcode,
                                      const APValue &RHSValue, APInt &Result) {
   // The result is always an int type, however operands match the first.
   if (LHSValue.getKind() == APValue::Int)
public:
     bool ZeroInitialization(const Expr *E);
 
     bool VisitUnaryReal(const UnaryOperator *E)
       { return Visit(E->getSubExpr()); }
     bool VisitCastExpr(const CastExpr* E);
     bool VisitInitListExpr(const InitListExpr *E);
     bool VisitUnaryImag(const UnaryOperator *E);
     bool VisitBinaryOperator(const BinaryOperator *E);
-    // FIXME: Missing: unary -, unary ~, conditional operator (for GNU
+    bool VisitUnaryOperator(const UnaryOperator *E);
+    // FIXME: Missing: conditional operator (for GNU
     //                 conditional select), shufflevector, ExtVectorElementExpr
   };
 } // end anonymous namespace
 
 static bool EvaluateVector(const Expr* E, APValue& Result, EvalInfo &Info) {
   assert(E->isPRValue() && E->getType()->isVectorType() &&
          "not a vector prvalue");
   return VectorExprEvaluator(Info, Result).Visit(E);
if (!Evaluate(RHSValue, Info, RHS) || !LHSOK)
     return false;
 
   if (!handleVectorVectorBinOp(Info, E, Op, LHSValue, RHSValue))
     return false;
 
   return Success(LHSValue, E);
 }
 
+static llvm::Optional<APValue> handleVectorUnaryOperator(ASTContext &Ctx,
+                                                         QualType ResultTy,
+                                                         UnaryOperatorKind Op,
+                                                         APValue Elt) {
+  switch (Op) {
+  case UO_Plus:
+    // Nothing to do here.
+    return Elt;
+  case UO_Minus:
+    if (Elt.getKind() == APValue::Int) {
+      Elt.getInt().negate();
+    } else {
+      assert(Elt.getKind() == APValue::Float &&
+             "Vector can only be int or float type");
+      Elt.getFloat().changeSign();
+    }
+    return Elt;
+  case UO_Not:
+    // This is only valid for integral types anyway, so we don't have to handle
+    // float here.
+    assert(Elt.getKind() == APValue::Int &&
+           "Vector operator ~ can only be int");
+    Elt.getInt().flipAllBits();
+    return Elt;
+  case UO_LNot: {
+    if (Elt.getKind() == APValue::Int) {
+      Elt.getInt() = !Elt.getInt();
+      // operator ! on vectors returns -1 for 'truth', so negate it.
+      Elt.getInt().negate();
+      return Elt;
+    }
+    assert(Elt.getKind() == APValue::Float &&
+           "Vector can only be int or float type");
+    // Float types result in an int of the same size, but -1 for true, or 0 for
+    // false.
+    APSInt EltResult{Ctx.getIntWidth(ResultTy),
+                     ResultTy->isUnsignedIntegerType()};
+    if (Elt.getFloat().isZero())
+      EltResult.setAllBits();
+    else
+      EltResult.clearAllBits();
+
+    return APValue{EltResult};
+  }
+  default:
+    // FIXME: Implement the rest of the unary operators.
+    return llvm::None;
+  }
+}
+
+bool VectorExprEvaluator::VisitUnaryOperator(const UnaryOperator *E) {
+  Expr *SubExpr = E->getSubExpr();
+  const auto *VD = SubExpr->getType()->castAs<VectorType>();
+  // This result element type differs in the case of negating a floating point
+  // vector, since the result type is the a vector of the equivilant sized
+  // integer.
+  const QualType ResultEltTy = VD->getElementType();
+  UnaryOperatorKind Op = E->getOpcode();
+
+  APValue SubExprValue;
+  if (!Evaluate(SubExprValue, Info, SubExpr))
+    return false;
+
+  assert(SubExprValue.getVectorLength() == VD->getNumElements() &&
+         "Vector length doesn't match type?");
+
+  SmallVector<APValue, 4> ResultElements;
+  for (unsigned EltNum = 0; EltNum < VD->getNumElements(); ++EltNum) {
+    llvm::Optional<APValue> Elt = handleVectorUnaryOperator(
+        Info.Ctx, ResultEltTy, Op, SubExprValue.getVectorElt(EltNum));
+    if (!Elt)
+      return false;
+    ResultElements.push_back(*Elt);
+  }
+  return Success(APValue(ResultElements.data(), ResultElements.size()), E);
+}
+
 //===----------------------------------------------------------------------===//
 // Array Evaluation
 //===----------------------------------------------------------------------===//
 
 namespace {
   class ArrayExprEvaluator
   : public ExprEvaluatorBase<ArrayExprEvaluator> {
     const LValue &This;

clang/lib/Sema/SemaExpr.cpp

 // size and number of elements. In the non ext_vector_type case, search from
 // the largest type to the smallest type to avoid cases where long long == long,
 // where long gets picked over long long.
 QualType Sema::GetSignedVectorType(QualType V) {
   const VectorType *VTy = V->castAs<VectorType>();
   unsigned TypeSize = Context.getTypeSize(VTy->getElementType());
 
   if (isa<ExtVectorType>(VTy)) {
     if (TypeSize == Context.getTypeSize(Context.CharTy))
       return Context.getExtVectorType(Context.CharTy, VTy->getNumElements());
-    else if (TypeSize == Context.getTypeSize(Context.ShortTy))
+    if (TypeSize == Context.getTypeSize(Context.ShortTy))
       return Context.getExtVectorType(Context.ShortTy, VTy->getNumElements());
-    else if (TypeSize == Context.getTypeSize(Context.IntTy))
+    if (TypeSize == Context.getTypeSize(Context.IntTy))
       return Context.getExtVectorType(Context.IntTy, VTy->getNumElements());
-    else if (TypeSize == Context.getTypeSize(Context.LongTy))
+    if (TypeSize == Context.getTypeSize(Context.Int128Ty))
+      return Context.getExtVectorType(Context.Int128Ty, VTy->getNumElements());
+    if (TypeSize == Context.getTypeSize(Context.LongTy))
       return Context.getExtVectorType(Context.LongTy, VTy->getNumElements());
     assert(TypeSize == Context.getTypeSize(Context.LongLongTy) &&
            "Unhandled vector element size in vector compare");
     return Context.getExtVectorType(Context.LongLongTy, VTy->getNumElements());
   }
 
+  if (TypeSize == Context.getTypeSize(Context.Int128Ty))
+    return Context.getVectorType(Context.Int128Ty, VTy->getNumElements(),
+                                 VectorType::GenericVector);
   if (TypeSize == Context.getTypeSize(Context.LongLongTy))
     return Context.getVectorType(Context.LongLongTy, VTy->getNumElements(),
                                  VectorType::GenericVector);
-  else if (TypeSize == Context.getTypeSize(Context.LongTy))
+  if (TypeSize == Context.getTypeSize(Context.LongTy))
     return Context.getVectorType(Context.LongTy, VTy->getNumElements(),
                                  VectorType::GenericVector);
-  else if (TypeSize == Context.getTypeSize(Context.IntTy))
+  if (TypeSize == Context.getTypeSize(Context.IntTy))
     return Context.getVectorType(Context.IntTy, VTy->getNumElements(),
                                  VectorType::GenericVector);
-  else if (TypeSize == Context.getTypeSize(Context.ShortTy))
+  if (TypeSize == Context.getTypeSize(Context.ShortTy))
     return Context.getVectorType(Context.ShortTy, VTy->getNumElements(),
                                  VectorType::GenericVector);
   assert(TypeSize == Context.getTypeSize(Context.CharTy) &&
          "Unhandled vector element size in vector compare");
   return Context.getVectorType(Context.CharTy, VTy->getNumElements(),
                                VectorType::GenericVector);
 }
 

clang/test/SemaCXX/constexpr-vectors.cpp

 // RUN: %clang_cc1 -std=c++14 -Wno-unused-value %s -disable-llvm-passes -triple x86_64-linux-gnu -emit-llvm -o - | FileCheck %s
 
 // FIXME: Unfortunately there is no good way to validate that our values are
 // correct since Vector types don't have operator [] implemented for constexpr.
 // Instead, we need to use filecheck to ensure the emitted IR is correct. Once
 // someone implements array subscript operator for these types as constexpr,
 // this test should modified to jsut use static asserts.
 
 using FourCharsVecSize __attribute__((vector_size(4))) = char;
 using FourIntsVecSize __attribute__((vector_size(16))) = int;
 using FourLongLongsVecSize __attribute__((vector_size(32))) = long long;
 using FourFloatsVecSize __attribute__((vector_size(16))) = float;
 using FourDoublesVecSize __attribute__((vector_size(32))) = double;
+using FourI128VecSize __attribute__((vector_size(64))) = __int128;
 
 using FourCharsExtVec __attribute__((ext_vector_type(4))) = char;
 using FourIntsExtVec __attribute__((ext_vector_type(4))) = int;
 using FourLongLongsExtVec __attribute__((ext_vector_type(4))) = long long;
 using FourFloatsExtVec __attribute__((ext_vector_type(4))) = float;
 using FourDoublesExtVec __attribute__((ext_vector_type(4))) = double;
+using FourI128ExtVec __attribute__((ext_vector_type(4))) = __int128;
+
 
 // Next a series of tests to make sure these operations are usable in
 // constexpr functions. Template instantiations don't emit Winvalid-constexpr,
 // so we have to do these as macros.
 #define MathShiftOps(Type)                            \
   constexpr auto MathShiftOps##Type(Type a, Type b) { \
     a = a + b;                                        \
     a = a - b;                                        \
void CharUsage() {
   // CHECK: store <4 x i8> <i8 9, i8 7, i8 5, i8 10>
   constexpr auto u = 12 << FourCharsVecSize{1, 2, 3, 3};
   // CHECK: store <4 x i8> <i8 24, i8 48, i8 96, i8 96>
   constexpr auto v = 12 >> FourCharsVecSize{1, 2, 2, 1};
   // CHECK: store <4 x i8> <i8 6, i8 3, i8 3, i8 6>
 
   constexpr auto w = FourCharsVecSize{1, 2, 3, 4} <
                      FourCharsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 0>
   constexpr auto x = FourCharsVecSize{1, 2, 3, 4} >
                      FourCharsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto y = FourCharsVecSize{1, 2, 3, 4} <=
                      FourCharsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 -1, i8 0>
   constexpr auto z = FourCharsVecSize{1, 2, 3, 4} >=
                      FourCharsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto A = FourCharsVecSize{1, 2, 3, 4} ==
                      FourCharsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 0>
   constexpr auto B = FourCharsVecSize{1, 2, 3, 4} !=
                      FourCharsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 -1>
 
   constexpr auto C = FourCharsVecSize{1, 2, 3, 4} < 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 0>
   constexpr auto D = FourCharsVecSize{1, 2, 3, 4} > 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 0, i8 -1>
   constexpr auto E = FourCharsVecSize{1, 2, 3, 4} <= 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 -1, i8 0>
   constexpr auto F = FourCharsVecSize{1, 2, 3, 4} >= 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto G = FourCharsVecSize{1, 2, 3, 4} == 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 0>
   constexpr auto H = FourCharsVecSize{1, 2, 3, 4} != 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 -1>
 
   constexpr auto I = FourCharsVecSize{1, 2, 3, 4} &
                      FourCharsVecSize{4, 3, 2, 1};
   // CHECK: store <4 x i8> <i8 0, i8 2, i8 2, i8 0>
   constexpr auto J = FourCharsVecSize{1, 2, 3, 4} ^
                      FourCharsVecSize { 4, 3, 2, 1 };
   // CHECK: store <4 x i8> <i8 5, i8 1, i8 1, i8 5>
   constexpr auto K = FourCharsVecSize{1, 2, 3, 4} |
void CharUsage() {
   // CHECK: store <4 x i8> <i8 0, i8 0, i8 7, i8 8>
 
   constexpr auto X = CmpAdd(a, b);
   // CHECK: store <4 x i8> <i8 24, i8 19, i8 15, i8 17>
 
   constexpr auto Y = CmpSub(a, b);
   // CHECK: store <4 x i8> <i8 12, i8 17, i8 -1, i8 -1>
 
-  constexpr auto Z = CmpLSH(a, H);
+  constexpr auto InvH = -H;
+  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+  constexpr auto Z = CmpLSH(a, InvH);
   // CHECK: store <4 x i8> <i8 36, i8 36, i8 7, i8 16>
 
-  constexpr auto aa = CmpRSH(a, H);
+  constexpr auto aa = CmpRSH(a, InvH);
   // CHECK: store <4 x i8> <i8 9, i8 9, i8 7, i8 4>
 
   constexpr auto ab = CmpBinAnd(a, b);
   // CHECK: store <4 x i8> <i8 2, i8 0, i8 0, i8 8>
 
   constexpr auto ac = CmpBinXOr(a, b);
   // CHECK: store <4 x i8> <i8 20, i8 19, i8 15, i8 1>
 
   constexpr auto ad = CmpBinOr(a, b);
   // CHECK: store <4 x i8> <i8 22, i8 19, i8 15, i8 9>
+
+  constexpr auto ae = ~FourCharsVecSize{1, 2, 10, 20};
+  // CHECK: store <4 x i8> <i8 -2, i8 -3, i8 -11, i8 -21>
+
+  constexpr auto af = !FourCharsVecSize{0, 1, 8, -1};
+  // CHECK: store <4 x i8> <i8 -1, i8 0, i8 0, i8 0>
 }
 
 void CharExtVecUsage() {
   constexpr auto a = FourCharsExtVec{6, 3, 2, 1} +
                      FourCharsExtVec{12, 15, 5, 7};
   // CHECK: store <4 x i8> <i8 18, i8 18, i8 7, i8 8>
   constexpr auto b = FourCharsExtVec{19, 15, 13, 12} -
                      FourCharsExtVec{13, 14, 5, 3};
void CharExtVecUsage() {
   // CHECK: store <4 x i8> <i8 9, i8 7, i8 5, i8 10>
   constexpr auto u = 12 << FourCharsExtVec{1, 2, 3, 3};
   // CHECK: store <4 x i8> <i8 24, i8 48, i8 96, i8 96>
   constexpr auto v = 12 >> FourCharsExtVec{1, 2, 2, 1};
   // CHECK: store <4 x i8> <i8 6, i8 3, i8 3, i8 6>
 
   constexpr auto w = FourCharsExtVec{1, 2, 3, 4} <
                      FourCharsExtVec{4, 3, 2, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 0>
   constexpr auto x = FourCharsExtVec{1, 2, 3, 4} >
                      FourCharsExtVec{4, 3, 2, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto y = FourCharsExtVec{1, 2, 3, 4} <=
                      FourCharsExtVec{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 -1, i8 0>
   constexpr auto z = FourCharsExtVec{1, 2, 3, 4} >=
                      FourCharsExtVec{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto A = FourCharsExtVec{1, 2, 3, 4} ==
                      FourCharsExtVec{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 0>
   constexpr auto B = FourCharsExtVec{1, 2, 3, 4} !=
                      FourCharsExtVec{4, 3, 3, 1};
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 -1>
 
   constexpr auto C = FourCharsExtVec{1, 2, 3, 4} < 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 0>
   constexpr auto D = FourCharsExtVec{1, 2, 3, 4} > 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 0, i8 -1>
   constexpr auto E = FourCharsExtVec{1, 2, 3, 4} <= 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 -1, i8 0>
   constexpr auto F = FourCharsExtVec{1, 2, 3, 4} >= 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 1>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 -1>
   constexpr auto G = FourCharsExtVec{1, 2, 3, 4} == 3;
-  // CHECK: store <4 x i8> <i8 0, i8 0, i8 1, i8 0>
+  // CHECK: store <4 x i8> <i8 0, i8 0, i8 -1, i8 0>
   constexpr auto H = FourCharsExtVec{1, 2, 3, 4} != 3;
-  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+  // CHECK: store <4 x i8> <i8 -1, i8 -1, i8 0, i8 -1>
 
   constexpr auto I = FourCharsExtVec{1, 2, 3, 4} &
                      FourCharsExtVec{4, 3, 2, 1};
   // CHECK: store <4 x i8> <i8 0, i8 2, i8 2, i8 0>
   constexpr auto J = FourCharsExtVec{1, 2, 3, 4} ^
                      FourCharsExtVec { 4, 3, 2, 1 };
   // CHECK: store <4 x i8> <i8 5, i8 1, i8 1, i8 5>
   constexpr auto K = FourCharsExtVec{1, 2, 3, 4} |
void CharExtVecUsage() {
   // CHECK: store <4 x i8> <i8 0, i8 0, i8 7, i8 8>
 
   constexpr auto X = CmpAdd(a, b);
   // CHECK: store <4 x i8> <i8 24, i8 19, i8 15, i8 17>
 
   constexpr auto Y = CmpSub(a, b);
   // CHECK: store <4 x i8> <i8 12, i8 17, i8 -1, i8 -1>
 
-  constexpr auto Z = CmpLSH(a, H);
+  constexpr auto InvH = -H;
+  // CHECK: store <4 x i8> <i8 1, i8 1, i8 0, i8 1>
+
+  constexpr auto Z = CmpLSH(a, InvH);
   // CHECK: store <4 x i8> <i8 36, i8 36, i8 7, i8 16>
 
-  constexpr auto aa = CmpRSH(a, H);
+  constexpr auto aa = CmpRSH(a, InvH);
   // CHECK: store <4 x i8> <i8 9, i8 9, i8 7, i8 4>
 
   constexpr auto ab = CmpBinAnd(a, b);
   // CHECK: store <4 x i8> <i8 2, i8 0, i8 0, i8 8>
 
   constexpr auto ac = CmpBinXOr(a, b);
   // CHECK: store <4 x i8> <i8 20, i8 19, i8 15, i8 1>
 
   constexpr auto ad = CmpBinOr(a, b);
   // CHECK: store <4 x i8> <i8 22, i8 19, i8 15, i8 9>
+
+  constexpr auto ae = ~FourCharsExtVec{1, 2, 10, 20};
+  // CHECK: store <4 x i8> <i8 -2, i8 -3, i8 -11, i8 -21>
+
+  constexpr auto af = !FourCharsExtVec{0, 1, 8, -1};
+  // CHECK: store <4 x i8> <i8 -1, i8 0, i8 0, i8 0>
 }
 
 void FloatUsage() {
   constexpr auto a = FourFloatsVecSize{6, 3, 2, 1} +
                      FourFloatsVecSize{12, 15, 5, 7};
   // CHECK: <4 x float> <float 1.800000e+01, float 1.800000e+01, float 7.000000e+00, float 8.000000e+00>
   constexpr auto b = FourFloatsVecSize{19, 15, 13, 12} -
                      FourFloatsVecSize{13, 14, 5, 3};
void FloatUsage() {
   // CHECK: store <4 x float> <float 1.000000e+00, float 5.000000e+00, float 8.000000e+00, float 1.000000e+01>
   constexpr auto n = 3 * FourFloatsVecSize{8, 4, 2, 1};
   // CHECK: store <4 x float> <float 2.400000e+01, float 1.200000e+01, float 6.000000e+00, float 3.000000e+00>
   constexpr auto o = 100 / FourFloatsVecSize{12, 15, 18, 21};
   // CHECK: store <4 x float> <float 0x4020AAAAA0000000, float 0x401AAAAAA0000000, float 0x401638E380000000, float 0x40130C30C0000000>
 
   constexpr auto w = FourFloatsVecSize{1, 2, 3, 4} <
                      FourFloatsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 0>
   constexpr auto x = FourFloatsVecSize{1, 2, 3, 4} >
                      FourFloatsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto y = FourFloatsVecSize{1, 2, 3, 4} <=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 -1, i32 0>
   constexpr auto z = FourFloatsVecSize{1, 2, 3, 4} >=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto A = FourFloatsVecSize{1, 2, 3, 4} ==
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 0>
   constexpr auto B = FourFloatsVecSize{1, 2, 3, 4} !=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 -1>
 
   constexpr auto C = FourFloatsVecSize{1, 2, 3, 4} < 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 0>
   constexpr auto D = FourFloatsVecSize{1, 2, 3, 4} > 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 0, i32 -1>
   constexpr auto E = FourFloatsVecSize{1, 2, 3, 4} <= 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 -1, i32 0>
   constexpr auto F = FourFloatsVecSize{1, 2, 3, 4} >= 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto G = FourFloatsVecSize{1, 2, 3, 4} == 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 0>
   constexpr auto H = FourFloatsVecSize{1, 2, 3, 4} != 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 -1>
 
   constexpr auto O = FourFloatsVecSize{5, 0, 6, 0} &&
                      FourFloatsVecSize{5, 5, 0, 0};
   // CHECK: store <4 x i32> <i32 1, i32 0, i32 0, i32 0>
   constexpr auto P = FourFloatsVecSize{5, 0, 6, 0} ||
                      FourFloatsVecSize{5, 5, 0, 0};
   // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
 
   constexpr auto Q = FourFloatsVecSize{5, 0, 6, 0} && 3;
   // CHECK: store <4 x i32> <i32 1, i32 0, i32 1, i32 0>
   constexpr auto R = FourFloatsVecSize{5, 0, 6, 0} || 3;
   // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 1>
 
   constexpr auto T = CmpMul(a, b);
   // CHECK: store <4 x float> <float 1.080000e+02, float 1.800000e+01, float 5.600000e+01, float 7.200000e+01>
 
   constexpr auto U = CmpDiv(a, b);
   // CHECK: store <4 x float> <float 3.000000e+00, float 1.800000e+01, float 8.750000e-01, float 0x3FEC71C720000000>
 
   constexpr auto X = CmpAdd(a, b);
   // CHECK: store <4 x float> <float 2.400000e+01, float 1.900000e+01, float 1.500000e+01, float 1.700000e+01>
 
   constexpr auto Y = CmpSub(a, b);
   // CHECK: store <4 x float> <float 1.200000e+01, float 1.700000e+01, float -1.000000e+00, float -1.000000e+00>
+
+  constexpr auto Z = -Y;
+  // CHECK: store <4 x float> <float -1.200000e+01, float -1.700000e+01, float 1.000000e+00, float 1.000000e+00>
+
+  // Operator ~ is illegal on floats, so no test for that.
+  constexpr auto af = !FourFloatsVecSize{0, 1, 8, -1};
+  // CHECK: store <4 x i32> <i32 -1, i32 0, i32 0, i32 0>
 }
 
 void FloatVecUsage() {
   constexpr auto a = FourFloatsVecSize{6, 3, 2, 1} +
                      FourFloatsVecSize{12, 15, 5, 7};
   // CHECK: <4 x float> <float 1.800000e+01, float 1.800000e+01, float 7.000000e+00, float 8.000000e+00>
   constexpr auto b = FourFloatsVecSize{19, 15, 13, 12} -
                      FourFloatsVecSize{13, 14, 5, 3};
void FloatVecUsage() {
   // CHECK: store <4 x float> <float 1.000000e+00, float 5.000000e+00, float 8.000000e+00, float 1.000000e+01>
   constexpr auto n = 3 * FourFloatsVecSize{8, 4, 2, 1};
   // CHECK: store <4 x float> <float 2.400000e+01, float 1.200000e+01, float 6.000000e+00, float 3.000000e+00>
   constexpr auto o = 100 / FourFloatsVecSize{12, 15, 18, 21};
   // CHECK: store <4 x float> <float 0x4020AAAAA0000000, float 0x401AAAAAA0000000, float 0x401638E380000000, float 0x40130C30C0000000>
 
   constexpr auto w = FourFloatsVecSize{1, 2, 3, 4} <
                      FourFloatsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 0>
   constexpr auto x = FourFloatsVecSize{1, 2, 3, 4} >
                      FourFloatsVecSize{4, 3, 2, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto y = FourFloatsVecSize{1, 2, 3, 4} <=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 -1, i32 0>
   constexpr auto z = FourFloatsVecSize{1, 2, 3, 4} >=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto A = FourFloatsVecSize{1, 2, 3, 4} ==
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 0>
   constexpr auto B = FourFloatsVecSize{1, 2, 3, 4} !=
                      FourFloatsVecSize{4, 3, 3, 1};
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 -1>
 
   constexpr auto C = FourFloatsVecSize{1, 2, 3, 4} < 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 0>
   constexpr auto D = FourFloatsVecSize{1, 2, 3, 4} > 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 0, i32 -1>
   constexpr auto E = FourFloatsVecSize{1, 2, 3, 4} <= 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 -1, i32 0>
   constexpr auto F = FourFloatsVecSize{1, 2, 3, 4} >= 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 1>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 -1>
   constexpr auto G = FourFloatsVecSize{1, 2, 3, 4} == 3;
-  // CHECK: store <4 x i32> <i32 0, i32 0, i32 1, i32 0>
+  // CHECK: store <4 x i32> <i32 0, i32 0, i32 -1, i32 0>
   constexpr auto H = FourFloatsVecSize{1, 2, 3, 4} != 3;
-  // CHECK: store <4 x i32> <i32 1, i32 1, i32 0, i32 1>
+  // CHECK: store <4 x i32> <i32 -1, i32 -1, i32 0, i32 -1>
 
   constexpr auto O = FourFloatsVecSize{5, 0, 6, 0} &&
                      FourFloatsVecSize{5, 5, 0, 0};
   // CHECK: store <4 x i32> <i32 1, i32 0, i32 0, i32 0>
   constexpr auto P = FourFloatsVecSize{5, 0, 6, 0} ||
                      FourFloatsVecSize{5, 5, 0, 0};
   // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 0>
 
   constexpr auto Q = FourFloatsVecSize{5, 0, 6, 0} && 3;
   // CHECK: store <4 x i32> <i32 1, i32 0, i32 1, i32 0>
   constexpr auto R = FourFloatsVecSize{5, 0, 6, 0} || 3;
   // CHECK: store <4 x i32> <i32 1, i32 1, i32 1, i32 1>
 
   constexpr auto T = CmpMul(a, b);
   // CHECK: store <4 x float> <float 1.080000e+02, float 1.800000e+01, float 5.600000e+01, float 7.200000e+01>
 
   constexpr auto U = CmpDiv(a, b);
   // CHECK: store <4 x float> <float 3.000000e+00, float 1.800000e+01, float 8.750000e-01, float 0x3FEC71C720000000>
 
   constexpr auto X = CmpAdd(a, b);
   // CHECK: store <4 x float> <float 2.400000e+01, float 1.900000e+01, float 1.500000e+01, float 1.700000e+01>
 
   constexpr auto Y = CmpSub(a, b);
   // CHECK: store <4 x float> <float 1.200000e+01, float 1.700000e+01, float -1.000000e+00, float -1.000000e+00>
+
+  constexpr auto Z = -Y;
+  // CHECK: store <4 x float> <float -1.200000e+01, float -1.700000e+01, float 1.000000e+00, float 1.000000e+00>
+
+  // Operator ~ is illegal on floats, so no test for that.
+  constexpr auto af = !FourFloatsVecSize{0, 1, 8, -1};
+  // CHECK: store <4 x i32> <i32 -1, i32 0, i32 0, i32 0>
+}
+
+void I128Usage() {
+  constexpr auto a = FourI128VecSize{1, 2, 3, 4};
+  // CHECK: store <4 x i128> <i128 1, i128 2, i128 3, i128 4>
+  constexpr auto b = a < 3;
+  // CHECK: store <4 x i128> <i128 -1, i128 -1, i128 0, i128 0>
+
+  // Operator ~ is illegal on floats, so no test for that.
+  constexpr auto c = ~FourI128VecSize{1, 2, 10, 20};
+  // CHECK: store <4 x i128> <i128 -2, i128 -3, i128 -11, i128 -21>
+
+  constexpr auto d = !FourI128VecSize{0, 1, 8, -1};
+  // CHECK: store <4 x i128> <i128 -1, i128 0, i128 0, i128 0>
+}
+
+void I128VecUsage() {
+  constexpr auto a = FourI128ExtVec{1, 2, 3, 4};
+  // CHECK: store <4 x i128> <i128 1, i128 2, i128 3, i128 4>
+  constexpr auto b = a < 3;
+  // CHECK: store <4 x i128> <i128 -1, i128 -1, i128 0, i128 0>
+
+  // Operator ~ is illegal on floats, so no test for that.
+  constexpr auto c = ~FourI128ExtVec{1, 2, 10, 20};
+  // CHECK: store <4 x i128>  <i128 -2, i128 -3, i128 -11, i128 -21>
+
+  constexpr auto d = !FourI128ExtVec{0, 1, 8, -1};
+  // CHECK: store <4 x i128>  <i128 -1, i128 0, i128 0, i128 0>
 }
+