Extract LaneBitmask into a separate type

[kparzysz]

Summary

The uses of lane masks are not consistent in terms of what type they use: some use LaneBitmask (which is defined in more than one place), others use plain unsigned. This patch defines LaneBitmask as a class with the underlying type of unsigned, and changes all uses of lane masks to use this class. The class is intended to avoid any overhead compared to using an integral type directly. It specifically avoids implicit conversions from/to integral types to avoid potential errors when changing the underlying type. For example, a typical initialization of a "full" mask was "LaneMask = ~0u", which would result in a value of 0x00000000FFFFFFFF if the type was extended to uint64_t.

[MatzeB]

I'm not a big fan of all the boilerplate necessary when wrapping up something simple like an unsigned value.

However seeing the little places fixed here where we didn't correctly use the LaneBitmask typedef is convincing, so this is a good plan!

I have some comments though:

• Creating a new LaneBitmask.h header seems overkill (esp. in llvm/Support which is even imported into non-codegen parts of llvm). I would just add the definition at the beginning of MCRegisterInfo.h.
• Possibly add constexpr to all the members of LaneBitmask (though I'm not sure if we care about constexpr that much in llvm, it's used extremely rarely today)

include/llvm/Support/LaneBitmask.h

9 ↗

(On Diff #80409)

Should use /// \file to tell doxygen about the file level comment.

36–37 ↗

(On Diff #80409)

Maybe just move the print function up here, so you immediately see the matching format string.

40 ↗

(On Diff #80409)

Just use a member initializer instead of a constructor.

43 ↗

(On Diff #80409)

Do we actually need an ordering for bitmasks?

56–61 ↗

(On Diff #80409)

Shifting seems like a very unnatural operator for a bitmask. It's only used as an encoding trick in the tablegen stuff, I don't think it makes any semantic sense as a lane bitmask. Therefore I'd rather force people to cast to LaneBitmask::Type and back for the shifting operations.

71–84 ↗

(On Diff #80409)

Same comment as operator<<, operator>> applies here

88–89 ↗

(On Diff #80409)

Maybe use

enum : Type {
  None = 0,
  All = ~(Type)0,
}

(or static const Type None = ...)
as that will make it obvious they are constants, will work in constant global initializers and is a few characters less typing.

97 ↗

(On Diff #80409)

Should we make this public and get rid of the get(Type V) function?

lib/CodeGen/DetectDeadLanes.cpp

213 ↗

(On Diff #80409)

(A & ~B) is a pretty common operation, maybe introduce:

LaneBitmask::common(LaneBitmask Other) { return *this & ~Other; }
Possibly with an implicit bool cast operator so we can write:
if (!UsedLanes.common(PrevUsedLanes)) return;

(union may be another possible candidate for a name, but it feels a bit out of place as the type is called a mask and not a set)

We can leave this addition to a followup patch of course.

lib/CodeGen/RegisterPressure.cpp

43 ↗

(On Diff #80409)

That sneaked in here and should probably be discussed in a separate patch (even though this appears to be a real problem).

lib/CodeGen/TargetRegisterInfo.cpp

130–134 ↗

(On Diff #80409)

Should probably move that function declaration/definition next to the LaneBitmask definition.

utils/TableGen/CodeGenRegisters.cpp

54 ↗

(On Diff #80409)

LaneMask() can be removed.

65 ↗

(On Diff #80409)

dito.

682 ↗

(On Diff #80409)

dito.

[kparzysz]

In D27454#616756, @MatzeB wrote:

• Creating a new LaneBitmask.h header seems overkill (esp. in llvm/Support which is even imported into non-codegen parts of llvm). I would just add the definition at the beginning of MCRegisterInfo.h.

I added a separate file because LaneBitmask is used in both TableGen and CodeGen. Otherwise I wouldn't bother.

include/llvm/Support/LaneBitmask.h

40 ↗

(On Diff #80409)

I did it and got errors: "error: constructor for 'llvm::CodeGenRegisterClass' must explicitly initialize the member 'LaneMask' which does not have a default constructor".

43 ↗

(On Diff #80409)

Ordering is required for storing it in std::set, std::map, etc.

88–89 ↗

(On Diff #80409)

This would require allowing implicit conversion from Type, at least if we want to save on the number of characters. I specifically wanted to avoid such conversions, because it makes it easy to make a hard-to-find mistake in the code. As per the example from the main comment, a person may write "LaneBitmask M = ~0u" by the force of habit, and that code will compile and run correctly at the moment. However it may fail if the lane masks later on become 64-bit integers, since ~0u will only bit-negate a 32-bit value.

97 ↗

(On Diff #80409)

This is related to the comment above regarding the implicit conversion from Type. Having to write "get" may serve as a reminder that constructing lane masks needs to be done with "type switchability" in mind.

lib/CodeGen/DetectDeadLanes.cpp

213 ↗

(On Diff #80409)

(A & ~B) is actually more like A - B (remove B's bits from A). It does occur a few times, so adding a function for it may be a good idea.

Also, union may not work as a name, it being a keyword and all... :)

[kparzysz]

Addressed several comments. The rest of the comments were replied to.

[kparzysz]

include/llvm/Support/LaneBitmask.h

42 ↗

(On Diff #80805)

Just to explain the use of enum: using "static const unsigned" has in the past caused problems with Debug vs Release builds. Release builds treated the constant as literal, while Debug builds treated it as a constant object, thus requiring a definition (i.e. "const Type Class::Member = val"). Release builds, in turn, didn't like that definition, so there was no satisfactory way to keep both happy.

[MatzeB]

In D27454#617500, @kparzysz wrote:

In D27454#616756, @MatzeB wrote:

• Creating a new LaneBitmask.h header seems overkill (esp. in llvm/Support which is even imported into non-codegen parts of llvm). I would just add the definition at the beginning of MCRegisterInfo.h.

I added a separate file because LaneBitmask is used in both TableGen and CodeGen. Otherwise I wouldn't bother.

Ah that's why. It still feels wrong to put a codegen related class into llvm/Support. Maybe we should just not use the custom type in tablegen (you can still add safety checks like letting tablegen generate a static_assert(std::is_same(LaneBitmask::Type, TypeUsedByTableGen)); to avoid accidental type mismatches between tablegen and libCodeGen. (A silly alternative would be to make sure LaneBitmask is a 100% header only implementation so we can include it into tablegen without needing to link libMC, but that seems brittle to me.)

include/llvm/Support/LaneBitmask.h

43 ↗

(On Diff #80409)

Sure I just couldn't come up with a sensible reason to put them into a set/map. Anyway I don't really care whether that operator exists or not.

88–89 ↗

(On Diff #80409)

Oh indeed enum : Type was not my intention and enum : LaneBitmask of course doesn't work, of course we do not want implicit conversions.

Anyway if you mark the respective methods/constructors as constexpr I would expect something like this to work:

static const LaneBitmask None = LaneBitmask(0);
static const LaneBitmask All = ~None;

97 ↗

(On Diff #80409)

I agree that we do not want implicit conversions here. I was thinking of an explicit LaneBitmask(Type Mask) : Mask(Mask) {} constructor.

42 ↗

(On Diff #80805)

enums are fine (I think with static const unsigned we will get an extra entry in the rodata segment which seems superfluous for a purely compiletime constant).

The Release vs. Debug build issue rather sounds like a constant vs. non-constant expression issue to me. I would expect it to properly work in both as long as everything involved is marked constexpr or using basic types (also looking around there are certainly other headers in llvm using static const type name =.

lib/CodeGen/DetectDeadLanes.cpp

213 ↗

(On Diff #80409)

I must have been tired writing the review. Of course it's not a union/intersection.

I would not like to overload operator- as it not clear whether that performs an integer subtraction with the underlying datatype or the "and not" operation.
The set theoretical name difference() feels unhelpful to me as well, so maybe a more colloquial without().

[kparzysz]

In D27454#617933, @MatzeB wrote:

Ah that's why. It still feels wrong to put a codegen related class into llvm/Support. Maybe we should just not use the custom type in tablegen (you can still add safety checks like letting tablegen generate a static_assert(std::is_same(LaneBitmask::Type, TypeUsedByTableGen)); to avoid accidental type mismatches between tablegen and libCodeGen.

I have done that, but I had serious reservations about it. I think it makes the tablegen code more error prone. It may look unnatural to have LaneBitmask definition in Support, but it being used by two different layers justifies it, in my opinion.

include/llvm/Support/LaneBitmask.h

88–89 ↗

(On Diff #80409)

I kept the get... functions in the end. I wanted the None/All to be nested in the LaneBitmask namespace. Declaring them outside created a conflict with "None" from Optional, and declaring them inside of LaneBitmask was rejected by the compiler, since the LaneBitmask type was not yet complete (until the closing bracket).

I made all applicable members "constexpr".

lib/CodeGen/DetectDeadLanes.cpp

213 ↗

(On Diff #80409)

I left it out from this patch.

[kparzysz]

Removed LaneBitmask.h, merged it into MCRegisterInfo.h.

Used a separate type for lane masks in TableGen.

[kparzysz]

TableGen includes CodeGen/MachineValueType.h. Could we have LaneBitmask.h in CodeGen and include it here as well?

[MatzeB]

In D27454#622191, @kparzysz wrote:

TableGen includes CodeGen/MachineValueType.h. Could we have LaneBitmask.h in CodeGen and include it here as well?

Ah I didn't notice that file before, but that looks like the "100% header only implementation" I mentioned as an alternative earlier. Maybe the idea isn't so silly and we should just do it with LaneBitmask as well, at least it is all implemented in the header at the moment anyway.

[MatzeB]

LGTM with the declaration moved to an MC header. If you want to pursue the direction of having a MC/LaneBitmask.h with a header only implementation and including that in TableGen go for it (I'm fine with that as well with the solution currently here).

Some more nitpicks/suggestions below but nothing that should stop the patch.

include/llvm/MC/MCRegisterInfo.h

29 ↗

(On Diff #81110)

Should use ///

52–55 ↗

(On Diff #81110)

One slightly annoying thing here is that we need to add #include "raw_ostream.h" and #include "Format.h" to the header. Would it make sense to maybe not add the print() function to LaneBitmask at all, just declaring Printable PrintLaneMask() here and implementing it in MCRegisterInfo.cpp and using something like static_assert(sizeof(LaneBitmask::Type) == 4) (or alternatively leaving the format string but not the actual call to format() in the LaneBitmask struct).

57 ↗

(On Diff #81110)

The : Mask(0) is unnecessary.

63–64 ↗

(On Diff #81110)

Looking at the users !X.none() seems to be common enough that it warrants a bool any() const { return Mask != 0; } maybe even a operator bool() const { return Mask != 0; }.

utils/TableGen/RegisterInfoEmitter.cpp

616 ↗

(On Diff #81110)

You could add a static_assert(sizeof(LaneBitmaskType) == 4) to be safe.

[kparzysz]

include/llvm/MC/MCRegisterInfo.h

63–64 ↗

(On Diff #81110)

I will add it in a subsequent patch.

llvm/trunk/include/llvm/CodeGen/LiveIntervalAnalysis.h

private:
 
     /// Helper function for repairIntervalsInRange(), walks backwards and
     /// creates/modifies live segments in @p LR to match the operands found.
     /// Only full operands or operands with subregisters matching @p LaneMask
     /// are considered.
     void repairOldRegInRange(MachineBasicBlock::iterator Begin,
                              MachineBasicBlock::iterator End,
                              const SlotIndex endIdx, LiveRange &LR,
-                             unsigned Reg, LaneBitmask LaneMask = ~0u);
+                             unsigned Reg,
+                             LaneBitmask LaneMask = LaneBitmask::getAll());
 
     class HMEditor;
   };
 } // End llvm namespace
 
 #endif

llvm/trunk/include/llvm/CodeGen/MachineBasicBlock.h

 #ifndef LLVM_CODEGEN_MACHINEBASICBLOCK_H
 #define LLVM_CODEGEN_MACHINEBASICBLOCK_H
 
 #include "llvm/ADT/GraphTraits.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/CodeGen/MachineInstrBundleIterator.h"
 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/Support/BranchProbability.h"
+#include "llvm/MC/LaneBitmask.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/Support/DataTypes.h"
 #include <functional>
 
 namespace llvm {
 
 class Pass;
 class BasicBlock;
 class MachineFunction;
 class MCSymbol;
 class MIPrinter;
 class SlotIndexes;
 class StringRef;
 class raw_ostream;
 class MachineBranchProbabilityInfo;
 
-// Forward declaration to avoid circular include problem with TargetRegisterInfo
-typedef unsigned LaneBitmask;
-
 template <> struct ilist_traits<MachineInstr> {
 private:
   friend class MachineBasicBlock; // Set by the owning MachineBasicBlock.
   MachineBasicBlock *Parent;
 
   typedef simple_ilist<MachineInstr, ilist_sentinel_tracking<true>>::iterator
       instr_iterator;
 
inline iterator_range<const_succ_iterator> successors() const {
     return make_range(succ_begin(), succ_end());
   }
 
   // LiveIn management methods.
 
   /// Adds the specified register as a live in. Note that it is an error to add
   /// the same register to the same set more than once unless the intention is
   /// to call sortUniqueLiveIns after all registers are added.
-  void addLiveIn(MCPhysReg PhysReg, LaneBitmask LaneMask = ~0u) {
+  void addLiveIn(MCPhysReg PhysReg,
+                 LaneBitmask LaneMask = LaneBitmask::getAll()) {
     LiveIns.push_back(RegisterMaskPair(PhysReg, LaneMask));
   }
   void addLiveIn(const RegisterMaskPair &RegMaskPair) {
     LiveIns.push_back(RegMaskPair);
   }
 
   /// Sorts and uniques the LiveIns vector. It can be significantly faster to do
   /// this than repeatedly calling isLiveIn before calling addLiveIn for every
   /// LiveIn insertion.
   void sortUniqueLiveIns();
 
   /// Add PhysReg as live in to this block, and ensure that there is a copy of
   /// PhysReg to a virtual register of class RC. Return the virtual register
   /// that is a copy of the live in PhysReg.
   unsigned addLiveIn(MCPhysReg PhysReg, const TargetRegisterClass *RC);
 
   /// Remove the specified register from the live in set.
-  void removeLiveIn(MCPhysReg Reg, LaneBitmask LaneMask = ~0u);
+  void removeLiveIn(MCPhysReg Reg,
+                    LaneBitmask LaneMask = LaneBitmask::getAll());
 
   /// Return true if the specified register is in the live in set.
-  bool isLiveIn(MCPhysReg Reg, LaneBitmask LaneMask = ~0u) const;
+  bool isLiveIn(MCPhysReg Reg,
+                LaneBitmask LaneMask = LaneBitmask::getAll()) const;
 
   // Iteration support for live in sets.  These sets are kept in sorted
   // order by their register number.
   typedef LiveInVector::const_iterator livein_iterator;
   livein_iterator livein_begin() const { return LiveIns.begin(); }
   livein_iterator livein_end()   const { return LiveIns.end(); }
   bool            livein_empty() const { return LiveIns.empty(); }
   iterator_range<livein_iterator> liveins() const {

llvm/trunk/include/llvm/CodeGen/RegisterPressure.h

 public:
   void clear();
   void init(const MachineRegisterInfo &MRI);
 
   LaneBitmask contains(unsigned Reg) const {
     unsigned SparseIndex = getSparseIndexFromReg(Reg);
     RegSet::const_iterator I = Regs.find(SparseIndex);
     if (I == Regs.end())
-      return 0;
+      return LaneBitmask::getNone();
     return I->LaneMask;
   }
 
   /// Mark the \p Pair.LaneMask lanes of \p Pair.Reg as live.
   /// Returns the previously live lanes of \p Pair.Reg.
   LaneBitmask insert(RegisterMaskPair Pair) {
     unsigned SparseIndex = getSparseIndexFromReg(Pair.RegUnit);
     auto InsertRes = Regs.insert(IndexMaskPair(SparseIndex, Pair.LaneMask));
     if (!InsertRes.second) {
-      unsigned PrevMask = InsertRes.first->LaneMask;
+      LaneBitmask PrevMask = InsertRes.first->LaneMask;
       InsertRes.first->LaneMask |= Pair.LaneMask;
       return PrevMask;
     }
-    return 0;
+    return LaneBitmask::getNone();
   }
 
   /// Clears the \p Pair.LaneMask lanes of \p Pair.Reg (mark them as dead).
   /// Returns the previously live lanes of \p Pair.Reg.
   LaneBitmask erase(RegisterMaskPair Pair) {
     unsigned SparseIndex = getSparseIndexFromReg(Pair.RegUnit);
     RegSet::iterator I = Regs.find(SparseIndex);
     if (I == Regs.end())
-      return 0;
-    unsigned PrevMask = I->LaneMask;
+      return LaneBitmask::getNone();
+    LaneBitmask PrevMask = I->LaneMask;
     I->LaneMask &= ~Pair.LaneMask;
     return PrevMask;
   }
 
   size_t size() const {
     return Regs.size();
   }
 
   template<typename ContainerT>
   void appendTo(ContainerT &To) const {
     for (const IndexMaskPair &P : Regs) {
       unsigned Reg = getRegFromSparseIndex(P.Index);
-      if (P.LaneMask != 0)
+      if (!P.LaneMask.none())
         To.push_back(RegisterMaskPair(Reg, P.LaneMask));
     }
   }
 };
 
 /// Track the current register pressure at some position in the instruction
 /// stream, and remember the high water mark within the region traversed. This
 /// does not automatically consider live-through ranges. The client may

llvm/trunk/include/llvm/CodeGen/RegisterScavenging.h

public:
   /// Returns the scavenged register.
   unsigned scavengeRegister(const TargetRegisterClass *RegClass,
                             MachineBasicBlock::iterator I, int SPAdj);
   unsigned scavengeRegister(const TargetRegisterClass *RegClass, int SPAdj) {
     return scavengeRegister(RegClass, MBBI, SPAdj);
   }
 
   /// Tell the scavenger a register is used.
-  void setRegUsed(unsigned Reg, LaneBitmask LaneMask = ~0u);
+  void setRegUsed(unsigned Reg, LaneBitmask LaneMask = LaneBitmask::getAll());
 private:
   /// Returns true if a register is reserved. It is never "unused".
   bool isReserved(unsigned Reg) const { return MRI->isReserved(Reg); }
 
   /// setUsed / setUnused - Mark the state of one or a number of register units.
   ///
   void setUsed(BitVector &RegUnits) {
     RegUnitsAvailable.reset(RegUnits);

llvm/trunk/include/llvm/MC/LaneBitmask.h

+//===-- llvm/MC/LaneBitmask.h -----------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// A common definition of LaneBitmask for use in TableGen and CodeGen.
+///
+/// A lane mask is a bitmask representing the covering of a register with
+/// sub-registers.
+///
+/// This is typically used to track liveness at sub-register granularity.
+/// Lane masks for sub-register indices are similar to register units for
+/// physical registers. The individual bits in a lane mask can't be assigned
+/// any specific meaning. They can be used to check if two sub-register
+/// indices overlap.
+///
+/// Iff the target has a register such that:
+///
+///   getSubReg(Reg, A) overlaps getSubReg(Reg, B)
+///
+/// then:
+///
+///   (getSubRegIndexLaneMask(A) & getSubRegIndexLaneMask(B)) != 0
+
+#ifndef LLVM_MC_LANEBITMASK_H
+#define LLVM_MC_LANEBITMASK_H
+
+#include "llvm/Support/Format.h"
+#include "llvm/Support/Printable.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+  struct LaneBitmask {
+    // When changing the underlying type, change the format string as well.
+    typedef unsigned Type;
+    enum : unsigned { BitWidth = 8*sizeof(Type) };
+    constexpr static const char *const FormatStr = "%08X";
+
+    constexpr LaneBitmask() = default;
+    explicit constexpr LaneBitmask(Type V) : Mask(V) {}
+
+    constexpr bool operator== (LaneBitmask M) const { return Mask == M.Mask; }
+    constexpr bool operator!= (LaneBitmask M) const { return Mask != M.Mask; }
+    constexpr bool operator< (LaneBitmask M)  const { return Mask < M.Mask; }
+    constexpr bool none() const { return Mask == 0; }
+    constexpr bool all()  const { return ~Mask == 0; }
+
+    constexpr LaneBitmask operator~() const {
+      return LaneBitmask(~Mask);
+    }
+    constexpr LaneBitmask operator|(LaneBitmask M) const {
+      return LaneBitmask(Mask | M.Mask);
+    }
+    constexpr LaneBitmask operator&(LaneBitmask M) const {
+      return LaneBitmask(Mask & M.Mask);
+    }
+    LaneBitmask &operator|=(LaneBitmask M) {
+      Mask |= M.Mask;
+      return *this;
+    }
+    LaneBitmask &operator&=(LaneBitmask M) {
+      Mask &= M.Mask;
+      return *this;
+    }
+
+    constexpr Type getAsInteger() const { return Mask; }
+
+    static LaneBitmask getNone() { return LaneBitmask(0); }
+    static LaneBitmask getAll()  { return ~LaneBitmask(0); }
+
+  private:
+    Type Mask = 0;
+  };
+
+  /// Create Printable object to print LaneBitmasks on a \ref raw_ostream.
+  static LLVM_ATTRIBUTE_UNUSED Printable PrintLaneMask(LaneBitmask LaneMask) {
+    return Printable([LaneMask](raw_ostream &OS) {
+      OS << format(LaneBitmask::FormatStr, LaneMask.getAsInteger());
+    });
+  }
+}
+
+#endif // LLVM_MC_LANEBITMASK_H

llvm/trunk/include/llvm/MC/MCRegisterInfo.h

 // purposed, especially register allocation.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_MC_MCREGISTERINFO_H
 #define LLVM_MC_MCREGISTERINFO_H
 
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/MC/LaneBitmask.h"
 #include "llvm/Support/ErrorHandling.h"
 #include <cassert>
 
 namespace llvm {
 
 /// An unsigned integer type large enough to represent all physical registers,
 /// but not necessarily virtual registers.
 typedef uint16_t MCPhysReg;
private:
   unsigned NumRegs;                           // Number of entries in the array
   unsigned RAReg;                             // Return address register
   unsigned PCReg;                             // Program counter register
   const MCRegisterClass *Classes;             // Pointer to the regclass array
   unsigned NumClasses;                        // Number of entries in the array
   unsigned NumRegUnits;                       // Number of regunits.
   const MCPhysReg (*RegUnitRoots)[2];         // Pointer to regunit root table.
   const MCPhysReg *DiffLists;                 // Pointer to the difflists array
-  const unsigned *RegUnitMaskSequences;       // Pointer to lane mask sequences
+  const LaneBitmask *RegUnitMaskSequences;    // Pointer to lane mask sequences
                                               // for register units.
   const char *RegStrings;                     // Pointer to the string table.
   const char *RegClassStrings;                // Pointer to the class strings.
   const uint16_t *SubRegIndices;              // Pointer to the subreg lookup
                                               // array.
   const SubRegCoveredBits *SubRegIdxRanges;   // Pointer to the subreg covered
                                               // bit ranges array.
   unsigned NumSubRegIndices;                  // Number of subreg indices.
public:
   /// \brief Initialize MCRegisterInfo, called by TableGen
   /// auto-generated routines. *DO NOT USE*.
   void InitMCRegisterInfo(const MCRegisterDesc *D, unsigned NR, unsigned RA,
                           unsigned PC,
                           const MCRegisterClass *C, unsigned NC,
                           const MCPhysReg (*RURoots)[2],
                           unsigned NRU,
                           const MCPhysReg *DL,
-                          const unsigned *RUMS,
+                          const LaneBitmask *RUMS,
                           const char *Strings,
                           const char *ClassStrings,
                           const uint16_t *SubIndices,
                           unsigned NumIndices,
                           const SubRegCoveredBits *SubIdxRanges,
                           const uint16_t *RET) {
     Desc = D;
     NumRegs = NR;
public:
   }
 };
 
 /// MCRegUnitMaskIterator enumerates a list of register units and their
 /// associated lane masks for Reg. The register units are in ascending
 /// numerical order.
 class MCRegUnitMaskIterator {
   MCRegUnitIterator RUIter;
-  const unsigned *MaskListIter;
+  const LaneBitmask *MaskListIter;
 public:
   MCRegUnitMaskIterator() {}
   /// Constructs an iterator that traverses the register units and their
   /// associated LaneMasks in Reg.
   MCRegUnitMaskIterator(unsigned Reg, const MCRegisterInfo *MCRI)
     : RUIter(Reg, MCRI) {
       uint16_t Idx = MCRI->get(Reg).RegUnitLaneMasks;
       MaskListIter = &MCRI->RegUnitMaskSequences[Idx];
   }
 
   /// Returns a (RegUnit, LaneMask) pair.
-  std::pair<unsigned,unsigned> operator*() const {
+  std::pair<unsigned,LaneBitmask> operator*() const {
     return std::make_pair(*RUIter, *MaskListIter);
   }
 
   /// Returns true if this iterator is not yet at the end.
   bool isValid() const { return RUIter.isValid(); }
 
   /// Moves to the next position.
   void operator++() {

llvm/trunk/include/llvm/Target/TargetRegisterInfo.h

 class BitVector;
 class MachineFunction;
 class RegScavenger;
 template<class T> class SmallVectorImpl;
 class VirtRegMap;
 class raw_ostream;
 class LiveRegMatrix;
 
-/// A bitmask representing the covering of a register with sub-registers.
-///
-/// This is typically used to track liveness at sub-register granularity.
-/// Lane masks for sub-register indices are similar to register units for
-/// physical registers. The individual bits in a lane mask can't be assigned
-/// any specific meaning. They can be used to check if two sub-register
-/// indices overlap.
-///
-/// Iff the target has a register such that:
-///
-///   getSubReg(Reg, A) overlaps getSubReg(Reg, B)
-///
-/// then:
-///
-///   (getSubRegIndexLaneMask(A) & getSubRegIndexLaneMask(B)) != 0
-typedef unsigned LaneBitmask;
-
 class TargetRegisterClass {
 public:
   typedef const MCPhysReg* iterator;
   typedef const MCPhysReg* const_iterator;
   typedef const MVT::SimpleValueType* vt_iterator;
   typedef const TargetRegisterClass* const * sc_iterator;
 
   // Instance variables filled by tablegen, do not use!
public:
   typedef const TargetRegisterClass * const * regclass_iterator;
 private:
   const TargetRegisterInfoDesc *InfoDesc;     // Extra desc array for codegen
   const char *const *SubRegIndexNames;        // Names of subreg indexes.
   // Pointer to array of lane masks, one per sub-reg index.
   const LaneBitmask *SubRegIndexLaneMasks;
 
   regclass_iterator RegClassBegin, RegClassEnd;   // List of regclasses
-  unsigned CoveringLanes;
+  LaneBitmask CoveringLanes;
 
 protected:
   TargetRegisterInfo(const TargetRegisterInfoDesc *ID,
                      regclass_iterator RegClassBegin,
                      regclass_iterator RegClassEnd,
                      const char *const *SRINames,
                      const LaneBitmask *SRILaneMasks,
-                     unsigned CoveringLanes);
+                     LaneBitmask CoveringLanes);
   virtual ~TargetRegisterInfo();
 public:
 
   // Register numbers can represent physical registers, virtual registers, and
   // sometimes stack slots. The unsigned values are divided into these ranges:
   //
   //   0           Not a register, can be used as a sentinel.
   //   [1;2^30)    Physical registers assigned by TableGen.

 ///
 /// Usage: OS << PrintRegUnit(Unit, TRI) << '\n';
 Printable PrintRegUnit(unsigned Unit, const TargetRegisterInfo *TRI);
 
 /// \brief Create Printable object to print virtual registers and physical
 /// registers on a \ref raw_ostream.
 Printable PrintVRegOrUnit(unsigned VRegOrUnit, const TargetRegisterInfo *TRI);
 
-/// Create Printable object to print LaneBitmasks on a \ref raw_ostream.
-Printable PrintLaneMask(LaneBitmask LaneMask);
-
 } // End llvm namespace
 
 #endif

llvm/trunk/lib/CodeGen/DetectDeadLanes.cpp

void DetectDeadLanes::addUsedLanesOnOperand(const MachineOperand &MO,
   if (MOSubReg != 0)
     UsedLanes = TRI->composeSubRegIndexLaneMask(MOSubReg, UsedLanes);
   UsedLanes &= MRI->getMaxLaneMaskForVReg(MOReg);
 
   unsigned MORegIdx = TargetRegisterInfo::virtReg2Index(MOReg);
   VRegInfo &MORegInfo = VRegInfos[MORegIdx];
   LaneBitmask PrevUsedLanes = MORegInfo.UsedLanes;
   // Any change at all?
-  if ((UsedLanes & ~PrevUsedLanes) == 0)
+  if ((UsedLanes & ~PrevUsedLanes).none())
     return;
 
   // Set UsedLanes and remember instruction for further propagation.
   MORegInfo.UsedLanes = PrevUsedLanes | UsedLanes;
   if (DefinedByCopy.test(MORegIdx))
     PutInWorklist(MORegIdx);
 }
 
void DetectDeadLanes::transferDefinedLanesStep(const MachineOperand &Use,
   unsigned OpNum = MI.getOperandNo(&Use);
   DefinedLanes =
       TRI->reverseComposeSubRegIndexLaneMask(Use.getSubReg(), DefinedLanes);
   DefinedLanes = transferDefinedLanes(Def, OpNum, DefinedLanes);
 
   VRegInfo &RegInfo = VRegInfos[DefRegIdx];
   LaneBitmask PrevDefinedLanes = RegInfo.DefinedLanes;
   // Any change at all?
-  if ((DefinedLanes & ~PrevDefinedLanes) == 0)
+  if ((DefinedLanes & ~PrevDefinedLanes).none())
     return;
 
   RegInfo.DefinedLanes = PrevDefinedLanes | DefinedLanes;
   PutInWorklist(DefRegIdx);
 }
 
 LaneBitmask DetectDeadLanes::transferDefinedLanes(const MachineOperand &Def,
     unsigned OpNum, LaneBitmask DefinedLanes) const {
LaneBitmask DetectDeadLanes::transferDefinedLanes(const MachineOperand &Def,
   DefinedLanes &= MRI->getMaxLaneMaskForVReg(Def.getReg());
   return DefinedLanes;
 }
 
 LaneBitmask DetectDeadLanes::determineInitialDefinedLanes(unsigned Reg) {
   // Live-In or unused registers have no definition but are considered fully
   // defined.
   if (!MRI->hasOneDef(Reg))
-    return ~0u;
+    return LaneBitmask::getAll();
 
   const MachineOperand &Def = *MRI->def_begin(Reg);
   const MachineInstr &DefMI = *Def.getParent();
   if (lowersToCopies(DefMI)) {
     // Start optimisatically with no used or defined lanes for copy
     // instructions. The following dataflow analysis will add more bits.
     unsigned RegIdx = TargetRegisterInfo::virtReg2Index(Reg);
     DefinedByCopy.set(RegIdx);
     PutInWorklist(RegIdx);
 
     if (Def.isDead())
-      return 0;
+      return LaneBitmask::getNone();
 
     // COPY/PHI can copy across unrelated register classes (example: float/int)
     // with incompatible subregister structure. Do not include these in the
     // dataflow analysis since we cannot transfer lanemasks in a meaningful way.
     const TargetRegisterClass *DefRC = MRI->getRegClass(Reg);
 
     // Determine initially DefinedLanes.
-    LaneBitmask DefinedLanes = 0;
+    LaneBitmask DefinedLanes;
     for (const MachineOperand &MO : DefMI.uses()) {
       if (!MO.isReg() || !MO.readsReg())
         continue;
       unsigned MOReg = MO.getReg();
       if (!MOReg)
         continue;
 
       LaneBitmask MODefinedLanes;
       if (TargetRegisterInfo::isPhysicalRegister(MOReg)) {
-        MODefinedLanes = ~0u;
+        MODefinedLanes = LaneBitmask::getAll();
       } else if (isCrossCopy(*MRI, DefMI, DefRC, MO)) {
-        MODefinedLanes = ~0u;
+        MODefinedLanes = LaneBitmask::getAll();
       } else {
         assert(TargetRegisterInfo::isVirtualRegister(MOReg));
         if (MRI->hasOneDef(MOReg)) {
           const MachineOperand &MODef = *MRI->def_begin(MOReg);
           const MachineInstr &MODefMI = *MODef.getParent();
           // Bits from copy-like operations will be added later.
           if (lowersToCopies(MODefMI) || MODefMI.isImplicitDef())
             continue;
         }
         unsigned MOSubReg = MO.getSubReg();
         MODefinedLanes = MRI->getMaxLaneMaskForVReg(MOReg);
         MODefinedLanes = TRI->reverseComposeSubRegIndexLaneMask(
             MOSubReg, MODefinedLanes);
       }
 
       unsigned OpNum = DefMI.getOperandNo(&MO);
       DefinedLanes |= transferDefinedLanes(Def, OpNum, MODefinedLanes);
     }
     return DefinedLanes;
   }
   if (DefMI.isImplicitDef() || Def.isDead())
-    return 0;
+    return LaneBitmask::getNone();
 
   assert(Def.getSubReg() == 0 &&
          "Should not have subregister defs in machine SSA phase");
   return MRI->getMaxLaneMaskForVReg(Reg);
 }
 
 LaneBitmask DetectDeadLanes::determineInitialUsedLanes(unsigned Reg) {
-  LaneBitmask UsedLanes = 0;
+  LaneBitmask UsedLanes = LaneBitmask::getNone();
   for (const MachineOperand &MO : MRI->use_nodbg_operands(Reg)) {
     if (!MO.readsReg())
       continue;
 
     const MachineInstr &UseMI = *MO.getParent();
     if (UseMI.isKill())
       continue;
 
LaneBitmask DetectDeadLanes::determineInitialUsedLanes(unsigned Reg) {
   }
   return UsedLanes;
 }
 
 bool DetectDeadLanes::isUndefRegAtInput(const MachineOperand &MO,
                                         const VRegInfo &RegInfo) const {
   unsigned SubReg = MO.getSubReg();
   LaneBitmask Mask = TRI->getSubRegIndexLaneMask(SubReg);
-  return (RegInfo.DefinedLanes & RegInfo.UsedLanes & Mask) == 0;
+  return (RegInfo.DefinedLanes & RegInfo.UsedLanes & Mask).none();
 }
 
 bool DetectDeadLanes::isUndefInput(const MachineOperand &MO,
                                    bool *CrossCopy) const {
   if (!MO.isUse())
     return false;
   const MachineInstr &MI = *MO.getParent();
   if (!lowersToCopies(MI))
     return false;
   const MachineOperand &Def = MI.getOperand(0);
   unsigned DefReg = Def.getReg();
   if (!TargetRegisterInfo::isVirtualRegister(DefReg))
     return false;
   unsigned DefRegIdx = TargetRegisterInfo::virtReg2Index(DefReg);
   if (!DefinedByCopy.test(DefRegIdx))
     return false;
 
   const VRegInfo &DefRegInfo = VRegInfos[DefRegIdx];
   LaneBitmask UsedLanes = transferUsedLanes(MI, DefRegInfo.UsedLanes, MO);
-  if (UsedLanes != 0)
+  if (!UsedLanes.none())
     return false;
 
   unsigned MOReg = MO.getReg();
   if (TargetRegisterInfo::isVirtualRegister(MOReg)) {
     const TargetRegisterClass *DstRC = MRI->getRegClass(DefReg);
     *CrossCopy = isCrossCopy(*MRI, MI, DstRC, MO);
   }
   return true;
for (MachineInstr &MI : MBB) {
       for (MachineOperand &MO : MI.operands()) {
         if (!MO.isReg())
           continue;
         unsigned Reg = MO.getReg();
         if (!TargetRegisterInfo::isVirtualRegister(Reg))
           continue;
         unsigned RegIdx = TargetRegisterInfo::virtReg2Index(Reg);
         const VRegInfo &RegInfo = VRegInfos[RegIdx];
-        if (MO.isDef() && !MO.isDead() && RegInfo.UsedLanes == 0) {
+        if (MO.isDef() && !MO.isDead() && RegInfo.UsedLanes.none()) {
           DEBUG(dbgs() << "Marking operand '" << MO << "' as dead in " << MI);
           MO.setIsDead();
         }
         if (MO.readsReg()) {
           bool CrossCopy = false;
           if (isUndefRegAtInput(MO, RegInfo)) {
             DEBUG(dbgs() << "Marking operand '" << MO << "' as undef in "
                   << MI);

llvm/trunk/lib/CodeGen/LiveInterval.cpp

 }
 
 void LiveInterval::computeSubRangeUndefs(SmallVectorImpl<SlotIndex> &Undefs,
                                          LaneBitmask LaneMask,
                                          const MachineRegisterInfo &MRI,
                                          const SlotIndexes &Indexes) const {
   assert(TargetRegisterInfo::isVirtualRegister(reg));
   LaneBitmask VRegMask = MRI.getMaxLaneMaskForVReg(reg);
-  assert((VRegMask & LaneMask) != 0);
+  assert(!(VRegMask & LaneMask).none());
   const TargetRegisterInfo &TRI = *MRI.getTargetRegisterInfo();
   for (const MachineOperand &MO : MRI.def_operands(reg)) {
     if (!MO.isUndef())
       continue;
     unsigned SubReg = MO.getSubReg();
     assert(SubReg != 0 && "Undef should only be set on subreg defs");
     LaneBitmask DefMask = TRI.getSubRegIndexLaneMask(SubReg);
     LaneBitmask UndefMask = VRegMask & ~DefMask;
-    if ((UndefMask & LaneMask) != 0) {
+    if (!(UndefMask & LaneMask).none()) {
       const MachineInstr &MI = *MO.getParent();
       bool EarlyClobber = MO.isEarlyClobber();
       SlotIndex Pos = Indexes.getInstructionIndex(MI).getRegSlot(EarlyClobber);
       Undefs.push_back(Pos);
     }
   }
 }
 
for (const_iterator I = begin(), E = end(); I != E; ++I) {
     }
   }
 }
 
 void LiveInterval::verify(const MachineRegisterInfo *MRI) const {
   super::verify();
 
   // Make sure SubRanges are fine and LaneMasks are disjunct.
-  LaneBitmask Mask = 0;
-  LaneBitmask MaxMask = MRI != nullptr ? MRI->getMaxLaneMaskForVReg(reg) : ~0u;
+  LaneBitmask Mask;
+  LaneBitmask MaxMask = MRI != nullptr ? MRI->getMaxLaneMaskForVReg(reg)
+                                       : LaneBitmask::getAll();
   for (const SubRange &SR : subranges()) {
     // Subrange lanemask should be disjunct to any previous subrange masks.
-    assert((Mask & SR.LaneMask) == 0);
+    assert((Mask & SR.LaneMask).none());
     Mask |= SR.LaneMask;
 
     // subrange mask should not contained in maximum lane mask for the vreg.
-    assert((Mask & ~MaxMask) == 0);
+    assert((Mask & ~MaxMask).none());
     // empty subranges must be removed.
     assert(!SR.empty());
 
     SR.verify();
     // Main liverange should cover subrange.
     assert(covers(SR));
   }
 }

llvm/trunk/lib/CodeGen/LiveIntervalAnalysis.cpp

void LiveIntervals::shrinkToUses(LiveInterval::SubRange &SR, unsigned Reg) {
   for (MachineOperand &MO : MRI->use_nodbg_operands(Reg)) {
     // Skip "undef" uses.
     if (!MO.readsReg())
       continue;
     // Maybe the operand is for a subregister we don't care about.
     unsigned SubReg = MO.getSubReg();
     if (SubReg != 0) {
       LaneBitmask LaneMask = TRI->getSubRegIndexLaneMask(SubReg);
-      if ((LaneMask & SR.LaneMask) == 0)
+      if ((LaneMask & SR.LaneMask).none())
         continue;
     }
     // We only need to visit each instruction once.
     MachineInstr *UseMI = MO.getParent();
     SlotIndex Idx = getInstructionIndex(*UseMI).getRegSlot();
     if (Idx == LastIdx)
       continue;
     LastIdx = Idx;
for (LiveInterval::const_iterator RI = LI.begin(), RE = LI.end(); RI != RE;
         //        = read %vreg1          ; R32: %vreg1
         // The <kill> flag is correct for %vreg2, but the register allocator may
         // assign R0L to %vreg1, and R0 to %vreg2 because the low 32bits of R0
         // are actually never written by %vreg2. After assignment the <kill>
         // flag at the read instruction is invalid.
         LaneBitmask DefinedLanesMask;
         if (!SRs.empty()) {
           // Compute a mask of lanes that are defined.
-          DefinedLanesMask = 0;
+          DefinedLanesMask = LaneBitmask::getNone();
           for (auto &SRP : SRs) {
             const LiveInterval::SubRange &SR = *SRP.first;
             LiveRange::const_iterator &I = SRP.second;
             if (I == SR.end())
               continue;
             I = SR.advanceTo(I, RI->end);
             if (I == SR.end() || I->start >= RI->end)
               continue;
             // I is overlapping RI
             DefinedLanesMask |= SR.LaneMask;
           }
         } else
-          DefinedLanesMask = ~0u;
+          DefinedLanesMask = LaneBitmask::getAll();
 
         bool IsFullWrite = false;
         for (const MachineOperand &MO : MI->operands()) {
           if (!MO.isReg() || MO.getReg() != Reg)
             continue;
           if (MO.isUse()) {
             // Reading any undefined lanes?
             LaneBitmask UseMask = TRI->getSubRegIndexLaneMask(MO.getSubReg());
-            if ((UseMask & ~DefinedLanesMask) != 0)
+            if (!(UseMask & ~DefinedLanesMask).none())
               goto CancelKill;
           } else if (MO.getSubReg() == 0) {
             // Writing to the full register?
             assert(MO.isDef());
             IsFullWrite = true;
           }
         }
 
for (MachineOperand &MO : MI->operands()) {
         continue;
       if (TargetRegisterInfo::isVirtualRegister(Reg)) {
         LiveInterval &LI = LIS.getInterval(Reg);
         if (LI.hasSubRanges()) {
           unsigned SubReg = MO.getSubReg();
           LaneBitmask LaneMask = SubReg ? TRI.getSubRegIndexLaneMask(SubReg)
                                         : MRI.getMaxLaneMaskForVReg(Reg);
           for (LiveInterval::SubRange &S : LI.subranges()) {
-            if ((S.LaneMask & LaneMask) == 0)
+            if ((S.LaneMask & LaneMask).none())
               continue;
             updateRange(S, Reg, S.LaneMask);
           }
         }
-        updateRange(LI, Reg, 0);
+        updateRange(LI, Reg, LaneBitmask::getNone());
         continue;
       }
 
       // For physregs, only update the regunits that actually have a
       // precomputed live range.
       for (MCRegUnitIterator Units(Reg, &TRI); Units.isValid(); ++Units)
         if (LiveRange *LR = getRegUnitLI(*Units))
-          updateRange(*LR, *Units, 0);
+          updateRange(*LR, *Units, LaneBitmask::getNone());
     }
     if (hasRegMask)
       updateRegMaskSlots();
   }
 
 private:
   /// Update a single live range, assuming an instruction has been moved from
   /// OldIdx to NewIdx.
   void updateRange(LiveRange &LR, unsigned Reg, LaneBitmask LaneMask) {
     if (!Updated.insert(&LR).second)
       return;
     DEBUG({
       dbgs() << "     ";
       if (TargetRegisterInfo::isVirtualRegister(Reg)) {
         dbgs() << PrintReg(Reg);
-        if (LaneMask != 0)
+        if (!LaneMask.none())
           dbgs() << " L" << PrintLaneMask(LaneMask);
       } else {
         dbgs() << PrintRegUnit(Reg, &TRI);
       }
       dbgs() << ":\t" << LR << '\n';
     });
     if (SlotIndex::isEarlierInstr(OldIdx, NewIdx))
       handleMoveDown(LR);
private:
   SlotIndex findLastUseBefore(SlotIndex Before, unsigned Reg,
                               LaneBitmask LaneMask) {
     if (TargetRegisterInfo::isVirtualRegister(Reg)) {
       SlotIndex LastUse = Before;
       for (MachineOperand &MO : MRI.use_nodbg_operands(Reg)) {
         if (MO.isUndef())
           continue;
         unsigned SubReg = MO.getSubReg();
-        if (SubReg != 0 && LaneMask != 0
-            && (TRI.getSubRegIndexLaneMask(SubReg) & LaneMask) == 0)
+        if (SubReg != 0 && !LaneMask.none()
+            && (TRI.getSubRegIndexLaneMask(SubReg) & LaneMask).none())
           continue;
 
         const MachineInstr &MI = *MO.getParent();
         SlotIndex InstSlot = LIS.getSlotIndexes()->getInstructionIndex(MI);
         if (InstSlot > LastUse && InstSlot < OldIdx)
           LastUse = InstSlot.getRegSlot();
       }
       return LastUse;
for (MachineInstr::mop_iterator OI = MI.operands_begin(),
                                     OE = MI.operands_end();
          OI != OE; ++OI) {
       const MachineOperand &MO = *OI;
       if (!MO.isReg() || MO.getReg() != Reg)
         continue;
 
       unsigned SubReg = MO.getSubReg();
       LaneBitmask Mask = TRI->getSubRegIndexLaneMask(SubReg);
-      if ((Mask & LaneMask) == 0)
+      if ((Mask & LaneMask).none())
         continue;
 
       if (MO.isDef()) {
         if (!isStartValid) {
           if (LII->end.isDead()) {
             SlotIndex prevStart;
             if (LII != LR.begin())
               prevStart = std::prev(LII)->start;

llvm/trunk/lib/CodeGen/LivePhysRegs.cpp

bool LivePhysRegs::available(const MachineRegisterInfo &MRI,
   }
   return true;
 }
 
 /// Add live-in registers of basic block \p MBB to \p LiveRegs.
 void LivePhysRegs::addBlockLiveIns(const MachineBasicBlock &MBB) {
   for (const auto &LI : MBB.liveins()) {
     MCSubRegIndexIterator S(LI.PhysReg, TRI);
-    if (LI.LaneMask == ~0u || (LI.LaneMask != 0 && !S.isValid())) {
+    if (LI.LaneMask.all() || (!LI.LaneMask.none() && !S.isValid())) {
       addReg(LI.PhysReg);
       continue;
     }
-    for (; S.isValid(); ++S)
-      if (LI.LaneMask & TRI->getSubRegIndexLaneMask(S.getSubRegIndex()))
+    for (; S.isValid(); ++S) {
+      unsigned SI = S.getSubRegIndex();
+      if (!(LI.LaneMask & TRI->getSubRegIndexLaneMask(SI)).none())
         addReg(S.getSubReg());
     }
   }
+}
 
 /// Add pristine registers to the given \p LiveRegs. This function removes
 /// actually saved callee save registers when \p InPrologueEpilogue is false.
 static void addPristines(LivePhysRegs &LiveRegs, const MachineFunction &MF,
                          const MachineFrameInfo &MFI,
                          const TargetRegisterInfo &TRI) {
   for (const MCPhysReg *CSR = TRI.getCalleeSavedRegs(&MF); CSR && *CSR; ++CSR)
     LiveRegs.addReg(*CSR);

llvm/trunk/lib/CodeGen/LiveRangeCalc.h

class LiveRangeCalc {
   /// Extend the live range of @p LR to reach all uses of Reg.
   ///
   /// If @p LR is a main range, or if @p LI is null, then all uses must be
   /// jointly dominated by the definitions from @p LR. If @p LR is a subrange
   /// of the live interval @p LI, corresponding to lane mask @p LaneMask,
   /// all uses must be jointly dominated by the definitions from @p LR
   /// together with definitions of other lanes where @p LR becomes undefined
   /// (via <def,read-undef> operands).
-  /// If @p LR is a main range, the @p LaneMask should be set to ~0.
+  /// If @p LR is a main range, the @p LaneMask should be set to ~0, i.e.
+  /// LaneBitmask::getAll().
   void extendToUses(LiveRange &LR, unsigned Reg, LaneBitmask LaneMask,
                     LiveInterval *LI = nullptr);
 
   /// Reset Map and Seen fields.
   void resetLiveOutMap();
 
 public:
   LiveRangeCalc() : MF(nullptr), MRI(nullptr), Indexes(nullptr),
public:
   /// minimal live range.
   void createDeadDefs(LiveRange &LR, unsigned Reg);
 
   /// Extend the live range of @p LR to reach all uses of Reg.
   ///
   /// All uses must be jointly dominated by existing liveness.  PHI-defs are
   /// inserted as needed to preserve SSA form.
   void extendToUses(LiveRange &LR, unsigned PhysReg) {
-    extendToUses(LR, PhysReg, ~0u);
+    extendToUses(LR, PhysReg, LaneBitmask::getAll());
   }
 
   /// Calculates liveness for the register specified in live interval @p LI.
   /// Creates subregister live ranges as needed if subreg liveness tracking is
   /// enabled.
   void calculate(LiveInterval &LI, bool TrackSubRegs);
 
   /// For live interval \p LI with correct SubRanges construct matching

llvm/trunk/lib/CodeGen/LiveRangeCalc.cpp

if (LI.hasSubRanges() || (SubReg != 0 && TrackSubRegs)) {
         LaneBitmask ClassMask = MRI->getMaxLaneMaskForVReg(Reg);
         LI.createSubRangeFrom(*Alloc, ClassMask, LI);
       }
 
       LaneBitmask Mask = SubMask;
       for (LiveInterval::SubRange &S : LI.subranges()) {
         // A Mask for subregs common to the existing subrange and current def.
         LaneBitmask Common = S.LaneMask & Mask;
-        if (Common == 0)
+        if (Common.none())
           continue;
         LiveInterval::SubRange *CommonRange;
         // A Mask for subregs covered by the subrange but not the current def.
-        if (LaneBitmask RM = S.LaneMask & ~Mask) {
+        LaneBitmask RM = S.LaneMask & ~Mask;
+        if (!RM.none()) {
           // Split the subrange S into two parts: one covered by the current
           // def (CommonRange), and the one not affected by it (updated S).
           S.LaneMask = RM;
           CommonRange = LI.createSubRangeFrom(*Alloc, Common, S);
         } else {
           assert(Common == S.LaneMask);
           CommonRange = &S;
         }
         if (MO.isDef())
           createDeadDef(*Indexes, *Alloc, *CommonRange, MO);
         Mask &= ~Common;
       }
       // Create a new SubRange for subregs we did not cover yet.
-      if (Mask != 0) {
+      if (!Mask.none()) {
         LiveInterval::SubRange *NewRange = LI.createSubRange(*Alloc, Mask);
         if (MO.isDef())
           createDeadDef(*Indexes, *Alloc, *NewRange, MO);
       }
     }
 
     // Create the def in the main liverange. We do not have to do this if
     // subranges are tracked as we recreate the main range later in this case.
for (LiveInterval::SubRange &S : LI.subranges()) {
       LiveRangeCalc SubLRC;
       SubLRC.reset(MF, Indexes, DomTree, Alloc);
       SubLRC.extendToUses(S, Reg, S.LaneMask, &LI);
     }
     LI.clear();
     constructMainRangeFromSubranges(LI);
   } else {
     resetLiveOutMap();
-    extendToUses(LI, Reg, ~0u);
+    extendToUses(LI, Reg, LaneBitmask::getAll());
   }
 }
 
 void LiveRangeCalc::constructMainRangeFromSubranges(LiveInterval &LI) {
   // First create dead defs at all defs found in subranges.
   LiveRange &MainRange = LI;
   assert(MainRange.segments.empty() && MainRange.valnos.empty() &&
          "Expect empty main liverange");
 
   for (const LiveInterval::SubRange &SR : LI.subranges()) {
     for (const VNInfo *VNI : SR.valnos) {
       if (!VNI->isUnused() && !VNI->isPHIDef())
         MainRange.createDeadDef(VNI->def, *Alloc);
     }
   }
   resetLiveOutMap();
-  extendToUses(MainRange, LI.reg, ~0U, &LI);
+  extendToUses(MainRange, LI.reg, LaneBitmask::getAll(), &LI);
 }
 
 void LiveRangeCalc::createDeadDefs(LiveRange &LR, unsigned Reg) {
   assert(MRI && Indexes && "call reset() first");
 
   // Visit all def operands. If the same instruction has multiple defs of Reg,
   // LR.createDeadDef() will deduplicate.
   for (MachineOperand &MO : MRI->def_operands(Reg))
     createDeadDef(*Indexes, *Alloc, LR, MO);
 }
 
 
 void LiveRangeCalc::extendToUses(LiveRange &LR, unsigned Reg, LaneBitmask Mask,
                                  LiveInterval *LI) {
   SmallVector<SlotIndex, 4> Undefs;
   if (LI != nullptr)
     LI->computeSubRangeUndefs(Undefs, Mask, *MRI, *Indexes);
 
   // Visit all operands that read Reg. This may include partial defs.
-  bool IsSubRange = (Mask != ~0U);
+  bool IsSubRange = !Mask.all();
   const TargetRegisterInfo &TRI = *MRI->getTargetRegisterInfo();
   for (MachineOperand &MO : MRI->reg_nodbg_operands(Reg)) {
     // Clear all kill flags. They will be reinserted after register allocation
     // by LiveIntervalAnalysis::addKillFlags().
     if (MO.isUse())
       MO.setIsKill(false);
     // MO::readsReg returns "true" for subregister defs. This is for keeping
     // liveness of the entire register (i.e. for the main range of the live
     // interval). For subranges, definitions of non-overlapping subregisters
     // do not count as uses.
     if (!MO.readsReg() || (IsSubRange && MO.isDef()))
       continue;
 
     unsigned SubReg = MO.getSubReg();
     if (SubReg != 0) {
       LaneBitmask SLM = TRI.getSubRegIndexLaneMask(SubReg);
       if (MO.isDef())
         SLM = ~SLM;
       // Ignore uses not reading the current (sub)range.
-      if ((SLM & Mask) == 0)
+      if ((SLM & Mask).none())
         continue;
     }
 
     // Determine the actual place of the use.
     const MachineInstr *MI = MO.getParent();
     unsigned OpNo = (&MO - &MI->getOperand(0));
     SlotIndex UseIdx;
     if (MI->isPHI()) {

llvm/trunk/lib/CodeGen/LiveRangeEdit.cpp

bool LiveRangeEdit::useIsKill(const LiveInterval &LI,
   const MachineInstr &MI = *MO.getParent();
   SlotIndex Idx = LIS.getInstructionIndex(MI).getRegSlot();
   if (LI.Query(Idx).isKill())
     return true;
   const TargetRegisterInfo &TRI = *MRI.getTargetRegisterInfo();
   unsigned SubReg = MO.getSubReg();
   LaneBitmask LaneMask = TRI.getSubRegIndexLaneMask(SubReg);
   for (const LiveInterval::SubRange &S : LI.subranges()) {
-    if ((S.LaneMask & LaneMask) != 0 && S.Query(Idx).isKill())
+    if (!(S.LaneMask & LaneMask).none() && S.Query(Idx).isKill())
       return true;
   }
   return false;
 }
 
 /// Find all live intervals that need to shrink, then remove the instruction.
 void LiveRangeEdit::eliminateDeadDef(MachineInstr *MI, ToShrinkSet &ToShrink,
                                      AliasAnalysis *AA) {

llvm/trunk/lib/CodeGen/LiveRegMatrix.cpp

 static bool foreachUnit(const TargetRegisterInfo *TRI,
                         LiveInterval &VRegInterval, unsigned PhysReg,
                         Callable Func) {
   if (VRegInterval.hasSubRanges()) {
     for (MCRegUnitMaskIterator Units(PhysReg, TRI); Units.isValid(); ++Units) {
       unsigned Unit = (*Units).first;
       LaneBitmask Mask = (*Units).second;
       for (LiveInterval::SubRange &S : VRegInterval.subranges()) {
-        if (S.LaneMask & Mask) {
+        if (!(S.LaneMask & Mask).none()) {
           if (Func(Unit, S))
             return true;
           break;
         }
       }
     }
   } else {
     for (MCRegUnitIterator Units(PhysReg, TRI); Units.isValid(); ++Units) {

llvm/trunk/lib/CodeGen/MIRParser/MIParser.cpp

if (Token.isNewlineOrEOF()) // Allow an empty list of liveins.
     return false;
   do {
     if (Token.isNot(MIToken::NamedRegister))
       return error("expected a named register");
     unsigned Reg = 0;
     if (parseNamedRegister(Reg))
       return true;
     lex();
-    LaneBitmask Mask = ~LaneBitmask(0);
+    LaneBitmask Mask = LaneBitmask::getAll();
     if (consumeIfPresent(MIToken::colon)) {
       // Parse lane mask.
       if (Token.isNot(MIToken::IntegerLiteral) &&
           Token.isNot(MIToken::HexLiteral))
         return error("expected a lane mask");
-      static_assert(sizeof(LaneBitmask) == sizeof(unsigned), "");
-      if (getUnsigned(Mask))
+      static_assert(sizeof(LaneBitmask::Type) == sizeof(unsigned),
+                    "Use correct get-function for lane mask");
+      LaneBitmask::Type V;
+      if (getUnsigned(V))
         return error("invalid lane mask value");
+      Mask = LaneBitmask(V);
       lex();
     }
     MBB.addLiveIn(Reg, Mask);
   } while (consumeIfPresent(MIToken::comma));
   return false;
 }
 
 bool MIParser::parseBasicBlockSuccessors(MachineBasicBlock &MBB) {

llvm/trunk/lib/CodeGen/MIRPrinter.cpp

void MIPrinter::print(const MachineBasicBlock &MBB) {
   if (!MBB.livein_empty()) {
     OS.indent(2) << "liveins: ";
     bool First = true;
     for (const auto &LI : MBB.liveins()) {
       if (!First)
         OS << ", ";
       First = false;
       printReg(LI.PhysReg, OS, TRI);
-      if (LI.LaneMask != ~0u)
+      if (!LI.LaneMask.all())
         OS << ":0x" << PrintLaneMask(LI.LaneMask);
     }
     OS << "\n";
     HasLineAttributes = true;
   }
 
   if (HasLineAttributes)
     OS << "\n";

llvm/trunk/lib/CodeGen/MachineBasicBlock.cpp

void MachineBasicBlock::print(raw_ostream &OS, ModuleSlotTracker &MST,
   OS << '\n';
 
   const TargetRegisterInfo *TRI = MF->getSubtarget().getRegisterInfo();
   if (!livein_empty()) {
     if (Indexes) OS << '\t';
     OS << "    Live Ins:";
     for (const auto &LI : make_range(livein_begin(), livein_end())) {
       OS << ' ' << PrintReg(LI.PhysReg, TRI);
-      if (LI.LaneMask != ~0u)
+      if (!LI.LaneMask.all())
         OS << ':' << PrintLaneMask(LI.LaneMask);
     }
     OS << '\n';
   }
   // Print the preds of this block according to the CFG.
   if (!pred_empty()) {
     if (Indexes) OS << '\t';
     OS << "    Predecessors according to CFG:";

 
 void MachineBasicBlock::removeLiveIn(MCPhysReg Reg, LaneBitmask LaneMask) {
   LiveInVector::iterator I = find_if(
       LiveIns, [Reg](const RegisterMaskPair &LI) { return LI.PhysReg == Reg; });
   if (I == LiveIns.end())
     return;
 
   I->LaneMask &= ~LaneMask;
-  if (I->LaneMask == 0)
+  if (I->LaneMask.none())
     LiveIns.erase(I);
 }
 
 bool MachineBasicBlock::isLiveIn(MCPhysReg Reg, LaneBitmask LaneMask) const {
   livein_iterator I = find_if(
       LiveIns, [Reg](const RegisterMaskPair &LI) { return LI.PhysReg == Reg; });
-  return I != livein_end() && (I->LaneMask & LaneMask) != 0;
+  return I != livein_end() && !(I->LaneMask & LaneMask).none();
 }
 
 void MachineBasicBlock::sortUniqueLiveIns() {
   std::sort(LiveIns.begin(), LiveIns.end(),
             [](const RegisterMaskPair &LI0, const RegisterMaskPair &LI1) {
               return LI0.PhysReg < LI1.PhysReg;
             });
   // Liveins are sorted by physreg now we can merge their lanemasks.

llvm/trunk/lib/CodeGen/MachinePipeliner.cpp

for (const MachineOperand &MO : MI->operands())
       }
   }
   for (SUnit *SU : NS)
     for (const MachineOperand &MO : SU->getInstr()->operands())
       if (MO.isReg() && MO.isDef() && !MO.isDead()) {
         unsigned Reg = MO.getReg();
         if (TargetRegisterInfo::isVirtualRegister(Reg)) {
           if (!Uses.count(Reg))
-            LiveOutRegs.push_back(RegisterMaskPair(Reg, 0));
+            LiveOutRegs.push_back(RegisterMaskPair(Reg,
+                                                   LaneBitmask::getNone()));
         } else if (MRI.isAllocatable(Reg)) {
           for (MCRegUnitIterator Units(Reg, TRI); Units.isValid(); ++Units)
             if (!Uses.count(*Units))
-              LiveOutRegs.push_back(RegisterMaskPair(*Units, 0));
+              LiveOutRegs.push_back(RegisterMaskPair(*Units,
+                                                     LaneBitmask::getNone()));
         }
       }
   RPTracker.addLiveRegs(LiveOutRegs);
 }
 
 /// A heuristic to filter nodes in recurrent node-sets if the register
 /// pressure of a set is too high.
 void SwingSchedulerDAG::registerPressureFilter(NodeSetType &NodeSets) {

llvm/trunk/lib/CodeGen/MachineScheduler.cpp

for (const MachineOperand &MO : MI.operands()) {
 
     // Record this local VReg use.
     VReg2SUnitMultiMap::iterator UI = VRegUses.find(Reg);
     for (; UI != VRegUses.end(); ++UI) {
       if (UI->SU == &SU)
         break;
     }
     if (UI == VRegUses.end())
-      VRegUses.insert(VReg2SUnit(Reg, 0, &SU));
+      VRegUses.insert(VReg2SUnit(Reg, LaneBitmask::getNone(), &SU));
   }
 }
 
 /// enterRegion - Called back from MachineScheduler::runOnMachineFunction after
 /// crossing a scheduling boundary. [begin, end) includes all instructions in
 /// the region, including the boundary itself and single-instruction regions
 /// that don't get scheduled.
 void ScheduleDAGMILive::enterRegion(MachineBasicBlock *bb,
for (const RegisterMaskPair &P : LiveUses) {
     if (!TRI->isVirtualRegister(Reg))
       continue;
 
     if (ShouldTrackLaneMasks) {
       // If the register has just become live then other uses won't change
       // this fact anymore => decrement pressure.
       // If the register has just become dead then other uses make it come
       // back to life => increment pressure.
-      bool Decrement = P.LaneMask != 0;
+      bool Decrement = !P.LaneMask.none();
 
       for (const VReg2SUnit &V2SU
            : make_range(VRegUses.find(Reg), VRegUses.end())) {
         SUnit &SU = *V2SU.SU;
         if (SU.isScheduled || &SU == &ExitSU)
           continue;
 
         PressureDiff &PDiff = getPressureDiff(&SU);
         PDiff.addPressureChange(Reg, Decrement, &MRI);
         DEBUG(
           dbgs() << "  UpdateRegP: SU(" << SU.NodeNum << ") "
                  << PrintReg(Reg, TRI) << ':' << PrintLaneMask(P.LaneMask)
                  << ' ' << *SU.getInstr();
           dbgs() << "              to ";
           PDiff.dump(*TRI);
         );
       }
     } else {
-      assert(P.LaneMask != 0);
+      assert(!P.LaneMask.none());
       DEBUG(dbgs() << "  LiveReg: " << PrintVRegOrUnit(Reg, TRI) << "\n");
       // This may be called before CurrentBottom has been initialized. However,
       // BotRPTracker must have a valid position. We want the value live into the
       // instruction or live out of the block, so ask for the previous
       // instruction's live-out.
       const LiveInterval &LI = LIS->getInterval(Reg);
       VNInfo *VNI;
       MachineBasicBlock::const_iterator I =

llvm/trunk/lib/CodeGen/MachineVerifier.cpp

struct MachineVerifier {
     void report_context_vreg(unsigned VReg) const;
     void report_context_vreg_regunit(unsigned VRegOrRegUnit) const;
 
     void verifyInlineAsm(const MachineInstr *MI);
 
     void checkLiveness(const MachineOperand *MO, unsigned MONum);
     void checkLivenessAtUse(const MachineOperand *MO, unsigned MONum,
                             SlotIndex UseIdx, const LiveRange &LR, unsigned Reg,
-                            LaneBitmask LaneMask = 0);
+                            LaneBitmask LaneMask = LaneBitmask::getNone());
     void checkLivenessAtDef(const MachineOperand *MO, unsigned MONum,
                             SlotIndex DefIdx, const LiveRange &LR, unsigned Reg,
-                            LaneBitmask LaneMask = 0);
+                            LaneBitmask LaneMask = LaneBitmask::getNone());
 
     void markReachable(const MachineBasicBlock *MBB);
     void calcRegsPassed();
     void checkPHIOps(const MachineBasicBlock *MBB);
 
     void calcRegsRequired();
     void verifyLiveVariables();
     void verifyLiveIntervals();
     void verifyLiveInterval(const LiveInterval&);
     void verifyLiveRangeValue(const LiveRange&, const VNInfo*, unsigned,
-                              unsigned);
+                              LaneBitmask);
     void verifyLiveRangeSegment(const LiveRange&,
                                 const LiveRange::const_iterator I, unsigned,
-                                unsigned);
-    void verifyLiveRange(const LiveRange&, unsigned, LaneBitmask LaneMask = 0);
+                                LaneBitmask);
+    void verifyLiveRange(const LiveRange&, unsigned,
+                         LaneBitmask LaneMask = LaneBitmask::getNone());
 
     void verifyStackFrame();
 
     void verifySlotIndexes() const;
     void verifyProperties(const MachineFunction &MF);
   };
 
   struct MachineVerifierPass : public MachineFunctionPass {

 void MachineVerifier::report_context(const LiveInterval &LI) const {
   errs() << "- interval:    " << LI << '\n';
 }
 
 void MachineVerifier::report_context(const LiveRange &LR, unsigned VRegUnit,
                                      LaneBitmask LaneMask) const {
   report_context_liverange(LR);
   report_context_vreg_regunit(VRegUnit);
-  if (LaneMask != 0)
+  if (!LaneMask.none())
     report_context_lanemask(LaneMask);
 }
 
 void MachineVerifier::report_context(const LiveRange::Segment &S) const {
   errs() << "- segment:     " << S << '\n';
 }
 
 void MachineVerifier::report_context(const VNInfo &VNI) const {

 }
 
 void MachineVerifier::checkLivenessAtUse(const MachineOperand *MO,
     unsigned MONum, SlotIndex UseIdx, const LiveRange &LR, unsigned VRegOrUnit,
     LaneBitmask LaneMask) {
   LiveQueryResult LRQ = LR.Query(UseIdx);
   // Check if we have a segment at the use, note however that we only need one
   // live subregister range, the others may be dead.
-  if (!LRQ.valueIn() && LaneMask == 0) {
+  if (!LRQ.valueIn() && LaneMask.none()) {
     report("No live segment at use", MO, MONum);
     report_context_liverange(LR);
     report_context_vreg_regunit(VRegOrUnit);
     report_context(UseIdx);
   }
   if (MO->isKill() && !LRQ.isKill()) {
     report("Live range continues after kill flag", MO, MONum);
     report_context_liverange(LR);
     report_context_vreg_regunit(VRegOrUnit);
-    if (LaneMask != 0)
+    if (!LaneMask.none())
       report_context_lanemask(LaneMask);
     report_context(UseIdx);
   }
 }
 
 void MachineVerifier::checkLivenessAtDef(const MachineOperand *MO,
     unsigned MONum, SlotIndex DefIdx, const LiveRange &LR, unsigned VRegOrUnit,
     LaneBitmask LaneMask) {
   if (const VNInfo *VNI = LR.getVNInfoAt(DefIdx)) {
     assert(VNI && "NULL valno is not allowed");
     if (VNI->def != DefIdx) {
       report("Inconsistent valno->def", MO, MONum);
       report_context_liverange(LR);
       report_context_vreg_regunit(VRegOrUnit);
-      if (LaneMask != 0)
+      if (!LaneMask.none())
         report_context_lanemask(LaneMask);
       report_context(*VNI);
       report_context(DefIdx);
     }
   } else {
     report("No live segment at def", MO, MONum);
     report_context_liverange(LR);
     report_context_vreg_regunit(VRegOrUnit);
-    if (LaneMask != 0)
+    if (!LaneMask.none())
       report_context_lanemask(LaneMask);
     report_context(DefIdx);
   }
   // Check that, if the dead def flag is present, LiveInts agree.
   if (MO->isDead()) {
     LiveQueryResult LRQ = LR.Query(DefIdx);
     if (!LRQ.isDeadDef()) {
       // In case of physregs we can have a non-dead definition on another
if (!LRQ.isDeadDef()) {
           }
         }
       }
 
       if (!otherDef) {
         report("Live range continues after dead def flag", MO, MONum);
         report_context_liverange(LR);
         report_context_vreg_regunit(VRegOrUnit);
-        if (LaneMask != 0)
+        if (!LaneMask.none())
           report_context_lanemask(LaneMask);
       }
     }
   }
 }
 
 void MachineVerifier::checkLiveness(const MachineOperand *MO, unsigned MONum) {
   const MachineInstr *MI = MO->getParent();
if (LiveInts && !LiveInts->isNotInMIMap(*MI)) {
           const LiveInterval &LI = LiveInts->getInterval(Reg);
           checkLivenessAtUse(MO, MONum, UseIdx, LI, Reg);
 
           if (LI.hasSubRanges() && !MO->isDef()) {
             unsigned SubRegIdx = MO->getSubReg();
             LaneBitmask MOMask = SubRegIdx != 0
                                ? TRI->getSubRegIndexLaneMask(SubRegIdx)
                                : MRI->getMaxLaneMaskForVReg(Reg);
-            LaneBitmask LiveInMask = 0;
+            LaneBitmask LiveInMask;
             for (const LiveInterval::SubRange &SR : LI.subranges()) {
-              if ((MOMask & SR.LaneMask) == 0)
+              if ((MOMask & SR.LaneMask).none())
                 continue;
               checkLivenessAtUse(MO, MONum, UseIdx, SR, Reg, SR.LaneMask);
               LiveQueryResult LRQ = SR.Query(UseIdx);
               if (LRQ.valueIn())
                 LiveInMask |= SR.LaneMask;
             }
             // At least parts of the register has to be live at the use.
-            if ((LiveInMask & MOMask) == 0) {
+            if ((LiveInMask & MOMask).none()) {
               report("No live subrange at use", MO, MONum);
               report_context(LI);
               report_context(UseIdx);
             }
           }
         } else {
           report("Virtual register has no live interval", MO, MONum);
         }
if (LiveInts && !LiveInts->isNotInMIMap(*MI)) {
           checkLivenessAtDef(MO, MONum, DefIdx, LI, Reg);
 
           if (LI.hasSubRanges()) {
             unsigned SubRegIdx = MO->getSubReg();
             LaneBitmask MOMask = SubRegIdx != 0
               ? TRI->getSubRegIndexLaneMask(SubRegIdx)
               : MRI->getMaxLaneMaskForVReg(Reg);
             for (const LiveInterval::SubRange &SR : LI.subranges()) {
-              if ((SR.LaneMask & MOMask) == 0)
+              if ((SR.LaneMask & MOMask).none())
                 continue;
               checkLivenessAtDef(MO, MONum, DefIdx, SR, Reg, SR.LaneMask);
             }
           }
         } else {
           report("Virtual register has no Live interval", MO, MONum);
         }
       }
for (ConstMIBundleOperands MOI(*MI); MOI.isValid(); ++MOI) {
       if (TargetRegisterInfo::isVirtualRegister(Reg)) {
         if (MOI->getReg() != Reg)
           continue;
       } else {
         if (!TargetRegisterInfo::isPhysicalRegister(MOI->getReg()) ||
             !TRI->hasRegUnit(MOI->getReg(), Reg))
           continue;
       }
-      if (LaneMask != 0 &&
-          (TRI->getSubRegIndexLaneMask(MOI->getSubReg()) & LaneMask) == 0)
+      if (!LaneMask.none() &&
+          (TRI->getSubRegIndexLaneMask(MOI->getSubReg()) & LaneMask).none())
         continue;
       hasDef = true;
       if (MOI->isEarlyClobber())
         isEarlyClobber = true;
     }
 
     if (!hasDef) {
       report("Defining instruction does not modify register", MI);
if (TargetRegisterInfo::isVirtualRegister(Reg)) {
     // use, or a dead flag on a def.
     bool hasRead = false;
     bool hasSubRegDef = false;
     bool hasDeadDef = false;
     for (ConstMIBundleOperands MOI(*MI); MOI.isValid(); ++MOI) {
       if (!MOI->isReg() || MOI->getReg() != Reg)
         continue;
       unsigned Sub = MOI->getSubReg();
-      LaneBitmask SLM = Sub != 0 ? TRI->getSubRegIndexLaneMask(Sub) : ~0U;
+      LaneBitmask SLM = Sub != 0 ? TRI->getSubRegIndexLaneMask(Sub)
+                                 : LaneBitmask::getAll();
       if (MOI->isDef()) {
         if (Sub != 0) {
           hasSubRegDef = true;
           // An operand vreg0:sub0<def> reads vreg0:sub1..n. Invert the lane
           // mask for subregister defs. Read-undef defs will be handled by
           // readsReg below.
           SLM = ~SLM;
         }
         if (MOI->isDead())
           hasDeadDef = true;
       }
-      if (LaneMask != 0 && !(LaneMask & SLM))
+      if (!LaneMask.none() && (LaneMask & SLM).none())
         continue;
       if (MOI->readsReg())
         hasRead = true;
     }
     if (S.end.isDead()) {
       // Make sure that the corresponding machine operand for a "dead" live
       // range has the dead flag. We cannot perform this check for subregister
       // liveranges as partially dead values are allowed.
-      if (LaneMask == 0 && !hasDeadDef) {
+      if (LaneMask.none() && !hasDeadDef) {
         report("Instruction ending live segment on dead slot has no dead flag",
                MI);
         report_context(LR, Reg, LaneMask);
         report_context(S);
       }
     } else {
       if (!hasRead) {
         // When tracking subregister liveness, the main range must start new
         // values on partial register writes, even if there is no read.
-        if (!MRI->shouldTrackSubRegLiveness(Reg) || LaneMask != 0 ||
+        if (!MRI->shouldTrackSubRegLiveness(Reg) || !LaneMask.none() ||
             !hasSubRegDef) {
           report("Instruction ending live segment doesn't read the register",
                  MI);
           report_context(LR, Reg, LaneMask);
           report_context(S);
         }
       }
     }
for (;;) {
     // Check that VNI is live-out of all predecessors.
     for (MachineBasicBlock::const_pred_iterator PI = MFI->pred_begin(),
          PE = MFI->pred_end(); PI != PE; ++PI) {
       SlotIndex PEnd = LiveInts->getMBBEndIdx(*PI);
       const VNInfo *PVNI = LR.getVNInfoBefore(PEnd);
 
       // All predecessors must have a live-out value if this is not a
       // subregister liverange.
-      if (!PVNI && LaneMask == 0) {
+      if (!PVNI && LaneMask.none()) {
         report("Register not marked live out of predecessor", *PI);
         report_context(LR, Reg, LaneMask);
         report_context(*VNI);
         errs() << " live into BB#" << MFI->getNumber()
                << '@' << LiveInts->getMBBStartIdx(&*MFI) << ", not live before "
                << PEnd << '\n';
         continue;
       }
for (LiveRange::const_iterator I = LR.begin(), E = LR.end(); I != E; ++I)
     verifyLiveRangeSegment(LR, I, Reg, LaneMask);
 }
 
 void MachineVerifier::verifyLiveInterval(const LiveInterval &LI) {
   unsigned Reg = LI.reg;
   assert(TargetRegisterInfo::isVirtualRegister(Reg));
   verifyLiveRange(LI, Reg);
 
-  LaneBitmask Mask = 0;
+  LaneBitmask Mask;
   LaneBitmask MaxMask = MRI->getMaxLaneMaskForVReg(Reg);
   for (const LiveInterval::SubRange &SR : LI.subranges()) {
-    if ((Mask & SR.LaneMask) != 0) {
+    if (!(Mask & SR.LaneMask).none()) {
       report("Lane masks of sub ranges overlap in live interval", MF);
       report_context(LI);
     }
-    if ((SR.LaneMask & ~MaxMask) != 0) {
+    if (!(SR.LaneMask & ~MaxMask).none()) {
       report("Subrange lanemask is invalid", MF);
       report_context(LI);
     }
     if (SR.empty()) {
       report("Subrange must not be empty", MF);
       report_context(SR, LI.reg, SR.LaneMask);
     }
     Mask |= SR.LaneMask;

llvm/trunk/lib/CodeGen/PeepholeOptimizer.cpp

ValueTrackerResult ValueTracker::getNextSourceFromInsertSubreg() {
   if (MRI.getRegClass(MODef.getReg()) != MRI.getRegClass(BaseReg.Reg) ||
       BaseReg.SubReg)
     return ValueTrackerResult();
 
   // Get the TRI and check if the inserted sub-register overlaps with the
   // sub-register we are tracking.
   const TargetRegisterInfo *TRI = MRI.getTargetRegisterInfo();
   if (!TRI ||
-      (TRI->getSubRegIndexLaneMask(DefSubReg) &
-       TRI->getSubRegIndexLaneMask(InsertedReg.SubIdx)) != 0)
+      !(TRI->getSubRegIndexLaneMask(DefSubReg) &
+        TRI->getSubRegIndexLaneMask(InsertedReg.SubIdx)).none())
     return ValueTrackerResult();
   // At this point, the value is available in v0 via the same subreg
   // we used for Def.
   return ValueTrackerResult(BaseReg.Reg, DefSubReg);
 }
 
 ValueTrackerResult ValueTracker::getNextSourceFromExtractSubreg() {
   assert((Def->isExtractSubreg() ||

llvm/trunk/lib/CodeGen/RegisterCoalescer.cpp

if (IntB.hasSubRanges()) {
     for (LiveInterval::SubRange &SA : IntA.subranges()) {
       VNInfo *ASubValNo = SA.getVNInfoAt(AIdx);
       assert(ASubValNo != nullptr);
 
       LaneBitmask AMask = SA.LaneMask;
       for (LiveInterval::SubRange &SB : IntB.subranges()) {
         LaneBitmask BMask = SB.LaneMask;
         LaneBitmask Common = BMask & AMask;
-        if (Common == 0)
+        if (Common.none())
           continue;
 
         DEBUG( dbgs() << "\t\tCopy_Merge " << PrintLaneMask(BMask)
                       << " into " << PrintLaneMask(Common) << '\n');
         LaneBitmask BRest = BMask & ~AMask;
         LiveInterval::SubRange *CommonRange;
-        if (BRest != 0) {
+        if (!BRest.none()) {
           SB.LaneMask = BRest;
           DEBUG(dbgs() << "\t\tReduce Lane to " << PrintLaneMask(BRest)
                        << '\n');
           // Duplicate SubRange for newly merged common stuff.
           CommonRange = IntB.createSubRangeFrom(Allocator, Common, SB);
         } else {
           // We van reuse the L SubRange.
           SB.LaneMask = Common;
           CommonRange = &SB;
         }
         LiveRange RangeCopy(SB, Allocator);
 
         VNInfo *BSubValNo = CommonRange->getVNInfoAt(CopyIdx);
         assert(BSubValNo->def == CopyIdx);
         BSubValNo->def = ASubValNo->def;
         addSegmentsWithValNo(*CommonRange, BSubValNo, SA, ASubValNo);
         AMask &= ~BMask;
       }
-      if (AMask != 0) {
+      if (!AMask.none()) {
         DEBUG(dbgs() << "\t\tNew Lane " << PrintLaneMask(AMask) << '\n');
         LiveRange *NewRange = IntB.createSubRange(Allocator, AMask);
         VNInfo *BSubValNo = NewRange->getNextValue(CopyIdx, Allocator);
         addSegmentsWithValNo(*NewRange, BSubValNo, SA, ASubValNo);
       }
     }
   }
 
if (NewIdx == 0 && DstInt.hasSubRanges()) {
           CurrIdx.getRegSlot(NewMI.getOperand(0).isEarlyClobber());
       LaneBitmask MaxMask = MRI->getMaxLaneMaskForVReg(DstReg);
       VNInfo::Allocator& Alloc = LIS->getVNInfoAllocator();
       for (LiveInterval::SubRange &SR : DstInt.subranges()) {
         if (!SR.liveAt(DefIndex))
           SR.createDeadDef(DefIndex, Alloc);
         MaxMask &= ~SR.LaneMask;
       }
-      if (MaxMask != 0) {
+      if (!MaxMask.none()) {
         LiveInterval::SubRange *SR = DstInt.createSubRange(Alloc, MaxMask);
         SR->createDeadDef(DefIndex, Alloc);
       }
     }
   } else if (NewMI.getOperand(0).getReg() != CopyDstReg) {
     // The New instruction may be defining a sub-register of what's actually
     // been asked for. If so it must implicitly define the whole thing.
     assert(TargetRegisterInfo::isPhysicalRegister(DstReg) &&
bool RegisterCoalescer::eliminateUndefCopy(MachineInstr *CopyMI) {
   isMoveInstr(*TRI, CopyMI, SrcReg, DstReg, SrcSubIdx, DstSubIdx);
 
   SlotIndex Idx = LIS->getInstructionIndex(*CopyMI);
   const LiveInterval &SrcLI = LIS->getInterval(SrcReg);
   // CopyMI is undef iff SrcReg is not live before the instruction.
   if (SrcSubIdx != 0 && SrcLI.hasSubRanges()) {
     LaneBitmask SrcMask = TRI->getSubRegIndexLaneMask(SrcSubIdx);
     for (const LiveInterval::SubRange &SR : SrcLI.subranges()) {
-      if ((SR.LaneMask & SrcMask) == 0)
+      if ((SR.LaneMask & SrcMask).none())
         continue;
       if (SR.liveAt(Idx))
         return false;
     }
   } else if (SrcLI.liveAt(Idx))
     return false;
 
   DEBUG(dbgs() << "\tEliminating copy of <undef> value\n");
 
   // Remove any DstReg segments starting at the instruction.
   LiveInterval &DstLI = LIS->getInterval(DstReg);
   SlotIndex RegIndex = Idx.getRegSlot();
   // Remove value or merge with previous one in case of a subregister def.
   if (VNInfo *PrevVNI = DstLI.getVNInfoAt(Idx)) {
     VNInfo *VNI = DstLI.getVNInfoAt(RegIndex);
     DstLI.MergeValueNumberInto(VNI, PrevVNI);
 
     // The affected subregister segments can be removed.
     LaneBitmask DstMask = TRI->getSubRegIndexLaneMask(DstSubIdx);
     for (LiveInterval::SubRange &SR : DstLI.subranges()) {
-      if ((SR.LaneMask & DstMask) == 0)
+      if ((SR.LaneMask & DstMask).none())
         continue;
 
       VNInfo *SVNI = SR.getVNInfoAt(RegIndex);
       assert(SVNI != nullptr && SlotIndex::isSameInstr(SVNI->def, RegIndex));
       SR.removeValNo(SVNI);
     }
     DstLI.removeEmptySubRanges();
   } else
     LIS->removeVRegDefAt(DstLI, RegIndex);
 
   // Mark uses as undef.
   for (MachineOperand &MO : MRI->reg_nodbg_operands(DstReg)) {
     if (MO.isDef() /*|| MO.isUndef()*/)
       continue;
     const MachineInstr &MI = *MO.getParent();
     SlotIndex UseIdx = LIS->getInstructionIndex(MI);
     LaneBitmask UseMask = TRI->getSubRegIndexLaneMask(MO.getSubReg());
     bool isLive;
-    if (UseMask != ~0u && DstLI.hasSubRanges()) {
+    if (!UseMask.all() && DstLI.hasSubRanges()) {
       isLive = false;
       for (const LiveInterval::SubRange &SR : DstLI.subranges()) {
-        if ((SR.LaneMask & UseMask) == 0)
+        if ((SR.LaneMask & UseMask).none())
           continue;
         if (SR.liveAt(UseIdx)) {
           isLive = true;
           break;
         }
       }
     } else
       isLive = DstLI.liveAt(UseIdx);

 
 void RegisterCoalescer::addUndefFlag(const LiveInterval &Int, SlotIndex UseIdx,
                                      MachineOperand &MO, unsigned SubRegIdx) {
   LaneBitmask Mask = TRI->getSubRegIndexLaneMask(SubRegIdx);
   if (MO.isDef())
     Mask = ~Mask;
   bool IsUndef = true;
   for (const LiveInterval::SubRange &S : Int.subranges()) {
-    if ((S.LaneMask & Mask) == 0)
+    if ((S.LaneMask & Mask).none())
       continue;
     if (S.liveAt(UseIdx)) {
       IsUndef = false;
       break;
     }
   }
   if (IsUndef) {
     MO.setIsUndef(true);
DEBUG({
                << PrintReg(CP.getSrcReg()) << " in "
                << TRI->getSubRegIndexName(CP.getSrcIdx()) << '\n';
       else
         dbgs() << PrintReg(CP.getSrcReg(), TRI) << " in "
                << PrintReg(CP.getDstReg(), TRI, CP.getSrcIdx()) << '\n';
     });
   }
 
-  ShrinkMask = 0;
+  ShrinkMask = LaneBitmask::getNone();
   ShrinkMainRange = false;
 
   // Okay, attempt to join these two intervals.  On failure, this returns false.
   // Otherwise, if one of the intervals being joined is a physreg, this method
   // always canonicalizes DstInt to be it.  The output "SrcInt" will not have
   // been modified, so we can use this information below to update aliases.
   if (!joinIntervals(CP)) {
     // Coalescing failed.
bool RegisterCoalescer::joinCopy(MachineInstr *CopyMI, bool &Again) {
 
   // Rewrite all SrcReg operands to DstReg.
   // Also update DstReg operands to include DstIdx if it is set.
   if (CP.getDstIdx())
     updateRegDefsUses(CP.getDstReg(), CP.getDstReg(), CP.getDstIdx());
   updateRegDefsUses(CP.getSrcReg(), CP.getDstReg(), CP.getSrcIdx());
 
   // Shrink subregister ranges if necessary.
-  if (ShrinkMask != 0) {
+  if (!ShrinkMask.none()) {
     LiveInterval &LI = LIS->getInterval(CP.getDstReg());
     for (LiveInterval::SubRange &S : LI.subranges()) {
-      if ((S.LaneMask & ShrinkMask) == 0)
+      if ((S.LaneMask & ShrinkMask).none())
         continue;
       DEBUG(dbgs() << "Shrink LaneUses (Lane " << PrintLaneMask(S.LaneMask)
                    << ")\n");
       LIS->shrinkToUses(S, LI.reg);
     }
     LI.removeEmptySubRanges();
   }
   if (ShrinkMainRange) {
struct Val {
 
     /// True when the live range of this value will be pruned because of an
     /// overlapping CR_Replace value in the other live range.
     bool Pruned;
 
     /// True once Pruned above has been computed.
     bool PrunedComputed;
 
-    Val() : Resolution(CR_Keep), WriteLanes(0), ValidLanes(0),
+    Val() : Resolution(CR_Keep), WriteLanes(), ValidLanes(),
             RedefVNI(nullptr), OtherVNI(nullptr), ErasableImplicitDef(false),
             Pruned(false), PrunedComputed(false) {}
 
-    bool isAnalyzed() const { return WriteLanes != 0; }
+    bool isAnalyzed() const { return !WriteLanes.none(); }
   };
 
   /// One entry per value number in LI.
   SmallVector<Val, 8> Vals;
 
   /// Compute the bitmask of lanes actually written by DefMI.
   /// Set Redef if there are any partial register definitions that depend on the
   /// previous value of the register.
public:
 
   /// Get the value assignments suitable for passing to LiveInterval::join.
   const int *getAssignments() const { return Assignments.data(); }
 };
 } // end anonymous namespace
 
 LaneBitmask JoinVals::computeWriteLanes(const MachineInstr *DefMI, bool &Redef)
   const {
-  LaneBitmask L = 0;
+  LaneBitmask L;
   for (const MachineOperand &MO : DefMI->operands()) {
     if (!MO.isReg() || MO.getReg() != Reg || !MO.isDef())
       continue;
     L |= TRI->getSubRegIndexLaneMask(
            TRI->composeSubRegIndices(SubIdx, MO.getSubReg()));
     if (MO.readsReg())
       Redef = true;
   }
if (!SubRangeJoin || !LI.hasSubRanges()) {
       LiveQueryResult LRQ = LI.Query(Def);
       ValueIn = LRQ.valueIn();
     } else {
       // Query subranges. Pick the first matching one.
       ValueIn = nullptr;
       for (const LiveInterval::SubRange &S : LI.subranges()) {
         // Transform lanemask to a mask in the joined live interval.
         LaneBitmask SMask = TRI->composeSubRegIndexLaneMask(SubIdx, S.LaneMask);
-        if ((SMask & LaneMask) == 0)
+        if ((SMask & LaneMask).none())
           continue;
         LiveQueryResult LRQ = S.Query(Def);
         ValueIn = LRQ.valueIn();
         break;
       }
     }
     if (ValueIn == nullptr)
       break;

 }
 
 JoinVals::ConflictResolution
 JoinVals::analyzeValue(unsigned ValNo, JoinVals &Other) {
   Val &V = Vals[ValNo];
   assert(!V.isAnalyzed() && "Value has already been analyzed!");
   VNInfo *VNI = LR.getValNumInfo(ValNo);
   if (VNI->isUnused()) {
-    V.WriteLanes = ~0u;
+    V.WriteLanes = LaneBitmask::getAll();
     return CR_Keep;
   }
 
   // Get the instruction defining this value, compute the lanes written.
   const MachineInstr *DefMI = nullptr;
   if (VNI->isPHIDef()) {
     // Conservatively assume that all lanes in a PHI are valid.
-    LaneBitmask Lanes = SubRangeJoin ? 1 : TRI->getSubRegIndexLaneMask(SubIdx);
+    LaneBitmask Lanes = SubRangeJoin ? LaneBitmask(1)
+                                     : TRI->getSubRegIndexLaneMask(SubIdx);
     V.ValidLanes = V.WriteLanes = Lanes;
   } else {
     DefMI = Indexes->getInstructionFromIndex(VNI->def);
     assert(DefMI != nullptr);
     if (SubRangeJoin) {
       // We don't care about the lanes when joining subregister ranges.
-      V.WriteLanes = V.ValidLanes = 1;
+      V.WriteLanes = V.ValidLanes = LaneBitmask(1);
       if (DefMI->isImplicitDef()) {
-        V.ValidLanes = 0;
+        V.ValidLanes = LaneBitmask::getNone();
         V.ErasableImplicitDef = true;
       }
     } else {
       bool Redef = false;
       V.ValidLanes = V.WriteLanes = computeWriteLanes(DefMI, Redef);
 
       // If this is a read-modify-write instruction, there may be more valid
       // lanes than the ones written by this instruction.
if (VNInfo *OtherVNI = OtherLRQ.valueDefined()) {
     // Keep this value, check for conflicts when analyzing OtherVNI.
     if (!OtherV.isAnalyzed())
       return CR_Keep;
     // Both sides have been analyzed now.
     // Allow overlapping PHI values. Any real interference would show up in a
     // predecessor, the PHI itself can't introduce any conflicts.
     if (VNI->isPHIDef())
       return CR_Merge;
-    if (V.ValidLanes & OtherV.ValidLanes)
+    if (!(V.ValidLanes & OtherV.ValidLanes).none())
       // Overlapping lanes can't be resolved.
       return CR_Impossible;
     else
       return CR_Merge;
   }
 
   // No simultaneous def. Is Other live at the def?
   V.OtherVNI = OtherLRQ.valueIn();
JoinVals::analyzeValue(unsigned ValNo, JoinVals &Other) {
   if (VNI->isPHIDef())
     return CR_Replace;
 
   // Check for simple erasable conflicts.
   if (DefMI->isImplicitDef()) {
     // We need the def for the subregister if there is nothing else live at the
     // subrange at this point.
     if (TrackSubRegLiveness
-        && (V.WriteLanes & (OtherV.ValidLanes | OtherV.WriteLanes)) == 0)
+        && (V.WriteLanes & (OtherV.ValidLanes | OtherV.WriteLanes)).none())
       return CR_Replace;
     return CR_Erase;
   }
 
   // Include the non-conflict where DefMI is a coalescable copy that kills
   // OtherVNI. We still want the copy erased and value numbers merged.
   if (CP.isCoalescable(DefMI)) {
     // Some of the lanes copied from OtherVNI may be undef, making them undef
JoinVals::analyzeValue(unsigned ValNo, JoinVals &Other) {
   //   1 %dst:ssub0 = FOO                <-- OtherVNI
   //   2 %src = BAR                      <-- VNI
   //   3 %dst:ssub1 = COPY %src<kill>    <-- Eliminate this copy.
   //   4 BAZ %dst<kill>
   //   5 QUUX %src<kill>
   //
   // Here OtherVNI will map to itself in [1;2), but to VNI in [2;5). CR_Replace
   // handles this complex value mapping.
-  if ((V.WriteLanes & OtherV.ValidLanes) == 0)
+  if ((V.WriteLanes & OtherV.ValidLanes).none())
     return CR_Replace;
 
   // If the other live range is killed by DefMI and the live ranges are still
   // overlapping, it must be because we're looking at an early clobber def:
   //
   //   %dst<def,early-clobber> = ASM %src<kill>
   //
   // In this case, it is illegal to merge the two live ranges since the early
   // clobber def would clobber %src before it was read.
   if (OtherLRQ.isKill()) {
     // This case where the def doesn't overlap the kill is handled above.
     assert(VNI->def.isEarlyClobber() &&
            "Only early clobber defs can overlap a kill");
     return CR_Impossible;
   }
 
   // VNI is clobbering live lanes in OtherVNI, but there is still the
   // possibility that no instructions actually read the clobbered lanes.
   // If we're clobbering all the lanes in OtherVNI, at least one must be read.
   // Otherwise Other.RI wouldn't be live here.
-  if ((TRI->getSubRegIndexLaneMask(Other.SubIdx) & ~V.WriteLanes) == 0)
+  if ((TRI->getSubRegIndexLaneMask(Other.SubIdx) & ~V.WriteLanes).none())
     return CR_Impossible;
 
   // We need to verify that no instructions are reading the clobbered lanes. To
   // save compile time, we'll only check that locally. Don't allow the tainted
   // value to escape the basic block.
   MachineBasicBlock *MBB = Indexes->getMBBFromIndex(VNI->def);
   if (OtherLRQ.endPoint() >= Indexes->getMBBEndIdx(MBB))
     return CR_Impossible;
case CR_Merge:
     break;
   case CR_Replace:
   case CR_Unresolved: {
     // The other value is going to be pruned if this join is successful.
     assert(V.OtherVNI && "OtherVNI not assigned, can't prune");
     Val &OtherV = Other.Vals[V.OtherVNI->id];
     // We cannot erase an IMPLICIT_DEF if we don't have valid values for all
     // its lanes.
-    if ((OtherV.WriteLanes & ~V.ValidLanes) != 0 && TrackSubRegLiveness)
+    if (!(OtherV.WriteLanes & ~V.ValidLanes).none() && TrackSubRegLiveness)
       OtherV.ErasableImplicitDef = false;
     OtherV.Pruned = true;
     LLVM_FALLTHROUGH;
   }
   default:
     // This value number needs to go in the final joined live range.
     Assignments[ValNo] = NewVNInfo.size();
     NewVNInfo.push_back(LR.getValNumInfo(ValNo));
do {
     if (++OtherI == Other.LR.end() || OtherI->start >= MBBEnd)
       break;
 
     // Lanes written by the new def are no longer tainted.
     const Val &OV = Other.Vals[OtherI->valno->id];
     TaintedLanes &= ~OV.WriteLanes;
     if (!OV.RedefVNI)
       break;
-  } while (TaintedLanes);
+  } while (!TaintedLanes.none());
   return true;
 }
 
 bool JoinVals::usesLanes(const MachineInstr &MI, unsigned Reg, unsigned SubIdx,
                          LaneBitmask Lanes) const {
   if (MI.isDebugValue())
     return false;
   for (const MachineOperand &MO : MI.operands()) {
     if (!MO.isReg() || MO.isDef() || MO.getReg() != Reg)
       continue;
     if (!MO.readsReg())
       continue;
-    if (Lanes & TRI->getSubRegIndexLaneMask(
-                  TRI->composeSubRegIndices(SubIdx, MO.getSubReg())))
+    unsigned S = TRI->composeSubRegIndices(SubIdx, MO.getSubReg());
+    if (!(Lanes & TRI->getSubRegIndexLaneMask(S)).none())
       return true;
   }
   return false;
 }
 
 bool JoinVals::resolveConflicts(JoinVals &Other) {
   for (unsigned i = 0, e = LR.getNumValNums(); i != e; ++i) {
     Val &V = Vals[i];
void RegisterCoalescer::mergeSubRangeInto(LiveInterval &LI,
                                           LaneBitmask LaneMask,
                                           CoalescerPair &CP) {
   BumpPtrAllocator &Allocator = LIS->getVNInfoAllocator();
   for (LiveInterval::SubRange &R : LI.subranges()) {
     LaneBitmask RMask = R.LaneMask;
     // LaneMask of subregisters common to subrange R and ToMerge.
     LaneBitmask Common = RMask & LaneMask;
     // There is nothing to do without common subregs.
-    if (Common == 0)
+    if (Common.none())
       continue;
 
     DEBUG(dbgs() << "\t\tCopy+Merge " << PrintLaneMask(RMask) << " into "
                  << PrintLaneMask(Common) << '\n');
     // LaneMask of subregisters contained in the R range but not in ToMerge,
     // they have to split into their own subrange.
     LaneBitmask LRest = RMask & ~LaneMask;
     LiveInterval::SubRange *CommonRange;
-    if (LRest != 0) {
+    if (!LRest.none()) {
       R.LaneMask = LRest;
       DEBUG(dbgs() << "\t\tReduce Lane to " << PrintLaneMask(LRest) << '\n');
       // Duplicate SubRange for newly merged common stuff.
       CommonRange = LI.createSubRangeFrom(Allocator, Common, R);
     } else {
       // Reuse the existing range.
       R.LaneMask = Common;
       CommonRange = &R;
     }
     LiveRange RangeCopy(ToMerge, Allocator);
     joinSubRegRanges(*CommonRange, RangeCopy, Common, CP);
     LaneMask &= ~RMask;
   }
 
-  if (LaneMask != 0) {
+  if (!LaneMask.none()) {
     DEBUG(dbgs() << "\t\tNew Lane " << PrintLaneMask(LaneMask) << '\n');
     LI.createSubRangeFrom(Allocator, LaneMask, ToMerge);
   }
 }
 
 bool RegisterCoalescer::joinVirtRegs(CoalescerPair &CP) {
   SmallVector<VNInfo*, 16> NewVNInfo;
   LiveInterval &RHS = LIS->getInterval(CP.getSrcReg());
   LiveInterval &LHS = LIS->getInterval(CP.getDstReg());
   bool TrackSubRegLiveness = MRI->shouldTrackSubRegLiveness(*CP.getNewRC());
-  JoinVals RHSVals(RHS, CP.getSrcReg(), CP.getSrcIdx(), 0, NewVNInfo, CP, LIS,
-                   TRI, false, TrackSubRegLiveness);
-  JoinVals LHSVals(LHS, CP.getDstReg(), CP.getDstIdx(), 0, NewVNInfo, CP, LIS,
-                   TRI, false, TrackSubRegLiveness);
+  JoinVals RHSVals(RHS, CP.getSrcReg(), CP.getSrcIdx(), LaneBitmask::getNone(),
+                   NewVNInfo, CP, LIS, TRI, false, TrackSubRegLiveness);
+  JoinVals LHSVals(LHS, CP.getDstReg(), CP.getDstIdx(), LaneBitmask::getNone(),
+                   NewVNInfo, CP, LIS, TRI, false, TrackSubRegLiveness);
 
   DEBUG(dbgs() << "\t\tRHS = " << RHS
                << "\n\t\tLHS = " << LHS
                << '\n');
 
   // First compute NewVNInfo and the simple value mappings.
   // Detect impossible conflicts early.
   if (!LHSVals.mapValues(RHSVals) || !RHSVals.mapValues(LHSVals))
if (RHS.hasSubRanges() || LHS.hasSubRanges()) {
 
     // Transform lanemasks from the LHS to masks in the coalesced register and
     // create initial subranges if necessary.
     unsigned DstIdx = CP.getDstIdx();
     if (!LHS.hasSubRanges()) {
       LaneBitmask Mask = DstIdx == 0 ? CP.getNewRC()->getLaneMask()
                                      : TRI->getSubRegIndexLaneMask(DstIdx);
       // LHS must support subregs or we wouldn't be in this codepath.
-      assert(Mask != 0);
+      assert(!Mask.none());
       LHS.createSubRangeFrom(Allocator, Mask, LHS);
     } else if (DstIdx != 0) {
       // Transform LHS lanemasks to new register class if necessary.
       for (LiveInterval::SubRange &R : LHS.subranges()) {
         LaneBitmask Mask = TRI->composeSubRegIndexLaneMask(DstIdx, R.LaneMask);
         R.LaneMask = Mask;
       }
     }
if (MRI->recomputeRegClass(Reg)) {
         if (!MRI->shouldTrackSubRegLiveness(Reg)) {
           LI.clearSubRanges();
         } else {
 #ifndef NDEBUG
           LaneBitmask MaxMask = MRI->getMaxLaneMaskForVReg(Reg);
           // If subranges are still supported, then the same subregs
           // should still be supported.
           for (LiveInterval::SubRange &S : LI.subranges()) {
-            assert((S.LaneMask & ~MaxMask) == 0);
+            assert((S.LaneMask & ~MaxMask).none());
           }
 #endif
         }
       }
     }
   }
 
   DEBUG(dump());
   if (VerifyCoalescing)
     MF->verify(this, "After register coalescing");
   return true;
 }
 
 void RegisterCoalescer::print(raw_ostream &O, const Module* m) const {
    LIS->print(O, m);
 }

llvm/trunk/lib/CodeGen/RegisterPressure.cpp

 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 
 /// Increase pressure for each pressure set provided by TargetRegisterInfo.
 static void increaseSetPressure(std::vector<unsigned> &CurrSetPressure,
                                 const MachineRegisterInfo &MRI, unsigned Reg,
                                 LaneBitmask PrevMask, LaneBitmask NewMask) {
-  assert((PrevMask & ~NewMask) == 0 && "Must not remove bits");
-  if (PrevMask != 0 || NewMask == 0)
+  assert((PrevMask & ~NewMask).none() && "Must not remove bits");
+  if (!PrevMask.none() || NewMask.none())
     return;
 
   PSetIterator PSetI = MRI.getPressureSets(Reg);
   unsigned Weight = PSetI.getWeight();
   for (; PSetI.isValid(); ++PSetI)
     CurrSetPressure[*PSetI] += Weight;
 }
 
 /// Decrease pressure for each pressure set provided by TargetRegisterInfo.
 static void decreaseSetPressure(std::vector<unsigned> &CurrSetPressure,
                                 const MachineRegisterInfo &MRI, unsigned Reg,
                                 LaneBitmask PrevMask, LaneBitmask NewMask) {
-  assert((NewMask & !PrevMask) == 0 && "Must not add bits");
-  if (NewMask != 0 || PrevMask == 0)
+  //assert((NewMask & !PrevMask) == 0 && "Must not add bits");
+  if (!NewMask.none() || PrevMask.none())
     return;
 
   PSetIterator PSetI = MRI.getPressureSets(Reg);
   unsigned Weight = PSetI.getWeight();
   for (; PSetI.isValid(); ++PSetI) {
     assert(CurrSetPressure[*PSetI] >= Weight && "register pressure underflow");
     CurrSetPressure[*PSetI] -= Weight;
   }

 
 LLVM_DUMP_METHOD
 void RegisterPressure::dump(const TargetRegisterInfo *TRI) const {
   dbgs() << "Max Pressure: ";
   dumpRegSetPressure(MaxSetPressure, TRI);
   dbgs() << "Live In: ";
   for (const RegisterMaskPair &P : LiveInRegs) {
     dbgs() << PrintVRegOrUnit(P.RegUnit, TRI);
-    if (P.LaneMask != ~0u)
+    if (!P.LaneMask.all())
       dbgs() << ':' << PrintLaneMask(P.LaneMask);
     dbgs() << ' ';
   }
   dbgs() << '\n';
   dbgs() << "Live Out: ";
   for (const RegisterMaskPair &P : LiveOutRegs) {
     dbgs() << PrintVRegOrUnit(P.RegUnit, TRI);
-    if (P.LaneMask != ~0u)
+    if (!P.LaneMask.all())
       dbgs() << ':' << PrintLaneMask(P.LaneMask);
     dbgs() << ' ';
   }
   dbgs() << '\n';
 }
 
 LLVM_DUMP_METHOD
 void RegPressureTracker::dump() const {
for (const PressureChange &Change : *this) {
     sep = "    ";
   }
   dbgs() << '\n';
 }
 
 void RegPressureTracker::increaseRegPressure(unsigned RegUnit,
                                              LaneBitmask PreviousMask,
                                              LaneBitmask NewMask) {
-  if (PreviousMask != 0 || NewMask == 0)
+  if (!PreviousMask.none() || NewMask.none())
     return;
 
   PSetIterator PSetI = MRI->getPressureSets(RegUnit);
   unsigned Weight = PSetI.getWeight();
   for (; PSetI.isValid(); ++PSetI) {
     CurrSetPressure[*PSetI] += Weight;
     P.MaxSetPressure[*PSetI] =
         std::max(P.MaxSetPressure[*PSetI], CurrSetPressure[*PSetI]);

 /// can never drop below this pressure.
 void RegPressureTracker::initLiveThru(const RegPressureTracker &RPTracker) {
   LiveThruPressure.assign(TRI->getNumRegPressureSets(), 0);
   assert(isBottomClosed() && "need bottom-up tracking to intialize.");
   for (const RegisterMaskPair &Pair : P.LiveOutRegs) {
     unsigned RegUnit = Pair.RegUnit;
     if (TargetRegisterInfo::isVirtualRegister(RegUnit)
         && !RPTracker.hasUntiedDef(RegUnit))
-      increaseSetPressure(LiveThruPressure, *MRI, RegUnit, 0, Pair.LaneMask);
+      increaseSetPressure(LiveThruPressure, *MRI, RegUnit,
+                          LaneBitmask::getNone(), Pair.LaneMask);
   }
 }
 
 static LaneBitmask getRegLanes(ArrayRef<RegisterMaskPair> RegUnits,
                                unsigned RegUnit) {
   auto I = find_if(RegUnits, [RegUnit](const RegisterMaskPair Other) {
     return Other.RegUnit == RegUnit;
   });
   if (I == RegUnits.end())
-    return 0;
+    return LaneBitmask::getNone();
   return I->LaneMask;
 }
 
 static void addRegLanes(SmallVectorImpl<RegisterMaskPair> &RegUnits,
                         RegisterMaskPair Pair) {
   unsigned RegUnit = Pair.RegUnit;
-  assert(Pair.LaneMask != 0);
+  assert(!Pair.LaneMask.none());
   auto I = find_if(RegUnits, [RegUnit](const RegisterMaskPair Other) {
     return Other.RegUnit == RegUnit;
   });
   if (I == RegUnits.end()) {
     RegUnits.push_back(Pair);
   } else {
     I->LaneMask |= Pair.LaneMask;
   }
 }
 
 static void setRegZero(SmallVectorImpl<RegisterMaskPair> &RegUnits,
                        unsigned RegUnit) {
   auto I = find_if(RegUnits, [RegUnit](const RegisterMaskPair Other) {
     return Other.RegUnit == RegUnit;
   });
   if (I == RegUnits.end()) {
-    RegUnits.push_back(RegisterMaskPair(RegUnit, 0));
+    RegUnits.push_back(RegisterMaskPair(RegUnit, LaneBitmask::getNone()));
   } else {
-    I->LaneMask = 0;
+    I->LaneMask = LaneBitmask::getNone();
   }
 }
 
 static void removeRegLanes(SmallVectorImpl<RegisterMaskPair> &RegUnits,
                            RegisterMaskPair Pair) {
   unsigned RegUnit = Pair.RegUnit;
-  assert(Pair.LaneMask != 0);
+  assert(!Pair.LaneMask.none());
   auto I = find_if(RegUnits, [RegUnit](const RegisterMaskPair Other) {
     return Other.RegUnit == RegUnit;
   });
   if (I != RegUnits.end()) {
     I->LaneMask &= ~Pair.LaneMask;
-    if (I->LaneMask == 0)
+    if (I->LaneMask.none())
       RegUnits.erase(I);
   }
 }
 
 static LaneBitmask getLanesWithProperty(const LiveIntervals &LIS,
     const MachineRegisterInfo &MRI, bool TrackLaneMasks, unsigned RegUnit,
     SlotIndex Pos, LaneBitmask SafeDefault,
     bool(*Property)(const LiveRange &LR, SlotIndex Pos)) {
   if (TargetRegisterInfo::isVirtualRegister(RegUnit)) {
     const LiveInterval &LI = LIS.getInterval(RegUnit);
-    LaneBitmask Result = 0;
+    LaneBitmask Result;
     if (TrackLaneMasks && LI.hasSubRanges()) {
         for (const LiveInterval::SubRange &SR : LI.subranges()) {
           if (Property(SR, Pos))
             Result |= SR.LaneMask;
         }
     } else if (Property(LI, Pos)) {
-      Result = TrackLaneMasks ? MRI.getMaxLaneMaskForVReg(RegUnit) : ~0u;
+      Result = TrackLaneMasks ? MRI.getMaxLaneMaskForVReg(RegUnit)
+                              : LaneBitmask::getAll();
     }
 
     return Result;
   } else {
     const LiveRange *LR = LIS.getCachedRegUnit(RegUnit);
     // Be prepared for missing liveranges: We usually do not compute liveranges
     // for physical registers on targets with many registers (GPUs).
     if (LR == nullptr)
       return SafeDefault;
-    return Property(*LR, Pos) ? ~0u : 0;
+    return Property(*LR, Pos) ? LaneBitmask::getAll() : LaneBitmask::getNone();
   }
 }
 
 static LaneBitmask getLiveLanesAt(const LiveIntervals &LIS,
                                   const MachineRegisterInfo &MRI,
                                   bool TrackLaneMasks, unsigned RegUnit,
                                   SlotIndex Pos) {
-  return getLanesWithProperty(LIS, MRI, TrackLaneMasks, RegUnit, Pos, ~0u,
+  return getLanesWithProperty(LIS, MRI, TrackLaneMasks, RegUnit, Pos,
+                              LaneBitmask::getAll(),
                               [](const LiveRange &LR, SlotIndex Pos) {
                                 return LR.liveAt(Pos);
                               });
 }
 
 
 namespace {
 
if (MO.isUse()) {
       } else
         pushReg(Reg, RegOpers.Defs);
     }
   }
 
   void pushReg(unsigned Reg,
                SmallVectorImpl<RegisterMaskPair> &RegUnits) const {
     if (TargetRegisterInfo::isVirtualRegister(Reg)) {
-      addRegLanes(RegUnits, RegisterMaskPair(Reg, ~0u));
+      addRegLanes(RegUnits, RegisterMaskPair(Reg, LaneBitmask::getAll()));
     } else if (MRI.isAllocatable(Reg)) {
       for (MCRegUnitIterator Units(Reg, &TRI); Units.isValid(); ++Units)
-        addRegLanes(RegUnits, RegisterMaskPair(*Units, ~0u));
+        addRegLanes(RegUnits, RegisterMaskPair(*Units, LaneBitmask::getAll()));
     }
   }
 
   void collectOperandLanes(const MachineOperand &MO) const {
     if (!MO.isReg() || !MO.getReg())
       return;
     unsigned Reg = MO.getReg();
     unsigned SubRegIdx = MO.getSubReg();
void pushRegLanes(unsigned Reg, unsigned SubRegIdx,
                     SmallVectorImpl<RegisterMaskPair> &RegUnits) const {
     if (TargetRegisterInfo::isVirtualRegister(Reg)) {
       LaneBitmask LaneMask = SubRegIdx != 0
                              ? TRI.getSubRegIndexLaneMask(SubRegIdx)
                              : MRI.getMaxLaneMaskForVReg(Reg);
       addRegLanes(RegUnits, RegisterMaskPair(Reg, LaneMask));
     } else if (MRI.isAllocatable(Reg)) {
       for (MCRegUnitIterator Units(Reg, &TRI); Units.isValid(); ++Units)
-        addRegLanes(RegUnits, RegisterMaskPair(*Units, ~0u));
+        addRegLanes(RegUnits, RegisterMaskPair(*Units, LaneBitmask::getAll()));
     }
   }
 
   friend class llvm::RegisterOperands;
 };
 
 } // namespace
 
void RegisterOperands::adjustLaneLiveness(const LiveIntervals &LIS,
                                           MachineInstr *AddFlagsMI) {
   for (auto I = Defs.begin(); I != Defs.end(); ) {
     LaneBitmask LiveAfter = getLiveLanesAt(LIS, MRI, true, I->RegUnit,
                                            Pos.getDeadSlot());
     // If the the def is all that is live after the instruction, then in case
     // of a subregister def we need a read-undef flag.
     unsigned RegUnit = I->RegUnit;
     if (TargetRegisterInfo::isVirtualRegister(RegUnit) &&
-        AddFlagsMI != nullptr && (LiveAfter & ~I->LaneMask) == 0)
+        AddFlagsMI != nullptr && (LiveAfter & ~I->LaneMask).none())
       AddFlagsMI->setRegisterDefReadUndef(RegUnit);
 
     LaneBitmask ActualDef = I->LaneMask & LiveAfter;
-    if (ActualDef == 0) {
+    if (ActualDef.none()) {
       I = Defs.erase(I);
     } else {
       I->LaneMask = ActualDef;
       ++I;
     }
   }
   for (auto I = Uses.begin(); I != Uses.end(); ) {
     LaneBitmask LiveBefore = getLiveLanesAt(LIS, MRI, true, I->RegUnit,
                                             Pos.getBaseIndex());
     LaneBitmask LaneMask = I->LaneMask & LiveBefore;
-    if (LaneMask == 0) {
+    if (LaneMask.none()) {
       I = Uses.erase(I);
     } else {
       I->LaneMask = LaneMask;
       ++I;
     }
   }
   if (AddFlagsMI != nullptr) {
     for (const RegisterMaskPair &P : DeadDefs) {
       unsigned RegUnit = P.RegUnit;
       if (!TargetRegisterInfo::isVirtualRegister(RegUnit))
         continue;
       LaneBitmask LiveAfter = getLiveLanesAt(LIS, MRI, true, RegUnit,
                                              Pos.getDeadSlot());
-      if (LiveAfter == 0)
+      if (LiveAfter.none())
         AddFlagsMI->setRegisterDefReadUndef(RegUnit);
     }
   }
 }
 
 /// Initialize an array of N PressureDiffs.
 void PressureDiffs::init(unsigned N) {
   Size = N;
for (const RegisterMaskPair &P : Regs) {
     LaneBitmask PrevMask = LiveRegs.insert(P);
     LaneBitmask NewMask = PrevMask | P.LaneMask;
     increaseRegPressure(P.RegUnit, PrevMask, NewMask);
   }
 }
 
 void RegPressureTracker::discoverLiveInOrOut(RegisterMaskPair Pair,
     SmallVectorImpl<RegisterMaskPair> &LiveInOrOut) {
-  assert(Pair.LaneMask != 0);
+  assert(!Pair.LaneMask.none());
 
   unsigned RegUnit = Pair.RegUnit;
   auto I = find_if(LiveInOrOut, [RegUnit](const RegisterMaskPair &Other) {
     return Other.RegUnit == RegUnit;
   });
   LaneBitmask PrevMask;
   LaneBitmask NewMask;
   if (I == LiveInOrOut.end()) {
-    PrevMask = 0;
+    PrevMask = LaneBitmask::getNone();
     NewMask = Pair.LaneMask;
     LiveInOrOut.push_back(Pair);
   } else {
     PrevMask = I->LaneMask;
     NewMask = PrevMask | Pair.LaneMask;
     I->LaneMask = NewMask;
   }
   increaseSetPressure(P.MaxSetPressure, *MRI, RegUnit, PrevMask, NewMask);
void RegPressureTracker::recede(const RegisterOperands &RegOpers,
   // TODO: consider earlyclobbers?
   for (const RegisterMaskPair &Def : RegOpers.Defs) {
     unsigned Reg = Def.RegUnit;
 
     LaneBitmask PreviousMask = LiveRegs.erase(Def);
     LaneBitmask NewMask = PreviousMask & ~Def.LaneMask;
 
     LaneBitmask LiveOut = Def.LaneMask & ~PreviousMask;
-    if (LiveOut != 0) {
+    if (!LiveOut.none()) {
       discoverLiveOut(RegisterMaskPair(Reg, LiveOut));
       // Retroactively model effects on pressure of the live out lanes.
-      increaseSetPressure(CurrSetPressure, *MRI, Reg, 0, LiveOut);
+      increaseSetPressure(CurrSetPressure, *MRI, Reg, LaneBitmask::getNone(),
+                          LiveOut);
       PreviousMask = LiveOut;
     }
 
-    if (NewMask == 0) {
+    if (NewMask.none()) {
       // Add a 0 entry to LiveUses as a marker that the complete vreg has become
       // dead.
       if (TrackLaneMasks && LiveUses != nullptr)
         setRegZero(*LiveUses, Reg);
     }
 
     decreaseRegPressure(Reg, PreviousMask, NewMask);
   }
 
   SlotIndex SlotIdx;
   if (RequireIntervals)
     SlotIdx = LIS->getInstructionIndex(*CurrPos).getRegSlot();
 
   // Generate liveness for uses.
   for (const RegisterMaskPair &Use : RegOpers.Uses) {
     unsigned Reg = Use.RegUnit;
-    assert(Use.LaneMask != 0);
+    assert(!Use.LaneMask.none());
     LaneBitmask PreviousMask = LiveRegs.insert(Use);
     LaneBitmask NewMask = PreviousMask | Use.LaneMask;
     if (NewMask == PreviousMask)
       continue;
 
     // Did the register just become live?
-    if (PreviousMask == 0) {
+    if (PreviousMask.none()) {
       if (LiveUses != nullptr) {
         if (!TrackLaneMasks) {
           addRegLanes(*LiveUses, RegisterMaskPair(Reg, NewMask));
         } else {
           auto I = find_if(*LiveUses, [Reg](const RegisterMaskPair Other) {
             return Other.RegUnit == Reg;
           });
           bool IsRedef = I != LiveUses->end();
           if (IsRedef) {
             // ignore re-defs here...
-            assert(I->LaneMask == 0);
+            assert(I->LaneMask.none());
             removeRegLanes(*LiveUses, RegisterMaskPair(Reg, NewMask));
           } else {
             addRegLanes(*LiveUses, RegisterMaskPair(Reg, NewMask));
           }
         }
       }
 
       // Discover live outs if this may be the first occurance of this register.
       if (RequireIntervals) {
         LaneBitmask LiveOut = getLiveThroughAt(Reg, SlotIdx);
-        if (LiveOut != 0)
+        if (!LiveOut.none())
           discoverLiveOut(RegisterMaskPair(Reg, LiveOut));
       }
     }
 
     increaseRegPressure(Reg, PreviousMask, NewMask);
   }
   if (TrackUntiedDefs) {
     for (const RegisterMaskPair &Def : RegOpers.Defs) {
       unsigned RegUnit = Def.RegUnit;
       if (TargetRegisterInfo::isVirtualRegister(RegUnit) &&
-          (LiveRegs.contains(RegUnit) & Def.LaneMask) == 0)
+          (LiveRegs.contains(RegUnit) & Def.LaneMask).none())
         UntiedDefs.insert(RegUnit);
     }
   }
 }
 
 void RegPressureTracker::recedeSkipDebugValues() {
   assert(CurrPos != MBB->begin());
   if (!isBottomClosed())
if (isBottomClosed()) {
     else
       static_cast<RegionPressure&>(P).openBottom(CurrPos);
   }
 
   for (const RegisterMaskPair &Use : RegOpers.Uses) {
     unsigned Reg = Use.RegUnit;
     LaneBitmask LiveMask = LiveRegs.contains(Reg);
     LaneBitmask LiveIn = Use.LaneMask & ~LiveMask;
-    if (LiveIn != 0) {
+    if (!LiveIn.none()) {
       discoverLiveIn(RegisterMaskPair(Reg, LiveIn));
       increaseRegPressure(Reg, LiveMask, LiveMask | LiveIn);
       LiveRegs.insert(RegisterMaskPair(Reg, LiveIn));
     }
     // Kill liveness at last uses.
     if (RequireIntervals) {
       LaneBitmask LastUseMask = getLastUsedLanes(Reg, SlotIdx);
-      if (LastUseMask != 0) {
+      if (!LastUseMask.none()) {
         LiveRegs.erase(RegisterMaskPair(Reg, LastUseMask));
         decreaseRegPressure(Reg, LiveMask, LiveMask & ~LastUseMask);
       }
     }
   }
 
   // Generate liveness for defs.
   for (const RegisterMaskPair &Def : RegOpers.Defs) {
for (const MachineOperand &MO : MRI.use_nodbg_operands(Reg)) {
     if (MO.isUndef())
       continue;
     const MachineInstr *MI = MO.getParent();
     SlotIndex InstSlot = LIS->getInstructionIndex(*MI).getRegSlot();
     if (InstSlot >= PriorUseIdx && InstSlot < NextUseIdx) {
       unsigned SubRegIdx = MO.getSubReg();
       LaneBitmask UseMask = TRI.getSubRegIndexLaneMask(SubRegIdx);
       LastUseMask &= ~UseMask;
-      if (LastUseMask == 0)
-        return 0;
+      if (LastUseMask.none())
+        return LaneBitmask::getNone();
     }
   }
   return LastUseMask;
 }
 
 LaneBitmask RegPressureTracker::getLiveLanesAt(unsigned RegUnit,
                                                SlotIndex Pos) const {
   assert(RequireIntervals);
-  return getLanesWithProperty(*LIS, *MRI, TrackLaneMasks, RegUnit, Pos, ~0u,
+  return getLanesWithProperty(*LIS, *MRI, TrackLaneMasks, RegUnit, Pos,
+                              LaneBitmask::getAll(),
       [](const LiveRange &LR, SlotIndex Pos) {
         return LR.liveAt(Pos);
       });
 }
 
 LaneBitmask RegPressureTracker::getLastUsedLanes(unsigned RegUnit,
                                                  SlotIndex Pos) const {
   assert(RequireIntervals);
   return getLanesWithProperty(*LIS, *MRI, TrackLaneMasks, RegUnit,
-                              Pos.getBaseIndex(), 0,
+                              Pos.getBaseIndex(), LaneBitmask::getNone(),
       [](const LiveRange &LR, SlotIndex Pos) {
         const LiveRange::Segment *S = LR.getSegmentContaining(Pos);
         return S != nullptr && S->end == Pos.getRegSlot();
       });
 }
 
 LaneBitmask RegPressureTracker::getLiveThroughAt(unsigned RegUnit,
                                                  SlotIndex Pos) const {
   assert(RequireIntervals);
-  return getLanesWithProperty(*LIS, *MRI, TrackLaneMasks, RegUnit, Pos, 0u,
+  return getLanesWithProperty(*LIS, *MRI, TrackLaneMasks, RegUnit, Pos,
+                              LaneBitmask::getNone(),
       [](const LiveRange &LR, SlotIndex Pos) {
         const LiveRange::Segment *S = LR.getSegmentContaining(Pos);
         return S != nullptr && S->start < Pos.getRegSlot(true) &&
                S->end != Pos.getDeadSlot();
       });
 }
 
 /// Record the downward impact of a single instruction on current register
void RegPressureTracker::bumpDownwardPressure(const MachineInstr *MI) {
   RegOpers.collect(*MI, *TRI, *MRI, TrackLaneMasks, false);
   if (TrackLaneMasks)
     RegOpers.adjustLaneLiveness(*LIS, *MRI, SlotIdx);
 
   if (RequireIntervals) {
     for (const RegisterMaskPair &Use : RegOpers.Uses) {
       unsigned Reg = Use.RegUnit;
       LaneBitmask LastUseMask = getLastUsedLanes(Reg, SlotIdx);
-      if (LastUseMask == 0)
+      if (LastUseMask.none())
         continue;
       // The LastUseMask is queried from the liveness information of instruction
       // which may be further down the schedule. Some lanes may actually not be
       // last uses for the current position.
       // FIXME: allow the caller to pass in the list of vreg uses that remain
       // to be bottom-scheduled to avoid searching uses at each query.
       SlotIndex CurrIdx = getCurrSlot();
       LastUseMask
         = findUseBetween(Reg, LastUseMask, CurrIdx, SlotIdx, *MRI, LIS);
-      if (LastUseMask == 0)
+      if (LastUseMask.none())
         continue;
 
       LaneBitmask LiveMask = LiveRegs.contains(Reg);
       LaneBitmask NewMask = LiveMask & ~LastUseMask;
       decreaseRegPressure(Reg, LiveMask, NewMask);
     }
   }
 

llvm/trunk/lib/CodeGen/RegisterScavenging.cpp

 #include "llvm/Target/TargetSubtargetInfo.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "reg-scavenging"
 
 void RegScavenger::setRegUsed(unsigned Reg, LaneBitmask LaneMask) {
   for (MCRegUnitMaskIterator RUI(Reg, TRI); RUI.isValid(); ++RUI) {
     LaneBitmask UnitMask = (*RUI).second;
-    if (UnitMask == 0 || (LaneMask & UnitMask) != 0)
+    if (UnitMask.none() || !(LaneMask & UnitMask).none())
       RegUnitsAvailable.reset((*RUI).first);
   }
 }
 
 void RegScavenger::init(MachineBasicBlock &MBB) {
   MachineFunction &MF = *MBB.getParent();
   TII = MF.getSubtarget().getInstrInfo();
   TRI = MF.getSubtarget().getRegisterInfo();

llvm/trunk/lib/CodeGen/RenameIndependentSubregs.cpp

bool RenameIndependentSubregs::findComponents(IntEqClasses &Classes,
   for (const MachineOperand &MO : MRI->reg_nodbg_operands(Reg)) {
     if (!MO.isDef() && !MO.readsReg())
       continue;
     unsigned SubRegIdx = MO.getSubReg();
     LaneBitmask LaneMask = TRI.getSubRegIndexLaneMask(SubRegIdx);
     unsigned MergedID = ~0u;
     for (RenameIndependentSubregs::SubRangeInfo &SRInfo : SubRangeInfos) {
       const LiveInterval::SubRange &SR = *SRInfo.SR;
-      if ((SR.LaneMask & LaneMask) == 0)
+      if ((SR.LaneMask & LaneMask).none())
         continue;
       SlotIndex Pos = LIS->getInstructionIndex(*MO.getParent());
       Pos = MO.isDef() ? Pos.getRegSlot(MO.isEarlyClobber())
                        : Pos.getBaseIndex();
       const VNInfo *VNI = SR.getVNInfoAt(Pos);
       if (VNI == nullptr)
         continue;
 
for (MachineRegisterInfo::reg_nodbg_iterator I = MRI->reg_nodbg_begin(Reg),
     Pos = MO.isDef() ? Pos.getRegSlot(MO.isEarlyClobber())
                      : Pos.getBaseIndex();
     unsigned SubRegIdx = MO.getSubReg();
     LaneBitmask LaneMask = TRI.getSubRegIndexLaneMask(SubRegIdx);
 
     unsigned ID = ~0u;
     for (const SubRangeInfo &SRInfo : SubRangeInfos) {
       const LiveInterval::SubRange &SR = *SRInfo.SR;
-      if ((SR.LaneMask & LaneMask) == 0)
+      if ((SR.LaneMask & LaneMask).none())
         continue;
       const VNInfo *VNI = SR.getVNInfoAt(Pos);
       if (VNI == nullptr)
         continue;
 
       // Map to local representant ID.
       unsigned LocalID = SRInfo.ConEQ.getEqClass(VNI);
       // Global ID

llvm/trunk/lib/CodeGen/ScheduleDAGInstrs.cpp

 }
 
 LaneBitmask ScheduleDAGInstrs::getLaneMaskForMO(const MachineOperand &MO) const
 {
   unsigned Reg = MO.getReg();
   // No point in tracking lanemasks if we don't have interesting subregisters.
   const TargetRegisterClass &RC = *MRI.getRegClass(Reg);
   if (!RC.HasDisjunctSubRegs)
-    return ~0u;
+    return LaneBitmask::getAll();
 
   unsigned SubReg = MO.getSubReg();
   if (SubReg == 0)
     return RC.getLaneMask();
   return TRI->getSubRegIndexLaneMask(SubReg);
 }
 
 /// addVRegDefDeps - Add register output and data dependencies from this SUnit
void ScheduleDAGInstrs::addVRegDefDeps(SUnit *SU, unsigned OperIdx) {
 
   LaneBitmask DefLaneMask;
   LaneBitmask KillLaneMask;
   if (TrackLaneMasks) {
     bool IsKill = MO.getSubReg() == 0 || MO.isUndef();
     DefLaneMask = getLaneMaskForMO(MO);
     // If we have a <read-undef> flag, none of the lane values comes from an
     // earlier instruction.
-    KillLaneMask = IsKill ? ~0u : DefLaneMask;
+    KillLaneMask = IsKill ? LaneBitmask::getAll() : DefLaneMask;
 
     // Clear undef flag, we'll re-add it later once we know which subregister
     // Def is first.
     MO.setIsUndef(false);
   } else {
-    DefLaneMask = ~0u;
-    KillLaneMask = ~0u;
+    DefLaneMask = LaneBitmask::getAll();
+    KillLaneMask = LaneBitmask::getAll();
   }
 
   if (MO.isDead()) {
     assert(CurrentVRegUses.find(Reg) == CurrentVRegUses.end() &&
            "Dead defs should have no uses");
   } else {
     // Add data dependence to all uses we found so far.
     const TargetSubtargetInfo &ST = MF.getSubtarget();
     for (VReg2SUnitOperIdxMultiMap::iterator I = CurrentVRegUses.find(Reg),
          E = CurrentVRegUses.end(); I != E; /*empty*/) {
       LaneBitmask LaneMask = I->LaneMask;
       // Ignore uses of other lanes.
-      if ((LaneMask & KillLaneMask) == 0) {
+      if ((LaneMask & KillLaneMask).none()) {
         ++I;
         continue;
       }
 
-      if ((LaneMask & DefLaneMask) != 0) {
+      if (!(LaneMask & DefLaneMask).none()) {
         SUnit *UseSU = I->SU;
         MachineInstr *Use = UseSU->getInstr();
         SDep Dep(SU, SDep::Data, Reg);
         Dep.setLatency(SchedModel.computeOperandLatency(MI, OperIdx, Use,
                                                         I->OperandIndex));
         ST.adjustSchedDependency(SU, UseSU, Dep);
         UseSU->addPred(Dep);
       }
 
       LaneMask &= ~KillLaneMask;
       // If we found a Def for all lanes of this use, remove it from the list.
-      if (LaneMask != 0) {
+      if (!LaneMask.none()) {
         I->LaneMask = LaneMask;
         ++I;
       } else
         I = CurrentVRegUses.erase(I);
     }
   }
 
   // Shortcut: Singly defined vregs do not have output/anti dependencies.
   if (MRI.hasOneDef(Reg))
     return;
 
   // Add output dependence to the next nearest defs of this vreg.
   //
   // Unless this definition is dead, the output dependence should be
   // transitively redundant with antidependencies from this definition's
   // uses. We're conservative for now until we have a way to guarantee the uses
   // are not eliminated sometime during scheduling. The output dependence edge
   // is also useful if output latency exceeds def-use latency.
   LaneBitmask LaneMask = DefLaneMask;
   for (VReg2SUnit &V2SU : make_range(CurrentVRegDefs.find(Reg),
                                      CurrentVRegDefs.end())) {
     // Ignore defs for other lanes.
-    if ((V2SU.LaneMask & LaneMask) == 0)
+    if ((V2SU.LaneMask & LaneMask).none())
       continue;
     // Add an output dependence.
     SUnit *DefSU = V2SU.SU;
     // Ignore additional defs of the same lanes in one instruction. This can
     // happen because lanemasks are shared for targets with too many
     // subregisters. We also use some representration tricks/hacks where we
     // add super-register defs/uses, to imply that although we only access parts
     // of the reg we care about the full one.
     if (DefSU == SU)
       continue;
     SDep Dep(SU, SDep::Output, Reg);
     Dep.setLatency(
       SchedModel.computeOutputLatency(MI, OperIdx, DefSU->getInstr()));
     DefSU->addPred(Dep);
 
     // Update current definition. This can get tricky if the def was about a
     // bigger lanemask before. We then have to shrink it and create a new
     // VReg2SUnit for the non-overlapping part.
     LaneBitmask OverlapMask = V2SU.LaneMask & LaneMask;
     LaneBitmask NonOverlapMask = V2SU.LaneMask & ~LaneMask;
     V2SU.SU = SU;
     V2SU.LaneMask = OverlapMask;
-    if (NonOverlapMask != 0)
+    if (!NonOverlapMask.none())
       CurrentVRegDefs.insert(VReg2SUnit(Reg, NonOverlapMask, DefSU));
   }
   // If there was no CurrentVRegDefs entry for some lanes yet, create one.
-  if (LaneMask != 0)
+  if (!LaneMask.none())
     CurrentVRegDefs.insert(VReg2SUnit(Reg, LaneMask, SU));
 }
 
 /// addVRegUseDeps - Add a register data dependency if the instruction that
 /// defines the virtual register used at OperIdx is mapped to an SUnit. Add a
 /// register antidependency from this SUnit to instructions that occur later in
 /// the same scheduling region if they write the virtual register.
 ///
 /// TODO: Handle ExitSU "uses" properly.
 void ScheduleDAGInstrs::addVRegUseDeps(SUnit *SU, unsigned OperIdx) {
   const MachineInstr *MI = SU->getInstr();
   const MachineOperand &MO = MI->getOperand(OperIdx);
   unsigned Reg = MO.getReg();
 
   // Remember the use. Data dependencies will be added when we find the def.
-  LaneBitmask LaneMask = TrackLaneMasks ? getLaneMaskForMO(MO) : ~0u;
+  LaneBitmask LaneMask = TrackLaneMasks ? getLaneMaskForMO(MO)
+                                        : LaneBitmask::getAll();
   CurrentVRegUses.insert(VReg2SUnitOperIdx(Reg, LaneMask, OperIdx, SU));
 
   // Add antidependences to the following defs of the vreg.
   for (VReg2SUnit &V2SU : make_range(CurrentVRegDefs.find(Reg),
                                      CurrentVRegDefs.end())) {
     // Ignore defs for unrelated lanes.
     LaneBitmask PrevDefLaneMask = V2SU.LaneMask;
-    if ((PrevDefLaneMask & LaneMask) == 0)
+    if ((PrevDefLaneMask & LaneMask).none())
       continue;
     if (V2SU.SU == SU)
       continue;
 
     V2SU.SU->addPred(SDep(SU, SDep::Anti, Reg));
   }
 }
 

llvm/trunk/lib/CodeGen/SplitKit.cpp

for (LiveInterval::SubRange &S : LI.subranges()) {
         S.createDeadDef(Def, LIS.getVNInfoAllocator());
     }
   } else {
     // This is a new def: either from rematerialization, or from an inserted
     // copy. Since rematerialization can regenerate a definition of a sub-
     // register, we need to check which subranges need to be updated.
     const MachineInstr *DefMI = LIS.getInstructionFromIndex(Def);
     assert(DefMI != nullptr);
-    LaneBitmask LM = 0;
+    LaneBitmask LM;
     for (const MachineOperand &DefOp : DefMI->defs()) {
       unsigned R = DefOp.getReg();
       if (R != LI.reg)
         continue;
       if (unsigned SR = DefOp.getSubReg())
         LM |= TRI.getSubRegIndexLaneMask(SR);
       else {
         LM = MRI.getMaxLaneMaskForVReg(R);
         break;
       }
     }
     for (LiveInterval::SubRange &S : LI.subranges())
-      if (S.LaneMask & LM)
+      if (!(S.LaneMask & LM).none())
         S.createDeadDef(Def, LIS.getVNInfoAllocator());
   }
 }
 
 VNInfo *SplitEditor::defValue(unsigned RegIdx,
                               const VNInfo *ParentVNI,
                               SlotIndex Idx,
                               bool Original) {
void SplitEditor::extendPHIRange(MachineBasicBlock &B, LiveRangeCalc &LRC,
   for (MachineBasicBlock *P : B.predecessors()) {
     SlotIndex End = LIS.getMBBEndIdx(P);
     SlotIndex LastUse = End.getPrevSlot();
     // The predecessor may not have a live-out value. That is OK, like an
     // undef PHI operand.
     LiveInterval &PLI = Edit->getParent();
     // Need the cast because the inputs to ?: would otherwise be deemed
     // "incompatible": SubRange vs LiveInterval.
-    LiveRange &PSR = (LM != ~0u) ? getSubRangeForMask(LM, PLI)
-                                 : static_cast<LiveRange&>(PLI);
+    LiveRange &PSR = !LM.all() ? getSubRangeForMask(LM, PLI)
     if (PSR.liveAt(LastUse))
       LRC.extend(LR, End, /*PhysReg=*/0, Undefs);
   }
 }
 
 void SplitEditor::extendPHIKillRanges() {
   // Extend live ranges to be live-out for successor PHI values.
 
   // Visit each PHI def slot in the parent live interval. If the def is dead,
   // remove it. Otherwise, extend the live interval to reach the end indexes
   // of all predecessor blocks.
 
   LiveInterval &ParentLI = Edit->getParent();
   for (const VNInfo *V : ParentLI.valnos) {
     if (V->isUnused() || !V->isPHIDef())
       continue;
 
     unsigned RegIdx = RegAssign.lookup(V->def);
     LiveInterval &LI = LIS.getInterval(Edit->get(RegIdx));
     LiveRangeCalc &LRC = getLRCalc(RegIdx);
     MachineBasicBlock &B = *LIS.getMBBFromIndex(V->def);
     if (!removeDeadSegment(V->def, LI))
-      extendPHIRange(B, LRC, LI, ~0u, /*Undefs=*/{});
+      extendPHIRange(B, LRC, LI, LaneBitmask::getAll(), /*Undefs=*/{});
   }
 
   SmallVector<SlotIndex, 4> Undefs;
   LiveRangeCalc SubLRC;
 
   for (LiveInterval::SubRange &PS : ParentLI.subranges()) {
     for (const VNInfo *V : PS.valnos) {
       if (V->isUnused() || !V->isPHIDef())
for (ExtPoint &EP : ExtPoints) {
     LiveInterval &LI = LIS.getInterval(Edit->get(EP.RegIdx));
     assert(LI.hasSubRanges());
 
     LiveRangeCalc SubLRC;
     unsigned Reg = EP.MO.getReg(), Sub = EP.MO.getSubReg();
     LaneBitmask LM = Sub != 0 ? TRI.getSubRegIndexLaneMask(Sub)
                               : MRI.getMaxLaneMaskForVReg(Reg);
     for (LiveInterval::SubRange &S : LI.subranges()) {
-      if (!(S.LaneMask & LM))
+      if ((S.LaneMask & LM).none())
         continue;
       // The problem here can be that the new register may have been created
       // for a partially defined original register. For example:
       //   %vreg827:subreg_hireg<def,read-undef> = ...
       //   ...
       //   %vreg828<def> = COPY %vreg827
       if (S.empty())
         continue;

llvm/trunk/lib/CodeGen/TargetRegisterInfo.cpp

 
 #define DEBUG_TYPE "target-reg-info"
 
 using namespace llvm;
 
 TargetRegisterInfo::TargetRegisterInfo(const TargetRegisterInfoDesc *ID,
                              regclass_iterator RCB, regclass_iterator RCE,
                              const char *const *SRINames,
-                             const unsigned *SRILaneMasks,
-                             unsigned SRICoveringLanes)
+                             const LaneBitmask *SRILaneMasks,
+                             LaneBitmask SRICoveringLanes)
   : InfoDesc(ID), SubRegIndexNames(SRINames),
     SubRegIndexLaneMasks(SRILaneMasks),
     RegClassBegin(RCB), RegClassEnd(RCE),
     CoveringLanes(SRICoveringLanes) {
 }
 
 TargetRegisterInfo::~TargetRegisterInfo() {}
 
return Printable([Unit, TRI](raw_ostream &OS) {
     if (TRI && TRI->isVirtualRegister(Unit)) {
       OS << "%vreg" << TargetRegisterInfo::virtReg2Index(Unit);
     } else {
       OS << PrintRegUnit(Unit, TRI);
     }
   });
 }
 
-Printable PrintLaneMask(LaneBitmask LaneMask) {
-  return Printable([LaneMask](raw_ostream &OS) {
-    OS << format("%08X", LaneMask);
-  });
-}
-
 } // End of llvm namespace
 
 /// getAllocatableClass - Return the maximal subclass of the given register
 /// class that is alloctable, or NULL.
 const TargetRegisterClass *
 TargetRegisterInfo::getAllocatableClass(const TargetRegisterClass *RC) const {
   if (!RC || RC->isAllocatable())
     return RC;

llvm/trunk/lib/CodeGen/VirtRegMap.cpp

void VirtRegRewriter::addLiveInsForSubRanges(const LiveInterval &LI,
 
   // Check all mbb start positions between First and Last while
   // simulatenously advancing an iterator for each subrange.
   for (SlotIndexes::MBBIndexIterator MBBI = Indexes->findMBBIndex(First);
        MBBI != Indexes->MBBIndexEnd() && MBBI->first <= Last; ++MBBI) {
     SlotIndex MBBBegin = MBBI->first;
     // Advance all subrange iterators so that their end position is just
     // behind MBBBegin (or the iterator is at the end).
-    LaneBitmask LaneMask = 0;
+    LaneBitmask LaneMask;
     for (auto &RangeIterPair : SubRanges) {
       const LiveInterval::SubRange *SR = RangeIterPair.first;
       LiveInterval::const_iterator &SRI = RangeIterPair.second;
       while (SRI != SR->end() && SRI->end <= MBBBegin)
         ++SRI;
       if (SRI == SR->end())
         continue;
       if (SRI->start <= MBBBegin)
         LaneMask |= SR->LaneMask;
     }
-    if (LaneMask == 0)
+    if (LaneMask.none())
       continue;
     MachineBasicBlock *MBB = MBBI->second;
     MBB->addLiveIn(PhysReg, LaneMask);
   }
 }
 
 // Compute MBB live-in lists from virtual register live ranges and their
 // assignments.
bool VirtRegRewriter::readsUndefSubreg(const MachineOperand &MO) const {
   // we couldn't properly detect before.
   assert(LI.liveAt(BaseIndex) &&
          "Reads of completely dead register should be marked undef already");
   unsigned SubRegIdx = MO.getSubReg();
   assert(SubRegIdx != 0 && LI.hasSubRanges());
   LaneBitmask UseMask = TRI->getSubRegIndexLaneMask(SubRegIdx);
   // See if any of the relevant subregister liveranges is defined at this point.
   for (const LiveInterval::SubRange &SR : LI.subranges()) {
-    if ((SR.LaneMask & UseMask) != 0 && SR.liveAt(BaseIndex))
+    if (!(SR.LaneMask & UseMask).none() && SR.liveAt(BaseIndex))
       return false;
   }
   return true;
 }
 
 void VirtRegRewriter::handleIdentityCopy(MachineInstr &MI) const {
   if (!MI.isIdentityCopy())
     return;

llvm/trunk/lib/MC/MCRegisterInfo.cpp

 //=== MC/MCRegisterInfo.cpp - Target Register Description -------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements MCRegisterInfo functions.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/MC/MCRegisterInfo.h"
+#include "llvm/Support/Format.h"
+#include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 
 unsigned MCRegisterInfo::getMatchingSuperReg(unsigned Reg, unsigned SubIdx,
                                              const MCRegisterClass *RC) const {
   for (MCSuperRegIterator Supers(Reg, this); Supers.isValid(); ++Supers)
     if (RC->contains(*Supers) && Reg == getSubReg(*Supers, SubIdx))
       return *Supers;

llvm/trunk/lib/Target/AMDGPU/SIRegisterInfo.cpp

 }
 
 const TargetRegisterClass *SIRegisterInfo::getSubRegClass(
                          const TargetRegisterClass *RC, unsigned SubIdx) const {
   if (SubIdx == AMDGPU::NoSubRegister)
     return RC;
 
   // We can assume that each lane corresponds to one 32-bit register.
-  unsigned Count = countPopulation(getSubRegIndexLaneMask(SubIdx));
+  LaneBitmask::Type Mask = getSubRegIndexLaneMask(SubIdx).getAsInteger();
+  unsigned Count = countPopulation(Mask);
   if (isSGPRClass(RC)) {
     switch (Count) {
     case 1:
       return &AMDGPU::SGPR_32RegClass;
     case 2:
       return &AMDGPU::SReg_64RegClass;
     case 4:
       return &AMDGPU::SReg_128RegClass;

llvm/trunk/lib/Target/Hexagon/HexagonBlockRanges.cpp

 }
 
 HexagonBlockRanges::RegisterSet HexagonBlockRanges::getLiveIns(
       const MachineBasicBlock &B, const MachineRegisterInfo &MRI,
       const TargetRegisterInfo &TRI) {
   RegisterSet LiveIns;
   RegisterSet Tmp;
   for (auto I : B.liveins()) {
-    if (I.LaneMask == ~LaneBitmask(0)) {
+    if (I.LaneMask.all()) {
       Tmp.insert({I.PhysReg,0});
       continue;
     }
     for (MCSubRegIndexIterator S(I.PhysReg, &TRI); S.isValid(); ++S) {
       LaneBitmask M = TRI.getSubRegIndexLaneMask(S.getSubRegIndex());
-      if (M & I.LaneMask)
+      if (!(M & I.LaneMask).none())
         Tmp.insert({S.getSubReg(), 0});
     }
   }
 
   for (auto R : Tmp) {
     if (!Reserved[R.Reg])
       LiveIns.insert(R);
     for (auto S : expandToSubRegs(R, MRI, TRI))

llvm/trunk/lib/Target/Hexagon/HexagonExpandCondsets.cpp

void HexagonExpandCondsets::updateDeadsInRange(unsigned Reg, LaneBitmask LM,
 
   auto IsRegDef = [this,Reg,LM] (MachineOperand &Op) -> bool {
     if (!Op.isReg() || !Op.isDef())
       return false;
     unsigned DR = Op.getReg(), DSR = Op.getSubReg();
     if (!TargetRegisterInfo::isVirtualRegister(DR) || DR != Reg)
       return false;
     LaneBitmask SLM = getLaneMask(DR, DSR);
-    return (SLM & LM) != 0;
+    return !(SLM & LM).none();
   };
 
   // The splitting step will create pairs of predicated definitions without
   // any implicit uses (since implicit uses would interfere with predication).
   // This can cause the reaching defs to become dead after live range
   // recomputation, even though they are not really dead.
   // We need to identify predicated defs that need implicit uses, and
   // dead defs that are not really dead, and correct both problems.

llvm/trunk/lib/Target/Hexagon/RDFGraph.h

private:
     AllocatorTy MemPool;
   };
 
   struct RegisterRef {
     RegisterId Reg;
     LaneBitmask Mask;
 
     RegisterRef() : RegisterRef(0) {}
-    explicit RegisterRef(RegisterId R, LaneBitmask M = ~LaneBitmask(0))
-      : Reg(R), Mask(R != 0 ? M : 0) {}
-    operator bool() const { return Reg != 0 && Mask != LaneBitmask(0); }
+    explicit RegisterRef(RegisterId R, LaneBitmask M = LaneBitmask::getAll())
+      : Reg(R), Mask(R != 0 ? M : LaneBitmask::getNone()) {}
+    operator bool() const { return Reg != 0 && !Mask.none(); }
     bool operator== (const RegisterRef &RR) const {
       return Reg == RR.Reg && Mask == RR.Mask;
     }
     bool operator!= (const RegisterRef &RR) const {
       return !operator==(RR);
     }
     bool operator< (const RegisterRef &RR) const {
       return Reg < RR.Reg || (Reg == RR.Reg && Mask < RR.Mask);
uint32_t insert(T Val) {
       if (F != Map.end())
         return F - Map.begin() + 1;
       Map.push_back(Val);
       return Map.size();  // Return actual_index + 1.
     }
     uint32_t find(T Val) const {
       auto F = llvm::find(Map, Val);
       assert(F != Map.end());
-      return *F;
+      return F - Map.begin();
     }
   private:
     std::vector<T> Map;
   };
 
   struct LaneMaskIndex : private IndexedSet<LaneBitmask> {
     LaneMaskIndex() = default;
 
     LaneBitmask getLaneMaskForIndex(uint32_t K) const {
-      return K == 0 ? ~LaneBitmask(0) : get(K);
+      return K == 0 ? LaneBitmask::getAll() : get(K);
     }
     uint32_t getIndexForLaneMask(LaneBitmask LM) {
-      assert(LM != LaneBitmask(0));
-      return LM == ~LaneBitmask(0) ? 0 : insert(LM);
+      assert(!LM.none());
+      return LM.all() ? 0 : insert(LM);
     }
     uint32_t getIndexForLaneMask(LaneBitmask LM) const {
-      assert(LM != LaneBitmask(0));
-      return LM == ~LaneBitmask(0) ? 0 : find(LM);
+      assert(!LM.none());
+      return LM.all() ? 0 : find(LM);
     }
     PackedRegisterRef pack(RegisterRef RR) {
       return { RR.Reg, getIndexForLaneMask(RR.Mask) };
     }
     PackedRegisterRef pack(RegisterRef RR) const {
       return { RR.Reg, getIndexForLaneMask(RR.Mask) };
     }
     RegisterRef unpack(PackedRegisterRef PR) const {

llvm/trunk/lib/Target/Hexagon/RDFGraph.cpp

 using namespace rdf;
 
 // Printing functions. Have them here first, so that the rest of the code
 // can use them.
 namespace llvm {
 namespace rdf {
 
 raw_ostream &operator<< (raw_ostream &OS, const PrintLaneMaskOpt &P) {
-  if (P.Mask != ~LaneBitmask(0))
+  if (!P.Mask.all())
     OS << ':' << PrintLaneMask(P.Mask);
   return OS;
 }
 
 template<>
 raw_ostream &operator<< (raw_ostream &OS, const Print<RegisterRef> &P) {
   auto &TRI = P.G.getTRI();
   if (P.Obj.Reg > 0 && P.Obj.Reg < TRI.getNumRegs())
RegisterRef RegisterAggr::normalize(RegisterRef RR) const {
   LaneBitmask SuperMask = TRI.composeSubRegIndexLaneMask(Sub, Common);
   return RegisterRef(SuperReg, SuperMask);
 }
 
 bool RegisterAggr::hasAliasOf(RegisterRef RR) const {
   RegisterRef NR = normalize(RR);
   auto F = Masks.find(NR.Reg);
   if (F != Masks.end()) {
-    if (F->second & NR.Mask)
+    if (!(F->second & NR.Mask).none())
       return true;
   }
   if (CheckUnits) {
     for (MCRegUnitIterator U(RR.Reg, &TRI); U.isValid(); ++U)
       if (ExpAliasUnits.test(*U))
         return true;
   }
   return false;
 }
 
 bool RegisterAggr::hasCoverOf(RegisterRef RR) const {
   // Always have a cover for empty lane mask.
   RegisterRef NR = normalize(RR);
-  if (!NR.Mask)
+  if (NR.Mask.none())
     return true;
   auto F = Masks.find(NR.Reg);
   if (F == Masks.end())
     return false;
   return (NR.Mask & F->second) == NR.Mask;
 }
 
 RegisterAggr &RegisterAggr::insert(RegisterRef RR) {

 }
 
 RegisterAggr &RegisterAggr::clear(RegisterRef RR) {
   RegisterRef NR = normalize(RR);
   auto F = Masks.find(NR.Reg);
   if (F == Masks.end())
     return *this;
   LaneBitmask NewM = F->second & ~NR.Mask;
-  if (NewM == LaneBitmask(0))
+  if (NewM.none())
     Masks.erase(F);
   else
     F->second = NewM;
   return *this;
 }
 
 RegisterAggr &RegisterAggr::clear(const RegisterAggr &RG) {
   for (std::pair<RegisterId,LaneBitmask> P : RG.Masks)
RegisterRef DataFlowGraph::normalizeRef(RegisterRef RR) const {
   LaneBitmask SuperMask = RR.Mask &
                           TRI.composeSubRegIndexLaneMask(Sub, RC.LaneMask);
   return RegisterRef(SuperReg, SuperMask);
 }
 
 RegisterRef DataFlowGraph::restrictRef(RegisterRef AR, RegisterRef BR) const {
   if (AR.Reg == BR.Reg) {
     LaneBitmask M = AR.Mask & BR.Mask;
-    return M ? RegisterRef(AR.Reg, M) : RegisterRef();
+    return !M.none() ? RegisterRef(AR.Reg, M) : RegisterRef();
   }
 #ifndef NDEBUG
   RegisterRef NAR = normalizeRef(AR);
   RegisterRef NBR = normalizeRef(BR);
   assert(NAR.Reg != NBR.Reg);
 #endif
   // This isn't strictly correct, because the overlap may happen in the
   // part masked out.
bool DataFlowGraph::alias(RegisterRef RA, RegisterRef RB) const {
   while (UMA.isValid() && UMB.isValid()) {
     std::pair<uint32_t,LaneBitmask> PA = *UMA;
     std::pair<uint32_t,LaneBitmask> PB = *UMB;
     if (PA.first == PB.first) {
       // Lane mask of 0 (given by the iterator) should be treated as "full".
       // This can happen when the register has only one unit, or when the
       // unit corresponds to explicit aliasing. In such cases, the lane mask
       // from RegisterRef should be ignored.
-      if (!PA.second || !PB.second)
+      if (PA.second.none() || PB.second.none())
         return true;
 
       // At this point the common unit corresponds to a subregister. The lane
       // masks correspond to the lane mask of that unit within the original
       // register, for example assuming register quadruple q0 = r3:0, and
       // a register pair d1 = r3:2, the lane mask of r2 in q0 may be 0b0100,
       // while the lane mask of r2 in d1 may be 0b0001.
       LaneBitmask LA = PA.second & RA.Mask;
       LaneBitmask LB = PB.second & RB.Mask;
-      if (LA != 0 && LB != 0) {
+      if (!LA.none() && !LB.none()) {
         unsigned Root = *MCRegUnitRootIterator(PA.first, &TRI);
         // If register units were guaranteed to only have 1 bit in any lane
         // mask, the code below would not be necessary. This is because LA
         // and LB would have at most 1 bit set each, and that bit would be
         // guaranteed to correspond to the given register unit.
         uint32_t SubA = TRI.getSubRegIndex(RA.Reg, Root);
         uint32_t SubB = TRI.getSubRegIndex(RB.Reg, Root);
         const TargetRegisterClass &RC = *TRI.getMinimalPhysRegClass(Root);
         LaneBitmask MaskA = TRI.reverseComposeSubRegIndexLaneMask(SubA, LA);
         LaneBitmask MaskB = TRI.reverseComposeSubRegIndexLaneMask(SubB, LB);
-        if (MaskA & MaskB & RC.LaneMask)
+        if (!(MaskA & MaskB & RC.LaneMask).none())
           return true;
       }
 
       ++UMA;
       ++UMB;
       continue;
     }
     if (PA.first < PB.first)

llvm/trunk/lib/Target/Hexagon/RDFLiveness.cpp

for (NodeAddr<PhiNode*> PA : Phis) {
         // predecessor.
         // Remap all the RUs so that they have a correct reaching def.
         auto PrA = DFG.addr<BlockNode*>(PUA.Addr->getPredecessor());
         RefMap &LOX = PhiLOX[PrA.Addr->getCode()];
 
         RegisterRef UR = DFG.normalizeRef(getRestrictedRegRef(PUA));
         for (const std::pair<RegisterId,NodeRefSet> &T : RUs) {
           // Check if T.first aliases UR?
-          LaneBitmask M = 0;
+          LaneBitmask M;
           for (std::pair<NodeId,LaneBitmask> P : T.second)
             M |= P.second;
 
           RegisterRef S = DFG.restrictRef(RegisterRef(T.first, M), UR);
           if (!S)
             continue;
           for (NodeAddr<DefNode*> D : getAllReachingDefs(S, PUA))
             LOX[S.Reg].insert({D.Id, S.Mask});
for (MachineBasicBlock &B : MF) {
       for (std::pair<RegisterId,LaneBitmask> P : LiveMap[&B]) {
         MCSubRegIndexIterator S(P.first, &TRI);
         if (!S.isValid()) {
           LV.push_back(RegisterRef(P.first));
           continue;
         }
         do {
           LaneBitmask M = TRI.getSubRegIndexLaneMask(S.getSubRegIndex());
-          if (M & P.second)
+          if (!(M & P.second).none())
             LV.push_back(RegisterRef(S.getSubReg()));
           ++S;
         } while (S.isValid());
       }
       std::sort(LV.begin(), LV.end());
       dbgs() << "\tcomp = {";
       for (auto I : LV)
         dbgs() << ' ' << Print<RegisterRef>(I, DFG);
auto CopyLiveIns = [this] (MachineBasicBlock *B, BitVector &LV) -> void {
     for (auto I : B->liveins()) {
       MCSubRegIndexIterator S(I.PhysReg, &TRI);
       if (!S.isValid()) {
         LV.set(I.PhysReg);
         continue;
       }
       do {
         LaneBitmask M = TRI.getSubRegIndexLaneMask(S.getSubRegIndex());
-        if (M & I.LaneMask)
+        if (!(M & I.LaneMask).none())
           LV.set(S.getSubReg());
         ++S;
       } while (S.isValid());
     }
   };
 
   BitVector LiveIn(TRI.getNumRegs()), Live(TRI.getNumRegs());
   CopyLiveIns(B, LiveIn);
if (Trace) {
     dbgs() << "  Local:  " << Print<RegisterAggr>(LiveMap[B], DFG) << '\n';
   }
 
   // Phi uses should not be propagated up the dominator tree, since they
   // are not dominated by their corresponding reaching defs.
   RegisterAggr &Local = LiveMap[B];
   RefMap &LON = PhiLON[B];
   for (auto &R : LON) {
-    LaneBitmask M = 0;
+    LaneBitmask M;
     for (auto P : R.second)
       M |= P.second;
     Local.insert(RegisterRef(R.first,M));
   }
 
   if (Trace) {
     dbgs() << "after phi uses in block\n";
     dbgs() << "  LiveIn: " << Print<RefMap>(LiveIn, DFG) << '\n';

llvm/trunk/utils/TableGen/CodeGenRegisters.h

 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/SparseBitVector.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/CodeGen/MachineValueType.h"
+#include "llvm/MC/LaneBitmask.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/TableGen/Record.h"
 #include "llvm/TableGen/SetTheory.h"
 #include <cassert>
 #include <cstdint>
 #include <deque>
 #include <list>
 #include <map>
 #include <string>
 #include <utility>
 #include <vector>
 
 namespace llvm {
 
   class CodeGenRegBank;
   template <typename T, typename Vector, typename Set> class SetVector;
 
   /// Used to encode a step in a register lane mask transformation.
   /// Mask the bits specified in Mask, then rotate them Rol bits to the left
   /// assuming a wraparound at 32bits.
   struct MaskRolPair {
-    unsigned Mask;
+    LaneBitmask Mask;
     uint8_t RotateLeft;
 
     bool operator==(const MaskRolPair Other) const {
       return Mask == Other.Mask && RotateLeft == Other.RotateLeft;
     }
     bool operator!=(const MaskRolPair Other) const {
       return Mask != Other.Mask || RotateLeft != Other.RotateLeft;
     }
   };
 
   /// CodeGenSubRegIndex - Represents a sub-register index.
   class CodeGenSubRegIndex {
     Record *const TheDef;
     std::string Name;
     std::string Namespace;
 
   public:
     uint16_t Size;
     uint16_t Offset;
     const unsigned EnumValue;
-    mutable unsigned LaneMask;
+    mutable LaneBitmask LaneMask;
     mutable SmallVector<MaskRolPair,1> CompositionLaneMaskTransform;
 
     // Are all super-registers containing this SubRegIndex covered by their
     // sub-registers?
     bool AllSuperRegsCovered;
 
     CodeGenSubRegIndex(Record *R, unsigned Enum);
     CodeGenSubRegIndex(StringRef N, StringRef Nspace, unsigned Enum);
public:
 
     // Update the composite maps of components specified in 'ComposedOf'.
     void updateComponents(CodeGenRegBank&);
 
     // Return the map of composites.
     const CompMap &getComposites() const { return Composed; }
 
     // Compute LaneMask from Composed. Return LaneMask.
-    unsigned computeLaneMask() const;
+    LaneBitmask computeLaneMask() const;
 
   private:
     CompMap Composed;
   };
 
   inline bool operator<(const CodeGenSubRegIndex &A,
                         const CodeGenSubRegIndex &B) {
     return A.EnumValue < B.EnumValue;
struct CodeGenRegister {
     // (TopoSig-wise).
     unsigned getTopoSig() const {
       assert(SuperRegsComplete && "TopoSigs haven't been computed yet.");
       return TopoSig;
     }
 
     // List of register units in ascending order.
     typedef SparseBitVector<> RegUnitList;
-    typedef SmallVector<unsigned, 16> RegUnitLaneMaskList;
+    typedef SmallVector<LaneBitmask, 16> RegUnitLaneMaskList;
 
     // How many entries in RegUnitList are native?
     RegUnitList NativeRegUnits;
 
     // Get the list of register units.
     // This is only valid after computeSubRegs() completes.
     const RegUnitList &getRegUnits() const { return RegUnits; }
 
-    ArrayRef<unsigned> getRegUnitLaneMasks() const {
+    ArrayRef<LaneBitmask> getRegUnitLaneMasks() const {
       return makeArrayRef(RegUnitLaneMasks).slice(0, NativeRegUnits.count());
     }
 
     // Get the native register units. This is a prefix of getRegUnits().
     RegUnitList getNativeRegUnits() const {
       return NativeRegUnits;
     }
 
public:
     SmallVector<MVT::SimpleValueType, 4> VTs;
     unsigned SpillSize;
     unsigned SpillAlignment;
     int CopyCost;
     bool Allocatable;
     std::string AltOrderSelect;
     uint8_t AllocationPriority;
     /// Contains the combination of the lane masks of all subregisters.
-    unsigned LaneMask;
+    LaneBitmask LaneMask;
     /// True if there are at least 2 subregisters which do not interfere.
     bool HasDisjunctSubRegs;
     bool CoveredBySubRegs;
 
     // Return the Record that defined this class, or NULL if the class was
     // created by TableGen.
     Record *getDef() const { return TheDef; }
 
public:
     //
     // This is used to compute the mask of call-preserved registers from a list
     // of callee-saves.
     BitVector computeCoveredRegisters(ArrayRef<Record*> Regs);
 
     // Bit mask of lanes that cover their registers. A sub-register index whose
     // LaneMask is contained in CoveringLanes will be completely covered by
     // another sub-register with the same or larger lane mask.
-    unsigned CoveringLanes;
+    LaneBitmask CoveringLanes;
   };
 
 } // end namespace llvm
 
 #endif // LLVM_UTILS_TABLEGEN_CODEGENREGISTERS_H

llvm/trunk/utils/TableGen/CodeGenRegisters.cpp

 
 #define DEBUG_TYPE "regalloc-emitter"
 
 //===----------------------------------------------------------------------===//
 //                             CodeGenSubRegIndex
 //===----------------------------------------------------------------------===//
 
 CodeGenSubRegIndex::CodeGenSubRegIndex(Record *R, unsigned Enum)
-  : TheDef(R), EnumValue(Enum), LaneMask(0), AllSuperRegsCovered(true) {
+  : TheDef(R), EnumValue(Enum), AllSuperRegsCovered(true) {
   Name = R->getName();
   if (R->getValue("Namespace"))
     Namespace = R->getValueAsString("Namespace");
   Size = R->getValueAsInt("Size");
   Offset = R->getValueAsInt("Offset");
 }
 
 CodeGenSubRegIndex::CodeGenSubRegIndex(StringRef N, StringRef Nspace,
                                        unsigned Enum)
   : TheDef(nullptr), Name(N), Namespace(Nspace), Size(-1), Offset(-1),
-    EnumValue(Enum), LaneMask(0), AllSuperRegsCovered(true) {
+    EnumValue(Enum), AllSuperRegsCovered(true) {
 }
 
 std::string CodeGenSubRegIndex::getQualifiedName() const {
   std::string N = getNamespace();
   if (!N.empty())
     N += "::";
   N += getName();
   return N;
if (Parts.size() < 2)
                       "CoveredBySubRegs must have two or more entries");
     SmallVector<CodeGenSubRegIndex*, 8> IdxParts;
     for (unsigned i = 0, e = Parts.size(); i != e; ++i)
       IdxParts.push_back(RegBank.getSubRegIdx(Parts[i]));
     RegBank.addConcatSubRegIndex(IdxParts, this);
   }
 }
 
-unsigned CodeGenSubRegIndex::computeLaneMask() const {
+LaneBitmask CodeGenSubRegIndex::computeLaneMask() const {
   // Already computed?
-  if (LaneMask)
+  if (!LaneMask.none())
     return LaneMask;
 
   // Recursion guard, shouldn't be required.
-  LaneMask = ~0u;
+  LaneMask = LaneBitmask::getAll();
 
   // The lane mask is simply the union of all sub-indices.
-  unsigned M = 0;
+  LaneBitmask M;
   for (const auto &C : Composed)
     M |= C.second->computeLaneMask();
-  assert(M && "Missing lane mask, sub-register cycle?");
+  assert(!M.none() && "Missing lane mask, sub-register cycle?");
   LaneMask = M;
   return LaneMask;
 }
 
 //===----------------------------------------------------------------------===//
 //                              CodeGenRegister
 //===----------------------------------------------------------------------===//
 
static void sortAndUniqueRegisters(CodeGenRegister::Vec &M) {
   std::sort(M.begin(), M.end(), deref<llvm::less>());
   M.erase(std::unique(M.begin(), M.end(), deref<llvm::equal>()), M.end());
 }
 
 CodeGenRegisterClass::CodeGenRegisterClass(CodeGenRegBank &RegBank, Record *R)
   : TheDef(R),
     Name(R->getName()),
     TopoSigs(RegBank.getNumTopoSigs()),
-    EnumValue(-1),
-    LaneMask(0) {
+    EnumValue(-1) {
   // Rename anonymous register classes.
   if (R->getName().size() > 9 && R->getName()[9] == '.') {
     static unsigned AnonCounter = 0;
     R->setName("AnonRegClass_" + utostr(AnonCounter++));
   }
 
   std::vector<Record*> TypeList = R->getValueAsListOfDefs("RegTypes");
   for (unsigned i = 0, e = TypeList.size(); i != e; ++i) {

 // indices overlap in some register.
 //
 // Conservatively share a lane mask bit if two sub-register indices overlap in
 // some registers, but not in others. That shouldn't happen a lot.
 void CodeGenRegBank::computeSubRegLaneMasks() {
   // First assign individual bits to all the leaf indices.
   unsigned Bit = 0;
   // Determine mask of lanes that cover their registers.
-  CoveringLanes = ~0u;
+  CoveringLanes = LaneBitmask::getAll();
   for (auto &Idx : SubRegIndices) {
     if (Idx.getComposites().empty()) {
       if (Bit > 32) {
         PrintFatalError(
           Twine("Ran out of lanemask bits to represent subregister ")
           + Idx.getName());
       }
-      Idx.LaneMask = 1u << Bit;
+      Idx.LaneMask = LaneBitmask(1 << Bit);
       ++Bit;
     } else {
-      Idx.LaneMask = 0;
+      Idx.LaneMask = LaneBitmask::getNone();
     }
   }
 
   // Compute transformation sequences for composeSubRegIndexLaneMask. The idea
   // here is that for each possible target subregister we look at the leafs
   // in the subregister graph that compose for this target and create
   // transformation sequences for the lanemasks. Each step in the sequence
   // consists of a bitmask and a bitrotate operation. As the rotation amounts
   // are usually the same for many subregisters we can easily combine the steps
   // by combining the masks.
   for (const auto &Idx : SubRegIndices) {
     const auto &Composites = Idx.getComposites();
     auto &LaneTransforms = Idx.CompositionLaneMaskTransform;
 
     if (Composites.empty()) {
       // Moving from a class with no subregisters we just had a single lane:
       // The subregister must be a leaf subregister and only occupies 1 bit.
       // Move the bit from the class without subregisters into that position.
-      unsigned DstBit = Log2_32(Idx.LaneMask);
-      assert(Idx.LaneMask == 1u << DstBit && "Must be a leaf subregister");
-      MaskRolPair MaskRol = { 1, (uint8_t)DstBit };
+      static_assert(sizeof(Idx.LaneMask.getAsInteger()) == 4,
+                    "Change Log2_32 to a proper one");
+      unsigned DstBit = Log2_32(Idx.LaneMask.getAsInteger());
+      assert(Idx.LaneMask == LaneBitmask(1 << DstBit) &&
+             "Must be a leaf subregister");
+      MaskRolPair MaskRol = { LaneBitmask(1), (uint8_t)DstBit };
       LaneTransforms.push_back(MaskRol);
     } else {
       // Go through all leaf subregisters and find the ones that compose with
       // Idx. These make out all possible valid bits in the lane mask we want to
       // transform. Looking only at the leafs ensure that only a single bit in
       // the mask is set.
       unsigned NextBit = 0;
       for (auto &Idx2 : SubRegIndices) {
         // Skip non-leaf subregisters.
         if (!Idx2.getComposites().empty())
           continue;
         // Replicate the behaviour from the lane mask generation loop above.
         unsigned SrcBit = NextBit;
-        unsigned SrcMask = 1u << SrcBit;
-        if (NextBit < 31)
+        LaneBitmask SrcMask = LaneBitmask(1 << SrcBit);
+        if (NextBit < LaneBitmask::BitWidth-1)
           ++NextBit;
         assert(Idx2.LaneMask == SrcMask);
 
         // Get the composed subregister if there is any.
         auto C = Composites.find(&Idx2);
         if (C == Composites.end())
           continue;
         const CodeGenSubRegIndex *Composite = C->second;
         // The Composed subreg should be a leaf subreg too
         assert(Composite->getComposites().empty());
 
         // Create Mask+Rotate operation and merge with existing ops if possible.
-        unsigned DstBit = Log2_32(Composite->LaneMask);
+        static_assert(sizeof(Composite->LaneMask.getAsInteger()) == 4,
+                      "Change Log2_32 to a proper one");
+        unsigned DstBit = Log2_32(Composite->LaneMask.getAsInteger());
         int Shift = DstBit - SrcBit;
-        uint8_t RotateLeft = Shift >= 0 ? (uint8_t)Shift : 32+Shift;
+        uint8_t RotateLeft = Shift >= 0 ? (uint8_t)Shift
+                                        : LaneBitmask::BitWidth + Shift;
         for (auto &I : LaneTransforms) {
           if (I.RotateLeft == RotateLeft) {
             I.Mask |= SrcMask;
-            SrcMask = 0;
+            SrcMask = LaneBitmask::getNone();
           }
         }
-        if (SrcMask != 0) {
+        if (!SrcMask.none()) {
           MaskRolPair MaskRol = { SrcMask, RotateLeft };
           LaneTransforms.push_back(MaskRol);
         }
       }
     }
 
     // Optimize if the transformation consists of one step only: Set mask to
     // 0xffffffff (including some irrelevant invalid bits) so that it should
     // merge with more entries later while compressing the table.
     if (LaneTransforms.size() == 1)
-      LaneTransforms[0].Mask = ~0u;
+      LaneTransforms[0].Mask = LaneBitmask::getAll();
 
     // Further compression optimization: For invalid compositions resulting
     // in a sequence with 0 entries we can just pick any other. Choose
     // Mask 0xffffffff with Rotation 0.
     if (LaneTransforms.size() == 0) {
-      MaskRolPair P = { ~0u, 0 };
+      MaskRolPair P = { LaneBitmask::getAll(), 0 };
       LaneTransforms.push_back(P);
     }
   }
 
   // FIXME: What if ad-hoc aliasing introduces overlaps that aren't represented
   // by the sub-register graph? This doesn't occur in any known targets.
 
   // Inherit lanes from composites.
   for (const auto &Idx : SubRegIndices) {
-    unsigned Mask = Idx.computeLaneMask();
+    LaneBitmask Mask = Idx.computeLaneMask();
     // If some super-registers without CoveredBySubRegs use this index, we can
     // no longer assume that the lanes are covering their registers.
     if (!Idx.AllSuperRegsCovered)
       CoveringLanes &= ~Mask;
   }
 
   // Compute lane mask combinations for register classes.
   for (auto &RegClass : RegClasses) {
-    unsigned LaneMask = 0;
+    LaneBitmask LaneMask;
     for (const auto &SubRegIndex : SubRegIndices) {
       if (RegClass.getSubClassWithSubReg(&SubRegIndex) == nullptr)
         continue;
       LaneMask |= SubRegIndex.LaneMask;
     }
 
     // For classes without any subregisters set LaneMask to 1 instead of 0.
     // This makes it easier for client code to handle classes uniformly.
-    if (LaneMask == 0)
-      LaneMask = 1;
+    if (LaneMask.none())
+      LaneMask = LaneBitmask(1);
 
     RegClass.LaneMask = LaneMask;
   }
 }
 
 namespace {
 
 // UberRegSet is a helper class for computeRegUnitWeights. Each UberRegSet is
for (unsigned UnitIdx = 0, UnitEnd = NumNativeRegUnits;
     }
   }
 }
 
 void CodeGenRegBank::computeRegUnitLaneMasks() {
   for (auto &Register : Registers) {
     // Create an initial lane mask for all register units.
     const auto &RegUnits = Register.getRegUnits();
-    CodeGenRegister::RegUnitLaneMaskList RegUnitLaneMasks(RegUnits.count(), 0);
+    CodeGenRegister::RegUnitLaneMaskList
+        RegUnitLaneMasks(RegUnits.count(), LaneBitmask::getNone());
     // Iterate through SubRegisters.
     typedef CodeGenRegister::SubRegMap SubRegMap;
     const SubRegMap &SubRegs = Register.getSubRegs();
     for (SubRegMap::const_iterator S = SubRegs.begin(),
          SE = SubRegs.end(); S != SE; ++S) {
       CodeGenRegister *SubReg = S->second;
       // Ignore non-leaf subregisters, their lane masks are fully covered by
       // the leaf subregisters anyway.
       if (!SubReg->getSubRegs().empty())
         continue;
       CodeGenSubRegIndex *SubRegIndex = S->first;
       const CodeGenRegister *SubRegister = S->second;
-      unsigned LaneMask = SubRegIndex->LaneMask;
+      LaneBitmask LaneMask = SubRegIndex->LaneMask;
       // Distribute LaneMask to Register Units touched.
       for (unsigned SUI : SubRegister->getRegUnits()) {
         bool Found = false;
         unsigned u = 0;
         for (unsigned RU : RegUnits) {
           if (SUI == RU) {
             RegUnitLaneMasks[u] |= LaneMask;
             assert(!Found);

llvm/trunk/utils/TableGen/RegisterInfoEmitter.cpp

 //
 //   out[0] = InitVal;
 //   out[n+1] = out[n] + diff[n]; // n = 0, 1, ...
 //
 // The initial value depends on the specific list. The list is terminated by a
 // 0 differential which means we can't encode repeated elements.
 
 typedef SmallVector<uint16_t, 4> DiffVec;
-typedef SmallVector<unsigned, 4> MaskVec;
+typedef SmallVector<LaneBitmask, 4> MaskVec;
 
 // Differentially encode a sequence of numbers into V. The starting value and
 // terminating 0 are not added to V, so it will have the same size as List.
 static
 DiffVec &diffEncode(DiffVec &V, unsigned InitVal, SparseBitVector<> List) {
   assert(V.empty() && "Clear DiffVec before diffEncode.");
   uint16_t Val = uint16_t(InitVal);
 
DiffVec &diffEncode(DiffVec &V, unsigned InitVal, Iter Begin, Iter End) {
   }
   return V;
 }
 
 static void printDiff16(raw_ostream &OS, uint16_t Val) {
   OS << Val;
 }
 
-static void printMask(raw_ostream &OS, unsigned Val) {
-  OS << format("0x%08X", Val);
+static void printMask(raw_ostream &OS, LaneBitmask Val) {
+  OS << "LaneBitmask(0x" << PrintLaneMask(Val) << ')';
 }
 
 // Try to combine Idx's compose map into Vec if it is compatible.
 // Return false if it's not possible.
 static bool combine(const CodeGenSubRegIndex *Idx,
                     SmallVectorImpl<CodeGenSubRegIndex*> &Vec) {
   const CodeGenSubRegIndex::CompMap &Map = Idx->getComposites();
   for (const auto &I : Map) {
for (const auto &Idx : SubRegIndices) {
     if (Found == ~0u) {
       Sequences.push_back(IdxSequence);
       Found = SIdx;
     }
     SubReg2SequenceIndexMap.push_back(Found);
   }
 
   OS << "  struct MaskRolOp {\n"
-        "    unsigned Mask;\n"
+        "    LaneBitmask Mask;\n"
         "    uint8_t  RotateLeft;\n"
         "  };\n"
         "  static const MaskRolOp LaneMaskComposeSequences[] = {\n";
   unsigned Idx = 0;
   for (size_t s = 0, se = Sequences.size(); s != se; ++s) {
     OS << "    ";
     const SmallVectorImpl<MaskRolPair> &Sequence = Sequences[s];
     for (size_t p = 0, pe = Sequence.size(); p != pe; ++p) {
       const MaskRolPair &P = Sequence[p];
-      OS << format("{ 0x%08X, %2u }, ", P.Mask, P.RotateLeft);
+      printMask(OS << "{ ", P.Mask);
+      OS << format(", %2u }, ", P.RotateLeft);
     }
-    OS << "{ 0, 0 }";
+    OS << "{ LaneBitmask::getNone(), 0 }";
     if (s+1 != se)
       OS << ", ";
     OS << "  // Sequence " << Idx << "\n";
     Idx += Sequence.size() + 1;
   }
   OS << "  };\n"
         "  static const MaskRolOp *const CompositeSequences[] = {\n";
   for (size_t i = 0, e = SubRegIndices.size(); i != e; ++i) {
     OS << "    ";
     unsigned Idx = SubReg2SequenceIndexMap[i];
     OS << format("&LaneMaskComposeSequences[%u]", Idx);
     if (i+1 != e)
       OS << ",";
     OS << " // to " << SubRegIndices[i].getName() << "\n";
   }
   OS << "  };\n\n";
 
   OS << "LaneBitmask " << ClName
      << "::composeSubRegIndexLaneMaskImpl(unsigned IdxA, LaneBitmask LaneMask)"
         " const {\n"
         "  --IdxA; assert(IdxA < " << SubRegIndices.size()
      << " && \"Subregister index out of bounds\");\n"
-        "  LaneBitmask Result = 0;\n"
-        "  for (const MaskRolOp *Ops = CompositeSequences[IdxA]; Ops->Mask != 0; ++Ops)"
-        " {\n"
-        "    LaneBitmask Masked = LaneMask & Ops->Mask;\n"
-        "    Result |= (Masked << Ops->RotateLeft) & 0xFFFFFFFF;\n"
-        "    Result |= (Masked >> ((32 - Ops->RotateLeft) & 0x1F));\n"
+        "  LaneBitmask Result;\n"
+        "  for (const MaskRolOp *Ops = CompositeSequences[IdxA]; !Ops->Mask.none(); ++Ops) {\n"
+        "    LaneBitmask::Type M = LaneMask.getAsInteger() & Ops->Mask.getAsInteger();\n"
+        "    unsigned S = Ops->RotateLeft;\n"
+        "    Result |= LaneBitmask((M << S) | (M >> (LaneBitmask::BitWidth - S)));\n"
         "  }\n"
         "  return Result;\n"
         "}\n\n";
 
   OS << "LaneBitmask " << ClName
      << "::reverseComposeSubRegIndexLaneMaskImpl(unsigned IdxA, "
         " LaneBitmask LaneMask) const {\n"
         "  LaneMask &= getSubRegIndexLaneMask(IdxA);\n"
         "  --IdxA; assert(IdxA < " << SubRegIndices.size()
      << " && \"Subregister index out of bounds\");\n"
-        "  LaneBitmask Result = 0;\n"
-        "  for (const MaskRolOp *Ops = CompositeSequences[IdxA]; Ops->Mask != 0; ++Ops)"
-        " {\n"
-        "    LaneBitmask Rotated = (LaneMask >> Ops->RotateLeft) |\n"
-        "                          ((LaneMask << ((32 - Ops->RotateLeft) & 0x1F)) & 0xFFFFFFFF);\n"
-        "    Result |= Rotated & Ops->Mask;\n"
+        "  LaneBitmask Result;\n"
+        "  for (const MaskRolOp *Ops = CompositeSequences[IdxA]; !Ops->Mask.none(); ++Ops) {\n"
+        "    LaneBitmask::Type M = LaneMask.getAsInteger();\n"
+        "    unsigned S = Ops->RotateLeft;\n"
+        "    Result |= LaneBitmask((M >> S) | (M << (LaneBitmask::BitWidth - S)));\n"
         "  }\n"
         "  return Result;\n"
         "}\n\n";
 }
 
 //
 // runMCDesc - Print out MC register descriptions.
 //
for (auto I = Regs.begin(), E = Regs.end(); I != E; ++I, ++i) {
     DiffSeqs.add(diffEncode(RegUnitLists[i], Scale * Reg.EnumValue, RUs));
 
     const auto &RUMasks = Reg.getRegUnitLaneMasks();
     MaskVec &LaneMaskVec = RegUnitLaneMasks[i];
     assert(LaneMaskVec.empty());
     LaneMaskVec.insert(LaneMaskVec.begin(), RUMasks.begin(), RUMasks.end());
     // Terminator mask should not be used inside of the list.
 #ifndef NDEBUG
-    for (unsigned M : LaneMaskVec) {
-      assert(M != ~0u && "terminator mask should not be part of the list");
+    for (LaneBitmask M : LaneMaskVec) {
+      assert(!M.all() && "terminator mask should not be part of the list");
     }
 #endif
     LaneMaskSeqs.add(LaneMaskVec);
   }
 
   // Compute the final layout of the sequence table.
   DiffSeqs.layout();
   LaneMaskSeqs.layout();
   SubRegIdxSeqs.layout();
 
   OS << "namespace llvm {\n\n";
 
   const std::string &TargetName = Target.getName();
 
   // Emit the shared table of differential lists.
   OS << "extern const MCPhysReg " << TargetName << "RegDiffLists[] = {\n";
   DiffSeqs.emit(OS, printDiff16);
   OS << "};\n\n";
 
   // Emit the shared table of regunit lane mask sequences.
-  OS << "extern const unsigned " << TargetName << "LaneMaskLists[] = {\n";
-  LaneMaskSeqs.emit(OS, printMask, "~0u");
+  OS << "extern const LaneBitmask " << TargetName << "LaneMaskLists[] = {\n";
+  LaneMaskSeqs.emit(OS, printMask, "LaneBitmask::getAll()");
   OS << "};\n\n";
 
   // Emit the table of sub-register indexes.
   OS << "extern const uint16_t " << TargetName << "SubRegIdxLists[] = {\n";
   SubRegIdxSeqs.emit(OS, printSubRegIndex);
   OS << "};\n\n";
 
   // Emit the table of sub-register index sizes.
RegisterInfoEmitter::runTargetDesc(raw_ostream &OS, CodeGenTarget &Target,
 
   for (const auto &Idx : SubRegIndices) {
     OS << Idx.getName();
     OS << "\", \"";
   }
   OS << "\" };\n\n";
 
   // Emit SubRegIndex lane masks, including 0.
-  OS << "\nstatic const unsigned SubRegIndexLaneMaskTable[] = {\n  ~0u,\n";
+  OS << "\nstatic const LaneBitmask SubRegIndexLaneMaskTable[] = {\n  LaneBitmask::getAll(),\n";
   for (const auto &Idx : SubRegIndices) {
-    OS << format("  0x%08x, // ", Idx.LaneMask) << Idx.getName() << '\n';
+    printMask(OS << "  ", Idx.LaneMask);
+    OS << ", // " << Idx.getName() << '\n';
   }
   OS << " };\n\n";
 
   OS << "\n";
 
   // Now that all of the structs have been emitted, emit the instances.
   if (!RegisterClasses.empty()) {
     OS << "\nstatic const TargetRegisterClass *const "
OS << "\nnamespace " << RegisterClasses.front().Namespace
        << " {   // Register class instances\n";
 
     for (const auto &RC : RegisterClasses) {
       OS << "  extern const TargetRegisterClass " << RC.getName()
          << "RegClass = {\n    " << '&' << Target.getName()
          << "MCRegisterClasses[" << RC.getName() << "RegClassID],\n    "
          << "VTLists + " << VTSeqs.get(RC.VTs) << ",\n    " << RC.getName()
          << "SubClassMask,\n    SuperRegIdxSeqs + "
-         << SuperRegIdxSeqs.get(SuperRegIdxLists[RC.EnumValue]) << ",\n    "
-         << format("0x%08x,\n    ", RC.LaneMask)
-         << (unsigned)RC.AllocationPriority << ",\n    "
+         << SuperRegIdxSeqs.get(SuperRegIdxLists[RC.EnumValue]) << ",\n    ";
+      printMask(OS, RC.LaneMask);
+      OS << ",\n    " << (unsigned)RC.AllocationPriority << ",\n    "
          << (RC.HasDisjunctSubRegs?"true":"false")
          << ", /* HasDisjunctSubRegs */\n    "
          << (RC.CoveredBySubRegs?"true":"false")
          << ", /* CoveredBySubRegs */\n    ";
       if (RC.getSuperClasses().empty())
         OS << "NullRegClasses,\n    ";
       else
         OS << RC.getName() << "Superclasses,\n    ";
OS << "  };\n  assert(RC && \"Missing regclass\");\n"
        << "  return TV ? getRegClass(TV - 1) : nullptr;\n}\n\n";
   }
 
   EmitRegUnitPressure(OS, RegBank, ClassName);
 
   // Emit the constructor of the class...
   OS << "extern const MCRegisterDesc " << TargetName << "RegDesc[];\n";
   OS << "extern const MCPhysReg " << TargetName << "RegDiffLists[];\n";
-  OS << "extern const unsigned " << TargetName << "LaneMaskLists[];\n";
+  OS << "extern const LaneBitmask " << TargetName << "LaneMaskLists[];\n";
   OS << "extern const char " << TargetName << "RegStrings[];\n";
   OS << "extern const char " << TargetName << "RegClassStrings[];\n";
   OS << "extern const MCPhysReg " << TargetName << "RegUnitRoots[][2];\n";
   OS << "extern const uint16_t " << TargetName << "SubRegIdxLists[];\n";
   OS << "extern const MCRegisterInfo::SubRegCoveredBits "
      << TargetName << "SubRegIdxRanges[];\n";
   OS << "extern const uint16_t " << TargetName << "RegEncodingTable[];\n";
 
   EmitRegMappingTables(OS, Regs, true);
 
   OS << ClassName << "::\n" << ClassName
      << "(unsigned RA, unsigned DwarfFlavour, unsigned EHFlavour, unsigned PC)\n"
      << "  : TargetRegisterInfo(" << TargetName << "RegInfoDesc"
      << ", RegisterClasses, RegisterClasses+" << RegisterClasses.size() <<",\n"
-     << "             SubRegIndexNameTable, SubRegIndexLaneMaskTable, 0x";
-  OS.write_hex(RegBank.CoveringLanes);
+     << "             SubRegIndexNameTable, SubRegIndexLaneMaskTable, ";
+  printMask(OS, RegBank.CoveringLanes);
   OS << ") {\n"
      << "  InitMCRegisterInfo(" << TargetName << "RegDesc, " << Regs.size() + 1
      << ", RA, PC,\n                     " << TargetName
      << "MCRegisterClasses, " << RegisterClasses.size() << ",\n"
      << "                     " << TargetName << "RegUnitRoots,\n"
      << "                     " << RegBank.getNumNativeRegUnits() << ",\n"
      << "                     " << TargetName << "RegDiffLists,\n"
      << "                     " << TargetName << "LaneMaskLists,\n"