[TableGen][GlobalISel] Account for HwMode in RegisterBank register sizes

[nitinjohnraj]

Summary

This patch adds logic for determining RegisterBank size to RegisterBankInfo, which allows accounting for the HwMode of the target. Individual RegisterBanks cannot be constructed with HwMode information as construction is generated by TableGen, but a RegisterBankInfo subclass can provide the HwMode as a constructor argument. The HwMode is used to select the appropriate RegisterBank size from an array relating sizes to RegisterBanks.

Targets simply need to provide the HwMode argument to the <target>GenRegisterBankInfo constructor. The RISC-V RegisterBankInfo constructor has been updated accordingly (plus an unused argument removed).

[simoncook]

Patch makes sense, and looking at the newly generated XRegisterBank.inc files, all LGTM

[akuegel]

llvm/lib/CodeGen/GlobalISel/RegisterBankInfo.cpp

61

bkramer already commented on this on the llvm-commits mailing list. Copying it here for visibility:

This can write to a global variable, which is thread-hostile. I see
tsan complain about concurrent writes to llvm::X86::GPRRegBank.

I reverted the patch (commit baa6f6a7828a46c37b96227282938717220f8b34).

[lewis-revill]

My initial thoughts are that this may only be an issue if there can be potentially more than one instance of RegisterBankInfo? And indeed, if it is possible for each of these to have been constructed with differing HwModes? I'll try and look into it in more detail and find out.

[lewis-revill]

I'm lacking an understanding of how concurrency is expected to work within LLVM, but since there is only one RegisterBankInfo constructed per subtarget I don't see how there is a problem with updating the global RegBanks array? Likewise the HwMode is constant for the subtarget, so the write will be of the same value.

[bkramer]

In D76007#1944582, @lewis-revill wrote:

I'm lacking an understanding of how concurrency is expected to work within LLVM, but since there is only one RegisterBankInfo constructed per subtarget I don't see how there is a problem with updating the global RegBanks array? Likewise the HwMode is constant for the subtarget, so the write will be of the same value.

There can be many subtargets around, and those can be created concurrently. Having state in a global variable is not going to work.

[lewis-revill]

Rewrote the global RegBanks array to be constant. Instead of modifying RegisterBanks after construction, provide HwMode to the constructor of RegisterBankInfo and move size calculation logic for RegisterBanks to that class.

[lewis-revill]

Rebased.

[lewis-revill]

Rebased.

[arsenm]

I'm not sure I understand why the RegisterBankInfo needs to be aware of the hwmode. HWmode changes the size of the classes during selection, but the registerbankinfo just needs to assign sizes to banks. The concrete classes don't matter so much

[lewis-revill]

In D76007#3249742, @arsenm wrote:

I'm not sure I understand why the RegisterBankInfo needs to be aware of the hwmode. HWmode changes the size of the classes during selection, but the registerbankinfo just needs to assign sizes to banks. The concrete classes don't matter so much

Unsure if I understood your comment! The patch itself was necessary since the assignment of a single size to each register bank is wrong for targets with different register sizes for different hardware modes. If the RegisterBankInfo is aware of the hardware mode then we can query what the size of a given register bank actually is for the current hardware mode. The register bank array itself is also constructed once by TableGen so we can't change sizes when they're constructed. And it isn't safe to change the size after construction.

[lewis-revill]

Bump - is anyone able to review this?

[arsenm]

In D76007#3287539, @lewis-revill wrote:

In D76007#3249742, @arsenm wrote:

I'm not sure I understand why the RegisterBankInfo needs to be aware of the hwmode. HWmode changes the size of the classes during selection, but the registerbankinfo just needs to assign sizes to banks. The concrete classes don't matter so much

Unsure if I understood your comment! The patch itself was necessary since the assignment of a single size to each register bank is wrong for targets with different register sizes for different hardware modes. If the RegisterBankInfo is aware of the hardware mode then we can query what the size of a given register bank actually is for the current hardware mode. The register bank array itself is also constructed once by TableGen so we can't change sizes when they're constructed. And it isn't safe to change the size after construction.

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

[lewis-revill]

In D76007#3308144, @arsenm wrote:

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

Right. But before this patch there was a max size associated to the bank itself, which is queried by the verifier. And if we want to keep that concept of a maximum size surely we should have it be accurate? So for different hardware modes it should be possible for it to be different.

[arsenm]

In D76007#3310559, @lewis-revill wrote:

In D76007#3308144, @arsenm wrote:

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

Right. But before this patch there was a max size associated to the bank itself, which is queried by the verifier. And if we want to keep that concept of a maximum size surely we should have it be accurate? So for different hardware modes it should be possible for it to be different.

Why is there a maximum size concept? What is it used for? I haven't noticed thisb efore

[arsenm]

In D76007#3412653, @arsenm wrote:

In D76007#3310559, @lewis-revill wrote:

In D76007#3308144, @arsenm wrote:

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

Right. But before this patch there was a max size associated to the bank itself, which is queried by the verifier. And if we want to keep that concept of a maximum size surely we should have it be accurate? So for different hardware modes it should be possible for it to be different.

Why is there a maximum size concept? What is it used for? I haven't noticed thisb efore

If it is a maximum, why can't you set to the maximum possible size?

[lewis-revill]

In D76007#3412660, @arsenm wrote:

In D76007#3412653, @arsenm wrote:

In D76007#3310559, @lewis-revill wrote:

In D76007#3308144, @arsenm wrote:

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

Right. But before this patch there was a max size associated to the bank itself, which is queried by the verifier. And if we want to keep that concept of a maximum size surely we should have it be accurate? So for different hardware modes it should be possible for it to be different.

Why is there a maximum size concept? What is it used for? I haven't noticed thisb efore

If it is a maximum, why can't you set to the maximum possible size?

Because this size isn't 'set' by a target, it's determined by tablegen to be the maximum size of the register class covered by the register bank. So for the RISC-V GPR banks that size will always depend on hardware mode, and as such the bank size should also reflect that - that's what this patch addresses.

[lewis-revill]

In D76007#3412653, @arsenm wrote:

In D76007#3310559, @lewis-revill wrote:

In D76007#3308144, @arsenm wrote:

Register banks do not have sizes. A register bank mapping is a bank plus a size/offset that you can change based on the target's size, not part of the bank itself.

Right. But before this patch there was a max size associated to the bank itself, which is queried by the verifier. And if we want to keep that concept of a maximum size surely we should have it be accurate? So for different hardware modes it should be possible for it to be different.

Why is there a maximum size concept? What is it used for? I haven't noticed thisb efore

It's purely used for the verifier (and for similar assertions about type sizes in backends).

[arsenm]

llvm/include/llvm/CodeGen/GlobalISel/RegisterBankInfo.h

587

getMaximumSize?

588

Braces

[nitinjohnraj]

Rebased

[craig.topper]

llvm/include/llvm/CodeGen/GlobalISel/RegisterBankInfo.h

588

Can we get the ID from the RegisterBank object instead of searching?

[nitinjohnraj]

Updating getSize to getMaximumSize and using the register bank ID

[nitinjohnraj]

Use DefaultMode instead of 0.

[nitinjohnraj]

llvm/utils/TableGen/RegisterBankEmitter.cpp

270

Use predefined constant DefaultMode

[nitinjohnraj]

This was approved a long while ago, could I get a review to confirm that it's good to go?

[craig.topper]

llvm/include/llvm/CodeGen/RegisterBankInfo.h

589 ↗

(On Diff #527686)

Do we need a loop at al

Can it just be return Sizes[RegBankID + HwMode * NumRegBanks];

[nitinjohnraj]

Removed loop in getMaximumSize.

[craig.topper]

LGTM

llvm/include/llvm/CodeGen/GlobalISel/RegisterBank.h

 
 /// This class implements the register bank concept.
 /// Two instances of RegisterBank must have different ID.
 /// This property is enforced by the RegisterBankInfo class.
 class RegisterBank {
 private:
   unsigned ID;
   const char *Name;
-  unsigned Size;
   BitVector ContainedRegClasses;
 
   /// Sentinel value used to recognize register bank not properly
   /// initialized yet.
   static const unsigned InvalidID;
 
   /// Only the RegisterBankInfo can initialize RegisterBank properly.
   friend RegisterBankInfo;
 
 public:
-  RegisterBank(unsigned ID, const char *Name, unsigned Size,
-               const uint32_t *CoveredClasses, unsigned NumRegClasses);
+  RegisterBank(unsigned ID, const char *Name, const uint32_t *CoveredClasses,
+               unsigned NumRegClasses);
 
   /// Get the identifier of this register bank.
   unsigned getID() const { return ID; }
 
   /// Get a user friendly name of this register bank.
   /// Should be used only for debugging purposes.
   const char *getName() const { return Name; }
 
-  /// Get the maximal size in bits that fits in this register bank.
-  unsigned getSize() const { return Size; }
-
   /// Check whether this instance is ready to be used.
   bool isValid() const;
 
   /// Check if this register bank is valid. In other words,
   /// if it has been properly constructed.
   ///
   /// \note This method does not check anything when assertions are disabled.
   ///
   /// \return True is the check was successful.
-  bool verify(const TargetRegisterInfo &TRI) const;
+  bool verify(const RegisterBankInfo &RBI, const TargetRegisterInfo &TRI) const;
 
   /// Check whether this register bank covers \p RC.
   /// In other words, check if this register bank fully covers
   /// the registers that \p RC contains.
   /// \pre isValid()
   bool covers(const TargetRegisterClass &RC) const;
 
   /// Check whether \p OtherRB is the same as this.

llvm/include/llvm/CodeGen/GlobalISel/RegisterBankInfo.h

struct PartialMapping {
 
     /// Check that the Mask is compatible with the RegBank.
     /// Indeed, if the RegBank cannot accomadate the "active bits" of the mask,
     /// there is no way this mapping is valid.
     ///
     /// \note This method does not check anything when assertions are disabled.
     ///
     /// \return True is the check was successful.
-    bool verify() const;
+    bool verify(const RegisterBankInfo &RBI) const;
   };
 
   /// Helper struct that represents how a value is mapped through
   /// different register banks.
   ///
   /// \note: So far we do not have any users of the complex mappings
   /// (mappings with more than one partial mapping), but when we do,
   /// we would have needed to duplicate partial mappings.
struct ValueMapping {
     /// Check if this ValueMapping is valid.
     bool isValid() const { return BreakDown && NumBreakDowns; }
 
     /// Verify that this mapping makes sense for a value of
     /// \p MeaningfulBitWidth.
     /// \note This method does not check anything when assertions are disabled.
     ///
     /// \return True is the check was successful.
-    bool verify(unsigned MeaningfulBitWidth) const;
+    bool verify(const RegisterBankInfo &RBI, unsigned MeaningfulBitWidth) const;
 
     /// Print this on dbgs() stream.
     void dump() const;
 
     /// Print this on \p OS;
     void print(raw_ostream &OS) const;
   };
 
public:
     void dump() const;
 
     /// Print this operands mapper on \p OS stream.
     void print(raw_ostream &OS, bool ForDebug = false) const;
   };
 
 protected:
   /// Hold the set of supported register banks.
-  RegisterBank **RegBanks;
+  const RegisterBank **RegBanks;
 
   /// Total number of register banks.
   unsigned NumRegBanks;
 
+  /// Hold the sizes of the register banks for all HwModes.
+  const unsigned *Sizes;
+
+  /// Current HwMode for the target.
+  unsigned HwMode;
+
   /// Keep dynamically allocated PartialMapping in a separate map.
   /// This shouldn't be needed when everything gets TableGen'ed.
   mutable DenseMap<unsigned, std::unique_ptr<const PartialMapping>>
       MapOfPartialMappings;
 
   /// Keep dynamically allocated ValueMapping in a separate map.
   /// This shouldn't be needed when everything gets TableGen'ed.
   mutable DenseMap<unsigned, std::unique_ptr<const ValueMapping>>
mutable DenseMap<unsigned, std::unique_ptr<const InstructionMapping>>
       MapOfInstructionMappings;
 
   /// Getting the minimal register class of a physreg is expensive.
   /// Cache this information as we get it.
   mutable DenseMap<unsigned, const TargetRegisterClass *> PhysRegMinimalRCs;
 
   /// Create a RegisterBankInfo that can accommodate up to \p NumRegBanks
   /// RegisterBank instances.
-  RegisterBankInfo(RegisterBank **RegBanks, unsigned NumRegBanks);
+  RegisterBankInfo(const RegisterBank **RegBanks, unsigned NumRegBanks,
+                   const unsigned *Sizes, unsigned HwMode);
 
   /// This constructor is meaningless.
   /// It just provides a default constructor that can be used at link time
   /// when GlobalISel is not built.
   /// That way, targets can still inherit from this class without doing
   /// crazy gymnastic to avoid link time failures.
   /// \note That works because the constructor is inlined.
   RegisterBankInfo() {
     llvm_unreachable("This constructor should not be executed");
   }
 
   /// Get the register bank identified by \p ID.
-  RegisterBank &getRegBank(unsigned ID) {
+  const RegisterBank &getRegBank(unsigned ID) {
     assert(ID < getNumRegBanks() && "Accessing an unknown register bank");
     return *RegBanks[ID];
   }
 
   /// Get the MinimalPhysRegClass for Reg.
   /// \pre Reg is a physical register.
   const TargetRegisterClass &
   getMinimalPhysRegClass(Register Reg, const TargetRegisterInfo &TRI) const;

 public:
   virtual ~RegisterBankInfo() = default;
 
   /// Get the register bank identified by \p ID.
   const RegisterBank &getRegBank(unsigned ID) const {
     return const_cast<RegisterBankInfo *>(this)->getRegBank(ID);
   }
 
+  /// Get the maximal size in bits that fits in the given register bank.
+  unsigned getSize(const RegisterBank *RegBank) const {
+    for (size_t Index = 0; Index < NumRegBanks; ++Index)
+      if (RegBanks[Index] == RegBank)
+        return Sizes[Index + HwMode * NumRegBanks];
+
+    assert(false && "Could not get size for unknown register bank");
+  }
+
   /// Get the register bank of \p Reg.
   /// If Reg has not been assigned a register, a register class,
   /// or a register bank, then this returns nullptr.
   ///
   /// \pre Reg != 0 (NoRegister)
   const RegisterBank *getRegBank(Register Reg, const MachineRegisterInfo &MRI,
                                  const TargetRegisterInfo &TRI) const;
 

llvm/lib/CodeGen/GlobalISel/RegisterBank.cpp

 //===- llvm/CodeGen/GlobalISel/RegisterBank.cpp - Register Bank --*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 /// \file
 /// This file implements the RegisterBank class.
 //===----------------------------------------------------------------------===//
 
 #include "llvm/CodeGen/GlobalISel/RegisterBank.h"
+#include "llvm/CodeGen/GlobalISel/RegisterBankInfo.h"
 #include "llvm/CodeGen/TargetRegisterInfo.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/Support/Debug.h"
 
 #define DEBUG_TYPE "registerbank"
 
 using namespace llvm;
 
 const unsigned RegisterBank::InvalidID = UINT_MAX;
 
-RegisterBank::RegisterBank(
-    unsigned ID, const char *Name, unsigned Size,
-    const uint32_t *CoveredClasses, unsigned NumRegClasses)
-    : ID(ID), Name(Name), Size(Size) {
+RegisterBank::RegisterBank(unsigned ID, const char *Name,
+                           const uint32_t *CoveredClasses,
+                           unsigned NumRegClasses)
+    : ID(ID), Name(Name) {
   ContainedRegClasses.resize(NumRegClasses);
   ContainedRegClasses.setBitsInMask(CoveredClasses);
 }
 
-bool RegisterBank::verify(const TargetRegisterInfo &TRI) const {
+bool RegisterBank::verify(const RegisterBankInfo &RBI,
+                          const TargetRegisterInfo &TRI) const {
   assert(isValid() && "Invalid register bank");
   for (unsigned RCId = 0, End = TRI.getNumRegClasses(); RCId != End; ++RCId) {
     const TargetRegisterClass &RC = *TRI.getRegClass(RCId);
 
     if (!covers(RC))
       continue;
     // Verify that the register bank covers all the sub classes of the
     // classes it covers.
 
     // Use a different (slow in that case) method than
     // RegisterBankInfo to find the subclasses of RC, to make sure
     // both agree on the covers.
     for (unsigned SubRCId = 0; SubRCId != End; ++SubRCId) {
       const TargetRegisterClass &SubRC = *TRI.getRegClass(RCId);
 
       if (!RC.hasSubClassEq(&SubRC))
         continue;
 
       // Verify that the Size of the register bank is big enough to cover
       // all the register classes it covers.
-      assert(getSize() >= TRI.getRegSizeInBits(SubRC) &&
+      assert(RBI.getSize(this) >= TRI.getRegSizeInBits(SubRC) &&
              "Size is not big enough for all the subclasses!");
       assert(covers(SubRC) && "Not all subclasses are covered");
     }
   }
   return true;
 }
 
 bool RegisterBank::covers(const TargetRegisterClass &RC) const {
   assert(isValid() && "RB hasn't been initialized yet");
   return ContainedRegClasses.test(RC.getID());
 }
 
 bool RegisterBank::isValid() const {
-  return ID != InvalidID && Name != nullptr && Size != 0 &&
+  return ID != InvalidID && Name != nullptr &&
          // A register bank that does not cover anything is useless.
          !ContainedRegClasses.empty();
 }
 
 bool RegisterBank::operator==(const RegisterBank &OtherRB) const {
   // There must be only one instance of a given register bank alive
   // for the whole compilation.
   // The RegisterBankInfo is supposed to enforce that.
   assert((OtherRB.getID() != getID() || &OtherRB == this) &&
          "ID does not uniquely identify a RegisterBank");
   return &OtherRB == this;
 }
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void RegisterBank::dump(const TargetRegisterInfo *TRI) const {
   print(dbgs(), /* IsForDebug */ true, TRI);
 }
 #endif
 
 void RegisterBank::print(raw_ostream &OS, bool IsForDebug,
                          const TargetRegisterInfo *TRI) const {
   OS << getName();
   if (!IsForDebug)
     return;
-  OS << "(ID:" << getID() << ", Size:" << getSize() << ")\n"
+  OS << "(ID:" << getID() << ")\n"
      << "isValid:" << isValid() << '\n'
      << "Number of Covered register classes: " << ContainedRegClasses.count()
      << '\n';
   // Print all the subclasses if we can.
   // This register classes may not be properly initialized yet.
   if (!TRI || ContainedRegClasses.empty())
     return;
   assert(ContainedRegClasses.size() == TRI->getNumRegClasses() &&

llvm/lib/CodeGen/GlobalISel/RegisterBankInfo.cpp

STATISTIC(NumInstructionMappingsAccessed,
           "Number of instruction mappings dynamically accessed");
 
 const unsigned RegisterBankInfo::DefaultMappingID = UINT_MAX;
 const unsigned RegisterBankInfo::InvalidMappingID = UINT_MAX - 1;
 
 //------------------------------------------------------------------------------
 // RegisterBankInfo implementation.
 //------------------------------------------------------------------------------
-RegisterBankInfo::RegisterBankInfo(RegisterBank **RegBanks,
-                                   unsigned NumRegBanks)
-    : RegBanks(RegBanks), NumRegBanks(NumRegBanks) {
+RegisterBankInfo::RegisterBankInfo(const RegisterBank **RegBanks,
+                                   unsigned NumRegBanks, const unsigned *Sizes,
+                                   unsigned HwMode)
+    : RegBanks(RegBanks), NumRegBanks(NumRegBanks), Sizes(Sizes),
+      HwMode(HwMode) {
 #ifndef NDEBUG
   for (unsigned Idx = 0, End = getNumRegBanks(); Idx != End; ++Idx) {
     assert(RegBanks[Idx] != nullptr && "Invalid RegisterBank");
     assert(RegBanks[Idx]->isValid() && "RegisterBank should be valid");
   }
 #endif // NDEBUG
 }
 
 bool RegisterBankInfo::verify(const TargetRegisterInfo &TRI) const {
 #ifndef NDEBUG
   for (unsigned Idx = 0, End = getNumRegBanks(); Idx != End; ++Idx) {
     const RegisterBank &RegBank = getRegBank(Idx);
     assert(Idx == RegBank.getID() &&
            "ID does not match the index in the array");
     LLVM_DEBUG(dbgs() << "Verify " << RegBank << '\n');
-    assert(RegBank.verify(TRI) && "RegBank is invalid");
+    assert(RegBank.verify(*this, TRI) && "RegBank is invalid");
   }
 #endif // NDEBUG
   return true;
 }
 
 const RegisterBank *
 RegisterBankInfo::getRegBank(Register Reg, const MachineRegisterInfo &MRI,
                              const TargetRegisterInfo &TRI) const {

 //------------------------------------------------------------------------------
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void RegisterBankInfo::PartialMapping::dump() const {
   print(dbgs());
   dbgs() << '\n';
 }
 #endif
 
-bool RegisterBankInfo::PartialMapping::verify() const {
+bool RegisterBankInfo::PartialMapping::verify(
+    const RegisterBankInfo &RBI) const {
   assert(RegBank && "Register bank not set");
   assert(Length && "Empty mapping");
   assert((StartIdx <= getHighBitIdx()) && "Overflow, switch to APInt?");
   // Check if the minimum width fits into RegBank.
-  assert(RegBank->getSize() >= Length && "Register bank too small for Mask");
+  assert(RBI.getSize(RegBank) >= Length && "Register bank too small for Mask");
   return true;
 }
 
 void RegisterBankInfo::PartialMapping::print(raw_ostream &OS) const {
   OS << "[" << StartIdx << ", " << getHighBitIdx() << "], RegBank = ";
   if (RegBank)
     OS << *RegBank;
   else
     OS << "nullptr";
 }
 
 bool RegisterBankInfo::ValueMapping::partsAllUniform() const {
   if (NumBreakDowns < 2)
     return true;
 
   const PartialMapping *First = begin();
   for (const PartialMapping *Part = First + 1; Part != end(); ++Part) {
     if (Part->Length != First->Length || Part->RegBank != First->RegBank)
       return false;
   }
 
   return true;
 }
 
-bool RegisterBankInfo::ValueMapping::verify(unsigned MeaningfulBitWidth) const {
+bool RegisterBankInfo::ValueMapping::verify(const RegisterBankInfo &RBI,
+                                            unsigned MeaningfulBitWidth) const {
   assert(NumBreakDowns && "Value mapped nowhere?!");
   unsigned OrigValueBitWidth = 0;
   for (const RegisterBankInfo::PartialMapping &PartMap : *this) {
     // Check that each register bank is big enough to hold the partial value:
     // this check is done by PartialMapping::verify
-    assert(PartMap.verify() && "Partial mapping is invalid");
+    assert(PartMap.verify(RBI) && "Partial mapping is invalid");
     // The original value should completely be mapped.
     // Thus the maximum accessed index + 1 is the size of the original value.
     OrigValueBitWidth =
         std::max(OrigValueBitWidth, PartMap.getHighBitIdx() + 1);
   }
   assert(OrigValueBitWidth >= MeaningfulBitWidth &&
          "Meaningful bits not covered by the mapping");
   APInt ValueMask(OrigValueBitWidth, 0);
for (unsigned Idx = 0; Idx < NumOperands; ++Idx) {
     if (!Reg)
       continue;
     assert(getOperandMapping(Idx).isValid() &&
            "We must have a mapping for reg operands");
     const RegisterBankInfo::ValueMapping &MOMapping = getOperandMapping(Idx);
     (void)MOMapping;
     // Register size in bits.
     // This size must match what the mapping expects.
-    assert(MOMapping.verify(RBI->getSizeInBits(
-               Reg, MF.getRegInfo(), *MF.getSubtarget().getRegisterInfo())) &&
+    assert(MOMapping.verify(*RBI, RBI->getSizeInBits(
+                                      Reg, MF.getRegInfo(),
+                                      *MF.getSubtarget().getRegisterInfo())) &&
            "Value mapping is invalid");
   }
   return true;
 }
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void RegisterBankInfo::InstructionMapping::dump() const {
   print(dbgs());

llvm/lib/CodeGen/MachineVerifier.cpp

 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SetOperations.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Analysis/EHPersonalities.h"
 #include "llvm/CodeGen/GlobalISel/RegisterBank.h"
+#include "llvm/CodeGen/GlobalISel/RegisterBankInfo.h"
 #include "llvm/CodeGen/LiveInterval.h"
 #include "llvm/CodeGen/LiveIntervals.h"
 #include "llvm/CodeGen/LiveRangeCalc.h"
 #include "llvm/CodeGen/LiveStacks.h"
 #include "llvm/CodeGen/LiveVariables.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineFunction.h"
if (Register::isPhysicalRegister(Reg)) {
         LLT Ty = MRI->getType(Reg);
         if (!Ty.isValid()) {
           report("Generic virtual register must have a valid type", MO,
                  MONum);
           return;
         }
 
         const RegisterBank *RegBank = MRI->getRegBankOrNull(Reg);
+        const RegisterBankInfo *RBI = MF->getSubtarget().getRegBankInfo();
 
         // If we're post-RegBankSelect, the gvreg must have a bank.
         if (!RegBank && isFunctionRegBankSelected) {
           report("Generic virtual register must have a bank in a "
                  "RegBankSelected function",
                  MO, MONum);
           return;
         }
 
         // Make sure the register fits into its register bank if any.
         if (RegBank && Ty.isValid() &&
-            RegBank->getSize() < Ty.getSizeInBits()) {
+            RBI->getSize(RegBank) < Ty.getSizeInBits()) {
           report("Register bank is too small for virtual register", MO,
                  MONum);
           errs() << "Register bank " << RegBank->getName() << " too small("
-                 << RegBank->getSize() << ") to fit " << Ty.getSizeInBits()
+                 << RBI->getSize(RegBank) << ") to fit " << Ty.getSizeInBits()
                  << "-bits\n";
           return;
         }
         if (SubIdx)  {
           report("Generic virtual register does not allow subregister index", MO,
                  MONum);
           return;
         }

llvm/lib/Target/AArch64/AArch64RegisterBankInfo.cpp

static auto InitializeRegisterBankOnce = [&]() {
     (void)RBCCR;
     assert(&AArch64::CCRegBank == &RBCCR &&
            "The order in RegBanks is messed up");
 
     // The GPR register bank is fully defined by all the registers in
     // GR64all + its subclasses.
     assert(RBGPR.covers(*TRI.getRegClass(AArch64::GPR32RegClassID)) &&
            "Subclass not added?");
-    assert(RBGPR.getSize() == 64 && "GPRs should hold up to 64-bit");
+    assert(getSize(&RBGPR) == 64 && "GPRs should hold up to 64-bit");
 
     // The FPR register bank is fully defined by all the registers in
     // GR64all + its subclasses.
     assert(RBFPR.covers(*TRI.getRegClass(AArch64::QQRegClassID)) &&
            "Subclass not added?");
     assert(RBFPR.covers(*TRI.getRegClass(AArch64::FPR64RegClassID)) &&
            "Subclass not added?");
-    assert(RBFPR.getSize() == 512 &&
+    assert(getSize(&RBFPR) == 512 &&
            "FPRs should hold up to 512-bit via QQQQ sequence");
 
     assert(RBCCR.covers(*TRI.getRegClass(AArch64::CCRRegClassID)) &&
            "Class not added?");
-    assert(RBCCR.getSize() == 32 && "CCR should hold up to 32-bit");
+    assert(getSize(&RBCCR) == 32 && "CCR should hold up to 32-bit");
 
     // Check that the TableGen'ed like file is in sync we our expectations.
     // First, the Idx.
     assert(checkPartialMappingIdx(PMI_FirstGPR, PMI_LastGPR,
                                   {PMI_GPR32, PMI_GPR64}) &&
            "PartialMappingIdx's are incorrectly ordered");
     assert(checkPartialMappingIdx(PMI_FirstFPR, PMI_LastFPR,
                                   {PMI_FPR16, PMI_FPR32, PMI_FPR64, PMI_FPR128,

llvm/lib/Target/ARM/ARMRegisterBankInfo.cpp

assert(RBGPR.covers(*TRI.getRegClass(ARM::tcGPRRegClassID)) &&
            "Subclass not added?");
     assert(RBGPR.covers(*TRI.getRegClass(ARM::tGPR_and_tcGPRRegClassID)) &&
            "Subclass not added?");
     assert(RBGPR.covers(
                *TRI.getRegClass(ARM::tGPREven_and_tGPR_and_tcGPRRegClassID)) &&
            "Subclass not added?");
     assert(RBGPR.covers(*TRI.getRegClass(ARM::tGPROdd_and_tcGPRRegClassID)) &&
            "Subclass not added?");
-    assert(RBGPR.getSize() == 32 && "GPRs should hold up to 32-bit");
+    assert(getSize(&RBGPR) == 32 && "GPRs should hold up to 32-bit");
 
 #ifndef NDEBUG
     ARM::checkPartialMappings();
     ARM::checkValueMappings();
 #endif
   };
 
   llvm::call_once(InitializeRegisterBankFlag, InitializeRegisterBankOnce);

llvm/lib/Target/RISCV/RISCVRegisterBankInfo.h

 protected:
 #define GET_TARGET_REGBANK_CLASS
 #include "RISCVGenRegisterBank.inc"
 };
 
 /// This class provides the information for the target register banks.
 class RISCVRegisterBankInfo final : public RISCVGenRegisterBankInfo {
 public:
-  RISCVRegisterBankInfo(const TargetRegisterInfo &TRI);
+  RISCVRegisterBankInfo(unsigned HwMode);
 };
 } // end namespace llvm
 #endif

llvm/lib/Target/RISCV/RISCVRegisterBankInfo.cpp

 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/TargetRegisterInfo.h"
 
 #define GET_TARGET_REGBANK_IMPL
 #include "RISCVGenRegisterBank.inc"
 
 using namespace llvm;
 
-RISCVRegisterBankInfo::RISCVRegisterBankInfo(const TargetRegisterInfo &TRI)
-    : RISCVGenRegisterBankInfo() {}
+RISCVRegisterBankInfo::RISCVRegisterBankInfo(unsigned HwMode)
+    : RISCVGenRegisterBankInfo(HwMode) {}

llvm/lib/Target/RISCV/RISCVSubtarget.cpp

RISCVSubtarget::RISCVSubtarget(const Triple &TT, StringRef CPU, StringRef FS,
                                StringRef ABIName, const TargetMachine &TM)
     : RISCVGenSubtargetInfo(TT, CPU, FS),
       UserReservedRegister(RISCV::NUM_TARGET_REGS),
       FrameLowering(initializeSubtargetDependencies(TT, CPU, FS, ABIName)),
       InstrInfo(*this), RegInfo(getHwMode()), TLInfo(TM, *this) {
   CallLoweringInfo.reset(new RISCVCallLowering(*getTargetLowering()));
   Legalizer.reset(new RISCVLegalizerInfo(*this));
 
-  auto *RBI = new RISCVRegisterBankInfo(*getRegisterInfo());
+  auto *RBI = new RISCVRegisterBankInfo(getHwMode());
   RegBankInfo.reset(RBI);
   InstSelector.reset(createRISCVInstructionSelector(
       *static_cast<const RISCVTargetMachine *>(&TM), *this, *RBI));
 }
 
 const CallLowering *RISCVSubtarget::getCallLowering() const {
   return CallLoweringInfo.get();
 }

llvm/lib/Target/X86/X86RegisterBankInfo.cpp

X86RegisterBankInfo::X86RegisterBankInfo(const TargetRegisterInfo &TRI)
   const RegisterBank &RBGPR = getRegBank(X86::GPRRegBankID);
   (void)RBGPR;
   assert(&X86::GPRRegBank == &RBGPR && "Incorrect RegBanks inizalization.");
 
   // The GPR register bank is fully defined by all the registers in
   // GR64 + its subclasses.
   assert(RBGPR.covers(*TRI.getRegClass(X86::GR64RegClassID)) &&
          "Subclass not added?");
-  assert(RBGPR.getSize() == 64 && "GPRs should hold up to 64-bit");
+  assert(getSize(&RBGPR) == 64 && "GPRs should hold up to 64-bit");
 }
 
 const RegisterBank &
 X86RegisterBankInfo::getRegBankFromRegClass(const TargetRegisterClass &RC,
                                             LLT) const {
 
   if (X86::GR8RegClass.hasSubClassEq(&RC) ||
       X86::GR16RegClass.hasSubClassEq(&RC) ||

llvm/utils/TableGen/RegisterBankEmitter.cpp

class RegisterBank {
   typedef std::vector<const CodeGenRegisterClass *> RegisterClassesTy;
 
 private:
   const Record &TheDef;
 
   /// The register classes that are covered by the register bank.
   RegisterClassesTy RCs;
 
-  /// The register class with the largest register size.
-  const CodeGenRegisterClass *RCWithLargestRegsSize;
+  /// The register classes with the largest register size for each HwMode.
+  std::vector<const CodeGenRegisterClass *> RCsWithLargestRegSize;
 
 public:
-  RegisterBank(const Record &TheDef)
-      : TheDef(TheDef), RCs(), RCWithLargestRegsSize(nullptr) {}
+  RegisterBank(const Record &TheDef, unsigned NumModeIds)
+      : TheDef(TheDef), RCs(), RCsWithLargestRegSize(NumModeIds) {}
 
   /// Get the human-readable name for the bank.
   StringRef getName() const { return TheDef.getValueAsString("Name"); }
   /// Get the name of the enumerator in the ID enumeration.
   std::string getEnumeratorName() const { return (TheDef.getName() + "ID").str(); }
 
   /// Get the name of the array holding the register class coverage data;
   std::string getCoverageArrayName() const {
if (std::find_if(RCs.begin(), RCs.end(),
       return;
 
     // FIXME? We really want the register size rather than the spill size
     //        since the spill size may be bigger on some targets with
     //        limited load/store instructions. However, we don't store the
     //        register size anywhere (we could sum the sizes of the subregisters
     //        but there may be additional bits too) and we can't derive it from
     //        the VT's reliably due to Untyped.
-    if (RCWithLargestRegsSize == nullptr)
-      RCWithLargestRegsSize = RC;
-    else if (RCWithLargestRegsSize->RSI.get(DefaultMode).SpillSize <
-             RC->RSI.get(DefaultMode).SpillSize)
-      RCWithLargestRegsSize = RC;
-    assert(RCWithLargestRegsSize && "RC was nullptr?");
+    unsigned NumModeIds = RCsWithLargestRegSize.size();
+    for (unsigned M = 0; M < NumModeIds; ++M) {
+      if (RCsWithLargestRegSize[M] == nullptr)
+        RCsWithLargestRegSize[M] = RC;
+      else if (RCsWithLargestRegSize[M]->RSI.get(M).SpillSize <
+               RC->RSI.get(M).SpillSize)
+        RCsWithLargestRegSize[M] = RC;
+      assert(RCsWithLargestRegSize[M] && "RC was nullptr?");
+    }
 
     RCs.emplace_back(RC);
   }
 
-  const CodeGenRegisterClass *getRCWithLargestRegsSize() const {
-    return RCWithLargestRegsSize;
+  const CodeGenRegisterClass *getRCWithLargestRegSize(unsigned HwMode) const {
+    return RCsWithLargestRegSize[HwMode];
   }
 
   iterator_range<typename RegisterClassesTy::const_iterator>
   register_classes() const {
     return llvm::make_range(RCs.begin(), RCs.end());
   }
 };
 
OS << "  NumRegisterBanks,\n"
      << "} // end namespace llvm\n";
 }
 
 /// Emit declarations of the <Target>GenRegisterBankInfo class.
 void RegisterBankEmitter::emitBaseClassDefinition(
     raw_ostream &OS, const StringRef TargetName,
     const std::vector<RegisterBank> &Banks) {
   OS << "private:\n"
-     << "  static RegisterBank *RegBanks[];\n\n"
+     << "  static const RegisterBank *RegBanks[];\n"
+     << "  static const unsigned Sizes[];\n\n"
      << "protected:\n"
-     << "  " << TargetName << "GenRegisterBankInfo();\n"
+     << "  " << TargetName << "GenRegisterBankInfo(unsigned HwMode = 0);\n"
      << "\n";
 }
 
 /// Visit each register class belonging to the given register bank.
 ///
 /// A class belongs to the bank iff any of these apply:
 /// * It is explicitly specified
 /// * It is a subclass of a class that is a member.
for (const auto &PossibleSubclass : RegisterClassHierarchy.getRegClasses()) {
     }
   }
 }
 
 void RegisterBankEmitter::emitBaseClassImplementation(
     raw_ostream &OS, StringRef TargetName,
     std::vector<RegisterBank> &Banks) {
   const CodeGenRegBank &RegisterClassHierarchy = Target.getRegBank();
+  const CodeGenHwModes &CGH = Target.getHwModes();
 
   OS << "namespace llvm {\n"
      << "namespace " << TargetName << " {\n";
   for (const auto &Bank : Banks) {
     std::vector<std::vector<const CodeGenRegisterClass *>> RCsGroupedByWord(
         (RegisterClassHierarchy.getRegClasses().size() + 31) / 32);
 
     for (const auto &RC : Bank.register_classes())
for (const auto &Bank : Banks) {
     }
     OS << "};\n";
   }
   OS << "\n";
 
   for (const auto &Bank : Banks) {
     std::string QualifiedBankID =
         (TargetName + "::" + Bank.getEnumeratorName()).str();
-    const CodeGenRegisterClass &RC = *Bank.getRCWithLargestRegsSize();
-    unsigned Size = RC.RSI.get(DefaultMode).SpillSize;
-    OS << "RegisterBank " << Bank.getInstanceVarName() << "(/* ID */ "
-       << QualifiedBankID << ", /* Name */ \"" << Bank.getName()
-       << "\", /* Size */ " << Size << ", "
+    OS << "const RegisterBank " << Bank.getInstanceVarName() << "(/* ID */ "
+       << QualifiedBankID << ", /* Name */ \"" << Bank.getName() << "\", "
        << "/* CoveredRegClasses */ " << Bank.getCoverageArrayName()
        << ", /* NumRegClasses */ "
        << RegisterClassHierarchy.getRegClasses().size() << ");\n";
   }
   OS << "} // end namespace " << TargetName << "\n"
      << "\n";
 
-  OS << "RegisterBank *" << TargetName
+  OS << "const RegisterBank *" << TargetName
      << "GenRegisterBankInfo::RegBanks[] = {\n";
   for (const auto &Bank : Banks)
     OS << "    &" << TargetName << "::" << Bank.getInstanceVarName() << ",\n";
   OS << "};\n\n";
 
+  unsigned NumModeIds = CGH.getNumModeIds();
+  OS << "const unsigned " << TargetName << "GenRegisterBankInfo::Sizes[] = {\n";
+  for (unsigned M = 0; M < NumModeIds; ++M) {
+    OS << "    // Mode = " << M << " (";
+    if (M == 0)
+      OS << "Default";
+    else
+      OS << CGH.getMode(M).Name;
+    OS << ")\n";
+    for (const auto &Bank : Banks) {
+      const CodeGenRegisterClass &RC = *Bank.getRCWithLargestRegSize(M);
+      unsigned Size = RC.RSI.get(M).SpillSize;
+      OS << "    " << Size << ",\n";
+    }
+  }
+  OS << "};\n\n";
+
   OS << TargetName << "GenRegisterBankInfo::" << TargetName
-     << "GenRegisterBankInfo()\n"
+     << "GenRegisterBankInfo(unsigned HwMode)\n"
      << "    : RegisterBankInfo(RegBanks, " << TargetName
-     << "::NumRegisterBanks) {\n"
+     << "::NumRegisterBanks, Sizes, HwMode) {\n"
      << "  // Assert that RegBank indices match their ID's\n"
      << "#ifndef NDEBUG\n"
      << "  unsigned Index = 0;\n"
      << "  for (const auto &RB : RegBanks)\n"
      << "    assert(Index++ == RB->getID() && \"Index != ID\");\n"
      << "#endif // NDEBUG\n"
      << "}\n"
      << "} // end namespace llvm\n";
 }
 
 void RegisterBankEmitter::run(raw_ostream &OS) {
   StringRef TargetName = Target.getName();
   const CodeGenRegBank &RegisterClassHierarchy = Target.getRegBank();
+  const CodeGenHwModes &CGH = Target.getHwModes();
 
   std::vector<RegisterBank> Banks;
   for (const auto &V : Records.getAllDerivedDefinitions("RegisterBank")) {
     SmallPtrSet<const CodeGenRegisterClass *, 8> VisitedRCs;
-    RegisterBank Bank(*V);
+    RegisterBank Bank(*V, CGH.getNumModeIds());
 
     for (const CodeGenRegisterClass *RC :
          Bank.getExplicitlySpecifiedRegisterClasses(RegisterClassHierarchy)) {
       visitRegisterBankClasses(
           RegisterClassHierarchy, RC, "explicit",
           [&Bank](const CodeGenRegisterClass *RC, StringRef Kind) {
             LLVM_DEBUG(dbgs()
                        << "Added " << RC->getName() << "(" << Kind << ")\n");