[llvm-mca] Refactor class RegisterFile to allow the definition of multiple register files.

[andreadb]

Summary

This is a refactoring in preparation for other two changes that will allow scheduling models to define multiple register files. This is the first step towards fixing PR36662.

class RegisterFile (in Dispatch.h) now can be used to emulate multiple register files.
Internally, it tracks the number of temporary registers dynamically allocated by each register file (described by class RegisterFileInfo).
A RegisterFileInfo stores the number of temporary registers available, and the maximum number of temporary registers used at runtime. There is a RegisterFileInfo for every register file defined by the processor scheduling model. In the absence of register file definitions, a "default" RegisterFileInfo is created; the default RegisterFileInfo describes a register file with an unlimited number of temporaries.

Each register file is associated with a list of MCRegisterClass indices. Knowinb the register class indices allows to map physical registers to register files.

The long term goal is to allow processor models to optionally specify how many register files are implemented using the following tablegen syntax:

`def FPRegFile : RegisterFile<[RegisterClassA, RegisterClassB, ...], #NumTemps>`

Where: [RegisterClassA, RegisterClassB,...] is a list of RegisterClass records, and #NumTemps is the number of temporaries implemented by the register file.

This patch is a "NFCI" change that restructures the code in preparation for the future development to fix PR36662.
Once this patch is accepted, the plan is to send two other patches:

1. A patch that adds the new tablegen syntax for defining RegisterFiles
2. An llvm-mca patch that allows to query the model to obtain the register file descriptors.

Please let me know what you think.

Thanks,
Andrea

[RKSimon]

You can probably commit the Instruction/InstrDesc argument changes straightaway and simplify this patch

[RKSimon]

<pedantic> I think physical registers is a better term than temporary registers, I don't know what other people think?

Jaguar treats 1 ymm logical as (upto) 2 x 128-bit physical entries - real world perf effect is minimal, but it does mean that it performs (upto) 2 read/writes into the PRF. 128-bit physicals that are known zero don't take up an PRF entry and the read/write should be quicker.

[courbet]

In D44488#1038972, @RKSimon wrote:

<pedantic> I think physical registers is a better term than temporary registers, I don't know what other people think?

I agree in principle, but unfortunately LLVM already uses MCPhysReg to denote RAX/RBX/... so there is a risk of confusion. Agner sometimes uses "microarchitectural registers", which is a nice name, albeit a bit long. We could abbreviate to UarchReg ?

Jaguar treats 1 ymm logical as (upto) 2 x 128-bit physical entries - real world perf effect is minimal, but it does mean that it performs (upto) 2 read/writes into the PRF. 128-bit physicals that are known zero don't take up an PRF entry and the read/write should be quicker.

tools/llvm-mca/Dispatch.cpp

41

why not use a range ?

tools/llvm-mca/Dispatch.h

37

typo: "tracks"

39

XXXInfo ususally represents immutable information about XXX. What about calling this RegisterMappingTracker or something along those lines ?

43

const ?

72

typo: specific

108

Please document these.

[andreadb]

Thanks for the feedback.

I am going to upload an updated patch soon.

In D44488#1039016, @courbet wrote:

In D44488#1038972, @RKSimon wrote:

<pedantic> I think physical registers is a better term than temporary registers, I don't know what other people think?

I agree in principle, but unfortunately LLVM already uses MCPhysReg to denote RAX/RBX/... so there is a risk of confusion. Agner sometimes uses "microarchitectural registers", which is a nice name, albeit a bit long. We could abbreviate to UarchReg ?

I like "microarchitectural register". I will use it for now.

tools/llvm-mca/Dispatch.h

39

I will use RegisterMappingTracker.

43

It can be changed by 'RegisterFile::isAvailable'.

Users can override the register file size using flag -register-file-size. If the number of "microarchitectural registers" is too small, there is a risk of deadlock in the dispatch logic. Method 'isAvailable' spots if there is a problems with the total number of mappings, and artificially increases to avoid problems. When the total number of mappings is created, a warning is generated to standard error to notify the user.

If you want, I can remove that check, and make this field const. We still want to identify potential deadlocks and abort with a fatal_error. What do you think?

108

Will do.

[andreadb]

In D44488#1038972, @RKSimon wrote:

Jaguar treats 1 ymm logical as (upto) 2 x 128-bit physical entries - real world perf effect is minimal, but it does mean that it performs (upto) 2 read/writes into the PRF. 128-bit physicals that are known zero don't take up an PRF entry and the read/write should be quicker.

Right. Ideally we should be able to emulate that. That can be a future work.

[andreadb]

Patch updated.
Please let me know what you think.

Cheers
-Andrea

[RKSimon]

A few minor comments but otherwise looks almost ready.

tools/llvm-mca/Dispatch.cpp

154

Better to do this in the constructor?

tools/llvm-mca/Dispatch.h

31–32

and and

78

Assert that there 32 or less PRFs?

80

for each

84

Please can you make it clear in the comment this WIP?

91

VR256RegClass

138

(style) Asserts should have a message as well as a condition.

[courbet]

tools/llvm-mca/Dispatch.cpp

153

remove "(0U)" ?

tools/llvm-mca/Dispatch.h

43

I see. Dying sounds reasonable. It does not really make sense for a microarchitecture to not even have enough microarchitectural registers to handle the registers its ISA defines. The user should know (or be told :) ) that.

[andreadb]

Patch updated.
Address review comments.

[andreadb]

Patch updated.
Forgot to add a string message to the new asserts.

[RKSimon]

LGTM with a couple of minors

tools/llvm-mca/Dispatch.cpp

30

Assert that RegisterFileIndex < 32 ? There may be other cases but this is the only one that I can find.

tools/llvm-mca/Dispatch.h

42

unbounded

tools/llvm-mca/Backend.cpp

 void Backend::runCycle(unsigned Cycle) {
   notifyCycleBegin(Cycle);
 
   while (SM.hasNext()) {
     InstRef IR = SM.peekNext();
     std::unique_ptr<Instruction> NewIS(
         IB->createInstruction(STI, IR.first, *IR.second));
     const InstrDesc &Desc = NewIS->getDesc();
-    if (!DU->isAvailable(Desc.NumMicroOps) || !DU->canDispatch(Desc))
+    if (!DU->isAvailable(Desc.NumMicroOps) || !DU->canDispatch(*NewIS))
       break;
 
     Instruction *IS = NewIS.get();
     Instructions[IR.first] = std::move(NewIS);
     IS->setRCUTokenID(DU->dispatch(IR.first, IS, STI));
     SM.updateNext();
   }
 

tools/llvm-mca/Dispatch.h

 
 namespace mca {
 
 class WriteState;
 class DispatchUnit;
 class Scheduler;
 class Backend;
 
-/// \brief Keeps track of register definitions.
-///
-/// This class tracks register definitions, and performs register renaming
-/// to break anti dependencies.
-/// By default, there is no limit in the number of register aliases which
-/// can be created for the purpose of register renaming. However, users can
-/// specify at object construction time a limit in the number of temporary
-/// registers which can be used by the register renaming logic.
+/// \brief Manages hardware register files, and and tracks data dependencies
+/// between registers.
 class RegisterFile {
   const llvm::MCRegisterInfo &MRI;
-  // Currently used mappings and maximum used mappings.
-  // These are to generate statistics only.
+
+  // Each register file is described by an instance of RegisterFileInfo.
+  // RegisterFileInfo track the number of temporary registers dynamically
+  // allocated during the execution.
+  struct RegisterFileInfo {
+    // Total number of temporary registers that are available for register
+    // renaming. A value of zero for this field means: this register file has
+    // an unbound number of temporaries.
+    unsigned TotalMappings;
+    // Number of temporary registers that are currently in use.
     unsigned NumUsedMappings;
+    // Maximum number of temporary registers used.
     unsigned MaxUsedMappings;
-  // Total number of mappings created over time.
+    // Total number of temporaries dynamically allocated.
     unsigned TotalMappingsCreated;
 
-  // The maximum number of register aliases which can be used by the
-  // register renamer. Defaut value for this field is zero.
-  // A value of zero for this field means that there is no limit in the
-  // amount of register mappings which can be created. That is equivalent
-  // to having a theoretically infinite number of temporary registers.
-  unsigned TotalMappings;
+    RegisterFileInfo(unsigned NumTemporaries)
+        : TotalMappings(NumTemporaries), NumUsedMappings(0), MaxUsedMappings(0),
+          TotalMappingsCreated(0) {}
+  };
 
-  // This map contains an entry for every physical register.
-  // A register index is used as a key value to access a WriteState.
-  // This is how we track RAW dependencies for dispatched
-  // instructions. For every register, we track the last seen write only.
-  // This assumes that all writes fully update both super and sub registers.
-  // We need a flag in MCInstrDesc to check if a write also updates super
-  // registers. We can then have a extra tablegen flag to set for instructions.
-  // This is a separate patch on its own.
-  std::vector<WriteState *> RegisterMappings;
-  // Assumptions are:
-  //  a) a false dependencies is always removed by the register renamer.
-  //  b) the register renamer can create an "infinite" number of mappings.
-  // Since we track the number of mappings created, in future we may
-  // introduce constraints on the number of mappings that can be created.
-  // For example, the maximum number of registers that are available for
-  // register renaming purposes may default to the size of the register file.
-
-  // In future, we can extend this design to allow multiple register files, and
-  // apply different restrictions on the register mappings and the number of
-  // temporary registers used by mappings.
+  // This is where information related to the various register files is kept.
+  // This set always contains at least one register file at index #0. That
+  // register file "sees" all the physical registers declared by the target, and
+  // (by default) it has un unbound number of temporaries.
+  // Users can override the number of temporary registers available in register
+  // file #0 through the command line flag `-register-file-size`.
+  llvm::SmallVector<RegisterFileInfo, 4> RegisterFiles;
+
+  // RegisterMapping objects are used to maps physical registers to WriteState
+  // objects. WriteState describes a register definition, and it is used to
+  // track RAW dependencies (see Instruction.h).
+  // A RegisterMapping object also specifies the set of register files
+  // associated to a physical register. The mapping between physreg and register
+  // files is done using a "register file mask".
+  //
+  // A register file mask identifies a set of register files. Each bit of the
+  // mask representation identifies a specif register file.
+  // For example:
+  //    0b0001     -->  Register file #0
+  //    0b0010     -->  Register file #1
+  //    0b0100     -->  Register file #2
+  //
+  // Note that this implementation allows register files to overlap. Also,
+  // by design, register file #0 contains all the physical registers.
+  using RegisterMapping = std::pair<WriteState *, unsigned>;
+
+  // This map contains an entry for every physical register defined by the
+  // processor scheduling model. The goal is to track the temporary registers
+  // usage for every register file.
+  std::vector<RegisterMapping> RegisterMappings;
+
+  // Initializes a RegisterFileInfo object for every register file
+  // declared by the target.
+  //
+  // This method is called by the constructor of RegisterFile for every register
+  // file defined by the scheduling model.
+  // The idea is that every scheduling model should be able to describe the
+  // composition of a register file, and how many temporaries are available.
+  // For example, a tablegen definitions for a x86 FP register file that
+  // features AVX would look like this:
+  //
+  //    def FPRegisterFile : RegisterFile<[VR128RegClass, VR256RegClass], 60>
+  //
+  // Here FPRegisterFile contains all the registers within the two specified
+  // register classes. FPRegisterFile implements 60 temporary registers which
+  // can be used for register renaming purpose.
+  //
+  // The list of register classes is then converted by the tablegen backend into
+  // a list of register class indices. That list, along with the number of
+  // temporaries, is then used to create a new RegisterFileInfo object.
+  void addRegisterFile(llvm::ArrayRef<unsigned> RegisterClasses,
+                       unsigned NumTemps);
+  void reserveTemporaries(unsigned RegisterFileMask);
+  void releaseTemporaries(unsigned RegisterFileMask);
 
 public:
-  RegisterFile(const llvm::MCRegisterInfo &mri, unsigned Mappings = 0)
-      : MRI(mri), NumUsedMappings(0), MaxUsedMappings(0),
-        TotalMappingsCreated(0), TotalMappings(Mappings),
-        RegisterMappings(MRI.getNumRegs(), nullptr) {}
+  RegisterFile(const llvm::MCRegisterInfo &mri, unsigned TempRegs = 0)
+      : MRI(mri), RegisterMappings(MRI.getNumRegs(), { nullptr, 0U }) {
+    // Add the "default" register file to the set.
+    addRegisterFile({}, TempRegs);
+    // TODO: teach the scheduling models how to specify multiple registers.
+  }
 
   // Creates a new register mapping for RegID.
   // This reserves a temporary register in the register file.
   void addRegisterMapping(WriteState &WS);
 
   // Invalidates register mappings associated to the input WriteState object.
   // This releases temporary registers in the register file.
   void invalidateRegisterMapping(const WriteState &WS);
 
-  bool isAvailable(unsigned NumRegWrites);
+  // Checks if there are enough temporary registers in the register files.
+  // Returns a "response mask" where each bit is the response from a
+  // RegisterFileInfo.
+  // For example: if all register files are available, then the response mask
+  // is a bitmask of all zeroes. If Instead register file #1 is not available,
+  // then the response mask is 0b10.
+  unsigned isAvailable(const llvm::ArrayRef<unsigned> Regs);
   void collectWrites(llvm::SmallVectorImpl<WriteState *> &Writes,
                      unsigned RegID) const;
   void updateOnRead(ReadState &RS, unsigned RegID);
-  unsigned getMaxUsedRegisterMappings() const { return MaxUsedMappings; }
-  unsigned getTotalRegisterMappingsCreated() const {
-    return TotalMappingsCreated;
+  unsigned getMaxUsedRegisterMappings(unsigned RegisterFileIndex) const {
+    assert(RegisterFileIndex < getNumRegisterFiles());
+    return RegisterFiles[RegisterFileIndex].MaxUsedMappings;
+  }
+  unsigned getTotalRegisterMappingsCreated(unsigned RegisterFileIndex) const {
+    assert(RegisterFileIndex < getNumRegisterFiles());
+    return RegisterFiles[RegisterFileIndex].TotalMappingsCreated;
   }
+  unsigned getNumRegisterFiles() const { return RegisterFiles.size(); }
 
 #ifndef NDEBUG
   void dump() const;
 #endif
 };
 
 /// \brief tracks which instructions are in-flight (i.e. dispatched but not
 /// retired) in the OoO backend.
class DispatchUnit {
   };
 
   // The DispatchUnit track dispatch stall events caused by unavailable
   // of hardware resources. Events are classified based on the stall kind;
   // so we have a counter for every source of dispatch stall. Counters are
   // stored into a vector `DispatchStall` which is always of size DS_LAST.
   std::vector<unsigned> DispatchStalls;
 
-  bool checkRAT(const InstrDesc &Desc);
+  bool checkRAT(const Instruction &Inst);
   bool checkRCU(const InstrDesc &Desc);
   bool checkScheduler(const InstrDesc &Desc);
 
   void updateRAWDependencies(ReadState &RS, const llvm::MCSubtargetInfo &STI);
   void notifyInstructionDispatched(unsigned IID);
 
 public:
   DispatchUnit(Backend *B, const llvm::MCRegisterInfo &MRI,
public:
   unsigned getDispatchWidth() const { return DispatchWidth; }
 
   bool isAvailable(unsigned NumEntries) const {
     return NumEntries <= AvailableEntries || AvailableEntries == DispatchWidth;
   }
 
   bool isRCUEmpty() const { return RCU->isEmpty(); }
 
-  bool canDispatch(const InstrDesc &Desc) {
+  bool canDispatch(const Instruction &Inst) {
+    const InstrDesc &Desc = Inst.getDesc();
     assert(isAvailable(Desc.NumMicroOps));
-    return checkRCU(Desc) && checkRAT(Desc) && checkScheduler(Desc);
+    return checkRCU(Desc) && checkRAT(Inst) && checkScheduler(Desc);
   }
 
   unsigned dispatch(unsigned IID, Instruction *NewInst,
                     const llvm::MCSubtargetInfo &STI);
 
   void collectWrites(llvm::SmallVectorImpl<WriteState *> &Vec,
                      unsigned RegID) const {
     return RAT->collectWrites(Vec, RegID);
unsigned getNumLDQStalls() const {
     return DispatchStalls[DS_LDQ_TOKEN_UNAVAILABLE];
   }
   unsigned getNumSTQStalls() const {
     return DispatchStalls[DS_STQ_TOKEN_UNAVAILABLE];
   }
   unsigned getNumDispatchGroupStalls() const {
     return DispatchStalls[DS_DISPATCH_GROUP_RESTRICTION];
   }
-  unsigned getMaxUsedRegisterMappings() const {
-    return RAT->getMaxUsedRegisterMappings();
+  unsigned getMaxUsedRegisterMappings(unsigned RegFileIndex = 0) const {
+    return RAT->getMaxUsedRegisterMappings(RegFileIndex);
   }
-  unsigned getTotalRegisterMappingsCreated() const {
-    return RAT->getTotalRegisterMappingsCreated();
+  unsigned getTotalRegisterMappingsCreated(unsigned RegFileIndex = 0) const {
+    return RAT->getTotalRegisterMappingsCreated(RegFileIndex);
   }
   void addNewRegisterMapping(WriteState &WS) { RAT->addRegisterMapping(WS); }
 
   void cycleEvent(unsigned Cycle) {
     RCU->cycleEvent();
     AvailableEntries =
         CarryOver >= DispatchWidth ? 0 : DispatchWidth - CarryOver;
     CarryOver = CarryOver >= DispatchWidth ? CarryOver - DispatchWidth : 0U;

tools/llvm-mca/Dispatch.cpp

 #include "llvm/Support/Debug.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "llvm-mca"
 
 namespace mca {
 
+void RegisterFile::addRegisterFile(ArrayRef<unsigned> RegisterClasses,
+                                   unsigned NumTemps) {
+  unsigned RegisterFileIndex = RegisterFiles.size();
+  RegisterFiles.emplace_back(NumTemps);
+
+  // Special case where there are no register classes specified.
+  // An empty register class set means *all* registers.
+  if (RegisterClasses.empty()) {
+    for (std::pair<WriteState *, unsigned> &Mapping : RegisterMappings)
+      Mapping.second |= 1U << RegisterFileIndex;
+  } else {
+    for (const unsigned RegClassIndex : RegisterClasses) {
+      const MCRegisterClass &RC = MRI.getRegClass(RegClassIndex);
+      for (auto I = RC.begin(), E = RC.end(); I < E; ++I)
+        RegisterMappings[*I].second |= 1U << RegisterFileIndex;
+    }
+  }
+}
+
+void RegisterFile::reserveTemporaries(unsigned RegisterFileMask) {
+  assert(RegisterFileMask && "RegisterFileMask cannot be zero!");
+  // Notify each register file that contains RegID.
+  do {
+    unsigned NextRegisterFile = llvm::PowerOf2Floor(RegisterFileMask);
+    unsigned RegisterFileIndex = llvm::countTrailingZeros(NextRegisterFile);
+    RegisterFileInfo &RFI = RegisterFiles[RegisterFileIndex];
+    RFI.NumUsedMappings++;
+    RFI.MaxUsedMappings = std::max(RFI.MaxUsedMappings, RFI.NumUsedMappings);
+    RFI.TotalMappingsCreated++;
+    RegisterFileMask ^= NextRegisterFile;
+  } while (RegisterFileMask);
+}
+
+void RegisterFile::releaseTemporaries(unsigned RegisterFileMask) {
+  assert(RegisterFileMask && "RegisterFileMask cannot be zero!");
+  // Notify each register file that contains RegID.
+  do {
+    unsigned NextRegisterFile = llvm::PowerOf2Floor(RegisterFileMask);
+    unsigned RegisterFileIndex = llvm::countTrailingZeros(NextRegisterFile);
+    RegisterFileInfo &RFI = RegisterFiles[RegisterFileIndex];
+    assert(RFI.NumUsedMappings);
+    RFI.NumUsedMappings--;
+    RegisterFileMask ^= NextRegisterFile;
+  } while (RegisterFileMask);
+}
+
 void RegisterFile::addRegisterMapping(WriteState &WS) {
   unsigned RegID = WS.getRegisterID();
   assert(RegID && "Adding an invalid register definition?");
 
-  RegisterMappings[RegID] = &WS;
+  RegisterMapping &Mapping = RegisterMappings[RegID];
+  Mapping.first = &WS;
   for (MCSubRegIterator I(RegID, &MRI); I.isValid(); ++I)
-    RegisterMappings[*I] = &WS;
-  if (MaxUsedMappings == NumUsedMappings)
-    MaxUsedMappings++;
-  NumUsedMappings++;
-  TotalMappingsCreated++;
+    RegisterMappings[*I].first = &WS;
+
+  reserveTemporaries(Mapping.second);
   // If this is a partial update, then we are done.
   if (!WS.fullyUpdatesSuperRegs())
     return;
 
   for (MCSuperRegIterator I(RegID, &MRI); I.isValid(); ++I)
-    RegisterMappings[*I] = &WS;
+    RegisterMappings[*I].first = &WS;
 }
 
 void RegisterFile::invalidateRegisterMapping(const WriteState &WS) {
   unsigned RegID = WS.getRegisterID();
   bool ShouldInvalidateSuperRegs = WS.fullyUpdatesSuperRegs();
 
   assert(RegID != 0 && "Invalidating an already invalid register?");
   assert(WS.getCyclesLeft() != -512 &&
          "Invalidating a write of unknown cycles!");
   assert(WS.getCyclesLeft() <= 0 && "Invalid cycles left for this write!");
-  if (!RegisterMappings[RegID])
+  RegisterMapping &Mapping = RegisterMappings[RegID];
+  if (!Mapping.first)
     return;
 
-  assert(NumUsedMappings);
-  NumUsedMappings--;
+  releaseTemporaries(Mapping.second);
 
-  if (RegisterMappings[RegID] == &WS)
-    RegisterMappings[RegID] = nullptr;
+  if (Mapping.first == &WS)
+    Mapping.first = nullptr;
 
   for (MCSubRegIterator I(RegID, &MRI); I.isValid(); ++I)
-    if (RegisterMappings[*I] == &WS)
-      RegisterMappings[*I] = nullptr;
+    if (RegisterMappings[*I].first == &WS)
+      RegisterMappings[*I].first = nullptr;
 
   if (!ShouldInvalidateSuperRegs)
     return;
 
   for (MCSuperRegIterator I(RegID, &MRI); I.isValid(); ++I)
-    if (RegisterMappings[*I] == &WS)
-      RegisterMappings[*I] = nullptr;
+    if (RegisterMappings[*I].first == &WS)
+      RegisterMappings[*I].first = nullptr;
 }
 
 // Update the number of used mappings in the event of instruction retired.
 // This mehod delegates to the register file the task of invalidating
 // register mappings that were created for instruction IS.
 void DispatchUnit::invalidateRegisterMappings(const Instruction &IS) {
   for (const std::unique_ptr<WriteState> &WS : IS.getDefs()) {
     DEBUG(dbgs() << "[RAT] Invalidating mapping for: ");
     DEBUG(WS->dump());
     RAT->invalidateRegisterMapping(*WS.get());
   }
 }
 
 void RegisterFile::collectWrites(SmallVectorImpl<WriteState *> &Writes,
                                  unsigned RegID) const {
   assert(RegID && RegID < RegisterMappings.size());
-  WriteState *WS = RegisterMappings[RegID];
+  WriteState *WS = RegisterMappings[RegID].first;
   if (WS) {
     DEBUG(dbgs() << "Found a dependent use of RegID=" << RegID << '\n');
     Writes.push_back(WS);
   }
 
   // Handle potential partial register updates.
   for (MCSubRegIterator I(RegID, &MRI); I.isValid(); ++I) {
-    WS = RegisterMappings[*I];
+    WS = RegisterMappings[*I].first;
     if (WS && std::find(Writes.begin(), Writes.end(), WS) == Writes.end()) {
       DEBUG(dbgs() << "Found a dependent use of subReg " << *I << " (part of "
                    << RegID << ")\n");
       Writes.push_back(WS);
     }
   }
 }
 
-bool RegisterFile::isAvailable(unsigned NumRegWrites) {
-  if (!TotalMappings)
-    return true;
-  if (NumRegWrites > TotalMappings) {
-    // The user specified a too small number of registers.
-    // Artificially set the number of temporaries to NumRegWrites.
-    errs() << "warning: not enough temporaries in the register file. "
-           << "The register file size has been automatically increased to "
-           << NumRegWrites << '\n';
-    TotalMappings = NumRegWrites;
+unsigned RegisterFile::isAvailable(const ArrayRef<unsigned> Regs) {
+  SmallVector<unsigned, 4> NumTemporaries(0U);
+  NumTemporaries.resize(getNumRegisterFiles());
+
+  // Find out how many temporary registers are required from each register file.
+  for (const unsigned RegID : Regs) {
+    unsigned RegisterFileMask = RegisterMappings[RegID].second;
+    do {
+      unsigned NextRegisterFileID = llvm::PowerOf2Floor(RegisterFileMask);
+      NumTemporaries[llvm::countTrailingZeros(NextRegisterFileID)]++;
+      RegisterFileMask ^= NextRegisterFileID;
+    } while (RegisterFileMask);
+  }
+
+  unsigned Response = 0;
+  for (unsigned I = 0, E = getNumRegisterFiles(); I < E; ++I) {
+    unsigned Temporaries = NumTemporaries[I];
+    if (!Temporaries)
+      continue;
+
+    RegisterFileInfo &RFI = RegisterFiles[I];
+    if (!RFI.TotalMappings) {
+      // The register file has an unbound number of registers. This is the
+      // default for register file #0, unless users override the number of
+      // temporaries via command line flag -reg-file-size.
+      continue;
+    }
+
+    if (RFI.TotalMappings < Temporaries) {
+      // The current register file is too small. This may occur if the number of
+      // temporaries for register file #0 was changed by the users via flag
+      // -reg-file-size. Alternatively, the scheduling model specified a too
+      // small number of registers for this register file. Artificially increase
+      // the total number of mappings.
+      errs() << "warning: not enough temporaries in register file " << I
+             << ". The register file size has been increased to " << Temporaries
+             << '\n';
+      RFI.TotalMappings = Temporaries;
+    }
+
+    if (RFI.TotalMappings < RFI.NumUsedMappings + Temporaries)
+      Response |= (1U << I);
   }
 
-  return NumRegWrites + NumUsedMappings <= TotalMappings;
+  return Response;
 }
 
 #ifndef NDEBUG
 void RegisterFile::dump() const {
-  for (unsigned I = 0, E = MRI.getNumRegs(); I < E; ++I)
-    if (RegisterMappings[I]) {
-      dbgs() << MRI.getName(I) << ", " << I << ", ";
-      RegisterMappings[I]->dump();
-    }
-
-  dbgs() << "TotalMappingsCreated: " << TotalMappingsCreated
-         << ", MaxUsedMappings: " << MaxUsedMappings
-         << ", NumUsedMappings: " << NumUsedMappings << '\n';
+  for (unsigned I = 0, E = MRI.getNumRegs(); I < E; ++I) {
+    const RegisterMapping &RM = RegisterMappings[I];
+    dbgs() << MRI.getName(I) << ", " << I << ", Map=" << RM.second << ", ";
+    if (RM.first)
+      RM.first->dump();
+    else
+      dbgs() << "(null)\n";
+  }
+
+  for (unsigned I = 0, E = getNumRegisterFiles(); I < E; ++I) {
+    dbgs() << "Register File #" << I;
+    const RegisterFileInfo &RFI = RegisterFiles[I];
+    dbgs() << "\n  TotalMappings:        " << RFI.TotalMappings
+           << "\n  TotalMappingsCreated: " << RFI.TotalMappingsCreated
+           << "\n  MaxUsedMappings:      " << RFI.MaxUsedMappings
+           << "\n  NumUsedMappings:      " << RFI.NumUsedMappings << '\n';
 }
 #endif
 
 // Reserves a number of slots, and returns a new token.
 unsigned RetireControlUnit::reserveSlot(unsigned Index, unsigned NumMicroOps) {
   assert(isAvailable(NumMicroOps));
   unsigned NormalizedQuantity =
       std::min(NumMicroOps, static_cast<unsigned>(Queue.size()));

 
 #ifndef NDEBUG
 void RetireControlUnit::dump() const {
   dbgs() << "Retire Unit: { Total Slots=" << Queue.size()
          << ", Available Slots=" << AvailableSlots << " }\n";
 }
 #endif
 
-bool DispatchUnit::checkRAT(const InstrDesc &Desc) {
-  unsigned NumWrites = Desc.Writes.size();
-  if (RAT->isAvailable(NumWrites))
-    return true;
+bool DispatchUnit::checkRAT(const Instruction &Instr) {
+  // Collect register definitions from the WriteStates.
+  SmallVector<unsigned, 8> RegDefs;
+
+  for (const std::unique_ptr<WriteState> &Def : Instr.getDefs())
+    RegDefs.push_back(Def->getRegisterID());
+
+  unsigned RegisterMask = RAT->isAvailable(RegDefs);
+  // A mask with all zeroes means: register files are available.
+  if (RegisterMask) {
+    // TODO: We currently implement a single hardware counter for all the
+    // dispatch stalls caused by the unavailability of temporaries in one of the
+    // register files.
+    // In future, we want to let register files directly notify hardware
+    // listeners in the event of a dispatch stall.  This would simplify the
+    // logic in Dispatch.[h/cpp], and move all the "hardware counting logic"
+    // into a View (for example: BackendStatistics).
     DispatchStalls[DS_RAT_REG_UNAVAILABLE]++;
     return false;
   }
 
+  return true;
+}
+
 bool DispatchUnit::checkRCU(const InstrDesc &Desc) {
   unsigned NumMicroOps = Desc.NumMicroOps;
   if (RCU->isAvailable(NumMicroOps))
     return true;
   DispatchStalls[DS_RCU_TOKEN_UNAVAILABLE]++;
   return false;
 }