[Mips][DSP] Fix physregs incorrectly marked as dead.

[mbrkusanin]

Summary

Implicit operands that use DSPControl register such as RDDSP and WRDSP are added
after InstrEmitter::EmitMachineNode determines which ones should be marked as
dead. This will add a manual check of these flags and correct them if necessary.

We also don't need removing of dead flags in LiveVariable analysis pass anymore.

[mbrkusanin]

The changes can be seen in some functions in test CodeGen/Mips/dsp-r1.ll. Where we previously had:

CMPU_EQ_QB killed %4:gpr32, killed %3:gpr32, implicit-def dead $dspccond
%5:gpr32 = RDDSP 31, implicit undef $dsppos, implicit undef $dspscount, implicit undef $dspcarry, implicit undef $dspoutflag, implicit undef $dspccond

We now have:

CMPU_EQ_QB killed %4:gpr32, killed %3:gpr32, implicit-def $dspccond
%5:gpr32 = RDDSP 31, implicit $dsppos, implicit $dspscount, implicit $dspcarry, implicit $dspoutflag, implicit killed $dspccond

[atanasyan]

Is it possible to add tests to check the changes?

[efriedma]

I would like to avoid having "undead" registers, which are marked dead but are semantically necessary, at any point. That means never calling setIsDead(false); as part of instruction selection, even if the flags are eventually correct after post-isel hooks run. The reason for this is that otherwise it's very confusing for anyone reading the code, or MIR dumps, to understand how it's supposed to work.

Why aren't these registers modeled precisely during isel?

[mbrkusanin]

Added test.

[mbrkusanin]

I would like to avoid having "undead" registers, which are marked dead but are semantically necessary, at any point. That means never calling setIsDead(false); as part of instruction selection, even if the flags are eventually correct after post-isel hooks run.

The idea was to remove setIsDead(false); which was already in LiveVariables.cpp. This way flags are corrected earlier.
Other possible solution would be to mark them as always live. Another solution would be to leave rL272647 as is, which lets the test CodeGen/Mips/dsp-r1.ll pass with -verify-machineinstrs flag, but then we have wrong dead flags.

The reason for this is that otherwise it's very confusing for anyone reading the code, or MIR dumps, to understand how it's supposed to work.
Why aren't these registers modeled precisely during isel?

That is the main issue. Because some registers for rddsp or wrdsp are added in post isel hook, the previous check which looks at uses in glue chain does not see it. As for modeling the instruction, I don't see any possible way of adding these registers to .td for these instructions. Is there some other place that you know of or have any suggestion where that could be done? I would prefer that too.

[mbrkusanin]

Updated test.

[efriedma]

I think fundamentally, the problem here is that the registers in question aren't being handled consistently.

Looking briefly, at the IR level, the DSPCtrl registers are implicit: it's equivalent to some unknown location in memory. Then, during isel, some instructions acquire explicit references to those registers. Then, after isel, some other instructions acquire explicit references to those registers. This is fundamentally broken; you have to transition from modeling DSPCtrl as unknown memory to modeling it as an explicit set of registers at exactly one point, for all operations which refer to DSPCtrl. Otherwise the dependencies are wrong. This doesn't just affect the dead/undef markings; this could also have implications for scheduling. For example, if we don't model the dependency between wrdsp and pick.qb, we could schedule them in the wrong order.

Not sure where the best place to convert from arbitrary "memory" to specific registers is. Probably the simplest is to do it all in post-isel hooks?

[mbrkusanin]

Problem with handling everything in post-isel hooks is that it is after marking registers as dead. So we would need to mark every instruction that uses any part of DSPControl as having an post-isel hook and there are more than 100 instructions (including MicroMips variants).

[mbrkusanin]

All changes are now in code for Mips target. Only change in common code is in removing there lines in LiveVariables.cpp that are preceded by a FIXME comment. Also, tests that check other DSPControl registers are added.

[mbrkusanin]

Fixed comment typos.

[efriedma]

So we would need to mark every instruction that uses any part of DSPControl as having an post-isel hook and there are more than 100 instructions

It should be possible to do this without too much boilerplate using a TSFlags bit; you can mark all the instructions that need this particular rewrite, and check the bit in processFunctionAfterISel, instead of explicitly listing all the opcodes. The Mips backend uses TSFlags for other instruction properties already; see, for example, IsCTI.

[mbrkusanin]

Marked all instructions that define part of DSPControl register with TSFlag named defsDSPCtrl. This helps us avoid running unnecessary code on instructions that don't need it.

While we could add something similar for RDDSP and WRDSP instructions to make sure all implicit operands are added (which appears that it needs to be done in common CodeGen not Mips) before dead flag are set, it would still not help. This is because other instructions that use DSPControl are not glued to RDDSP and/or WRDSP. So we would still need to check for those instructions some other way which would not be any more efficient then current solution.

[efriedma]

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

122

I'm still not really happy with the setIsDead(false), but I guess it's not a big deal if we aren't actually depending on those register defs for correct scheduling. Could probably use a comment explaining that, though.

133

Does this setIsDead() call actually run for some testcase? I don't see any dead flag in your testcase.

[mbrkusanin]

• Added a comment in: removeOperandFromDeadImplRegs()
• Removed MO.setIsDead() from markOperandInDeadImplRegs() and renamed to checkOperandForDeadImplRegs().
• Added a comment in: checkOperandForDeadImplRegs().
• Decided to remove the change from addDSPCtrlRegOperands() which marked RDDSP operands as implicit undef. Machine verifier would fail if only RDDSP was used without some other instruction previously defining a register. We could handle these Undef flags the same way we handle dead flags for other instructions (when we remove dead from previous we can also remove undef from current), but I don't think we need to touch these flags.

[mbrkusanin]

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

133

You're right. We don't need it anymore. This was needed in an older version of the patch. Now we can just check if it's dead.

[efriedma]

Machine verifier would fail if only RDDSP was used without some other instruction previously defining a register

If you're going to allow calling llvm.mips.rddsp without some other instruction defining the register, you need to mark the registers live-in to the function. Marking them "undef" means you aren't using the value in the register, which could cause issues later.

[mbrkusanin]

• Marked undef DSPControl registers as live-in to the function and basic block.

[mbrkusanin]

Sorry for the delay. I was away for a while.

[mbrkusanin]

• Rebase

include/llvm/CodeGen/MachineInstr.h

 // the back end.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CODEGEN_MACHINEINSTR_H
 #define LLVM_CODEGEN_MACHINEINSTR_H
 
 #include "llvm/ADT/DenseMapInfo.h"
+#include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/PointerSumType.h"
 #include "llvm/ADT/ilist.h"
 #include "llvm/ADT/ilist_node.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/CodeGen/MachineMemOperand.h"
 #include "llvm/CodeGen/MachineOperand.h"
 #include "llvm/CodeGen/TargetOpcodes.h"

   /// otherwise undefined super register.
   void setRegisterDefReadUndef(unsigned Reg, bool IsUndef = true);
 
   /// We have determined MI defines a register. Make sure there is an operand
   /// defining Reg.
   void addRegisterDefined(unsigned Reg,
                           const TargetRegisterInfo *RegInfo = nullptr);
 
+  /// We have determined that MI uses a register. Make sure there is an operand
+  /// using Reg.
+  void addRegisterUse(unsigned Reg,
+                      const TargetRegisterInfo *RegInfo = nullptr);
+
   /// Mark every physreg used by this instruction as
   /// dead except those in the UsedRegs list.
   ///
   /// On instructions with register mask operands, also add implicit-def
   /// operands for all registers in UsedRegs.
   void setPhysRegsDeadExcept(ArrayRef<unsigned> UsedRegs,
                              const TargetRegisterInfo &TRI);
 
+  /// Mark every physreg used in implicit operands for this instruction as dead
+  /// as some of them will be added later than others if 'hasPostISelHook' is
+  /// true. Also keep track of them so the dead flag can be removed upon use by
+  /// some other instruction.
+  void setImplicitRegsDead(DenseMap<unsigned, MachineOperand *> &ImplicitRegs);
+
   /// Return true if it is safe to move this instruction. If
   /// SawStore is set to true, it means that there is a store (or call) between
   /// the instruction's location and its intended destination.
   bool isSafeToMove(AliasAnalysis *AA, bool &SawStore) const;
 
   /// Returns true if this instruction's memory access aliases the memory
   /// access of Other.
   //

include/llvm/CodeGen/TargetLowering.h

 
   /// This method should be implemented by targets that mark instructions with
   /// the 'hasPostISelHook' flag. These instructions must be adjusted after
   /// instruction selection by target hooks.  e.g. To fill in optional defs for
   /// ARM 's' setting instructions.
   virtual void AdjustInstrPostInstrSelection(MachineInstr &MI,
                                              SDNode *Node) const;
 
+  /// This method should be implemented by targets if they add any new operands
+  /// in AdjustInstrPostInstrSelection controled by 'hasPostISelHook' flag. They
+  /// might have been marked as dead regardless of their later use and should be
+  /// adjusted. ImplicitRegs keeps track of previously added operands and should
+  /// be checked against operands of current instruction. Afterwards they should
+  /// be marked as dead and added to ImplicitRegs.
+  virtual void
+  AdjustImplicitRegsDead(MachineInstr &MI,
+                         DenseMap<unsigned, MachineOperand *> &ImplicitRegs,
+                         const MCInstrDesc &II) const;
+
   /// If this function returns true, SelectionDAGBuilder emits a
   /// LOAD_STACK_GUARD node when it is lowering Intrinsic::stackprotector.
   virtual bool useLoadStackGuardNode() const {
     return false;
   }
 
   virtual SDValue emitStackGuardXorFP(SelectionDAG &DAG, SDValue Val,
                                       const SDLoc &DL) const {

lib/CodeGen/LiveVariables.cpp

for (unsigned i = 0; i != NumOperandsToProcess; ++i) {
     if (MO.isUse()) {
       if (!(TargetRegisterInfo::isPhysicalRegister(MOReg) &&
             MRI->isReserved(MOReg)))
         MO.setIsKill(false);
       if (MO.readsReg())
         UseRegs.push_back(MOReg);
     } else {
       assert(MO.isDef());
-      // FIXME: We should not remove any dead flags. However the MIPS RDDSP
-      // instruction needs it at the moment: http://llvm.org/PR27116.
-      if (TargetRegisterInfo::isPhysicalRegister(MOReg) &&
-          !MRI->isReserved(MOReg))
-        MO.setIsDead(false);
       DefRegs.push_back(MOReg);
     }
   }
 
   MachineBasicBlock *MBB = MI.getParent();
   // Process all uses.
   for (unsigned i = 0, e = UseRegs.size(); i != e; ++i) {
     unsigned MOReg = UseRegs[i];

lib/CodeGen/MachineInstr.cpp

void MachineInstr::setPhysRegsDeadExcept(ArrayRef<unsigned> UsedRegs,
   // This is a call with a register mask operand.
   // Mask clobbers are always dead, so add defs for the non-dead defines.
   if (HasRegMask)
     for (ArrayRef<unsigned>::iterator I = UsedRegs.begin(), E = UsedRegs.end();
          I != E; ++I)
       addRegisterDefined(*I, &TRI);
 }
 
+void MachineInstr::setImplicitRegsDead(
+    DenseMap<unsigned, MachineOperand *> &ImplicitRegs) {
+  for (MachineOperand &MO : implicit_operands()) {
+    if (MO.isRegMask() || !MO.isReg() || !MO.isDef())
+      continue;
+    unsigned Reg = MO.getReg();
+    if (!TargetRegisterInfo::isPhysicalRegister(Reg))
+      continue;
+    ImplicitRegs[Reg] = &MO;
+  }
+}
+
 unsigned
 MachineInstrExpressionTrait::getHashValue(const MachineInstr* const &MI) {
   // Build up a buffer of hash code components.
   SmallVector<size_t, 8> HashComponents;
   HashComponents.reserve(MI->getNumOperands() + 1);
   HashComponents.push_back(MI->getOpcode());
   for (const MachineOperand &MO : MI->operands()) {
     if (MO.isReg() && MO.isDef() &&

 
 Optional<unsigned>
 MachineInstr::getFoldedRestoreSize(const TargetInstrInfo *TII) const {
   MMOList Accesses;
   if (TII->hasLoadFromStackSlot(*this, Accesses))
     return getSpillSlotSize(Accesses, getMF()->getFrameInfo());
   return None;
 }
+
+void MachineInstr::addRegisterUse(unsigned Reg,
+                                  const TargetRegisterInfo *RegInfo) {
+  if (TargetRegisterInfo::isPhysicalRegister(Reg)) {
+    MachineOperand *MO = findRegisterUseOperand(Reg, false, RegInfo);
+    if (MO)
+      return;
+  } else {
+    for (const MachineOperand &MO : operands()) {
+      if (MO.isReg() && MO.getReg() == Reg && MO.isUse() &&
+          MO.getSubReg() == 0)
+        return;
+    }
+  }
+  addOperand(MachineOperand::CreateReg(Reg,
+                                       false /*IsDef*/,
+                                       true  /*IsImp*/));
+}

lib/CodeGen/SelectionDAG/InstrEmitter.h

class LLVM_LIBRARY_VISIBILITY InstrEmitter {
   MachineRegisterInfo *MRI;
   const TargetInstrInfo *TII;
   const TargetRegisterInfo *TRI;
   const TargetLowering *TLI;
 
   MachineBasicBlock *MBB;
   MachineBasicBlock::iterator InsertPos;
 
+  /// Tracks implicit operands that are added by post-isel target hook as they
+  /// might be incorrectly marked as dead.
+  DenseMap<unsigned, MachineOperand*> ImplicitRegs;
+
   /// EmitCopyFromReg - Generate machine code for an CopyFromReg node or an
   /// implicit physical register output.
   void EmitCopyFromReg(SDNode *Node, unsigned ResNo,
                        bool IsClone, bool IsCloned,
                        unsigned SrcReg,
                        DenseMap<SDValue, unsigned> &VRBaseMap);
 
   void CreateVirtualRegisters(SDNode *Node,

lib/CodeGen/SelectionDAG/InstrEmitter.cpp

#endif
 
   // Finally mark unused registers as dead.
   if (!UsedRegs.empty() || II.getImplicitDefs() || II.hasOptionalDef())
     MIB->setPhysRegsDeadExcept(UsedRegs, *TRI);
 
   // Run post-isel target hook to adjust this instruction if needed.
   if (II.hasPostISelHook())
     TLI->AdjustInstrPostInstrSelection(*MIB, Node);
+
+  // Cleanup and track any new implicit operands added by post-isel hook.
+  TLI->AdjustImplicitRegsDead(*MIB, ImplicitRegs, II);
 }
 
 /// EmitSpecialNode - Generate machine code for a target-independent node and
 /// needed dependencies.
 void InstrEmitter::
 EmitSpecialNode(SDNode *Node, bool IsClone, bool IsCloned,
                 DenseMap<SDValue, unsigned> &VRBaseMap) {
   switch (Node->getOpcode()) {
case ISD::INLINEASM_BR: {
 
     // GCC inline assembly allows input operands to also be early-clobber
     // output operands (so long as the operand is written only after it's
     // used), but this does not match the semantics of our early-clobber flag.
     // If an early-clobber operand register is also an input operand register,
     // then remove the early-clobber flag.
     for (unsigned Reg : ECRegs) {
       if (MIB->readsRegister(Reg, TRI)) {
-        MachineOperand *MO = 
+        MachineOperand *MO =
             MIB->findRegisterDefOperand(Reg, false, false, TRI);
         assert(MO && "No def operand for clobbered register?");
         MO->setIsEarlyClobber(false);
       }
     }
 
     // Get the mdnode from the asm if it exists and add it to the instruction.
     SDValue MDV = Node->getOperand(InlineAsm::Op_MDNode);

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

 
 void TargetLowering::AdjustInstrPostInstrSelection(MachineInstr &MI,
                                                    SDNode *Node) const {
   assert(!MI.hasPostISelHook() &&
          "If a target marks an instruction with 'hasPostISelHook', "
          "it must implement TargetLowering::AdjustInstrPostInstrSelection!");
 }
 
+void TargetLowering::AdjustImplicitRegsDead(
+    MachineInstr &MI, DenseMap<unsigned, MachineOperand *> &ImplicitRegs,
+    const MCInstrDesc &II) const {}
+
 //===----------------------------------------------------------------------===//
 // SelectionDAGISel code
 //===----------------------------------------------------------------------===//
 
 SelectionDAGISel::SelectionDAGISel(TargetMachine &tm,
                                    CodeGenOpt::Level OL) :
   MachineFunctionPass(ID), TM(tm),
   FuncInfo(new FunctionLoweringInfo()),

lib/Target/Mips/MicroMipsDSPInstrInfo.td

 }
 
 class RDDSP_MM_DESC {
   dag OutOperandList = (outs GPR32Opnd:$rt);
   dag InOperandList = (ins uimm7:$mask);
   string AsmString = !strconcat("rddsp", "\t$rt, $mask");
   list<dag> Pattern = [(set GPR32Opnd:$rt, (int_mips_rddsp immZExt7:$mask))];
   InstrItinClass Itinerary = NoItinerary;
+  bit hasPostISelHook = 1;
 }
 
 class REPL_QB_MM_DESC {
   dag OutOperandList = (outs DSPROpnd:$rt);
   dag InOperandList = (ins uimm8:$imm);
   string AsmString = !strconcat("repl.qb", "\t$rt, $imm");
   list<dag> Pattern = [(set DSPROpnd:$rt, (int_mips_repl_qb immZExt8:$imm))];
   InstrItinClass Itinerary = NoItinerary;
 }
 
 class REPLV_PH_MM_DESC : ABSQ_S_PH_MM_R2_DESC_BASE<"replv.ph", int_mips_repl_ph,
                                                    NoItinerary, DSPROpnd,
                                                    GPR32Opnd>;
 class REPLV_QB_MM_DESC : ABSQ_S_PH_MM_R2_DESC_BASE<"replv.qb", int_mips_repl_qb,
                                                    NoItinerary, DSPROpnd,
                                                    GPR32Opnd>;
 
 class WRDSP_MM_DESC {
   dag OutOperandList = (outs);
   dag InOperandList = (ins GPR32Opnd:$rt, uimm7:$mask);
   string AsmString = !strconcat("wrdsp", "\t$rt, $mask");
   list<dag> Pattern = [(int_mips_wrdsp GPR32Opnd:$rt, immZExt7:$mask)];
   InstrItinClass Itinerary = NoItinerary;
   bit isMoveReg = 1;
+  bit hasPostISelHook = 1;
 }
 
 class BPOSGE32C_MMR3_DESC {
   dag OutOperandList = (outs);
   dag InOperandList = (ins brtarget1SImm16:$offset);
   string AsmString = !strconcat("bposge32c", "\t$offset");
   InstrItinClass Itinerary = NoItinerary;
   bit isBranch = 1;

lib/Target/Mips/MipsDSPInstrInfo.td

class RDDSP_DESC_BASE<string instr_asm, SDPatternOperator OpNode,
                       InstrItinClass itin> {
   dag OutOperandList = (outs GPR32Opnd:$rd);
   dag InOperandList = (ins uimm10:$mask);
   string AsmString = !strconcat(instr_asm, "\t$rd, $mask");
   list<dag> Pattern = [(set GPR32Opnd:$rd, (OpNode immZExt10:$mask))];
   InstrItinClass Itinerary = itin;
   string BaseOpcode = instr_asm;
   bit isMoveReg = 1;
+  bit hasPostISelHook = 1;
 }
 
 class WRDSP_DESC_BASE<string instr_asm, SDPatternOperator OpNode,
                       InstrItinClass itin> {
   dag OutOperandList = (outs);
   dag InOperandList = (ins GPR32Opnd:$rs, uimm10:$mask);
   string AsmString = !strconcat(instr_asm, "\t$rs, $mask");
   list<dag> Pattern = [(OpNode GPR32Opnd:$rs, immZExt10:$mask)];
   InstrItinClass Itinerary = itin;
   string BaseOpcode = instr_asm;
   bit isMoveReg = 1;
+  bit hasPostISelHook = 1;
 }
 
 class DPA_W_PH_DESC_BASE<string instr_asm, SDPatternOperator OpNode> {
   dag OutOperandList = (outs ACC64DSPOpnd:$ac);
   dag InOperandList = (ins GPR32Opnd:$rs, GPR32Opnd:$rt, ACC64DSPOpnd:$acin);
   string AsmString = !strconcat(instr_asm, "\t$ac, $rs, $rt");
   list<dag> Pattern = [(set ACC64DSPOpnd:$ac,
                         (OpNode GPR32Opnd:$rs, GPR32Opnd:$rt, ACC64DSPOpnd:$acin))];

lib/Target/Mips/MipsISelLowering.h

EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
                            EVT VT) const override;
 
     SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
     MachineBasicBlock *
     EmitInstrWithCustomInserter(MachineInstr &MI,
                                 MachineBasicBlock *MBB) const override;
 
+    void
+    AdjustImplicitRegsDead(MachineInstr &MI,
+                           DenseMap<unsigned, MachineOperand *> &ImplicitRegs,
+                           const MCInstrDesc &II) const override;
+
     void AdjustInstrPostInstrSelection(MachineInstr &MI,
                                        SDNode *Node) const override;
 
     void HandleByVal(CCState *, unsigned &, unsigned) const override;
 
     unsigned getRegisterByName(const char* RegName, EVT VT,
                                SelectionDAG &DAG) const override;
 

lib/Target/Mips/MipsISelLowering.cpp

if (Subtarget.inMips16HardFloat()) {
     }
   }
   Ops.push_back(CLI.DAG.getRegisterMask(Mask));
 
   if (InFlag.getNode())
     Ops.push_back(InFlag);
 }
 
+static void addDSPCtrlRegisterImplicitDefs(MachineInstr &MI,
+                                           const TargetRegisterInfo &TRI) {
+  unsigned Mask = MI.getOperand(1).getImm();
+  if (Mask & 1)
+    MI.addRegisterDefined(Mips::DSPPos, &TRI);
+  if (Mask & 2)
+    MI.addRegisterDefined(Mips::DSPSCount, &TRI);
+  if (Mask & 4)
+    MI.addRegisterDefined(Mips::DSPCarry, &TRI);
+  if (Mask & 8)
+    MI.addRegisterDefined(Mips::DSPOutFlag, &TRI);
+  if (Mask & 16)
+    MI.addRegisterDefined(Mips::DSPCCond, &TRI);
+  if (Mask & 32)
+    MI.addRegisterDefined(Mips::DSPEFI, &TRI);
+}
+
+static void addDSPCtrlRegisterImplicitUses(MachineInstr &MI,
+                                           const TargetRegisterInfo &TRI) {
+  unsigned Mask = MI.getOperand(1).getImm();
+  if (Mask & 1)
+    MI.addRegisterUse(Mips::DSPPos, &TRI);
+  if (Mask & 2)
+    MI.addRegisterUse(Mips::DSPSCount, &TRI);
+  if (Mask & 4)
+    MI.addRegisterUse(Mips::DSPCarry, &TRI);
+  if (Mask & 8)
+    MI.addRegisterUse(Mips::DSPOutFlag, &TRI);
+  if (Mask & 16)
+    MI.addRegisterUse(Mips::DSPCCond, &TRI);
+  if (Mask & 32)
+    MI.addRegisterUse(Mips::DSPEFI, &TRI);
+}
+
 void MipsTargetLowering::AdjustInstrPostInstrSelection(MachineInstr &MI,
                                                        SDNode *Node) const {
+  const TargetRegisterInfo &TRI = *Subtarget.getRegisterInfo();
   switch (MI.getOpcode()) {
     default:
       return;
+    case Mips::RDDSP:
+    case Mips::RDDSP_MM:
+      addDSPCtrlRegisterImplicitUses(MI, TRI);
+      break;
+    case Mips::WRDSP:
+    case Mips::WRDSP_MM:
+      addDSPCtrlRegisterImplicitDefs(MI, TRI);
+      break;
     case Mips::JALR:
     case Mips::JALRPseudo:
     case Mips::JALR64:
     case Mips::JALR64Pseudo:
     case Mips::JALR16_MM:
     case Mips::JALRC16_MMR6:
     case Mips::TAILCALLREG:
     case Mips::TAILCALLREG64:
case Mips::TAILCALLREG_MMR6: {
 
       MachineFunction *MF = MI.getParent()->getParent();
       MCSymbol *S = MF->getContext().getOrCreateSymbol(Sym);
       MI.addOperand(MachineOperand::CreateMCSymbol(S, MipsII::MO_JALR));
     }
   }
 }
 
+void MipsTargetLowering::AdjustImplicitRegsDead(
+    MachineInstr &MI, DenseMap<unsigned, MachineOperand *> &ImplicitRegs,
+    const MCInstrDesc &II) const {
+  // Check current operands against previously added (if any) if they are used
+  // and remove dead mark.
+  if (!ImplicitRegs.empty())
+    for (MachineOperand &MO : MI.implicit_operands())
+      if (MO.isReg()) {
+        auto Res = ImplicitRegs.find(MO.getReg());
+        if (Res != ImplicitRegs.end()) {
+          if (Res->second->isDead())
+            Res->second->setIsDead(false);
+          ImplicitRegs.erase(Res);
+        }
+      }
+
+  // Mark all newly added implicit operands as dead.
+  if (II.getImplicitDefs() || II.hasOptionalDef())
+    MI.setImplicitRegsDead(ImplicitRegs);
+}
+
 /// LowerCall - functions arguments are copied from virtual regs to
 /// (physical regs)/(stack frame), CALLSEQ_START and CALLSEQ_END are emitted.
 SDValue
 MipsTargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
                               SmallVectorImpl<SDValue> &InVals) const {
   SelectionDAG &DAG                     = CLI.DAG;
   SDLoc DL                              = CLI.DL;
   SmallVectorImpl<ISD::OutputArg> &Outs = CLI.Outs;

lib/Target/Mips/MipsSEISelDAGToDAG.cpp

bool MipsSEDAGToDAGISel::runOnMachineFunction(MachineFunction &MF) {
   return MipsDAGToDAGISel::runOnMachineFunction(MF);
 }
 
 void MipsSEDAGToDAGISel::getAnalysisUsage(AnalysisUsage &AU) const {
   AU.addRequired<DominatorTreeWrapperPass>();
   SelectionDAGISel::getAnalysisUsage(AU);
 }
 
-void MipsSEDAGToDAGISel::addDSPCtrlRegOperands(bool IsDef, MachineInstr &MI,
-                                               MachineFunction &MF) {
-  MachineInstrBuilder MIB(MF, &MI);
-  unsigned Mask = MI.getOperand(1).getImm();
-  unsigned Flag =
-      IsDef ? RegState::ImplicitDefine : RegState::Implicit | RegState::Undef;
-
-  if (Mask & 1)
-    MIB.addReg(Mips::DSPPos, Flag);
-
-  if (Mask & 2)
-    MIB.addReg(Mips::DSPSCount, Flag);
-
-  if (Mask & 4)
-    MIB.addReg(Mips::DSPCarry, Flag);
-
-  if (Mask & 8)
-    MIB.addReg(Mips::DSPOutFlag, Flag);
-
-  if (Mask & 16)
-    MIB.addReg(Mips::DSPCCond, Flag);
-
-  if (Mask & 32)
-    MIB.addReg(Mips::DSPEFI, Flag);
-}
-
 unsigned MipsSEDAGToDAGISel::getMSACtrlReg(const SDValue RegIdx) const {
   uint64_t RegNum = cast<ConstantSDNode>(RegIdx)->getZExtValue();
   return Mips::MSACtrlRegClass.getRegister(RegNum);
 }
 
 bool MipsSEDAGToDAGISel::replaceUsesWithZeroReg(MachineRegisterInfo *MRI,
                                                 const MachineInstr& MI) {
   unsigned DstReg = 0, ZeroReg = 0;

 void MipsSEDAGToDAGISel::processFunctionAfterISel(MachineFunction &MF) {
   MF.getInfo<MipsFunctionInfo>()->initGlobalBaseReg();
 
   MachineRegisterInfo *MRI = &MF.getRegInfo();
 
   for (auto &MBB: MF) {
     for (auto &MI: MBB) {
       switch (MI.getOpcode()) {
-      case Mips::RDDSP:
-        addDSPCtrlRegOperands(false, MI, MF);
-        break;
-      case Mips::WRDSP:
-        addDSPCtrlRegOperands(true, MI, MF);
-        break;
       case Mips::BuildPairF64_64:
       case Mips::ExtractElementF64_64:
         if (!Subtarget->useOddSPReg()) {
           MI.addOperand(MachineOperand::CreateReg(Mips::SP, false, true));
           break;
         }
         LLVM_FALLTHROUGH;
       case Mips::BuildPairF64:
       case Mips::ExtractElementF64:
         if (Subtarget->isABI_FPXX() && !Subtarget->hasMTHC1())
           MI.addOperand(MachineOperand::CreateReg(Mips::SP, false, true));
         break;
       default:
         replaceUsesWithZeroReg(MRI, MI);
       }
     }
   }
 }
 
 void MipsSEDAGToDAGISel::selectAddE(SDNode *Node, const SDLoc &DL) const {
   SDValue InFlag = Node->getOperand(2);
   unsigned Opc = InFlag.getOpcode();
   SDValue LHS = Node->getOperand(0), RHS = Node->getOperand(1);
   EVT VT = LHS.getValueType();
 
   // In the base case, we can rely on the carry bit from the addsc
   // instruction.
   if (Opc == ISD::ADDC) {
     SDValue Ops[3] = {LHS, RHS, InFlag};
     CurDAG->SelectNodeTo(Node, Mips::ADDWC, VT, MVT::Glue, Ops);
     return;
   }
 

test/CodeGen/Mips/dsp-implicit-dead.ll

+; RUN: llc -march=mipsel -mcpu=mips32r2 -mattr=+dsp -verify-machineinstrs \
+; RUN:     -stop-before=prologepilog -o - < %s | FileCheck %s
+
+; Checks that implicit operands added in post isel hook for RDDSP and WRDSP are
+; not incorrectly marked dead in previous instructions.
+
+declare i32 @llvm.mips.rddsp(i32) nounwind readonly
+declare void @llvm.mips.cmpu.le.qb(<4 x i8>, <4 x i8>) nounwind
+
+define i32 @test__builtin_mips_cmpu_le_qb_before_rddsp(i32 %i0, i32 %a0.coerce, i32 %a1.coerce) nounwind {
+entry:
+; CHECK:      CMPU_LE_QB killed renamable $a1, killed renamable $a2, implicit-def $dspccond
+; CHECK-NEXT: renamable $v0 = RDDSP 31, implicit $dsppos, implicit $dspscount, implicit $dspcarry, implicit $dspoutflag, implicit killed $dspccond
+  %0 = bitcast i32 %a0.coerce to <4 x i8>
+  %1 = bitcast i32 %a1.coerce to <4 x i8>
+  tail call void @llvm.mips.cmpu.le.qb(<4 x i8> %0, <4 x i8> %1)
+  %2 = tail call i32 @llvm.mips.rddsp(i32 31)
+  ret i32 %2
+}
+
+declare void @llvm.mips.wrdsp(i32, i32) nounwind
+declare <4 x i8> @llvm.mips.pick.qb(<4 x i8>, <4 x i8>) nounwind readonly
+
+define i32 @test__builtin_mips_pick_qb_before_wrdsp(i32 %i0, i32 %a0.coerce, i32 %a1.coerce) nounwind readonly {
+entry:
+; CHECK:      WRDSP killed renamable $a0, 16, implicit-def $dspccond
+; CHECK-NEXT: renamable $v0 = PICK_QB killed renamable $a1, killed renamable $a2, implicit killed $dspccond
+  %0 = bitcast i32 %a0.coerce to <4 x i8>
+  %1 = bitcast i32 %a1.coerce to <4 x i8>
+  tail call void @llvm.mips.wrdsp(i32 %i0, i32 16)
+  %2 = tail call <4 x i8> @llvm.mips.pick.qb(<4 x i8> %0, <4 x i8> %1)
+  %3 = bitcast <4 x i8> %2 to i32
+  ret i32 %3
+}