[RISCV] Add lowering of global TLS addresses

[lewis-revill]

Summary

This patch adds lowering for global TLS addresses for the TLS models of InitialExec, GlobalDynamic, LocalExec and LocalDynamic.

LocalExec support required using a 4-operand add instruction, which uses the fourth operand to express a relocation on the symbol. The necessary fixup is emitted when the instruction is emitted.

[jrtc27]

I'd rather see all the local-exec support moved into D55306; I don't think it makes much sense to add things here that don't yet work. Alternatively, to avoid lines that are then going to be modified again in D55306, you could merge the two? There aren't many extra changes in D55306.

[asb]

Thanks for sharing this patch series Lewis. As it happens @luismarques has been finishing off his own implementation of TLS support which this unfortunately clashes with. The upside is that we're in a good position to review this work!

Could you please refactor the patches so that MC layer changes occur separately to codegen. e.g. having one patch that adds the MC layer changes for for TLS and a second which adds the codegen changes.

[lewis-revill]

In D55305#1319651, @jrtc27 wrote:

I'd rather see all the local-exec support moved into D55306; I don't think it makes much sense to add things here that don't yet work. Alternatively, to avoid lines that are then going to be modified again in D55306, you could merge the two? There aren't many extra changes in D55306.

Yes sorry about that, I can do that.

In D55305#1319850, @asb wrote:

Thanks for sharing this patch series Lewis. As it happens @luismarques has been finishing off his own implementation of TLS support which this unfortunately clashes with. The upside is that we're in a good position to review this work!

Could you please refactor the patches so that MC layer changes occur separately to codegen. e.g. having one patch that adds the MC layer changes for for TLS and a second which adds the codegen changes.

OK, I'll look at doing that and incorporating existing MC patches where they exist.

[lewis-revill]

Merging D55306 into this, then splitting this patch into MC layer and Codegen

[lewis-revill]

Merged D55306 into this patch.

[lewis-revill]

Split into MC layer (patches D55341 D55342) and codegen (this patch)

[lewis-revill]

Rebased with updated dependencies.

[lewis-revill]

Rebase with updated dependencies. Use and expand the la.tls.ie and la.tls.gd pseudo instructions.

[lewis-revill]

Rebased and fixed FileCheck lines.

[lewis-revill]

lib/Target/RISCV/RISCVISelLowering.cpp

468

Am I correct in thinking this sequence should actually be opaque here, it causes a DAG with multiple load/stores of the same GlobalTLSAddress to only duplicate the ADDI relative to a single AddTPRel node... I know this would be safe for absolute addressing, but not for thread-pointer relative addressing?

[lewis-revill]

lib/Target/RISCV/RISCVISelLowering.cpp

468

Actually it appears GCC will do common subexpression elimination on this sequence with optimizations enabled so I would take that to mean it is safe to emit as it is.

[rogfer01]

lib/Target/RISCV/RISCVISelLowering.cpp

445

Because we run BranchRelaxation in addPreEmitPass but the expansion of these pseudos happens in addPreEmitPass2 I think we need to update RISCVInstrInfo::getInstSizeInBytes to state that PseudoLA_TLS_IE and PseudoLA_TLS_GD will take 8 bytes (otherwise we run into "fixup errors" because the relaxation has been done using an underestimation).

(Alternatively we could use the Size attribute in tablegen but currently only PseudoLI does this and we don't use that one in codegen, so that is probably OK)

Also I wonder if the attribute isAsmParserOnly = 1 should be updated to 0 so we are more honest with the current usage of these pseudos. That said from a cursory check in tablegen, that attribute does not seem very important at the moment.

[lewis-revill]

lib/Target/RISCV/RISCVISelLowering.cpp

445

Thanks, I'll make that change, I did the same for the PIC PseudoLA so I must have forgotten it for this.

The isAsmParser attribute seems to enforce that the pseudo instruction doesn't cross the barrier between CodeGen and MC. Might be worth removing it to prevent confusion, I could do that in a later patch since there are now a couple of pseudos like this.

[lewis-revill]

Add pseudo instructions to getInstSizeInBytes.

[rogfer01]

lib/Target/RISCV/RISCVISelLowering.cpp

523

I'm curious, is there a reason to have this unused parameter here? (other than symmetry with getStaticTLSAddr, that is).

Removing it could allow you to combine the two cases in lowerGlobalTLSAddress. That could make more obvious that in RISC-V we use the same sequence in both cases.

[lewis-revill]

Added nounwind to test functions.

[asb]

I added some very minor nits in comments, but other than those I think this is looking good to land. Thanks! It's also worth checking @rogfer01's suggestion before committing https://reviews.llvm.org/D55305#inline-553463

lib/Target/RISCV/RISCVISelLowering.cpp

500

Line is longer than 80 chars, git clang-format will reformat niclely (assuming you have the git-clang-format helper on your PATH)

538

Line is longer than 80 chars, git clang-format will reformat niclely (assuming you have the git-clang-format helper on your PATH)

574

The enumeration is fully covered, so we shouldn't have the default case here: https://llvm.org/docs/CodingStandards.html#don-t-use-default-labels-in-fully-covered-switches-over-enumerations

[lewis-revill]

Updated to address comments

lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp

   OperandMatchResultTy parseMemOpBaseReg(OperandVector &Operands);
   OperandMatchResultTy parseOperandWithModifier(OperandVector &Operands);
   OperandMatchResultTy parseBareSymbol(OperandVector &Operands);
+  OperandMatchResultTy parseTPRelAddSymbol(OperandVector &Operands);
   OperandMatchResultTy parseJALOffset(OperandVector &Operands);
 
   bool parseOperand(OperandVector &Operands, StringRef Mnemonic);

             VK == RISCVMCExpr::VK_RISCV_CALL_PLT);
   }
 
+  bool isTPRelAddSymbol() const {
+    int64_t Imm;
+    RISCVMCExpr::VariantKind VK;
+    // Must be of 'immediate' type but not a constant.
+    if (!isImm() || evaluateConstantImm(getImm(), Imm, VK))
+      return false;
+    return RISCVAsmParser::classifySymbolRef(getImm(), VK, Imm) &&
+           VK == RISCVMCExpr::VK_RISCV_TPREL_ADD;
+  }
+
   bool isCSRSystemRegister() const { return isSystemRegister(); }
 
   /// Return true if the operand is a valid for the fence instruction e.g.

       IsValid = isInt<12>(Imm);
     return IsValid && ((IsConstantImm && VK == RISCVMCExpr::VK_RISCV_None) ||
                        VK == RISCVMCExpr::VK_RISCV_LO ||
-                       VK == RISCVMCExpr::VK_RISCV_PCREL_LO);
+                       VK == RISCVMCExpr::VK_RISCV_PCREL_LO ||
+                       VK == RISCVMCExpr::VK_RISCV_TPREL_LO);
   }
 
   bool isSImm12Lsb0() const { return isBareSimmNLsb0<12>(); }

     bool IsConstantImm = evaluateConstantImm(getImm(), Imm, VK);
     if (!IsConstantImm) {
       IsValid = RISCVAsmParser::classifySymbolRef(getImm(), VK, Imm);
-      return IsValid && VK == RISCVMCExpr::VK_RISCV_HI;
+      return IsValid && (VK == RISCVMCExpr::VK_RISCV_HI ||
+                         VK == RISCVMCExpr::VK_RISCV_TPREL_HI);
     } else {
       return isUInt<20>(Imm) && (VK == RISCVMCExpr::VK_RISCV_None ||
-                                 VK == RISCVMCExpr::VK_RISCV_HI);
+                                 VK == RISCVMCExpr::VK_RISCV_HI ||
+                                 VK == RISCVMCExpr::VK_RISCV_TPREL_HI);
     }
   }
 

     if (!IsConstantImm) {
       IsValid = RISCVAsmParser::classifySymbolRef(getImm(), VK, Imm);
       return IsValid && (VK == RISCVMCExpr::VK_RISCV_PCREL_HI ||
-                         VK == RISCVMCExpr::VK_RISCV_GOT_HI);
+                         VK == RISCVMCExpr::VK_RISCV_GOT_HI ||
+                         VK == RISCVMCExpr::VK_RISCV_TLS_GOT_HI ||
+                         VK == RISCVMCExpr::VK_RISCV_TLS_GD_HI);
     } else {
       return isUInt<20>(Imm) && (VK == RISCVMCExpr::VK_RISCV_None ||
                                  VK == RISCVMCExpr::VK_RISCV_PCREL_HI ||
-                                 VK == RISCVMCExpr::VK_RISCV_GOT_HI);
+                                 VK == RISCVMCExpr::VK_RISCV_GOT_HI ||
+                                 VK == RISCVMCExpr::VK_RISCV_TLS_GOT_HI ||
+                                 VK == RISCVMCExpr::VK_RISCV_TLS_GD_HI);
     }
   }
 

   case Match_InvalidSImm12:
     return generateImmOutOfRangeError(
         Operands, ErrorInfo, -(1 << 11), (1 << 11) - 1,
-        "operand must be a symbol with %lo/%pcrel_lo modifier or an integer in "
-        "the range");
+        "operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an "
+        "integer in the range");
   case Match_InvalidSImm12Lsb0:
     return generateImmOutOfRangeError(
         Operands, ErrorInfo, -(1 << 11), (1 << 11) - 2,

         "immediate must be a multiple of 2 bytes in the range");
   case Match_InvalidUImm20LUI:
     return generateImmOutOfRangeError(Operands, ErrorInfo, 0, (1 << 20) - 1,
-                                      "operand must be a symbol with %hi() "
-                                      "modifier or an integer in the range");
+                                      "operand must be a symbol with "
+                                      "%hi/%tprel_hi() modifier or an integer "
+                                      "in the range");
   case Match_InvalidUImm20AUIPC:
     return generateImmOutOfRangeError(
         Operands, ErrorInfo, 0, (1 << 20) - 1,
-        "operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an "
-        "integer in the range");
+        "operand must be a symbol with "
+        "%pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in "
+        "the range");
   case Match_InvalidSImm21Lsb0JAL:
     return generateImmOutOfRangeError(
         Operands, ErrorInfo, -(1 << 20), (1 << 20) - 2,

     SMLoc ErrorLoc = ((RISCVOperand &)*Operands[ErrorInfo]).getStartLoc();
     return Error(ErrorLoc, "operand must be a bare symbol name");
   }
+  case Match_InvalidTPRelAddSymbol: {
+    SMLoc ErrorLoc = ((RISCVOperand &)*Operands[ErrorInfo]).getStartLoc();
+    return Error(ErrorLoc, "operand must be a symbol with %tprel_add modifier");
+  }
   }
 
   llvm_unreachable("Unknown match type detected!");

   return MatchOperand_Success;
 }
 
+OperandMatchResultTy
+RISCVAsmParser::parseTPRelAddSymbol(OperandVector &Operands) {
+  SMLoc S = getLoc();
+  SMLoc E = SMLoc::getFromPointer(S.getPointer() - 1);
+
+  if (getLexer().getKind() != AsmToken::Percent)
+    return MatchOperand_NoMatch;
+
+  getParser().Lex(); // Eat '%'
+
+  if (getLexer().getKind() != AsmToken::Identifier) {
+    Error(getLoc(), "expected valid identifier for operand modifier");
+    return MatchOperand_ParseFail;
+  }
+  StringRef Identifier = getParser().getTok().getIdentifier();
+  RISCVMCExpr::VariantKind VK = RISCVMCExpr::getVariantKindForName(Identifier);
+  if (VK != RISCVMCExpr::VK_RISCV_TPREL_ADD) {
+    Error(getLoc(), "expected %tprel_add modifier");
+    return MatchOperand_ParseFail;
+  }
+
+  getParser().Lex(); // Eat the identifier
+  if (getLexer().getKind() != AsmToken::LParen) {
+    Error(getLoc(), "expected '('");
+    return MatchOperand_ParseFail;
+  }
+  getParser().Lex(); // Eat '('
+
+  const MCExpr *SubExpr;
+  if (getParser().parseParenExpression(SubExpr, E))
+    return MatchOperand_ParseFail;
+
+  const MCExpr *ModExpr = RISCVMCExpr::create(SubExpr, VK, getContext());
+  Operands.push_back(RISCVOperand::createImm(ModExpr, S, E, isRV64()));
+  return MatchOperand_Success;
+}
+
 OperandMatchResultTy RISCVAsmParser::parseJALOffset(OperandVector &Operands) {
   // Parsing jal operands is fiddly due to the `jal foo` and `jal ra, foo`
   // both being acceptable forms. When parsing `jal ra, foo` this function

lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.h

       { "fixup_riscv_pcrel_lo12_i", 20,     12,  MCFixupKindInfo::FKF_IsPCRel },
       { "fixup_riscv_pcrel_lo12_s",  0,     32,  MCFixupKindInfo::FKF_IsPCRel },
       { "fixup_riscv_got_hi20",     12,     20,  MCFixupKindInfo::FKF_IsPCRel },
+      { "fixup_riscv_tls_got_hi20", 12,     20,  MCFixupKindInfo::FKF_IsPCRel },
+      { "fixup_riscv_tprel_hi20",   12,     20,  0 },
+      { "fixup_riscv_tprel_lo12_i", 20,     12,  0 },
+      { "fixup_riscv_tprel_lo12_s",  0,     32,  0 },
+      { "fixup_riscv_tprel_add",     0,      0,  0 },
+      { "fixup_riscv_tls_gd_hi20",  12,     20,  MCFixupKindInfo::FKF_IsPCRel },
       { "fixup_riscv_jal",          12,     20,  MCFixupKindInfo::FKF_IsPCRel },
       { "fixup_riscv_branch",        0,     32,  MCFixupKindInfo::FKF_IsPCRel },
       { "fixup_riscv_rvc_jump",      2,     11,  MCFixupKindInfo::FKF_IsPCRel },

lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.cpp

     return Value;
   case RISCV::fixup_riscv_lo12_i:
   case RISCV::fixup_riscv_pcrel_lo12_i:
+  case RISCV::fixup_riscv_tprel_lo12_i:
     return Value & 0xfff;
   case RISCV::fixup_riscv_lo12_s:
   case RISCV::fixup_riscv_pcrel_lo12_s:
+  case RISCV::fixup_riscv_tprel_lo12_s:
     return (((Value >> 5) & 0x7f) << 25) | ((Value & 0x1f) << 7);
   case RISCV::fixup_riscv_hi20:
   case RISCV::fixup_riscv_pcrel_hi20:
   case RISCV::fixup_riscv_got_hi20:
+  case RISCV::fixup_riscv_tls_got_hi20:
+  case RISCV::fixup_riscv_tprel_hi20:
+  case RISCV::fixup_riscv_tls_gd_hi20:
     // Add 1 if bit 11 is 1, to compensate for low 12 bits being negative.
     return ((Value + 0x800) >> 12) & 0xfffff;
   case RISCV::fixup_riscv_jal: {

lib/Target/RISCV/MCTargetDesc/RISCVELFObjectWriter.cpp

     return ELF::R_RISCV_PCREL_LO12_S;
   case RISCV::fixup_riscv_got_hi20:
     return ELF::R_RISCV_GOT_HI20;
+  case RISCV::fixup_riscv_tls_got_hi20:
+    return ELF::R_RISCV_TLS_GOT_HI20;
+  case RISCV::fixup_riscv_tprel_hi20:
+    return ELF::R_RISCV_TPREL_HI20;
+  case RISCV::fixup_riscv_tprel_lo12_i:
+    return ELF::R_RISCV_TPREL_LO12_I;
+  case RISCV::fixup_riscv_tprel_lo12_s:
+    return ELF::R_RISCV_TPREL_LO12_S;
+  case RISCV::fixup_riscv_tprel_add:
+    return ELF::R_RISCV_TPREL_ADD;
+  case RISCV::fixup_riscv_tls_gd_hi20:
+    return ELF::R_RISCV_TLS_GD_HI20;
   case RISCV::fixup_riscv_jal:
     return ELF::R_RISCV_JAL;
   case RISCV::fixup_riscv_branch:

lib/Target/RISCV/MCTargetDesc/RISCVFixupKinds.h

   // fixup_riscv_got_hi20 - 20-bit fixup corresponding to got_pcrel_hi(foo) for
   // instructions like auipc
   fixup_riscv_got_hi20,
+  // fixup_riscv_tls_got_hi20 - 20-bit fixup corresponding to tls_got_hi(foo)
+  // for instructions like auipc
+  fixup_riscv_tls_got_hi20,
+  // fixup_riscv_tprel_hi20 - 20-bit fixup corresponding to tprel_hi(foo) for
+  // instructions like lui
+  fixup_riscv_tprel_hi20,
+  // fixup_riscv_tprel_lo12_i - 12-bit fixup corresponding to tprel_lo(foo) for
+  // instructions like addi
+  fixup_riscv_tprel_lo12_i,
+  // fixup_riscv_tprel_lo12_s - 12-bit fixup corresponding to tprel_lo(foo) for
+  // the S-type store instructions
+  fixup_riscv_tprel_lo12_s,
+  // fixup_riscv_tprel_add - A fixup corresponding to %tprel_add(foo) for the
+  // add_tls instruction. Used to provide a hint to the linker.
+  fixup_riscv_tprel_add,
+  // fixup_riscv_tls_gd_hi20 - 20-bit fixup corresponding to gd_hi(foo) for
+  // instructions like auipc
+  fixup_riscv_tls_gd_hi20,
   // fixup_riscv_jal - 20-bit fixup for symbol references in the jal
   // instruction
   fixup_riscv_jal,

lib/Target/RISCV/MCTargetDesc/RISCVMCCodeEmitter.cpp

                           SmallVectorImpl<MCFixup> &Fixups,
                           const MCSubtargetInfo &STI) const;
 
+  void expandAddTPRel(const MCInst &MI, raw_ostream &OS,
+                      SmallVectorImpl<MCFixup> &Fixups,
+                      const MCSubtargetInfo &STI) const;
+
   /// TableGen'erated function for getting the binary encoding for an
   /// instruction.
   uint64_t getBinaryCodeForInstr(const MCInst &MI,

   support::endian::write(OS, Binary, support::little);
 }
 
+// Expand PseudoAddTPRel to a simple ADD with the correct relocation.
+void RISCVMCCodeEmitter::expandAddTPRel(const MCInst &MI, raw_ostream &OS,
+                                        SmallVectorImpl<MCFixup> &Fixups,
+                                        const MCSubtargetInfo &STI) const {
+  MCOperand DestReg = MI.getOperand(0);
+  MCOperand SrcReg = MI.getOperand(1);
+  MCOperand TPReg = MI.getOperand(2);
+  assert(TPReg.isReg() && TPReg.getReg() == RISCV::X4 &&
+         "Expected thread pointer as second input to TP-relative add");
+
+  MCOperand SrcSymbol = MI.getOperand(3);
+  assert(SrcSymbol.isExpr() &&
+         "Expected expression as third input to TP-relative add");
+
+  const RISCVMCExpr *Expr = dyn_cast<RISCVMCExpr>(SrcSymbol.getExpr());
+  assert(Expr && Expr->getKind() == RISCVMCExpr::VK_RISCV_TPREL_ADD &&
+         "Expected tprel_add relocation on TP-relative symbol");
+
+  // Emit the correct tprel_add relocation for the symbol.
+  Fixups.push_back(MCFixup::create(
+      0, Expr, MCFixupKind(RISCV::fixup_riscv_tprel_add), MI.getLoc()));
+
+  // Emit a normal ADD instruction with the given operands.
+  MCInst TmpInst = MCInstBuilder(RISCV::ADD)
+                       .addOperand(DestReg)
+                       .addOperand(SrcReg)
+                       .addOperand(TPReg);
+  uint32_t Binary = getBinaryCodeForInstr(TmpInst, Fixups, STI);
+  support::endian::write(OS, Binary, support::little);
+}
+
 void RISCVMCCodeEmitter::encodeInstruction(const MCInst &MI, raw_ostream &OS,
                                            SmallVectorImpl<MCFixup> &Fixups,
                                            const MCSubtargetInfo &STI) const {

     return;
   }
 
+  if (MI.getOpcode() == RISCV::PseudoAddTPRel) {
+    expandAddTPRel(MI, OS, Fixups, STI);
+    MCNumEmitted += 1;
+    return;
+  }
+
   switch (Size) {
   default:
     llvm_unreachable("Unhandled encodeInstruction length!");

     switch (RVExpr->getKind()) {
     case RISCVMCExpr::VK_RISCV_None:
     case RISCVMCExpr::VK_RISCV_Invalid:
+    case RISCVMCExpr::VK_RISCV_TPREL_ADD:
       llvm_unreachable("Unhandled fixup kind!");
     case RISCVMCExpr::VK_RISCV_LO:
       if (MIFrm == RISCVII::InstFormatI)

     case RISCVMCExpr::VK_RISCV_GOT_HI:
       FixupKind = RISCV::fixup_riscv_got_hi20;
       break;
+    case RISCVMCExpr::VK_RISCV_TLS_GOT_HI:
+      FixupKind = RISCV::fixup_riscv_tls_got_hi20;
+      break;
+    case RISCVMCExpr::VK_RISCV_TPREL_LO:
+      if (MIFrm == RISCVII::InstFormatI)
+        FixupKind = RISCV::fixup_riscv_tprel_lo12_i;
+      else if (MIFrm == RISCVII::InstFormatS)
+        FixupKind = RISCV::fixup_riscv_tprel_lo12_s;
+      else
+        llvm_unreachable(
+            "VK_RISCV_TPREL_LO used with unexpected instruction format");
+      break;
+    case RISCVMCExpr::VK_RISCV_TPREL_HI:
+      FixupKind = RISCV::fixup_riscv_tprel_hi20;
+      break;
+    case RISCVMCExpr::VK_RISCV_TLS_GD_HI:
+      FixupKind = RISCV::fixup_riscv_tls_gd_hi20;
+      break;
     case RISCVMCExpr::VK_RISCV_CALL:
       FixupKind = RISCV::fixup_riscv_call;
       break;

lib/Target/RISCV/MCTargetDesc/RISCVMCExpr.h

     VK_RISCV_PCREL_LO,
     VK_RISCV_PCREL_HI,
     VK_RISCV_GOT_HI,
+    VK_RISCV_TLS_GOT_HI,
+    VK_RISCV_TPREL_LO,
+    VK_RISCV_TPREL_HI,
+    VK_RISCV_TPREL_ADD,
+    VK_RISCV_TLS_GD_HI,
     VK_RISCV_CALL,
     VK_RISCV_CALL_PLT,
     VK_RISCV_Invalid

     return getSubExpr()->findAssociatedFragment();
   }
 
-  // There are no TLS RISCVMCExprs at the moment.
-  void fixELFSymbolsInTLSFixups(MCAssembler &Asm) const override {}
+  void fixELFSymbolsInTLSFixups(MCAssembler &Asm) const override;
 
   bool evaluateAsConstant(int64_t &Res) const;
 

lib/Target/RISCV/MCTargetDesc/RISCVMCExpr.cpp

 
 #include "RISCV.h"
 #include "RISCVMCExpr.h"
+#include "llvm/BinaryFormat/ELF.h"
 #include "llvm/MC/MCAssembler.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCStreamer.h"

     case VK_RISCV_PCREL_LO:
     case VK_RISCV_PCREL_HI:
     case VK_RISCV_GOT_HI:
+    case VK_RISCV_TLS_GOT_HI:
+    case VK_RISCV_TPREL_LO:
+    case VK_RISCV_TPREL_HI:
+    case VK_RISCV_TPREL_ADD:
+    case VK_RISCV_TLS_GD_HI:
       return false;
     }
   }

       .Case("pcrel_lo", VK_RISCV_PCREL_LO)
       .Case("pcrel_hi", VK_RISCV_PCREL_HI)
       .Case("got_pcrel_hi", VK_RISCV_GOT_HI)
+      .Case("tls_got_hi", VK_RISCV_TLS_GOT_HI)
+      .Case("tprel_lo", VK_RISCV_TPREL_LO)
+      .Case("tprel_hi", VK_RISCV_TPREL_HI)
+      .Case("tprel_add", VK_RISCV_TPREL_ADD)
+      .Case("gd_hi", VK_RISCV_TLS_GD_HI)
       .Default(VK_RISCV_Invalid);
 }
 

     return "pcrel_hi";
   case VK_RISCV_GOT_HI:
     return "got_pcrel_hi";
+  case VK_RISCV_TLS_GOT_HI:
+    return "tls_got_hi";
+  case VK_RISCV_TPREL_LO:
+    return "tprel_lo";
+  case VK_RISCV_TPREL_HI:
+    return "tprel_hi";
+  case VK_RISCV_TPREL_ADD:
+    return "tprel_add";
+  case VK_RISCV_TLS_GD_HI:
+    return "gd_hi";
   }
 }
 
+static void fixELFSymbolsInTLSFixupsImpl(const MCExpr *Expr, MCAssembler &Asm) {
+  switch (Expr->getKind()) {
+  case MCExpr::Target:
+    llvm_unreachable("Can't handle nested target expression");
+    break;
+  case MCExpr::Constant:
+    break;
+
+  case MCExpr::Binary: {
+    const MCBinaryExpr *BE = cast<MCBinaryExpr>(Expr);
+    fixELFSymbolsInTLSFixupsImpl(BE->getLHS(), Asm);
+    fixELFSymbolsInTLSFixupsImpl(BE->getRHS(), Asm);
+    break;
+  }
+
+  case MCExpr::SymbolRef: {
+    // We're known to be under a TLS fixup, so any symbol should be
+    // modified. There should be only one.
+    const MCSymbolRefExpr &SymRef = *cast<MCSymbolRefExpr>(Expr);
+    cast<MCSymbolELF>(SymRef.getSymbol()).setType(ELF::STT_TLS);
+    break;
+  }
+
+  case MCExpr::Unary:
+    fixELFSymbolsInTLSFixupsImpl(cast<MCUnaryExpr>(Expr)->getSubExpr(), Asm);
+    break;
+  }
+}
+
+void RISCVMCExpr::fixELFSymbolsInTLSFixups(MCAssembler &Asm) const {
+  switch (getKind()) {
+  default:
+    return;
+  case VK_RISCV_TLS_GOT_HI:
+  case VK_RISCV_TPREL_HI:
+  case VK_RISCV_TLS_GD_HI:
+    break;
+  }
+
+  fixELFSymbolsInTLSFixupsImpl(getSubExpr(), Asm);
+}
+
 bool RISCVMCExpr::evaluateAsConstant(int64_t &Res) const {
   MCValue Value;
 
   if (Kind == VK_RISCV_PCREL_HI || Kind == VK_RISCV_PCREL_LO ||
-      Kind == VK_RISCV_GOT_HI || Kind == VK_RISCV_CALL ||
-      Kind == VK_RISCV_CALL_PLT)
+      Kind == VK_RISCV_GOT_HI || Kind == VK_RISCV_TLS_GOT_HI ||
+      Kind == VK_RISCV_TPREL_HI || Kind == VK_RISCV_TPREL_LO ||
+      Kind == VK_RISCV_TPREL_ADD || Kind == VK_RISCV_TLS_GD_HI ||
+      Kind == VK_RISCV_CALL || Kind == VK_RISCV_CALL_PLT)
     return false;
 
   if (!getSubExpr()->evaluateAsRelocatable(Value, nullptr, nullptr))

lib/Target/RISCV/RISCVISelLowering.h

   template <class NodeTy>
   SDValue getAddr(NodeTy *N, SelectionDAG &DAG, unsigned Flags = 0) const;
 
+  SDValue getStaticTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG,
+                           bool UseGOT) const;
+  SDValue getDynamicTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG,
+                            bool UseGOT) const;
+
   SDValue lowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
+  SDValue lowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerSELECT(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerVASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;

lib/Target/RISCV/RISCVISelLowering.cpp

   setOperationAction(ISD::BlockAddress, XLenVT, Custom);
   setOperationAction(ISD::ConstantPool, XLenVT, Custom);
 
+  setOperationAction(ISD::GlobalTLSAddress, XLenVT, Custom);
+
   if (Subtarget.hasStdExtA()) {
     setMaxAtomicSizeInBitsSupported(Subtarget.getXLen());
     setMinCmpXchgSizeInBits(32);

     return lowerBlockAddress(Op, DAG);
   case ISD::ConstantPool:
     return lowerConstantPool(Op, DAG);
+  case ISD::GlobalTLSAddress:
+    return lowerGlobalTLSAddress(Op, DAG);
   case ISD::SELECT:
     return lowerSELECT(Op, DAG);
   case ISD::VASTART:

   return getAddr(N, DAG);
 }
 
+SDValue RISCVTargetLowering::getStaticTLSAddr(GlobalAddressSDNode *N,
+                                              SelectionDAG &DAG,
+                                              bool UseGOT) const {
+  SDLoc DL(N);
+  EVT Ty = getPointerTy(DAG.getDataLayout());
+  const GlobalValue *GV = N->getGlobal();
+  MVT XLenVT = Subtarget.getXLenVT();
+
+  if (UseGOT) {
+    // Use PC-relative addressing to access the GOT for this TLS symbol, then
+    // load the address from the GOT and add the thread pointer. This generates
+    // the pattern (add (ld (WrapperPCRel %tls_got_hi(sym))) tp)
+    SDValue Addr =
+        DAG.getTargetGlobalAddress(GV, DL, Ty, 0, RISCVII::MO_TLS_GOT_HI);
+    SDValue Wrapper = DAG.getNode(RISCVISD::WRAPPER_PCREL, DL, Ty, Addr);
+    SDValue Base =
+        DAG.getLoad(Ty, DL, DAG.getEntryNode(), Wrapper, MachinePointerInfo());
+    SDValue TPReg = DAG.getRegister(RISCV::X4, XLenVT);
+    return DAG.getNode(ISD::ADD, DL, Ty, Base, TPReg);
+  }
+
+  // Generate a sequence for accessing the address relative to the thread
+  // pointer, with the appropriate adjustment for the thread pointer offset.
+  // This generates the pattern
+  // (add (add_tprel (lui %tprel_hi(sym)) tp %tprel_add(sym)) %tprel_lo(sym))
+  SDValue AddrHi =
+      DAG.getTargetGlobalAddress(GV, DL, Ty, 0, RISCVII::MO_TPREL_HI);
+  SDValue AddrAdd =
+      DAG.getTargetGlobalAddress(GV, DL, Ty, 0, RISCVII::MO_TPREL_ADD);
+  SDValue AddrLo =
+      DAG.getTargetGlobalAddress(GV, DL, Ty, 0, RISCVII::MO_TPREL_LO);
+
+  SDValue MNHi = SDValue(DAG.getMachineNode(RISCV::LUI, DL, Ty, AddrHi), 0);
+  SDValue TPReg = DAG.getRegister(RISCV::X4, XLenVT);
+  SDValue MNAdd = SDValue(
+      DAG.getMachineNode(RISCV::PseudoAddTPRel, DL, Ty, MNHi, TPReg, AddrAdd),
+      0);
+  return SDValue(DAG.getMachineNode(RISCV::ADDI, DL, Ty, MNAdd, AddrLo), 0);
+}
+
+SDValue RISCVTargetLowering::getDynamicTLSAddr(GlobalAddressSDNode *N,
+                                               SelectionDAG &DAG,
+                                               bool UseGOT) const {
+  SDLoc DL(N);
+  EVT Ty = getPointerTy(DAG.getDataLayout());
+  IntegerType *CallTy = Type::getIntNTy(*DAG.getContext(), Ty.getSizeInBits());
+  const GlobalValue *GV = N->getGlobal();
+
+  // Use a PC-relative addressing mode to access the global dynamic GOT address.
+  // This generates the pattern (WrapperPCRel %gd_hi(sym)).
+  SDValue Addr = DAG.getTargetGlobalAddress(GV, DL, Ty, 0, RISCVII::MO_GD_HI);
+  SDValue Wrapper = DAG.getNode(RISCVISD::WRAPPER_PCREL, DL, Ty, Addr);
+
+  // Prepare argument list to generate call.
+  ArgListTy Args;
+  ArgListEntry Entry;
+  Entry.Node = Wrapper;
+  Entry.Ty = CallTy;
+  Args.push_back(Entry);
+
+  // Setup call to __tls_get_addr.
+  TargetLowering::CallLoweringInfo CLI(DAG);
+  CLI.setDebugLoc(DL)
+      .setChain(DAG.getEntryNode())
+      .setLibCallee(CallingConv::C, CallTy,
+                    DAG.getExternalSymbol("__tls_get_addr", Ty),
+                    std::move(Args));
+
+  return LowerCallTo(CLI).first;
+}
+
+SDValue RISCVTargetLowering::lowerGlobalTLSAddress(SDValue Op,
+                                                   SelectionDAG &DAG) const {
+  SDLoc DL(Op);
+  EVT Ty = Op.getValueType();
+  GlobalAddressSDNode *N = cast<GlobalAddressSDNode>(Op);
+  int64_t Offset = N->getOffset();
+  MVT XLenVT = Subtarget.getXLenVT();
+
+  // Non-PIC TLS lowering should always use the LocalExec model.
+  TLSModel::Model Model = isPositionIndependent()
+                              ? getTargetMachine().getTLSModel(N->getGlobal())
+                              : TLSModel::LocalExec;
+
+  SDValue Addr;
+  switch (Model) {
+  default:
+    report_fatal_error("Unsupported TLS model");
+  case TLSModel::LocalExec:
+    Addr = getStaticTLSAddr(N, DAG, /*UseGOT=*/false);
+    break;
+  case TLSModel::InitialExec:
+    Addr = getStaticTLSAddr(N, DAG, /*UseGOT=*/true);
+    break;
+  case TLSModel::LocalDynamic:
+    Addr = getDynamicTLSAddr(N, DAG, /*UseGOT=*/false);
+    break;
+  case TLSModel::GeneralDynamic:
+    Addr = getDynamicTLSAddr(N, DAG, /*UseGOT=*/true);
+    break;
+  }
+
+  // In order to maximise the opportunity for common subexpression elimination,
+  // emit a separate ADD node for the global address offset instead of folding
+  // it in the global address node. Later peephole optimisations may choose to
+  // fold it back in when profitable.
+  if (Offset != 0)
+    return DAG.getNode(ISD::ADD, DL, Ty, Addr,
+                       DAG.getConstant(Offset, DL, XLenVT));
+  return Addr;
+}
+
 SDValue RISCVTargetLowering::lowerSELECT(SDValue Op, SelectionDAG &DAG) const {
   SDValue CondV = Op.getOperand(0);
   SDValue TrueV = Op.getOperand(1);

lib/Target/RISCV/RISCVInstrInfo.td

   let ParserMatchClass = BareSymbol;
 }
 
+def TPRelAddSymbol : AsmOperandClass {
+  let Name = "TPRelAddSymbol";
+  let RenderMethod = "addImmOperands";
+  let DiagnosticType = "InvalidTPRelAddSymbol";
+  let ParserMethod = "parseTPRelAddSymbol";
+}
+
+// A bare symbol with the %tprel_add variant.
+def tprel_add_symbol : Operand<XLenVT> {
+  let ParserMatchClass = TPRelAddSymbol;
+}
+
 def CSRSystemRegister : AsmOperandClass {
   let Name = "CSRSystemRegister";
   let ParserMethod = "parseCSRSystemRegister";

 def : PatGprGpr<shiftop<srl>, SRL>;
 def : PatGprGpr<shiftop<sra>, SRA>;
 
+// This is a special case of the ADD instruction used to facilitate the use of a
+// fourth operand to emit a relocation on a symbol relating to this instruction.
+// The relocation does not affect any bits of the instruction itself but is used
+// as a hint to the linker.
+let hasSideEffects = 0, mayLoad = 0, mayStore = 0, isCodeGenOnly = 0 in
+def PseudoAddTPRel : Pseudo<(outs GPR:$rd),
+                            (ins GPR:$rs1, GPR:$rs2, tprel_add_symbol:$src), [],
+                            "add", "$rd, $rs1, $rs2, $src">;
+
 /// FrameIndex calculations
 
 def : Pat<(add (i32 AddrFI:$Rs), simm12:$imm12),

 
 def : Pat<(WrapperPCRel tglobaladdr:$addr),
           (PseudoAddrPCRel tglobaladdr:$addr)>;
+def : Pat<(WrapperPCRel tglobaltlsaddr:$addr),
+          (PseudoAddrPCRel tglobaltlsaddr:$addr)>;
 def : Pat<(WrapperPCRel tblockaddress:$addr),
           (PseudoAddrPCRel tblockaddress:$addr)>;
 def : Pat<(WrapperPCRel tconstpool:$addr),

lib/Target/RISCV/RISCVMCInstLower.cpp

   case RISCVII::MO_GOT_HI:
     Kind = RISCVMCExpr::VK_RISCV_GOT_HI;
     break;
+  case RISCVII::MO_TLS_GOT_HI:
+    Kind = RISCVMCExpr::VK_RISCV_TLS_GOT_HI;
+    break;
+  case RISCVII::MO_TPREL_LO:
+    Kind = RISCVMCExpr::VK_RISCV_TPREL_LO;
+    break;
+  case RISCVII::MO_TPREL_HI:
+    Kind = RISCVMCExpr::VK_RISCV_TPREL_HI;
+    break;
+  case RISCVII::MO_TPREL_ADD:
+    Kind = RISCVMCExpr::VK_RISCV_TPREL_ADD;
+    break;
+  case RISCVII::MO_GD_HI:
+    Kind = RISCVMCExpr::VK_RISCV_TLS_GD_HI;
+    break;
   case RISCVII::MO_PLT:
     Kind = RISCVMCExpr::VK_RISCV_CALL_PLT;
     break;

lib/Target/RISCV/Utils/RISCVBaseInfo.h

   MO_PCREL_LO,
   MO_PCREL_HI,
   MO_GOT_HI,
+  MO_TLS_GOT_HI,
+  MO_TPREL_LO,
+  MO_TPREL_HI,
+  MO_TPREL_ADD,
+  MO_GD_HI,
   MO_PLT,
 };
 } // namespace RISCVII

test/CodeGen/RISCV/tls-models.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -relocation-model=pic < %s \
+; RUN:     | FileCheck -check-prefix=RV32-PIC %s
+
+; Check that TLS symbols are lowered correctly based on the specified
+; model.
+
+@unspecified = thread_local global i32 42
+@ld = thread_local(localdynamic) global i32 42
+@ie = thread_local(initialexec) global i32 42
+@le = thread_local(localexec) global i32 42
+
+
+; No model specified
+
+define i32* @f1() {
+; RV32-PIC-LABEL: f1:
+; RV32-PIC:       # %bb.0: # %entry
+; RV32-PIC-NEXT:    addi sp, sp, -16
+; RV32-PIC-NEXT:    sw ra, 12(sp)
+; RV32-PIC-NEXT:  .LBB0_1: # %entry
+; RV32-PIC-NEXT:    # Label of block must be emitted
+; RV32-PIC-NEXT:    auipc a0, %gd_hi(unspecified)
+; RV32-PIC-NEXT:    addi a0, a0, %pcrel_lo(.LBB0_1)
+; RV32-PIC-NEXT:    call __tls_get_addr@plt
+; RV32-PIC-NEXT:    lw ra, 12(sp)
+; RV32-PIC-NEXT:    addi sp, sp, 16
+; RV32-PIC-NEXT:    ret
+entry:
+  ret i32* @unspecified
+}
+
+
+; localdynamic specified
+
+define i32* @f2() {
+; RV32-PIC-LABEL: f2:
+; RV32-PIC:       # %bb.0: # %entry
+; RV32-PIC-NEXT:    addi sp, sp, -16
+; RV32-PIC-NEXT:    sw ra, 12(sp)
+; RV32-PIC-NEXT:  .LBB1_1: # %entry
+; RV32-PIC-NEXT:    # Label of block must be emitted
+; RV32-PIC-NEXT:    auipc a0, %gd_hi(ld)
+; RV32-PIC-NEXT:    addi a0, a0, %pcrel_lo(.LBB1_1)
+; RV32-PIC-NEXT:    call __tls_get_addr@plt
+; RV32-PIC-NEXT:    lw ra, 12(sp)
+; RV32-PIC-NEXT:    addi sp, sp, 16
+; RV32-PIC-NEXT:    ret
+entry:
+  ret i32* @ld
+}
+
+
+; initialexec specified
+
+define i32* @f3() {
+; RV32-PIC-LABEL: f3:
+; RV32-PIC:       # %bb.0: # %entry
+; RV32-PIC-NEXT:  .LBB2_1: # %entry
+; RV32-PIC-NEXT:    # Label of block must be emitted
+; RV32-PIC-NEXT:    auipc a0, %tls_got_hi(ie)
+; RV32-PIC-NEXT:    addi a0, a0, %pcrel_lo(.LBB2_1)
+; RV32-PIC-NEXT:    lw a0, 0(a0)
+; RV32-PIC-NEXT:    add a0, a0, tp
+; RV32-PIC-NEXT:    ret
+entry:
+  ret i32* @ie
+}
+
+
+; localexec specified
+
+define i32* @f4() {
+; RV32-PIC-LABEL: f4:
+; RV32-PIC:       # %bb.0: # %entry
+; RV32-PIC-NEXT:    lui a0, %tprel_hi(le)
+; RV32-PIC-NEXT:    add a0, a0, tp, %tprel_add(le)
+; RV32-PIC-NEXT:    addi a0, a0, %tprel_lo(le)
+; RV32-PIC-NEXT:    ret
+entry:
+  ret i32* @le
+}

test/MC/RISCV/rv32d-invalid.s

 
 # Out of range immediates
 ## simm12
-fld ft1, -2049(a0) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-fsd ft2, 2048(a1) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+fld ft1, -2049(a0) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+fsd ft2, 2048(a1) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Memory operand not formatted correctly
-fld ft1, a0, -200 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-fsd ft2, a1, 100 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+fld ft1, a0, -200 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+fsd ft2, a1, 100 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Invalid register names
 fld ft15, 100(a0) # CHECK: :[[@LINE]]:5: error: invalid operand for instruction

test/MC/RISCV/rv32f-invalid.s

 
 # Out of range immediates
 ## simm12
-flw ft1, -2049(a0) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-fsw ft2, 2048(a1) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+flw ft1, -2049(a0) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+fsw ft2, 2048(a1) # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Memory operand not formatted correctly
-flw ft1, a0, -200 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-fsw ft2, a1, 100 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+flw ft1, a0, -200 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+fsw ft2, a1, 100 # CHECK: :[[@LINE]]:10: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Invalid register names
 flw ft15, 100(a0) # CHECK: :[[@LINE]]:5: error: invalid operand for instruction

test/MC/RISCV/rv32i-invalid.s

 csrrci x0, 43, -90 # CHECK: :[[@LINE]]:16: error: immediate must be an integer in the range [0, 31]
 
 ## simm12
-ori a0, a1, -2049 # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-andi ra, sp, 2048 # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+ori a0, a1, -2049 # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+andi ra, sp, 2048 # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 ## uimm12
 csrrw a0, -1, a0 # CHECK: :[[@LINE]]:11: error: immediate must be an integer in the range [0, 4095]

 bgeu t0, t1, -13 # CHECK: :[[@LINE]]:14: error: immediate must be a multiple of 2 bytes in the range [-4096, 4094]
 
 ## uimm20
-lui a0, -1 # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-lui s0, 1048576 # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-auipc zero, -0xf # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
+lui a0, -1 # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+lui s0, 1048576 # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+auipc zero, -0xf # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
 
 ## simm21_lsb0
 jal gp, -1048578 # CHECK: :[[@LINE]]:9: error: immediate must be a multiple of 2 bytes in the range [-1048576, 1048574]

 csrrci x0, 43, %pcrel_lo(d) # CHECK: :[[@LINE]]:16: error: immediate must be an integer in the range [0, 31]
 
 ## simm12
-ori a0, a1, %hi(foo) # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-andi ra, sp, %pcrel_hi(123) # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-xori a2, a3, %hi(345) # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-add a1, a2, (a3) # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-add a1, a2, foo # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+ori a0, a1, %hi(foo) # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+andi ra, sp, %pcrel_hi(123) # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+xori a2, a3, %hi(345) # CHECK: :[[@LINE]]:14: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+add a1, a2, (a3) # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+add a1, a2, foo # CHECK: :[[@LINE]]:13: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 ## uimm12
 csrrw a0, %lo(1), a0 # CHECK: :[[@LINE]]:11: error: immediate must be an integer in the range [0, 4095]

 bgeu t0, t1, %pcrel_lo(d) # CHECK: :[[@LINE]]:14: error: immediate must be a multiple of 2 bytes in the range [-4096, 4094]
 
 ## uimm20
-lui a0, %lo(1) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-auipc a1, %lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
+lui a0, %lo(1) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+auipc a1, %lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
 
 ## simm21_lsb0
 jal gp, %lo(1) # CHECK: :[[@LINE]]:9: error: immediate must be a multiple of 2 bytes in the range [-1048576, 1048574]

 # Bare symbol names when an operand modifier is required and unsupported 
 # operand modifiers.
 
-lui a0, foo # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-lui a0, %lo(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-lui a0, %pcrel_lo(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
-lui a0, %pcrel_hi(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi() modifier or an integer in the range [0, 1048575]
+lui a0, foo # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+lui a0, %lo(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+lui a0, %pcrel_lo(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
+lui a0, %pcrel_hi(foo) # CHECK: :[[@LINE]]:9: error: operand must be a symbol with %hi/%tprel_hi() modifier or an integer in the range [0, 1048575]
 
-auipc a0, foo # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
-auipc a0, %lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
-auipc a0, %hi(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
-auipc a0, %pcrel_lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi() modifier or an integer in the range [0, 1048575]
+auipc a0, foo # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
+auipc a0, %lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
+auipc a0, %hi(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
+auipc a0, %pcrel_lo(foo) # CHECK: :[[@LINE]]:11: error: operand must be a symbol with %pcrel_hi/%got_pcrel_hi/%tls_got_hi/%gd_hi() modifier or an integer in the range [0, 1048575]
+
+# TP-relative symbol names require a %tprel_add modifier.
+add a0, a0, tp, zero # CHECK: :[[@LINE]]:17: error: operand must be a symbol with %tprel_add modifier
+add a0, a0, tp, %hi(foo) # CHECK: :[[@LINE]]:18: error: expected %tprel_add modifier
 
 # Unrecognized operand modifier
 addi t0, sp, %modifer(255) # CHECK: :[[@LINE]]:15: error: unrecognized operand modifier

 sub t0, t2, 1 # CHECK: :[[@LINE]]:13: error: invalid operand for instruction
 
 # Too many operands
-add ra, zero, zero, zero # CHECK: :[[@LINE]]:21: error: invalid operand for instruction
 sltiu s2, s3, 0x50, 0x60 # CHECK: :[[@LINE]]:21: error: invalid operand for instruction
 
 # Memory operand not formatted correctly
-lw a4, a5, 111 # CHECK: :[[@LINE]]:8: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+lw a4, a5, 111 # CHECK: :[[@LINE]]:8: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Too few operands
 ori a0, a1 # CHECK: :[[@LINE]]:1: error: too few operands for instruction

test/MC/RISCV/rv32i-valid.s

 # CHECK-OBJ: lui a0, 0
 # CHECK-OBJ: R_RISCV_HI20 foo
 lui a0, %hi(foo)
+# CHECK-ASM: lui a0, %tprel_hi(foo)
+# CHECK-ASM: encoding: [0x37,0bAAAA0101,A,A]
+# CHECK-OBJ: lui a0, 0
+# CHECK-OBJ: R_RISCV_TPREL_HI20 foo
+lui a0, %tprel_hi(foo)
 
 # CHECK-ASM-AND-OBJ: auipc a0, 2
 # CHECK-ASM: encoding: [0x17,0x25,0x00,0x00]

 # CHECK-OBJ: auipc a0, 0
 # CHECK-OBJ: R_RISCV_GOT_HI20 foo
 auipc a0, %got_pcrel_hi(foo)
+# CHECK-ASM: auipc a0, %tls_got_hi(foo)
+# CHECK-ASM: encoding: [0x17,0bAAAA0101,A,A]
+# CHECK-OBJ: auipc a0, 0
+# CHECK-OBJ: R_RISCV_TLS_GOT_HI20 foo
+auipc a0, %tls_got_hi(foo)
+# CHECK-ASM: auipc a0, %gd_hi(foo)
+# CHECK-ASM: encoding: [0x17,0bAAAA0101,A,A]
+# CHECK-OBJ: auipc a0, 0
+# CHECK-OBJ: R_RISCV_TLS_GD_HI20 foo
+auipc a0, %gd_hi(foo)
 
 # CHECK-ASM-AND-OBJ: jal a2, 1048574
 # CHECK-ASM: encoding: [0x6f,0xf6,0xff,0x7f]

 # CHECK-ASM-AND-OBJ: add ra, zero, zero
 # CHECK-ASM: encoding: [0xb3,0x00,0x00,0x00]
 add x1, x0, x0
+# CHECK-ASM: add a0, a0, tp, %tprel_add(foo)
+# CHECK-ASM: encoding: [0x33,0x05,0x45,0x00]
+# CHECK-OBJ: add a0, a0, tp
+# CHECK-OBJ: R_RISCV_TPREL_ADD foo
+add a0, a0, tp, %tprel_add(foo)
 # CHECK-ASM-AND-OBJ: sub t0, t2, t1
 # CHECK-ASM: encoding: [0xb3,0x82,0x63,0x40]
 sub t0, t2, t1

test/MC/RISCV/rv64i-invalid.s

 sraiw a0, a0, -19 # CHECK: :[[@LINE]]:15: error: immediate must be an integer in the range [0, 31]
 
 ## simm12
-addiw a0, a1, -2049 # CHECK: :[[@LINE]]:15: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
-ld ra, 2048(sp) # CHECK: :[[@LINE]]:8: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+addiw a0, a1, -2049 # CHECK: :[[@LINE]]:15: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
+ld ra, 2048(sp) # CHECK: :[[@LINE]]:8: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]
 
 # Illegal operand modifier
 ## uimm5

 sraiw a0, a0, %hi(2) # CHECK: :[[@LINE]]:15: error: immediate must be an integer in the range [0, 31]
 
 ## simm12
-addiw a0, a1, %hi(foo) # CHECK: :[[@LINE]]:15: error: operand must be a symbol with %lo/%pcrel_lo modifier or an integer in the range [-2048, 2047]
+addiw a0, a1, %hi(foo) # CHECK: :[[@LINE]]:15: error: operand must be a symbol with %lo/%pcrel_lo/%tprel_lo modifier or an integer in the range [-2048, 2047]