Add "RPRED" mechanism for relocation predicates. (5% speedup for `ld.lld -O0`)

[silvas]

Summary

In various places in LLD's hot loops, we have expressions of the form "E == R_FOO || E == R_BAR || ..." (E is a RelExpr).

Some of these expressions are quite long, and even though they usually go just a very small number of ways[1] and so should be well predicted, they can still occupy branch predictor resources harming other parts of the code, or they won't be predicted well if they overflow branch predictor resources or if the branches are too dense and the branch predictor can't track them all (the compiler can in theory avoid this, at a cost in text size). And some of these expressions are so large and executed so frequently that even when well-predicted they probably still have a nontrivial cost.

This patch aims to accelerate these operations by assigning the RelExpr enum values in such a way that these predicates can be written as a single expression E & RPRED_FOO. The patch also converts a couple use cases in the hot path as a proof of concept (it is easy to add or remove predicates, so we can decide later which predicates to accelerate, based on benchmarking).

The predicates are encoded in the high bits of the RelExpr value. There is one high bit of the RelExpr value per predicate. So each predicate can become a single bit test.

This speedup should be pretty portable. The cost of these simple bit tests is independent of:

• the target we are linking for
• the distribution of RelExpr's for a given link (which can depend on how the input files were compiled)
• what compiler was used to compile LLD (it is just a simple bit test; hopefully the compiler gets it right!)
• adding new target-dependent relocations (e.g. needsPlt doesn't pay any extra cost checking R_PPC_PLT_OPD on x86-64 builds)

Rafael, I tried running this through your benchmarks that I grabbed from your google drive link, but it uses a bunch of stuff that seem to be local to your machine. I was able to get clang-fsds working though. I did some rough measurements on clang-fsds and this patch gives over about 4% speedup for a regular -O1 link, about 2.5% for -O3 --gc-sections and over 5% for -O0. Sorry, I don't have my current machine set up for doing really accurate measurements right now; Rafael, if you could run it through your full benchmark suite that would be really nice.

[1] see e.g. the quick analysis in https://reviews.llvm.org/F2621745 (derived from the trace in https://reviews.llvm.org/F2621770)

[silvas]

Add a comment. More comments are probably necessary but this one was particularly bad to have missing.

[silvas]

This time actually update the diff.

[ruiu]

4% speedup in -O1 sounds like a nice improvement.

ELF/Relocations.def

22–24

Are these numbers correct? It seems that

PRED_RELAX_MARKER == PRED_ALWAYS_CONSTANT | PRED_REFERS_TO_GOT

holds, which is odd. Don't you need power-of-twos?

[silvas]

ELF/Relocations.def

22–24

The shifting is done in construction of the enum. Sorry, I should have some comments in this file.

enum RelExprPredicate {
#define PREDICATE(Name, PredicateIndex)                                        \
  Name = (1 << (PredicateIndex + NumRelExprIndexBits)),
#include "Relocations.def"
};

[silvas]

Simplify the definition of the predicates to be easier to follow. There is no longer any need to have the predicate definitions in Relocations.def

Rui, let me know if you like this approach better (or have suggestions for a simpler approach).

[silvas]

This time with the right patch.

[silvas]

Mark some comments as done. They aren't relevant with the way that the predicates are currently defined.

[grandinj]

Do the use-sites need to be updated? Won't the compiler already simplify such expressions (just concerned that the use-sites may be come less legible)

Also, for future source code readers, you may want a comment at the definition site of the enum that it is a regular (exclusive) enum and not some kind of flags/bit-field type thing.

[silvas]

In D27145#606147, @grandinj wrote:

Do the use-sites need to be updated? Won't the compiler already simplify such expressions (just concerned that the use-sites may be come less legible)

The use sites are updated by this patch (search for the expressions that look like & RPRED_FOO).

The expressions that are being replaced are of the form x == 7 || x == 9 || x == 35 || x == 12 || .... The best the compiler could do is infer range information (based on the C language rules for enums) and make a lookup table, but I'm not aware of any compiler that will currently do that for this kind of expression.

This patch is a type of data representation optimization that a compiler can't be expected to do, even with whole-program analysis.

The use sites might become less legible, but ideally we can hide each of these operations behind a helper with a useful name, such as refersToGotEntry, needsPlt, etc. so I don't expect it to matter much. I'm not as much of an expert on relocations to come up with good names for each of them, so some of them I have just left as a raw bit test. Hopefully Rafael and others can help identify good names for these (and for the RPRED_* enum values).

Also, for future source code readers, you may want a comment at the definition site of the enum that it is a regular (exclusive) enum and not some kind of flags/bit-field type thing.

Once we settle down on the design, I'll need to do another pass through the code and add some strategic comments.

[joerg]

I'm not convinced that a better switch encoding logic by the compiler can't be done. In fact it seems to be quite reasonable to expect

if (x == C1 || x == C2 || ... || x == C_N)

to be turned into

if (x>=0 && x < 64 && (1ULL << (x&63)) & ((1ULL << C1) | (1ULL << C2) | ....| (1ULL << C_N))

as a general optimisation if all C_i<64.

[silvas]

Joerg, that transformation is interesting (and Hans' new switch lowering has something like that). However, I don't think we should assume that compilers implement that, nor that we can rely on it to generate code as good as the one with this RPRED mechanism.

One issue that I see with that transformation is that it is not a form of canonizalization and so won't kick in until later (or in the extreme case like Hans' switch lowering, in the backend).
So there is a phase ordering issue because, for example, JumpThreading would like to see the single combined expression in order for needsPlt or refersToGotEntry to get properly jump-threaded in the hot relocation loop.
Even worse, profile-guided branch reordering could easily mangle these expressions, causing them to not CSE appropriately (again, the hot relocation loop has precisely this situation; e.g. needsPlt and refersToGotEntry are duplicated).

With this RPRED mechanism we can get reliable results from any compiler.

[joerg]

Your change is IMO a non-trivial uglification of the code and will only one shift at runtime. I'm not sure that justifies this complexity. I would expect it to be trivial to transform the suggested codegen change into a constexpr function with array argument to do the same (and include an assert about the value range of x where necessary.

[ruiu]

It is a legitimate concern that the new code is harder to read, especially when compared to the original simple-minded one.

I wonder if a table lookup will work here. A RelExpr is a small integer, so we can do something like this

static bool refersToGotEntry(RelExpr Expr) {
  return RefersToGotEntry[static_cast<int>(Expr)];
}

where RefersToGotEntry is defined as

bool RefersToGotEntry[R_MAX] = {
  .R_GOT = true, .R_GOT_OFF = true, .R_MIPS_GOT_LOCAL_PAGE = true, ...
}

[silvas]

In D27145#606205, @ruiu wrote:

It is a legitimate concern that the new code is harder to read, especially when compared to the original simple-minded one.

I don't think it actually is so bad. These predicates are rarely modified and usually have a larger semantic meaning like needsPlt or refersToGotEntry. I don't think that most people reading this code actually look at the specific set of RelExpr's being checked very often.

I wonder if a table lookup will work here. A RelExpr is a small integer, so we can do something like this

static bool refersToGotEntry(RelExpr Expr) {
  return RefersToGotEntry[static_cast<int>(Expr)];
}

where RefersToGotEntry is defined as

bool RefersToGotEntry[R_MAX] = {
  .R_GOT = true, .R_GOT_OFF = true, .R_MIPS_GOT_LOCAL_PAGE = true, ...
}

If you want I can try an approach like that. However, the syntax you have there isn't possible in C++. It would probably end up looking something like:

RelExprPredicateTable RefersToGotEntry([](RelExpr Expr) {
  return Expr == R_GOT || Expr == R_GOT_OFF || Expr == R_MIPS_GOT_LOCAL_PAGE ||
         Expr == R_MIPS_GOT_OFF || Expr == R_MIPS_GOT_OFF32 ||
         Expr == R_MIPS_TLSGD || Expr == R_MIPS_TLSLD ||
         Expr == R_GOT_PAGE_PC || Expr == R_GOT_PC || Expr == R_GOT_FROM_END ||
         Expr == R_TLSGD || Expr == R_TLSGD_PC || Expr == R_TLSDESC ||
         Expr == R_TLSDESC_PAGE;
});

The code for RelExprPredicateTable would be some annoying constexpr metaprogramming that calls the lambda on each possible RelExpr value to build the table (and then we have to worry about which compilers support the necessary C++ features to do it; if not all of them do, then it may not be possible or it may have to be done in an ugly way). I'm not sure that would actually be that much more readable.

Do you want me to try it?

Also, it may not be as fast (hard to beat a single AND), but that will have to be measured.

[joerg]

You can build the preciate functions with a constexpr helper? E.g.

static bool refersToGotEntry(RelExpr Expr) {
  if (uint64_t(Expr) > 63)
    return false;
  return (uint64_t(1) << Expr) & buildMask({R_GOT, R_GOT_OFF, ...});
}

where buildMask computes (uint64_t(1) << R_GOT) | (uint64_t(1) << R_GOT_OFF) | ... via constexpr.

I don't think precomputing the shift and passing that around is much help -- it's a single cheap assembler instruction without any memory access. It can also be trivially CSEed. The first check can likely be replaced by an assert, assuming input is sanitzed already here.

[silvas]

In D27145#606254, @joerg wrote:

You can build the preciate functions with a constexpr helper? E.g.

static bool refersToGotEntry(RelExpr Expr) {
  if (uint64_t(Expr) > 63)
    return false;
  return (uint64_t(1) << Expr) & buildMask({R_GOT, R_GOT_OFF, ...});
}

where buildMask computes (uint64_t(1) << R_GOT) | (uint64_t(1) << R_GOT_OFF) | ... via constexpr.

I don't think precomputing the shift and passing that around is much help -- it's a single cheap assembler instruction without any memory access. It can also be trivially CSEed. The first check can likely be replaced by an assert, assuming input is sanitzed already here.

Oh, nice. Sorry, I misunderstood what you meant. That sounds really nice actually.

[silvas]

Okay, here's what I've come up with:
https://godbolt.org/g/NNSthI

If this looks good then I'll go back and just use this.

[joerg]

That looks good.

[silvas]

I have posted D27156 as an alternative to this patch. The approach that Joerg suggested turned out really well actually. PTAL.

[grandinj]

wow, that is neat.
I wonder if it would be legal to teach the compiler to do this if both sides of the expression were enum values of the same type (or possibly more legal if they were scoped enums) ?

[joerg]

The compiler would generally either have to know the overflow behavior of the left shift or add the one conditional I mentioned earlier. But otherwise I don't see why it can't create the transform.

[ruiu]

It is unfortunate that we cannot write like that. I wish we had the C99 desginated initializer in C++.

ELF/Relocations.h

42–43

I think I prefer writing the definitions inline here in a macro-expanded form rather than in an included file. The file is not included more than once (except for static_assert), so the #include hack does not make much sense. I also want to make code less exciting by writing constants inline.

This code

enum RelExprPredicateBit {
  RPRED_NEEDS_PLT       = 1 << 31,
  RPRED_ALWAYS_CONSTANT = 1 << 30,
  RPRED_REFERS_TO_GOT   = 1 << 29,
  RPRED_RELAX_MARKER    = 1 << 28,
  RPRED_RELATIVE        = 1 << 27,
};

is boring but I believe it's easier to read. Likewise, I prefer a style like this because it is obvious what it is doing without looking at other places.

enum RelExpr {
  R_ABS = 1,
  R_GOT = 2 | RPRED_REFERS_TO_GOT,
  R_GOTONLY_PC = 3,
  R_GOTONLY_PC_FROM_END = 4,
  R_GOTREL = 5 | RPRED_RELATIVE,
  R_GOTREL_FROM_END = 6 | RPRED_RELATIVE,
  R_GOT_FROM_END = 7 | RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT,
  ...
};

[silvas]

Abandoning now that D27156 has landed.

ELF/Relocations.h

 
 namespace lld {
 namespace elf {
 class SymbolBody;
 class InputSectionData;
 template <class ELFT> class InputSection;
 template <class ELFT> class InputSectionBase;
 
+// There must be less than 2^PredicateShift RelExpr's.
+// Bits beyond the low PredicateShift bits each encode a predicate
+// that lets us efficiently do a test of the form
+// `E == R_FOO || E == R_BAR || ...` with a single expression `E & RPRED_BAZ`.
+enum { PredicateShift = 6 };
+
+constexpr uint32_t predicateBit(int N) { return 1 << (N + PredicateShift); }
+
+enum RelExprPredicateBit {
+  RPRED_NEEDS_PLT       = predicateBit(0),
+  RPRED_ALWAYS_CONSTANT = predicateBit(1),
+  RPRED_REFERS_TO_GOT   = predicateBit(2),
+  RPRED_RELAX_MARKER    = predicateBit(3),
+  RPRED_RELATIVE        = predicateBit(4),
+};
+
 // List of target-independent relocation types. Relocations read
 // from files are converted to these types so that the main code
 // doesn't have to know about architecture-specific details.
 enum RelExpr {
-  R_ABS,
-  R_GOT,
-  R_GOTONLY_PC,
-  R_GOTONLY_PC_FROM_END,
-  R_GOTREL,
-  R_GOTREL_FROM_END,
-  R_GOT_FROM_END,
-  R_GOT_OFF,
-  R_GOT_PAGE_PC,
-  R_GOT_PC,
-  R_HINT,
-  R_MIPS_GOT_LOCAL_PAGE,
-  R_MIPS_GOT_OFF,
-  R_MIPS_GOT_OFF32,
-  R_MIPS_GOTREL,
-  R_MIPS_TLSGD,
-  R_MIPS_TLSLD,
-  R_NEG_TLS,
-  R_PAGE_PC,
-  R_PC,
-  R_PLT,
-  R_PLT_PC,
-  R_PLT_PAGE_PC,
-  R_PPC_OPD,
-  R_PPC_PLT_OPD,
-  R_PPC_TOC,
-  R_RELAX_GOT_PC,
-  R_RELAX_GOT_PC_NOPIC,
-  R_RELAX_TLS_GD_TO_IE,
-  R_RELAX_TLS_GD_TO_IE_END,
-  R_RELAX_TLS_GD_TO_IE_ABS,
-  R_RELAX_TLS_GD_TO_IE_PAGE_PC,
-  R_RELAX_TLS_GD_TO_LE,
-  R_RELAX_TLS_GD_TO_LE_NEG,
-  R_RELAX_TLS_IE_TO_LE,
-  R_RELAX_TLS_LD_TO_LE,
-  R_SIZE,
-  R_THUNK_ABS,
-  R_THUNK_PC,
-  R_THUNK_PLT_PC,
-  R_TLS,
-  R_TLSDESC,
-  R_TLSDESC_PAGE,
-  R_TLSDESC_CALL,
-  R_TLSGD,
-  R_TLSGD_PC,
-  R_TLSLD,
-  R_TLSLD_PC,
+#define RELOCATION(Name, Index, PredicateBits) Name = (Index | PredicateBits),
+#include "Relocations.def"
 };
 
+// Sanity-check that all relocation indices are in-bounds.
+// If this ever fails, just increase PredicateShift above.
+#define RELOCATION(Name, Index, PredicateBits)                                 \
+  static_assert(Index < (1 << PredicateShift),                                 \
+                "Increase PredicateShift so that all relocs fit!");
+#include "Relocations.def"
+
+
 // Architecture-neutral representation of relocation.
 struct Relocation {
   RelExpr Expr;
   uint32_t Type;
   uint64_t Offset;
   uint64_t Addend;
   SymbolBody *Sym;
 };

ELF/Relocations.cpp

 using namespace llvm::ELF;
 using namespace llvm::object;
 using namespace llvm::support::endian;
 
 namespace lld {
 namespace elf {
 
 static bool refersToGotEntry(RelExpr Expr) {
-  return Expr == R_GOT || Expr == R_GOT_OFF || Expr == R_MIPS_GOT_LOCAL_PAGE ||
-         Expr == R_MIPS_GOT_OFF || Expr == R_MIPS_GOT_OFF32 ||
-         Expr == R_MIPS_TLSGD || Expr == R_MIPS_TLSLD ||
-         Expr == R_GOT_PAGE_PC || Expr == R_GOT_PC || Expr == R_GOT_FROM_END ||
-         Expr == R_TLSGD || Expr == R_TLSGD_PC || Expr == R_TLSDESC ||
-         Expr == R_TLSDESC_PAGE;
+  return Expr & RPRED_REFERS_TO_GOT;
 }
 
 static bool isPreemptible(const SymbolBody &Body, uint32_t Type) {
   // In case of MIPS GP-relative relocations always resolve to a definition
   // in a regular input file, ignoring the one-definition rule. So we,
   // for example, should not attempt to create a dynamic relocation even
   // if the target symbol is preemptible. There are two two MIPS GP-relative
   // relocations R_MIPS_GPREL16 and R_MIPS_GPREL32. But only R_MIPS_GPREL16
if (const auto *DR = dyn_cast<DefinedRegular<ELFT>>(&Body))
     return DR->Section == nullptr; // Absolute symbol.
   return false;
 }
 
 template <class ELFT> static bool isAbsoluteValue(const SymbolBody &Body) {
   return isAbsolute<ELFT>(Body) || Body.isTls();
 }
 
-static bool needsPlt(RelExpr Expr) {
-  return Expr == R_PLT_PC || Expr == R_PPC_PLT_OPD || Expr == R_PLT ||
-         Expr == R_PLT_PAGE_PC || Expr == R_THUNK_PLT_PC;
-}
+static bool needsPlt(RelExpr Expr) { return Expr & RPRED_NEEDS_PLT; }
 
 // True if this expression is of the form Sym - X, where X is a position in the
 // file (PC, or GOT for example).
-static bool isRelExpr(RelExpr Expr) {
-  return Expr == R_PC || Expr == R_GOTREL || Expr == R_GOTREL_FROM_END ||
-         Expr == R_MIPS_GOTREL || Expr == R_PAGE_PC || Expr == R_RELAX_GOT_PC ||
-         Expr == R_THUNK_PC || Expr == R_THUNK_PLT_PC;
-}
+static bool isRelExpr(RelExpr Expr) { return Expr & RPRED_RELATIVE; }
 
 template <class ELFT>
 static bool isStaticLinkTimeConstant(RelExpr E, uint32_t Type,
                                      const SymbolBody &Body,
                                      InputSectionBase<ELFT> &S,
                                      typename ELFT::uint RelOff) {
   // These expressions always compute a constant
-  if (E == R_SIZE || E == R_GOT_FROM_END || E == R_GOT_OFF ||
-      E == R_MIPS_GOT_LOCAL_PAGE || E == R_MIPS_GOT_OFF ||
-      E == R_MIPS_GOT_OFF32 || E == R_MIPS_TLSGD || E == R_GOT_PAGE_PC ||
-      E == R_GOT_PC || E == R_PLT_PC || E == R_TLSGD_PC || E == R_TLSGD ||
-      E == R_PPC_PLT_OPD || E == R_TLSDESC_CALL || E == R_TLSDESC_PAGE ||
-      E == R_HINT || E == R_THUNK_PC || E == R_THUNK_PLT_PC)
+  if (E & RPRED_ALWAYS_CONSTANT)
     return true;
 
   // These never do, except if the entire file is position dependent or if
   // only the low bits are used.
   if (E == R_GOT || E == R_PLT || E == R_TLSDESC)
     return Target->usesOnlyLowPageBits(Type) || !Config->Pic;
 
   if (isPreemptible(Body, Type))
for (auto I = Rels.begin(), E = Rels.end(); I != E; ++I) {
     if (unsigned Processed =
             handleTlsRelocation<ELFT>(Type, Body, C, Offset, Addend, Expr)) {
       I += (Processed - 1);
       continue;
     }
 
     // Ignore "hint" and TLS Descriptor call relocation because they are
     // only markers for relaxation.
-    if (Expr == R_HINT || Expr == R_TLSDESC_CALL)
+    if (Expr & RPRED_RELAX_MARKER)
       continue;
 
     if (needsPlt(Expr) || Expr == R_THUNK_ABS || Expr == R_THUNK_PC ||
         Expr == R_THUNK_PLT_PC || refersToGotEntry(Expr) ||
         !isPreemptible(Body, Type)) {
       // If the relocation points to something in the file, we can process it.
       bool Constant =
           isStaticLinkTimeConstant<ELFT>(Expr, Type, Body, C, RI.r_offset);

ELF/Relocations.def

+//===- Relocations.def -----------------------------------------*- C++ -*-===//
+//
+//                             The LLVM Linker
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+// NOTE: No include guard desired!
+
+#ifndef RELOCATION
+#define RELOCATION(Name, Index, PredicateBits)
+#endif
+
+// RELOCATION(Name, Index, PredicateBits)
+//
+// Index and PredicateBits are just OR'd together to get the RelExpr
+// value, but they are kept separate here so that we can do a
+// compile-time check that the Index values don't collide with the
+// PredicateBits values.
+RELOCATION(R_ABS                        ,   0, 0)
+RELOCATION(R_GOT                        ,   1, RPRED_REFERS_TO_GOT)
+RELOCATION(R_GOTONLY_PC                 ,   2, 0)
+RELOCATION(R_GOTONLY_PC_FROM_END        ,   3, 0)
+RELOCATION(R_GOTREL                     ,   4, RPRED_RELATIVE)
+RELOCATION(R_GOTREL_FROM_END            ,   5, RPRED_RELATIVE)
+RELOCATION(R_GOT_FROM_END               ,   6, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_GOT_OFF                    ,   7, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_GOT_PAGE_PC                ,   8, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_GOT_PC                     ,   9, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_HINT                       ,  10, RPRED_ALWAYS_CONSTANT | RPRED_RELAX_MARKER)
+RELOCATION(R_MIPS_GOT_LOCAL_PAGE        ,  11, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_MIPS_GOT_OFF               ,  12, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_MIPS_GOT_OFF32             ,  13, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_MIPS_GOTREL                ,  14, RPRED_RELATIVE)
+RELOCATION(R_MIPS_TLSGD                 ,  15, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_MIPS_TLSLD                 ,  16, RPRED_REFERS_TO_GOT)
+RELOCATION(R_NEG_TLS                    ,  17, 0)
+RELOCATION(R_PAGE_PC                    ,  18, RPRED_RELATIVE)
+RELOCATION(R_PC                         ,  19, RPRED_RELATIVE)
+RELOCATION(R_PLT                        ,  20, RPRED_NEEDS_PLT)
+RELOCATION(R_PLT_PC                     ,  21, RPRED_NEEDS_PLT | RPRED_ALWAYS_CONSTANT)
+RELOCATION(R_PLT_PAGE_PC                ,  22, RPRED_NEEDS_PLT)
+RELOCATION(R_PPC_OPD                    ,  23, 0)
+RELOCATION(R_PPC_PLT_OPD                ,  24, RPRED_NEEDS_PLT | RPRED_ALWAYS_CONSTANT)
+RELOCATION(R_PPC_TOC                    ,  25, 0)
+RELOCATION(R_RELAX_GOT_PC               ,  26, RPRED_RELATIVE)
+RELOCATION(R_RELAX_GOT_PC_NOPIC         ,  27, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_IE         ,  28, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_IE_END     ,  29, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_IE_ABS     ,  30, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_IE_PAGE_PC ,  31, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_LE         ,  32, 0)
+RELOCATION(R_RELAX_TLS_GD_TO_LE_NEG     ,  33, 0)
+RELOCATION(R_RELAX_TLS_IE_TO_LE         ,  34, 0)
+RELOCATION(R_RELAX_TLS_LD_TO_LE         ,  35, 0)
+RELOCATION(R_SIZE                       ,  36, RPRED_ALWAYS_CONSTANT)
+RELOCATION(R_THUNK_ABS                  ,  37, 0)
+RELOCATION(R_THUNK_PC                   ,  38, RPRED_ALWAYS_CONSTANT | RPRED_RELATIVE)
+RELOCATION(R_THUNK_PLT_PC               ,  39, RPRED_NEEDS_PLT | RPRED_ALWAYS_CONSTANT | RPRED_RELATIVE)
+RELOCATION(R_TLS                        ,  40, 0)
+RELOCATION(R_TLSDESC                    ,  41, RPRED_REFERS_TO_GOT)
+RELOCATION(R_TLSDESC_PAGE               ,  42, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_TLSDESC_CALL               ,  43, RPRED_ALWAYS_CONSTANT | RPRED_RELAX_MARKER)
+RELOCATION(R_TLSGD                      ,  44, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_TLSGD_PC                   ,  45, RPRED_ALWAYS_CONSTANT | RPRED_REFERS_TO_GOT)
+RELOCATION(R_TLSLD                      ,  46, 0)
+RELOCATION(R_TLSLD_PC                   ,  47, 0)
+
+// Undefine macros so that users of this .def file don't have to do so
+// themselves.
+
+#ifdef RELOCATION
+#undef RELOCATION
+#endif