[llvm][IR] Add dso_local_equivalent Constant

[leonardchan]

Summary

The dso_local_equivalent constant is a wrapper for functions that represents a value which is functionally equivalent to the global passed to this. That is, if this accepts a function, calling this constant should have the same effects as calling the function directly. This could be a direct reference to the function, the @plt modifier on X86/AArch64, a thunk, or anything that's equivalent to the resolved function as a call target.

When lowered, the returned address must have a constant offset at link time from some other symbol defined within the same binary. The address of this value is also insignificant. The name is leveraged from dso_local where use of a function or variable is resolved to a symbol in the same linkage unit.

In this patch:

• Addition of dso_local_equivalent and handling it
• Update Constant::needsRelocation() to strip constant inbound GEPs and take advantage of dso_local_equivalent for relative references

This is useful for the Relative VTables C++ ABI which makes vtables readonly. This works by replacing the dynamic relocations for function pointers in them with static relocations that represent the offset between the vtable and virtual functions. If a function is externally defined, dso_local_equivalent can be used as a generic wrapper for the function to still allow for this static offset calculation to be done.

See RFC for more details.

[leonardchan]

*ping*

[leonardchan]

*ping*

[arsenm]

Was there an RFC for this?

Missing GlobalISel support

[leonardchan]

In D77248#2224589, @arsenm wrote:

Was there an RFC for this?

Missing GlobalISel support

There hasn't, but I can send one out. The idea of a new constant was from discussing with @pcc offline.

[lebedev.ri]

Missing RFC + langref changes.

llvm/include/llvm/IR/Constants.h

892

What's PLT entry?

[leonardchan]

Clean up some of the code.

Also sent out an RFC at http://lists.llvm.org/pipermail/llvm-dev/2020-August/144469.html.

Still need to add and test GlobalISel support.

[leonardchan]

Update to reflect comments in the RFC.

[mcgrathr]

Rather than "equivalent to the resolved function at runtime", I'd say "equivalent to the resolved function as a call target". It's not equivalent to the function at runtime when the runtime use is "ptr == &func".

[phosek]

This is just a naming suggestion, but how about naming it unnamed_func (instead of unnamedfunc) so it's closer to unnamed_addr.

[leonardchan]

• Renamed unnamedfunc to unnamed_func.
• Replaced "equivalent to the resolved function at runtime" with "equivalent to the resolved function as a call target".
• Added section to LangRef.

[leonardchan]

Updated to reflect the most recent comments at http://lists.llvm.org/pipermail/llvm-dev/2020-September/145177.html.

[rjmccall]

llvm/docs/LangRef.rst

3771

There has to be a type for the global in the syntax, right? Or is that an assumed prefix on this, and then we don't need it separately on the global because it's the same type?

Also, do we actually need the parens? I think the other places we use parens here, it actually does meaningful grouping.

On wording, allow me to suggest:

A '``dso_local_equivalent``' constant represents a global value which is functionally equivalent to a given global value, but is always defined in the current linkage unit.  The resulting pointer has the same type as the underlying global.  The resulting pointer is permitted, but not required, to be different from a pointer to the global, and it may have different values in different translation units.

The target global may not have ``extern_weak`` linkage.

``dso_local_equivalent`` can be implemented in several ways:

- If the global has private or internal linkage, has hidden visibility, or is ``dso_local``, ``dso_local_equivalent`` can be implemented as simply a pointer to the global.

- If the global is a function, ``dso_local_equivalent`` can be implemented with a stub that tail-calls the function.  Many targets support relocations that resolve at link time to either a function or a stub for it, depending on if the function is defined within the linkage unit; LLVM will use this when available.  (This is commonly called a "PLT stub".)  On other targets, the stub may have to be emitted explicitly.

- If the global is a variable, this can be implemented by copying the contents of the variable into the current linkage unit (as if by `memcpy`).  This is not supported on all targets, and target-specific restrictions may apply.  For example, on ELF this can be implemented as a copy relocation, which requires that the global has the same size at load time as it did at at link time.

I don't think you need to keep anything from the last paragraph.

[leonardchan]

llvm/docs/LangRef.rst

3771

There has to be a type for the global in the syntax, right? Or is that an assumed prefix on this, and then we don't need it separately on the global because it's the same type?

The latter. I was aiming for a syntax similar to blockaddress, but we would know the global type since we still need to write it before dso_local_equivalent.

Also, do we actually need the parens? I think the other places we use parens here, it actually does meaningful grouping.

You're right. They can be removed.

Thanks for the suggestion also! The only thing I'd probably clarify on is if this constant accepts "global values" vs "global objects" since global value implies that this could also accept global aliases or global ifuncs. I suppose there's nothing wrong with this: ifuncs can have the same handling as functions, and depending on the aliasee, an alias can have the same handling as a function or variable. It could be that this is already implied, but I'll update to explicitly mention them also.

[leonardchan]

Previous diff was against HEAD rather than master. Updated.

[leonardchan]

Add generic lowering for targets that do not support PLTs.

I wasn't sure what the best way to lower the constant was, so I ended up making a module pass that runs before ISel and replaces dso_local_equivalents with dso_local stubs that tail call to the original function. I'm open to changing it if there's a cleaner way. I thought the easiest approach was to provide the generic implementation as IR that could be lowered naturally than through SDNodes in ISel.

[leonardchan]

ping

[pcc]

llvm/docs/LangRef.rst

3797

I would make this specific to functions. I don't think this can be made to work with globals at least with existing ELF linkers because I don't think you can get the copy relocation behavior without the symbol's value being overridden by the linkage unit with the copy relocation, and it doesn't make sense to have more than one linkage unit override the address of the same global.

[rjmccall]

llvm/docs/LangRef.rst

3797

Oh, does ELF's copy relocation necessarily steal the symbol so that it resolves into the copying linkage unit? I thought it might be separable so that you could just get a local copy of the data without changing symbol resolution. That would be a much more generically useful feature.

[pcc]

llvm/docs/LangRef.rst

3797

I think the behaviors may be separable at the binary level (i.e. you could hand-craft an executable that only copies but doesn't override), but at the linker level the copying and overriding happen at the same time when the linker sees a non-GOT-generating relocation pointing to the symbol.

[leonardchan]

llvm/docs/LangRef.rst

3797

I'm fine with applying this to just functions for now since that was what I was initially planning to support. If possible, support for variables could be rolled out in the future. It looks like, off the bat, we could just use a GOT entry if we know the symbol is dso_local, but I think we can already achieve this by having a hidden global that points to your symbol.

Took me some time to research copy relocations, so correct me if I'm wrong, but should it be R_*_GLOB_DATA that overrides the symbol address during linking, whereas the copy relocation (R_*_COPY) just does the copying as @rjmccall suggests? I think it's R_*_GLOB_DATA that the GOT uses for getting addresses(?).

[pcc]

llvm/docs/LangRef.rst

3797

The R_*_COPY is normally generated by the linker and only present in the executable or shared object. It isn't present in an object file. The symbol address override is implemented by a symbol table entry, not by a relocation.

I'm not sure what would happen if you had an R_*_COPY in an object file without a definition of the symbol. You would need to make sure that it's passed through to the binary correctly and that dynamic loaders have the expected behavior (because normally dynamic loaders expect the symbol that's being copied to also be defined locally). The size of the symbol would also need to be known at compile time (as opposed to at link time in the normal case where the linker is inserting the relocation) in order to allocate enough space for the copy.

[MaskRay]

llvm/docs/LangRef.rst

3797

I agree with pcc that the variable case should be dropped.

R_*_COPY could be used for shared objects in some ld.so (e.g. musl, which just skips the first definition; but not in glibc, which special cases that a COPY skips an executable definition). Generally it does not make sense for a shared object.

R_*_COPY on an executable copying its own symbol does not work -> the behavior is like RTLD_NEXT.

[rjmccall]

llvm/docs/LangRef.rst

3797

Is that avoidable? What happens if you ask for a copy relocation in an object file, or is that not expressible?

3797

I'm fine with just supporting functions given the absence of a target that can support this for variables.

3797

I thought I deleted this comment, but apparently not; please ignore it.

[leonardchan]

Update to only support functions since we don't have a fully flushed out way of supporting global variables.

[leonardchan]

ping

[leonardchan]

ping, any more comments for this?

[phosek]

LGTM but someone else should take a look as well.

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

37

Propagate

[leonardchan]

*ping* for other comments. I'll probably commit this at the end of the week unless others have more to add.

[MaskRay]

llc -filetype=obj doesn't. Please fix it. It is a bit unfortunate but the ELFObjectWriter generally has more restrictions than the assembler writer.

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

2289

https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

25

Use static

https://llvm.org/docs/CodingStandards.html#anonymous-namespaces

lowerCase

82

lowerCase.

This either needs static or should stick with https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

84

Shouldn't protected visibility use GV as well?

Is isDSOLocal implied by other properties?

llvm/test/CodeGen/X86/dso_local_equivalent.ll

39

_hidden_func{{$}}

44

ditto

49

ditto

54

ditto

72

ifunc stub may seem strange. Can you add a comment?

[leonardchan]

In D77248#2358172, @MaskRay wrote:

llc -filetype=obj doesn't. Please fix it. It is a bit unfortunate but the ELFObjectWriter generally has more restrictions than the assembler writer.

Ah, didn't see this. Yeah I was using this incorrectly in some of my tests. Fixed.

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

84

Yeah, it seems that there's actually a function that covers this already: GlobalValue::isImplicitDSOLocal.

llvm/test/CodeGen/X86/dso_local_equivalent.ll

72

Added. This is here mostly to ensure correctness since the ifunc may not be dso_local so we'd still need a stub to ensure that.

[leonardchan]

@MaskRay any more comments?

[leonardchan]

Rebased

[leonardchan]

Updated some renamed variables that I didn't check prior.

[MaskRay]

Apologies for my late reply. A couple of comments inlined

See RFC for more details.

Neither the RFC nor the patch mentions the immediate use case. It is possible that the merit is buries in a deep message but for a few feature the description itself should justify its value (dblaikie mentioned that:

We should do this because I need it" (we shouldn't be implementing features for especially niche use cases/if they don't generalize) isn't always a compelling motivation but "we should do this because someone might need it" isn't either (we shouldn't be implementing features that have no users).

Please add the use cases.

llvm/include/llvm-c/Core.h

2159

This is not defined. If you don't add a C implementation, please drop the declaration here and above

llvm/include/llvm/IR/Constants.h

893

The comment says "function" then "variable". Is "variable" a typo?

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

21

https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

52

ELF has comdat but does not need this pass.
Mach-O needs this pass but does not have comdat.

I assume that this code is untested.

llvm/test/CodeGen/X86/dso_local_equivalent.ll

5

This comment appears to be unrelated to the main purpose of the test. A file-level comment is needed to explain why darwin is picked (it does not support the lowering)

[leonardchan]

In D77248#2390009, @MaskRay wrote:

Apologies for my late reply. A couple of comments inlined

See RFC for more details.

Neither the RFC nor the patch mentions the immediate use case. It is possible that the merit is buries in a deep message but for a few feature the description itself should justify its value (dblaikie mentioned that:

We should do this because I need it" (we shouldn't be implementing features for especially niche use cases/if they don't generalize) isn't always a compelling motivation but "we should do this because someone might need it" isn't either (we shouldn't be implementing features that have no users).

Please add the use cases.

No worries. Thanks for the reviews! I added the use case in the LangRef. Essentially the immediate use case is for using static relocations in vtables under the new relative vtables ABI. This constant gives us a generic way to essentially "request" a PLT relocation from Clang without needing to explicitly say in the frontend "these targets support PLT entries".

llvm/include/llvm/IR/Constants.h

893

Yup, it should still be function. Fixed.

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

52

Added some cases for COFF where this pass is needed and comdats are supported.

[MaskRay]

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

10

Transform uses of dso_local_equivalent to a function stub for targets which do not support lowering dso_local_equivalent to MCExpr.

21

Please use llvm::foo to define functions/variables in .cpp files

31

Nit: Strings in assert usually do not need to end with a period

37

I think the stub should be InternalLinkage (if you want to retain a symbol) or PrivateLinkage (if you don't want a symbol). You would not need hidden visibility below.

If you use a hidden function, and two translation units have dso_local declarations of the same function, the produced stubs will cause a duplicate definition error at link time.

llvm/test/CodeGen/X86/dso_local_equivalent.ll

11

-o - -> -o /dev/null

129

Use one single NO-PLT-NOT: .stub

145

Use one single NO-PLT-COMDAT-NOT: .stub

[leonardchan]

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

37

Hmm. I'm guessing the link error would only occur though if comdats weren't supported? In that case I think perhaps the stubs should be internal/private to avoid multiple definitions.

[MaskRay]

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

37

For stubs for declarations (not defined), there are some pitfalls. On ELF, if in a comdat, a hidden symbol works better because it can be merged with symbols in other translations; if not in a comdat, private/internal linkage has to be used to avoid linker issues. Such a comdat concept probably does not work on COFF. Mach-O does not have comdat.

The pass serves for binary formats which I think are not your main concern, am I right? :) If you cannot find people willing to review the Mach-O/COFF part, I recommend that you drop this pass from this patch because it is not needed by ELF (and your use case will condition the support on ELF anyway)

[leonardchan]

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

37

Ah, ok I'll drop this pass then. The immediate use case for us is only ELF formats. I wasn't sure to what extent this should be supported for other binary formats, but I suppose those could be added down the line.

[MaskRay]

llvm/include/llvm-c/Core.h

266

Adding a member in the middle breaks prebuilt binaries linking against llvm-c. How about just dropping the change?

[leonardchan]

llvm/include/llvm-c/Core.h

266

Done. The only concern is that there is no generated value for this in the Value::ValueTy enum which would've led to some errors, so I added it manually with a comment explaining why it's there.

[leonardchan]

Welp, it turns out that if I went with the approach of adding the constant manually everywhere the macro was used, I'd be adding a lot of special cases everywhere. The only case I needed to cover was in Core which depends on Value.def:

/usr/local/google/home/leonardchan/llvm-monorepo/llvm-project-1/llvm/lib/IR/Core.cpp:832:12: note: expanded from macro 'HANDLE_VALUE'
    return LLVM##Name##ValueKind;
           ^
<scratch space>:239:1: note: expanded from here
LLVMDSOLocalEquivalentValueKind

The only way I could think of preventing the constant from being used here while still being used in other places HANDLE_VALUE/CONSTANT are used is a "banlist" approach via more macros in Value.def. Open to other ideas if it doesn't look that good.

llvm/docs/LangRef.rst

 opaque pointer sized value as long as the bits are not inspected. This
 allows ``ptrtoint`` and arithmetic to be performed on these values so
 long as the original value is reconstituted before the ``indirectbr`` or
 ``callbr`` instruction.
 
 Finally, some targets may provide defined semantics when using the value
 as the operand to an inline assembly, but that is target specific.
 
+.. _dso_local_equivalent:
+
+DSO Local Equivalent
+--------------------
+
+``dso_local_equivalent @func``
+
+A '``dso_local_equivalent``' constant represents a function which is
+functionally equivalent to a given function, but is always defined in the
+current linkage unit. The resulting pointer has the same type as the underlying
+function. The resulting pointer is permitted, but not required, to be different
+from a pointer to the function, and it may have different values in different
+translation units.
+
+The target function may not have ``extern_weak`` linkage.
+
+``dso_local_equivalent`` can be implemented as such:
+
+- If the function has local linkage, hidden visibility, or is
+  ``dso_local``, ``dso_local_equivalent`` can be implemented as simply a pointer
+  to the function.
+- ``dso_local_equivalent`` can be implemented with a stub that tail-calls the
+  function. Many targets support relocations that resolve at link time to either
+  a function or a stub for it, depending on if the function is defined within the
+  linkage unit; LLVM will use this when available. (This is commonly called a
+  "PLT stub".) On other targets, the stub may need to be emitted explicitly.
+
 .. _constantexprs:
 
 Constant Expressions
 --------------------
 
 Constant expressions are used to allow expressions involving other
 constants to be used as constants. Constant expressions may be of any
 :ref:`first class <t_firstclass>` type and may involve any LLVM operation
 that does not have side effects (e.g. load and call are not supported).
 The following is the syntax for constant expressions:

llvm/include/llvm-c/Core.h

typedef enum {
   LLVMMemoryDefValueKind,
   LLVMMemoryPhiValueKind,
 
   LLVMFunctionValueKind,
   LLVMGlobalAliasValueKind,
   LLVMGlobalIFuncValueKind,
   LLVMGlobalVariableValueKind,
   LLVMBlockAddressValueKind,
+  LLVMDSOLocalEquivalentValueKind,
   LLVMConstantExprValueKind,
   LLVMConstantArrayValueKind,
   LLVMConstantStructValueKind,
   LLVMConstantVectorValueKind,
 
   LLVMUndefValueValueKind,
   LLVMConstantAggregateZeroValueKind,
   LLVMConstantDataArrayValueKind,
macro(Constant)                         \
       macro(ConstantTokenNone)              \
       macro(ConstantVector)                 \
       macro(GlobalValue)                    \
         macro(GlobalAlias)                  \
         macro(GlobalIFunc)                  \
         macro(GlobalObject)                 \
           macro(Function)                   \
           macro(GlobalVariable)             \
+      macro(DSOLocalEquivalent)             \
       macro(UndefValue)                     \
     macro(Instruction)                      \
       macro(UnaryOperator)                  \
       macro(BinaryOperator)                 \
       macro(CallInst)                       \
         macro(IntrinsicInst)                \
           macro(DbgInfoIntrinsic)           \
             macro(DbgVariableIntrinsic)     \
LLVMValueRef LLVMConstShuffleVector(LLVMValueRef VectorAConstant,
                                     LLVMValueRef VectorBConstant,
                                     LLVMValueRef MaskConstant);
 LLVMValueRef LLVMConstExtractValue(LLVMValueRef AggConstant, unsigned *IdxList,
                                    unsigned NumIdx);
 LLVMValueRef LLVMConstInsertValue(LLVMValueRef AggConstant,
                                   LLVMValueRef ElementValueConstant,
                                   unsigned *IdxList, unsigned NumIdx);
 LLVMValueRef LLVMBlockAddress(LLVMValueRef F, LLVMBasicBlockRef BB);
+LLVMValueRef LLVMDSOLocalEquivalent(LLVMValueRef F);
 
 /** Deprecated: Use LLVMGetInlineAsm instead. */
 LLVMValueRef LLVMConstInlineAsm(LLVMTypeRef Ty,
                                 const char *AsmString, const char *Constraints,
                                 LLVMBool HasSideEffects, LLVMBool IsAlignStack);
 
 /**
  * @}

llvm/include/llvm/CodeGen/Passes.h

/// MachineDominanaceFrontier - This pass is a machine dominators analysis pass.
   ModulePass *createLowerEmuTLSPass();
 
   /// This pass lowers the \@llvm.load.relative and \@llvm.objc.* intrinsics to
   /// instructions.  This is unsafe to do earlier because a pass may combine the
   /// constant initializer into the load, which may result in an overflowing
   /// evaluation.
   ModulePass *createPreISelIntrinsicLoweringPass();
 
+  /// This pass lowers references to dso_local_equivalent to generic IR
+  /// implementations on targets that don't have a native lowering of them.
+  ModulePass *createDSOLocalEquivalentLoweringPass(const TargetMachine *TM);
+
   /// GlobalMerge - This pass merges internal (by default) globals into structs
   /// to enable reuse of a base pointer by indexed addressing modes.
   /// It can also be configured to focus on size optimizations only.
   ///
   Pass *createGlobalMergePass(const TargetMachine *TM, unsigned MaximalOffset,
                               bool OnlyOptimizeForSize = false,
                               bool MergeExternalByDefault = false);
 

llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h

class TargetLoweringObjectFileELF : public TargetLoweringObjectFile {
   mutable unsigned NextUniqueID = 1;  // ID 0 is reserved for execute-only sections
 
 protected:
   MCSymbolRefExpr::VariantKind PLTRelativeVariantKind =
       MCSymbolRefExpr::VK_None;
   const TargetMachine *TM = nullptr;
 
 public:
-  TargetLoweringObjectFileELF() = default;
+  TargetLoweringObjectFileELF();
   ~TargetLoweringObjectFileELF() override = default;
 
   void Initialize(MCContext &Ctx, const TargetMachine &TM) override;
 
   /// Emit Obj-C garbage collection and linker options.
   void emitModuleMetadata(MCStreamer &Streamer, Module &M) const override;
 
   void emitPersonalityValue(MCStreamer &Streamer, const DataLayout &DL,
MCSection *getStaticCtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
   MCSection *getStaticDtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
 
   const MCExpr *lowerRelativeReference(const GlobalValue *LHS,
                                        const GlobalValue *RHS,
                                        const TargetMachine &TM) const override;
 
+  const MCExpr *lowerDSOLocalEquivalent(const DSOLocalEquivalent *Equiv,
+                                        const TargetMachine &TM) const override;
+
   MCSection *getSectionForCommandLines() const override;
 };
 
 class TargetLoweringObjectFileMachO : public TargetLoweringObjectFile {
 public:
   TargetLoweringObjectFileMachO();
   ~TargetLoweringObjectFileMachO() override = default;
 

llvm/include/llvm/IR/Constants.h

 
 template <>
 struct OperandTraits<BlockAddress> :
   public FixedNumOperandTraits<BlockAddress, 2> {
 };
 
 DEFINE_TRANSPARENT_OPERAND_ACCESSORS(BlockAddress, Value)
 
+/// Wrapper for a function that represents a value that
+/// functionally represents the original variable. This can be a function,
+/// global alias to a function, or an ifunc.
+class DSOLocalEquivalent final : public Constant {
+  friend class Constant;
+
+  DSOLocalEquivalent(GlobalValue *GV);
+
+  void *operator new(size_t s) { return User::operator new(s, 1); }
+
+  void destroyConstantImpl();
+  Value *handleOperandChangeImpl(Value *From, Value *To);
+
+public:
+  /// Return a DSOLocalEquivalent for the specified global value.
+  static DSOLocalEquivalent *get(GlobalValue *GV);
+
+  /// Transparently provide more efficient getOperand methods.
+  DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value);
+
+  GlobalValue *getGlobalValue() const {
+    return cast<GlobalValue>(Op<0>().get());
+  }
+
+  /// Methods for support type inquiry through isa, cast, and dyn_cast:
+  static bool classof(const Value *V) {
+    return V->getValueID() == DSOLocalEquivalentVal;
+  }
+};
+
+template <>
+struct OperandTraits<DSOLocalEquivalent>
+    : public FixedNumOperandTraits<DSOLocalEquivalent, 1> {};
+
+DEFINE_TRANSPARENT_OPERAND_ACCESSORS(DSOLocalEquivalent, Value)
+
 //===----------------------------------------------------------------------===//
 /// A constant value that is initialized with an expression using
 /// other constant values.
 ///
 /// This class uses the standard Instruction opcodes to define the various
 /// constant expressions.  The Opcode field for the ConstantExpr class is
 /// maintained in the Value::SubclassData field.
 class ConstantExpr : public Constant {

llvm/include/llvm/IR/Value.def

 
 // Constant
 HANDLE_GLOBAL_VALUE(Function)
 HANDLE_GLOBAL_VALUE(GlobalAlias)
 HANDLE_GLOBAL_VALUE(GlobalIFunc)
 HANDLE_GLOBAL_VALUE(GlobalVariable)
 HANDLE_CONSTANT(BlockAddress)
 HANDLE_CONSTANT(ConstantExpr)
+HANDLE_CONSTANT(DSOLocalEquivalent)
 
 // ConstantAggregate.
 HANDLE_CONSTANT(ConstantArray)
 HANDLE_CONSTANT(ConstantStruct)
 HANDLE_CONSTANT(ConstantVector)
 
 // ConstantData.
 HANDLE_CONSTANT(UndefValue)

llvm/include/llvm/Target/TargetLoweringObjectFile.h

 class MCSymbol;
 class MCSymbolRefExpr;
 class MCStreamer;
 class MCValue;
 class Module;
 class SectionKind;
 class StringRef;
 class TargetMachine;
+class DSOLocalEquivalent;
 
 class TargetLoweringObjectFile : public MCObjectFileInfo {
   /// Name-mangler for global names.
   Mangler *Mang = nullptr;
 
 protected:
   bool SupportIndirectSymViaGOTPCRel = false;
   bool SupportGOTPCRelWithOffset = true;
   bool SupportDebugThreadLocalLocation = true;
+  bool SupportDSOLocalEquivalentLowering = false;
 
   /// PersonalityEncoding, LSDAEncoding, TTypeEncoding - Some encoding values
   /// for EH.
   unsigned PersonalityEncoding = 0;
   unsigned LSDAEncoding = 0;
   unsigned TTypeEncoding = 0;
   unsigned CallSiteEncoding = 0;
 
public:
   virtual const MCExpr *getDebugThreadLocalSymbol(const MCSymbol *Sym) const;
 
   virtual const MCExpr *lowerRelativeReference(const GlobalValue *LHS,
                                                const GlobalValue *RHS,
                                                const TargetMachine &TM) const {
     return nullptr;
   }
 
+  /// Target supports a native lowering of a dso_local_equivalent constant
+  /// without needing to replace it with equivalent IR.
+  bool supportDSOLocalEquivalentLowering() const {
+    return SupportDSOLocalEquivalentLowering;
+  }
+
+  virtual const MCExpr *lowerDSOLocalEquivalent(const DSOLocalEquivalent *Equiv,
+                                                const TargetMachine &TM) const {
+    return nullptr;
+  }
+
   /// Target supports replacing a data "PC"-relative access to a symbol
   /// through another symbol, by accessing the later via a GOT entry instead?
   bool supportIndirectSymViaGOTPCRel() const {
     return SupportIndirectSymViaGOTPCRel;
   }
 
   /// Target GOT "PC"-relative relocation supports encoding an additional
   /// binary expression with an offset?

llvm/lib/Analysis/ConstantFolding.cpp

bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
                                       APInt &Offset, const DataLayout &DL) {
   // Trivial case, constant is the global.
   if ((GV = dyn_cast<GlobalValue>(C))) {
     unsigned BitWidth = DL.getIndexTypeSizeInBits(GV->getType());
     Offset = APInt(BitWidth, 0);
     return true;
   }
 
+  if (auto *Equiv = dyn_cast<DSOLocalEquivalent>(C))
+    return IsConstantOffsetFromGlobal(Equiv->getGlobalValue(), GV, Offset, DL);
+
   // Otherwise, if this isn't a constant expr, bail out.
   auto *CE = dyn_cast<ConstantExpr>(C);
   if (!CE) return false;
 
   // Look through ptr->int and ptr->ptr casts.
   if (CE->getOpcode() == Instruction::PtrToInt ||
       CE->getOpcode() == Instruction::BitCast)
     return IsConstantOffsetFromGlobal(CE->getOperand(0), GV, Offset, DL);

llvm/lib/AsmParser/LLLexer.cpp

#define KEYWORD(STR)                                                           \
   KEYWORD(oge); KEYWORD(ord); KEYWORD(uno); KEYWORD(ueq); KEYWORD(une);
 
   KEYWORD(xchg); KEYWORD(nand); KEYWORD(max); KEYWORD(min); KEYWORD(umax);
   KEYWORD(umin);
 
   KEYWORD(vscale);
   KEYWORD(x);
   KEYWORD(blockaddress);
+  KEYWORD(dso_local_equivalent);
 
   // Metadata types.
   KEYWORD(distinct);
 
   // Use-list order directives.
   KEYWORD(uselistorder);
   KEYWORD(uselistorder_bb);
 

llvm/lib/AsmParser/LLParser.cpp

if (BlockAddressPFS && F == &BlockAddressPFS->getFunction()) {
         return Error(Label.Loc, "referenced value is not a basic block");
     }
 
     ID.ConstantVal = BlockAddress::get(F, BB);
     ID.Kind = ValID::t_Constant;
     return false;
   }
 
+  case lltok::kw_dso_local_equivalent: {
+    // ValID ::= 'dso_local_equivalent' @foo
+    Lex.Lex();
+
+    ValID Fn;
+
+    if (ParseValID(Fn))
+      return true;
+
+    if (Fn.Kind != ValID::t_GlobalID && Fn.Kind != ValID::t_GlobalName)
+      return Error(Fn.Loc,
+                   "expected global value name in dso_local_equivalent");
+
+    // Try to find the function (but skip it if it's forward-referenced).
+    GlobalValue *GV = nullptr;
+    if (Fn.Kind == ValID::t_GlobalID) {
+      if (Fn.UIntVal < NumberedVals.size())
+        GV = NumberedVals[Fn.UIntVal];
+    } else if (!ForwardRefVals.count(Fn.StrVal)) {
+      GV = M->getNamedValue(Fn.StrVal);
+    }
+
+    assert(GV && "Could not find a corresponding global variable");
+
+    if (!GV->getValueType()->isFunctionTy())
+      return Error(Fn.Loc, "expected a function, alias to function, or ifunc "
+                           "in dso_local_equivalent");
+
+    ID.ConstantVal = DSOLocalEquivalent::get(GV);
+    ID.Kind = ValID::t_Constant;
+    return false;
+  }
+
   case lltok::kw_trunc:
   case lltok::kw_zext:
   case lltok::kw_sext:
   case lltok::kw_fptrunc:
   case lltok::kw_fpext:
   case lltok::kw_bitcast:
   case lltok::kw_addrspacecast:
   case lltok::kw_uitofp:

llvm/lib/AsmParser/LLToken.h

enum Kind {
   kw_getelementptr,
 
   kw_extractelement,
   kw_insertelement,
   kw_shufflevector,
   kw_extractvalue,
   kw_insertvalue,
   kw_blockaddress,
+  kw_dso_local_equivalent,
 
   kw_freeze,
 
   // Metadata types.
   kw_distinct,
 
   // Use-list order directives.
   kw_uselistorder,

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

if (const ConstantInt *CI = dyn_cast<ConstantInt>(CV))
     return MCConstantExpr::create(CI->getZExtValue(), Ctx);
 
   if (const GlobalValue *GV = dyn_cast<GlobalValue>(CV))
     return MCSymbolRefExpr::create(getSymbol(GV), Ctx);
 
   if (const BlockAddress *BA = dyn_cast<BlockAddress>(CV))
     return MCSymbolRefExpr::create(GetBlockAddressSymbol(BA), Ctx);
 
+  if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(CV)) {
+    return getObjFileLowering().lowerDSOLocalEquivalent(Equiv, TM);
+  }
+
   const ConstantExpr *CE = dyn_cast<ConstantExpr>(CV);
   if (!CE) {
     llvm_unreachable("Unknown constant value to lower!");
   }
 
   switch (CE->getOpcode()) {
   case Instruction::AddrSpaceCast: {
     const Constant *Op = CE->getOperand(0);

llvm/lib/CodeGen/CMakeLists.txt

add_llvm_component_library(LLVMCodeGen
   CFIInstrInserter.cpp
   CodeGen.cpp
   CodeGenPrepare.cpp
   CommandFlags.cpp
   CriticalAntiDepBreaker.cpp
   DeadMachineInstructionElim.cpp
   DetectDeadLanes.cpp
   DFAPacketizer.cpp
+  DSOLocalEquivalentLowering.cpp
   DwarfEHPrepare.cpp
   EarlyIfConversion.cpp
   EdgeBundles.cpp
   ExecutionDomainFix.cpp
   ExpandMemCmp.cpp
   ExpandPostRAPseudos.cpp
   ExpandReductions.cpp
   FaultMaps.cpp

llvm/lib/CodeGen/DSOLocalEquivalentLowering.cpp

+//===- DSOLocalEquivalentLowering.cpp - dso_local_equivalent lowering pass ===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// If this target cannot lower the dso_local_equivalent pass to an MCExpr, the
+// expression is lowered with a generic IR implementation.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/Triple.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/Target/TargetLoweringObjectFile.h"
+#include "llvm/Target/TargetMachine.h"
+
+namespace llvm {
+
+namespace {
+
+Function *GetOrCreateRelativeStub(GlobalValue *GV) {
+  SmallString<16> StubName(GV->getName());
+  StubName.append(".stub");
+
+  auto &M = *GV->getParent();
+  Function *Stub = M.getFunction(StubName);
+  if (Stub) {
+    assert(Stub->isDSOLocal() &&
+           "The previous definition of this stub should be dso_local.");
+    return Stub;
+  }
+
+  if (const auto *F = dyn_cast<Function>(GV)) {
+    // Propogate function attributes.
+    Stub = Function::Create(F->getFunctionType(), F->getLinkage(), StubName, M);
+    Stub->setAttributes(F->getAttributes());
+  } else if (isa<GlobalIFunc, GlobalAlias>(GV)) {
+    Stub = Function::Create(cast<FunctionType>(GV->getValueType()),
+                            GV->getLinkage(), StubName, M);
+  } else {
+    llvm_unreachable("Expected the global value to be a function, ifunc, or "
+                     "alias to a function or ifunc.");
+  }
+
+  Stub->setDSOLocal(true);
+  Stub->setUnnamedAddr(GlobalValue::UnnamedAddr::Global);
+  if (!Stub->hasLocalLinkage()) {
+    Stub->setVisibility(GlobalValue::HiddenVisibility);
+    if (Triple(M.getTargetTriple()).supportsCOMDAT())
+      Stub->setComdat(M.getOrInsertComdat(StubName));
+  }
+
+  // Fill the Stub with a tail call that will be optimized.
+  BasicBlock *block = BasicBlock::Create(M.getContext(), "entry", Stub);
+  IRBuilder<> block_builder(block);
+  SmallVector<Value *, 8> args;
+  for (auto &arg : Stub->args())
+    args.push_back(&arg);
+
+  CallInst *call;
+  if (auto *F = dyn_cast<Function>(GV)) {
+    call = block_builder.CreateCall(F, args);
+    call->setAttributes(F->getAttributes());
+  } else {
+    call = block_builder.CreateCall(cast<FunctionType>(GV->getValueType()), GV,
+                                    args);
+  }
+
+  call->setTailCall();
+  if (call->getType()->isVoidTy())
+    block_builder.CreateRetVoid();
+  else
+    block_builder.CreateRet(call);
+
+  return Stub;
+}
+
+Value *LowerDSOLocalEquivalent(DSOLocalEquivalent *Equiv) {
+  GlobalValue *GV = Equiv->getGlobalValue();
+  if (GV->hasLocalLinkage() || GV->hasHiddenVisibility() || GV->isDSOLocal())
+    return GV;
+  return GetOrCreateRelativeStub(GV);
+}
+
+bool CheckValue(Value *Val) {
+  if (auto *Equiv = dyn_cast<DSOLocalEquivalent>(Val)) {
+    Value *NewVal = LowerDSOLocalEquivalent(Equiv);
+    Equiv->replaceAllUsesWith(NewVal);
+    return true;
+  }
+  return false;
+}
+
+bool CheckFunction(Function &F) {
+  bool Changed = false;
+  for (BasicBlock &BB : F) {
+    for (auto BBI = BB.begin(); BBI != BB.end(); ++BBI) {
+      for (Instruction &I : BB) {
+        for (auto ValIt = I.value_op_begin(); ValIt != I.value_op_end();
+             ++ValIt)
+          Changed |= CheckValue(*ValIt);
+      }
+    }
+  }
+  return Changed;
+}
+
+class DSOLocalEquivalentLoweringLegacyPass : public ModulePass {
+public:
+  static char ID;
+  const TargetMachine *TM;
+
+  DSOLocalEquivalentLoweringLegacyPass(const TargetMachine *TM)
+      : ModulePass(ID), TM(TM) {
+    assert(!TM->getObjFileLowering()->supportDSOLocalEquivalentLowering() &&
+           "This pass should not need to be run on targets that can natively "
+           "lower dso_local_equivalent");
+  }
+
+  bool runOnModule(Module &M) override {
+    bool Changed = false;
+    for (GlobalVariable &GV : M.globals()) {
+      if (GV.hasInitializer())
+        Changed |= CheckValue(GV.getInitializer());
+    }
+    for (Function &F : M)
+      Changed |= CheckFunction(F);
+    return Changed;
+  }
+};
+
+} // namespace
+
+char DSOLocalEquivalentLoweringLegacyPass::ID;
+
+ModulePass *createDSOLocalEquivalentLoweringPass(const TargetMachine *TM) {
+  return new DSOLocalEquivalentLoweringLegacyPass(TM);
+}
+
+} // namespace llvm

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

if (C->getType()->isStructTy() || C->getType()->isArrayTy()) {
       }
 
       return DAG.getMergeValues(Constants, getCurSDLoc());
     }
 
     if (const BlockAddress *BA = dyn_cast<BlockAddress>(C))
       return DAG.getBlockAddress(BA, VT);
 
+    if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(C))
+      return getValue(Equiv->getGlobalValue());
+
     VectorType *VecTy = cast<VectorType>(V->getType());
 
     // Now that we know the number and type of the elements, get that number of
     // elements into the Ops array based on what kind of constant it is.
     if (const ConstantVector *CV = dyn_cast<ConstantVector>(C)) {
       SmallVector<SDValue, 16> Ops;
       unsigned NumElements = cast<FixedVectorType>(VecTy)->getNumElements();
       for (unsigned i = 0; i != NumElements; ++i)

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

for (const auto &MFE: ModuleFlags) {
     }
   }
 }
 
 //===----------------------------------------------------------------------===//
 //                                  ELF
 //===----------------------------------------------------------------------===//
 
+TargetLoweringObjectFileELF::TargetLoweringObjectFileELF()
+    : TargetLoweringObjectFile() {
+  SupportDSOLocalEquivalentLowering = true;
+}
+
 void TargetLoweringObjectFileELF::Initialize(MCContext &Ctx,
                                              const TargetMachine &TgtM) {
   TargetLoweringObjectFile::Initialize(Ctx, TgtM);
   TM = &TgtM;
 
   CodeModel::Model CM = TgtM.getCodeModel();
   InitializeELF(TgtM.Options.UseInitArray);
 
if (LHS->getType()->getPointerAddressSpace() != 0 ||
     return nullptr;
 
   return MCBinaryExpr::createSub(
       MCSymbolRefExpr::create(TM.getSymbol(LHS), PLTRelativeVariantKind,
                               getContext()),
       MCSymbolRefExpr::create(TM.getSymbol(RHS), getContext()), getContext());
 }
 
+const MCExpr *TargetLoweringObjectFileELF::lowerDSOLocalEquivalent(
+    const DSOLocalEquivalent *Equiv, const TargetMachine &TM) const {
+  assert(supportDSOLocalEquivalentLowering());
+  return MCSymbolRefExpr::create(TM.getSymbol(Equiv->getGlobalValue()),
+                                 PLTRelativeVariantKind, getContext());
+}
+
 MCSection *TargetLoweringObjectFileELF::getSectionForCommandLines() const {
   // Use ".GCC.command.line" since this feature is to support clang's
   // -frecord-gcc-switches which in turn attempts to mimic GCC's switch of the
   // same name.
   return getContext().getELFSection(".GCC.command.line", ELF::SHT_PROGBITS,
                                     ELF::SHF_MERGE | ELF::SHF_STRINGS, 1, "");
 }
 

llvm/lib/CodeGen/TargetPassConfig.cpp

 #include "llvm/Pass.h"
 #include "llvm/Support/CodeGen.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/SaveAndRestore.h"
 #include "llvm/Support/Threading.h"
+#include "llvm/Target/TargetLoweringObjectFile.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Utils.h"
 #include "llvm/Transforms/Utils/SymbolRewriter.h"
 #include <cassert>
 #include <string>
 
 using namespace llvm;
bool TargetPassConfig::addCoreISelPasses() {
   printAndVerify("After Instruction Selection");
 
   return false;
 }
 
 bool TargetPassConfig::addISelPasses() {
   if (TM->useEmulatedTLS())
     addPass(createLowerEmuTLSPass());
+  if (!TM->getObjFileLowering()->supportDSOLocalEquivalentLowering())
+    addPass(createDSOLocalEquivalentLoweringPass(TM));
 
   addPass(createPreISelIntrinsicLoweringPass());
   PM->add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
   addIRPasses();
   addCodeGenPrepare();
   addPassesToHandleExceptions();
   addISelPrepare();
 

llvm/lib/IR/AsmWriter.cpp

WriteAsOperandInternal(Out, BA->getFunction(), &TypePrinter, Machine,
                            Context);
     Out << ", ";
     WriteAsOperandInternal(Out, BA->getBasicBlock(), &TypePrinter, Machine,
                            Context);
     Out << ")";
     return;
   }
 
+  if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(CV)) {
+    Out << "dso_local_equivalent ";
+    WriteAsOperandInternal(Out, Equiv->getGlobalValue(), &TypePrinter, Machine,
+                           Context);
+    return;
+  }
+
   if (const ConstantArray *CA = dyn_cast<ConstantArray>(CV)) {
     Type *ETy = CA->getType()->getElementType();
     Out << '[';
     TypePrinter.print(ETy, Out);
     Out << ' ';
     WriteAsOperandInternal(Out, CA->getOperand(0),
                            &TypePrinter, Machine,
                            Context);

llvm/lib/IR/Constants.cpp

case Constant::ConstantDataVectorVal:
     delete static_cast<ConstantDataVector *>(C);
     break;
   case Constant::ConstantTokenNoneVal:
     delete static_cast<ConstantTokenNone *>(C);
     break;
   case Constant::BlockAddressVal:
     delete static_cast<BlockAddress *>(C);
     break;
+  case Constant::DSOLocalEquivalentVal:
+    delete static_cast<DSOLocalEquivalent *>(C);
+    break;
   case Constant::UndefValueVal:
     delete static_cast<UndefValue *>(C);
     break;
   case Constant::ConstantExprVal:
     if (isa<UnaryConstantExpr>(C))
       delete static_cast<UnaryConstantExpr *>(C);
     else if (isa<BinaryConstantExpr>(C))
       delete static_cast<BinaryConstantExpr *>(C);
if (CE->getOpcode() == Instruction::Sub) {
         // function.  This is a common idiom when creating a table for the
         // indirect goto extension, so we handle it efficiently here.
         if (isa<BlockAddress>(LHSOp0) && isa<BlockAddress>(RHSOp0) &&
             cast<BlockAddress>(LHSOp0)->getFunction() ==
                 cast<BlockAddress>(RHSOp0)->getFunction())
           return false;
 
         // Relative pointers do not need to be dynamically relocated.
-        if (auto *LHSGV = dyn_cast<GlobalValue>(LHSOp0->stripPointerCasts()))
-          if (auto *RHSGV = dyn_cast<GlobalValue>(RHSOp0->stripPointerCasts()))
+        if (auto *RHSGV =
+                dyn_cast<GlobalValue>(RHSOp0->stripInBoundsConstantOffsets())) {
+          auto *LHS = LHSOp0->stripInBoundsConstantOffsets();
+          if (auto *LHSGV = dyn_cast<GlobalValue>(LHS)) {
             if (LHSGV->isDSOLocal() && RHSGV->isDSOLocal())
               return false;
+          } else if (isa<DSOLocalEquivalent>(LHS)) {
+            if (RHSGV->isDSOLocal())
+              return false;
+          }
+        }
       }
     }
   }
 
   bool Result = false;
   for (unsigned i = 0, e = getNumOperands(); i != e; ++i)
     Result |= cast<Constant>(getOperand(i))->needsRelocation();
 
Value *BlockAddress::handleOperandChangeImpl(Value *From, Value *To) {
   setOperand(1, NewBB);
   getBasicBlock()->AdjustBlockAddressRefCount(1);
 
   // If we just want to keep the existing value, then return null.
   // Callers know that this means we shouldn't delete this value.
   return nullptr;
 }
 
+DSOLocalEquivalent *DSOLocalEquivalent::get(GlobalValue *GV) {
+  DSOLocalEquivalent *&Equiv = GV->getContext().pImpl->DSOLocalEquivalents[GV];
+  if (!Equiv)
+    Equiv = new DSOLocalEquivalent(GV);
+
+  assert(Equiv->getGlobalValue() == GV &&
+         "DSOLocalFunction does not match the expected global value");
+  return Equiv;
+}
+
+DSOLocalEquivalent::DSOLocalEquivalent(GlobalValue *GV)
+    : Constant(GV->getType(), Value::DSOLocalEquivalentVal, &Op<0>(), 1) {
+  setOperand(0, GV);
+}
+
+/// Remove the constant from the constant table.
+void DSOLocalEquivalent::destroyConstantImpl() {
+  const GlobalValue *GV = getGlobalValue();
+  GV->getContext().pImpl->DSOLocalEquivalents.erase(GV);
+}
+
+Value *DSOLocalEquivalent::handleOperandChangeImpl(Value *From, Value *To) {
+  assert(From == getGlobalValue() && "Changing value does not match operand.");
+  assert(To->getType() == getType() && "Mismatched types");
+  assert(isa<Constant>(To) && "Can only replace the operands with a constant");
+
+  // The replacement is with another global value.
+  if (const auto *ToObj = dyn_cast<GlobalValue>(To)) {
+    DSOLocalEquivalent *&NewEquiv =
+        getContext().pImpl->DSOLocalEquivalents[ToObj];
+    if (NewEquiv)
+      return NewEquiv;
+  }
+
+  // The replacement could be a bitcast or an alias to another function. We can
+  // replace it with a bitcast to the dso_local_equivalent of that function.
+  auto *Func = cast<Function>(To->stripPointerCastsAndAliases());
+  DSOLocalEquivalent *&NewEquiv = getContext().pImpl->DSOLocalEquivalents[Func];
+  if (NewEquiv)
+    return llvm::ConstantExpr::getBitCast(NewEquiv, getType());
+
+  // Replace this with the new one.
+  getContext().pImpl->DSOLocalEquivalents.erase(getGlobalValue());
+  NewEquiv = this;
+  setOperand(0, Func);
+  return nullptr;
+}
+
 //---- ConstantExpr::get() implementations.
 //
 
 /// This is a utility function to handle folding of casts and lookup of the
 /// cast in the ExprConstants map. It is used by the various get* methods below.
 static Constant *getFoldedCast(Instruction::CastOps opc, Constant *C, Type *Ty,
                                bool OnlyIfReduced = false) {
   assert(Ty->isFirstClassType() && "Cannot cast to an aggregate type!");

llvm/lib/IR/LLVMContextImpl.h

#include "llvm/IR/Metadata.def"
   DenseMap<PointerType *, std::unique_ptr<ConstantPointerNull>> CPNConstants;
 
   DenseMap<Type *, std::unique_ptr<UndefValue>> UVConstants;
 
   StringMap<ConstantDataSequential*> CDSConstants;
 
   DenseMap<std::pair<const Function *, const BasicBlock *>, BlockAddress *>
     BlockAddresses;
+
+  DenseMap<const GlobalValue *, DSOLocalEquivalent *> DSOLocalEquivalents;
+
   ConstantUniqueMap<ConstantExpr> ExprConstants;
 
   ConstantUniqueMap<InlineAsm> InlineAsms;
 
   ConstantInt *TheTrueVal = nullptr;
   ConstantInt *TheFalseVal = nullptr;
 
   std::unique_ptr<ConstantTokenNone> TheNoneToken;

llvm/test/CodeGen/X86/dso_local_equivalent.ll

+; RUN: llc -mtriple=x86_64-linux-gnu -relocation-model=pic -data-sections -o - %s | FileCheck %s -check-prefix=PLT
+; RUN: llc -mtriple=x86_64-apple-darwin -relocation-model=pic -data-sections -o - %s | FileCheck %s -check-prefix=NO-PLT
+
+declare void @extern_func()
+
+; NO-PLT:      _extern_func.stub:
+; NO-PLT:         jmp     _extern_func
+; NO-PLT-NOT:  _hidden_func.stub
+; NO-PLT-NOT:  _dso_local_func.stub
+; NO-PLT-NOT:  _internal_func.stub
+; NO-PLT-NOT:  l_private_func.stub
+; NO-PLT:      _ifunc_func.stub:
+; NO-PLT:         jmp     _ifunc_func
+; NO-PLT:      _alias_func.stub:
+; NO-PLT:         jmp     _alias_func
+; NO-PLT-NOT:  _dso_local_alias_func.stub:
+
+; PLT:         data:
+; PLT-NEXT:      .quad   extern_func@PLT
+; NO-PLT:      _data:
+; NO-PLT-NEXT:   .quad   _extern_func.stub
+@data = constant void ()* dso_local_equivalent @extern_func
+
+declare hidden void @hidden_func()
+declare dso_local void @dso_local_func()
+define internal void @internal_func() {
+entry:
+  ret void
+}
+define private void @private_func() {
+entry:
+  ret void
+}
+
+; PLT:         data2:
+; PLT-NEXT:      .quad   hidden_func@PLT
+; NO-PLT:      _data2:
+; NO-PLT-NEXT:   .quad   _hidden_func
+
+; PLT:         data3:
+; PLT-NEXT:      .quad   dso_local_func@PLT
+; NO-PLT:      _data3:
+; NO-PLT-NEXT:   .quad   _dso_local_func
+
+; PLT:         data4:
+; PLT-NEXT:      .quad   internal_func@PLT
+; NO-PLT:      _data4:
+; NO-PLT-NEXT:   .quad   _internal_func
+
+; PLT:         data5:
+; PLT-NEXT:      .quad   .Lprivate_func@PLT
+; NO-PLT:      _data5:
+; NO-PLT-NEXT:   .quad   l_private_func
+@data2 = constant void ()* dso_local_equivalent @hidden_func
+@data3 = constant void ()* dso_local_equivalent @dso_local_func
+@data4 = constant void ()* dso_local_equivalent @internal_func
+@data5 = constant void ()* dso_local_equivalent @private_func
+
+
+@ifunc_func = ifunc i32 (i32), i64 ()* @resolver
+
+define internal i64 @resolver() {
+entry:
+  ret i64 0
+}
+
+; PLT:         data_ifunc:
+; PLT-NEXT:      .quad   ifunc_func@PLT
+; NO-PLT:      _data_ifunc:
+; NO-PLT-NEXT:   .quad   _ifunc_func.stub
+@data_ifunc = constant i32 (i32)* dso_local_equivalent @ifunc_func
+
+define void @aliasee_func() {
+entry:
+  ret void
+}
+
+@alias_func = alias void (), void ()* @aliasee_func
+@dso_local_alias_func = dso_local alias void (), void ()* @aliasee_func
+
+; PLT:         data_alias:
+; PLT-NEXT:      .quad   alias_func@PLT
+; NO-PLT:      _data_alias:
+; NO-PLT-NEXT:   .quad   _alias_func.stub
+@data_alias = constant void ()* dso_local_equivalent @alias_func
+
+; PLT:         data_dso_local_alias:
+; PLT-NEXT:      .quad   dso_local_alias_func@PLT
+; NO-PLT:      _data_dso_local_alias:
+; NO-PLT-NEXT:   .quad   _dso_local_alias_func
+@data_dso_local_alias = constant void ()* dso_local_equivalent @dso_local_alias_func

llvm/test/CodeGen/X86/relptr-rodata.ll

 ; CHECK: relro1:
 ; CHECK: .long default-relro1
 @relro1 = hidden constant i32 trunc (i64 sub (i64 ptrtoint (i8* @default to i64), i64 ptrtoint (i32* @relro1 to i64)) to i32)
 
 ; CHECK: .section .data.rel.ro.relro2
 ; CHECK: relro2:
 ; CHECK: .long hidden-relro2
 @relro2 = constant i32 trunc (i64 sub (i64 ptrtoint (i8* @hidden to i64), i64 ptrtoint (i32* @relro2 to i64)) to i32)
+
+; CHECK:      .section .rodata.cst8
+; CHECK-NEXT: .globl obj
+; CHECK:      obj:
+; CHECK:      .long 0
+; CHECK:      .long (hidden_func-obj)-4
+
+declare hidden void @hidden_func()
+
+; Ensure that inbound GEPs with constant offsets are also resolved.
+@obj = dso_local unnamed_addr constant { { i32, i32 } } {
+  { i32, i32 } {
+    i32 0,
+    i32 trunc (i64 sub (i64 ptrtoint (void ()* dso_local_equivalent @hidden_func to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i32, i32 } }, { { i32, i32 } }* @obj, i32 0, i32 0, i32 1) to i64)) to i32)
+  } }, align 4
+
+; CHECK:      .section .rodata.rodata2
+; CHECK-NEXT: .globl rodata2
+; CHECK:      rodata2:
+; CHECK:      .long extern_func@PLT-rodata2
+
+declare void @extern_func() unnamed_addr
+
+@rodata2 = hidden constant i32 trunc (i64 sub (i64 ptrtoint (void ()* dso_local_equivalent @extern_func to i64), i64 ptrtoint (i32* @rodata2 to i64)) to i32)