Make it easier to use DwarfContext with MCJIT

[loladiro]

Summary

This supersedes http://reviews.llvm.org/D4010, hopefully properly dealing with the JIT case and also adds an actual test case. DwarfContext was basically already usable for the JIT (and back when we were overwriting ELF files it actually worked out of the box by accident), but in order to resolve relocations correctly it needs to know the load address of the section. Rather than trying to get this out of the ObjectFile or requiring the user to create a new ObjectFile just to get some debug info, this adds the capability to pass in that info directly. As part of this I separated out part of the LoadedObjectInfo struct from RuntimeDyld, since it is now required add a higher layer. For now I put that in ObjectFile.h, but I would be happy to receive suggestions for more appropriate places.

[loladiro]

Accidentally forgot newly added test cases in previous diff.

[• rafael]

include/llvm/Object/ELFObjectFile.h

569 ↗

(On Diff #18122)

Not directly, but It is valid to write

.long .text

and it is represented as a relocation to a symbol, and the symbol has type STT_SECTION.

Would it be desirable to check for that here? If not, please expand the comment a bit.

include/llvm/Object/MachO.h

249 ↗

(On Diff #18122)

Please return an ErrorOr<uint8_t>.

include/llvm/Object/RelocVisitor.h

278 ↗

(On Diff #18122)

As you can see it is way too easy to ignore a std::error_code return :-)

lib/Object/COFFObjectFile.cpp

1040 ↗

(On Diff #18122)

Someone familiar with COFF should check this.

lib/Object/MachOObjectFile.cpp

1010 ↗

(On Diff #18122)

Actually, if this is never going to fail, just return a uint8_t.

[lhames]

Hi Keno,

Is the plan here just to get debugging working for MachO objects?

My intention was that people wanting to interface between the debugger and the JIT should use LoadedObjectInfo::getObjectForDebug(), and that should return a new file that's appropriate for parsing with DWARFContext.

For ELF files, LoadedELFObjectInfo::getObjectForDebug() is doing the same thing that RuntimeDyldELF used to and overwriting the vmaddrs in the ObjectFile, but it's duplicating the ObjectFile first to ensure that that's safe.

For MachO files my plan was to do the same thing, using your old patch for MachO debugging. We'd just need to hook that up in LoadedMachOObjectInfo.

• Lang.

[loladiro]

Hi Lang,

No, this is not for the debugger. This is for the client itself to be able to parse the DWARF information. This is the last piece I need to put MCJIT back on par functionality wise with the old jit (was was able to give you line table information).

Keno

[lhames]

Sorry - that was a bit vague of me. I should have said that I intend for getObjectForDebug to work with DWARFContext, not just the debugger.

Is creating a new object file too much overhead? We haven't gone this far in LoadedELFObjectFile yet, but rather than duplicating the original object file, I think it should be reasonable to build a new object file that just contains a symbol table and load commands.

I'm not hugely opposed to the direction of this patch. It's actually very close to what I pitched to echristo when I first cleaned up RuntimeDyld's memory ownership model. At the time though, he convinced me that it would be unpleasant to thread the optional "object interpretation metadata" through the interfaces, hence the decision to just form a new self-contained object file.

[loladiro]

Well, I see three reasons for this direction over the other:

1. We can avoid relocation the debug sections. And in fact, since all this is done lazily, we only ever need to process the sections that we actually need (quite a win from a memory and performance standpoint).
2. I was never really happy with the way I ended up solving the debug object problem for MachO. A lot of things in MachO refer to the original layout of the object file and fixing them all up is a pain. It can be done, but it always feels like a hack to me. In contrast, other than the admittedly ugly need to pass the LoadedObjectInfo to the DwarfContext, this seems rather sane to me.
3. I think it improves reusability of the code. I think at some point the whole DWARF library will need to be split into a separate library (it's already duplicated between LLVM and lldb) and in other contexts people might not like to create such a debug object. Even with this patch, I imagine it might be quite possible to create a LoadedObjectInfo subclass for a different JIT and use the DWARF parser with it (admittedly this is probably a minor point).

Also note that this patch isn't actually that big. It's a little inflated by adding test cases, exposing this functionality through llvm-rtdyld and adding the missing ObjectFile methods for handling MachO relocations.

[loladiro]

Addresses @rafael's review comments. Added implementations of getRelocationSection for ELF and COFF, which aren't strictly necessary, since it's equivalent on ELF/COFF to get the symbols' section, which is what we do anyway. Nevertheless, better to have the API complete in case somebody wants to use it for a different use case.

[friss]

Your patch seems to touch a few different and pretty independent parts on the compiler/tools. It looks like this could be split up quite a bit. Just commenting on the libDebugInfo code:

include/llvm/Object/RelocVisitor.h

228–240 ↗

(On Diff #18303)

I'd suggest to submit that in a separate patch too. It can be tested quite simply by verifying that llvm-dwarfdump doesn't emit any relocation processing "errors" anymore on x86_64 macho object files.

lib/DebugInfo/DWARFContext.cpp

557–559 ↗

(On Diff #18303)

The { should be on the same line as the if. If you don't use the LoadAddress variable out of the if, you can move its declaration inside the condition.

561 ↗

(On Diff #18303)

Wouldn't it be cleaner to have a method inside the LoadedObjectInfo that directly returns a StringRef (or ArrayRef) to the section data?

633–637 ↗

(On Diff #18303)

No need for the variable, just put the condition in the if. We only care about debug sections here, can this ever be true for the debug sections? Will the JIT load them and relocate them?

680–701 ↗

(On Diff #18303)

You introduce section lookups by name for every relocation. Isn't there a more efficient interface? Also, I think I would prefer the code for the LoadedObjectInfo case to be in its own if() rather than mixed with the standard relocation processing (So that it's more self-evident that it's only adding a new code path).

lib/DebugInfo/DWARFDebugLine.cpp

608–610 ↗

(On Diff #18303)

This part of the change should be submitted separately with appropriate test coverage.

Not suggesting to do it for this patch, but looking more globally, I wonder if this function should really be returning results that span multiple sequences or if we shouldn't simply modify it to only look in the first sequence containing the start address.
Or maybe we need to change DWARFContext::getLineInfoForAddressRange() to interpret the results differently. Right now, if your result spans multiple sequences, it'll return line mappings for the EndSequence rows which could be very misleading.

[loladiro]

I'll split up the patch for further in depth review, but I wanted to have a chance to have everybody see how everything plays together. In particular, I'd like @echristo to comment whether the general direction of this patch seems reasonable. If that's the case, I'll split the patch into three parts (The debug info changes, the LoadedObjectInfo integration with DwarfContext and the Darwin support for RelocVisitor).

lib/DebugInfo/DWARFContext.cpp

561 ↗

(On Diff #18303)

That's a good idea.

633–637 ↗

(On Diff #18303)

Yes, the JIT will load and relocate debug info in some cases.

680–701 ↗

(On Diff #18303)

Currently there's only the lookup by name. I'm not sure if the RuntimeDyld SectionIDs have any relation to the section order in the object file. @lhames?

lib/DebugInfo/DWARFDebugLine.cpp

608–610 ↗

(On Diff #18303)

Will do. I had just noticed it while testing this.

[loladiro]

This rebases the current diff and splits out the MachO RelocVisitor changes into http://reviews.llvm.org/D8148.

[loladiro]

@echristo @lhames As discussed on IRC, rebased to make it independent of the earlier MachO patch (because that one is not easily testable without this).

[lhames]

The RuntimeDyld side of this looks good to me.

[loladiro]

@echristo Can you take a look?

[loladiro]

Could somebody please take a look at the RelocVisitor part of this? This patch shouldn't break anything and it's very useful when working with MCJIT.

[echristo]

Added some comments, looks like bits overlap with the other patch. Some more inline comments as well. I'll take another look after you get this stuff.

-eric

include/llvm/DebugInfo/DIContext.h

147–156 ↗

(On Diff #25369)

Needs doxygen docs for the functions.

lib/DebugInfo/DWARF/DWARFContext.cpp

557–562 ↗

(On Diff #25369)

Style.

679–704 ↗

(On Diff #25369)

Can you comment this out? It looks a little odd and it's hard to get straight.

[loladiro]

Updated with more comments. I also flattened the if nest. Hopefully that should be easier to understand now.

[echristo]

Few more inline comments.

include/llvm/DebugInfo/DIContext.h

147 ↗

(On Diff #26269)

Sentences end with a period. Also we have \brief support turned on so you don't need to use \brief anymore. (And in the other places :)

173 ↗

(On Diff #26269)

clone is entirely unused.

lib/DebugInfo/DWARF/DWARFContext.cpp

685–687 ↗

(On Diff #26269)

LLVM style is that single statement blocks don't get braces if possible.

700–702 ↗

(On Diff #26269)

Ditto.

tools/llvm-rtdyld/llvm-rtdyld.cpp

261–269 ↗

(On Diff #26269)

Explanatory comment here. (It looks obvious, but I like them in general. :)

[loladiro]

include/llvm/DebugInfo/DIContext.h

147 ↗

(On Diff #26269)

Will fix.

173 ↗

(On Diff #26269)

It's supposed to be available for the client to be able to obtain a copy of the LoadedObjectInfo, since the one it gets from MCJIT is passed by reference. We could probably change that one to a shared_ptr, but I didn't want to put that API breakage into the same commit.

[loladiro]

More comments and style updates.

[echristo]

include/llvm/DebugInfo/DIContext.h

173 ↗

(On Diff #26269)

I guess, it might be nicer to make this clear. I'm not a fan of requiring people to clone it etc. Maybe I'm just not seeing where it's going to be used off the top of my head, and can be added in the future if we have a user of the functionality?

[loladiro]

include/llvm/DebugInfo/DIContext.h

173 ↗

(On Diff #26269)

I added this because I need it in Julia. The MCJIT listener interface is

NotifyObjectEmitted(const ObjectFile &Object, const RuntimeDyld::LoadedObjectInfo &L)

but I need to store the LoadedObjectInfo for later (to query the DWARFContext), so I added the clone method as a workaround. Better alternatives would be most appreciated of course.

[echristo]

include/llvm/DebugInfo/DIContext.h

173 ↗

(On Diff #26269)

Not my favorite set of interfaces, no, but I don't have anything else in mind at the moment and hopefully someone else can come up with something better.

[dblaikie]

include/llvm/DebugInfo/DIContext.h

173 ↗

(On Diff #26269)

This is a bit problematic as the code is entirely untested...

For example - I've fixed the API to use unique_ptr, which may break your usage & I have little idea if my change is correct.

Please add some kind of test coverage in-tree or it'd be best to remove this API.

[grimar]

This change introduced getLoadedSectionContents(), but it is not used anywhere in LLVM code I think. Do we need it ?

llvm/trunk/include/llvm/DebugInfo/DIContext.h

public:
   virtual DILineInfoTable getLineInfoForAddressRange(uint64_t Address,
       uint64_t Size, DILineInfoSpecifier Specifier = DILineInfoSpecifier()) = 0;
   virtual DIInliningInfo getInliningInfoForAddress(uint64_t Address,
       DILineInfoSpecifier Specifier = DILineInfoSpecifier()) = 0;
 private:
   const DIContextKind Kind;
 };
 
+/// An inferface for inquiring the load address of a loaded object file
+/// to be used by the DIContext implementations when applying relocations
+/// on the fly.
+class LoadedObjectInfo {
+public:
+  LoadedObjectInfo() {}
+  virtual ~LoadedObjectInfo() {}
+
+  /// Obtain the Load Address of a section by Name.
+  ///
+  /// Calculate the address of the section identified by the passed in Name.
+  /// The section need not be present in the local address space. The addresses
+  /// need to be consistent with the addresses used to query the DIContext and
+  /// the output of this function should be deterministic, i.e. repeated calls with
+  /// the same Name should give the same address.
+  virtual uint64_t getSectionLoadAddress(StringRef Name) const = 0;
+
+  /// If conveniently available, return the content of the given Section.
+  ///
+  /// When the section is available in the local address space, in relocated (loaded)
+  /// form, e.g. because it was relocated by a JIT for execution, this function
+  /// should provide the contents of said section in `Data`. If the loaded section
+  /// is not available, or the cost of retrieving it would be prohibitive, this
+  /// function should return false. In that case, relocations will be read from the
+  /// local (unrelocated) object file and applied on the fly. Note that this method
+  /// is used purely for optimzation purposes in the common case of JITting in the
+  /// local address space, so returning false should always be correct.
+  virtual bool getLoadedSectionContents(StringRef Name, StringRef &Data) const {
+    return false;
+  }
+
+  /// Obtain a copy of this LoadedObjectInfo.
+  ///
+  /// The caller is responsible for deallocation once the copy is no longer required.
+  virtual LoadedObjectInfo *clone() const = 0;
+};
+
 }
 
 #endif

llvm/trunk/include/llvm/DebugInfo/DWARF/DWARFContext.h

class DWARFContextInMemory : public DWARFContext {
   DWARFSection AppleNamesSection;
   DWARFSection AppleTypesSection;
   DWARFSection AppleNamespacesSection;
   DWARFSection AppleObjCSection;
 
   SmallVector<SmallString<32>, 4> UncompressedSections;
 
 public:
-  DWARFContextInMemory(const object::ObjectFile &Obj);
+  DWARFContextInMemory(const object::ObjectFile &Obj,
+    const LoadedObjectInfo *L = nullptr);
   bool isLittleEndian() const override { return IsLittleEndian; }
   uint8_t getAddressSize() const override { return AddressSize; }
   const DWARFSection &getInfoSection() override { return InfoSection; }
   const TypeSectionMap &getTypesSections() override { return TypesSections; }
   StringRef getAbbrevSection() override { return AbbrevSection; }
   const DWARFSection &getLocSection() override { return LocSection; }
   StringRef getARangeSection() override { return ARangeSection; }
   StringRef getDebugFrameSection() override { return DebugFrameSection; }

llvm/trunk/include/llvm/ExecutionEngine/RuntimeDyld.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_EXECUTIONENGINE_RUNTIMEDYLD_H
 #define LLVM_EXECUTIONENGINE_RUNTIMEDYLD_H
 
 #include "JITSymbolFlags.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Memory.h"
+#include "llvm/DebugInfo/DIContext.h"
 #include <memory>
 
 namespace llvm {
 
 namespace object {
   class ObjectFile;
   template <typename T> class OwningBinary;
 }
SymbolInfo(uint64_t Address, JITSymbolFlags Flags)
       : JITSymbolBase(Flags), Address(Address) {}
     explicit operator bool() const { return Address != 0; }
     uint64_t getAddress() const { return Address; }
   private:
     uint64_t Address;
   };
 
   /// \brief Information about the loaded object.
-  class LoadedObjectInfo {
+  class LoadedObjectInfo : public llvm::LoadedObjectInfo {
     friend class RuntimeDyldImpl;
   public:
     LoadedObjectInfo(RuntimeDyldImpl &RTDyld, unsigned BeginIdx,
                      unsigned EndIdx)
       : RTDyld(RTDyld), BeginIdx(BeginIdx), EndIdx(EndIdx) { }
 
     virtual ~LoadedObjectInfo() {}
 

llvm/trunk/include/llvm/Object/COFF.h

public:
   bool isAnyUndefined() const {
     return isUndefined() || isWeakExternal();
   }
 
   bool isFileRecord() const {
     return getStorageClass() == COFF::IMAGE_SYM_CLASS_FILE;
   }
 
+  bool isSection() const {
+    return getStorageClass() == COFF::IMAGE_SYM_CLASS_SECTION;
+  }
+
   bool isSectionDefinition() const {
     // C++/CLI creates external ABS symbols for non-const appdomain globals.
     // These are also followed by an auxiliary section definition.
     bool isAppdomainGlobal =
         getStorageClass() == COFF::IMAGE_SYM_CLASS_EXTERNAL &&
         getSectionNumber() == COFF::IMAGE_SYM_ABSOLUTE;
     bool isOrdinarySection = getStorageClass() == COFF::IMAGE_SYM_CLASS_STATIC;
     if (!getNumberOfAuxSymbols())
protected:
   relocation_iterator section_rel_end(DataRefImpl Sec) const override;
 
   void moveRelocationNext(DataRefImpl &Rel) const override;
   std::error_code getRelocationAddress(DataRefImpl Rel,
                                        uint64_t &Res) const override;
   std::error_code getRelocationOffset(DataRefImpl Rel,
                                       uint64_t &Res) const override;
   symbol_iterator getRelocationSymbol(DataRefImpl Rel) const override;
+  section_iterator getRelocationSection(DataRefImpl Rel) const override;
   std::error_code getRelocationType(DataRefImpl Rel,
                                     uint64_t &Res) const override;
   std::error_code
   getRelocationTypeName(DataRefImpl Rel,
                         SmallVectorImpl<char> &Result) const override;
   std::error_code
   getRelocationValueString(DataRefImpl Rel,
                            SmallVectorImpl<char> &Result) const override;

llvm/trunk/include/llvm/Object/ELFObjectFile.h

std::error_code getSymbolAddress(DataRefImpl Symb,
                                    uint64_t &Res) const override;
   std::error_code getSymbolAlignment(DataRefImpl Symb,
                                      uint32_t &Res) const override;
   std::error_code getSymbolSize(DataRefImpl Symb, uint64_t &Res) const override;
   uint32_t getSymbolFlags(DataRefImpl Symb) const override;
   std::error_code getSymbolOther(DataRefImpl Symb, uint8_t &Res) const override;
   std::error_code getSymbolType(DataRefImpl Symb,
                                 SymbolRef::Type &Res) const override;
+  section_iterator getSymbolSection(const Elf_Sym *Symb) const;
   std::error_code getSymbolSection(DataRefImpl Symb,
                                    section_iterator &Res) const override;
 
   void moveSectionNext(DataRefImpl &Sec) const override;
   std::error_code getSectionName(DataRefImpl Sec,
                                  StringRef &Res) const override;
   uint64_t getSectionAddress(DataRefImpl Sec) const override;
   uint64_t getSectionSize(DataRefImpl Sec) const override;
protected:
   section_iterator getRelocatedSection(DataRefImpl Sec) const override;
 
   void moveRelocationNext(DataRefImpl &Rel) const override;
   std::error_code getRelocationAddress(DataRefImpl Rel,
                                        uint64_t &Res) const override;
   std::error_code getRelocationOffset(DataRefImpl Rel,
                                       uint64_t &Res) const override;
   symbol_iterator getRelocationSymbol(DataRefImpl Rel) const override;
+  section_iterator getRelocationSection(DataRefImpl Rel) const override;
   std::error_code getRelocationType(DataRefImpl Rel,
                                     uint64_t &Res) const override;
   std::error_code
   getRelocationTypeName(DataRefImpl Rel,
                         SmallVectorImpl<char> &Result) const override;
   std::error_code
   getRelocationValueString(DataRefImpl Rel,
                            SmallVectorImpl<char> &Result) const override;
uint32_t ELFObjectFile<ELFT>::getSymbolFlags(DataRefImpl Symb) const {
 
   if (ESym->getVisibility() == ELF::STV_HIDDEN)
     Result |= SymbolRef::SF_Hidden;
 
   return Result;
 }
 
 template <class ELFT>
-std::error_code
-ELFObjectFile<ELFT>::getSymbolSection(DataRefImpl Symb,
-                                      section_iterator &Res) const {
-  const Elf_Sym *ESym = getSymbol(Symb);
+section_iterator
+ELFObjectFile<ELFT>::getSymbolSection(const Elf_Sym *ESym) const {
   const Elf_Shdr *ESec = EF.getSection(ESym);
   if (!ESec)
-    Res = section_end();
+    return section_end();
   else {
     DataRefImpl Sec;
     Sec.p = reinterpret_cast<intptr_t>(ESec);
-    Res = section_iterator(SectionRef(Sec, this));
-  }
+    return section_iterator(SectionRef(Sec, this));
+  }
+
+template <class ELFT>
+std::error_code
+ELFObjectFile<ELFT>::getSymbolSection(DataRefImpl Symb,
+                                      section_iterator &Res) const {
+  Res = getSymbolSection(getSymbol(Symb));
   return object_error::success;
 }
 
 template <class ELFT>
 void ELFObjectFile<ELFT>::moveSectionNext(DataRefImpl &Sec) const {
   Sec = toDRI(++toELFShdrIter(Sec));
 }
 
ELFObjectFile<ELFT>::getRelocationSymbol(DataRefImpl Rel) const {
   case ELF::SHT_DYNSYM:
     SymbolData = toDRI(EF.begin_dynamic_symbols() + symbolIdx);
     break;
   }
 
   return symbol_iterator(SymbolRef(SymbolData, this));
 }
 
+// ELF relocations can target sections, by targetting a symbol of type
+// STT_SECTION
+template <class ELFT>
+section_iterator
+ELFObjectFile<ELFT>::getRelocationSection(DataRefImpl Rel) const {
+  symbol_iterator Sym = getRelocationSymbol(Rel);
+  if (Sym == symbol_end())
+    return section_end();
+  const Elf_Sym *ESym = getSymbol(Sym->getRawDataRefImpl());
+  if (ESym->getType() != ELF::STT_SECTION)
+    return section_end();
+  return getSymbolSection(ESym);
+}
+
 template <class ELFT>
 std::error_code
 ELFObjectFile<ELFT>::getRelocationAddress(DataRefImpl Rel,
                                           uint64_t &Result) const {
   uint64_t ROffset = getROffset(Rel);
   const Elf_Ehdr *Header = EF.getHeader();
 
   if (Header->e_type == ELF::ET_REL) {

llvm/trunk/include/llvm/Object/MachO.h

public:
   relocation_iterator section_rel_end(DataRefImpl Sec) const override;
 
   void moveRelocationNext(DataRefImpl &Rel) const override;
   std::error_code getRelocationAddress(DataRefImpl Rel,
                                        uint64_t &Res) const override;
   std::error_code getRelocationOffset(DataRefImpl Rel,
                                       uint64_t &Res) const override;
   symbol_iterator getRelocationSymbol(DataRefImpl Rel) const override;
+  section_iterator getRelocationSection(DataRefImpl Rel) const override;
   std::error_code getRelocationType(DataRefImpl Rel,
                                     uint64_t &Res) const override;
   std::error_code
   getRelocationTypeName(DataRefImpl Rel,
                         SmallVectorImpl<char> &Result) const override;
   std::error_code
   getRelocationValueString(DataRefImpl Rel,
                            SmallVectorImpl<char> &Result) const override;
public:
   uint32_t getScatteredRelocationValue(
                                     const MachO::any_relocation_info &RE) const;
   uint32_t getScatteredRelocationType(
                                     const MachO::any_relocation_info &RE) const;
   unsigned getAnyRelocationAddress(const MachO::any_relocation_info &RE) const;
   unsigned getAnyRelocationPCRel(const MachO::any_relocation_info &RE) const;
   unsigned getAnyRelocationLength(const MachO::any_relocation_info &RE) const;
   unsigned getAnyRelocationType(const MachO::any_relocation_info &RE) const;
-  SectionRef getRelocationSection(const MachO::any_relocation_info &RE) const;
+  SectionRef getAnyRelocationSection(const MachO::any_relocation_info &RE) const;
 
   // Walk load commands.
   LoadCommandInfo getFirstLoadCommandInfo() const;
   LoadCommandInfo getNextLoadCommandInfo(const LoadCommandInfo &L) const;
 
   // MachO specific structures.
   MachO::section getSection(DataRefImpl DRI) const;
   MachO::section_64 getSection64(DataRefImpl DRI) const;

llvm/trunk/include/llvm/Object/ObjectFile.h

 namespace object {
 
 class ObjectFile;
 class COFFObjectFile;
 class MachOObjectFile;
 
 class SymbolRef;
 class symbol_iterator;
+class SectionRef;
+typedef content_iterator<SectionRef> section_iterator;
 
 /// RelocationRef - This is a value type class that represents a single
 /// relocation in the list of relocations in the object file.
 class RelocationRef {
   DataRefImpl RelocationPimpl;
   const ObjectFile *OwningObject;
 
 public:
   RelocationRef() : OwningObject(nullptr) { }
 
   RelocationRef(DataRefImpl RelocationP, const ObjectFile *Owner);
 
   bool operator==(const RelocationRef &Other) const;
 
   void moveNext();
 
   std::error_code getAddress(uint64_t &Result) const;
   std::error_code getOffset(uint64_t &Result) const;
   symbol_iterator getSymbol() const;
+  section_iterator getSection() const;
   std::error_code getType(uint64_t &Result) const;
 
   /// @brief Indicates whether this relocation should hidden when listing
   /// relocations, usually because it is the trailing part of a multipart
   /// relocation that will be printed as part of the leading relocation.
   std::error_code getHidden(bool &Result) const;
 
   /// @brief Get a string that represents the type of this relocation.
public:
 
   DataRefImpl getRawDataRefImpl() const;
   const ObjectFile *getObjectFile() const;
 };
 typedef content_iterator<RelocationRef> relocation_iterator;
 
 /// SectionRef - This is a value type class that represents a single section in
 /// the list of sections in the object file.
-class SectionRef;
-typedef content_iterator<SectionRef> section_iterator;
 class SectionRef {
   friend class SymbolRef;
   DataRefImpl SectionPimpl;
   const ObjectFile *OwningObject;
 
 public:
   SectionRef() : OwningObject(nullptr) { }
 
protected:
   // Same as above for RelocationRef.
   friend class RelocationRef;
   virtual void moveRelocationNext(DataRefImpl &Rel) const = 0;
   virtual std::error_code getRelocationAddress(DataRefImpl Rel,
                                                uint64_t &Res) const = 0;
   virtual std::error_code getRelocationOffset(DataRefImpl Rel,
                                               uint64_t &Res) const = 0;
   virtual symbol_iterator getRelocationSymbol(DataRefImpl Rel) const = 0;
+  virtual section_iterator getRelocationSection(DataRefImpl Rel) const = 0;
   virtual std::error_code getRelocationType(DataRefImpl Rel,
                                             uint64_t &Res) const = 0;
   virtual std::error_code
   getRelocationTypeName(DataRefImpl Rel,
                         SmallVectorImpl<char> &Result) const = 0;
   virtual std::error_code
   getRelocationValueString(DataRefImpl Rel,
                            SmallVectorImpl<char> &Result) const = 0;

 inline std::error_code RelocationRef::getOffset(uint64_t &Result) const {
   return OwningObject->getRelocationOffset(RelocationPimpl, Result);
 }
 
 inline symbol_iterator RelocationRef::getSymbol() const {
   return OwningObject->getRelocationSymbol(RelocationPimpl);
 }
 
+inline section_iterator RelocationRef::getSection() const {
+  return OwningObject->getRelocationSection(RelocationPimpl);
+}
+
 inline std::error_code RelocationRef::getType(uint64_t &Result) const {
   return OwningObject->getRelocationType(RelocationPimpl, Result);
 }
 
 inline std::error_code
 RelocationRef::getTypeName(SmallVectorImpl<char> &Result) const {
   return OwningObject->getRelocationTypeName(RelocationPimpl, Result);
 }

llvm/trunk/lib/DebugInfo/DWARF/DWARFContext.cpp

static bool consumeCompressedDebugSectionHeader(StringRef &data,
   uint32_t Offset = 0;
   OriginalSize = extractor.getU64(&Offset);
   if (Offset == 0)
     return false;
   data = data.substr(Offset);
   return true;
 }
 
-DWARFContextInMemory::DWARFContextInMemory(const object::ObjectFile &Obj)
+DWARFContextInMemory::DWARFContextInMemory(const object::ObjectFile &Obj,
+    const LoadedObjectInfo *L)
     : IsLittleEndian(Obj.isLittleEndian()),
       AddressSize(Obj.getBytesInAddress()) {
   for (const SectionRef &Section : Obj.sections()) {
     StringRef name;
     Section.getName(name);
     // Skip BSS and Virtual sections, they aren't interesting.
     bool IsBSS = Section.isBSS();
     if (IsBSS)
       continue;
     bool IsVirtual = Section.isVirtual();
     if (IsVirtual)
       continue;
     StringRef data;
+
+    // Try to obtain an already relocated version of this section.
+    // Else use the unrelocated section from the object file. We'll have to
+    // apply relocations ourselves later.
+    if (!L || !L->getLoadedSectionContents(name,data))
       Section.getContents(data);
 
     name = name.substr(name.find_first_not_of("._")); // Skip . and _ prefixes.
 
     // Check if debug info section is compressed with zlib.
     if (name.startswith("zdebug_")) {
       uint64_t OriginalSize;
       if (!zlib::isAvailable() ||
           !consumeCompressedDebugSectionHeader(data, OriginalSize))
if (SectionData) {
       TypesDWOSections[Section].Data = data;
     }
 
     section_iterator RelocatedSection = Section.getRelocatedSection();
     if (RelocatedSection == Obj.section_end())
       continue;
 
     StringRef RelSecName;
+    StringRef RelSecData;
     RelocatedSection->getName(RelSecName);
+
+    // If the section we're relocating was relocated already by the JIT,
+    // then we used the relocated version above, so we do not need to process
+    // relocations for it now.
+    if (L && L->getLoadedSectionContents(RelSecName,RelSecData))
+      continue;
+
     RelSecName = RelSecName.substr(
         RelSecName.find_first_not_of("._")); // Skip . and _ prefixes.
 
     // TODO: Add support for relocations in other sections as needed.
     // Record relocations for the debug_info and debug_line sections.
     RelocAddrMap *Map = StringSwitch<RelocAddrMap*>(RelSecName)
         .Case("debug_info", &InfoSection.Relocs)
         .Case("debug_loc", &LocSection.Relocs)
for (const SectionRef &Section : Obj.sections()) {
     if (Section.relocation_begin() != Section.relocation_end()) {
       uint64_t SectionSize = RelocatedSection->getSize();
       for (const RelocationRef &Reloc : Section.relocations()) {
         uint64_t Address;
         Reloc.getOffset(Address);
         uint64_t Type;
         Reloc.getType(Type);
         uint64_t SymAddr = 0;
+        uint64_t SectionLoadAddress = 0;
         object::symbol_iterator Sym = Reloc.getSymbol();
-        if (Sym != Obj.symbol_end())
+        object::section_iterator RSec = Reloc.getSection();
+
+        // First calculate the address of the symbol or section as it appears
+        // in the objct file
+        if (Sym != Obj.symbol_end()) {
           Sym->getAddress(SymAddr);
+          // Also remember what section this symbol is in for later
+          Sym->getSection(RSec);
+        } else if (RSec != Obj.section_end())
+          SymAddr = RSec->getAddress();
+
+        // If we are given load addresses for the sections, we need to adjust:
+        // SymAddr = (Address of Symbol Or Section in File) -
+        //           (Address of Section in File) +
+        //           (Load Address of Section)
+        if (L != nullptr && RSec != Obj.section_end()) {
+          // RSec is now either the section being targetted or the section
+          // containing the symbol being targetted. In either case,
+          // we need to perform the same computation.
+          StringRef SecName;
+          RSec->getName(SecName);
+          SectionLoadAddress = L->getSectionLoadAddress(SecName);
+          if (SectionLoadAddress != 0)
+            SymAddr += SectionLoadAddress - RSec->getAddress();
+        }
 
         object::RelocVisitor V(Obj);
         object::RelocToApply R(V.visit(Type, Reloc, SymAddr));
         if (V.error()) {
           SmallString<32> Name;
           std::error_code ec(Reloc.getTypeName(Name));
           if (ec) {
             errs() << "Aaaaaa! Nameless relocation! Aaaaaa!\n";

llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp

public:
   LoadedCOFFObjectInfo(RuntimeDyldImpl &RTDyld, unsigned BeginIdx,
                        unsigned EndIdx)
       : RuntimeDyld::LoadedObjectInfo(RTDyld, BeginIdx, EndIdx) {}
 
   OwningBinary<ObjectFile>
   getObjectForDebug(const ObjectFile &Obj) const override {
     return OwningBinary<ObjectFile>();
   }
+
+  RuntimeDyld::LoadedObjectInfo *clone() const { return new LoadedCOFFObjectInfo(*this); }
 };
 }
 
 namespace llvm {
 
 std::unique_ptr<RuntimeDyldCOFF>
 llvm::RuntimeDyldCOFF::create(Triple::ArchType Arch,
                               RuntimeDyld::MemoryManager &MemMgr,

llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp

 class LoadedELFObjectInfo : public RuntimeDyld::LoadedObjectInfo {
 public:
   LoadedELFObjectInfo(RuntimeDyldImpl &RTDyld, unsigned BeginIdx,
                       unsigned EndIdx)
     : RuntimeDyld::LoadedObjectInfo(RTDyld, BeginIdx, EndIdx) {}
 
   OwningBinary<ObjectFile>
   getObjectForDebug(const ObjectFile &Obj) const override;
+
+  RuntimeDyld::LoadedObjectInfo *clone() const { return new LoadedELFObjectInfo(*this); }
 };
 
 template <typename ELFT>
 std::unique_ptr<DyldELFObject<ELFT>>
 createRTDyldELFObject(MemoryBufferRef Buffer,
                       const LoadedELFObjectInfo &L,
                       std::error_code &ec) {
   typedef typename ELFFile<ELFT>::Elf_Shdr Elf_Shdr;

llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h

protected:
   RuntimeDyldCheckerImpl *Checker;
 
   // A list of all sections emitted by the dynamic linker.  These sections are
   // referenced in the code by means of their index in this list - SectionID.
   typedef SmallVector<SectionEntry, 64> SectionList;
   SectionList Sections;
 
   typedef unsigned SID; // Type for SectionIDs
-#define RTDYLD_INVALID_SECTION_ID ((SID)(-1))
+#define RTDYLD_INVALID_SECTION_ID ((RuntimeDyldImpl::SID)(-1))
 
   // Keep a map of sections from object file to the SectionID which
   // references it.
   typedef std::map<SectionRef, unsigned> ObjSectionToIDMap;
 
   // A global symbol table for symbols from all loaded modules.
   RTDyldSymbolTable GlobalSymbolTable;
 

llvm/trunk/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp

public:
   LoadedMachOObjectInfo(RuntimeDyldImpl &RTDyld, unsigned BeginIdx,
                         unsigned EndIdx)
     : RuntimeDyld::LoadedObjectInfo(RTDyld, BeginIdx, EndIdx) {}
 
   OwningBinary<ObjectFile>
   getObjectForDebug(const ObjectFile &Obj) const override {
     return OwningBinary<ObjectFile>();
   }
+
+  RuntimeDyld::LoadedObjectInfo *clone() const { return new LoadedMachOObjectInfo(*this); }
 };
 
 }
 
 namespace llvm {
 
 int64_t RuntimeDyldMachO::memcpyAddend(const RelocationEntry &RE) const {
   unsigned NumBytes = 1 << RE.Size;
if (SI != GlobalSymbolTable.end()) {
       const auto &SymInfo = SI->second;
       Value.SectionID = SymInfo.getSectionID();
       Value.Offset = SymInfo.getOffset() + RE.Addend;
     } else {
       Value.SymbolName = TargetName.data();
       Value.Offset = RE.Addend;
     }
   } else {
-    SectionRef Sec = Obj.getRelocationSection(RelInfo);
+    SectionRef Sec = Obj.getAnyRelocationSection(RelInfo);
     bool IsCode = Sec.isText();
     Value.SectionID = findOrEmitSection(Obj, Sec, IsCode, ObjSectionToID);
     uint64_t Addr = Sec.getAddress();
     Value.Offset = RE.Addend - Addr;
   }
 
   return Value;
 }

llvm/trunk/lib/Object/COFFObjectFile.cpp

if (SymbolTable16)
     Ref.p = reinterpret_cast<uintptr_t>(SymbolTable16 + R->SymbolTableIndex);
   else if (SymbolTable32)
     Ref.p = reinterpret_cast<uintptr_t>(SymbolTable32 + R->SymbolTableIndex);
   else
     llvm_unreachable("no symbol table pointer!");
   return symbol_iterator(SymbolRef(Ref, this));
 }
 
+section_iterator COFFObjectFile::getRelocationSection(DataRefImpl Rel) const {
+  symbol_iterator Sym = getRelocationSymbol(Rel);
+  if (Sym == symbol_end())
+    return section_end();
+  COFFSymbolRef Symb = getCOFFSymbol(*Sym);
+  if (!Symb.isSection())
+    return section_end();
+  section_iterator Res(section_end());
+  if (getSymbolSection(Sym->getRawDataRefImpl(),Res))
+    return section_end();
+  return Res;
+}
+
 std::error_code COFFObjectFile::getRelocationType(DataRefImpl Rel,
                                                   uint64_t &Res) const {
   const coff_relocation* R = toRel(Rel);
   Res = R->Type;
   return object_error::success;
 }
 
 const coff_section *

llvm/trunk/lib/Object/MachOObjectFile.cpp

unsigned SymbolTableEntrySize = is64Bit() ?
     sizeof(MachO::nlist_64) :
     sizeof(MachO::nlist);
   uint64_t Offset = S.symoff + SymbolIdx * SymbolTableEntrySize;
   DataRefImpl Sym;
   Sym.p = reinterpret_cast<uintptr_t>(getPtr(this, Offset));
   return symbol_iterator(SymbolRef(Sym, this));
 }
 
+section_iterator
+MachOObjectFile::getRelocationSection(DataRefImpl Rel) const {
+  return section_iterator(getAnyRelocationSection(getRelocation(Rel)));
+}
+
 std::error_code MachOObjectFile::getRelocationType(DataRefImpl Rel,
                                                    uint64_t &Res) const {
   MachO::any_relocation_info RE = getRelocation(Rel);
   Res = getAnyRelocationType(RE);
   return object_error::success;
 }
 
 std::error_code

 MachOObjectFile::getAnyRelocationType(
                                    const MachO::any_relocation_info &RE) const {
   if (isRelocationScattered(RE))
     return getScatteredRelocationType(RE);
   return getPlainRelocationType(this, RE);
 }
 
 SectionRef
-MachOObjectFile::getRelocationSection(
+MachOObjectFile::getAnyRelocationSection(
                                    const MachO::any_relocation_info &RE) const {
   if (isRelocationScattered(RE) || getPlainRelocationExternal(RE))
     return *section_end();
   unsigned SecNum = getPlainRelocationSymbolNum(RE) - 1;
   DataRefImpl DRI;
   DRI.d.a = SecNum;
   return SectionRef(DRI, this);
 }

llvm/trunk/test/DebugInfo/debuglineinfo.test

 RUN: llvm-rtdyld -printline %p/Inputs/test-inline.o \
 RUN:   | FileCheck %s -check-prefix TEST_INLINE
+RUN: llvm-rtdyld -printdebugline %p/Inputs/test-inline.o \
+RUN:   | FileCheck %s -check-prefix TEST_INLINE
 RUN: llvm-rtdyld -printline %p/Inputs/test-parameters.o \
 RUN:   | FileCheck %s -check-prefix TEST_PARAMETERS
+RUN: llvm-rtdyld -printdebugline %p/Inputs/test-parameters.o \
+RUN:   | FileCheck %s -check-prefix TEST_PARAMETERS
 
 ; This test verifies that relocations are correctly applied to the
 ; .debug_line section and exercises DIContext::getLineInfoForAddressRange().
 ; If relocations are not applied the first two functions will be reported as
 ; both starting at address zero in the; line number table.
 TEST_INLINE:      Function: _Z15test_parametersPfPA2_dR11char_structPPitm, Size = 170
 TEST_INLINE-NEXT: Line info @ 0: test-inline.cpp, line:33
 TEST_INLINE-NEXT: Line info @ 35: test-inline.cpp, line:34

llvm/trunk/tools/llvm-objdump/MachODump.cpp

static void findUnwindRelocNameAddend(const MachOObjectFile *Obj,
                                       StringRef &Name, uint64_t &Addend) {
   if (Reloc.getSymbol() != Obj->symbol_end()) {
     Reloc.getSymbol()->getName(Name);
     Addend = Addr;
     return;
   }
 
   auto RE = Obj->getRelocation(Reloc.getRawDataRefImpl());
-  SectionRef RelocSection = Obj->getRelocationSection(RE);
+  SectionRef RelocSection = Obj->getAnyRelocationSection(RE);
 
   uint64_t SectionAddr = RelocSection.getAddress();
 
   auto Sym = Symbols.upper_bound(Addr);
   if (Sym == Symbols.begin()) {
     // The first symbol in the object is after this reference, the best we can
     // do is section-relative notation.
     RelocSection.getName(Name);

llvm/trunk/tools/llvm-rtdyld/llvm-rtdyld.cpp

 
 static cl::list<std::string>
 InputFileList(cl::Positional, cl::ZeroOrMore,
               cl::desc("<input file>"));
 
 enum ActionType {
   AC_Execute,
   AC_PrintLineInfo,
+  AC_PrintDebugLineInfo,
   AC_Verify
 };
 
 static cl::opt<ActionType>
 Action(cl::desc("Action to perform:"),
        cl::init(AC_Execute),
        cl::values(clEnumValN(AC_Execute, "execute",
                              "Load, link, and execute the inputs."),
                   clEnumValN(AC_PrintLineInfo, "printline",
                              "Load, link, and print line information for each function."),
+                  clEnumValN(AC_PrintDebugLineInfo, "printdebugline",
+                             "Load, link, and print line information for each function using the debug object"),
                   clEnumValN(AC_Verify, "verify",
                              "Load, link and verify the resulting memory image."),
                   clEnumValEnd));
 
 static cl::opt<std::string>
 EntryPoint("entry",
            cl::desc("Function to call as entry point."),
            cl::init("_main"));
if (sys::fs::is_regular_file(Dylib)) {
                      << ErrMsg << "\n";
     } else
       llvm::errs() << "Dylib not found: '" << Dylib << "'.\n";
   }
 }
 
 /* *** */
 
-static int printLineInfoForInput() {
+static int printLineInfoForInput(bool LoadObjects, bool UseDebugObj) {
+  assert(LoadObjects || !UseDebugObj);
+
   // Load any dylibs requested on the command line.
   loadDylibs();
 
   // If we don't have any input files, read from stdin.
   if (!InputFileList.size())
     InputFileList.push_back("-");
   for(unsigned i = 0, e = InputFileList.size(); i != e; ++i) {
     // Instantiate a dynamic linker.
for(unsigned i = 0, e = InputFileList.size(); i != e; ++i) {
     ErrorOr<std::unique_ptr<ObjectFile>> MaybeObj(
       ObjectFile::createObjectFile((*InputBuffer)->getMemBufferRef()));
 
     if (std::error_code EC = MaybeObj.getError())
       return Error("unable to create object file: '" + EC.message() + "'");
 
     ObjectFile &Obj = **MaybeObj;
 
+    OwningBinary<ObjectFile> DebugObj;
+    std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo = nullptr;
+    ObjectFile *SymbolObj = &Obj;
+    if (LoadObjects) {
       // Load the object file
-    std::unique_ptr<RuntimeDyld::LoadedObjectInfo> LoadedObjInfo =
+      LoadedObjInfo =
         Dyld.loadObject(Obj);
 
       if (Dyld.hasError())
         return Error(Dyld.getErrorString());
 
       // Resolve all the relocations we can.
       Dyld.resolveRelocations();
 
-    OwningBinary<ObjectFile> DebugObj = LoadedObjInfo->getObjectForDebug(Obj);
+      if (UseDebugObj) {
+        DebugObj = LoadedObjInfo->getObjectForDebug(Obj);
+        SymbolObj = DebugObj.getBinary();
+      }
+    }
 
     std::unique_ptr<DIContext> Context(
-      new DWARFContextInMemory(*DebugObj.getBinary()));
+      new DWARFContextInMemory(*SymbolObj,LoadedObjInfo.get()));
 
     // Use symbol info to iterate functions in the object.
-    for (object::symbol_iterator I = DebugObj.getBinary()->symbol_begin(),
-                                 E = DebugObj.getBinary()->symbol_end();
+    for (object::symbol_iterator I = SymbolObj->symbol_begin(),
+                                 E = SymbolObj->symbol_end();
          I != E; ++I) {
       object::SymbolRef::Type SymType;
       if (I->getType(SymType)) continue;
       if (SymType == object::SymbolRef::ST_Function) {
         StringRef  Name;
         uint64_t   Addr;
         uint64_t   Size;
         if (I->getName(Name)) continue;
         if (I->getAddress(Addr)) continue;
         if (I->getSize(Size)) continue;
 
-        outs() << "Function: " << Name << ", Size = " << Size << "\n";
+        // If we're not using the debug object, compute the address of the
+        // symbol in memory (rather than that in the unrelocated object file)
+        // and use that to query the DWARFContext.
+        if (!UseDebugObj && LoadObjects) {
+          object::section_iterator Sec(SymbolObj->section_end());
+          I->getSection(Sec);
+          StringRef SecName;
+          Sec->getName(SecName);
+          uint64_t SectionLoadAddress =
+            LoadedObjInfo->getSectionLoadAddress(SecName);
+          if (SectionLoadAddress != 0)
+            Addr += SectionLoadAddress - Sec->getAddress();
+        }
+
+        outs() << "Function: " << Name << ", Size = " << Size << ", Addr = " << Addr << "\n";
 
         DILineInfoTable Lines = Context->getLineInfoForAddressRange(Addr, Size);
         DILineInfoTable::iterator  Begin = Lines.begin();
         DILineInfoTable::iterator  End = Lines.end();
         for (DILineInfoTable::iterator It = Begin; It != End; ++It) {
           outs() << "  Line info @ " << It->first - Addr << ": "
                  << It->second.FileName << ", line:" << It->second.Line << "\n";
         }
int main(int argc, char **argv) {
   llvm::InitializeAllTargetMCs();
   llvm::InitializeAllDisassemblers();
 
   cl::ParseCommandLineOptions(argc, argv, "llvm MC-JIT tool\n");
 
   switch (Action) {
   case AC_Execute:
     return executeInput();
+  case AC_PrintDebugLineInfo:
+    return printLineInfoForInput(true,true);
   case AC_PrintLineInfo:
-    return printLineInfoForInput();
+    return printLineInfoForInput(true,false);
   case AC_Verify:
     return linkAndVerify();
   }
 }