Set up basic infrastructure for ELF/i386 backend support in JITLink .

[jain98]

Summary

Changes in this CR enable jit-linking a minimal ELF 32-bit object file. No relocations are supported yet.

[jain98]

Runs clang-format

[lhames]

Thanks very much for working on this Kshitij -- this looks like a great start to the i386 backend!

I have a couple of minor comments -- could you address them and update the review? Once that's done I think that this is ready to land. :)

llvm/lib/ExecutionEngine/JITLink/i386.cpp

2–10

Comments should be updated to i386.

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

4

Tests should be -noexec (except in special circumstances): We want them to be able to run on architectures other than the target.

I would simplify to:

# RUN: llvm-mc -triple=i386-unknown-linux-gnu -position-independent -filetype=obj -o %t.o %s
# RUN: llvm-jitlink -noexec %t.o

[sunho]

Thanks for the patch! It's really exciting to see i386 backend coming up.

llvm/include/llvm/ExecutionEngine/JITLink/i386.h

27

The enum name here is better to be non platform-specific like in x86_64.h and aarch64.h. The edges are going to be reused across other platforms too. (e.g. COFF/i386)

llvm/lib/ExecutionEngine/JITLink/ELF_i386.cpp

50

If you were not aware of it, there is isInt<> function in support library.

[jain98]

llvm/include/llvm/ExecutionEngine/JITLink/i386.h

27

Makes sense. Changing it to NONE.

llvm/lib/ExecutionEngine/JITLink/ELF_i386.cpp

50

This is actually a remnant from the starter code that I used from the introductory aarch64 jitlink commit.

Removing the function entirely.

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

4

Does this also mean that the contents of this directory's lit.local.cfg should be changed? Currently this is what's in there -

if not 'i386' in config.root.targets:
  config.unsupported = True

I may be wrong, but I think it's saying run this test only if the current host architecture is i386?

4

Additionally, what the reasoning behind making these tests --noexec - it looks like most JitLink tests are? I'm wondering what makes the code in these tests non-runnable if we're able to load it into memory? I can see how the generated code for one architecture might be straight up rubbish for some other architecture, and therefore it does not make sense to actually try and execute the generated code. But then why go to the point where we're loading things into memory even? Why not stop after generating the LinkGraph?

[jain98]

Addressed comments on revision 2

[tschuett]

llvm/include/llvm/ExecutionEngine/JITLink/i386.h

20

The codebase switched to C++17. You are allowed to use nested namespaces.

[lhames]

@tschuett's suggestion is a good one.

Otherwise LGTM. Just let me know what name and email you would like me to use for attribution and I can land this tomorrow. :)

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

4

I may be wrong, but I think it's saying run this test only if the current host architecture is i386?

That check is saying "if this LLVM build isn't configured with i386 backend support". You want to keep it -- it tells us whether or not the tools can handle i386 assembly. :)

4

I can see how the generated code for one architecture might be straight up rubbish for some other architecture, and therefore it does not make sense to actually try and execute the generated code.

Yep -- we want to run this on all architectures, and that means not executing it when target arch != host arch. We just go one step further and never execute it at all.

But then why go to the point where we're loading things into memory even? Why not stop after generating the LinkGraph?

We want to check that all relocations have been applied correctly. Originally this testing system was designed for RuntimeDyld, which had no exposed data structure that we could evaluate, hence the decision to inspect final linked memory instead. Now that we have JITLink we could inspect the graph, but the advantage of checking the final linked memory is that it verifies that linking worked end-to-end, including layout in target memory.

It wouldn't hurt to rethink all this test infrastructure. It was written in a hurry back in 2015 and barely touched since. That said it has been remarkably effective -- a cleanup and new functionality would be welcome (LinkGraph inspection? Maybe a more user-friendly expression language and robust parser), but we'd definitely want to keep some variation of the end-to-end scheme that we have.

[jain98]

In D131347#3721558, @lhames wrote:

@tschuett's suggestion is a good one.

I think so too. However, I was thinking maybe we should make that change separately and perhaps group it with similar namespace changes across other JitLink files? I'm happy to send out a code review for the same.

Otherwise LGTM. Just let me know what name and email you would like me to use for attribution and I can land this tomorrow. :)

Name: Kshitij Jain
Email: [e-mail]

Thanks!

[jain98]

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

4

But then why go to the point where we're loading things into memory even? Why not stop after generating the LinkGraph?

We want to check that all relocations have been applied correctly. Originally this testing system was designed for RuntimeDyld, which had no exposed data structure that we could evaluate, hence the decision to inspect final linked memory instead. Now that we have JITLink we could inspect the graph, but the advantage of checking the final linked memory is that it verifies that linking worked end-to-end, including layout in target memory.

Perhaps this is my lack of understanding/knowledge, but are we truly checking whether the relocations have been applied correctly? I'm thinking of a very simple Pointer32 type relocation for a non-static global var. Is it possible that we calculate the address of the variable to be relocated incorrectly and since we never run the program we never run into any issues?

I'm guessing perhaps the more concrete issues, such as, invalid memory access are checked for, but can the program be linked and still be logically incorrect?

[lhames]

I ran out of time to test this today, but should be able to commit early tomorrow.

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

4

Is it possible that we calculate the address of the variable to be relocated incorrectly and since we never run the program we never run into any issues?

It is, but the address plumbing is shared infrastructure used by all backends and implicitly tested by the execution tests elsewhere in llvm/test/ExecutionEngine, so we don't need to re-test it in the JITLink backend tests.

[chapuni]

llvm/lib/ExecutionEngine/JITLink/ELF_i386.cpp

67

Unused?

[jain98]

llvm/lib/ExecutionEngine/JITLink/ELF_i386.cpp

67

Yes. It looks like it's been fixed

67

Yes looks like it's been fixed here - https://github.com/llvm/llvm-project/commit/eca990702d5676151419655098afb57b22cab600

llvm/include/llvm/ExecutionEngine/JITLink/ELF_i386.h

+//===--- ELF_i386.h - JIT link functions for ELF/i386 --*- C++ -*----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+//===----------------------------------------------------------------------===//
+//
+// jit-link functions for ELF/i386.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_JITLINK_ELF_I386_H
+#define LLVM_EXECUTIONENGINE_JITLINK_ELF_I386_H
+
+#include "llvm/ExecutionEngine/JITLink/JITLink.h"
+
+namespace llvm {
+namespace jitlink {
+
+/// Create a LinkGraph from an ELF/i386 relocatable object
+///
+/// Note: The graph does not take ownership of the underlying buffer, nor copy
+/// its contents. The caller is responsible for ensuring that the object buffer
+/// outlives the graph.
+Expected<std::unique_ptr<LinkGraph>>
+createLinkGraphFromELFObject_i386(MemoryBufferRef ObjectBuffer);
+
+/// jit-link the given object buffer, which must be a ELF i386 relocatable
+/// object file.
+void link_ELF_i386(std::unique_ptr<LinkGraph> G,
+                   std::unique_ptr<JITLinkContext> Ctx);
+
+} // end namespace jitlink
+} // end namespace llvm
+
+#endif // LLVM_EXECUTIONENGINE_JITLINK_ELF_I386_H

llvm/include/llvm/ExecutionEngine/JITLink/i386.h

+//=== i386.h - Generic JITLink i386 edge kinds, utilities -*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Generic utilities for graphs representing i386 objects.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_JITLINK_I386_H
+#define LLVM_EXECUTIONENGINE_JITLINK_I386_H
+
+#include "llvm/ExecutionEngine/JITLink/JITLink.h"
+
+namespace llvm {
+namespace jitlink {
+namespace i386 {
+
+/// Represets i386 fixups
+enum EdgeKind_i386 : Edge::Kind {
+
+  /// None
+  None = Edge::FirstRelocation,
+
+};
+
+/// Returns a string name for the given i386 edge. For debugging purposes
+/// only
+const char *getEdgeKindName(Edge::Kind K);
+
+} // namespace i386
+} // namespace jitlink
+} // namespace llvm
+
+#endif // LLVM_EXECUTIONENGINE_JITLINK_I386_H
 No newline at end of file

llvm/lib/ExecutionEngine/JITLink/CMakeLists.txt

add_llvm_component_library(LLVMJITLink
   MachOLinkGraphBuilder.cpp
 
   # ELF
   ELF.cpp
   ELFLinkGraphBuilder.cpp
   ELF_aarch64.cpp
   ELF_riscv.cpp
   ELF_x86_64.cpp
+  ELF_i386.cpp
 
   # Architectures:
   aarch64.cpp
   riscv.cpp
   x86_64.cpp
+  i386.cpp
   
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/JITLink
 
   DEPENDS
   intrinsics_gen
 
   LINK_COMPONENTS
   BinaryFormat

llvm/lib/ExecutionEngine/JITLink/ELF.cpp

 //===-------------- ELF.cpp - JIT linker function for ELF -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // ELF jit-link function.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/JITLink/ELF.h"
 
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/ExecutionEngine/JITLink/ELF_aarch64.h"
+#include "llvm/ExecutionEngine/JITLink/ELF_i386.h"
 #include "llvm/ExecutionEngine/JITLink/ELF_riscv.h"
 #include "llvm/ExecutionEngine/JITLink/ELF_x86_64.h"
 #include "llvm/Object/ELF.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include <cstring>
 
createLinkGraphFromELFObject(MemoryBufferRef ObjectBuffer) {
 
   switch (*TargetMachineArch) {
   case ELF::EM_AARCH64:
     return createLinkGraphFromELFObject_aarch64(ObjectBuffer);
   case ELF::EM_RISCV:
     return createLinkGraphFromELFObject_riscv(ObjectBuffer);
   case ELF::EM_X86_64:
     return createLinkGraphFromELFObject_x86_64(ObjectBuffer);
+  case ELF::EM_386:
+    return createLinkGraphFromELFObject_i386(ObjectBuffer);
   default:
     return make_error<JITLinkError>(
         "Unsupported target machine architecture in ELF object " +
         ObjectBuffer.getBufferIdentifier());
   }
 }
 
 void link_ELF(std::unique_ptr<LinkGraph> G,
               std::unique_ptr<JITLinkContext> Ctx) {
   switch (G->getTargetTriple().getArch()) {
   case Triple::aarch64:
     link_ELF_aarch64(std::move(G), std::move(Ctx));
     return;
   case Triple::riscv32:
   case Triple::riscv64:
     link_ELF_riscv(std::move(G), std::move(Ctx));
     return;
   case Triple::x86_64:
     link_ELF_x86_64(std::move(G), std::move(Ctx));
     return;
+  case Triple::x86:
+    link_ELF_i386(std::move(G), std::move(Ctx));
+    return;
   default:
     Ctx->notifyFailed(make_error<JITLinkError>(
         "Unsupported target machine architecture in ELF link graph " +
         G->getName()));
     return;
   }
 }
 
 } // end namespace jitlink
 } // end namespace llvm

llvm/lib/ExecutionEngine/JITLink/ELF_i386.cpp

+//===----- ELF_i386.cpp - JIT linker implementation for ELF/i386 ----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// ELF/i386 jit-link implementation.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ExecutionEngine/JITLink/ELF_i386.h"
+#include "ELFLinkGraphBuilder.h"
+#include "JITLinkGeneric.h"
+#include "llvm/BinaryFormat/ELF.h"
+#include "llvm/ExecutionEngine/JITLink/i386.h"
+#include "llvm/Object/ELFObjectFile.h"
+
+#define DEBUG_TYPE "jitlink"
+
+using namespace llvm;
+using namespace llvm::jitlink;
+
+namespace llvm {
+namespace jitlink {
+
+class ELFJITLinker_i386 : public JITLinker<ELFJITLinker_i386> {
+  friend class JITLinker<ELFJITLinker_i386>;
+
+public:
+  ELFJITLinker_i386(std::unique_ptr<JITLinkContext> Ctx,
+                    std::unique_ptr<LinkGraph> G, PassConfiguration PassConfig)
+      : JITLinker(std::move(Ctx), std::move(G), std::move(PassConfig)) {}
+
+private:
+  Error applyFixup(LinkGraph &G, Block &B, const Edge &E) const {
+    using namespace i386;
+    using namespace llvm::support;
+
+    switch (E.getKind()) {
+    case i386::None: {
+      break;
+    }
+    }
+    return Error::success();
+  }
+};
+
+template <typename ELFT>
+class ELFLinkGraphBuilder_i386 : public ELFLinkGraphBuilder<ELFT> {
+private:
+  static Expected<i386::EdgeKind_i386> getRelocationKind(const uint32_t Type) {
+    using namespace i386;
+    switch (Type) {
+    case ELF::R_386_NONE:
+      return EdgeKind_i386::None;
+    }
+
+    return make_error<JITLinkError>("Unsupported i386 relocation:" +
+                                    formatv("{0:d}", Type));
+  }
+
+  Error addRelocations() override {
+    LLVM_DEBUG(dbgs() << "Adding relocations\n");
+    using Base = ELFLinkGraphBuilder<ELFT>;
+
+    return Error::success();
+  }
+
+public:
+  ELFLinkGraphBuilder_i386(StringRef FileName, const object::ELFFile<ELFT> &Obj,
+                           const Triple T)
+      : ELFLinkGraphBuilder<ELFT>(Obj, std::move(T), FileName,
+                                  i386::getEdgeKindName) {}
+};
+
+Expected<std::unique_ptr<LinkGraph>>
+createLinkGraphFromELFObject_i386(MemoryBufferRef ObjectBuffer) {
+  LLVM_DEBUG({
+    dbgs() << "Building jitlink graph for new input "
+           << ObjectBuffer.getBufferIdentifier() << "...\n";
+  });
+
+  auto ELFObj = object::ObjectFile::createELFObjectFile(ObjectBuffer);
+  if (!ELFObj)
+    return ELFObj.takeError();
+
+  assert((*ELFObj)->getArch() == Triple::x86 &&
+         "Only i386 (little endian) is supported for now");
+
+  auto &ELFObjFile = cast<object::ELFObjectFile<object::ELF32LE>>(**ELFObj);
+  return ELFLinkGraphBuilder_i386<object::ELF32LE>((*ELFObj)->getFileName(),
+                                                   ELFObjFile.getELFFile(),
+                                                   (*ELFObj)->makeTriple())
+      .buildGraph();
+}
+
+void link_ELF_i386(std::unique_ptr<LinkGraph> G,
+                   std::unique_ptr<JITLinkContext> Ctx) {
+  PassConfiguration Config;
+  const Triple &TT = G->getTargetTriple();
+  if (Ctx->shouldAddDefaultTargetPasses(TT)) {
+    if (auto MarkLive = Ctx->getMarkLivePass(TT))
+      Config.PrePrunePasses.push_back(std::move(MarkLive));
+    else
+      Config.PrePrunePasses.push_back(markAllSymbolsLive);
+  }
+  if (auto Err = Ctx->modifyPassConfig(*G, Config))
+    return Ctx->notifyFailed(std::move(Err));
+
+  ELFJITLinker_i386::link(std::move(Ctx), std::move(G), std::move(Config));
+}
+
+} // namespace jitlink
+} // namespace llvm
 No newline at end of file

llvm/lib/ExecutionEngine/JITLink/i386.cpp

+//===---- i386.cpp - Generic JITLink i386 edge kinds, utilities -----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Generic utilities for graphs representing i386 objects.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ExecutionEngine/JITLink/i386.h"
+
+#define DEBUG_TYPE "jitlink"
+
+namespace llvm {
+namespace jitlink {
+namespace i386 {
+
+const char *getEdgeKindName(Edge::Kind K) {
+  switch (K) {
+  case None:
+    return "None";
+  }
+  return getGenericEdgeKindName(K);
+}
+} // namespace i386
+} // namespace jitlink
+} // namespace llvm
 No newline at end of file

llvm/test/ExecutionEngine/JITLink/i386/ELF_i386_minimal.s

+# RUN: llvm-mc -triple=i386-unknown-linux-gnu -position-independent -filetype=obj -o %t.o %s
+# RUN: llvm-jitlink -noexec %t.o
+
+  .text
+  .globl  main
+  .p2align  4
+  .type main,@function
+main:
+    pushl   %ebp
+    movl    %esp, %ebp
+    pushl   %eax
+    movl    $0, -4(%ebp)
+    movl    $42, %eax
+    addl    $4, %esp
+    popl    %ebp
+    retl
+
+  .size main, .-main
 No newline at end of file

llvm/test/ExecutionEngine/JITLink/i386/lit.local.cfg

+if not 'i386' in config.root.targets:
+  config.unsupported = True
 No newline at end of file