[ELF] - Linkerscript: implemented BYTE/SHORT/LONG/QUAD commands.

[grimar]

Summary

Previously our scripts handles these commands incorrectly. For example:

SECTIONS  {
  .foo : {
 *(.foo.1)
 BYTE(0x11)
...

We accepted the script above treating BYTE as input section description. These commands are used in the wild though, eg:
https://searchcode.com/codesearch/view/42878166/

The BYTE, SHORT, LONG, and QUAD commands store one, two, four, and eight bytes (respectively). After storing the bytes, the location counter is incremented by the number of bytes
stored.

Patch implements them.

[ruiu]

ELF/LinkerScript.cpp

72–1

This

72–1

this needs comments.

73

this and

74–79

I don't think we need these enums. Can't this be

int Size = StringSwitch(Tok)
  .Case("BYTE", 1)
  .Case("SHORT", 2)
  ...

?

77

Don't we have a function to find an output section by name?

80–83

This code seems to write the bytes in the host endianess instead of the target endianness.

ELF/LinkerScript.h

11–1

It's time to add comments for each command.

AssignmentKind, // . = expr or <sym> = expr
OutputSectionKind,
InputSectionKind,
AssertKind, // ASSERT(expr)
BytesDataKind, // BYTE(expr), SHORT(expr), LONG(expr) or QUAD(expr)

ELF/OutputSections.cpp

4

It needs a comment.

// Linker scripts may have BYTE()-family commands with which you
// can write arbitrary bytes to the output. Process them if any.

[grimar]

ELF/LinkerScript.cpp

72–1

Done.

72–1

Done.

73

Done.

74–79

Done.

77

I do not think we have, I created new one, it can be reused in getFiller() in following patch.

80–83

Ah, right. Fixed.

ELF/LinkerScript.h

11–1

Done. (btw comments looks a bit displaced because are clang-formatted)

ELF/OutputSections.cpp

4

Done.

[• rafael]

ELF/LinkerScript.h

16

This could just store a StringRef or ArrayRef, no?

We can compute the correct (target endian) representation upfront and simplify the remaining code.

[grimar]

ELF/LinkerScript.h

16

Not sure about StringRef/ArrayRef, that could be std::vector<uint_8t>, computed in theory, but I think I can't do it.

Problem is that readBytesDataCommand() that reads command know nothing about target endianess because is not ELFT templated.

[grimar]

And I mean it is too early to know it as script can go before any other files, so I need to keep the value anyways, and in that case have a vector or something else in addition does not make much sence ?

[• rafael]

ELF/LinkerScript.cpp

74

You can probably use getSectionIndex.

[grimar]

• Addressed review comments.

ELF/LinkerScript.cpp

713

Done.

[• rafael]

LGTM with nits, but wait so see if Rui is OK with it too.

test/ELF/linkerscript/data-commands.s

2

Add a big endian test.

21

Don't check the address.

[ruiu]

ELF/LinkerScript.cpp

424

Add a blank line before this.

Maybe just

// Handle BYTE(), SHORT(), LONG(), or QUAD().

717

This looks odd. You can return early.

if (I == INT_MAX)
  return;

726

I'd factor this switch out as an independent functions so that we can do something like this here.

writeInt<ELFT>(Buf + DataCmd->Offset, DataCmd->Data, DataCmd->Size);

where writeInt is defined as

writeInt(uint8_t *Buf, uint64_t Data, uint64_t Size);

1519

Use int instead of unsigned so to remove the cast to unsigned below.

[grimar]

• Addressed review comments.

ELF/LinkerScript.cpp

424

Done.

717

Done.

1519

Done.

test/ELF/linkerscript/data-commands.s

2

Done.

21

Done.

[ruiu]

LGTM with a nit.

ELF/LinkerScript.cpp

105–106

This comment doesn't make sense to be written here. Probably I'd just remove.

[grimar]

ELF/LinkerScript.cpp

105–106

Yeah, initially I wrote it because we have comment for filler saying it works with big-endian,
but for this place there is no disagree between ld, gold and spec, so it was probably excessive, I removed it from commit.

ELF/LinkerScript.h

 //===- LinkerScript.h -------------------------------------------*- C++ -*-===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_ELF_LINKER_SCRIPT_H
 #define LLD_ELF_LINKER_SCRIPT_H
 
 #include "Config.h"
 #include "Strings.h"
 #include "Writer.h"
 #include "lld/Core/LLVM.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/MapVector.h"
 #include "llvm/Support/Allocator.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Regex.h"
 #include <functional>
 

 // Config and ScriptConfig.
 void readLinkerScript(MemoryBufferRef MB);
 
 void readVersionScript(MemoryBufferRef MB);
 
 // This enum is used to implement linker script SECTIONS command.
 // https://sourceware.org/binutils/docs/ld/SECTIONS.html#SECTIONS
 enum SectionsCommandKind {
-  AssignmentKind,
+  AssignmentKind, // . = expr or <sym> = expr
   OutputSectionKind,
   InputSectionKind,
-  AssertKind
+  AssertKind,   // ASSERT(expr)
+  BytesDataKind // BYTE(expr), SHORT(expr), LONG(expr) or QUAD(expr)
 };
 
 struct BaseCommand {
   BaseCommand(int K) : Kind(K) {}
   virtual ~BaseCommand() {}
   int Kind;
 };
 

 };
 
 struct AssertCommand : BaseCommand {
   AssertCommand(Expr E) : BaseCommand(AssertKind), Expression(E) {}
   static bool classof(const BaseCommand *C);
   Expr Expression;
 };
 
+struct BytesDataCommand : BaseCommand {
+  BytesDataCommand(uint64_t Data, unsigned Size)
+      : BaseCommand(BytesDataKind), Data(Data), Size(Size) {}
+  static bool classof(const BaseCommand *C);
+  uint64_t Data;
+  unsigned Offset;
+  unsigned Size;
+};
+
 struct PhdrsCommand {
   StringRef Name;
   unsigned Type;
   bool HasFilehdr;
   bool HasPhdrs;
   unsigned Flags;
   Expr LMAExpr;
 };
public:
   void processCommands(OutputSectionFactory<ELFT> &Factory);
   void createSections(OutputSectionFactory<ELFT> &Factory);
   void adjustSectionsBeforeSorting();
 
   std::vector<PhdrEntry<ELFT>> createPhdrs();
   bool ignoreInterpSection();
 
   ArrayRef<uint8_t> getFiller(StringRef Name);
+  void writeDataBytes(StringRef Name, uint8_t *Buf);
   Expr getLma(StringRef Name);
   bool shouldKeep(InputSectionBase<ELFT> *S);
   void assignOffsets(OutputSectionCommand *Cmd);
   void assignAddresses();
   bool hasPhdrsCommands();
   uint64_t getOutputSectionAddress(StringRef Name) override;
   uint64_t getOutputSectionSize(StringRef Name) override;
   uint64_t getOutputSectionAlign(StringRef Name) override;

ELF/LinkerScript.cpp

 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/StringSaver.h"
 
 using namespace llvm;
 using namespace llvm::ELF;
 using namespace llvm::object;
+using namespace llvm::support::endian;
 using namespace lld;
 using namespace lld::elf;
 
 LinkerScriptBase *elf::ScriptBase;
 ScriptConfiguration *elf::ScriptConfig;
 
 template <class ELFT> static void addRegular(SymbolAssignment *Cmd) {
   Symbol *Sym = Symtab<ELFT>::X->addRegular(Cmd->Name, STB_GLOBAL, STV_DEFAULT);

 template <class ELFT> static void addSymbol(SymbolAssignment *Cmd) {
   if (Cmd->IsAbsolute)
     addRegular<ELFT>(Cmd);
   else
     addSynthetic<ELFT>(Cmd);
 }
 // If a symbol was in PROVIDE(), we need to define it only when
 // it is an undefined symbol.
 template <class ELFT> static bool shouldDefine(SymbolAssignment *Cmd) {
   if (Cmd->Name == ".")
     return false;
   if (!Cmd->Provide)
     return true;
   SymbolBody *B = Symtab<ELFT>::X->find(Cmd->Name);
   return B && B->isUndefined();
 }
 
 bool SymbolAssignment::classof(const BaseCommand *C) {
   return C->Kind == AssignmentKind;
 }
 
 bool OutputSectionCommand::classof(const BaseCommand *C) {
   return C->Kind == OutputSectionKind;
 }
 
 bool InputSectionDescription::classof(const BaseCommand *C) {
   return C->Kind == InputSectionKind;
 }
 
 bool AssertCommand::classof(const BaseCommand *C) {
   return C->Kind == AssertKind;
 }
 
+bool BytesDataCommand::classof(const BaseCommand *C) {
+  return C->Kind == BytesDataKind;
+}
+
 template <class ELFT> static bool isDiscarded(InputSectionBase<ELFT> *S) {
   return !S || !S->Live;
 }
 
 template <class ELFT> LinkerScript<ELFT>::LinkerScript() {}
 template <class ELFT> LinkerScript<ELFT>::~LinkerScript() {}
 
 template <class ELFT>
 bool LinkerScript<ELFT>::shouldKeep(InputSectionBase<ELFT> *S) {
   for (Regex *Re : Opt.KeptSections)
     if (Re->match(S->Name))
       return true;
   return false;
void LinkerScript<ELFT>::switchTo(OutputSectionBase<ELFT> *Sec) {
   flush();
   CurOutSec = Sec;
 
   Dot = alignTo(Dot, CurOutSec->getAlignment());
   CurOutSec->setVA(isTbss(CurOutSec) ? Dot + ThreadBssOffset : Dot);
 }
 
 template <class ELFT> void LinkerScript<ELFT>::process(BaseCommand &Base) {
+  // This handles the assignments to symbol or to a location counter (.)
   if (auto *AssignCmd = dyn_cast<SymbolAssignment>(&Base)) {
     if (AssignCmd->Name == ".") {
       // Update to location counter means update to section size.
       Dot = AssignCmd->Expression(Dot);
       CurOutSec->setSize(Dot - CurOutSec->getVA());
       return;
     }
     assignSectionSymbol<ELFT>(AssignCmd, CurOutSec, Dot - CurOutSec->getVA());
     return;
   }
+  // It handles the bytes data command, which places some byte value into output
+  // section and moves the location counter in according to value data size.
+  if (auto *DataCmd = dyn_cast<BytesDataCommand>(&Base)) {
+    DataCmd->Offset = Dot - CurOutSec->getVA();
+    Dot += DataCmd->Size;
+    CurOutSec->setSize(Dot - CurOutSec->getVA());
+    return;
+  }
+
+  // It handles single input section description command,
+  // calculates and assigns the offsets for each section and also
+  // updates the output section size.
   auto &ICmd = cast<InputSectionDescription>(Base);
   for (InputSectionData *ID : ICmd.Sections) {
     auto *IB = static_cast<InputSectionBase<ELFT> *>(ID);
     switchTo(IB->OutSec);
     if (auto *I = dyn_cast<InputSection<ELFT>>(IB))
       output(I);
     else if (AlreadyOutputOS.insert(CurOutSec).second)
       Dot += CurOutSec->getSize();

 ArrayRef<uint8_t> LinkerScript<ELFT>::getFiller(StringRef Name) {
   for (const std::unique_ptr<BaseCommand> &Base : Opt.Commands)
     if (auto *Cmd = dyn_cast<OutputSectionCommand>(Base.get()))
       if (Cmd->Name == Name)
         return Cmd->Filler;
   return {};
 }
 
+template <class ELFT>
+void LinkerScript<ELFT>::writeDataBytes(StringRef Name, uint8_t *Buf) {
+  const endianness E = ELFT::TargetEndianness;
+
+  int I = getSectionIndex(Name);
+  OutputSectionCommand *Cmd = dyn_cast_or_null<OutputSectionCommand>(
+      (I == INT_MAX) ? nullptr : Opt.Commands[I].get());
+  if (!Cmd)
+    return;
+
+  for (const std::unique_ptr<BaseCommand> &Base2 : Cmd->Commands) {
+    if (auto *DataCmd = dyn_cast<BytesDataCommand>(Base2.get())) {
+      // If the object file format of the output file has an explicit
+      // endianness, the value will be stored in that endianness.
+      switch (DataCmd->Size) {
+      case 1:
+        Buf[DataCmd->Offset] = (uint8_t)DataCmd->Data;
+        break;
+      case 2:
+        write16<E>(&Buf[DataCmd->Offset], (uint16_t)DataCmd->Data);
+        break;
+      case 4:
+        write32<E>(&Buf[DataCmd->Offset], (uint32_t)DataCmd->Data);
+        break;
+      case 8:
+        write64<E>(&Buf[DataCmd->Offset], DataCmd->Data);
+        break;
+      }
+    }
+  }
+}
+
 template <class ELFT> Expr LinkerScript<ELFT>::getLma(StringRef Name) {
   for (const std::unique_ptr<BaseCommand> &Base : Opt.Commands)
     if (auto *Cmd = dyn_cast<OutputSectionCommand>(Base.get()))
       if (Cmd->LmaExpr && Cmd->Name == Name)
         return Cmd->LmaExpr;
   return {};
 }
 
private:
   void readOutputFormat();
   void readPhdrs();
   void readSearchDir();
   void readSections();
   void readVersion();
   void readVersionScriptCommand();
 
   SymbolAssignment *readAssignment(StringRef Name);
+  BytesDataCommand *readBytesDataCommand(StringRef Tok);
   std::vector<uint8_t> readFill();
   OutputSectionCommand *readOutputSectionDescription(StringRef OutSec);
   std::vector<uint8_t> readOutputSectionFiller(StringRef Tok);
   std::vector<StringRef> readOutputSectionPhdrs();
   InputSectionDescription *readInputSectionDescription(StringRef Tok);
   Regex readFilePatterns();
   std::vector<SectionPattern> readInputSectionsList();
   InputSectionDescription *readInputSectionRules(StringRef FilePattern);
ScriptParser::readOutputSectionDescription(StringRef OutSec) {
   if (skip("ONLY_IF_RW"))
     Cmd->Constraint = ConstraintKind::ReadWrite;
   expect("{");
 
   while (!Error && !skip("}")) {
     StringRef Tok = next();
     if (SymbolAssignment *Assignment = readProvideOrAssignment(Tok, false))
       Cmd->Commands.emplace_back(Assignment);
+    else if (BytesDataCommand *Data = readBytesDataCommand(Tok))
+      Cmd->Commands.emplace_back(Data);
     else if (Tok == "FILL")
       Cmd->Filler = readFill();
     else if (Tok == "SORT")
       readSort();
     else if (peek() == "(")
       Cmd->Commands.emplace_back(readInputSectionDescription(Tok));
     else
       setError("unknown command " + Tok);
if (Tok.endswith_lower("K")) {
     Tok = Tok.drop_back();
   }
   if (Tok.getAsInteger(10, Result))
     return false;
   Result *= Suffix;
   return true;
 }
 
+BytesDataCommand *ScriptParser::readBytesDataCommand(StringRef Tok) {
+  unsigned Size = StringSwitch<unsigned>(Tok)
+                      .Case("BYTE", 1)
+                      .Case("SHORT", 2)
+                      .Case("LONG", 4)
+                      .Case("QUAD", 8)
+                      .Default(-1);
+  if (Size == (unsigned)-1)
+    return nullptr;
+
+  expect("(");
+  uint64_t Val = 0;
+  StringRef S = next();
+  if (!readInteger(S, Val))
+    setError("unexpected value: " + S);
+  expect(")");
+  return new BytesDataCommand(Val, Size);
+}
+
 Expr ScriptParser::readPrimary() {
   if (peek() == "(")
     return readParenExpr();
 
   StringRef Tok = next();
 
   if (Tok == "~") {
     Expr E = readPrimary();

ELF/OutputSections.cpp

 //===- OutputSections.cpp -------------------------------------------------===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "OutputSections.h"
 #include "Config.h"
 #include "EhFrame.h"
if (!Filler.empty())
     fill(Buf, this->getSize(), Filler);
   if (Config->Threads) {
     parallel_for_each(Sections.begin(), Sections.end(),
                       [=](InputSection<ELFT> *C) { C->writeTo(Buf); });
   } else {
     for (InputSection<ELFT> *C : Sections)
       C->writeTo(Buf);
   }
+  // Linker scripts may have BYTE()-family commands with which you
+  // can write arbitrary bytes to the output. Process them if any.
+  Script<ELFT>::X->writeDataBytes(this->Name, Buf);
 }
 
 template <class ELFT>
 EhOutputSection<ELFT>::EhOutputSection()
     : OutputSectionBase<ELFT>(".eh_frame", SHT_PROGBITS, SHF_ALLOC) {}
 
 // Search for an existing CIE record or create a new one.
 // CIE records from input object files are uniquified by their contents

test/ELF/linkerscript/data-commands.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+
+# RUN: echo "SECTIONS                \
+# RUN:  {                            \
+# RUN:    .foo : {                   \
+# RUN:      *(.foo.1)                \
+# RUN:      BYTE(0x11)               \
+# RUN:      *(.foo.2)                \
+# RUN:      SHORT(0x1122)            \
+# RUN:      *(.foo.3)                \
+# RUN:      LONG(0x11223344)         \
+# RUN:      *(.foo.4)                \
+# RUN:      QUAD(0x1122334455667788) \
+# RUN:    }                          \
+# RUN:  }" > %t.script
+# RUN: ld.lld -o %t --script %t.script %t.o
+# RUN: llvm-objdump -s %t | FileCheck %s
+
+# CHECK:      Contents of section .foo:
+# CHECK-NEXT:  00e8 ff11ff22 11ff4433 2211ff88 77665544
+# CHECK-NEXT:  00f8 332211
+
+.section .foo.1, "a"
+ .byte 0xFF
+
+.section .foo.2, "a"
+ .byte 0xFF
+
+.section .foo.3, "a"
+ .byte 0xFF
+
+.section .foo.4, "a"
+ .byte 0xFF