[ELF] - Reveal more information in -Map file about assignments.

[grimar]

Summary

Currently, LLD print symbol assignment commands to the map file,
but it does not do that for assignments that are outside of the section
descriptions. Such assignments can affect the layout though.

The patch implements the following:

1. Teaches LLD to print symbol assignments outside of section declaration.
2. Teaches LLD to print PROVIDE/HIDDEN/PROVIDE hidden commands.

In case when symbol is not provided, nothing will be printed.

Second part slightly differs from gold. It prints information about all
symbols to the map file (both about provided and not provided):

.bar3       0x0000000000000030        0x8 test.o
               [!provide]                        PROVIDE (sym4 = 0x2a)
               0x000000000000002a     PROVIDE (sym3 = 0x2a)

I think it probably does not make much sense to print not-provided
symbols, so patch ignores them. (Though it would be easy to print them too).

[ruiu]

ELF/LinkerScript.h

113–114

I think the original comment makes sense. I can understand what "before doing the assignment" means, but it feels easier to understand if you just say it's an address of an assignment.

118

Remove "

202–207

Why do you need to add these field to two different places? We generally should avoid doing this.

ELF/MapFile.cpp

153

Let's not generalize. Just pass a boolean value instead of a string.

OK, so you are using this function only twice. I'd inline this function.

[grimar]

• Addressed review comments.

ELF/LinkerScript.h

202–207

I am not adding Size in this patch, just adjusting the comments.

Now we have SymbolAssignment, ByteCommand and OutputSection users of field Size.
I thought about the possibility to move this field to BaseCommand but my concern is
that for each of these commands, Size has the different meaning. Size of the output section is obvious,
as well as the size of the data command, but the size of the assignment is very different.
We do not have Size in AssertCommand (and it probably makes no sense to have it).
Currently, all of the instances has its own comments and it looks good to me.

[• espindola]

Fine by me, but wait for a final OK from Rui.

Some parts of this could be an independent cleanup. Replacing Offset with Addr for example. Please commit that first.

[grimar]

• Rebased after cleanup committed (rL329162).

[• espindola]

Fine by me, but do wait for a second OK from Rui.

[ruiu]

LGTM

ELF/LinkerScript.h

 
 struct BaseCommand {
   BaseCommand(int K) : Kind(K) {}
   int Kind;
 };
 
 // This represents ". = <expr>" or "<symbol> = <expr>".
 struct SymbolAssignment : BaseCommand {
-  SymbolAssignment(StringRef Name, Expr E, std::string Loc,
-                   std::string CommandString)
-      : BaseCommand(AssignmentKind), Name(Name), Expression(E), Location(Loc),
-        CommandString(CommandString) {}
+  SymbolAssignment(StringRef Name, Expr E, std::string Loc)
+      : BaseCommand(AssignmentKind), Name(Name), Expression(E), Location(Loc) {}
 
   static bool classof(const BaseCommand *C) {
     return C->Kind == AssignmentKind;
   }
 
   // The LHS of an expression. Name is either a symbol name or ".".
   StringRef Name;
   Defined *Sym = nullptr;
 
   // The RHS of an expression.
   Expr Expression;
 
   // Command attributes for PROVIDE, HIDDEN and PROVIDE_HIDDEN.
   bool Provide = false;
   bool Hidden = false;
 
   // Holds file name and line number for error reporting.
   std::string Location;
 
   // A string representation of this command. We use this for -Map.
   std::string CommandString;
 
-  // This is just an offset of this assignment command in the output section.
-  unsigned Offset;
+  // Address of this assignment command. This corresponds to the value
+  // of Dot for the moment before doing the assignment.
+  unsigned Addr;
 
-  // Size of this assignment command. This is usually 0, but if you move '.'
-  // or use a BYTE()-family command, this may be greater than 0."
+  // Size of this assignment command. This is usually 0, but if
+  // you move '.' this may be greater than 0."
   unsigned Size;
 };
 
 // Linker scripts allow additional constraints to be put on ouput sections.
 // If an output section is marked as ONLY_IF_RO, the section is created
 // only if its input sections are read-only. Likewise, an output section
 // with ONLY_IF_RW is created if all input sections are RW.
 enum class ConstraintKind { NoConstraint, ReadOnly, ReadWrite };
ByteCommand(Expr E, unsigned Size, std::string CommandString)
         Size(Size) {}
 
   static bool classof(const BaseCommand *C) { return C->Kind == ByteKind; }
 
   // Keeps string representing the command. Used for -Map" is perhaps better.
   std::string CommandString;
 
   Expr Expression;
+
+  // This is just an offset of this assignment command in the output section.
   unsigned Offset;
+
+  // Size of this data command.
   unsigned Size;
 };
 
 struct PhdrsCommand {
   StringRef Name;
   unsigned Type = llvm::ELF::PT_NULL;
   bool HasFilehdr = false;
   bool HasPhdrs = false;
   llvm::Optional<unsigned> Flags;

ELF/LinkerScript.cpp

void LinkerScript::assignOffsets(OutputSection *Sec) {
   Sec->Size = 0;
 
   // We visited SectionsCommands from processSectionCommands to
   // layout sections. Now, we visit SectionsCommands again to fix
   // section offsets.
   for (BaseCommand *Base : Sec->SectionCommands) {
     // This handles the assignments to symbol or to the dot.
     if (auto *Cmd = dyn_cast<SymbolAssignment>(Base)) {
-      Cmd->Offset = Dot - Ctx->OutSec->Addr;
+      Cmd->Addr = Dot;
       assignSymbol(Cmd, true);
-      Cmd->Size = Dot - Ctx->OutSec->Addr - Cmd->Offset;
+      Cmd->Size = Dot - Cmd->Addr;
       continue;
     }
 
     // Handle BYTE(), SHORT(), LONG(), or QUAD().
     if (auto *Cmd = dyn_cast<ByteCommand>(Base)) {
       Cmd->Offset = Dot - Ctx->OutSec->Addr;
       Dot += Cmd->Size;
       expandOutputSection(Cmd->Size);
void LinkerScript::assignAddresses() {
 
   auto Deleter = make_unique<AddressState>();
   Ctx = Deleter.get();
   ErrorOnMissingSection = true;
   switchTo(Aether);
 
   for (BaseCommand *Base : SectionCommands) {
     if (auto *Cmd = dyn_cast<SymbolAssignment>(Base)) {
+      Cmd->Addr = Dot;
       assignSymbol(Cmd, false);
+      Cmd->Size = Dot - Cmd->Addr;
       continue;
     }
 
     if (auto *Cmd = dyn_cast<AssertCommand>(Base)) {
       Cmd->Expression();
       continue;
     }
 

ELF/MapFile.cpp

static void printEhFrame(raw_ostream &OS, OutputSection *OSec) {
   // Print out section pieces.
   for (EhSectionPiece &P : Pieces) {
     writeHeader(OS, OSec->Addr + P.OutputOff, P.Size, 0);
     OS << Indent8 << toString(P.Sec->File) << ":(" << P.Sec->Name << "+0x"
        << Twine::utohexstr(P.InputOff) + ")\n";
   }
 }
 
+static void writeAssignment(raw_ostream &OS, SymbolAssignment *Cmd,
+                            const Twine &Padding = "") {
+  // Do not print symbols that were not provided.
+  if (Cmd->Provide && !Cmd->Sym)
+    return;
+  writeHeader(OS, Cmd->Addr, Cmd->Size, 1);
+  OS << Padding << Cmd->CommandString << '\n';
+}
+
 void elf::writeMapFile() {
   if (Config->MapFile.empty())
     return;
 
   // Open a map file for writing.
   std::error_code EC;
   raw_fd_ostream OS(Config->MapFile, EC, sys::fs::F_None);
   if (EC) {
     error("cannot open " + Config->MapFile + ": " + EC.message());
     return;
   }
 
   // Collect symbol info that we want to print out.
   std::vector<Symbol *> Syms = getSymbols();
   SymbolMapTy SectionSyms = getSectionSyms(Syms);
   DenseMap<Symbol *, std::string> SymStr = getSymbolStrings(Syms);
 
   // Print out the header line.
   int W = Config->Is64 ? 16 : 8;
   OS << left_justify("Address", W) << ' ' << left_justify("Size", W)
      << " Align Out     In      Symbol\n";
 
-  // Print out file contents.
-  for (OutputSection *OSec : OutputSections) {
+  for (BaseCommand *Base : Script->SectionCommands) {
+    if (auto *Cmd = dyn_cast<SymbolAssignment>(Base)) {
+      writeAssignment(OS, Cmd);
+      continue;
+    }
+
+    auto *OSec = dyn_cast<OutputSection>(Base);
+    if (!OSec)
+      continue;
+
     writeHeader(OS, OSec->Addr, OSec->Size, OSec->Alignment);
     OS << OSec->Name << '\n';
 
     // Dump symbols for each input section.
     for (BaseCommand *Base : OSec->SectionCommands) {
       if (auto *ISD = dyn_cast<InputSectionDescription>(Base)) {
         for (InputSection *IS : ISD->Sections) {
           if (IS == InX::EhFrame) {
for (BaseCommand *Base : OSec->SectionCommands) {
 
       if (auto *Cmd = dyn_cast<ByteCommand>(Base)) {
         writeHeader(OS, OSec->Addr + Cmd->Offset, Cmd->Size, 1);
         OS << Indent8 << Cmd->CommandString << '\n';
         continue;
       }
 
       if (auto *Cmd = dyn_cast<SymbolAssignment>(Base)) {
-        writeHeader(OS, OSec->Addr + Cmd->Offset, Cmd->Size, 1);
-        OS << Indent8 << Cmd->CommandString << '\n';
+        writeAssignment(OS, Cmd, Indent8);
         continue;
       }
     }
   }
 }
 
 static void print(StringRef A, StringRef B) {
   outs() << left_justify(A, 49) << " " << B << "\n";

ELF/ScriptParser.cpp

while (!atEOF()) {
     }
   }
 }
 
 void ScriptParser::readDefsym(StringRef Name) {
   Expr E = readExpr();
   if (!atEOF())
     setError("EOF expected, but got " + next());
-  SymbolAssignment *Cmd = make<SymbolAssignment>(Name, E, getCurrentLocation(),
-                                                 "" /*CommandString*/);
+  SymbolAssignment *Cmd = make<SymbolAssignment>(Name, E, getCurrentLocation());
   Script->SectionCommands.push_back(Cmd);
 }
 
 void ScriptParser::addFile(StringRef S) {
   if (IsUnderSysroot && S.startswith("/")) {
     SmallString<128> PathData;
     StringRef Path = (Config->Sysroot + S).toStringRef(PathData);
     if (sys::fs::exists(Path)) {

 }
 
 SymbolAssignment *ScriptParser::readProvideHidden(bool Provide, bool Hidden) {
   expect("(");
   SymbolAssignment *Cmd = readAssignment(next());
   Cmd->Provide = Provide;
   Cmd->Hidden = Hidden;
   expect(")");
-  expect(";");
   return Cmd;
 }
 
 SymbolAssignment *ScriptParser::readProvideOrAssignment(StringRef Tok) {
+  size_t OldPos = Pos;
   SymbolAssignment *Cmd = nullptr;
-  if (peek() == "=" || peek() == "+=") {
+  if (peek() == "=" || peek() == "+=")
     Cmd = readAssignment(Tok);
-    expect(";");
-  } else if (Tok == "PROVIDE") {
+  else if (Tok == "PROVIDE")
     Cmd = readProvideHidden(true, false);
-  } else if (Tok == "HIDDEN") {
+  else if (Tok == "HIDDEN")
     Cmd = readProvideHidden(false, true);
-  } else if (Tok == "PROVIDE_HIDDEN") {
+  else if (Tok == "PROVIDE_HIDDEN")
     Cmd = readProvideHidden(true, true);
+
+  if (Cmd) {
+    Cmd->CommandString =
+        Tok.str() + " " +
+        llvm::join(Tokens.begin() + OldPos, Tokens.begin() + Pos, " ");
+    expect(";");
   }
   return Cmd;
 }
 
 SymbolAssignment *ScriptParser::readAssignment(StringRef Name) {
-  size_t OldPos = Pos;
   StringRef Op = next();
   assert(Op == "=" || Op == "+=");
   Expr E = readExpr();
   if (Op == "+=") {
     std::string Loc = getCurrentLocation();
     E = [=] { return add(Script->getSymbolValue(Name, Loc), E()); };
   }
-
-  std::string CommandString =
-      Name.str() + " " +
-      llvm::join(Tokens.begin() + OldPos, Tokens.begin() + Pos, " ");
-  return make<SymbolAssignment>(Name, E, getCurrentLocation(), CommandString);
+  return make<SymbolAssignment>(Name, E, getCurrentLocation());
 }
 
 // This is an operator-precedence parser to parse a linker
 // script expression.
 Expr ScriptParser::readExpr() {
   // Our lexer is context-aware. Set the in-expression bit so that
   // they apply different tokenization rules.
   bool Orig = InExpr;

test/ELF/linkerscript/map-file.test

 # REQUIRES: x86
 
-# RUN: echo '.section .foo.1, "a"; .quad 1' > %t.s
+# RUN: echo '.quad sym3; .quad sym4; .section .foo.1, "a"; .quad 1' > %t.s
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %t.s -o %t.o
 
 # RUN: ld.lld -o %t %t.o -Map=%t.map --script %s
 # RUN: FileCheck -strict-whitespace %s < %t.map
 
 SECTIONS {
   . = 0x1000;
   .foo : {
     BYTE(0x11)
     SHORT(0x1122)
     LONG(0x11223344)
     QUAD(0x1122334455667788)
+    PROVIDE_HIDDEN(sym4 = .);
     . += 0x1000;
     *(.foo.1)
+    PROVIDE(unused1 = 0xff);
+    HIDDEN(sym6 = .);
     . += 0x123 *
          (1 + 1);
     foo = .;
     bar = 0x42 - 0x26;
   }
+  sym1 = .;
+  . += 0x500;
+  sym2 = .;
+  PROVIDE(unused2 = 0xff);
+  PROVIDE(sym3 = 42);
 }
 
 # CHECK:      Address          Size             Align Out     In      Symbol
+# CHECK-NEXT: 0000000000000000 0000000000001000     1 . = 0x1000
 # CHECK-NEXT: 0000000000001000 000000000000125d     1 .foo
 # CHECK-NEXT: 0000000000001000 0000000000000001     1         BYTE ( 0x11 )
 # CHECK-NEXT: 0000000000001001 0000000000000002     1         SHORT ( 0x1122 )
 # CHECK-NEXT: 0000000000001003 0000000000000004     1         LONG ( 0x11223344 )
 # CHECK-NEXT: 0000000000001007 0000000000000008     1         QUAD ( 0x1122334455667788 )
+# CHECK-NEXT: 000000000000100f 0000000000000000     1         PROVIDE_HIDDEN ( sym4 = . )
 # CHECK-NEXT: 000000000000100f 0000000000001000     1         . += 0x1000
 # CHECK-NEXT: 000000000000200f 0000000000000008     1         {{.*}}{{/|\\}}map-file.test.tmp.o:(.foo.1)
+# CHECK-NEXT: 0000000000002017 0000000000000000     1         HIDDEN ( sym6 = . )
 # CHECK-NEXT: 0000000000002017 0000000000000246     1         . += 0x123 * ( 1 + 1 )
 # CHECK-NEXT: 000000000000225d 0000000000000000     1         foo = .
 # CHECK-NEXT: 000000000000225d 0000000000000000     1         bar = 0x42 - 0x26
-# CHECK-NEXT: 0000000000002260 0000000000000000     4 .text
-# CHECK-NEXT: 0000000000002260 0000000000000000     4         {{.*}}{{/|\\}}map-file.test.tmp.o:(.text)
+# CHECK-NEXT: 000000000000225d 0000000000000000     1 sym1 = .
+# CHECK-NEXT: 000000000000225d 0000000000000500     1 . += 0x500
+# CHECK-NEXT: 000000000000275d 0000000000000000     1 sym2 = .
+# CHECK-NEXT: 000000000000275d 0000000000000000     1 PROVIDE ( sym3 = 42 )
+# CHECK-NEXT: 0000000000002760 0000000000000010     4 .text
+# CHECK-NEXT: 0000000000002760 0000000000000010     4         {{.*}}{{/|\\}}map-file.test.tmp.o:(.text)