Add --print-icf-sections flag

[gbreynoo]

Summary

Currently ICF information is output through stderr if the "--verbose" flag is used. This differs to Gold for example, which uses an explicit flag to output this to stdout. This diff adds the "--print-icf-sections" and "--no-print-icf-sections" flags and changes the output message format for clarity and consistency with "--print-gc-sections". These messages are still output to stderr if using the verbose flag. However to avoid intermingled message output to console, this will not occur when the "--print-icf-sections" flag is used.

Existing tests have been modified to expect the new message format from stderr.

[gbreynoo]

Updated tests that used existing icf output

[ruiu]

ELF/Driver.cpp

4

hasFlag ( -> hasFlag(

ELF/ICF.cpp

18–22

Can you define this function

static void print(const Twine &S) {
  if (Config->PrintIcf)
    message(S);
  else
    log(S);
}

and use it?

[gbreynoo]

Changes following Rui's comments

[gbreynoo]

Changes following Rafael's comment on the mailing list:
"gold calls this option --print-icf-sections, why not use the same name?"

[grimar]

test/ELF/print-icf.s

8

I think it is better to use explicit way of checking the output:

1. RUN: ld.lld %t %t1 -o %t2 --icf=all --print-icf-sections --no-print-icf-sections | FileCheck %s --check-prefix=PREFIX
2. PREFIX-NOT: selected
3. PREFIX-NOT: removing

Otherwise test can pass by mistake.

[gbreynoo]

Changes following George and Rafaels comments.

[gbreynoo]

Changes following Rafael's comment:

+ auto PrintICF = [&](const Twine &prefix, size_t I) {

prefix -> Prefix

+ const auto *F = Sections[I]->File;
+ Twine S = prefix + " section '" + Sections[I]->Name + "' from file '" +
+ (F ? Twine(F->getName()) : "<internal>") + "'";

It is in general dangerous to use a Twine Value. I would suggest
writting this as:

if (!Config->Verbose && !Config->PrintIcfSections)
return;

std::string S = ....;

[ruiu]

ELF/ICF.cpp

15

Everything in this file is related to ICF, so "ICF" is redundant unless it is in the global namespace. I'd name this just Print.

18

Please avoid auto unless the actual type is obvious.

Maybe this code is a bit better.

std::string Filename = Sections[I]->File ? Sections[I]->File->getName() : "<internal>";

19–21

I believe you can do this (which creates a Twine and call its str() function).

std::string S = (Prefix + " section '" + Sections[I]->Name +
                  "' from file '" + (F ? F->getName() : "<internal>") +
                  "'").str();

[gbreynoo]

Changes following Rui's comments

[gbreynoo]

Change following Rui's comments, missed in last diff

[ruiu]

LGTM

ELF/Config.h

struct Configuration {
   bool NoUndefinedVersion;
   bool NoinhibitExec;
   bool Nostdlib;
   bool OFormatBinary;
   bool Omagic;
   bool OptRemarksWithHotness;
   bool Pie;
   bool PrintGcSections;
+  bool PrintIcfSections;
   bool Relocatable;
   bool SaveTemps;
   bool SingleRoRx;
   bool Shared;
   bool Static = false;
   bool SysvHash = false;
   bool Target1Rel;
   bool Trace;

ELF/Driver.cpp

 //===- Driver.cpp ---------------------------------------------------------===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // The driver drives the entire linking process. It is responsible for
 // parsing command line options and doing whatever it is instructed to do.
 //
void LinkerDriver::readConfigs(opt::InputArgList &Args) {
   Config->OFormatBinary = isOutputFormatBinary(Args);
   Config->Omagic = Args.hasFlag(OPT_omagic, OPT_no_omagic, false);
   Config->OptRemarksFilename = Args.getLastArgValue(OPT_opt_remarks_filename);
   Config->OptRemarksWithHotness = Args.hasArg(OPT_opt_remarks_with_hotness);
   Config->Optimize = args::getInteger(Args, OPT_O, 1);
   Config->OrphanHandling = getOrphanHandling(Args);
   Config->OutputFile = Args.getLastArgValue(OPT_o);
   Config->Pie = Args.hasFlag(OPT_pie, OPT_nopie, false);
+  Config->PrintIcfSections =
+      Args.hasFlag(OPT_print_icf_sections, OPT_no_print_icf_sections, false);
   Config->PrintGcSections =
       Args.hasFlag(OPT_print_gc_sections, OPT_no_print_gc_sections, false);
   Config->Rpath = getRpath(Args);
   Config->Relocatable = Args.hasArg(OPT_relocatable);
   Config->SaveTemps = Args.hasArg(OPT_save_temps);
   Config->SearchPaths = args::getStrings(Args, OPT_library_path);
   Config->SectionStartMap = getSectionStartMap(Args);
   Config->Shared = Args.hasArg(OPT_shared);

ELF/ICF.cpp

 //===- ICF.cpp ------------------------------------------------------------===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // ICF is short for Identical Code Folding. This is a size optimization to
 // identify and merge two or more read-only sections (typically functions)
 // that happened to have the same contents. It usually reduces output size
 // by a few percent.
 //
 // In ICF, two sections are considered identical if they have the same
 // section flags, section data, and relocations. Relocations are tricky,
 // because two relocations are considered the same if they have the same
 // relocation types, values, and if they point to the same sections *in
 // terms of ICF*.
 //
 // Here is an example. If foo and bar defined below are compiled to the
 // same machine instructions, ICF can and should merge the two, although
 // their relocations point to each other.
 //
 //   void foo() { bar(); }
 //   void bar() { foo(); }
 //
 // If you merge the two, their relocations point to the same section and
 // thus you know they are mergeable, but how do you know they are
 // mergeable in the first place? This is not an easy problem to solve.
template <class ELFT> void ICF<ELFT>::run() {
   do {
     Repeat = false;
     forEachClass(
         [&](size_t Begin, size_t End) { segregate(Begin, End, false); });
   } while (Repeat);
 
   log("ICF needed " + Twine(Cnt) + " iterations");
 
+  auto Print = [&](const Twine &Prefix, size_t I) {
+    if (!Config->Verbose && !Config->PrintIcfSections)
+      return;
+    std::string Filename =
+        Sections[I]->File ? Sections[I]->File->getName() : "<internal>";
+    std::string S = (Prefix + " section '" + Sections[I]->Name +
+                     "' from file '" + Filename + "'")
+                        .str();
+    if (Config->PrintIcfSections)
+      message(S);
+    else
+      log(S);
+  };
+
   // Merge sections by the equivalence class.
   forEachClass([&](size_t Begin, size_t End) {
     if (End - Begin == 1)
       return;
 
-    log("selected " + Sections[Begin]->Name);
+    Print("selected", Begin);
     for (size_t I = Begin + 1; I < End; ++I) {
-      log("  removed " + Sections[I]->Name);
+      Print("  removing identical", I);
       Sections[Begin]->replace(Sections[I]);
     }
   });
 
   // Mark ARM Exception Index table sections that refer to folded code
   // sections as not live. These sections have an implict dependency
   // via the link order dependency.
   if (Config->EMachine == EM_ARM)

ELF/Options.td

 def nopie: F<"nopie">, HelpText<"Do not create a position independent executable">;
 
 def no_omagic: Flag<["--"], "no-omagic">, MetaVarName<"<magic>">,
   HelpText<"Do not set the text data sections to be writable">;
 
 def no_print_gc_sections: F<"no-print-gc-sections">,
   HelpText<"Do not list removed unused sections">;
 
+def no_print_icf_sections: F<"no-print-icf-sections">,
+  HelpText<"Do not list identical folded sections">;
+
 def no_rosegment: F<"no-rosegment">,
   HelpText<"Do not put read-only non-executable sections in their own segment">;
 
 def no_undefined: F<"no-undefined">,
   HelpText<"Report unresolved symbols even if the linker is creating a shared library">;
 
 def no_undefined_version: F<"no-undefined-version">,
   HelpText<"Report version scripts that refer undefined symbols">;

 def pack_dyn_relocs_eq: J<"pack-dyn-relocs=">, MetaVarName<"<format>">,
   HelpText<"Pack dynamic relocations in the given format (none or android)">;
 
 def pie: F<"pie">, HelpText<"Create a position independent executable">;
 
 def print_gc_sections: F<"print-gc-sections">,
   HelpText<"List removed unused sections">;
 
+def print_icf_sections: F<"print-icf-sections">,
+  HelpText<"List identical folded sections">;
+
 def print_map: F<"print-map">,
   HelpText<"Print a link map to the standard output">;
 
 defm reproduce: Eq<"reproduce">,
   HelpText<"Dump linker invocation and input files for debugging">;
 
 defm rpath: Eq<"rpath">, HelpText<"Add a DT_RUNPATH to the output">;
 

test/ELF/Inputs/print-icf.s

+.section .text.f6, "ax"
+f6:
+  mov $60, %rax
+  mov $42, %rdi
+  syscall
+
+  .section .text.f7, "ax"
+f7:
+  mov $0, %rax

test/ELF/icf-absolute.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/icf-absolute.s -o %t2
 # RUN: ld.lld %t %t2 -o %t3 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file
+# CHECK:   removing identical section '.text.f2' from file
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   .byte a1
 
 .section .text.f2, "ax"
 f2:
   .byte a2

test/ELF/icf-comdat.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file [[T:'.*']]
+# CHECK:   removing identical section '.text.f2' from file [[T]]
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1,"ax"
 f1:
   mov $60, %rax
   mov $42, %rdi
   syscall
 
 .section .text.f2,"axG",@progbits,foo,comdat
 f2:
   mov $60, %rax
   mov $42, %rdi
   syscall

test/ELF/icf-i386.s

 # REQUIRES: x86
 # This test is to make sure that we can handle implicit addends properly.
 
 # RUN: llvm-mc -filetype=obj -triple=i386-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK:     selected .text.f1
-# CHECK:       removed .text.f2
-# CHECK-NOT:   removed .text.f3
+# CHECK:     selected section '.text.f1' from file [[T:'.*']]
+# CHECK:       removing identical section '.text.f2' from file [[T]]
+# CHECK-NOT:   removing identical section '.text.f3' from file [[T]]
 
 .globl _start, f1, f2, f3
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   movl $42, 4(%edi)
 
 .section .text.f2, "ax"
 f2:
   movl $42, 4(%edi)
 
 .section .text.f3, "ax"
 f3:
   movl $42, 8(%edi)

test/ELF/icf-merge-sec.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/icf-merge-sec.s -o %t2
 # RUN: ld.lld %t %t2 -o %t3 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file
+# CHECK:   removing identical section '.text.f2' from file
 
 .section .rodata.str,"aMS",@progbits,1
 .asciz "foo"
 .asciz "string 1"
 .asciz "string 2"
 
 .section .text.f1,"ax"
 .globl f1
 f1:
 .quad .rodata.str

test/ELF/icf-merge.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/icf-merge.s -o %t1
 # RUN: ld.lld %t %t1 -o %t1.out --icf=all --verbose 2>&1 | FileCheck %s
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/icf-merge2.s -o %t2
 # RUN: ld.lld %t %t2 -o %t3.out --icf=all --verbose 2>&1 | FileCheck --check-prefix=NOMERGE %s
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/icf-merge3.s -o %t3
 # RUN: ld.lld %t %t3 -o %t3.out --icf=all --verbose 2>&1 | FileCheck --check-prefix=NOMERGE %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file
+# CHECK:   removing identical section '.text.f2' from file
 
-# NOMERGE-NOT: selected .text.f
+# NOMERGE-NOT: selected section '.text.f
 
 .section .rodata.str,"aMS",@progbits,1
 foo:
 .asciz "foo"
 .asciz "string 1"
 .asciz "string 2"
 
 .section .text.f1,"ax"
 .globl f1
 f1:
 lea foo+42(%rip), %rax

test/ELF/icf-non-mergeable.s

 // REQUIRES: x86
 
 // This file contains two functions. They are themselves identical,
 // but because they have relocations against different data sections,
 // they are not mergeable.
 
 // RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1
 // RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux \
 // RUN:    %p/Inputs/icf-non-mergeable.s -o %t2
 
 // RUN: ld.lld %t1 %t2 -o %t3 --icf=all --verbose 2>&1 | FileCheck %s
 
-// CHECK-NOT: selected .text.f1
-// CHECK-NOT:   removed .text.f2
+// CHECK-NOT: selected section '.text.f1'
+// CHECK-NOT:   removing identical section '.text.f2'
 
 .globl _start, f1, f2, d1, d2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   movl $5, d1
   ret
 
 .section .text.f2, "ax"
 f2:
   movl $5, d2
   ret

test/ELF/icf-none.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --icf=none --verbose 2>&1 | FileCheck %s
 
-# CHECK-NOT: selected .text.f1
+# CHECK-NOT: selected section '.text.f1'
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   mov $60, %rax
   mov $42, %rdi
   syscall
 
 .section .text.f2, "ax"
 f2:
   mov $60, %rax
   mov $42, %rdi
   syscall

test/ELF/icf1.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file [[T:'.*']]
+# CHECK:   removing identical section '.text.f2' from file [[T]]
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   mov $60, %rax
   mov $42, %rdi
   syscall
 
 .section .text.f2, "ax"
 f2:
   mov $60, %rax
   mov $42, %rdi
   syscall

test/ELF/icf2.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/icf2.s -o %t2
 # RUN: ld.lld %t1 %t2 -o %t --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file
+# CHECK:   removing identical section '.text.f2' from file
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   mov $60, %rdi
   call f2

test/ELF/icf3.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/icf2.s -o %t2
 # RUN: ld.lld %t1 %t2 -o %t --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK-NOT: Selected .text.f1
-# CHECK-NOT: Selected .text.f2
+# CHECK-NOT: selected section '.text.f1' from file
+# CHECK-NOT: selected section '.text.f2' from file
 
 .globl _start, f1, f2
 _start:
   ret
 
 # This section is not mergeable because the content is different from f2.
 .section .text.f1, "ax"
 f1:
   mov $60, %rdi
   call f2
   mov $0, %rax

test/ELF/icf4.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK-NOT: Selected .text.f1
-# CHECK-NOT: Selected .text.f2
+# CHECK-NOT: selected section '.text.f1'
+# CHECK-NOT: selected section '.text.f2'
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   mov $1, %rax
 
 .section .text.f2, "ax"
 f2:
   mov $0, %rax

test/ELF/icf5.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK-NOT: Selected .text.f1
-# CHECK-NOT: Selected .text.f2
+# CHECK-NOT: selected section '.text.f1'
+# CHECK-NOT: selected section '.text.f2'
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .text.f1, "ax"
 f1:
   mov $0, %rax
 
 .section .text.f2, "awx"
 f2:
   mov $0, %rax

test/ELF/icf6.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 
-# CHECK-NOT: Selected .text.f1
-# CHECK-NOT: Selected .text.f2
+# CHECK-NOT: selected section '.text.f1'
+# CHECK-NOT: selected section '.text.f2'
 
 .globl _start, f1, f2
 _start:
   ret
 
 .section .init, "ax"
 f1:
   mov $60, %rax
   mov $42, %rdi
   syscall
 
 .section .fini, "ax"
 f2:
   mov $60, %rax
   mov $42, %rdi
   syscall

test/ELF/icf7.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 # RUN: llvm-objdump -t %t2 | FileCheck -check-prefix=ALIGN %s
 
-# CHECK: selected .text.f1
-# CHECK:   removed .text.f2
+# CHECK: selected section '.text.f1' from file [[T:'.*']]
+# CHECK:   removing identical section '.text.f2' from file [[T]]
 
 # ALIGN: 0000000000201000 .text 00000000 _start
 # ALIGN: 0000000000201100 .text 00000000 f1
 
 .globl _start, f1, f2
 _start:
   ret
 

test/ELF/icf9.s

 # REQUIRES: x86
 
 ### Make sure that we do not merge data.
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
 # RUN: ld.lld %t -o %t2 --icf=all --verbose 2>&1 | FileCheck %s
 # RUN: llvm-readelf -S -W %t2 | FileCheck --check-prefix=SEC %s
 
 # SEC:  .rodata      PROGBITS  0000000000200120 000120 000002 00 A 0 0 1
 
-# CHECK-NOT: selected .rodata.d1
-# CHECK-NOT: selected .rodata.d2
+# CHECK-NOT: selected section '.rodata.d1'
+# CHECK-NOT: selected section '.rodata.d2'
 
 # We do merge rodata if passed --icf-data
 # RUN: ld.lld %t -o %t2 --icf=all --verbose --ignore-data-address-equality 2>&1 | FileCheck --check-prefix=DATA %s
 # RUN: llvm-readelf -S -W %t2 | FileCheck --check-prefix=DATA-SEC %s
 
-# DATA: selected .rodata.d1
-# DATA: removed .rodata.d2
+# DATA: selected section '.rodata.d1' from file [[T:'.*']]
+# DATA:   removing identical section '.rodata.d2' from file [[T]]
 
 # DATA-SEC:  .rodata      PROGBITS  0000000000200120 000120 000001 00 A 0 0 1
 
 .globl _start, d1, d2
 _start:
   ret
 
 .section .rodata.d1, "a"
 d1:
   .byte 1
 
 .section .rodata.d2, "a"
 d2:
   .byte 1

test/ELF/print-icf.s

+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/print-icf.s -o %t1
+# RUN: ld.lld %t %t1 -o %t2 --icf=all --print-icf-sections | FileCheck %s
+# RUN: ld.lld %t %t1 -o %t2 --icf=all --no-print-icf-sections --print-icf-sections | FileCheck %s
+# RUN: ld.lld %t %t1 -o %t2 --icf=all --print-icf-sections --no-print-icf-sections | FileCheck -allow-empty -check-prefix=PRINT %s
+
+# CHECK: selected section '.text.f2' from file [[T:'.*']]
+# CHECK:   removing identical section '.text.f4' from file [[T]]
+# CHECK:   removing identical section '.text.f7' from file [[T1:'.*']]
+# CHECK: selected section '.text.f1' from file [[T]]
+# CHECK:   removing identical section '.text.f3' from file [[T]]
+# CHECK:   removing identical section '.text.f5' from file [[T]]
+# CHECK:   removing identical section '.text.f6' from file [[T1]]
+
+# PRINT-NOT: selected
+# PRINT-NOT: removing
+
+.globl _start, f1, f2
+_start:
+  ret
+
+.section .text.f1, "ax"
+f1:
+  mov $60, %rax
+  mov $42, %rdi
+  syscall
+
+  .section .text.f2, "ax"
+f2:
+  mov $0, %rax
+
+.section .text.f3, "ax"
+f3:
+  mov $60, %rax
+  mov $42, %rdi
+  syscall
+
+.section .text.f4, "ax"
+f4:
+  mov $0, %rax
+
+.section .text.f5, "ax"
+f5:
+  mov $60, %rax
+  mov $42, %rdi
+  syscall