[ELF] Add convenience TableGen classes to enforce two dashes for options not supported by GNU ld

[MaskRay]

Summary

Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

For many options, we have to support either one or two dash to be
compatible with GNU ld. For newer and lld specific options, we can enforce strict double dashes.

Affected options:

• --thinlto-*
• --lto-*
• --shuffle-sections=

This patch does not change -plugin-opt=* because clang driver passes
-plugin-opt=* and I don't intend to cause churn.

In 2000, GNU ld tried something similar
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4897a3288f37d5f69e8acd256a6e83e607fe8d8

[tejohnson]

Since this is a lot of options that in some cases may have been used for some time (e.g. many the -thinlto-* ones have been supported for a few years now), it might be worth an announcement on llvm-dev so no one is surprised that their build scripts need updating.

lld/ELF/Options.td

542–543

What about this one, looks like Eq takes both dash forms.

[MaskRay]

In D79371#2020539, @tejohnson wrote:

Since this is a lot of options that in some cases may have been used for some time (e.g. many the -thinlto-* ones have been supported for a few years now), it might be worth an announcement on llvm-dev so no one is surprised that their build scripts need updating.

(I did not the history of -thinlto-* but I can't find them referenced, probably because -plugin-opt=thinlto-* is the more common use case as gold only accepts -plugin-opt=*)

Created https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

lld/ELF/Options.td

542–543

I left this one because I don't have a good name for it. EqEq?

[psmith]

I definitely think we should do this for all new options. I agree that -o magic ought to be changed as I can see that happening in practice. The chance that someone intending -t hinlto-index-only rather than -thinlto-index-only is much lower. I'll defer to the users of thin-lto whether they think this is important.

Will be worth mentioning in the releases notes for LLD 11.0.

[MaskRay]

Add EEq
Update ReleaseNotes.rst

[tejohnson]

LGTM. But you should add -omagic in the summary and release notes.

[MaskRay]

In D79371#2026499, @tejohnson wrote:

LGTM. But you should add -omagic in the summary and release notes.

--omagic is already correct.

def omagic: Flag<["--"], "omagic">, MetaVarName<"<magic>">,
  HelpText<"Set the text and data sections to be readable and writable, do not page align sections, link against static libraries">;

It was 2000's GNU ld which changed the behavior.

[psmith]

LGTM too. Thanks for the update.

[tejohnson]

In D79371#2026540, @MaskRay wrote:

In D79371#2026499, @tejohnson wrote:

LGTM. But you should add -omagic in the summary and release notes.

--omagic is already correct.

def omagic: Flag<["--"], "omagic">, MetaVarName<"<magic>">,
  HelpText<"Set the text and data sections to be readable and writable, do not page align sections, link against static libraries">;

It was 2000's GNU ld which changed the behavior.

Got it, missed the fact that this was a noop change to use the new convenience class.

lld/ELF/Options.td

 include "llvm/Option/OptParser.td"
 
+// Convenience classes for long options which only accept two dashes. For lld
+// specific or newer long options, we prefer two dashes to avoid collision with
+// short options. For many others, we have to accept both forms to be compatible
+// with GNU ld.
+class FF<string name> : Flag<["--"], name>;
+class JJ<string name>: Joined<["--"], name>;
+
+multiclass EEq<string name, string help> {
+  def NAME: Separate<["--"], name>;
+  def NAME # _eq: Joined<["--"], name # "=">, Alias<!cast<Separate>(NAME)>,
+    HelpText<help>;
+}
+
 // For options whose names are multiple letters, either one dash or
 // two can precede the option name except those that start with 'o'.
 class F<string name>: Flag<["--", "-"], name>;
 class J<string name>: Joined<["--", "-"], name>;
 
 multiclass Eq<string name, string help> {
   def NAME: Separate<["--", "-"], name>;
   def NAME # _eq: Joined<["--", "-"], name # "=">, Alias<!cast<Separate>(NAME)>,
def no_undefined: F<"no-undefined">,
   HelpText<"Report unresolved symbols even if the linker is creating a shared library">;
 
 def o: JoinedOrSeparate<["-"], "o">, MetaVarName<"<path>">,
   HelpText<"Path to file to write output">;
 
 def oformat: Separate<["--"], "oformat">, MetaVarName<"<format>">,
   HelpText<"Specify the binary format for the output object file">;
 
-def omagic: Flag<["--"], "omagic">, MetaVarName<"<magic>">,
+def omagic: FF<"omagic">, MetaVarName<"<magic>">,
   HelpText<"Set the text and data sections to be readable and writable, do not page align sections, link against static libraries">;
 
 defm orphan_handling:
   Eq<"orphan-handling", "Control how orphan sections are handled when linker script used">;
 
 defm pack_dyn_relocs:
   Eq<"pack-dyn-relocs", "Pack dynamic relocations in the given format">,
   MetaVarName<"[none,android,relr,android+relr]">;

 def: Flag<["-"], "S">, Alias<strip_debug>, HelpText<"Alias for --strip-debug">;
 def: Flag<["-"], "t">, Alias<trace>, HelpText<"Alias for --trace">;
 def: Joined<["-", "--"], "Ttext-segment=">, Alias<Ttext_segment>;
 def: JoinedOrSeparate<["-"], "y">, Alias<trace_symbol>, HelpText<"Alias for --trace-symbol">;
 def: JoinedOrSeparate<["-"], "u">, Alias<undefined>, HelpText<"Alias for --undefined">;
 def: Flag<["-"], "V">, Alias<version>, HelpText<"Alias for --version">;
 
 // LTO-related options.
-def lto_aa_pipeline: J<"lto-aa-pipeline=">,
+def lto_aa_pipeline: JJ<"lto-aa-pipeline=">,
   HelpText<"AA pipeline to run during LTO. Used in conjunction with -lto-newpm-passes">;
-def lto_debug_pass_manager: F<"lto-debug-pass-manager">,
+def lto_debug_pass_manager: FF<"lto-debug-pass-manager">,
   HelpText<"Debug new pass manager">;
-def lto_emit_asm: F<"lto-emit-asm">,
+def lto_emit_asm: FF<"lto-emit-asm">,
   HelpText<"Emit assembly code">;
-def lto_new_pass_manager: F<"lto-new-pass-manager">,
+def lto_new_pass_manager: FF<"lto-new-pass-manager">,
   HelpText<"Use new pass manager">;
-def lto_newpm_passes: J<"lto-newpm-passes=">,
+def lto_newpm_passes: JJ<"lto-newpm-passes=">,
   HelpText<"Passes to run during LTO">;
-def lto_O: J<"lto-O">, MetaVarName<"<opt-level>">,
+def lto_O: JJ<"lto-O">, MetaVarName<"<opt-level>">,
   HelpText<"Optimization level for LTO">;
-def lto_partitions: J<"lto-partitions=">,
+def lto_partitions: JJ<"lto-partitions=">,
   HelpText<"Number of LTO codegen partitions">;
-def lto_cs_profile_generate: F<"lto-cs-profile-generate">,
+def lto_cs_profile_generate: FF<"lto-cs-profile-generate">,
   HelpText<"Perform context sensitive PGO instrumentation">;
-def lto_cs_profile_file: J<"lto-cs-profile-file=">,
+def lto_cs_profile_file: JJ<"lto-cs-profile-file=">,
   HelpText<"Context sensitive profile file path">;
-def lto_obj_path_eq: J<"lto-obj-path=">;
-def lto_sample_profile: J<"lto-sample-profile=">,
+def lto_obj_path_eq: JJ<"lto-obj-path=">;
+def lto_sample_profile: JJ<"lto-sample-profile=">,
   HelpText<"Sample profile file path">;
-def lto_whole_program_visibility: F<"lto-whole-program-visibility">,
+def lto_whole_program_visibility: FF<"lto-whole-program-visibility">,
   HelpText<"Asserts that the LTO link has whole program visibility">;
 def disable_verify: F<"disable-verify">;
 defm mllvm: Eq<"mllvm", "Additional arguments to forward to LLVM's option processing">;
 def opt_remarks_filename: Separate<["--"], "opt-remarks-filename">,
   HelpText<"YAML output file for optimization remarks">;
 def opt_remarks_passes: Separate<["--"], "opt-remarks-passes">,
   HelpText<"Regex for the passes that need to be serialized to the output file">;
-def opt_remarks_with_hotness: Flag<["--"], "opt-remarks-with-hotness">,
+def opt_remarks_with_hotness: FF<"opt-remarks-with-hotness">,
   HelpText<"Include hotness information in the optimization remarks file">;
 def opt_remarks_format: Separate<["--"], "opt-remarks-format">,
   HelpText<"The format used for serializing remarks (default: YAML)">;
 def save_temps: F<"save-temps">;
-def lto_basicblock_sections: J<"lto-basicblock-sections=">,
+def lto_basicblock_sections: JJ<"lto-basicblock-sections=">,
   HelpText<"Enable basic block sections for LTO">;
 defm lto_unique_bb_section_names: B<"lto-unique-bb-section-names",
     "Give unique names to every basic block section for LTO",
     "Do not give unique names to every basic block section for LTO (default)">;
-def shuffle_sections: J<"shuffle-sections=">, MetaVarName<"<seed>">,
+def shuffle_sections: JJ<"shuffle-sections=">, MetaVarName<"<seed>">,
   HelpText<"Shuffle input sections using the given seed. If 0, use a random seed">;
-def thinlto_cache_dir: J<"thinlto-cache-dir=">,
+def thinlto_cache_dir: JJ<"thinlto-cache-dir=">,
   HelpText<"Path to ThinLTO cached object file directory">;
-defm thinlto_cache_policy: Eq<"thinlto-cache-policy", "Pruning policy for the ThinLTO cache">;
-def thinlto_emit_imports_files: F<"thinlto-emit-imports-files">;
-def thinlto_index_only: F<"thinlto-index-only">;
-def thinlto_index_only_eq: J<"thinlto-index-only=">;
-def thinlto_jobs: J<"thinlto-jobs=">,
+defm thinlto_cache_policy: EEq<"thinlto-cache-policy", "Pruning policy for the ThinLTO cache">;
+def thinlto_emit_imports_files: FF<"thinlto-emit-imports-files">;
+def thinlto_index_only: FF<"thinlto-index-only">;
+def thinlto_index_only_eq: JJ<"thinlto-index-only=">;
+def thinlto_jobs: JJ<"thinlto-jobs=">,
   HelpText<"Number of ThinLTO jobs. Default to --threads=">;
-def thinlto_object_suffix_replace_eq: J<"thinlto-object-suffix-replace=">;
-def thinlto_prefix_replace_eq: J<"thinlto-prefix-replace=">;
+def thinlto_object_suffix_replace_eq: JJ<"thinlto-object-suffix-replace=">;
+def thinlto_prefix_replace_eq: JJ<"thinlto-prefix-replace=">;
 
 def: J<"plugin-opt=O">, Alias<lto_O>, HelpText<"Alias for --lto-O">;
 def: F<"plugin-opt=debug-pass-manager">,
   Alias<lto_debug_pass_manager>, HelpText<"Alias for --lto-debug-pass-manager">;
 def: F<"plugin-opt=disable-verify">, Alias<disable_verify>, HelpText<"Alias for --disable-verify">;
 def plugin_opt_dwo_dir_eq: J<"plugin-opt=dwo_dir=">,
   HelpText<"Directory to store .dwo files when LTO and debug fission are used">;
 def plugin_opt_emit_asm: F<"plugin-opt=emit-asm">,

lld/docs/ReleaseNotes.rst

 Non-comprehensive list of changes in this release
 =================================================
 
 ELF Improvements
 ----------------
 
 * ...
 
+Breaking changes
+----------------
+
+* One-dash form of some long option (``--thinlto-*``, ``--lto-*``, ``--shuffle-sections=``)
+  are no longer supported.
+
 COFF Improvements
 -----------------
 
 * ...
 
 MinGW Improvements
 ------------------
 

lld/test/ELF/lto/devirt_vcall_vis_public.ll

 ; REQUIRES: x86
-; Test that -lto-whole-program-visibility enables devirtualization.
+; Test that --lto-whole-program-visibility enables devirtualization.
 
 ; Index based WPD
 ; Generate unsplit module with summary for ThinLTO index-based WPD.
-; RUN: opt -thinlto-bc -o %t2.o %s
-; RUN: ld.lld %t2.o -o %t3 -save-temps -lto-whole-program-visibility \
+; RUN: opt --thinlto-bc -o %t2.o %s
+; RUN: ld.lld %t2.o -o %t3 -save-temps --lto-whole-program-visibility \
 ; RUN:   -mllvm -pass-remarks=. --export-dynamic 2>&1 | FileCheck %s --check-prefix=REMARK
 ; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s --check-prefix=CHECK-IR
 
 ; Hybrid WPD
 ; Generate split module with summary for hybrid Thin/Regular LTO WPD.
-; RUN: opt -thinlto-bc -thinlto-split-lto-unit -o %t.o %s
-; RUN: ld.lld %t.o -o %t3 -save-temps -lto-whole-program-visibility \
+; RUN: opt --thinlto-bc --thinlto-split-lto-unit -o %t.o %s
+; RUN: ld.lld %t.o -o %t3 -save-temps --lto-whole-program-visibility \
 ; RUN:   -mllvm -pass-remarks=. --export-dynamic 2>&1 | FileCheck %s --check-prefix=REMARK
 ; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck %s --check-prefix=CHECK-IR
 
 ; Regular LTO WPD
 ; RUN: opt -o %t4.o %s
-; RUN: ld.lld %t4.o -o %t3 -save-temps -lto-whole-program-visibility \
+; RUN: ld.lld %t4.o -o %t3 -save-temps --lto-whole-program-visibility \
 ; RUN:   -mllvm -pass-remarks=. --export-dynamic 2>&1 | FileCheck %s --check-prefix=REMARK
 ; RUN: llvm-dis %t3.0.4.opt.bc -o - | FileCheck %s --check-prefix=CHECK-IR
 
 ; REMARK-DAG: single-impl: devirtualized a call to _ZN1A1nEi
 ; REMARK-DAG: single-impl: devirtualized a call to _ZN1D1mEi
 
 ; Try everything again but without -whole-program-visibility to confirm
 ; WPD fails

lld/test/ELF/lto/thinlto-emit-imports.ll

 ; RUN: rm -f %t1.o.imports
 ; RUN: rm -f %t2.o.imports
 ; RUN: rm -f %t3.o.imports
 ; RUN: ld.lld --plugin-opt=thinlto-emit-imports-files -shared %t1.o %t2.o %t3.o -o %t4
 ; RUN: not ls %t1.o.imports
 ; RUN: not ls %t2.o.imports
 ; RUN: not ls %t3.o.imports
 
-; Check that imports files are generated also when -thinlto-index-only
+; Check that imports files are generated also when --thinlto-index-only
 ; is specified without --plugin-opt=.
 ; RUN: rm -f %t1.o.imports
-; RUN: ld.lld -thinlto-index-only -thinlto-emit-imports-files -shared %t1.o %t2.o %t3.o -o %t4
+; RUN: ld.lld --thinlto-index-only --thinlto-emit-imports-files -shared %t1.o %t2.o %t3.o -o %t4
 ; RUN: count 1 < %t1.o.imports
 ; RUN: FileCheck %s --check-prefix=IMPORTS1 < %t1.o.imports
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 declare void @g(...)
 
 define void @f() {
 entry:
   call void (...) @g()
   ret void
 }

lld/test/ELF/lto/thinlto-index-file.ll

 ; REQUIRES: x86
 
 ; Basic ThinLTO tests.
 ; RUN: opt -module-summary %s -o %t1.o
 ; RUN: opt -module-summary %p/Inputs/thinlto.ll -o %t2.o
 ; RUN: opt -module-summary %p/Inputs/thinlto_empty.ll -o %t3.o
 
 ; Ensure lld writes linked files to linked objects file
 ; RUN: ld.lld --plugin-opt=thinlto-index-only=%t.idx -shared %t1.o %t2.o %t3.o -o /dev/null
 ; RUN: FileCheck %s < %t.idx
 ; CHECK: {{.*}}thinlto-index-file.ll.tmp1.o
 ; CHECK: {{.*}}thinlto-index-file.ll.tmp2.o
 ; CHECK: {{.*}}thinlto-index-file.ll.tmp3.o
 
 ; Check that this also works without the --plugin-opt= prefix.
-; RUN: ld.lld -thinlto-index-only=%t.idx -shared %t1.o %t2.o %t3.o -o /dev/null
+; RUN: ld.lld --thinlto-index-only=%t.idx -shared %t1.o %t2.o %t3.o -o /dev/null
 ; RUN: FileCheck %s < %t.idx
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 declare void @g(...)
 
 define void @f() {
 entry:
   call void (...) @g()
   ret void
 }

lld/test/ELF/lto/thinlto-obj-path.ll

 ; REQUIRES: x86
 
 ; RUN: opt -module-summary %s -o %t1.o
 ; RUN: opt -module-summary %p/Inputs/thinlto.ll -o %t2.o
 
 ; Test to ensure that thinlto-index-only with obj-path creates the file.
 ; RUN: rm -f %t4.o
 ; RUN: ld.lld --plugin-opt=thinlto-index-only --plugin-opt=obj-path=%t4.o -shared %t1.o %t2.o -o /dev/null
 ; RUN: llvm-readobj -h %t4.o | FileCheck %s
 ; RUN: llvm-nm %t4.o 2>&1 | count 0
 
 ; Check that this also works without the --plugin-opt= prefix.
 ; RUN: rm -f %t4.o
-; RUN: ld.lld -thinlto-index-only -lto-obj-path=%t4.o -shared %t1.o %t2.o -o /dev/null
+; RUN: ld.lld --thinlto-index-only --lto-obj-path=%t4.o -shared %t1.o %t2.o -o /dev/null
 ; RUN: llvm-readobj -h %t4.o | FileCheck %s
 
 ;; Ensure lld emits empty combined module if specific obj-path.
 ; RUN: rm -fr %t.dir/objpath && mkdir -p %t.dir/objpath
 ; RUN: ld.lld --plugin-opt=obj-path=%t4.o -shared %t1.o %t2.o -o %t.dir/objpath/a.out --save-temps
 ; RUN: ls %t.dir/objpath/a.out*.lto.* | count 3
 
 ;; Ensure lld does not emit empty combined module in default.

lld/test/ELF/lto/thinlto-object-suffix-replace.ll

 ; REQUIRES: x86
 
 ; Test to make sure the thinlto-object-suffix-replace option is handled
 ; correctly.
 
 ; Generate bitcode file with summary, as well as a minimized bitcode without
 ; the debug metadata for the thin link.
-; RUN: opt -thinlto-bc %s -thin-link-bitcode-file=%t1.thinlink.bc -o %t1.o
+; RUN: opt --thinlto-bc %s -thin-link-bitcode-file=%t1.thinlink.bc -o %t1.o
 
 ; First perform the thin link on the normal bitcode file, and save the
 ; resulting index.
 ; RUN: ld.lld --plugin-opt=thinlto-index-only -shared %t1.o -o %t3
 ; RUN: cp %t1.o.thinlto.bc %t1.o.thinlto.bc.orig
 
 ; Next perform the thin link on the minimized bitcode file, and compare dump
 ; of the resulting index to the above dump to ensure they are identical.
 ; RUN: rm -f %t1.o.thinlto.bc
 ; Make sure it isn't inadvertently using the regular bitcode file.
 ; RUN: rm -f %t1.o
 ; RUN: ld.lld --plugin-opt=thinlto-index-only \
 ; RUN: --plugin-opt=thinlto-object-suffix-replace=".thinlink.bc;.o" \
 ; RUN: -shared %t1.thinlink.bc -o %t3
 ; RUN: diff %t1.o.thinlto.bc.orig %t1.o.thinlto.bc
 ; Also check that this works without the --plugin-opt= prefix.
-; RUN: ld.lld -thinlto-index-only \
-; RUN: -thinlto-object-suffix-replace=".thinlink.bc;.o" \
+; RUN: ld.lld --thinlto-index-only \
+; RUN: --thinlto-object-suffix-replace=".thinlink.bc;.o" \
 ; RUN: -shared %t1.thinlink.bc -o %t3
 ; RUN: diff %t1.o.thinlto.bc.orig %t1.o.thinlto.bc
 
 ; Ensure lld generates error if object suffix replace option does not have 'old;new' format
 ; RUN: rm -f %t1.o.thinlto.bc
 ; RUN: not ld.lld --plugin-opt=thinlto-index-only \
 ; RUN: --plugin-opt=thinlto-object-suffix-replace="abc:def" -shared %t1.thinlink.bc \
 ; RUN: -o %t3 2>&1 | FileCheck %s --check-prefix=ERR1

lld/test/ELF/lto/thinlto-prefix-replace.ll

 ; REQUIRES: x86
 ; Check that changing the output path via thinlto-prefix-replace works
 ; RUN: mkdir -p %t/oldpath
 ; RUN: opt -module-summary %s -o %t/oldpath/thinlto_prefix_replace.o
 
 ; Ensure that there is no existing file at the new path, so we properly
 ; test the creation of the new file there.
 ; RUN: rm -f %t/newpath/thinlto_prefix_replace.o.thinlto.bc
 ; RUN: ld.lld --plugin-opt=thinlto-index-only --plugin-opt=thinlto-prefix-replace="%t/oldpath/;%t/newpath/" -shared %t/oldpath/thinlto_prefix_replace.o -o %t/thinlto_prefix_replace
 ; RUN: ls %t/newpath/thinlto_prefix_replace.o.thinlto.bc
 
 ; Check that this also works without the --plugin-opt= prefix.
 ; RUN: rm -f %t/newpath/thinlto_prefix_replace.o.thinlto.bc
-; RUN: ld.lld -thinlto-index-only -thinlto-prefix-replace="%t/oldpath/;%t/newpath/" -shared %t/oldpath/thinlto_prefix_replace.o -o %t/thinlto_prefix_replace
+; RUN: ld.lld --thinlto-index-only --thinlto-prefix-replace="%t/oldpath/;%t/newpath/" -shared %t/oldpath/thinlto_prefix_replace.o -o %t/thinlto_prefix_replace
 ; RUN: ls %t/newpath/thinlto_prefix_replace.o.thinlto.bc
 
 ; Ensure that lld generates error if prefix replace option does not have 'old;new' format
 ; RUN: rm -f %t/newpath/thinlto_prefix_replace.o.thinlto.bc
 ; RUN: not ld.lld --plugin-opt=thinlto-index-only --plugin-opt=thinlto-prefix-replace=abc:def -shared %t/oldpath/thinlto_prefix_replace.o -o /dev/null 2>&1 | FileCheck %s --check-prefix=ERR
 ; ERR: --plugin-opt=thinlto-prefix-replace= expects 'old;new' format, but got abc:def
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @f() {
 entry:
   ret void
 }