[llvm-ifs] llvm Interface Stubs merging + object file generation tool.

[plotfi]

Summary

This tool merges interface stub files to produce either a merged interface stubs file or an ELF/COFF binary or tbe.
Currently it can write a merged interface stubs file or a merger of all the symbols into an ELF .so file.

The new IFS format is as follows:

--- !experimental-ifs-v1
IfsVersion:      1.0
Triple:          <llvm triple>
ObjectFileFormat: <ELF | TBD>
Symbols:
  _ZSymbolName: { Type: <type> }
...

[MaskRay]

There is another tool tools/llvm-elfabi that does something very similar. (To be honest I don't quite get the point why that tool is required and why it has to be a llvm tool.) Nevertheless, it exists there, though not actively developed. Does this tool intend to replace/complement it?

[plotfi]

In D66405#1634787, @MaskRay wrote:

There is another tool tools/llvm-elfabi that does something very similar. (To be honest I don't quite get the point why that tool is required and why it has to be a llvm tool.) Nevertheless, it exists there, though not actively developed. Does this tool intend to replace/complement it?

There was extensive discussion on https://reviews.llvm.org/D60974. The main purpose for llvm-ifs is because llvm-elfapi primarily deals with elf, and because we specifically do not want the mergeable interface stubs format to be the tbe format. Ideally, when the elfabi support for generating ELF binaries is finally landed, llvm-ifs will simply generate tbe in memory to product the elf. Ideally we want something that can generate ELF or COFF etc, and this is outside of the scope of llvm-elfabi.

[compnerd]

This definitely needs tests.

llvm/tools/llvm-ifs/llvm-ifs.cpp

54

Hmm, put this into an anonymous namespace and hoist it to the top of the file?

87

Braces are unnecessary.

101

Use a local variable for the Key.str(). You are creating two std::string here otherwise.

126

SOName is better - its an initialism for Shared Object

261

Probably should diagnose the actual mismatch so that its obvious why it doesn't match.

266

What about conflicts?

[plotfi]

• Addressing @compnerd's feedback, still need to do some smarter conflict resolution for symbols and add tests.
• Added rudimentary TBD support.

[plotfi]

llvm/tools/llvm-ifs/llvm-ifs.cpp

208

TODO: This is weird, I know. ELFSymbols for TBDs. I intend to fork away from ELFSymbol sometime soon.

[plotfi]

llvm/tools/llvm-ifs/LLVMBuild.txt

22

TODO: Add ObjectYAML

[plotfi]

Adding some tests. and added some really basic symbol conflict error checking (no resolution though).

[plotfi]

adding another test, to merge objects and funcs of different sizes.

[compnerd]

Probably is worth having a test for the error case as well.

llvm/test/tools/llvm-ifs/a.ifs

22 ↗

(On Diff #217269)

Why are they undefined?

llvm/test/tools/llvm-ifs/b.ifs

20 ↗

(On Diff #217269)

Why undefined?

llvm/test/tools/llvm-ifs/bd.ifs

1 ↗

(On Diff #217269)

Not sure what the value is of this test. You already tested 1 file, and 1+ files.

llvm/test/tools/llvm-ifs/c.ifs

1 ↗

(On Diff #217269)

Same.

llvm/test/tools/llvm-ifs/class-template-specialization.ifs

2

Okay, I suppose that this is C++. At the very least, emit a TBD to ensure that the user label prefix is handled correctly.

llvm/test/tools/llvm-ifs/ef.ifs

28 ↗

(On Diff #217269)

Why undefined

llvm/test/tools/llvm-ifs/object.ifs

2

Objects were already tested previously.

llvm/test/tools/llvm-ifs/template-namespace-function.ifs

1 ↗

(On Diff #217269)

Not sure what this is testing.

llvm/test/tools/llvm-ifs/virtual.ifs

1 ↗

(On Diff #217269)

Not sure what the virtuality testing is actually testing.

llvm/test/tools/llvm-ifs/visibility.ifs

1 ↗

(On Diff #217269)

Not sure if this is valuable.

[compnerd]

I suppose that the test cases that I think are valuable:

• 1 file
• 1+ file
• functions
• objects
• functions w/ different alignments
• objects w/ different alignments
• functions w/ different sizes
• objects w/ different sizes
• functions w/ different alignments, sizes
• objects w/ different alignments, sizes
• weak symbols
• mismatch (error cases)

[plotfi]

test updates based on feedback

[plotfi]

llvm/test/tools/llvm-ifs/ef.ifs

28 ↗

(On Diff #217269)

Undefined because I do not define a .text section for the symbol to belong to. I could do something like this to name the undefined go away:

--- !ELF
FileHeader:
  Class:           ELFCLASS64
  Data:            ELFDATA2LSB
  Type:            ET_REL
  Machine:         EM_X86_64
Sections:
  - Name:            .text
    Type:            SHT_PROGBITS
Symbols:
  Global:
    - Name:            _Z3foov
      Type:            STT_FUNC
      Section:         .text
...

[plotfi]

llvm/test/tools/llvm-ifs/class-template-specialization.ifs

2

I think in the first version, TBD is experimental. Ideally any IFS generation for Darwin TBDs would have come from a .*-apple-.* triple of some sort where clang generated the correct name manglings. Otherwise, there'd need to be some kind of demangler-remangler at play (which also wouldn't work as we've discussed for win32 mangling).

[plotfi]

Added some better conflict checking tests.

[plotfi]

Checking for conflicts in the Ifs header.

[plotfi]

More version checks

[compnerd]

I think it may make some sense to strip the user label prefix from the symbols that are listed in the interface, and then add that in when generating the ELF/TBD.

llvm/test/tools/llvm-ifs/a-conflict-header-version.ifs

7 ↗

(On Diff #217497)

Does it matter if the version is mismatched between the two versions as long as you can deserialise and merge?

llvm/test/tools/llvm-ifs/a-conflict-type.ifs

13 ↗

(On Diff #217497)

Missing space before the }.

llvm/test/tools/llvm-ifs/a-conflict-undefined.ifs

6 ↗

(On Diff #217497)

Why do we even need undefined symbols for the interfaces? Interfaces should be fully defined no?

llvm/test/tools/llvm-ifs/a-conflict-weak.ifs

7 ↗

(On Diff #217497)

mmm...I think that we should verify that this is appropriate behaviour. I think that the linker will normally accept the weak definition and replace it with the strong definition and we should just upgrade to the strong definition.

llvm/test/tools/llvm-ifs/b-conflict-size.ifs

13 ↗

(On Diff #217497)

Missing space before }.

[plotfi]

• Dropping vestigial ELFSymbols, now I have IFSSymbol.
• addressed @compnerd 's feedback

[plotfi]

llvm/test/tools/llvm-ifs/a-conflict-header-version.ifs

7 ↗

(On Diff #217497)

I think what i'll do is, if the major versions are different then error out but if the minors are different then resolve any difference.

llvm/test/tools/llvm-ifs/a-conflict-type.ifs

13 ↗

(On Diff #217497)

Aye

llvm/test/tools/llvm-ifs/a-conflict-weak.ifs

7 ↗

(On Diff #217497)

Can I add this resolution in a followup? it was my intention to do the more complicated resolution in a followup just so the patches stay simpler

[plotfi]

adding .data and .rodata to elf sections

[compnerd]

Please rename the tests to be more meaningful so that others can possibly identify what they are meant to test and can address things which may break when other changes are made.

llvm/test/tools/llvm-ifs/a-conflict-type.ifs

6 ↗

(On Diff #217928)

Should we diagnose the types which are conflicting?

llvm/test/tools/llvm-ifs/a-conflict-weak.ifs

7 ↗

(On Diff #217928)

The diagnostic is difficult to understand. Perhaps make this similar to the diagnostic for the mismatched types, and indicate that the failure is that the symbol is weak.

llvm/tools/llvm-ifs/llvm-ifs.cpp

41

Why no long name?

45

Same

53

Same

[plotfi]

better diag, better test names

llvm/test/tools/llvm-ifs/a-conflict-type.ifs

6 ↗

(On Diff #217928)

yes.

[compnerd]

llvm/test/tools/llvm-ifs/class-template-specialization.ifs

1

Is there anything special about C++ template specialization for the merge? I couldn't spot any. Perhaps this should be called c++/cxx or some such?

[plotfi]

llvm/test/tools/llvm-ifs/class-template-specialization.ifs

1

Hmm, actually I think the weak test might be sufficient to test something that actually came out of clang. I could drop this one.

[plotfi]

longer flags, no more =

[thakis]

llvm/trunk/test/lit.cfg.py

25 ↗

(On Diff #218138)

Since this list doesn't contain ".ifs", none of the tests you added actually run as part of check-llvm. I'd recommend renaming the tests to ".test" instead of adding ".ifs" to this list.

[thakis]

llvm/trunk/test/lit.cfg.py

25 ↗

(On Diff #218138)

A possibly even better fix is to add test/tools/llvm-ifs/lit.local.cfg that does config.suffixes = ['.ifs']. See e.g. llvm/test//TableGen/lit.local.cfg for an example.

[plotfi]

llvm/trunk/test/lit.cfg.py

25 ↗

(On Diff #218138)

Thank you for this heads up Nico. I'll land something shortly. I like the config.suffixes approach.

[cishida]

llvm/trunk/tools/llvm-ifs/llvm-ifs.cpp

238 ↗

(On Diff #218138)

I apologize for commenting on this retroactively, but if this hardcoded value is related to the experimental support for TBD files, I think there should be a comment detailing it.

[plotfi]

llvm/trunk/tools/llvm-ifs/llvm-ifs.cpp

238 ↗

(On Diff #218138)

Ah yeah, thanks for bringing this up @cishida. I’ll try and at the very least add some more comments on this soon. For now I’m still working on wiring up the clang driver and ifs generation to llvm-ifs.

[nickdesaulniers]

llvm/trunk/test/tools/llvm-ifs/conflict-size.ifs

15 ↗

(On Diff #218138)

Aplogies for asking on this patch (and very late after it was submitted), but playing with llvm-ifs, I see the following error running llvm-ifs against my copy of glibc.

$ llvm-ifs /lib/x86_64-linux-gnu/libc.so.6  --output-format=ELF -o -      
error: Interface Stub: Size Mismatch for sys_sigabbrev.
Filename: /lib/x86_64-linux-gnu/libc.so.6
Size Values: 512 520

$ llvm-readelf -s /lib/x86_64-linux-gnu/libc.so.6| grep sys_sigabbrev
   881: 00000000001e9d00   512 OBJECT  GLOBAL DEFAULT    27 sys_sigabbrev@GLIBC_2.2.5
   882: 00000000001e9d00   520 OBJECT  GLOBAL DEFAULT    27 sys_sigabbrev@@GLIBC_2.3.3

So it looks like glibc has 2 different sizes of sys_sigabbrev, but the symbols seem versioned, IIUC? ie. one is sys_sigabbrev@GLIBC_2.2.5 vs sys_sigabbrev@@GLIBC_2.3.3.

Is there an option to pick one, or perhaps both without removing the symbol versioning?

llvm/test/CMakeLists.txt

           llvm-exegesis
           llvm-extract
           llvm-isel-fuzzer
+          llvm-ifs
           llvm-jitlink
           llvm-lib
           llvm-link

llvm/test/lit.cfg.py

     'dsymutil', 'lli', 'lli-child-target', 'llvm-ar', 'llvm-as',
     'llvm-bcanalyzer', 'llvm-config', 'llvm-cov', 'llvm-cxxdump', 'llvm-cvtres',
     'llvm-diff', 'llvm-dis', 'llvm-dwarfdump', 'llvm-exegesis', 'llvm-extract',
-    'llvm-isel-fuzzer', 'llvm-jitlink', 'llvm-opt-fuzzer', 'llvm-lib',
+    'llvm-isel-fuzzer', 'llvm-ifs', 'llvm-jitlink', 'llvm-opt-fuzzer', 'llvm-lib',
     'llvm-link', 'llvm-lto', 'llvm-lto2', 'llvm-mc', 'llvm-mca',
     'llvm-modextract', 'llvm-nm', 'llvm-objcopy', 'llvm-objdump',
     'llvm-pdbutil', 'llvm-profdata', 'llvm-ranlib', 'llvm-rc', 'llvm-readelf',

llvm/test/tools/llvm-ifs/class-template-specialization.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s | FileCheck %s --check-prefixes=CHECK-IFS
+
+# CHECK-IFS:      --- !experimental-ifs-v1
+# CHECK-IFS-NEXT: IfsVersion: 1.0
+# CHECK-IFS-NEXT: Triple: x86_64-unknown-linux-gnu
+# CHECK-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-IFS-NEXT: Symbols:
+# CHECK-IFS-NEXT:   _Z1gv: { Type: Func }
+# CHECK-IFS-NEXT: ...
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  _Z1gv: { Type: Func }
+...

llvm/test/tools/llvm-ifs/conflict-header-format.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/object.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# CHECK-IFS: error: Interface Stub: ObjectFileFormat Mismatch.
+# CHECK-IFS-NEXT: Filenames:
+# CHECK-IFS-NEXT: ObjectFileFormat Values: TBD ELF
+
+--- !experimental-ifs-v1
+IfsVersion:      1.0
+Triple:          x86_64-apple-unknown
+ObjectFileFormat: TBD
+Symbols:
+  a:               { Type: Func }
+...

llvm/test/tools/llvm-ifs/conflict-header-triple.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/object.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# CHECK-IFS: error: Interface Stub: Triple Mismatch.
+# CHECK-IFS-NEXT: Filenames:
+# CHECK-IFS-NEXT: Triple Values: mips-unknown-linux x86_64-unknown-linux-gnu
+
+--- !experimental-ifs-v1
+IfsVersion:      1.0
+Triple:          mips-unknown-linux
+ObjectFileFormat: ELF
+Symbols:
+  a:               { Type: Func }
+...

llvm/test/tools/llvm-ifs/conflict-header-version.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/object.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# RUN: not llvm-ifs -a=write-ifs -o - %s 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS2
+
+# CHECK-IFS: error: Interface Stub: IfsVersion Mismatch.
+# CHECK-IFS2: error: Interface Stub: Bad IfsVersion: 0.0, llvm-ifs supported version: 1.2.
+
+--- !experimental-ifs-v1
+IfsVersion:      0.0
+Triple:          x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  a:               { Type: Func }
+...

llvm/test/tools/llvm-ifs/conflict-size.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/object.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# Here we are testing to see if two symbols with identical names will fail to
+# merge in conflict due to mismatching sizes.
+# CHECK-IFS: error: Interface Stub: Size Mismatch for b.
+# CHECK-IFS-NEXT: Filename:
+# CHECK-IFS-NEXT: Size Values: 1 4
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  b: { Type: Object, Size: 1 }
+...

llvm/test/tools/llvm-ifs/conflict-type.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/func.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# Here we are testing to see if two symbols with identical names will fail to
+# merge in conflict due to mismatched types.
+# CHECK-IFS: error: Interface Stub: Type Mismatch for a.
+# CHECK-IFS-NEXT: Filename:
+# CHECK-IFS-NEXT: Type Values: Object Func
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  a: { Type: Object, Size: 1 }
+...

llvm/test/tools/llvm-ifs/conflict-weak.ifs

+# RUN: not llvm-ifs -a=write-ifs -o - %s %S/func.ifs 2>&1 | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# Here we are testing to see if two symbols with identical names will fail to
+# merge in conflict due to one being weak and one not. Eventually this will work
+# when llvm-ifs has support for resolving these kinds of conflicts.
+# CHECK-IFS: error: Interface Stub: Weak Mismatch for a.
+# CHECK-IFS-NEXT: Filename:
+# CHECK-IFS-NEXT: Weak Values: 1 0
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  a: { Type: Func, Weak: true }
+...

llvm/test/tools/llvm-ifs/func.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s %S/object.ifs | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# RUN: llvm-ifs -a=write-bin -o - %s %S/object.ifs | \
+# RUN: llvm-readelf --all | FileCheck %s --check-prefixes=CHECK-ELF
+
+# RUN: llvm-ifs -a=write-bin -f=TBD -o - %s %S/object.ifs | \
+# RUN: FileCheck %s --check-prefixes=CHECK-DARWIN-TBD3
+
+# RUN: llvm-ifs -a=write-ifs -o - %s %s | \
+# RUN: FileCheck %s --check-prefixes=CHECK-MERGE-IFS
+
+# CHECK-IFS: --- !experimental-ifs-v1
+# CHECK-IFS-NEXT: IfsVersion:      1.0
+# CHECK-IFS-NEXT: Triple:          x86_64-unknown-linux-gnu
+# CHECK-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-IFS-NEXT: Symbols:
+# CHECK-IFS-DAG:   a:               { Type: Func }
+# CHECK-IFS-DAG:   b:               { Type: Object, Size: 4 }
+# CHECK-IFS: ...
+
+# CHECK-ELF: ELF Header:
+# CHECK-ELF:   Class:                             ELF64
+# CHECK-ELF:   Type:                              DYN (Shared object file)
+# CHECK-ELF:   FUNC    GLOBAL DEFAULT  1 a
+# CHECK-ELF:   OBJECT  GLOBAL DEFAULT  1 b
+
+# CHECK-DARWIN-TBD3: --- !tapi-tbd-v3
+# CHECK-DARWIN-TBD3-NEXT: archs:           [ x86_64 ]
+# CHECK-DARWIN-TBD3-NEXT: platform:        macosx
+# CHECK-DARWIN-TBD3-NEXT: flags:           [ flat_namespace, not_app_extension_safe ]
+# CHECK-DARWIN-TBD3-NEXT: install-name:    ''
+# CHECK-DARWIN-TBD3-NEXT: current-version: 0
+# CHECK-DARWIN-TBD3-NEXT: compatibility-version: 0
+# CHECK-DARWIN-TBD3-NEXT: objc-constraint: none
+# CHECK-DARWIN-TBD3-NEXT: exports:
+# CHECK-DARWIN-TBD3-NEXT:   - archs:           [ x86_64 ]
+# CHECK-DARWIN-TBD3-NEXT:     symbols:         [ a, b ]
+# CHECK-DARWIN-TBD3-NEXT: ...
+
+# Here we are testing to see if two identical symbols will merge.
+# CHECK-MERGE-IFS: --- !experimental-ifs-v1
+# CHECK-MERGE-IFS-NEXT: IfsVersion:      1.0
+# CHECK-MERGE-IFS-NEXT: Triple:          x86_64-unknown-linux-gnu
+# CHECK-MERGE-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-MERGE-IFS-NEXT: Symbols:
+# CHECK-MERGE-IFS-NEXT:   a:               { Type: Func }
+# CHECK-MERGE-IFS-NEXT: ...
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  a: { Type: Func }
+...

llvm/test/tools/llvm-ifs/object-function-size-weak-combo.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s %S/func.ifs %S/object.ifs %S/weak.ifs | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# RUN: llvm-ifs -a=write-bin -o - %s %S/func.ifs %S/object.ifs %S/weak.ifs | \
+# RUN: llvm-readelf --all | FileCheck %s --check-prefixes=CHECK-ELF
+
+# CHECK-IFS: --- !experimental-ifs-v1
+# CHECK-IFS-NEXT: IfsVersion:      1.0
+# CHECK-IFS-NEXT: Triple:          x86_64-unknown-linux-gnu
+# CHECK-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-IFS-NEXT: Symbols:
+# CHECK-IFS-DAG:   e:               { Type: Object, Size: 8 }
+# CHECK-IFS-DAG:   a:               { Type: Func }
+# CHECK-IFS-DAG:   f:               { Type: Object, Size: 2 }
+# CHECK-IFS-DAG:   _Z10strongFuncv: { Type: Func }
+# CHECK-IFS-DAG:   _Z8weakFuncv:    { Type: Func, Weak: true }
+# CHECK-IFS-DAG:   b:               { Type: Object, Size: 4 }
+# CHECK-IFS: ...
+
+# CHECK-ELF: FUNC    GLOBAL DEFAULT  1 _Z10strongFuncv
+# CHECK-ELF: FUNC    WEAK   DEFAULT  1 _Z8weakFuncv
+# CHECK-ELF: FUNC    GLOBAL DEFAULT  1 a
+# CHECK-ELF: OBJECT  GLOBAL DEFAULT  1 b
+# CHECK-ELF: OBJECT  GLOBAL DEFAULT  1 e
+# CHECK-ELF: OBJECT  GLOBAL DEFAULT  1 f
+
+--- !experimental-ifs-v1
+IfsVersion:      1.0
+Triple:          x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  e:               { Type: Object, Size: 8 }
+  f:               { Type: Object, Size: 2 }
+...

llvm/test/tools/llvm-ifs/object.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s | \
+# RUN: FileCheck %s --check-prefixes=CHECK-IFS
+
+# RUN: llvm-ifs -a=write-bin -o - %s | \
+# RUN: llvm-readelf --all | FileCheck %s --check-prefixes=CHECK-ELF
+
+# CHECK-IFS: --- !experimental-ifs-v1
+# CHECK-IFS-NEXT: IfsVersion:      1.0
+# CHECK-IFS-NEXT: Triple:          x86_64-unknown-linux-gnu
+# CHECK-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-IFS-NEXT: Symbols:
+# CHECK-IFS-NEXT:   b:               { Type: Object, Size: 4 }
+# CHECK-IFS-NEXT: ...
+
+# CHECK-ELF: ELF Header:
+# CHECK-ELF:   Class:                             ELF64
+# CHECK-ELF:   Data:                              2's complement, little endian
+# CHECK-ELF:   Type:                              DYN (Shared object file)
+# CHECK-ELF-NOT:   FUNC    GLOBAL DEFAULT  1 a
+# CHECK-ELF:   OBJECT  GLOBAL DEFAULT  1 b
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  b: { Type: Object, Size: 4 }
+...

llvm/test/tools/llvm-ifs/version-ok.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s %S/object.ifs
+
+--- !experimental-ifs-v1
+IfsVersion:      1.1
+Triple:          x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  a:               { Type: Func }
+...

llvm/test/tools/llvm-ifs/weak.ifs

+# RUN: llvm-ifs -a=write-ifs -o - %s | FileCheck %s --check-prefixes=CHECK-IFS
+
+# CHECK-IFS: --- !experimental-ifs-v1
+# CHECK-IFS-NEXT: IfsVersion: 1.0
+# CHECK-IFS-NEXT: Triple: x86_64-unknown-linux-gnu
+# CHECK-IFS-NEXT: ObjectFileFormat: ELF
+# CHECK-IFS-NEXT: Symbols:
+# CHECK-IFS-DAG:   _Z8weakFuncv: { Type: Func, Weak: true }
+# CHECK-IFS-DAG:   _Z10strongFuncv: { Type: Func }
+# CHECK-IFS: ...
+
+--- !experimental-ifs-v1
+IfsVersion: 1.0
+Triple: x86_64-unknown-linux-gnu
+ObjectFileFormat: ELF
+Symbols:
+  _Z8weakFuncv: { Type: Func, Weak: true }
+  _Z10strongFuncv: { Type: Func }
+...

llvm/tools/LLVMBuild.txt

  llvm-dwarfdump
  llvm-dwp
  llvm-elfabi
+ llvm-ifs
  llvm-exegesis
  llvm-extract
  llvm-jitlistener

llvm/tools/llvm-ifs/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Object
+  Support
+  TextAPI
+  ObjectYAML
+  )
+
+add_llvm_tool(llvm-ifs
+  llvm-ifs.cpp
+  )

llvm/tools/llvm-ifs/LLVMBuild.txt

+;===- ./tools/llvm-ifs/LLVMBuild.txt ---------------------------*- Conf -*--===;
+;
+; Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+; See https://llvm.org/LICENSE.txt for license information.
+; SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+;
+;===------------------------------------------------------------------------===;
+;
+; This is an LLVMBuild description file for the components in this subdirectory.
+;
+; For more information on the LLVMBuild system, please see:
+;
+;   http://llvm.org/docs/LLVMBuild.html
+;
+;===------------------------------------------------------------------------===;
+
+[component_0]
+type = Tool
+name = llvm-ifs
+parent = Tools
+required_libraries = Object Support TextAPI

llvm/tools/llvm-ifs/llvm-ifs.cpp

+//===- llvm-ifs.cpp -------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===-----------------------------------------------------------------------===/
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/ADT/Triple.h"
+#include "llvm/ObjectYAML/yaml2obj.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Support/Errc.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/VersionTuple.h"
+#include "llvm/Support/WithColor.h"
+#include "llvm/Support/YAMLTraits.h"
+#include "llvm/Support/raw_ostream.h"
+#include "llvm/TextAPI/MachO/InterfaceFile.h"
+#include "llvm/TextAPI/MachO/TextAPIReader.h"
+#include "llvm/TextAPI/MachO/TextAPIWriter.h"
+#include <set>
+#include <string>
+
+using namespace llvm;
+using namespace llvm::yaml;
+using namespace llvm::MachO;
+
+#define DEBUG_TYPE "llvm-ifs"
+
+namespace {
+const VersionTuple IFSVersionCurrent(1, 2);
+};
+
+static cl::opt<std::string> Action("a", cl::desc("<llvm-ifs action>"),
+                                   cl::value_desc("write-ifs | write-bin"),
+                                   cl::init("write-ifs"));
+
+static cl::opt<std::string> ForceFormat("f", cl::desc("<force object format>"),
+                                        cl::value_desc("ELF | TBD"),
+                                        cl::init(""));
+
+static cl::list<std::string> InputFilenames(cl::Positional,
+                                            cl::desc("<input ifs files>"),
+                                            cl::ZeroOrMore);
+
+static cl::opt<std::string> OutputFilename("o", cl::desc("<output file>"),
+                                           cl::value_desc("path"));
+
+enum class IFSSymbolType {
+  NoType = 0,
+  Object,
+  Func,
+  // Type information is 4 bits, so 16 is safely out of range.
+  Unknown = 16,
+};
+
+std::string getTypeName(IFSSymbolType Type) {
+  switch (Type) {
+  case IFSSymbolType::NoType:
+    return "NoType";
+  case IFSSymbolType::Func:
+    return "Func";
+  case IFSSymbolType::Object:
+    return "Object";
+  case IFSSymbolType::Unknown:
+    return "Unknown";
+  }
+}
+
+struct IFSSymbol {
+  IFSSymbol(std::string SymbolName) : Name(SymbolName) {}
+  std::string Name;
+  uint64_t Size;
+  IFSSymbolType Type;
+  bool Weak;
+  Optional<std::string> Warning;
+  bool operator<(const IFSSymbol &RHS) const { return Name < RHS.Name; }
+};
+
+/// YAML traits for IFSSymbolType.
+template <> struct ScalarEnumerationTraits<IFSSymbolType> {
+  static void enumeration(IO &IO, IFSSymbolType &SymbolType) {
+    IO.enumCase(SymbolType, "NoType", IFSSymbolType::NoType);
+    IO.enumCase(SymbolType, "Func", IFSSymbolType::Func);
+    IO.enumCase(SymbolType, "Object", IFSSymbolType::Object);
+    IO.enumCase(SymbolType, "Unknown", IFSSymbolType::Unknown);
+    // Treat other symbol types as noise, and map to Unknown.
+    if (!IO.outputting() && IO.matchEnumFallback())
+      SymbolType = IFSSymbolType::Unknown;
+  }
+};
+
+template <> struct ScalarTraits<VersionTuple> {
+  static void output(const VersionTuple &Value, void *,
+                     llvm::raw_ostream &Out) {
+    Out << Value.getAsString();
+  }
+
+  static StringRef input(StringRef Scalar, void *, VersionTuple &Value) {
+    if (Value.tryParse(Scalar))
+      return StringRef("Can't parse version: invalid version format.");
+
+    if (Value > IFSVersionCurrent)
+      return StringRef("Unsupported IFS version.");
+
+    // Returning empty StringRef indicates successful parse.
+    return StringRef();
+  }
+
+  // Don't place quotation marks around version value.
+  static QuotingType mustQuote(StringRef) { return QuotingType::None; }
+};
+
+/// YAML traits for IFSSymbol.
+template <> struct MappingTraits<IFSSymbol> {
+  static void mapping(IO &IO, IFSSymbol &Symbol) {
+    IO.mapRequired("Type", Symbol.Type);
+    // The need for symbol size depends on the symbol type.
+    if (Symbol.Type == IFSSymbolType::NoType)
+      IO.mapOptional("Size", Symbol.Size, (uint64_t)0);
+    else if (Symbol.Type == IFSSymbolType::Func)
+      Symbol.Size = 0;
+    else
+      IO.mapRequired("Size", Symbol.Size);
+    IO.mapOptional("Weak", Symbol.Weak, false);
+    IO.mapOptional("Warning", Symbol.Warning);
+  }
+
+  // Compacts symbol information into a single line.
+  static const bool flow = true;
+};
+
+/// YAML traits for set of IFSSymbols.
+template <> struct CustomMappingTraits<std::set<IFSSymbol>> {
+  static void inputOne(IO &IO, StringRef Key, std::set<IFSSymbol> &Set) {
+    std::string Name = Key.str();
+    IFSSymbol Sym(Name);
+    IO.mapRequired(Name.c_str(), Sym);
+    Set.insert(Sym);
+  }
+
+  static void output(IO &IO, std::set<IFSSymbol> &Set) {
+    for (auto &Sym : Set)
+      IO.mapRequired(Sym.Name.c_str(), const_cast<IFSSymbol &>(Sym));
+  }
+};
+
+// A cumulative representation of ELF stubs.
+// Both textual and binary stubs will read into and write from this object.
+class IFSStub {
+  // TODO: Add support for symbol versioning.
+public:
+  VersionTuple IfsVersion;
+  std::string Triple;
+  std::string ObjectFileFormat;
+  Optional<std::string> SOName;
+  std::vector<std::string> NeededLibs;
+  std::set<IFSSymbol> Symbols;
+
+  IFSStub() = default;
+  IFSStub(const IFSStub &Stub)
+      : IfsVersion(Stub.IfsVersion), Triple(Stub.Triple),
+        ObjectFileFormat(Stub.ObjectFileFormat), SOName(Stub.SOName),
+        NeededLibs(Stub.NeededLibs), Symbols(Stub.Symbols) {}
+  IFSStub(IFSStub &&Stub)
+      : IfsVersion(std::move(Stub.IfsVersion)), Triple(std::move(Stub.Triple)),
+        ObjectFileFormat(std::move(Stub.ObjectFileFormat)),
+        SOName(std::move(Stub.SOName)), NeededLibs(std::move(Stub.NeededLibs)),
+        Symbols(std::move(Stub.Symbols)) {}
+};
+
+/// YAML traits for IFSStub objects.
+template <> struct MappingTraits<IFSStub> {
+  static void mapping(IO &IO, IFSStub &Stub) {
+    if (!IO.mapTag("!experimental-ifs-v1", true))
+      IO.setError("Not a .ifs YAML file.");
+    IO.mapRequired("IfsVersion", Stub.IfsVersion);
+    IO.mapOptional("Triple", Stub.Triple);
+    IO.mapOptional("ObjectFileFormat", Stub.ObjectFileFormat);
+    IO.mapOptional("SOName", Stub.SOName);
+    IO.mapOptional("NeededLibs", Stub.NeededLibs);
+    IO.mapRequired("Symbols", Stub.Symbols);
+  }
+};
+
+static Expected<std::unique_ptr<IFSStub>> readInputFile(StringRef FilePath) {
+  // Read in file.
+  ErrorOr<std::unique_ptr<MemoryBuffer>> BufOrError =
+      MemoryBuffer::getFileOrSTDIN(FilePath);
+  if (!BufOrError)
+    return createStringError(BufOrError.getError(), "Could not open `%s`",
+                             FilePath.data());
+
+  std::unique_ptr<MemoryBuffer> FileReadBuffer = std::move(*BufOrError);
+  yaml::Input YamlIn(FileReadBuffer->getBuffer());
+  std::unique_ptr<IFSStub> Stub(new IFSStub());
+  YamlIn >> *Stub;
+
+  if (std::error_code Err = YamlIn.error())
+    return createStringError(Err, "Failed reading Interface Stub File.");
+
+  return std::move(Stub);
+}
+
+int writeTbdStub(const llvm::Triple &T, const std::set<IFSSymbol> &Symbols,
+                 const StringRef Format, raw_ostream &Out) {
+  auto ArchOrError =
+      [](const llvm::Triple &T) -> llvm::Expected<llvm::MachO::Architecture> {
+    switch (T.getArch()) {
+    default:
+      return createStringError(errc::not_supported, "Invalid Architecture.");
+    case llvm::Triple::ArchType::x86:
+      return AK_i386;
+    case llvm::Triple::ArchType::x86_64:
+      return AK_x86_64;
+    case llvm::Triple::ArchType::arm:
+      return AK_armv7;
+    case llvm::Triple::ArchType::aarch64:
+      return AK_arm64;
+    }
+  }(T);
+
+  if (!ArchOrError)
+    return -1;
+
+  Architecture Arch = ArchOrError.get();
+
+  InterfaceFile File;
+  File.setFileType(FileType::TBD_V3);
+  File.setArchitectures(Arch);
+  File.setPlatform(PlatformKind::macOS);
+
+  for (const auto &Symbol : Symbols) {
+    auto Name = Symbol.Name;
+    auto Kind = SymbolKind::GlobalSymbol;
+    switch (Symbol.Type) {
+    default:
+    case IFSSymbolType::NoType:
+      Kind = SymbolKind::GlobalSymbol;
+      break;
+    case IFSSymbolType::Object:
+      Kind = SymbolKind::GlobalSymbol;
+      break;
+    case IFSSymbolType::Func:
+      Kind = SymbolKind::GlobalSymbol;
+      break;
+    }
+    if (Symbol.Weak)
+      File.addSymbol(Kind, Name, Arch, SymbolFlags::WeakDefined);
+    else
+      File.addSymbol(Kind, Name, Arch);
+  }
+
+  SmallString<4096> Buffer;
+  raw_svector_ostream OS(Buffer);
+  if (Error Result = TextAPIWriter::writeToStream(OS, File))
+    return -1;
+  Out << OS.str();
+  return 0;
+}
+
+int writeElfStub(const llvm::Triple &T, const std::set<IFSSymbol> &Symbols,
+                 const StringRef Format, raw_ostream &Out) {
+  SmallString<0> Storage;
+  Storage.clear();
+  raw_svector_ostream OS(Storage);
+
+  OS << "--- !ELF\n";
+  OS << "FileHeader:\n";
+  OS << "  Class:           ELFCLASS";
+  OS << (T.isArch64Bit() ? "64" : "32");
+  OS << "\n";
+  OS << "  Data:            ELFDATA2";
+  OS << (T.isLittleEndian() ? "LSB" : "MSB");
+  OS << "\n";
+  OS << "  Type:            ET_DYN\n";
+  OS << "  Machine:         "
+     << llvm::StringSwitch<llvm::StringRef>(T.getArchName())
+            .Case("x86_64", "EM_X86_64")
+            .Case("i386", "EM_386")
+            .Case("i686", "EM_386")
+            .Case("aarch64", "EM_AARCH64")
+            .Case("amdgcn", "EM_AMDGPU")
+            .Case("r600", "EM_AMDGPU")
+            .Case("arm", "EM_ARM")
+            .Case("thumb", "EM_ARM")
+            .Case("avr", "EM_AVR")
+            .Case("mips", "EM_MIPS")
+            .Case("mipsel", "EM_MIPS")
+            .Case("mips64", "EM_MIPS")
+            .Case("mips64el", "EM_MIPS")
+            .Case("msp430", "EM_MSP430")
+            .Case("ppc", "EM_PPC")
+            .Case("ppc64", "EM_PPC64")
+            .Case("ppc64le", "EM_PPC64")
+            .Case("x86", T.isOSIAMCU() ? "EM_IAMCU" : "EM_386")
+            .Case("x86_64", "EM_X86_64")
+            .Default("EM_NONE")
+     << "\nSections:"
+     << "\n  - Name:            .text"
+     << "\n    Type:            SHT_PROGBITS"
+     << "\n  - Name:            .data"
+     << "\n    Type:            SHT_PROGBITS"
+     << "\n  - Name:            .rodata"
+     << "\n    Type:            SHT_PROGBITS"
+     << "\nSymbols:\n";
+  for (const auto &Symbol : Symbols) {
+    OS << "  - Name:            " << Symbol.Name << "\n"
+       << "    Type:            STT_";
+    switch (Symbol.Type) {
+    default:
+    case IFSSymbolType::NoType:
+      OS << "NOTYPE";
+      break;
+    case IFSSymbolType::Object:
+      OS << "OBJECT";
+      break;
+    case IFSSymbolType::Func:
+      OS << "FUNC";
+      break;
+    }
+    OS << "\n    Section:         .text"
+       << "\n    Binding:         STB_" << (Symbol.Weak ? "WEAK" : "GLOBAL")
+       << "\n";
+  }
+  OS << "...\n";
+
+  std::string YamlStr = OS.str();
+
+  // Only or debugging. Not an offical format.
+  LLVM_DEBUG({
+    if (ForceFormat == "ELFOBJYAML") {
+      Out << YamlStr;
+      return 0;
+    }
+  });
+
+  yaml::Input YIn(YamlStr);
+  if (Error E = convertYAML(YIn, Out)) {
+    logAllUnhandledErrors(std::move(E), WithColor::error(errs(), "llvm-ifs"));
+    return 1;
+  }
+
+  return 0;
+}
+
+int writeIfso(const IFSStub &Stub, bool IsWriteIfs, raw_ostream &Out) {
+  if (IsWriteIfs) {
+    yaml::Output YamlOut(Out, NULL, /*WrapColumn =*/0);
+    YamlOut << const_cast<IFSStub &>(Stub);
+    return 0;
+  }
+
+  std::string ObjectFileFormat =
+      ForceFormat.empty() ? Stub.ObjectFileFormat : ForceFormat;
+
+  if (ObjectFileFormat == "ELF" || ForceFormat == "ELFOBJYAML")
+    return writeElfStub(llvm::Triple(Stub.Triple), Stub.Symbols,
+                        Stub.ObjectFileFormat, Out);
+  if (ObjectFileFormat == "TBD")
+    return writeTbdStub(llvm::Triple(Stub.Triple), Stub.Symbols,
+                        Stub.ObjectFileFormat, Out);
+
+  WithColor::error()
+      << "Invalid ObjectFileFormat: Only ELF and TBD are supported.\n";
+  return -1;
+}
+
+// New Interface Stubs Yaml Format:
+// --- !experimental-ifs-v1
+// IfsVersion:      1.0
+// Triple:          <llvm triple>
+// ObjectFileFormat: <ELF | others not yet supported>
+// Symbols:
+//   _ZSymbolName: { Type: <type> }
+// ...
+
+int main(int argc, char *argv[]) {
+  // Parse arguments.
+  cl::ParseCommandLineOptions(argc, argv);
+
+  if (InputFilenames.empty())
+    InputFilenames.push_back("-");
+
+  IFSStub Stub;
+  std::map<std::string, IFSSymbol> SymbolMap;
+
+  std::string PreviousInputFilePath = "";
+  for (const std::string &InputFilePath : InputFilenames) {
+    Expected<std::unique_ptr<IFSStub>> StubOrErr = readInputFile(InputFilePath);
+    if (!StubOrErr) {
+      WithColor::error() << StubOrErr.takeError() << "\n";
+      return -1;
+    }
+    std::unique_ptr<IFSStub> TargetStub = std::move(StubOrErr.get());
+
+    if (Stub.Triple.empty()) {
+      PreviousInputFilePath = InputFilePath;
+      Stub.IfsVersion = TargetStub->IfsVersion;
+      Stub.Triple = TargetStub->Triple;
+      Stub.ObjectFileFormat = TargetStub->ObjectFileFormat;
+      Stub.SOName = TargetStub->SOName;
+      Stub.NeededLibs = TargetStub->NeededLibs;
+    } else {
+      if (Stub.IfsVersion != TargetStub->IfsVersion) {
+        if (Stub.IfsVersion.getMajor() != IFSVersionCurrent.getMajor()) {
+          WithColor::error()
+              << "Interface Stub: IfsVersion Mismatch."
+              << "\nFilenames: " << PreviousInputFilePath << " "
+              << InputFilePath << "\nIfsVersion Values: " << Stub.IfsVersion
+              << " " << TargetStub->IfsVersion << "\n";
+          return -1;
+        }
+        if (TargetStub->IfsVersion > Stub.IfsVersion)
+          Stub.IfsVersion = TargetStub->IfsVersion;
+      }
+      if (Stub.ObjectFileFormat != TargetStub->ObjectFileFormat) {
+        WithColor::error() << "Interface Stub: ObjectFileFormat Mismatch."
+                           << "\nFilenames: " << PreviousInputFilePath << " "
+                           << InputFilePath << "\nObjectFileFormat Values: "
+                           << Stub.ObjectFileFormat << " "
+                           << TargetStub->ObjectFileFormat << "\n";
+        return -1;
+      }
+      if (Stub.Triple != TargetStub->Triple) {
+        WithColor::error() << "Interface Stub: Triple Mismatch."
+                           << "\nFilenames: " << PreviousInputFilePath << " "
+                           << InputFilePath << "\nTriple Values: "
+                           << Stub.Triple << " "
+                           << TargetStub->Triple << "\n";
+        return -1;
+      }
+      if (Stub.SOName != TargetStub->SOName) {
+        WithColor::error() << "Interface Stub: SOName Mismatch."
+                           << "\nFilenames: " << PreviousInputFilePath << " "
+                           << InputFilePath
+                           << "\nSOName Values: " << Stub.SOName << " "
+                           << TargetStub->SOName << "\n";
+        return -1;
+      }
+      if (Stub.NeededLibs != TargetStub->NeededLibs) {
+        WithColor::error() << "Interface Stub: NeededLibs Mismatch."
+                           << "\nFilenames: " << PreviousInputFilePath << " "
+                           << InputFilePath << "\n";
+        return -1;
+      }
+    }
+
+    for (auto Symbol : TargetStub->Symbols) {
+      auto SI = SymbolMap.find(Symbol.Name);
+      if (SI == SymbolMap.end()) {
+        SymbolMap.insert(
+            std::pair<std::string, IFSSymbol>(Symbol.Name, Symbol));
+        continue;
+      }
+
+      assert(Symbol.Name == SI->second.Name && "Symbol Names Must Match.");
+
+      // Check conflicts:
+      if (Symbol.Type != SI->second.Type) {
+        WithColor::error() << "Interface Stub: Type Mismatch for "
+                           << Symbol.Name << ".\nFilename: " << InputFilePath
+                           << "\nType Values: " << getTypeName(SI->second.Type) << " "
+                           << getTypeName(Symbol.Type) << "\n";
+
+        return -1;
+      }
+      if (Symbol.Size != SI->second.Size) {
+        WithColor::error() << "Interface Stub: Size Mismatch for "
+                           << Symbol.Name << ".\nFilename: " << InputFilePath
+                           << "\nSize Values: " << SI->second.Size << " "
+                           << Symbol.Size << "\n";
+
+        return -1;
+      }
+      if (Symbol.Weak != SI->second.Weak) {
+        // TODO: Add conflict resolution for Weak vs non-Weak.
+        WithColor::error() << "Interface Stub: Weak Mismatch for "
+                           << Symbol.Name << ".\nFilename: " << InputFilePath
+                           << "\nWeak Values: " << SI->second.Weak << " "
+                           << Symbol.Weak << "\n";
+
+        return -1;
+      }
+      // TODO: Not checking Warning. Will be dropped.
+    }
+
+    PreviousInputFilePath = InputFilePath;
+  }
+
+  if (Stub.IfsVersion != IFSVersionCurrent)
+    if (Stub.IfsVersion.getMajor() != IFSVersionCurrent.getMajor()) {
+      WithColor::error() << "Interface Stub: Bad IfsVersion: "
+                         << Stub.IfsVersion << ", llvm-ifs supported version: "
+                         << IFSVersionCurrent << ".\n";
+      return -1;
+    }
+
+  for (auto &Entry : SymbolMap)
+    Stub.Symbols.insert(Entry.second);
+
+  std::error_code SysErr;
+
+  // Open file for writing.
+  raw_fd_ostream Out(OutputFilename, SysErr);
+  if (SysErr) {
+    WithColor::error() << "Couldn't open " << OutputFilename
+                       << " for writing.\n";
+    return -1;
+  }
+
+  return writeIfso(Stub, (Action == "write-ifs"), Out);
+}