This is an archive of the discontinued LLVM Phabricator instance.

Differential D149244

[BOLT] Make sure Mach-O binaries are actually linked
ClosedPublic

Authored by jobnoorman on Apr 26 2023, 1:33 AM.

Details

Reviewers
rafauler
maksfb
yota9
Amir
alexander-shaposhnikov
Commits
rGd755e10e7ac8: [BOLT] Make sure Mach-O binaries are actually linked
Summary

Note that this issue is also solved by D147544.

Diff Detail

Event Timeline

jobnoorman created this revision.Apr 26 2023, 1:33 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 26 2023, 1:33 AM
Herald added subscribers: asb, treapster, pmatos, ayermolo. · View Herald Transcript
jobnoorman requested review of this revision.Apr 26 2023, 1:33 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 26 2023, 1:33 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
jobnoorman mentioned this in D147544: [BOLT] Move from RuntimeDyld to JITLink.Apr 26 2023, 1:35 AM
Harbormaster completed remote builds in B228239: Diff 517091.Apr 26 2023, 1:54 AM
Amir added a comment.Apr 26 2023, 7:53 PM

Can you please add a test based on the issue found in D147544?

jobnoorman updated this revision to Diff 517473.Apr 27 2023, 1:02 AM

Add test.

The test was taken from D149113 but fixed and the executable was converted to yaml.

Harbormaster completed remote builds in B228498: Diff 517473.Apr 27 2023, 1:22 AM
maksfb added a subscriber: alexander-shaposhnikov.Apr 27 2023, 11:23 AM
alexander-shaposhnikov added a reviewer: alexander-shaposhnikov.Apr 27 2023, 7:39 PM
alexander-shaposhnikov accepted this revision.Apr 27 2023, 7:41 PM

to me LG

bolt/lib/Rewrite/MachORewriteInstance.cpp
530

yeah, i think it got lost

This revision is now accepted and ready to land.Apr 27 2023, 7:41 PM
This revision was landed with ongoing or failed builds.May 2 2023, 7:24 AM
Closed by commit rGd755e10e7ac8: [BOLT] Make sure Mach-O binaries are actually linked (authored by jobnoorman). · Explain Why
This revision was automatically updated to reflect the committed changes.
jobnoorman added a commit: rGd755e10e7ac8: [BOLT] Make sure Mach-O binaries are actually linked.

Revision Contents

PathSize
bolt/
lib/
Rewrite/
1 line
test/
X86/
MachO/
Inputs/
357 lines
55 lines

Diff 518720

bolt/lib/Rewrite/MachORewriteInstance.cpp

Show First 20 Lines • Show All 521 Lines • ▼ Show 20 Linesvoid MachORewriteInstance::emitAndLink() {
// Assign addresses to all sections. If key corresponds to the object // Assign addresses to all sections. If key corresponds to the object
// created by ourselves, call our regular mapping function. If we are // created by ourselves, call our regular mapping function. If we are
// loading additional objects as part of runtime libraries for // loading additional objects as part of runtime libraries for
// instrumentation, treat them as extra sections. // instrumentation, treat them as extra sections.
mapCodeSections(); mapCodeSections();
mapInstrumentationSection("__counters"); mapInstrumentationSection("__counters");
mapInstrumentationSection("__tables"); mapInstrumentationSection("__tables");
RTDyld->finalizeWithMemoryManagerLocking();
alexander-shaposhnikovUnsubmitted
Not Done
ReplyInline Actions

yeah, i think it got lost

alexander-shaposhnikov: yeah, i think it got lost
// TODO: Refactor addRuntimeLibSections to work properly on Mach-O // TODO: Refactor addRuntimeLibSections to work properly on Mach-O
// and use it here. // and use it here.
//FIXME! Put this in RtLibrary->link //FIXME! Put this in RtLibrary->link
// mapInstrumentationSection("I__setup"); // mapInstrumentationSection("I__setup");
// mapInstrumentationSection("I__fini"); // mapInstrumentationSection("I__fini");
// mapInstrumentationSection("I__data"); // mapInstrumentationSection("I__data");
// mapInstrumentationSection("I__text"); // mapInstrumentationSection("I__text");
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

bolt/test/X86/MachO/Inputs/external_symbol.yaml

  • This file was added.
--- !mach-o
FileHeader:
magic: 0xFEEDFACF
cputype: 0x1000007
cpusubtype: 0x80000003
filetype: 0x2
ncmds: 15
sizeofcmds: 1216
flags: 0x200085
reserved: 0x0
LoadCommands:
- cmd: LC_SEGMENT_64
cmdsize: 72
segname: __PAGEZERO
vmaddr: 0
vmsize: 4294967296
fileoff: 0
filesize: 0
maxprot: 0
initprot: 0
nsects: 0
flags: 0
- cmd: LC_SEGMENT_64
cmdsize: 392
segname: __TEXT
vmaddr: 4294967296
vmsize: 4096
fileoff: 0
filesize: 4096
maxprot: 5
initprot: 5
nsects: 4
flags: 0
Sections:
- sectname: __text
segname: __TEXT
addr: 0x100000F50
size: 70
offset: 0xF50
align: 4
reloff: 0x0
nreloc: 0
flags: 0x80000400
reserved1: 0x0
reserved2: 0x0
reserved3: 0x0
content: 554889E54883EC20C745FC00000000897DF8488975F0837DF8010F8E0D000000E8210000008945FCE910000000C745EC010000008B45EC83C0028945FC8B45FC4883C4205DC3
- sectname: __stubs
segname: __TEXT
addr: 0x100000F96
size: 6
offset: 0xF96
align: 1
reloff: 0x0
nreloc: 0
flags: 0x80000408
reserved1: 0x0
reserved2: 0x6
reserved3: 0x0
content: FF2574000000
- sectname: __stub_helper
segname: __TEXT
addr: 0x100000F9C
size: 28
offset: 0xF9C
align: 2
reloff: 0x0
nreloc: 0
flags: 0x80000400
reserved1: 0x0
reserved2: 0x0
reserved3: 0x0
content: 6800000000E90200000000004C8D1D510000004153FF255100000090
- sectname: __unwind_info
segname: __TEXT
addr: 0x100000FB8
size: 72
offset: 0xFB8
align: 2
reloff: 0x0
nreloc: 0
flags: 0x0
reserved1: 0x0
reserved2: 0x0
reserved3: 0x0
content: 010000001C000000000000001C000000000000001C00000002000000500F00003400000034000000970F00000000000034000000030000000C000100100001000000000000000001
- cmd: LC_SEGMENT_64
cmdsize: 312
segname: __DATA
vmaddr: 4294971392
vmsize: 4096
fileoff: 4096
filesize: 4096
maxprot: 3
initprot: 3
nsects: 3
flags: 0
Sections:
- sectname: __nl_symbol_ptr
segname: __DATA
addr: 0x100001000
size: 8
offset: 0x1000
align: 3
reloff: 0x0
nreloc: 0
flags: 0x6
reserved1: 0x1
reserved2: 0x0
reserved3: 0x0
content: '0000000000000000'
- sectname: __got
segname: __DATA
addr: 0x100001008
size: 8
offset: 0x1008
align: 3
reloff: 0x0
nreloc: 0
flags: 0x6
reserved1: 0x2
reserved2: 0x0
reserved3: 0x0
content: '0000000000000000'
- sectname: __la_symbol_ptr
segname: __DATA
addr: 0x100001010
size: 8
offset: 0x1010
align: 3
reloff: 0x0
nreloc: 0
flags: 0x7
reserved1: 0x3
reserved2: 0x0
reserved3: 0x0
content: 9C0F000001000000
- cmd: LC_SEGMENT_64
cmdsize: 72
segname: __LINKEDIT
vmaddr: 4294975488
vmsize: 4096
fileoff: 8192
filesize: 232
maxprot: 1
initprot: 1
nsects: 0
flags: 0
- cmd: LC_DYLD_INFO_ONLY
cmdsize: 48
rebase_off: 8192
rebase_size: 8
bind_off: 8200
bind_size: 24
weak_bind_off: 0
weak_bind_size: 0
lazy_bind_off: 8224
lazy_bind_size: 16
export_off: 8240
export_size: 48
- cmd: LC_SYMTAB
cmdsize: 24
symoff: 8296
nsyms: 4
stroff: 8376
strsize: 48
- cmd: LC_DYSYMTAB
cmdsize: 80
ilocalsym: 0
nlocalsym: 0
iextdefsym: 0
nextdefsym: 2
iundefsym: 2
nundefsym: 2
tocoff: 0
ntoc: 0
modtaboff: 0
nmodtab: 0
extrefsymoff: 0
nextrefsyms: 0
indirectsymoff: 8360
nindirectsyms: 4
extreloff: 0
nextrel: 0
locreloff: 0
nlocrel: 0
- cmd: LC_LOAD_DYLINKER
cmdsize: 32
name: 12
Content: '/usr/lib/dyld'
ZeroPadBytes: 7
- cmd: LC_UUID
cmdsize: 24
uuid: 4A03D302-35D2-3E3B-93AC-22375185ACFB
- cmd: LC_BUILD_VERSION
cmdsize: 32
platform: 1
minos: 658944
sdk: 658944
ntools: 1
Tools:
- tool: 3
version: 34734080
- cmd: LC_SOURCE_VERSION
cmdsize: 16
version: 0
- cmd: LC_MAIN
cmdsize: 24
entryoff: 3920
stacksize: 0
- cmd: LC_LOAD_DYLIB
cmdsize: 56
dylib:
name: 24
timestamp: 2
current_version: 82115073
compatibility_version: 65536
Content: '/usr/lib/libSystem.B.dylib'
ZeroPadBytes: 6
- cmd: LC_FUNCTION_STARTS
cmdsize: 16
dataoff: 8288
datasize: 8
- cmd: LC_DATA_IN_CODE
cmdsize: 16
dataoff: 8296
datasize: 0
LinkEditData:
RebaseOpcodes:
- Opcode: REBASE_OPCODE_SET_TYPE_IMM
Imm: 1
- Opcode: REBASE_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB
Imm: 2
ExtraData: [ 0x10 ]
- Opcode: REBASE_OPCODE_DO_REBASE_IMM_TIMES
Imm: 1
- Opcode: REBASE_OPCODE_DONE
Imm: 0
BindOpcodes:
- Opcode: BIND_OPCODE_SET_DYLIB_ORDINAL_IMM
Imm: 1
Symbol: ''
- Opcode: BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM
Imm: 0
Symbol: dyld_stub_binder
- Opcode: BIND_OPCODE_SET_TYPE_IMM
Imm: 1
Symbol: ''
- Opcode: BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB
Imm: 2
ULEBExtraData: [ 0x8 ]
Symbol: ''
- Opcode: BIND_OPCODE_DO_BIND
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
LazyBindOpcodes:
- Opcode: BIND_OPCODE_SET_SEGMENT_AND_OFFSET_ULEB
Imm: 2
ULEBExtraData: [ 0x10 ]
Symbol: ''
- Opcode: BIND_OPCODE_SET_DYLIB_SPECIAL_IMM
Imm: 14
Symbol: ''
- Opcode: BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM
Imm: 0
Symbol: _f
- Opcode: BIND_OPCODE_DO_BIND
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
- Opcode: BIND_OPCODE_DONE
Imm: 0
Symbol: ''
ExportTrie:
TerminalSize: 0
NodeOffset: 0
Name: ''
Flags: 0x0
Address: 0x0
Other: 0x0
ImportName: ''
Children:
- TerminalSize: 0
NodeOffset: 5
Name: _
Flags: 0x0
Address: 0x0
Other: 0x0
ImportName: ''
Children:
- TerminalSize: 2
NodeOffset: 33
Name: _mh_execute_header
Flags: 0x0
Address: 0x0
Other: 0x0
ImportName: ''
- TerminalSize: 3
NodeOffset: 37
Name: main
Flags: 0x0
Address: 0xF50
Other: 0x0
ImportName: ''
NameList:
- n_strx: 2
n_type: 0xF
n_sect: 1
n_desc: 16
n_value: 4294967296
- n_strx: 22
n_type: 0xF
n_sect: 1
n_desc: 0
n_value: 4294971216
- n_strx: 28
n_type: 0x1
n_sect: 0
n_desc: 65024
n_value: 0
- n_strx: 31
n_type: 0x1
n_sect: 0
n_desc: 256
n_value: 0
StringTable:
- ' '
- __mh_execute_header
- _main
- _f
- dyld_stub_binder
IndirectSymbols: [ 0x2, 0x40000000, 0x3, 0x2 ]
FunctionStarts: [ 0xF50 ]
...

bolt/test/X86/MachO/emit_new_binary_with_external_symbol.test

  • This file was added.
# RUN: yaml2obj -o %t.exe %p/Inputs/external_symbol.yaml
# RUN: llvm-bolt -o %t.original %t.exe
# RUN: llvm-objdump --macho --disassemble-all --no-print-imm-hex %t.original | FileCheck --check-prefix=ORIGINAL %s
# ORIGINAL: _main:
# ORIGINAL-NEXT: 100000f50: 55 pushq %rbp
# ORIGINAL-NEXT: 100000f51: 48 89 e5 movq %rsp, %rbp
# ORIGINAL-NEXT: 100000f54: 48 83 ec 20 subq $32, %rsp
# ORIGINAL-NEXT: 100000f58: c7 45 fc 00 00 00 00 movl $0, -4(%rbp)
# ORIGINAL-NEXT: 100000f5f: 89 7d f8 movl %edi, -8(%rbp)
# ORIGINAL-NEXT: 100000f62: 48 89 75 f0 movq %rsi, -16(%rbp)
# ORIGINAL-NEXT: 100000f66: 83 7d f8 01 cmpl $1, -8(%rbp)
# ORIGINAL-NEXT: 100000f6a: 7e 0a jle 0x100000f76
# ORIGINAL-NEXT: 100000f6c: e8 25 00 00 00 callq 0x100000f96
# ORIGINAL-NEXT: 100000f71: 89 45 fc movl %eax, -4(%rbp)
# ORIGINAL-NEXT: 100000f74: eb 10 jmp 0x100000f86
# ORIGINAL-NEXT: 100000f76: c7 45 ec 01 00 00 00 movl $1, -20(%rbp)
# ORIGINAL-NEXT: 100000f7d: 8b 45 ec movl -20(%rbp), %eax
# ORIGINAL-NEXT: 100000f80: 83 c0 02 addl $2, %eax
# ORIGINAL-NEXT: 100000f83: 89 45 fc movl %eax, -4(%rbp)
# ORIGINAL-NEXT: 100000f86: 8b 45 fc movl -4(%rbp), %eax
# ORIGINAL-NEXT: 100000f89: 48 83 c4 20 addq $32, %rsp
# ORIGINAL-NEXT: 100000f8d: 5d popq %rbp
# ORIGINAL-NEXT: 100000f8e: c3 retq
# ORIGINAL-NEXT: 100000f8f: fc cld
# ORIGINAL-NEXT: 100000f90: 48 83 c4 20 addq $32, %rsp
# ORIGINAL-NEXT: 100000f94: 5d popq %rbp
# ORIGINAL-NEXT: 100000f95: c3 retq
# RUN: llvm-bolt -o %t.reversed -reorder-blocks=reverse %t.exe
# RUN: llvm-objdump --macho --disassemble-all --no-print-imm-hex %t.reversed | FileCheck --check-prefix=REVERSED %s
# REVERSED: _main:
# REVERSED-NEXT: 100000f50: 55 pushq %rbp
# REVERSED-NEXT: 100000f51: 48 89 e5 movq %rsp, %rbp
# REVERSED-NEXT: 100000f54: 48 83 ec 20 subq $32, %rsp
# REVERSED-NEXT: 100000f58: c7 45 fc 00 00 00 00 movl $0, -4(%rbp)
# REVERSED-NEXT: 100000f5f: 89 7d f8 movl %edi, -8(%rbp)
# REVERSED-NEXT: 100000f62: 48 89 75 f0 movq %rsi, -16(%rbp)
# REVERSED-NEXT: 100000f66: 83 7d f8 01 cmpl $1, -8(%rbp)
# REVERSED-NEXT: 100000f6a: 7e 0b jle 0x100000f77
# REVERSED-NEXT: 100000f6c: eb 1b jmp 0x100000f89
# REVERSED-NEXT: 100000f6e: 8b 45 fc movl -4(%rbp), %eax
# REVERSED-NEXT: 100000f71: 48 83 c4 20 addq $32, %rsp
# REVERSED-NEXT: 100000f75: 5d popq %rbp
# REVERSED-NEXT: 100000f76: c3 retq
# REVERSED-NEXT: 100000f77: c7 45 ec 01 00 00 00 movl $1, -20(%rbp)
# REVERSED-NEXT: 100000f7e: 8b 45 ec movl -20(%rbp), %eax
# REVERSED-NEXT: 100000f81: 83 c0 02 addl $2, %eax
# REVERSED-NEXT: 100000f84: 89 45 fc movl %eax, -4(%rbp)
# REVERSED-NEXT: 100000f87: eb e5 jmp 0x100000f6e
# REVERSED-NEXT: 100000f89: e8 08 00 00 00 callq 0x100000f96
# REVERSED-NEXT: 100000f8e: 89 45 fc movl %eax, -4(%rbp)
# REVERSED-NEXT: 100000f91: eb db jmp 0x100000f6e
# REVERSED-NEXT: 100000f93: 20 5d c3 andb %bl, -61(%rbp)