[lld] Fill .text section gaps with INT3 only on x86 targets.

[jacek]

Summary

It doesn't make sense on ARM and using default 0 fill is compatible with MSVC.

(It's more noticeable ARM64EC targets, where additional padding mixed with alignment is used for entry thunk association, so there are more gaps).

[mstorsjo]

Thanks for the patch - I've consiered doing something about this at some time. At the time, I considered filling areas with a more specific pattern, mapping to trapping instructions similar to int3, but since it requires multibyte patterns instead of filling with a single byte, I never got to doing anything about it.

So, MS link.exe fills sections with zero? Is that a specific design choice (i.e. does it fill with 0xcc on x86) or is it simply the case that link.exe doesn't try to fill sections with sentinel instructions at all?

Overall I guess this patch is fine with me - the previous pattern with 0xcc doesn't seem to have mapped to any specific useful thumb/aarch64 instruction anyway (although I guess it's more likely that the previous one would have trapped, especially on aarch64 it seems to not be recognized by the disassembler) than this plain-zeros instruction.

lld/test/COFF/gaps-fill.test

12

Is this a case where llvm-objdump prints just ... for contiguous areas with plain zeros? (I guess that's primarily what this test should be looking for - it seems a bit odd that that checked aspect is so implicit here.)

[jacek]

Thanks for the review.

In D145962#4192325, @mstorsjo wrote:

So, MS link.exe fills sections with zero? Is that a specific design choice (i.e. does it fill with 0xcc on x86) or is it simply the case that link.exe doesn't try to fill sections with sentinel instructions at all?

It seems to be a design choice: link.exe does fill with 0xcc on x64. I will add a test case for x64.

FWIW, in on ARM64EC it's a bit more complicated, because we may have both x64 and true ARM64EC code linked together in a single module. Both use the same section (although according to my testing, never the same page, thus even more potential gaps). In theory, it could make sense to use 0xcc fill for x64 parts, but that's not what link.exe does: it uses 0 to fill everything.

[jacek]

lld/test/COFF/gaps-fill.test

12

Yes, that seems to be its feature (countSkippableZeroBytes in llvm-objdump.cpp). I noticed that there is -z/--disassemble-zeroes option to disable this feature, I will change the test to use that.

[mstorsjo]

LGTM, thanks!

In D145962#4193828, @jacek wrote:

In D145962#4192325, @mstorsjo wrote:

So, MS link.exe fills sections with zero? Is that a specific design choice (i.e. does it fill with 0xcc on x86) or is it simply the case that link.exe doesn't try to fill sections with sentinel instructions at all?

It seems to be a design choice: link.exe does fill with 0xcc on x64. I will add a test case for x64.

Ok, thanks for confirming!

FWIW, in on ARM64EC it's a bit more complicated, because we may have both x64 and true ARM64EC code linked together in a single module. Both use the same section (although according to my testing, never the same page, thus even more potential gaps). In theory, it could make sense to use 0xcc fill for x64 parts, but that's not what link.exe does: it uses 0 to fill everything.

Yeah that sounds more complex. I guess as you say, it could be possible to do varying filling, but that sounds quite complex, so I guess they've just haven't gone out of their way to implement that.

[chapuni]

lld/test/COFF/gaps-fill.test

38

Does it work if only aarch64 is configured?

[mstorsjo]

lld/test/COFF/gaps-fill.test

38

Right, this probably needs to have x86 added to the REQUIRES line.

[mstorsjo]

lld/test/COFF/gaps-fill.test

38

I pushed a fix for this now, in 09aa3f7bb54bf7f35809d3abfdf9f6a679ba1003.

lld/COFF/Writer.cpp

 // Write section contents to a mmap'ed file.
 void Writer::writeSections() {
   uint8_t *buf = buffer->getBufferStart();
   for (OutputSection *sec : ctx.outputSections) {
     uint8_t *secBuf = buf + sec->getFileOff();
     // Fill gaps between functions in .text with INT3 instructions
     // instead of leaving as NUL bytes (which can be interpreted as
     // ADD instructions).
-    if (sec->header.Characteristics & IMAGE_SCN_CNT_CODE)
+    if ((sec->header.Characteristics & IMAGE_SCN_CNT_CODE) &&
+        (ctx.config.machine == AMD64 || ctx.config.machine == I386))
       memset(secBuf, 0xCC, sec->getRawSize());
     parallelForEach(sec->chunks, [&](Chunk *c) {
       c->writeTo(secBuf + c->getRVA() - sec->getRVA());
     });
   }
 }
 
 void Writer::writeBuildId() {

lld/test/COFF/arm-thumb-thunks-multipass.s

 // The instruction above is padding from the .space
 // FUNC01-THUNKS: 404ffc:       f247 0c1e       movw    r12, #28702
 // FUNC01-THUNKS: 405000:       f2c0 0c20       movt    r12, #32
 // FUNC01-THUNKS: 405004:       44e7            add     pc,  r12
 // FUNC01-THUNKS: 405006:       f646 7cf0       movw    r12, #28656
 // FUNC01-THUNKS: 40500a:       f2c0 0c10       movt    r12, #16
 // FUNC01-THUNKS: 40500e:       44e7            add     pc, r12
 // The instruction below is padding from the .balign
-// FUNC01-THUNKS: 405010:       cccc            ldm     r4!, {r2, r3, r6, r7}
+// FUNC01-THUNKS: 405010:       0000            movs    r0, r0

lld/test/COFF/arm64-import2.test

 # BEFORE:        8:       d65f03c0        ret
 # BEFORE:        c:       ff              <unknown>
 
 # AFTER: Disassembly of section .text:
 # AFTER-EMPTY:
 # AFTER:  140001000:      94000004        bl      0x140001010
 # AFTER:  140001004:      94000006        bl      0x14000101c
 # AFTER:  140001008:      d65f03c0        ret
-# AFTER:  14000100c:      ccccccff        <unknown>
+# AFTER:  14000100c:      000000ff
 # AFTER:  140001010:      b0000010        adrp    x16, 0x140002000
 # AFTER:  140001014:      f9403210        ldr     x16, [x16, #96]
 # AFTER:  140001018:      d61f0200        br      x16
 # AFTER:  14000101c:      b0000010        adrp    x16, 0x140002000
 # AFTER:  140001020:      f9403a10        ldr     x16, [x16, #112]
 # AFTER:  140001024:      d61f0200        br      x16
 
 # IMPORTS: Import {

lld/test/COFF/gaps-fill.test

+# REQUIRES: aarch64
+# RUN: split-file %s %t.dir
+
+# RUN: llvm-mc -filetype=obj -triple=aarch64-windows %t.dir/arm64-dllmain.s -o %t.dir/arm64-dllmain.obj
+# RUN: llvm-mc -filetype=obj -triple=aarch64-windows %t.dir/arm64-p4sym.s -o %t.dir/arm64-p4sym.obj
+# RUN: lld-link -dll -machine:arm64 %t.dir/arm64-dllmain.obj %t.dir/arm64-p4sym.obj -out:%t.dll
+
+# RUN: llvm-objdump -dz %t.dll | FileCheck -check-prefix=CHECK-ARM64 %s
+# CHECK-ARM64: 180001000: 52800020     mov     w0, #0x1
+# CHECK-ARM64: 180001004: d65f03c0     ret
+# CHECK-ARM64: 180001008: 00000000
+# CHECK-ARM64: 18000100c: 00000000
+# CHECK-ARM64: 180001010: 52800040     mov     w0, #0x2
+# CHECK-ARM64: 180001014: d65f03c0     ret
+
+#--- arm64-dllmain.s
+        .def _DllMainCRTStartup;
+        .scl 2;
+        .type 32;
+        .endef
+        .globl _DllMainCRTStartup
+        .p2align 2
+_DllMainCRTStartup:
+        mov w0, #1
+        ret
+
+#--- arm64-p4sym.s
+        .def p4sym;
+        .scl 2;
+        .type 32;
+        .endef
+        .globl p4sym
+        .p2align 4
+p4sym:
+        mov w0, #2
+        ret
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-windows %t.dir/x86_64-dllmain.s -o %t.dir/x86_64-dllmain.obj
+# RUN: llvm-mc -filetype=obj -triple=x86_64-windows %t.dir/x86_64-p4sym.s -o %t.dir/x86_64-p4sym.obj
+# RUN: lld-link -dll -machine:amd64 %t.dir/x86_64-dllmain.obj %t.dir/x86_64-p4sym.obj -out:%t.dll
+
+# RUN: llvm-objdump -dz %t.dll | FileCheck -check-prefix=CHECK-X64 %s
+# CHECK-X64: 180001000: b8 01 00 00 00 movl $0x1, %eax
+# CHECK-X64: 180001005: c3 retq
+# CHECK-X64: 180001006: cc int3
+# CHECK-X64: 180001007: cc int3
+# CHECK-X64: 180001008: cc int3
+# CHECK-X64: 180001009: cc int3
+# CHECK-X64: 18000100a: cc int3
+# CHECK-X64: 18000100b: cc int3
+# CHECK-X64: 18000100c: cc int3
+# CHECK-X64: 18000100d: cc int3
+# CHECK-X64: 18000100e: cc int3
+# CHECK-X64: 18000100f: cc int3
+# CHECK-X64: 180001010: b8 02 00 00 00 movl $0x2, %eax
+# CHECK-X64: 180001015: c3 retq
+
+#--- x86_64-dllmain.s
+        .def _DllMainCRTStartup;
+        .scl 2;
+        .type 32;
+        .endef
+        .globl _DllMainCRTStartup
+        .p2align 4, 0x90
+_DllMainCRTStartup:
+        movl $1, %eax
+        retq
+
+#--- x86_64-p4sym.s
+        .def p4sym;
+        .scl 2;
+        .type 32;
+        .endef
+        .globl p4sym
+        .p2align 4, 0x90
+p4sym:
+        movl $2, %eax
+        retq