[BOLT] AArch64: Emit text objects

[yota9]

Summary

BOLT treats aarch64 objects located in text as empty functions with
contant islands. Emit them with at least 8-byte alignment to the new
text section.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

[yota9]

Move test from runtime

[yota9]

Rebase

[rafauler]

Overall LGTM.

The model that BOLT tries to create in memory is one in which every byte in an executable section has an associated BinaryFunction. From the point of view of the processor, an executable section should contain code, and BOLT models all code as the contents of a function.

BOLT deals poorly with data in code: it will mark the function containing data and will avoid processing it. Besides, in X86, it is frequently a bad practice to put data in the code section, as the processor has different caches for instruction and data, and if you put non-instruction bytes in the instruction stream, you will be unnecessarily polluting i-cache.

In AArch64, since it's a RISC processor that absolutely needs to put stuff in the executable sections due to limited range access, I expanded BOLT's IR to consider that a function might have data in code that is easily identifiable via $d symbol markers in the ELF file.

Given this context, an empty function with a constant island (freestanding data in code that is not part of any function) is a curious thing to see. Where is this happening?

bolt/lib/Passes/Aligner.cpp

177

Is this in sync with LongJmp?

bolt/test/AArch64/text_data.c

17 ↗

(On Diff #418466)

Just curious, how would this ever happen naturally (without forcing the compiler to do so)? If this is a synthetic test, it would be good to have a test that shows the real use case of freestanding data in code. But it's fine if it's not possible (e.g. if this is created as part of a non-C compiler implementing some specific runtime).

[maksfb]

nit: Could you rename text_data.c -> text-data.c ?

[yota9]

bolt/lib/Passes/Aligner.cpp

177

Sure, it is part of the previous commit https://github.com/llvm/llvm-project/commit/af9bdcfc46ef73179797ebd406ab413cd047a8c7

bolt/test/AArch64/text_data.c

17 ↗

(On Diff #418466)

This is rare case and not a compiler blame. Such things might appear in assembler and examples of such an issues were opened in previous BOLT repos like this: https://github.com/facebookincubator/BOLT/issues/48 . As you said thanks for mapping symbols we can handle such cases a bit better now for aarch64 - by moving the whole object in the newly created text section :)

bolt/lib/Core/BinaryEmitter.cpp

void BinaryEmitter::emitFunctions() {
   // Mark the end of hot text.
   if (opts::HotText) {
     Streamer.SwitchSection(BC.getTextSection());
     Streamer.emitLabel(BC.getHotTextEndSymbol());
   }
 }
 
 bool BinaryEmitter::emitFunction(BinaryFunction &Function, bool EmitColdPart) {
-  if (Function.size() == 0)
+  if (Function.size() == 0 && !Function.hasIslandsInfo())
     return false;
 
   if (Function.getState() == BinaryFunction::State::Empty)
     return false;
 
   MCSection *Section =
       BC.getCodeSection(EmitColdPart ? Function.getColdCodeSectionName()
                                      : Function.getCodeSectionName());

bolt/lib/Passes/Aligner.cpp

ParallelUtilities::WorkFuncTy WorkFun = [&](BinaryFunction &BF) {
     BinaryContext::IndependentCodeEmitter Emitter =
         BC.createIndependentMCCodeEmitter();
 
     if (opts::UseCompactAligner)
       alignCompact(BF, Emitter.MCE.get());
     else
       alignMaxBytes(BF);
 
+    // Align objects that contains constant islands and no code
+    // to at least 8 bytes.
+    if (!BF.size() && BF.hasIslandsInfo()) {
+      const uint16_t Alignment = BF.getConstantIslandAlignment();
+      if (BF.getAlignment() < Alignment)
+        BF.setAlignment(Alignment);
+
+      if (BF.getMaxAlignmentBytes() < Alignment)
+        BF.setMaxAlignmentBytes(Alignment);
+
+      if (BF.getMaxColdAlignmentBytes() < Alignment)
+        BF.setMaxColdAlignmentBytes(Alignment);
+    }
+
     if (opts::AlignBlocks && !opts::PreserveBlocksAlignment)
       alignBlocks(BF, Emitter.MCE.get());
   };
 
   ParallelUtilities::runOnEachFunction(
       BC, ParallelUtilities::SchedulingPolicy::SP_TRIVIAL, WorkFun,
       ParallelUtilities::PredicateTy(nullptr), "AlignerPass");
 

bolt/test/AArch64/text-data.c

+// This test checks that the data object located in text section
+// is properly emitted in the new section.
+
+// RUN: %clang %cflags %s -o %t.exe -Wl,-q
+// RUN: llvm-bolt %t.exe -o %t.bolt -lite=0 -use-old-text=0
+// RUN: llvm-objdump -j .text -d --disassemble-symbols=arr %t.bolt | \
+// RUN:   FileCheck %s
+
+// CHECK: {{.*}} <arr>:
+
+#include <stdlib.h>
+
+typedef void (*FooPtr)();
+
+void exitOk() { exit(0); }
+
+__attribute__((section(".text"))) const FooPtr arr[] = {exitOk, NULL};
+
+int main() {
+  arr[0]();
+  return -1;
+}