[XRay][CodeGen] Use PIC-friendly code in XRay sleds; remove synthetic references in .text

[dberris]

Summary

This change achieves two things:

• Redefine the Custom Event handling instrumentation points emitted by the compiler to not require dynamic relocation of references to the __xray_CustomEvent trampoline.
• Remove the synthetic reference we emit at the end of a function that we used to keep auxiliary sections alive in favour of SHF_LINK_ORDER associated with the section where the function is defined.

To achieve the custom event handling change, we've had to introduce the
concept of sled versioning -- this will need to be supported by the
runtime to allow us to understand how to turn on/off the new version of
the custom event handling sleds. That change has to land first before we
change the way we write the sleds.

To remove the synthetic reference, we rely on a relatively new linker
feature that preserves the sections that are associated with each other.
This allows us to limit the effects on the .text section of ELF
binaries.

Because we're still using absolute references that are resolved at
runtime for the instrumentation map (and function index) maps, we mark
these sections write-able. In the future we can re-define the entries in
the map to use relative relocations instead that can be statically
determined by the linker. That change will be a bit more invasive so we
defer this for later.

Depends on D36816.

[dblaikie]

Don't know enough about ELF attributes to review the point of this patch, sorry.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

2793–2803

Unrelated reformatting? (or I can't spot what the change is on these lines)

2821–2826

Repetitious - reformat to a common function, perhaps?

[dberris]

Missing test changes; re-titling and updating for review. Next update should contain updated (and new) tests.

[pcc]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

2779

This function (and the other places where you are doing the same thing) seems very questionable. In all cases the symbol is a temporary symbol, which means that it should not even appear in the symbol table, and I would expect these attribute assignments to have no effect. In fact if I patch in your change and build an object file like this:

$ cat foo.cpp
int main() {
  __xray_customevent(0, 0);
}
$ clang++ -fPIE -pie foo.cpp -fxray-instrument -c

I get exactly the same object file if I remove all of these things.

2798

I don't think this is right. I think you need to create a *separate* xray_instr_map and xray_fn_idx section for each function you emit. Otherwise the assembler will put every xray table in the same section and link the section with the first function section that you emit. That will mean that if the first function is dropped, the entire xray section is dropped. I can reproduce this with lld like this:

$ cat foo.cpp 
void f() {
  __xray_customevent(0, 0);
}

int main() {
  __xray_customevent(0, 0);
}
$ clang++ -fPIE -pie foo.cpp -fuse-ld=lld -fxray-instrument   -ffunction-sections -Wl,--gc-sections
$ readelf -S a.out | grep xray
[no output]

If I swap the order of main and f, the xray sections are kept (and f is kept alive because of references from the xray sections).

You can create multiple sections with the same name by incrementing the UniqueID that you pass to getELFSection.

[dberris]

Reword and update implementation, this time with tests.

[dberris]

Thanks, @pcc -- PTAL?

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

2821–2826

Looks like we didn't need to do that anyway, removed.

[dberris]

Rebase

[pcc]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

2807

Remove this comment.

2830

Remove this part of the comment.

lib/Target/X86/X86MCInstLower.cpp

1088–1089

(This is somewhat orthogonal to this patch, but...)

Is this code reachable? The reg-imm form of the mov instruction uses a variable number of bytes depending on the width of the immediate (I think MOV64ri will always select the 64-bit variant which uses 10 bytes), so if the reg-imm form is selected, the jmp instruction above will jump to the wrong instruction. Since this can't possibly work (nor could it have worked before), maybe assert that the operand is a register?

1093

Here you are pushing rdi and rsi onto the stack whereas you weren't doing that before.

1. I don't understand why the pushes are necessary now when they weren't before.
2. Since you are pushing one more value than before, is the stack still aligned correctly?

test/CodeGen/Mips/xray-section-group.ll

17

This test should make sure that the .section directive specifies a unique ID and associated section.
https://llvm.org/docs/Extensions.html#id1

27

Same here for the unique ID.

[dberris]

Address comments

[dberris]

lib/Target/X86/X86MCInstLower.cpp

1093

Because this is an intrinsic being lowered, defined in include/llvm/IR/Instrinsics.td, we aren't quite able to provide a mask of the registers that will be clobbered/used by this routine. While we haven't quite run into bugs without stashing the %rsi and %rdi registers here (we do actually do the right thing in the trampoline), We also haven't been defining the calling convention for calls to this intrinsic (which we couldn't define for the intrinsic too). I think I'm just being a bit cautious here.

The alignment question is an interesting one. Because it's an intrinsic being lowered, and that we're already past the stack adjustment phase (and side-stepping the call-lowering code for normal call instructions), we haven't quite been tracking the stack adjustments here. What we do assume is that if the stack was aligned already, pushing two 64-bit values onto the stack will keep it aligned -- then the trampoline starts running, assuming that the stack is set up properly.

Do you have suggestions on how else to do/test this?

[pcc]

lib/Target/X86/X86MCInstLower.cpp

1090

You can replace this if statement with the "then" body.

1093

Note that the callee is allowed to clobber more registers than just %rsi and %rdi, see the full list here: https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf

If the idea is that the __xray_CustomEvent function has a custom "callee-save almost all registers" calling convention, I'd expect you to need to save all registers in __xray_CustomEvent not marked as "preserved across function calls". It does at least seem to be necessary to save %rdi and %rsi in the function itself though.

For stack alignment, I don't think you can always assume %rsp to be properly aligned at the point where you emit the __xray_CustomEvent call. Here's a quick counterexample that shows that we can either align the stack correctly (in the case of foo) or incorrectly (in the case of bar) when calling the handler.

#include <xmmintrin.h>

__attribute__((weak)) __m128 f(__m128 *i) {
  return *i;
}

void foo() {
  __xray_customevent(0, 0);
  __m128 v = {};
  f(&v);
}

void bar() {
  __xray_customevent(0, 0);
}

If you compile this you should see that foo subtracts 24 from %rsp before calling __xray_CustomEvent, while bar does not subtract anything.

I'm no backend expert, so I don't know how to fix this properly, but I imagine that you could always realign the stack in the custom event handler function.

In terms of testing, the function foo above is an example of a function that requires the stack to be aligned, so you could try writing an integration test that uses something like that as a custom event handler function.

[dberris]

• fixup: remove unnecessary if condition

[dberris]

lib/Target/X86/X86MCInstLower.cpp

1093

Cool stuff!

I made the changes in D36816 to stash more registers in __xray_CustomEvent. We already have a special calling convention as you suggest (which is documented in the trampoline implementation). I'm not sure though whether the alignment in the case of the empty function does cause issues -- we use call instructions and already have a special calling convention for the trampolines.

[pcc]

LGTM

lib/Target/X86/X86MCInstLower.cpp

1093

Thanks, I've left some comments on that review.

include/llvm/CodeGen/AsmPrinter.h

enum class SledKind : uint8_t {
     FUNCTION_EXIT = 1,
     TAIL_CALL = 2,
     LOG_ARGS_ENTER = 3,
     CUSTOM_EVENT = 4,
   };
 
   // The table will contain these structs that point to the sled, the function
   // containing the sled, and what kind of sled (and whether they should always
-  // be instrumented).
+  // be instrumented). We also use a version identifier that the runtime can use
+  // to decide what to do with the sled, depending on the version of the sled.
   struct XRayFunctionEntry {
     const MCSymbol *Sled;
     const MCSymbol *Function;
     SledKind Kind;
     bool AlwaysInstrument;
     const class Function *Fn;
+    uint8_t Version;
 
     void emit(int, MCStreamer *, const MCSymbol *) const;
   };
 
   // All the sleds to be emitted.
   SmallVector<XRayFunctionEntry, 4> Sleds;
 
+  // A unique ID used for ELF sections associated with a particular function.
+  unsigned XRayFnUniqueID = 0;
+
   // Helper function to record a given XRay sled.
-  void recordSled(MCSymbol *Sled, const MachineInstr &MI, SledKind Kind);
+  void recordSled(MCSymbol *Sled, const MachineInstr &MI, SledKind Kind,
+                  uint8_t Version = 0);
 
   /// Emit a table with all XRay instrumentation points.
   void emitXRayTable();
 
   //===------------------------------------------------------------------===//
   // MachineFunctionPass Implementation.
   //===------------------------------------------------------------------===//
 

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

 // In the binary's "xray_instr_map" section, an array of these function entries
 // describes each instrumentation point.  When XRay patches your code, the index
 // into this table will be given to your handler as a patch point identifier.
 void AsmPrinter::XRayFunctionEntry::emit(int Bytes, MCStreamer *Out,
                                          const MCSymbol *CurrentFnSym) const {
   Out->EmitSymbolValue(Sled, Bytes);
   Out->EmitSymbolValue(CurrentFnSym, Bytes);
   auto Kind8 = static_cast<uint8_t>(Kind);
-  Out->EmitBytes(StringRef(reinterpret_cast<const char *>(&Kind8), 1));
-  Out->EmitBytes(
+  Out->EmitBinaryData(StringRef(reinterpret_cast<const char *>(&Kind8), 1));
+  Out->EmitBinaryData(
       StringRef(reinterpret_cast<const char *>(&AlwaysInstrument), 1));
-  Out->EmitZeros(2 * Bytes - 2);  // Pad the previous two entries
+  Out->EmitBinaryData(StringRef(reinterpret_cast<const char *>(&Version), 1));
+  auto Padding = (4 * Bytes) - ((2 * Bytes) + 3);
+  assert(Padding >= 0 && "Instrumentation map entry > 4 * Word Size");
+  Out->EmitZeros(Padding);
 }
 
 void AsmPrinter::emitXRayTable() {
   if (Sleds.empty())
     return;
 
   auto PrevSection = OutStreamer->getCurrentSectionOnly();
   auto Fn = MF->getFunction();
   MCSection *InstMap = nullptr;
   MCSection *FnSledIndex = nullptr;
   if (MF->getSubtarget().getTargetTriple().isOSBinFormatELF()) {
+    auto Associated = dyn_cast<MCSymbolELF>(PrevSection->getBeginSymbol());
+    assert(Associated != nullptr);
+    auto Flags = ELF::SHF_WRITE | ELF::SHF_ALLOC | ELF::SHF_LINK_ORDER;
+    std::string GroupName;
     if (Fn->hasComdat()) {
-      InstMap = OutContext.getELFSection("xray_instr_map", ELF::SHT_PROGBITS,
-                                         ELF::SHF_ALLOC | ELF::SHF_GROUP, 0,
-                                         Fn->getComdat()->getName());
-      FnSledIndex = OutContext.getELFSection("xray_fn_idx", ELF::SHT_PROGBITS,
-                                             ELF::SHF_ALLOC | ELF::SHF_GROUP, 0,
-                                             Fn->getComdat()->getName());
-    } else {
-      InstMap = OutContext.getELFSection("xray_instr_map", ELF::SHT_PROGBITS,
-                                         ELF::SHF_ALLOC);
-      FnSledIndex = OutContext.getELFSection("xray_fn_idx", ELF::SHT_PROGBITS,
-                                             ELF::SHF_ALLOC);
+      Flags |= ELF::SHF_GROUP;
+      GroupName = Fn->getComdat()->getName();
     }
+
+    auto UniqueID = ++XRayFnUniqueID;
+    InstMap =
+        OutContext.getELFSection("xray_instr_map", ELF::SHT_PROGBITS, Flags, 0,
+                                 GroupName, UniqueID, Associated);
+    FnSledIndex =
+        OutContext.getELFSection("xray_fn_idx", ELF::SHT_PROGBITS, Flags, 0,
+                                 GroupName, UniqueID, Associated);
   } else if (MF->getSubtarget().getTargetTriple().isOSBinFormatMachO()) {
     InstMap = OutContext.getMachOSection("__DATA", "xray_instr_map", 0,
                                          SectionKind::getReadOnlyWithRel());
     FnSledIndex = OutContext.getMachOSection("__DATA", "xray_fn_idx", 0,
                                              SectionKind::getReadOnlyWithRel());
   } else {
     llvm_unreachable("Unsupported target");
   }
 
-  // Before we switch over, we force a reference to a label inside the
-  // xray_fn_idx sections. This makes sure that the xray_fn_idx section is kept
-  // live by the linker if the function is not garbage-collected. Since this
-  // function is always called just before the function's end, we assume that
-  // this is happening after the last return instruction.
   auto WordSizeBytes = MAI->getCodePointerSize();
-  MCSymbol *IdxRef = OutContext.createTempSymbol("xray_fn_idx_synth_", true);
-  OutStreamer->EmitCodeAlignment(16);
-  OutStreamer->EmitSymbolValue(IdxRef, WordSizeBytes, false);
 
   // Now we switch to the instrumentation map section. Because this is done
   // per-function, we are able to create an index entry that will represent the
   // range of sleds associated with a function.
   MCSymbol *SledsStart = OutContext.createTempSymbol("xray_sleds_start", true);
   OutStreamer->SwitchSection(InstMap);
   OutStreamer->EmitLabel(SledsStart);
   for (const auto &Sled : Sleds)
     Sled.emit(WordSizeBytes, OutStreamer.get(), CurrentFnSym);
   MCSymbol *SledsEnd = OutContext.createTempSymbol("xray_sleds_end", true);
   OutStreamer->EmitLabel(SledsEnd);
 
   // We then emit a single entry in the index per function. We use the symbols
   // that bound the instrumentation map as the range for a specific function.
   // Each entry here will be 2 * word size aligned, as we're writing down two
   // pointers. This should work for both 32-bit and 64-bit platforms.
   OutStreamer->SwitchSection(FnSledIndex);
   OutStreamer->EmitCodeAlignment(2 * WordSizeBytes);
-  OutStreamer->EmitLabel(IdxRef);
-  OutStreamer->EmitSymbolValue(SledsStart, WordSizeBytes);
-  OutStreamer->EmitSymbolValue(SledsEnd, WordSizeBytes);
+  OutStreamer->EmitSymbolValue(SledsStart, WordSizeBytes, false);
+  OutStreamer->EmitSymbolValue(SledsEnd, WordSizeBytes, false);
   OutStreamer->SwitchSection(PrevSection);
   Sleds.clear();
 }
 
 void AsmPrinter::recordSled(MCSymbol *Sled, const MachineInstr &MI,
-  SledKind Kind) {
+                            SledKind Kind, uint8_t Version) {
   auto Fn = MI.getParent()->getParent()->getFunction();
   auto Attr = Fn->getFnAttribute("function-instrument");
   bool LogArgs = Fn->hasFnAttribute("xray-log-args");
   bool AlwaysInstrument =
     Attr.isStringAttribute() && Attr.getValueAsString() == "xray-always";
   if (Kind == SledKind::FUNCTION_ENTER && LogArgs)
     Kind = SledKind::LOG_ARGS_ENTER;
-  Sleds.emplace_back(
-    XRayFunctionEntry{ Sled, CurrentFnSym, Kind, AlwaysInstrument, Fn });
+  Sleds.emplace_back(XRayFunctionEntry{Sled, CurrentFnSym, Kind,
+                                       AlwaysInstrument, Fn, Version});
 }
 
 uint16_t AsmPrinter::getDwarfVersion() const {
   return OutStreamer->getContext().getDwarfVersion();
 }
 
 void AsmPrinter::setDwarfVersion(uint16_t Version) {
   OutStreamer->getContext().setDwarfVersion(Version);
 }

lib/Target/X86/X86MCInstLower.cpp

 
 void X86AsmPrinter::LowerPATCHABLE_EVENT_CALL(const MachineInstr &MI,
                                               X86MCInstLower &MCIL) {
   assert(Subtarget->is64Bit() && "XRay custom events only supports X86-64");
 
   // We want to emit the following pattern, which follows the x86 calling
   // convention to prepare for the trampoline call to be patched in.
   //
-  //   <args placement according SysV64 calling convention>
   //   .p2align 1, ...
   // .Lxray_event_sled_N:
-  //   jmp +N                    // jump across the call instruction
-  //   callq __xray_CustomEvent  // force relocation to symbol
-  //   <args cleanup, jump to here>
-  //
-  // The relative jump needs to jump forward 24 bytes:
-  // 10 (args) + 5 (nops) + 9 (cleanup)
+  //   jmp +N                    // jump across the instrumentation sled
+  //   ...                       // set up arguments in register
+  //   callq __xray_CustomEvent  // force dependency to symbol
+  //   ...
+  //   <jump here>
   //
   // After patching, it would look something like:
   //
   //   nopw (2-byte nop)
+  //   ...
   //   callq __xrayCustomEvent  // already lowered
+  //   ...
   //
   // ---
   // First we emit the label and the jump.
   auto CurSled = OutContext.createTempSymbol("xray_event_sled_", true);
   OutStreamer->AddComment("# XRay Custom Event Log");
   OutStreamer->EmitCodeAlignment(2);
   OutStreamer->EmitLabel(CurSled);
 
   // Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
   // an operand (computed as an offset from the jmp instruction).
   // FIXME: Find another less hacky way do force the relative jump.
-  OutStreamer->EmitBytes("\xeb\x14");
+  OutStreamer->EmitBinaryData("\xeb\x0f");
 
   // The default C calling convention will place two arguments into %rcx and
   // %rdx -- so we only work with those.
-  unsigned UsedRegs[] = {X86::RDI, X86::RSI, X86::RAX};
-
-  // Because we will use %rax, we preserve that across the call.
-  EmitAndCountInstruction(MCInstBuilder(X86::PUSH64r).addReg(X86::RAX));
+  unsigned UsedRegs[] = {X86::RDI, X86::RSI};
+  bool UsedMask[] = {false, false};
 
-  // Then we put the operands in the %rdi and %rsi registers.
+  // Then we put the operands in the %rdi and %rsi registers. We spill the
+  // values in the register before we clobber them, and mark them as used in
+  // UsedMask. In case the arguments are already in the correct register, we use
+  // emit nops appropriately sized to keep the sled the same size in every
+  // situation.
   for (unsigned I = 0; I < MI.getNumOperands(); ++I)
     if (auto Op = MCIL.LowerMachineOperand(&MI, MI.getOperand(I))) {
-      if (Op->isImm())
-        EmitAndCountInstruction(MCInstBuilder(X86::MOV64ri)
-                                    .addReg(UsedRegs[I])
-                                    .addImm(Op->getImm()));
-      else if (Op->isReg()) {
-        if (Op->getReg() != UsedRegs[I])
+      assert(Op->isReg() && "Only support arguments in registers");
+      if (Op->isReg()) {
+        if (Op->getReg() != UsedRegs[I]) {
+          UsedMask[I] = true;
+          EmitAndCountInstruction(
+              MCInstBuilder(X86::PUSH64r).addReg(UsedRegs[I]));
           EmitAndCountInstruction(MCInstBuilder(X86::MOV64rr)
                                       .addReg(UsedRegs[I])
                                       .addReg(Op->getReg()));
-        else
-          EmitNops(*OutStreamer, 3, Subtarget->is64Bit(), getSubtargetInfo());
+        } else {
+          EmitNops(*OutStreamer, 4, Subtarget->is64Bit(), getSubtargetInfo());
+        }
       }
     }
 
   // We emit a hard dependency on the __xray_CustomEvent symbol, which is the
-  // name of the trampoline to be implemented by the XRay runtime. We put this
-  // explicitly in the %rax register.
+  // name of the trampoline to be implemented by the XRay runtime.
   auto TSym = OutContext.getOrCreateSymbol("__xray_CustomEvent");
   MachineOperand TOp = MachineOperand::CreateMCSymbol(TSym);
-  EmitAndCountInstruction(MCInstBuilder(X86::MOV64ri)
-                              .addReg(X86::RAX)
-                              .addOperand(MCIL.LowerSymbolOperand(TOp, TSym)));
 
   // Emit the call instruction.
-  EmitAndCountInstruction(MCInstBuilder(X86::CALL64r).addReg(X86::RAX));
+  EmitAndCountInstruction(MCInstBuilder(X86::CALL64pcrel32)
+                              .addOperand(MCIL.LowerSymbolOperand(TOp, TSym)));
 
   // Restore caller-saved and used registers.
+  for (unsigned I = sizeof UsedMask; I-- > 0;)
+    if (UsedMask[I])
+      EmitAndCountInstruction(MCInstBuilder(X86::POP64r).addReg(UsedRegs[I]));
+    else
+      EmitNops(*OutStreamer, 1, Subtarget->is64Bit(), getSubtargetInfo());
+
   OutStreamer->AddComment("xray custom event end.");
-  EmitAndCountInstruction(MCInstBuilder(X86::POP64r).addReg(X86::RAX));
 
-  recordSled(CurSled, MI, SledKind::CUSTOM_EVENT);
+  // Record the sled version. Older versions of this sled were spelled
+  // differently, so we let the runtime handle the different offsets we're
+  // using.
+  recordSled(CurSled, MI, SledKind::CUSTOM_EVENT, 1);
 }
 
 void X86AsmPrinter::LowerPATCHABLE_FUNCTION_ENTER(const MachineInstr &MI,
                                                   X86MCInstLower &MCIL) {
   // We want to emit the following pattern:
   //
   //   .p2align 1, ...
   // .Lxray_sled_N:
   //   jmp .tmpN
   //   # 9 bytes worth of noops
-  // .tmpN
   //
   // We need the 9 bytes because at runtime, we'd be patching over the full 11
   // bytes with the following pattern:
   //
   //   mov %r10, <function id, 32-bit>   // 6 bytes
   //   call <relative offset, 32-bits>   // 5 bytes
   //
   auto CurSled = OutContext.createTempSymbol("xray_sled_", true);
   OutStreamer->EmitCodeAlignment(2);
   OutStreamer->EmitLabel(CurSled);
-  auto Target = OutContext.createTempSymbol();
 
   // Use a two-byte `jmp`. This version of JMP takes an 8-bit relative offset as
   // an operand (computed as an offset from the jmp instruction).
   // FIXME: Find another less hacky way do force the relative jump.
   OutStreamer->EmitBytes("\xeb\x09");
   EmitNops(*OutStreamer, 9, Subtarget->is64Bit(), getSubtargetInfo());
-  OutStreamer->EmitLabel(Target);
   recordSled(CurSled, MI, SledKind::FUNCTION_ENTER);
 }
 
 void X86AsmPrinter::LowerPATCHABLE_RET(const MachineInstr &MI,
                                        X86MCInstLower &MCIL) {
   // Since PATCHABLE_RET takes the opcode of the return statement as an
   // argument, we use that to emit the correct form of the RET that we want.
   // i.e. when we see this:

test/CodeGen/AArch64/xray-attribute-instrumentation.ll

 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-LABEL: Ltmp1:
 ; CHECK-NEXT:  ret
 }
-; CHECK:       .p2align 4
-; CHECK-NEXT:  .xword .Lxray_fn_idx_synth_0
-; CHECK-NEXT:  .section xray_instr_map,{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0
 ; CHECK:       .xword .Lxray_sled_0
 ; CHECK:       .xword .Lxray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0

test/CodeGen/AArch64/xray-tail-call-sled.ll

 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-LABEL: .Ltmp1:
 ; CHECK-NEXT:  ret
 }
-; CHECK:       .p2align 4
-; CHECK-NEXT:  .xword .Lxray_fn_idx_synth_0
-; CHECK-NEXT:  .section xray_instr_map,{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .xword .Lxray_sled_0
 ; CHECK:       .xword .Lxray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0:
-; CHECK:       .section xray_fn_idx,{{.*}}
-; CHECK-LABEL: Lxray_fn_idx_synth_0:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .xword .Lxray_sleds_start0
 ; CHECK-NEXT:  .xword .Lxray_sleds_end0
 
 define i32 @caller() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK:       .p2align 2
 ; CHECK-LABEL: Lxray_sled_2:
 ; CHECK-NEXT:  b  #32
 ; CHECK-NEXT:  nop

 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-LABEL: .Ltmp3:
   %retval = tail call i32 @callee()
 ; CHECK:       b  callee
   ret i32 %retval
 }
-; CHECK:       .p2align 4
-; CHECK-NEXT:  .xword .Lxray_fn_idx_synth_1
-; CHECK-NEXT:  .section xray_instr_map,{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start1:
 ; CHECK:       .xword .Lxray_sled_2
 ; CHECK:       .xword .Lxray_sled_3
 ; CHECK-LABEL: Lxray_sleds_end1:
 ; CHECK:       .section xray_fn_idx,{{.*}}
-; CHECK-LABEL: Lxray_fn_idx_synth_1:
 ; CHECK:       .xword .Lxray_sleds_start1
 ; CHECK-NEXT:  .xword .Lxray_sleds_end1

test/CodeGen/ARM/xray-armv6-attribute-instrumentation.ll

 ; CHECK-NEXT:  mov  r0, r0
 ; CHECK-NEXT:  mov  r0, r0
 ; CHECK-NEXT:  mov  r0, r0
 ; CHECK-NEXT:  mov  r0, r0
 ; CHECK-NEXT:  mov  r0, r0
 ; CHECK-LABEL: Ltmp1:
 ; CHECK-NEXT:  bx lr
 }
-; CHECK:       .p2align 4
-; CHECK-NEXT:  .long {{.*}}Lxray_fn_idx_synth_0
-; CHECK-NEXT:  .section {{.*}}xray_instr_map{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .long {{.*}}Lxray_sled_0
 ; CHECK:       .long {{.*}}Lxray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0:
-; CHECK:       .section {{.*}}xray_fn_idx{{.*}}
-; CHECK-LABEL: Lxray_fn_idx_synth_0:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .long {{.*}}Lxray_sleds_start0
 ; CHECK-NEXT:  .long {{.*}}Lxray_sleds_end0

test/CodeGen/ARM/xray-armv7-attribute-instrumentation.ll

 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-NEXT:  nop
 ; CHECK-LABEL: Ltmp1:
 ; CHECK-NEXT:  bx lr
 }
-; CHECK:       .p2align 4
-; CHECK-NEXT:  .long {{.*}}Lxray_fn_idx_synth_0
-; CHECK-NEXT:  .section {{.*}}xray_instr_map{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .long {{.*}}Lxray_sled_0
 ; CHECK:       .long {{.*}}Lxray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0:
-; CHECK:       .section {{.*}}xray_fn_idx{{.*}}
-; CHECK-LABEL: Lxray_fn_idx_synth_0:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .long {{.*}}xray_sleds_start0
 ; CHECK-NEXT:  .long {{.*}}xray_sleds_end0
 

test/CodeGen/Mips/xray-section-group.ll

 ; RUN: llc -filetype=asm -o - -mtriple=mips-unknown-linux-gnu -function-sections < %s | FileCheck %s
 ; RUN: llc -filetype=asm -o - -mtriple=mipsel-unknown-linux-gnu -function-sections < %s | FileCheck %s
 ; RUN: llc -filetype=obj -o %t -mtriple=mips-unknown-linux-gnu -function-sections < %s
 ; RUN: llvm-readobj -sections %t | FileCheck %s --check-prefix=CHECK-OBJ
 ; RUN: llc -filetype=obj -o %t -mtriple=mipsel-unknown-linux-gnu -function-sections < %s
 ; RUN: llvm-readobj -sections %t | FileCheck %s --check-prefix=CHECK-OBJ
 ; RUN: llc -filetype=asm -o - -mtriple=mips64-unknown-linux-gnu -function-sections < %s | FileCheck %s
 ; RUN: llc -filetype=asm -o - -mtriple=mips64el-unknown-linux-gnu -function-sections < %s | FileCheck %s
 ; RUN: llc -filetype=obj -o %t -mtriple=mips64-unknown-linux-gnu -function-sections < %s
 ; RUN: llvm-readobj -sections %t | FileCheck %s --check-prefix=CHECK-OBJ
 ; RUN: llc -filetype=obj -o %t -mtriple=mips64el-unknown-linux-gnu -function-sections < %s
 ; RUN: llvm-readobj -sections %t | FileCheck %s --check-prefix=CHECK-OBJ
 
 define i32 @foo() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK: .section .text.foo,"ax",@progbits
   ret i32 0
-; CHECK: .section xray_instr_map,"a",@progbits
+; CHECK: .section xray_instr_map,"awo",@progbits,.text.foo,unique,1
 }
 
 ; CHECK-OBJ: Section {
 ; CHECK-OBJ:   Name: xray_instr_map
 
 $bar = comdat any
 define i32 @bar() nounwind noinline uwtable "function-instrument"="xray-always" comdat($bar) {
 ; CHECK: .section .text.bar,"axG",@progbits,bar,comdat
   ret i32 1
-; CHECK: .section xray_instr_map,"aG",@progbits,bar,comdat
+; CHECK: .section xray_instr_map,"aGwo",@progbits,bar,comdat,.text.bar,unique,2
 }
 
 ; CHECK-OBJ: Section {
 ; CHECK-OBJ:   Name: xray_instr_map

test/CodeGen/X86/xray-attribute-instrumentation.ll

 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu < %s | FileCheck %s
+; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu \
+; RUN:    -relocation-model=pic < %s | FileCheck %s
 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-darwin-unknown    < %s | FileCheck %s
 
 define i32 @foo() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_0:
-; CHECK-NEXT:  .ascii "\353\t"
+; CHECK:       .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp0:
   ret i32 0
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_1:
-; CHECK-NEXT:  retq
+; CHECK:       retq
 ; CHECK-NEXT:  nopw %cs:512(%rax,%rax)
 }
-; CHECK:       .p2align 4, 0x90
-; CHECK-NEXT:  .quad {{.*}}xray_fn_idx_synth_0
-; CHECK-NEXT:  .section {{.*}}xray_instr_map
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .quad {{.*}}xray_sled_0
 ; CHECK:       .quad {{.*}}xray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0:
-; CHECK:       .section {{.*}}xray_fn_idx
-; CHECK-LABEL: Lxray_fn_idx_synth_0:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .quad {{.*}}xray_sleds_start0
 ; CHECK-NEXT:  .quad {{.*}}xray_sleds_end0
 
 
 ; We test multiple returns in a single function to make sure we're getting all
 ; of them with XRay instrumentation.
 define i32 @bar(i32 %i) nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_2:
-; CHECK-NEXT:  .ascii "\353\t"
+; CHECK:       .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp1:
 Test:
   %cond = icmp eq i32 %i, 0
   br i1 %cond, label %IsEqual, label %NotEqual
 IsEqual:
   ret i32 0
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_3:
-; CHECK-NEXT:  retq
+; CHECK:       retq
 ; CHECK-NEXT:  nopw %cs:512(%rax,%rax)
 NotEqual:
   ret i32 1
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_4:
-; CHECK-NEXT:  retq
+; CHECK:       retq
 ; CHECK-NEXT:  nopw %cs:512(%rax,%rax)
 }
-; CHECK:       .p2align 4, 0x90
-; CHECK-NEXT:  .quad {{.*}}xray_fn_idx_synth_1
-; CHECK-NEXT:  .section {{.*}}xray_instr_map
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start1:
 ; CHECK:       .quad {{.*}}xray_sled_2
 ; CHECK:       .quad {{.*}}xray_sled_3
 ; CHECK:       .quad {{.*}}xray_sled_4
 ; CHECK-LABEL: Lxray_sleds_end1:
-; CHECK:       .section {{.*}}xray_fn_idx
-; CHECK-LABEL: Lxray_fn_idx_synth_1:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .quad {{.*}}xray_sleds_start1
 ; CHECK-NEXT:  .quad {{.*}}xray_sleds_end1

test/CodeGen/X86/xray-custom-log.ll

 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu < %s | FileCheck %s
 
 define i32 @fn() nounwind noinline uwtable "function-instrument"="xray-always" {
     %eventptr = alloca i8
     %eventsize = alloca i32
     store i32 3, i32* %eventsize
     %val = load i32, i32* %eventsize
     call void @llvm.xray.customevent(i8* %eventptr, i32 %val)
     ; CHECK-LABEL: Lxray_event_sled_0:
-    ; CHECK-NEXT:  .ascii "\353\024
-    ; CHECK-NEXT:  pushq %rax
+    ; CHECK:       .byte 0xeb, 0x0f
+    ; CHECK-NEXT:  pushq %rdi
     ; CHECK-NEXT:  movq {{.*}}, %rdi
+    ; CHECK-NEXT:  pushq %rsi
     ; CHECK-NEXT:  movq {{.*}}, %rsi
-    ; CHECK-NEXT:  movabsq $__xray_CustomEvent, %rax
-    ; CHECK-NEXT:  callq *%rax
-    ; CHECK-NEXT:  popq %rax
+    ; CHECK-NEXT:  callq __xray_CustomEvent
+    ; CHECK-NEXT:  popq %rsi
+    ; CHECK-NEXT:  popq %rdi
     ret i32 0
 }
-; CHECK:       .section {{.*}}xray_instr_map
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .quad {{.*}}xray_event_sled_0
 
 declare void @llvm.xray.customevent(i8*, i32)

test/CodeGen/X86/xray-log-args.ll

 ; When logging arguments is specified, emit the entry sled accordingly.
 
 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu < %s | FileCheck %s
 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-darwin-unknown < %s | FileCheck %s
 
 define i32 @callee(i32 %arg) nounwind noinline uwtable "function-instrument"="xray-always" "xray-log-args"="1" {
   ret i32 %arg
 }
 ; CHECK-LABEL: Lxray_sleds_start0:
-; CHECK:  .quad {{\.?}}Lxray_sled_0
-; CHECK:  .quad {{_?}}callee
-; CHECK:  .byte 3
-; CHECK:  .byte 1
-; CHECK:  .{{(zero|space)}} 14
-; CHECK:  .quad {{\.?}}Lxray_sled_1
-; CHECK:  .quad {{_?}}callee
-; CHECK:  .byte 1
-; CHECK:  .byte 1
-; CHECK:  .{{(zero|space)}} 14
+; CHECK:  .quad {{\.?}}Lxray_sled_0
+; CHECK:  .quad {{_?}}callee
+; CHECK:  .byte 0x03
+; CHECK:  .byte 0x01
+; CHECK:  .byte 0x00
+; CHECK:  .{{(zero|space)}}  13
+; CHECK:  .quad {{\.?}}Lxray_sled_1
+; CHECK:  .quad {{_?}}callee
+; CHECK:  .byte 0x01
+; CHECK:  .byte 0x01
+; CHECK:  .byte 0x00
+; CHECK:  .{{(zero|space)}}  13
 
 define i32 @caller(i32 %arg) nounwind noinline uwtable "function-instrument"="xray-always" "xray-log-args"="1" {
   %retval = tail call i32 @callee(i32 %arg)
   ret i32 %retval
 }
 ; CHECK-LABEL: Lxray_sleds_start1:
-; CHECK:  .quad {{\.?}}Lxray_sled_2
-; CHECK:  .quad {{_?}}caller
-; CHECK:  .byte 3
-; CHECK:  .byte 1
-; CHECK:  .{{(zero|space)}} 14
-; CHECK:  .quad {{\.?}}Lxray_sled_3
-; CHECK:  .quad {{_?}}caller
-; CHECK:  .byte 2
-; CHECK:  .byte 1
-; CHECK:  .{{(zero|space)}} 14
+; CHECK:  .quad {{\.?}}Lxray_sled_2
+; CHECK:  .quad {{_?}}caller
+; CHECK:  .byte 0x03
+; CHECK:  .byte 0x01
+; CHECK:  .byte 0x00
+; CHECK:  .{{(zero|space)}}  13
+; CHECK:  .quad {{\.?}}Lxray_sled_3
+; CHECK:  .quad {{_?}}caller
+; CHECK:  .byte 0x02
+; CHECK:  .byte 0x01
+; CHECK:  .byte 0x00
+; CHECK:  .{{(zero|space)}}  13

test/CodeGen/X86/xray-loop-detection.ll

 Exit:
   %retval = phi i32 [%indvar, %Test]
   ret i32 %retval
 }
 
 ; CHECK-LABEL: xray_sled_0:
 ; CHECK-NEXT:  .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp0:
 

test/CodeGen/X86/xray-section-group.ll

 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu -function-sections < %s | FileCheck %s
 ; RUN: llc -filetype=obj -o %t -mtriple=x86_64-unknown-linux-gnu -function-sections < %s
 ; RUN: llvm-objdump -triple x86_64-unknown-linux-gnu -disassemble-all %t | FileCheck %s --check-prefix=CHECK-OBJ
 
 define i32 @foo() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK: .section .text.foo,"ax",@progbits
   ret i32 0
-; CHECK: .section xray_instr_map,"a",@progbits
+; CHECK: .section xray_instr_map,"awo",@progbits,.text.foo,unique,1
 }
 
 $bar = comdat any
 define i32 @bar() nounwind noinline uwtable "function-instrument"="xray-always" comdat($bar) {
 ; CHECK: .section .text.bar,"axG",@progbits,bar,comdat
   ret i32 1
-; CHECK: .section xray_instr_map,"aG",@progbits,bar,comdat
+; CHECK: .section xray_instr_map,"aGwo",@progbits,bar,comdat,.text.bar,unique,2
 }
 
 ; CHECK-OBJ:      section xray_instr_map:

test/CodeGen/X86/xray-tail-call-sled.ll

 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-unknown-linux-gnu < %s | FileCheck %s
 ; RUN: llc -filetype=asm -o - -mtriple=x86_64-darwin-unknown    < %s | FileCheck %s
 
 define i32 @callee() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_0:
-; CHECK-NEXT:  .ascii "\353\t"
+; CHECK:       .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp0:
   ret i32 0
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_1:
-; CHECK-NEXT:  retq
+; CHECK:       retq
 ; CHECK-NEXT:  nopw %cs:512(%rax,%rax)
 }
-; CHECK:       .p2align 4, 0x90
-; CHECK-NEXT:  .quad {{.*}}xray_fn_idx_synth_0{{.*}}
-; CHECK-NEXT:  .section {{.*}}xray_instr_map
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start0:
 ; CHECK:       .quad {{.*}}xray_sled_0
 ; CHECK:       .quad {{.*}}xray_sled_1
 ; CHECK-LABEL: Lxray_sleds_end0:
-; CHECK-NEXT:  .section {{.*}}xray_fn_idx
-; CHECK-LABEL: Lxray_fn_idx_synth_0:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .quad {{.*}}xray_sleds_start0
 ; CHECK-NEXT:  .quad {{.*}}xray_sleds_end0
 
 define i32 @caller() nounwind noinline uwtable "function-instrument"="xray-always" {
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_2:
-; CHECK-NEXT:  .ascii "\353\t"
+; CHECK:       .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp1:
 ; CHECK:       .p2align 1, 0x90
 ; CHECK-LABEL: Lxray_sled_3:
 ; CHECK-NEXT:  .ascii "\353\t"
 ; CHECK-NEXT:  nopw 512(%rax,%rax)
-; CHECK-LABEL: Ltmp2:
   %retval = tail call i32 @callee()
 ; CHECK:       jmp {{.*}}callee {{.*}}# TAILCALL
   ret i32 %retval
 }
-; CHECK:       .p2align 4, 0x90
-; CHECK-NEXT:  .quad {{.*}}xray_fn_idx_synth_1{{.*}}
+; CHECK-LABEL: xray_instr_map
 ; CHECK-LABEL: Lxray_sleds_start1:
 ; CHECK:       .quad {{.*}}xray_sled_2
 ; CHECK:       .quad {{.*}}xray_sled_3
 ; CHECK-LABEL: Lxray_sleds_end1:
-; CHECK:       .section {{.*}}xray_fn_idx
-; CHECK-LABEL: Lxray_fn_idx_synth_1:
+; CHECK-LABEL: xray_fn_idx
 ; CHECK:       .quad {{.*}}xray_sleds_start1
 ; CHECK:       .quad {{.*}}xray_sleds_end1