Add XRay flags to Clang

[dberris]

Summary

This depends on D19904 to land in LLVM so that the attributes for XRay
instrumentation in the IR are supported and generate the appropriate nop sleds
in function entry/exits. We implement two flags to control the XRay behaviour:

• -fxray-instrument: enables XRay annotation of IR
• -fxray-instruction-threshold: configures the threshold for function size (looking at IR instructions), and allow LLVM to decide whether to add the nop sleds later on in the process.

Also Depends on D21612 for the XRay runtime dependency.

[echristo]

Couple of inline comments. Also adding Aaron since he normally reviews the attributes :)

-eric

include/clang/Driver/Options.td

788

fno-xray-instrument

include/clang/Frontend/CodeGenOptions.def

75

Would it make sense to turn -finstrument-functions to -finstrument-functions=<type> to avoid tons of instrumentation type arguments? If we want to keep -fxray-instrument still it can then just be an alias.

[aaron.ballman]

include/clang/Basic/Attr.td

434

Would this also make sense for ObjC methods?

435

This gives a diagnostic suggesting ObjC methods, so this should either be removed (it will diagnose properly without it) or the Subjects list should be altered to include ObjC methods.

436

No new undocumented attributes. Also, TODO usually don't contain user names when we add them.

439

Same comments here as above.

I kind of think these are a good candidates to combine into a single attribute with multiple spellings.

def XRayInstrument : InheritableAttr {
  let Spellings = [GNU<"xray_always_instrument">, ...,
                           GNU<"xray_never_instrument">, ...];
}

I think this because I am guessing they will always appertain to the same subjects and take the same (lack of) arguments. You can tell which attribute you have at runtime by looking at the semantic spelling of the attribute. You could add an accessor to get that in a more friendly fashion if desired.

If my guess is wrong, feel free to ignore this suggestion. :-)

test/Sema/xray-always-instrument-attr.c

5

Should also be a test showing that it doesn't accept args. Also, CodeGen tests.

[dberris]

• Merge branch 'master' of http://llvm.org/git/clang into xray
• Merge branch 'master' of http://llvm.org/git/clang into xray
• Add runtime support for XRay

[dberris]

• Address review comments; see details.

[dberris]

include/clang/Basic/Attr.td

434

Yes it does, done.

435

Done the latter to include ObjC methods.

439

This actually makes a lot of sense. I've done some of the consolidation to make it a lot easier to check in the code and to expand later on too. I suppose other attributes that are XRay-specific with additional values/arguments would make a better candidate for additional attributes. Will cross that bridge when we get to that part of the implementation. :)

include/clang/Frontend/CodeGenOptions.def

75

I'm open to this idea, except this might break compatibility with GCC (as this seems to be a gcc-inspired flag).

I'm happy to explore consolidating the different instrumentation flags under -finstrument= which then has namespace values internally (profile-..., xray-...) for different modes, etc. For now the simplest thing to do might be to have XRay do this independently first, then have a cleanup later to consolidate flags on the driver and deprecate the flags.

[aaron.ballman]

It looks like you ran clang-format over the entire file in some instances, causing a lot of unrelated changes to creep in. Can you back those changes out and only run clang-format over just your patch instead (http://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting)?

include/clang/Basic/Attr.td

436

You can drop the WarnDiag and string literal; the defaults already do the right thing.

include/clang/Basic/AttrDocs.td

2457

Let's make sure that this patch and D19908 do not conflict, because they seem to be implementing competing semantics.

lib/CodeGen/CodeGenFunction.cpp

381

Can't you rangify this loop?

402

Any particular reason this isn't a const function?

696

This is not the right way to do this. ;-)

You should check the *semantic* spelling instead. e.g.,

if (const auto *A = D->getAttr<XRayInstrumentAttr>()) {
  switch (A->getSemanticSpelling()) {
  case XRayInstrumentAttr::GNU_blahblah:
  case XRayInstrumentAttr::CXX11_clang_blahblah:
  }
}

Alternatively (and I think this may be cleaner), you could add an additional member in Attr.td that exposes this as two Boolean getters bool alwaysInstrument() const and bool neverInstrument() const that do the semantic spelling switch.

[dberris]

• Undo formatting changes
• Re-apply changes post-review
• Address more comments

include/clang/Basic/Attr.td

436

I tried, but I'm getting this error:

.../Basic/Attr.td:436:18: error: Could not deduce diagnostic argument for Attr subjects

let Subjects = SubjectList<[CXXMethod, ObjCMethod, Function]>;

include/clang/Basic/AttrDocs.td

2457

Good catch. Let me comment on D19908 to see if it competes with how we define XRay instrumentation. As far as I can tell from the LLVM side, those should not interfere with each other.

lib/CodeGen/CodeGenFunction.cpp

381

Probably in a different change. :) For now I've reverted the formatting changes.

402

No good reason. :) Good catch, fixed.

696

I thought this was a little weird, thanks for pointing this out!

I've used accessors instead, maybe it looks better now?

[dberris]

Rebase

[dberris]

Rebase harder.

[aaron.ballman]

include/clang/Basic/Attr.td

436

Then ClangAttrEmitter.cpp needs to be updated as well (this is a reasonable case to automatically support, I think).

include/clang/Basic/AttrDocs.td

2457

I get the impression they do conflict because they have overlapping functionality (both provide prologue space for runtime patching, for instance). It would be best to have only one attribute if we can do it.

lib/CodeGen/CodeGenFunction.cpp

696

Yes, this is the right way to solve it. Thanks!

However, you should elide the braces when there's only a single statement (per our coding standards).

[dberris]

include/clang/Basic/Attr.td

436

That makes sense, I can do it in another change that's more focused on just that if you don't mind?

include/clang/Basic/AttrDocs.td

2457

I've explored that option (using patchable-function instead) and while I haven't excluded that outright, there needs to be a discussion about how we're going to implement complimentary instrumentation functionality that do nop-sled inserting. I suspect that has to happen in D19904 where we can still make those changes to instead use patchable-function.

The response to my comment in D19908 seems to be that there should be no conflicts in the platforms XRay currently cares about -- but that if we do ever want to get XRay implemented/supported in x86 32-bit, then we're going to have issues. This suggests that a harder look at patchable-function and exclusive modes (or special-casing the supported configurations) might need to happen sooner than later.

[dberris]

• Formatting changes

[aaron.ballman]

include/clang/Basic/Attr.td

436

That's fine by me (it's also fine to do as part of this patch since this patch will be what exercises that functionality anyway). Your choice. :-)

include/clang/Basic/AttrDocs.td

2457

I think that makes sense. I mostly want to avoid having multiple user-facing attributes that do very similar operations under different names. That makes it confusing for users, and makes for possibly more-complicated code (are these mutually exclusive attributes, etc). I am less worried about whether the backend has multiple attributes, though I suspect the same concerns may apply there as well.

[dberris]

• Check first whether D is actually not nullptr

[dberris]

Undo previous change -- updated the wrong patch. :(

[dberris]

• Check D is valid before using it

[rnk]

Looks good from a driver perspective. Aaron, you're happy with the attribute spelling stuff, right?

It looks like you can actually land this before landing the two dependent changes you mention and the tests will pass. I'd go ahead and land it to avoid the need to rebase, but it's up to you.

include/clang/Basic/AttrDocs.td

2459

These paragraphs can be wrapped to 80 cols like the others.

[aaron.ballman]

In D20352#477038, @rnk wrote:

Looks good from a driver perspective. Aaron, you're happy with the attribute spelling stuff, right?

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

[rnk]

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

[echristo]

In D20352#477203, @rnk wrote:

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

Agreed.

-eric

[aaron.ballman]

In D20352#477203, @rnk wrote:

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

Okay, that was the key piece I was missing -- I was under the impression that this was two different ways to achieve the same task. Thank you for the explanation (and sorry if I was being dense). :-)

When the MS hot patching attribute lands, that will be the time to discuss and document conflicts between the two (I am assuming you can't enable XRay and MS hot patching for the same function, which may be an incorrect assumption). So I think this LGTM now.

[dberris]

Thanks -- I don't have commit powers yet, do either of you mind landing this for me?

Cheers

[aaron.ballman]

I applied your patch locally and got a few compile errors with MSVC 2015:

'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 3187
'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 4608

There is no hasArg() overload that accepts a bool argument; perhaps that's a feature of one of the dependencies this patch depends on?

[dberris]

Rebase

[dberris]

• Use hasFlag instead of hasArg

[dberris]

It looks like I was using hasArg instead of hasFlag. The dangerous part here is that OptSpecifier has an unsigned non-explicit argument, and I suspect bool is being promoted to unsigned silently with clang/gcc.

[aaron.ballman]

In D20352#482590, @dberris wrote:

It looks like I was using hasArg instead of hasFlag. The dangerous part here is that OptSpecifier has an unsigned non-explicit argument, and I suspect bool is being promoted to unsigned silently with clang/gcc.

Still not quite right with the latest patch:

C2664 'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 4608

I changed this to use hasFlag() instead, and then it compiled and tests passed. I've commit in r275330.

include/clang/Basic/Attr.td

 }
 
 def AlwaysInline : InheritableAttr {
   let Spellings = [GCC<"always_inline">, Keyword<"__forceinline">];
   let Subjects = SubjectList<[Function]>;
   let Documentation = [Undocumented];
 }
 
+def XRayInstrument : InheritableAttr {
+  let Spellings = [GNU<"xray_always_instrument">,
+                   CXX11<"clang", "xray_always_instrument">,
+                   GNU<"xray_never_instrument">,
+                   CXX11<"clang", "xray_never_instrument">];
+  let Subjects = SubjectList<[CXXMethod, ObjCMethod, Function], WarnDiag,
+                              "ExpectedFunctionOrMethod">;
+  let Accessors = [Accessor<"alwaysXRayInstrument",
+                     [GNU<"xray_always_instrument">,
+                      CXX11<"clang", "xray_always_instrument">]>,
+                   Accessor<"neverXRayInstrument",
+                     [GNU<"xray_never_instrument">,
+                      CXX11<"clang", "xray_never_instrument">]>];
+  let Documentation = [XRayDocs];
+}
+
 def TLSModel : InheritableAttr {
   let Spellings = [GCC<"tls_model">];
   let Subjects = SubjectList<[TLSVar], ErrorDiag, "ExpectedTLSVar">;
   let Args = [StringArgument<"Model">];
   let Documentation = [TLSModelDocs];
 }
 
 def AnalyzerNoReturn : InheritableAttr {

include/clang/Basic/AttrDocs.td

 In RenderScript, ``kernel`` functions are used to express data-parallel
 computations.  The RenderScript runtime efficiently parallelizes ``kernel``
 functions to run on computational resources such as multi-core CPUs and GPUs.
 See the RenderScript_ documentation for more information.
 
 .. _RenderScript: https://developer.android.com/guide/topics/renderscript/compute.html
   }];
 }
+
+def XRayDocs : Documentation {
+  let Category = DocCatFunction;
+  let Content = [{
+``__attribute__((xray_always_instrument))`` or ``[[clang:xray_always_instrument]]`` is used to mark member functions (in C++), methods (in Objective C), and free functions (in C, C++, and Objective C) to be instrumented with XRay. This will cause the function to always have space at the beginning and exit points to allow for runtime patching.
+
+Conversely, ``__attribute__((xray_never_instrument))`` or ``[[clang:xray_never_instrument]]`` will inhibit the insertion of these instrumentation points.
+
+If a function has neither of these attributes, they become subject to the XRay heuristics used to determine whether a function should be instrumented or otherwise.
+  }];
+}

include/clang/Driver/Options.td

def finline_functions : Flag<["-"], "finline-functions">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Inline suitable functions">;
 def finline_hint_functions: Flag<["-"], "finline-hint-functions">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Inline functions wich are (explicitly or implicitly) marked inline">;
 def finline : Flag<["-"], "finline">, Group<clang_ignored_f_Group>;
 def finput_charset_EQ : Joined<["-"], "finput-charset=">, Group<f_Group>;
 def fexec_charset_EQ : Joined<["-"], "fexec-charset=">, Group<f_Group>;
 def finstrument_functions : Flag<["-"], "finstrument-functions">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Generate calls to instrument function entry and exit">;
+
+def fxray_instrument : Flag<["-"], "fxray-instrument">, Group<f_Group>,
+  Flags<[CC1Option]>,
+  HelpText<"Generate XRay instrumentation sleds on function entry and exit">;
+def fnoxray_instrument : Flag<["-"], "fno-xray-instrument">, Group<f_Group>,
+  Flags<[CC1Option]>;
+
+def fxray_instruction_threshold_EQ :
+  JoinedOrSeparate<["-"], "fxray-instruction-threshold=">,
+  Group<f_Group>, Flags<[CC1Option]>,
+  HelpText<"Sets the minimum function size to instrument with XRay">;
+def fxray_instruction_threshold_ :
+  JoinedOrSeparate<["-"], "fxray-instruction-threshold">,
+  Group<f_Group>, Flags<[CC1Option]>;
+
 def flat__namespace : Flag<["-"], "flat_namespace">;
 def flax_vector_conversions : Flag<["-"], "flax-vector-conversions">, Group<f_Group>;
 def flimited_precision_EQ : Joined<["-"], "flimited-precision=">, Group<f_Group>;
 def flto_EQ : Joined<["-"], "flto=">, Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Set LTO mode to either 'full' or 'thin'">;
 def flto : Flag<["-"], "flto">, Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group<f_Group>,

include/clang/Frontend/CodeGenOptions.def

 CODEGENOPT(EmulatedTLS       , 1, 0) ///< Set when -femulated-tls is enabled.
 /// \brief FP_CONTRACT mode (on/off/fast).
 ENUM_CODEGENOPT(FPContractMode, FPContractModeKind, 2, FPC_On)
 /// \brief Embed Bitcode mode (off/all/bitcode/marker).
 ENUM_CODEGENOPT(EmbedBitcode, EmbedBitcodeKind, 2, Embed_Off)
 CODEGENOPT(ForbidGuardVariables , 1, 0) ///< Issue errors if C++ guard variables
                                         ///< are required.
 CODEGENOPT(FunctionSections  , 1, 0) ///< Set when -ffunction-sections is enabled.
 CODEGENOPT(InstrumentFunctions , 1, 0) ///< Set when -finstrument-functions is
                                        ///< enabled.
+
+CODEGENOPT(XRayInstrumentFunctions , 1, 0) ///< Set when -fxray-instrument is
+                                           ///< enabled.
+
+///< Set the minimum number of instructions in a function to determine selective
+///< XRay instrumentation.
+VALUE_CODEGENOPT(XRayInstructionThreshold , 32, 200)
+
 CODEGENOPT(InstrumentForProfiling , 1, 0) ///< Set when -pg is enabled.
 CODEGENOPT(LessPreciseFPMAD  , 1, 0) ///< Enable less precise MAD instructions to
                                      ///< be generated.
 CODEGENOPT(PrepareForLTO     , 1, 0) ///< Set when -flto is enabled on the
                                      ///< compile step.
 CODEGENOPT(EmitSummaryIndex, 1, 0)   ///< Set when -flto=thin is enabled on the
                                      ///< compile step.
 CODEGENOPT(IncrementalLinkerCompatible, 1, 0) ///< Emit an object file which can

lib/CodeGen/CodeGenFunction.h

public:
   /// should call destructors on members and base classes in reverse
   /// order of their construction.
   void EnterDtorCleanups(const CXXDestructorDecl *Dtor, CXXDtorType Type);
 
   /// ShouldInstrumentFunction - Return true if the current function should be
   /// instrumented with __cyg_profile_func_* calls
   bool ShouldInstrumentFunction();
 
+  /// ShouldXRayInstrument - Return true if the current function should be
+  /// instrumented with XRay nop sleds.
+  bool ShouldXRayInstrumentFunction() const;
+
   /// EmitFunctionInstrumentation - Emit LLVM code to call the specified
   /// instrumentation function with the current function and the call site, if
   /// function instrumentation is enabled.
   void EmitFunctionInstrumentation(const char *Fn);
 
   /// EmitMCountInstrumentation - Emit call to .mcount.
   void EmitMCountInstrumentation();
 

lib/CodeGen/CodeGenFunction.cpp

void CodeGenFunction::FinishFunction(SourceLocation EndLoc) {
   EmitIfUsed(*this, EHResumeBlock);
   EmitIfUsed(*this, TerminateLandingPad);
   EmitIfUsed(*this, TerminateHandler);
   EmitIfUsed(*this, UnreachableBlock);
 
   if (CGM.getCodeGenOpts().EmitDeclMetadata)
     EmitDeclMetadata();
 
   for (SmallVectorImpl<std::pair<llvm::Instruction *, llvm::Value *> >::iterator
            I = DeferredReplacements.begin(),
            E = DeferredReplacements.end();
        I != E; ++I) {
     I->first->replaceAllUsesWith(I->second);
     I->first->eraseFromParent();
   }
 }
 
 /// ShouldInstrumentFunction - Return true if the current function should be
 /// instrumented with __cyg_profile_func_* calls
 bool CodeGenFunction::ShouldInstrumentFunction() {
   if (!CGM.getCodeGenOpts().InstrumentFunctions)
     return false;
   if (!CurFuncDecl || CurFuncDecl->hasAttr<NoInstrumentFunctionAttr>())
     return false;
   return true;
 }
 
+/// ShouldXRayInstrument - Return true if the current function should be
+/// instrumented with XRay nop sleds.
+bool CodeGenFunction::ShouldXRayInstrumentFunction() const {
+  return CGM.getCodeGenOpts().XRayInstrumentFunctions;
+}
+
 /// EmitFunctionInstrumentation - Emit LLVM code to call the specified
 /// instrumentation function with the current function and the call site, if
 /// function instrumentation is enabled.
 void CodeGenFunction::EmitFunctionInstrumentation(const char *Fn) {
   auto NL = ApplyDebugLocation::CreateArtificial(*this);
   // void __cyg_profile_func_{enter,exit} (void *this_fn, void *call_site);
   llvm::PointerType *PointerTy = Int8PtrTy;
   llvm::Type *ProfileFuncArgs[] = { PointerTy, PointerTy };
if (SanOpts.hasOneOf(SanitizerKind::Address | SanitizerKind::KernelAddress))
     Fn->addFnAttr(llvm::Attribute::SanitizeAddress);
   if (SanOpts.has(SanitizerKind::Thread))
     Fn->addFnAttr(llvm::Attribute::SanitizeThread);
   if (SanOpts.has(SanitizerKind::Memory))
     Fn->addFnAttr(llvm::Attribute::SanitizeMemory);
   if (SanOpts.has(SanitizerKind::SafeStack))
     Fn->addFnAttr(llvm::Attribute::SafeStack);
 
+  // Apply xray attributes to the function (as a string, for now)
+  if (D && ShouldXRayInstrumentFunction()) {
+    if (const auto *XRayAttr = D->getAttr<XRayInstrumentAttr>()) {
+      if (XRayAttr->alwaysXRayInstrument())
+        Fn->addFnAttr("function-instrument", "xray-always");
+      if (XRayAttr->neverXRayInstrument())
+        Fn->addFnAttr("function-instrument", "xray-never");
+    } else {
+      Fn->addFnAttr(
+          "xray-instruction-threshold",
+          llvm::itostr(CGM.getCodeGenOpts().XRayInstructionThreshold));
+    }
+  }
+
   // Pass inline keyword to optimizer if it appears explicitly on any
   // declaration. Also, in the case of -fno-inline attach NoInline
   // attribute to all functions that are not marked AlwaysInline, or
   // to all functions that are not marked inline or implicitly inline
   // in the case of -finline-hint-functions.
   if (const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(D)) {
     const CodeGenOptions& CodeGenOpts = CGM.getCodeGenOpts();
     if (!CodeGenOpts.NoInline) {

lib/Driver/Tools.cpp

static bool addSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
   }
   // If there is a static runtime with no dynamic list, force all the symbols
   // to be dynamic to be sure we export sanitizer interface functions.
   if (AddExportDynamic)
     CmdArgs.push_back("-export-dynamic");
   return !StaticRuntimes.empty();
 }
 
+static bool addXRayRuntime(const ToolChain &TC, const ArgList &Args,
+                           ArgStringList &CmdArgs) {
+  if (Args.hasFlag(options::OPT_fxray_instrument,
+                   options::OPT_fnoxray_instrument, false)) {
+    CmdArgs.push_back("-whole-archive");
+    CmdArgs.push_back(TC.getCompilerRTArgString(Args, "xray", false));
+    CmdArgs.push_back("-no-whole-archive");
+    return true;
+  }
+  return false;
+}
+
+static void linkXRayRuntimeDeps(const ToolChain &TC, ArgStringList &CmdArgs) {
+  CmdArgs.push_back("--no-as-needed");
+  CmdArgs.push_back("-lpthread");
+  CmdArgs.push_back("-lrt");
+  CmdArgs.push_back("-lm");
+  CmdArgs.push_back("-latomic");
+  if (TC.getTriple().getOS() != llvm::Triple::FreeBSD)
+    CmdArgs.push_back("-ldl");
+}
+
 static bool areOptimizationsEnabled(const ArgList &Args) {
   // Find the last -O arg and see if it is non-zero.
   if (Arg *A = Args.getLastArg(options::OPT_O_Group))
     return !A->getOption().matches(options::OPT_O0);
   // Defaults to -O0.
   return false;
 }
 
#endif
   }
 
   if (!Args.hasFlag(options::OPT_funique_section_names,
                     options::OPT_fno_unique_section_names, true))
     CmdArgs.push_back("-fno-unique-section-names");
 
   Args.AddAllArgs(CmdArgs, options::OPT_finstrument_functions);
 
+  if (Args.hasArg(options::OPT_fxray_instrument,
+                  options::OPT_fnoxray_instrument, false)) {
+    CmdArgs.push_back("-fxray-instrument");
+    if (Arg *A = Args.getLastArg(options::OPT_fxray_instruction_threshold_,
+                                 options::OPT_fxray_instruction_threshold_EQ)) {
+      CmdArgs.push_back("-fxray-instruction-threshold");
+      CmdArgs.push_back(A->getValue());
+    }
+  }
+
   addPGOAndCoverageFlags(C, D, Output, Args, CmdArgs);
 
   // Add runtime flag for PS4 when PGO or Coverage are enabled.
   if (getToolChain().getTriple().isPS4CPU())
     addPS4ProfileRTArgs(getToolChain(), Args, CmdArgs);
 
   // Pass options for controlling the default header search paths.
   if (Args.hasArg(options::OPT_nostdinc)) {
void gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
 
   if (D.isUsingLTO())
     AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
     CmdArgs.push_back("--no-demangle");
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
+  bool NeedsXRayDeps = addXRayRuntime(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
   // The profile runtime also needs access to system libraries.
   getToolChain().addProfileRTLibs(Args, CmdArgs);
 
   if (D.CCCIsCXX() &&
       !Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
     bool OnlyLibstdcxxStatic = Args.hasArg(options::OPT_static_libstdcxx) &&
                                !Args.hasArg(options::OPT_static);
void gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
   if (!Args.hasArg(options::OPT_nostdlib)) {
     if (!Args.hasArg(options::OPT_nodefaultlibs)) {
       if (Args.hasArg(options::OPT_static))
         CmdArgs.push_back("--start-group");
 
       if (NeedsSanitizerDeps)
         linkSanitizerRuntimeDeps(ToolChain, CmdArgs);
 
+      if (NeedsXRayDeps)
+        linkXRayRuntimeDeps(ToolChain, CmdArgs);
+
       bool WantPthread = Args.hasArg(options::OPT_pthread) ||
                          Args.hasArg(options::OPT_pthreads);
 
       if (Args.hasFlag(options::OPT_fopenmp, options::OPT_fopenmp_EQ,
                        options::OPT_fno_openmp, false)) {
         // OpenMP runtimes implies pthreads when using the GNU toolchain.
         // FIXME: Does this really make sense for all GNU toolchains?
         WantPthread = true;

lib/Frontend/CompilerInvocation.cpp

for (const auto &A : Args) {
         Opts.CmdArgs.insert(Opts.CmdArgs.end(), ArgStr.begin(), ArgStr.end());
         // using \00 to seperate each commandline options.
         Opts.CmdArgs.push_back('\0');
       }
     }
   }
 
   Opts.InstrumentFunctions = Args.hasArg(OPT_finstrument_functions);
+  Opts.XRayInstrumentFunctions = Args.hasArg(OPT_fxray_instrument);
+  Opts.XRayInstructionThreshold =
+      getLastArgIntValue(Args, OPT_fxray_instruction_threshold_, 200, Diags);
   Opts.InstrumentForProfiling = Args.hasArg(OPT_pg);
   Opts.EmitOpenCLArgMetadata = Args.hasArg(OPT_cl_kernel_arg_info);
   Opts.CompressDebugSections = Args.hasArg(OPT_compress_debug_sections);
   Opts.RelaxELFRelocations = Args.hasArg(OPT_mrelax_relocations);
   Opts.DebugCompilationDir = Args.getLastArgValue(OPT_fdebug_compilation_dir);
   for (auto A : Args.filtered(OPT_mlink_bitcode_file, OPT_mlink_cuda_bitcode)) {
     unsigned LinkFlags = llvm::Linker::Flags::None;
     if (A->getOption().matches(OPT_mlink_cuda_bitcode))

lib/Sema/SemaDeclAttr.cpp

static void ProcessDeclAttribute(Sema &S, Scope *scope, Decl *D,
 
   // Type safety attributes.
   case AttributeList::AT_ArgumentWithTypeTag:
     handleArgumentWithTypeTagAttr(S, D, Attr);
     break;
   case AttributeList::AT_TypeTagForDatatype:
     handleTypeTagForDatatypeAttr(S, D, Attr);
     break;
-
   case AttributeList::AT_RenderScriptKernel:
     handleSimpleAttribute<RenderScriptKernelAttr>(S, D, Attr);
     break;
+  // XRay attributes.
+  case AttributeList::AT_XRayInstrument:
+    handleSimpleAttribute<XRayInstrumentAttr>(S, D, Attr);
+    break;
   }
 }
 
 /// ProcessDeclAttributeList - Apply all the decl attributes in the specified
 /// attribute list to the specified decl, ignoring any type attributes.
 void Sema::ProcessDeclAttributeList(Scope *S, Decl *D,
                                     const AttributeList *AttrList,
                                     bool IncludeCXX11Attributes) {

test/CodeGen/xray-attributes-supported.cpp

+// RUN: %clang_cc1 %s -fxray-instrument -std=c++11 -x c++ -emit-llvm -o - -triple x86_64-unknown-linux-gnu | FileCheck %s
+
+// Make sure that the LLVM attribute for XRay-annotated functions do show up.
+[[clang::xray_always_instrument]] void foo() {
+// CHECK: define void @_Z3foov() #0
+};
+
+[[clang::xray_never_instrument]] void bar() {
+// CHECK: define void @_Z3barv() #1
+};
+
+// CHECK: #0 = {{.*}}"function-instrument"="xray-always"
+// CHECK: #1 = {{.*}}"function-instrument"="xray-never"

test/Sema/xray-always-instrument-attr.c

+// RUN: %clang_cc1 %s -verify -fsyntax-only -std=c11
+void foo() __attribute__((xray_always_instrument));
+
+struct __attribute__((xray_always_instrument)) a { int x; }; // expected-warning {{'xray_always_instrument' attribute only applies to functions and methods}}
+
+void bar() __attribute__((xray_always_instrument("not-supported"))); // expected-error {{'xray_always_instrument' attribute takes no arguments}}

test/Sema/xray-always-instrument-attr.cpp

+// RUN: %clang_cc1 %s -verify -fsyntax-only -std=c++11 -x c++
+void foo [[clang::xray_always_instrument]] ();
+
+struct [[clang::xray_always_instrument]] a { int x; }; // expected-warning {{'xray_always_instrument' attribute only applies to functions and methods}}
+
+class b {
+ void c [[clang::xray_always_instrument]] ();
+};
+
+void baz [[clang::xray_always_instrument("not-supported")]] (); // expected-error {{'xray_always_instrument' attribute takes no arguments}}