Add XRay flags to Clang

[dberris]

Summary

This depends on D19904 to land in LLVM so that the attributes for XRay
instrumentation in the IR are supported and generate the appropriate nop sleds
in function entry/exits. We implement two flags to control the XRay behaviour:

• -fxray-instrument: enables XRay annotation of IR
• -fxray-instruction-threshold: configures the threshold for function size (looking at IR instructions), and allow LLVM to decide whether to add the nop sleds later on in the process.

Also Depends on D21612 for the XRay runtime dependency.

[echristo]

Couple of inline comments. Also adding Aaron since he normally reviews the attributes :)

-eric

include/clang/Driver/Options.td

762

fno-xray-instrument

include/clang/Frontend/CodeGenOptions.def

75

Would it make sense to turn -finstrument-functions to -finstrument-functions=<type> to avoid tons of instrumentation type arguments? If we want to keep -fxray-instrument still it can then just be an alias.

[aaron.ballman]

include/clang/Basic/Attr.td

434

Would this also make sense for ObjC methods?

435

This gives a diagnostic suggesting ObjC methods, so this should either be removed (it will diagnose properly without it) or the Subjects list should be altered to include ObjC methods.

436

No new undocumented attributes. Also, TODO usually don't contain user names when we add them.

439

Same comments here as above.

I kind of think these are a good candidates to combine into a single attribute with multiple spellings.

def XRayInstrument : InheritableAttr {
  let Spellings = [GNU<"xray_always_instrument">, ...,
                           GNU<"xray_never_instrument">, ...];
}

I think this because I am guessing they will always appertain to the same subjects and take the same (lack of) arguments. You can tell which attribute you have at runtime by looking at the semantic spelling of the attribute. You could add an accessor to get that in a more friendly fashion if desired.

If my guess is wrong, feel free to ignore this suggestion. :-)

test/Sema/xray-always-instrument-attr.c

5

Should also be a test showing that it doesn't accept args. Also, CodeGen tests.

[dberris]

• Merge branch 'master' of http://llvm.org/git/clang into xray
• Merge branch 'master' of http://llvm.org/git/clang into xray
• Add runtime support for XRay

[dberris]

• Address review comments; see details.

[dberris]

include/clang/Basic/Attr.td

434

Yes it does, done.

435

Done the latter to include ObjC methods.

439

This actually makes a lot of sense. I've done some of the consolidation to make it a lot easier to check in the code and to expand later on too. I suppose other attributes that are XRay-specific with additional values/arguments would make a better candidate for additional attributes. Will cross that bridge when we get to that part of the implementation. :)

include/clang/Frontend/CodeGenOptions.def

75

I'm open to this idea, except this might break compatibility with GCC (as this seems to be a gcc-inspired flag).

I'm happy to explore consolidating the different instrumentation flags under -finstrument= which then has namespace values internally (profile-..., xray-...) for different modes, etc. For now the simplest thing to do might be to have XRay do this independently first, then have a cleanup later to consolidate flags on the driver and deprecate the flags.

[aaron.ballman]

It looks like you ran clang-format over the entire file in some instances, causing a lot of unrelated changes to creep in. Can you back those changes out and only run clang-format over just your patch instead (http://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting)?

include/clang/Basic/Attr.td

436

You can drop the WarnDiag and string literal; the defaults already do the right thing.

include/clang/Basic/AttrDocs.td

2455

Let's make sure that this patch and D19908 do not conflict, because they seem to be implementing competing semantics.

lib/CodeGen/CodeGenFunction.cpp

381

Can't you rangify this loop?

402

Any particular reason this isn't a const function?

691

This is not the right way to do this. ;-)

You should check the *semantic* spelling instead. e.g.,

if (const auto *A = D->getAttr<XRayInstrumentAttr>()) {
  switch (A->getSemanticSpelling()) {
  case XRayInstrumentAttr::GNU_blahblah:
  case XRayInstrumentAttr::CXX11_clang_blahblah:
  }
}

Alternatively (and I think this may be cleaner), you could add an additional member in Attr.td that exposes this as two Boolean getters bool alwaysInstrument() const and bool neverInstrument() const that do the semantic spelling switch.

[dberris]

• Undo formatting changes
• Re-apply changes post-review
• Address more comments

include/clang/Basic/Attr.td

436

I tried, but I'm getting this error:

.../Basic/Attr.td:436:18: error: Could not deduce diagnostic argument for Attr subjects

let Subjects = SubjectList<[CXXMethod, ObjCMethod, Function]>;

include/clang/Basic/AttrDocs.td

2455

Good catch. Let me comment on D19908 to see if it competes with how we define XRay instrumentation. As far as I can tell from the LLVM side, those should not interfere with each other.

lib/CodeGen/CodeGenFunction.cpp

381

Probably in a different change. :) For now I've reverted the formatting changes.

402

No good reason. :) Good catch, fixed.

691

I thought this was a little weird, thanks for pointing this out!

I've used accessors instead, maybe it looks better now?

[dberris]

Rebase

[dberris]

Rebase harder.

[aaron.ballman]

include/clang/Basic/Attr.td

436

Then ClangAttrEmitter.cpp needs to be updated as well (this is a reasonable case to automatically support, I think).

include/clang/Basic/AttrDocs.td

2455

I get the impression they do conflict because they have overlapping functionality (both provide prologue space for runtime patching, for instance). It would be best to have only one attribute if we can do it.

lib/CodeGen/CodeGenFunction.cpp

691

Yes, this is the right way to solve it. Thanks!

However, you should elide the braces when there's only a single statement (per our coding standards).

[dberris]

include/clang/Basic/Attr.td

436

That makes sense, I can do it in another change that's more focused on just that if you don't mind?

include/clang/Basic/AttrDocs.td

2455

I've explored that option (using patchable-function instead) and while I haven't excluded that outright, there needs to be a discussion about how we're going to implement complimentary instrumentation functionality that do nop-sled inserting. I suspect that has to happen in D19904 where we can still make those changes to instead use patchable-function.

The response to my comment in D19908 seems to be that there should be no conflicts in the platforms XRay currently cares about -- but that if we do ever want to get XRay implemented/supported in x86 32-bit, then we're going to have issues. This suggests that a harder look at patchable-function and exclusive modes (or special-casing the supported configurations) might need to happen sooner than later.

[dberris]

• Formatting changes

[aaron.ballman]

include/clang/Basic/Attr.td

436

That's fine by me (it's also fine to do as part of this patch since this patch will be what exercises that functionality anyway). Your choice. :-)

include/clang/Basic/AttrDocs.td

2455

I think that makes sense. I mostly want to avoid having multiple user-facing attributes that do very similar operations under different names. That makes it confusing for users, and makes for possibly more-complicated code (are these mutually exclusive attributes, etc). I am less worried about whether the backend has multiple attributes, though I suspect the same concerns may apply there as well.

[dberris]

• Check first whether D is actually not nullptr

[dberris]

Undo previous change -- updated the wrong patch. :(

[dberris]

• Check D is valid before using it

[rnk]

Looks good from a driver perspective. Aaron, you're happy with the attribute spelling stuff, right?

It looks like you can actually land this before landing the two dependent changes you mention and the tests will pass. I'd go ahead and land it to avoid the need to rebase, but it's up to you.

include/clang/Basic/AttrDocs.td

2457

These paragraphs can be wrapped to 80 cols like the others.

[aaron.ballman]

In D20352#477038, @rnk wrote:

Looks good from a driver perspective. Aaron, you're happy with the attribute spelling stuff, right?

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

[rnk]

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

[echristo]

In D20352#477203, @rnk wrote:

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

Agreed.

-eric

[aaron.ballman]

In D20352#477203, @rnk wrote:

In D20352#477084, @aaron.ballman wrote:

No, I'm not. I am worried about how this conflicts with another in-flight patch for supporting MS hotpatchable functions since it seems these two attributes do roughly the same thing. I'd like to understand how these two user-facing attributes will not be confusing to users, what should happen if both attributes wind up on a function, etc. I am hoping that we can wind up with only one attribute that covers both cases, if possible.

From a user perspective, I would definitely want to keep these attributes separate. It's only possible to completely replace an MS hotpatchable function, whereas XRay allows you to instrument function entry and exit with low overhead. You can achieve the same effect as XRay with MS hotpatchable prologues and a full trampoline with a stack frame, but probably not at the same runtime cost. They really are suited for different tasks.

Okay, that was the key piece I was missing -- I was under the impression that this was two different ways to achieve the same task. Thank you for the explanation (and sorry if I was being dense). :-)

When the MS hot patching attribute lands, that will be the time to discuss and document conflicts between the two (I am assuming you can't enable XRay and MS hot patching for the same function, which may be an incorrect assumption). So I think this LGTM now.

[dberris]

Thanks -- I don't have commit powers yet, do either of you mind landing this for me?

Cheers

[aaron.ballman]

I applied your patch locally and got a few compile errors with MSVC 2015:

'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 3187
'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 4608

There is no hasArg() overload that accepts a bool argument; perhaps that's a feature of one of the dependencies this patch depends on?

[dberris]

Rebase

[dberris]

• Use hasFlag instead of hasArg

[dberris]

It looks like I was using hasArg instead of hasFlag. The dangerous part here is that OptSpecifier has an unsigned non-explicit argument, and I suspect bool is being promoted to unsigned silently with clang/gcc.

[aaron.ballman]

In D20352#482590, @dberris wrote:

It looks like I was using hasArg instead of hasFlag. The dangerous part here is that OptSpecifier has an unsigned non-explicit argument, and I suspect bool is being promoted to unsigned silently with clang/gcc.

Still not quite right with the latest patch:

C2664 'bool llvm::opt::ArgList::hasArg(llvm::opt::OptSpecifier,llvm::opt::OptSpecifier,llvm::opt::OptSpecifier) const': cannot convert argument 3 from 'bool' to 'llvm::opt::OptSpecifier' E:\llvm\llvm\tools\clang\lib\Driver\Tools.cpp 4608

I changed this to use hasFlag() instead, and then it compiled and tests passed. I've commit in r275330.

include/clang/Basic/Attr.td

 }
 
 def AlwaysInline : InheritableAttr {
   let Spellings = [GCC<"always_inline">, Keyword<"__forceinline">];
   let Subjects = SubjectList<[Function]>;
   let Documentation = [Undocumented];
 }
 
+def XRayInstrument : InheritableAttr {
+  let Spellings = [GNU<"xray_always_instrument">,
+                   CXX11<"clang", "xray_always_instrument">,
+                   GNU<"xray_never_instrument">,
+                   CXX11<"clang", "xray_never_instrument">];
+  let Subjects = SubjectList<[CXXMethod, ObjCMethod, Function], WarnDiag,
+                              "ExpectedFunctionOrMethod">;
+  let Documentation = [XRayDocs];
+}
+
 def TLSModel : InheritableAttr {
   let Spellings = [GCC<"tls_model">];
   let Subjects = SubjectList<[TLSVar], ErrorDiag, "ExpectedTLSVar">;
   let Args = [StringArgument<"Model">];
   let Documentation = [TLSModelDocs];
 }
 
 def AnalyzerNoReturn : InheritableAttr {

include/clang/Basic/AttrDocs.td

 In RenderScript, ``kernel`` functions are used to express data-parallel
 computations.  The RenderScript runtime efficiently parallelizes ``kernel``
 functions to run on computational resources such as multi-core CPUs and GPUs.
 See the RenderScript_ documentation for more information.
 
 .. _RenderScript: https://developer.android.com/guide/topics/renderscript/compute.html
   }];
 }
+
+def XRayDocs : Documentation {
+  let Category = DocCatFunction;
+  let Content = [{
+``__attribute__((xray_always_instrument))`` or ``[[clang:xray_always_instrument]]`` is used to mark member functions (in C++), methods (in Objective C), and free functions (in C, C++, and Objective C) to be instrumented with XRay. This will cause the function to always have space at the beginning and exit points to allow for runtime patching.
+
+Conversely, ``__attribute__((xray_never_instrument))`` or ``[[clang:xray_never_instrument]]`` will inhibit the insertion of these instrumentation points.
+
+If a function has neither of these attributes, they become subject to the XRay heuristics used to determine whether a function should be instrumented or otherwise.
+  }];
+}

include/clang/Driver/Options.td

def finline_functions : Flag<["-"], "finline-functions">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Inline suitable functions">;
 def finline_hint_functions: Flag<["-"], "finline-hint-functions">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Inline functions wich are (explicitly or implicitly) marked inline">;
 def finline : Flag<["-"], "finline">, Group<clang_ignored_f_Group>;
 def finput_charset_EQ : Joined<["-"], "finput-charset=">, Group<f_Group>;
 def fexec_charset_EQ : Joined<["-"], "fexec-charset=">, Group<f_Group>;
 def finstrument_functions : Flag<["-"], "finstrument-functions">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Generate calls to instrument function entry and exit">;
+
+def fxray_instrument : Flag<["-"], "fxray-instrument">, Group<f_Group>,
+  Flags<[CC1Option]>,
+  HelpText<"Generate XRay instrumentation sleds on function entry and exit">;
+def fnoxray_instrument : Flag<["-"], "fno-xray-instrument">, Group<f_Group>,
+  Flags<[CC1Option]>;
+
+def fxray_instruction_threshold_EQ :
+  JoinedOrSeparate<["-"], "fxray-instruction-threshold=">,
+  Group<f_Group>, Flags<[CC1Option]>,
+  HelpText<"Sets the minimum function size to instrument with XRay">;
+def fxray_instruction_threshold_ :
+  JoinedOrSeparate<["-"], "fxray-instruction-threshold">,
+  Group<f_Group>, Flags<[CC1Option]>;
+
 def flat__namespace : Flag<["-"], "flat_namespace">;
 def flax_vector_conversions : Flag<["-"], "flax-vector-conversions">, Group<f_Group>;
 def flimited_precision_EQ : Joined<["-"], "flimited-precision=">, Group<f_Group>;
 def flto_EQ : Joined<["-"], "flto=">, Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Set LTO mode to either 'full' or 'thin'">;
 def flto : Flag<["-"], "flto">, Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group<f_Group>,

include/clang/Frontend/CodeGenOptions.def

 CODEGENOPT(EmulatedTLS       , 1, 0) ///< Set when -femulated-tls is enabled.
 /// \brief FP_CONTRACT mode (on/off/fast).
 ENUM_CODEGENOPT(FPContractMode, FPContractModeKind, 2, FPC_On)
 /// \brief Embed Bitcode mode (off/all/bitcode/marker).
 ENUM_CODEGENOPT(EmbedBitcode, EmbedBitcodeKind, 2, Embed_Off)
 CODEGENOPT(ForbidGuardVariables , 1, 0) ///< Issue errors if C++ guard variables
                                         ///< are required.
 CODEGENOPT(FunctionSections  , 1, 0) ///< Set when -ffunction-sections is enabled.
 CODEGENOPT(InstrumentFunctions , 1, 0) ///< Set when -finstrument-functions is
                                        ///< enabled.
+
+CODEGENOPT(XRayInstrumentFunctions , 1, 0) ///< Set when -fxray-instrument is
+                                           ///< enabled.
+
+///< Set the minimum number of instructions in a function to determine selective
+///< XRay instrumentation.
+VALUE_CODEGENOPT(XRayInstructionThreshold , 32, 200)
+
 CODEGENOPT(InstrumentForProfiling , 1, 0) ///< Set when -pg is enabled.
 CODEGENOPT(LessPreciseFPMAD  , 1, 0) ///< Enable less precise MAD instructions to
                                      ///< be generated.
 CODEGENOPT(PrepareForLTO     , 1, 0) ///< Set when -flto is enabled on the
                                      ///< compile step.
 CODEGENOPT(EmitSummaryIndex, 1, 0)   ///< Set when -flto=thin is enabled on the
                                      ///< compile step.
 CODEGENOPT(IncrementalLinkerCompatible, 1, 0) ///< Emit an object file which can

lib/CodeGen/CodeGenFunction.h

public:
   /// should call destructors on members and base classes in reverse
   /// order of their construction.
   void EnterDtorCleanups(const CXXDestructorDecl *Dtor, CXXDtorType Type);
 
   /// ShouldInstrumentFunction - Return true if the current function should be
   /// instrumented with __cyg_profile_func_* calls
   bool ShouldInstrumentFunction();
 
+  /// ShouldXRayInstrument - Return true if the current function should be
+  /// instrumented with XRay nop sleds.
+  bool ShouldXRayInstrumentFunction();
+
   /// EmitFunctionInstrumentation - Emit LLVM code to call the specified
   /// instrumentation function with the current function and the call site, if
   /// function instrumentation is enabled.
   void EmitFunctionInstrumentation(const char *Fn);
 
   /// EmitMCountInstrumentation - Emit call to .mcount.
   void EmitMCountInstrumentation();
 

lib/CodeGen/CodeGenFunction.cpp

 //===--- CodeGenFunction.cpp - Emit LLVM Code from ASTs for a Function ----===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This coordinates the per-function state used while generating code.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CodeGenFunction.h"
 #include "CGBlocks.h"
-#include "CGCleanup.h"
 #include "CGCUDARuntime.h"
 #include "CGCXXABI.h"
+#include "CGCleanup.h"
 #include "CGDebugInfo.h"
 #include "CGOpenMPRuntime.h"
 #include "CodeGenModule.h"
 #include "CodeGenPGO.h"
 #include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclCXX.h"

 using namespace clang;
 using namespace CodeGen;
 
 CodeGenFunction::CodeGenFunction(CodeGenModule &cgm, bool suppressNewContext)
     : CodeGenTypeCache(cgm), CGM(cgm), Target(cgm.getTarget()),
       Builder(cgm, cgm.getModule().getContext(), llvm::ConstantFolder(),
               CGBuilderInserterTy(this)),
       CurFn(nullptr), ReturnValue(Address::invalid()),
-      CapturedStmtInfo(nullptr),
-      SanOpts(CGM.getLangOpts().Sanitize), IsSanitizerScope(false),
-      CurFuncIsThunk(false), AutoreleaseResult(false), SawAsmBlock(false),
-      IsOutlinedSEHHelper(false),
-      BlockInfo(nullptr), BlockPointer(nullptr),
-      LambdaThisCaptureField(nullptr), NormalCleanupDest(nullptr),
-      NextCleanupDestIndex(1), FirstBlockInfo(nullptr), EHResumeBlock(nullptr),
-      ExceptionSlot(nullptr), EHSelectorSlot(nullptr),
-      DebugInfo(CGM.getModuleDebugInfo()),
+      CapturedStmtInfo(nullptr), SanOpts(CGM.getLangOpts().Sanitize),
+      IsSanitizerScope(false), CurFuncIsThunk(false), AutoreleaseResult(false),
+      SawAsmBlock(false), IsOutlinedSEHHelper(false), BlockInfo(nullptr),
+      BlockPointer(nullptr), LambdaThisCaptureField(nullptr),
+      NormalCleanupDest(nullptr), NextCleanupDestIndex(1),
+      FirstBlockInfo(nullptr), EHResumeBlock(nullptr), ExceptionSlot(nullptr),
+      EHSelectorSlot(nullptr), DebugInfo(CGM.getModuleDebugInfo()),
       DisableDebugInfo(false), DidCallStackSave(false), IndirectBranch(nullptr),
       PGO(cgm), SwitchInsn(nullptr), SwitchWeights(nullptr),
       CaseRangeBlock(nullptr), UnreachableBlock(nullptr), NumReturnExprs(0),
       NumSimpleReturnExprs(0), CXXABIThisDecl(nullptr),
       CXXABIThisValue(nullptr), CXXThisValue(nullptr),
       CXXStructorImplicitParamDecl(nullptr),
       CXXStructorImplicitParamValue(nullptr), OutermostConditional(nullptr),
       CurLexicalScope(nullptr), TerminateLandingPad(nullptr),
CodeGenFunction::~CodeGenFunction() {
   if (FirstBlockInfo)
     destroyBlockInfos(FirstBlockInfo);
 
   if (getLangOpts().OpenMP) {
     CGM.getOpenMPRuntime().functionFinished(*this);
   }
 }
 
-CharUnits CodeGenFunction::getNaturalPointeeTypeAlignment(QualType T,
-                                                     AlignmentSource *Source) {
+CharUnits
+CodeGenFunction::getNaturalPointeeTypeAlignment(QualType T,
   return getNaturalTypeAlignment(T->getPointeeType(), Source,
                                  /*forPointee*/ true);
 }
 
 CharUnits CodeGenFunction::getNaturalTypeAlignment(QualType T,
                                                    AlignmentSource *Source,
                                                    bool forPointeeType) {
   // Honor alignment typedef attributes even on incomplete types.
   // We also honor them straight for C++ class types, even as pointees;
   // there's an expressivity gap here.
   if (auto TT = T->getAs<TypedefType>()) {
     if (auto Align = TT->getDecl()->getMaxAlignment()) {
-      if (Source) *Source = AlignmentSource::AttributedType;
+      if (Source)
+        *Source = AlignmentSource::AttributedType;
       return getContext().toCharUnitsFromBits(Align);
     }
   }
 
-  if (Source) *Source = AlignmentSource::Type;
+  if (Source)
+    *Source = AlignmentSource::Type;
 
   CharUnits Alignment;
   if (T->isIncompleteType()) {
     Alignment = CharUnits::One(); // Shouldn't be used, but pessimistic is best.
   } else {
     // For C++ class pointees, we don't know whether we're pointing at a
     // base or a complete object, so we generally need to use the
     // non-virtual alignment.
LValue CodeGenFunction::MakeNaturalAlignAddrLValue(llvm::Value *V, QualType T) {
   AlignmentSource AlignSource;
   CharUnits Alignment = getNaturalTypeAlignment(T, &AlignSource);
   return LValue::MakeAddr(Address(V, Alignment), T, getContext(), AlignSource,
                           CGM.getTBAAInfo(T));
 }
 
 /// Given a value of type T* that may not be to a complete object,
 /// construct an l-value with the natural pointee alignment of T.
-LValue
-CodeGenFunction::MakeNaturalAlignPointeeAddrLValue(llvm::Value *V, QualType T) {
+LValue CodeGenFunction::MakeNaturalAlignPointeeAddrLValue(llvm::Value *V,
+                                                          QualType T) {
   AlignmentSource AlignSource;
   CharUnits Align = getNaturalTypeAlignment(T, &AlignSource, /*pointee*/ true);
   return MakeAddrLValue(Address(V, Align), T, AlignSource);
 }
 
-
 llvm::Type *CodeGenFunction::ConvertTypeForMem(QualType T) {
   return CGM.getTypes().ConvertTypeForMem(T);
 }
 
 llvm::Type *CodeGenFunction::ConvertType(QualType T) {
   return CGM.getTypes().ConvertType(T);
 }
 
if (CurBB) {
     return llvm::DebugLoc();
   }
 
   // Otherwise, if the return block is the target of a single direct
   // branch then we can just put the code in that block instead. This
   // cleans up functions which started with a unified return block.
   if (ReturnBlock.getBlock()->hasOneUse()) {
     llvm::BranchInst *BI =
         dyn_cast<llvm::BranchInst>(*ReturnBlock.getBlock()->user_begin());
     if (BI && BI->isUnconditional() &&
         BI->getSuccessor(0) == ReturnBlock.getBlock()) {
       // Record/return the DebugLoc of the simple 'return' expression to be used
       // later by the actual 'ret' instruction.
       llvm::DebugLoc Loc = BI->getDebugLoc();
       Builder.SetInsertPoint(BI->getParent());
       BI->eraseFromParent();
       delete ReturnBlock.getBlock();
       return Loc;
     }
   }
 
   // FIXME: We are at an unreachable point, there is no reason to emit the block
   // unless it has uses. However, we still need a place to put the debug
   // region.end for now.
 
   EmitBlock(ReturnBlock.getBlock());
   return llvm::DebugLoc();
 }
 
 static void EmitIfUsed(CodeGenFunction &CGF, llvm::BasicBlock *BB) {
-  if (!BB) return;
+  if (!BB)
+    return;
   if (!BB->use_empty())
     return CGF.CurFn->getBasicBlockList().push_back(BB);
   delete BB;
 }
 
 void CodeGenFunction::FinishFunction(SourceLocation EndLoc) {
   assert(BreakContinueStack.empty() &&
          "mismatched push/pop in break/continue stack!");
 
-  bool OnlySimpleReturnStmts = NumSimpleReturnExprs > 0
-    && NumSimpleReturnExprs == NumReturnExprs
-    && ReturnBlock.getBlock()->use_empty();
+  bool OnlySimpleReturnStmts = NumSimpleReturnExprs > 0 &&
+                               NumSimpleReturnExprs == NumReturnExprs &&
+                               ReturnBlock.getBlock()->use_empty();
   // Usually the return expression is evaluated before the cleanup
   // code.  If the function contains only a simple return statement,
   // such as a constant, the location before the cleanup code becomes
   // the last useful breakpoint in the function, because the simple
   // return expression will be evaluated after the cleanup code. To be
   // safe, set the debug location for cleanup code to the location of
   // the return statement.  Otherwise the cleanup code should be at the
   // end of the function's lexical scope.
if (CGDebugInfo *DI = getDebugInfo())
     DI->EmitFunctionEnd(Builder);
 
   // Reset the debug location to that of the simple 'return' expression, if any
   // rather than that of the end of the function's scope '}'.
   ApplyDebugLocation AL(*this, Loc);
   EmitFunctionEpilog(*CurFnInfo, EmitRetDbgLoc, EndLoc);
   EmitEndEHSpec(CurCodeDecl);
 
-  assert(EHStack.empty() &&
-         "did not remove all scopes from cleanup stack!");
+  assert(EHStack.empty() && "did not remove all scopes from cleanup stack!");
 
   // If someone did an indirect goto, emit the indirect goto block at the end of
   // the function.
   if (IndirectBranch) {
     EmitBlock(IndirectBranch->getParent());
     Builder.ClearInsertionPoint();
   }
 
void CodeGenFunction::FinishFunction(SourceLocation EndLoc) {
   EmitIfUsed(*this, EHResumeBlock);
   EmitIfUsed(*this, TerminateLandingPad);
   EmitIfUsed(*this, TerminateHandler);
   EmitIfUsed(*this, UnreachableBlock);
 
   if (CGM.getCodeGenOpts().EmitDeclMetadata)
     EmitDeclMetadata();
 
-  for (SmallVectorImpl<std::pair<llvm::Instruction *, llvm::Value *> >::iterator
+  for (SmallVectorImpl<std::pair<llvm::Instruction *, llvm::Value *>>::iterator
            I = DeferredReplacements.begin(),
            E = DeferredReplacements.end();
        I != E; ++I) {
     I->first->replaceAllUsesWith(I->second);
     I->first->eraseFromParent();
   }
 }
 
 /// ShouldInstrumentFunction - Return true if the current function should be
 /// instrumented with __cyg_profile_func_* calls
 bool CodeGenFunction::ShouldInstrumentFunction() {
   if (!CGM.getCodeGenOpts().InstrumentFunctions)
     return false;
   if (!CurFuncDecl || CurFuncDecl->hasAttr<NoInstrumentFunctionAttr>())
     return false;
   return true;
 }
 
+/// ShouldXRayInstrument - Return true if the current function should be
+/// instrumented with XRay nop sleds.
+bool CodeGenFunction::ShouldXRayInstrumentFunction() {
+  return CGM.getCodeGenOpts().XRayInstrumentFunctions;
+}
+
 /// EmitFunctionInstrumentation - Emit LLVM code to call the specified
 /// instrumentation function with the current function and the call site, if
 /// function instrumentation is enabled.
 void CodeGenFunction::EmitFunctionInstrumentation(const char *Fn) {
   auto NL = ApplyDebugLocation::CreateArtificial(*this);
   // void __cyg_profile_func_{enter,exit} (void *this_fn, void *call_site);
   llvm::PointerType *PointerTy = Int8PtrTy;
-  llvm::Type *ProfileFuncArgs[] = { PointerTy, PointerTy };
+  llvm::Type *ProfileFuncArgs[] = {PointerTy, PointerTy};
   llvm::FunctionType *FunctionTy =
       llvm::FunctionType::get(VoidTy, ProfileFuncArgs, false);
 
   llvm::Constant *F = CGM.CreateRuntimeFunction(FunctionTy, Fn);
-  llvm::CallInst *CallSite = Builder.CreateCall(
-    CGM.getIntrinsic(llvm::Intrinsic::returnaddress),
-    llvm::ConstantInt::get(Int32Ty, 0),
-    "callsite");
-
-  llvm::Value *args[] = {
-    llvm::ConstantExpr::getBitCast(CurFn, PointerTy),
-    CallSite
-  };
+  llvm::CallInst *CallSite =
+      Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::returnaddress),
+                         llvm::ConstantInt::get(Int32Ty, 0), "callsite");
+
+  llvm::Value *args[] = {llvm::ConstantExpr::getBitCast(CurFn, PointerTy),
+                         CallSite};
 
   EmitNounwindRuntimeCall(F, args);
 }
 
 void CodeGenFunction::EmitMCountInstrumentation() {
   llvm::FunctionType *FTy = llvm::FunctionType::get(VoidTy, false);
 
   llvm::Constant *MCountFn =
       CGM.CreateRuntimeFunction(FTy, getTarget().getMCountName());
   EmitNounwindRuntimeCall(MCountFn);
 }
 
 // OpenCL v1.2 s5.6.4.6 allows the compiler to store kernel argument
 // information in the program executable. The argument information stored
 // includes the argument name, its type, the address and access qualifiers used.
 static void GenOpenCLArgMetadata(const FunctionDecl *FD, llvm::Function *Fn,
                                  CodeGenModule &CGM, llvm::LLVMContext &Context,
if (ty->isPointerType()) {
 
       // Get argument type name.
       std::string typeName =
           pointeeTy.getUnqualifiedType().getAsString(Policy) + "*";
 
       // Turn "unsigned type" to "utype"
       std::string::size_type pos = typeName.find("unsigned");
       if (pointeeTy.isCanonical() && pos != std::string::npos)
-        typeName.erase(pos+1, 8);
+        typeName.erase(pos + 1, 8);
 
       argTypeNames.push_back(llvm::MDString::get(Context, typeName));
 
       std::string baseTypeName =
           pointeeTy.getUnqualifiedType().getCanonicalType().getAsString(
               Policy) +
           "*";
 
       // Turn "unsigned type" to "utype"
       pos = baseTypeName.find("unsigned");
       if (pos != std::string::npos)
-        baseTypeName.erase(pos+1, 8);
+        baseTypeName.erase(pos + 1, 8);
 
       argBaseTypeNames.push_back(llvm::MDString::get(Context, baseTypeName));
 
       // Get argument type qualifiers:
       if (ty.isRestrictQualified())
         typeQuals = "restrict";
       if (pointeeTy.isConstQualified() ||
           (pointeeTy.getAddressSpace() == LangAS::opencl_constant))
         typeQuals += typeQuals.empty() ? "const" : " const";
       if (pointeeTy.isVolatileQualified())
         typeQuals += typeQuals.empty() ? "volatile" : " volatile";
     } else {
       uint32_t AddrSpc = 0;
       bool isPipe = ty->isPipeType();
       if (ty->isImageType() || isPipe)
-        AddrSpc =
-          CGM.getContext().getTargetAddressSpace(LangAS::opencl_global);
+        AddrSpc = CGM.getContext().getTargetAddressSpace(LangAS::opencl_global);
 
       addressQuals.push_back(
           llvm::ConstantAsMetadata::get(Builder.getInt32(AddrSpc)));
 
       // Get argument type name.
       std::string typeName;
       if (isPipe)
         typeName = cast<PipeType>(ty)->getElementType().getAsString(Policy);
       else
         typeName = ty.getUnqualifiedType().getAsString(Policy);
 
       // Turn "unsigned type" to "utype"
       std::string::size_type pos = typeName.find("unsigned");
       if (ty.isCanonical() && pos != std::string::npos)
-        typeName.erase(pos+1, 8);
+        typeName.erase(pos + 1, 8);
 
       argTypeNames.push_back(llvm::MDString::get(Context, typeName));
 
       std::string baseTypeName;
       if (isPipe)
         baseTypeName =
-          cast<PipeType>(ty)->getElementType().getCanonicalType().getAsString(Policy);
+            cast<PipeType>(ty)->getElementType().getCanonicalType().getAsString(
+                Policy);
       else
         baseTypeName =
             ty.getUnqualifiedType().getCanonicalType().getAsString(Policy);
 
       // Turn "unsigned type" to "utype"
       pos = baseTypeName.find("unsigned");
       if (pos != std::string::npos)
-        baseTypeName.erase(pos+1, 8);
+        baseTypeName.erase(pos + 1, 8);
 
       argBaseTypeNames.push_back(llvm::MDString::get(Context, baseTypeName));
 
       // Get argument type qualifiers:
       if (ty.isConstQualified())
         typeQuals = "const";
       if (ty.isVolatileQualified())
         typeQuals += typeQuals.empty() ? "volatile" : " volatile";
       if (isPipe)
         typeQuals = "pipe";
     }
 
     argTypeQuals.push_back(llvm::MDString::get(Context, typeQuals));
 
     // Get image and pipe access qualifier:
-    if (ty->isImageType()|| ty->isPipeType()) {
+    if (ty->isImageType() || ty->isPipeType()) {
       const OpenCLAccessAttr *A = parm->getAttr<OpenCLAccessAttr>();
       if (A && A->isWriteOnly())
         accessQuals.push_back(llvm::MDString::get(Context, "write_only"));
       else if (A && A->isReadWrite())
         accessQuals.push_back(llvm::MDString::get(Context, "read_write"));
       else
         accessQuals.push_back(llvm::MDString::get(Context, "read_only"));
     } else
       accessQuals.push_back(llvm::MDString::get(Context, "none"));
 
     // Get argument name.
     argNames.push_back(llvm::MDString::get(Context, parm->getName()));
   }
 
   Fn->setMetadata("kernel_arg_addr_space",
                   llvm::MDNode::get(Context, addressQuals));
   Fn->setMetadata("kernel_arg_access_qual",
                   llvm::MDNode::get(Context, accessQuals));
-  Fn->setMetadata("kernel_arg_type",
-                  llvm::MDNode::get(Context, argTypeNames));
+  Fn->setMetadata("kernel_arg_type", llvm::MDNode::get(Context, argTypeNames));
   Fn->setMetadata("kernel_arg_base_type",
                   llvm::MDNode::get(Context, argBaseTypeNames));
   Fn->setMetadata("kernel_arg_type_qual",
                   llvm::MDNode::get(Context, argTypeQuals));
   if (CGM.getCodeGenOpts().EmitOpenCLArgMetadata)
-    Fn->setMetadata("kernel_arg_name",
-                    llvm::MDNode::get(Context, argNames));
+    Fn->setMetadata("kernel_arg_name", llvm::MDNode::get(Context, argNames));
 }
 
 void CodeGenFunction::EmitOpenCLKernelMetadata(const FunctionDecl *FD,
-                                               llvm::Function *Fn)
-{
+                                               llvm::Function *Fn) {
   if (!FD->hasAttr<OpenCLKernelAttr>())
     return;
 
   llvm::LLVMContext &Context = getLLVMContext();
 
   GenOpenCLArgMetadata(FD, Fn, CGM, Context, Builder, getContext());
 
   if (const VecTypeHintAttr *A = FD->getAttr<VecTypeHintAttr>()) {
if (const VecTypeHintAttr *A = FD->getAttr<VecTypeHintAttr>()) {
     Fn->setMetadata("vec_type_hint", llvm::MDNode::get(Context, attrMDArgs));
   }
 
   if (const WorkGroupSizeHintAttr *A = FD->getAttr<WorkGroupSizeHintAttr>()) {
     llvm::Metadata *attrMDArgs[] = {
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getXDim())),
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getYDim())),
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getZDim()))};
-    Fn->setMetadata("work_group_size_hint", llvm::MDNode::get(Context, attrMDArgs));
+    Fn->setMetadata("work_group_size_hint",
+                    llvm::MDNode::get(Context, attrMDArgs));
   }
 
   if (const ReqdWorkGroupSizeAttr *A = FD->getAttr<ReqdWorkGroupSizeAttr>()) {
     llvm::Metadata *attrMDArgs[] = {
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getXDim())),
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getYDim())),
         llvm::ConstantAsMetadata::get(Builder.getInt32(A->getZDim()))};
-    Fn->setMetadata("reqd_work_group_size", llvm::MDNode::get(Context, attrMDArgs));
+    Fn->setMetadata("reqd_work_group_size",
+                    llvm::MDNode::get(Context, attrMDArgs));
   }
 }
 
 /// Determine whether the function F ends with a return stmt.
-static bool endsWithReturn(const Decl* F) {
+static bool endsWithReturn(const Decl *F) {
   const Stmt *Body = nullptr;
   if (auto *FD = dyn_cast_or_null<FunctionDecl>(F))
     Body = FD->getBody();
   else if (auto *OMD = dyn_cast_or_null<ObjCMethodDecl>(F))
     Body = OMD->getBody();
 
   if (auto *CS = dyn_cast_or_null<CompoundStmt>(Body)) {
     auto LastStmt = CS->body_rbegin();
     if (LastStmt != CS->body_rend())
       return isa<ReturnStmt>(*LastStmt);
   }
   return false;
 }
 
-void CodeGenFunction::StartFunction(GlobalDecl GD,
-                                    QualType RetTy,
+void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
                                     llvm::Function *Fn,
                                     const CGFunctionInfo &FnInfo,
                                     const FunctionArgList &Args,
                                     SourceLocation Loc,
                                     SourceLocation StartLoc) {
   assert(!CurFn &&
          "Do not use a CodeGenFunction object for more than one function");
 
if (SanOpts.hasOneOf(SanitizerKind::Address | SanitizerKind::KernelAddress))
     Fn->addFnAttr(llvm::Attribute::SanitizeAddress);
   if (SanOpts.has(SanitizerKind::Thread))
     Fn->addFnAttr(llvm::Attribute::SanitizeThread);
   if (SanOpts.has(SanitizerKind::Memory))
     Fn->addFnAttr(llvm::Attribute::SanitizeMemory);
   if (SanOpts.has(SanitizerKind::SafeStack))
     Fn->addFnAttr(llvm::Attribute::SafeStack);
 
+  // Apply xray attributes to the function.
+  if (ShouldXRayInstrumentFunction()) {
+    if (D->hasAttr<XRayInstrumentAttr>()) {
+      switch (D->getAttr<XRayInstrumentAttr>()->getSpellingListIndex()) {
+        case 0:
+        case 1:
+          Fn->addFnAttr("function-instrument", "xray-always");
+          break;
+        case 2:
+        case 3:
+          Fn->addFnAttr("function-instrument", "xray-never");
+          break;
+        default:
+          llvm_unreachable("Unknown attribute index for XRay instrumentation");
+      }
+    } else {
+      Fn->addFnAttr(
+          "xray-instruction-threshold",
+          llvm::itostr(CGM.getCodeGenOpts().XRayInstructionThreshold));
+    }
+  }
+
   // Pass inline keyword to optimizer if it appears explicitly on any
   // declaration. Also, in the case of -fno-inline attach NoInline
   // attribute to all functions that are not marked AlwaysInline, or
   // to all functions that are not marked inline or implicitly inline
   // in the case of -finline-hint-functions.
   if (const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(D)) {
-    const CodeGenOptions& CodeGenOpts = CGM.getCodeGenOpts();
+    const CodeGenOptions &CodeGenOpts = CGM.getCodeGenOpts();
     if (!CodeGenOpts.NoInline) {
       for (auto RI : FD->redecls())
         if (RI->isInlineSpecified()) {
           Fn->addFnAttr(llvm::Attribute::InlineHint);
           break;
         }
       if (CodeGenOpts.getInlining() == CodeGenOptions::OnlyHintInlining &&
           !FD->isInlined() && !Fn->hasFnAttribute(llvm::Attribute::InlineHint))
void CodeGenFunction::StartFunction(GlobalDecl GD, QualType RetTy,
   // If we are checking function types, emit a function type signature as
   // prologue data.
   if (getLangOpts().CPlusPlus && SanOpts.has(SanitizerKind::Function)) {
     if (const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(D)) {
       if (llvm::Constant *PrologueSig =
               CGM.getTargetCodeGenInfo().getUBSanFunctionSignature(CGM)) {
         llvm::Constant *FTRTTIConst =
             CGM.GetAddrOfRTTIDescriptor(FD->getType(), /*ForEH=*/true);
-        llvm::Constant *PrologueStructElems[] = { PrologueSig, FTRTTIConst };
+        llvm::Constant *PrologueStructElems[] = {PrologueSig, FTRTTIConst};
         llvm::Constant *PrologueStructConst =
             llvm::ConstantStruct::getAnon(PrologueStructElems, /*Packed=*/true);
         Fn->setPrologueData(PrologueStructConst);
       }
     }
   }
 
   // If we're in C++ mode and the function name is "main", it is guaranteed
   // to be norecurse by the standard (3.6.1.3 "The function main shall not be
   // used within a program").
   if (getLangOpts().CPlusPlus)
     if (const FunctionDecl *FD = dyn_cast_or_null<FunctionDecl>(D))
       if (FD->isMain())
         Fn->addFnAttr(llvm::Attribute::NoRecurse);
-  
   llvm::BasicBlock *EntryBB = createBasicBlock("entry", CurFn);
 
   // Create a marker to make it easy to insert allocas into the entryblock
   // later.  Don't create this with the builder, because we don't want it
   // folded.
   llvm::Value *Undef = llvm::UndefValue::get(Int32Ty);
   AllocaInsertPt = new llvm::BitCastInst(Undef, Int32Ty, "allocapt", EntryBB);
 
if (RetTy->isVoidType()) {
     Addr = Builder.CreateAlignedLoad(Addr, getPointerAlign(), "agg.result");
     ReturnValue = Address(Addr, getNaturalTypeAlignment(RetTy));
   } else {
     ReturnValue = CreateIRTemp(RetTy, "retval");
 
     // Tell the epilog emitter to autorelease the result.  We do this
     // now so that various specialized functions can suppress it
     // during their IR-generation.
-    if (getLangOpts().ObjCAutoRefCount &&
-        !CurFnInfo->isReturnsRetained() &&
+    if (getLangOpts().ObjCAutoRefCount && !CurFnInfo->isReturnsRetained() &&
         RetTy->isObjCRetainableType())
       AutoreleaseResult = true;
   }
 
   EmitStartEHSpec(CurCodeDecl);
 
   PrologueCleanupDepth = EHStack.stable_begin();
   EmitFunctionProlog(*CurFnInfo, CurFn, Args);
 
   if (D && isa<CXXMethodDecl>(D) && cast<CXXMethodDecl>(D)->isInstance()) {
     CGM.getCXXABI().EmitInstanceFunctionProlog(*this);
     const CXXMethodDecl *MD = cast<CXXMethodDecl>(D);
-    if (MD->getParent()->isLambda() &&
-        MD->getOverloadedOperator() == OO_Call) {
+    if (MD->getParent()->isLambda() && MD->getOverloadedOperator() == OO_Call) {
       // We're in a lambda; figure out the captures.
       MD->getParent()->getCaptureFields(LambdaCaptureFields,
                                         LambdaThisCaptureField);
       if (LambdaThisCaptureField) {
         // If the lambda captures the object referred to by '*this' - either by
         // value or by reference, make sure CXXThisValue points to the correct
         // object.
 
         // Get the lvalue for the field (which is a copy of the enclosing object
         // or contains the address of the enclosing object).
-        LValue ThisFieldLValue = EmitLValueForLambdaField(LambdaThisCaptureField);
+        LValue ThisFieldLValue =
+            EmitLValueForLambdaField(LambdaThisCaptureField);
         if (!LambdaThisCaptureField->getType()->isPointerType()) {
-          // If the enclosing object was captured by value, just use its address.
+          // If the enclosing object was captured by value, just use its
+          // address.
           CXXThisValue = ThisFieldLValue.getAddress().getPointer();
         } else {
           // Load the lvalue pointed to by the field, since '*this' was captured
           // by reference.
-          CXXThisValue =
-              EmitLoadOfLValue(ThisFieldLValue, SourceLocation()).getScalarVal();
+          CXXThisValue = EmitLoadOfLValue(ThisFieldLValue, SourceLocation())
+                             .getScalarVal();
         }
       }
       for (auto *FD : MD->getParent()->fields()) {
         if (FD->hasCapturedVLAType()) {
-          auto *ExprArg = EmitLoadOfLValue(EmitLValueForLambdaField(FD),
-                                           SourceLocation()).getScalarVal();
+          auto *ExprArg =
+              EmitLoadOfLValue(EmitLValueForLambdaField(FD), SourceLocation())
+                  .getScalarVal();
           auto VAT = FD->getCapturedVLAType();
           VLASizeMap[VAT->getSizeExpr()] = ExprArg;
         }
       }
     } else {
       // Not in a lambda; just use 'this' from the method.
       // FIXME: Should we generate a new load for each use of 'this'?  The
       // fast register allocator would be happier...
       CXXThisValue = CXXABIThisValue;
     }
   }
 
   // If any of the arguments have a variably modified type, make sure to
   // emit the type size.
-  for (FunctionArgList::const_iterator i = Args.begin(), e = Args.end();
-       i != e; ++i) {
+  for (FunctionArgList::const_iterator i = Args.begin(), e = Args.end(); i != e;
+       ++i) {
     const VarDecl *VD = *i;
 
     // Dig out the type as written from ParmVarDecls; it's unclear whether
     // the standard (C99 6.9.1p10) requires this, but we're following the
     // precedent set by gcc.
     QualType Ty;
     if (const ParmVarDecl *PVD = dyn_cast<ParmVarDecl>(VD))
       Ty = PVD->getOriginalType();

 }
 
 /// Tries to mark the given function nounwind based on the
 /// non-existence of any throwing calls within it.  We believe this is
 /// lightweight enough to do at -O0.
 static void TryMarkNoThrow(llvm::Function *F) {
   // LLVM treats 'nounwind' on a function as part of the type, so we
   // can't do this on functions that can be overwritten.
-  if (F->isInterposable()) return;
+  if (F->isInterposable())
+    return;
 
   for (llvm::BasicBlock &BB : *F)
     for (llvm::Instruction &I : BB)
       if (I.mayThrow())
         return;
 
   F->setDoesNotThrow();
 }
for (auto *Param : FD->parameters()) {
     SizeArguments[Param] = Implicit;
     Args.push_back(Implicit);
   }
 
   if (MD && (isa<CXXConstructorDecl>(MD) || isa<CXXDestructorDecl>(MD)))
     CGM.getCXXABI().addImplicitStructorParams(*this, ResTy, Args);
 
   SourceRange BodyRange;
-  if (Stmt *Body = FD->getBody()) BodyRange = Body->getSourceRange();
+  if (Stmt *Body = FD->getBody())
+    BodyRange = Body->getSourceRange();
   CurEHLocation = BodyRange.getEnd();
 
   // Use the location of the start of the function to determine where
   // the function definition is located. By default use the location
   // of the declaration as the location for the subprogram. A function
   // may lack a declaration in the source code if it is created by code
   // gen. (examples: _GLOBAL__I_a, __cxx_global_array_dtor, thunk).
   SourceLocation Loc = FD->getLocation();
 
   // If this is a function specialization then use the pattern body
   // as the location for the function.
   if (const FunctionDecl *SpecDecl = FD->getTemplateInstantiationPattern())
     if (SpecDecl->hasBody(SpecDecl))
       Loc = SpecDecl->getLocation();
 
   // Emit the standard function prologue.
   StartFunction(GD, ResTy, Fn, FnInfo, Args, Loc, BodyRange.getBegin());
 
   // Generate the body of the function.
   PGO.assignRegionCounters(GD, CurFn);
   if (isa<CXXDestructorDecl>(FD))
     EmitDestructorBody(Args);
   else if (isa<CXXConstructorDecl>(FD))
     EmitConstructorBody(Args);
-  else if (getLangOpts().CUDA &&
-           !getLangOpts().CUDAIsDevice &&
+  else if (getLangOpts().CUDA && !getLangOpts().CUDAIsDevice &&
            FD->hasAttr<CUDAGlobalAttr>())
     CGM.getCUDARuntime().emitDeviceStub(*this, Args);
   else if (isa<CXXConversionDecl>(FD) &&
            cast<CXXConversionDecl>(FD)->isLambdaToBlockPointerConversion()) {
     // The lambda conversion to block pointer is special; the semantics can't be
     // expressed in the AST, so IRGen needs to special-case it.
     EmitLambdaToBlockPointerBody(Args);
   } else if (isa<CXXMethodDecl>(FD) &&
if (!CurFn->doesNotThrow())
     TryMarkNoThrow(CurFn);
 }
 
 /// ContainsLabel - Return true if the statement contains a label in it.  If
 /// this statement is not executed normally, it not containing a label means
 /// that we can just remove the code.
 bool CodeGenFunction::ContainsLabel(const Stmt *S, bool IgnoreCaseStmts) {
   // Null statement, not a label!
-  if (!S) return false;
+  if (!S)
+    return false;
 
   // If this is a label, we have to emit the code, consider something like:
   // if (0) {  ...  foo:  bar(); }  goto foo;
   //
   // TODO: If anyone cared, we could track __label__'s, since we know that you
   // can't jump to one from outside their declared region.
   if (isa<LabelStmt>(S))
     return true;
bool CodeGenFunction::ContainsLabel(const Stmt *S, bool IgnoreCaseStmts) {
   return false;
 }
 
 /// containsBreak - Return true if the statement contains a break out of it.
 /// If the statement (recursively) contains a switch or loop with a break
 /// inside of it, this is fine.
 bool CodeGenFunction::containsBreak(const Stmt *S) {
   // Null statement, not a label!
-  if (!S) return false;
+  if (!S)
+    return false;
 
   // If this is a switch or loop that defines its own break scope, then we can
   // include it and anything inside of it.
   if (isa<SwitchStmt>(S) || isa<WhileStmt>(S) || isa<DoStmt>(S) ||
       isa<ForStmt>(S))
     return false;
 
   if (isa<BreakStmt>(S))
     return true;
 
   // Scan subexpressions for verboten breaks.
   for (const Stmt *SubStmt : S->children())
     if (containsBreak(SubStmt))
       return true;
 
   return false;
 }
 
-
 /// ConstantFoldsToSimpleInteger - If the specified expression does not fold
 /// to a constant, or if it does but contains a label, return false.  If it
 /// constant folds return true and set the boolean result in Result.
 bool CodeGenFunction::ConstantFoldsToSimpleInteger(const Expr *Cond,
                                                    bool &ResultBool,
                                                    bool AllowLabels) {
   llvm::APSInt ResultInt;
   if (!ConstantFoldsToSimpleInteger(Cond, ResultInt, AllowLabels))
     return false;
 
   ResultBool = ResultInt.getBoolValue();
   return true;
 }
 
 /// ConstantFoldsToSimpleInteger - If the specified expression does not fold
 /// to a constant, or if it does but contains a label, return false.  If it
 /// constant folds return true and set the folded value.
 bool CodeGenFunction::ConstantFoldsToSimpleInteger(const Expr *Cond,
                                                    llvm::APSInt &ResultInt,
                                                    bool AllowLabels) {
   // FIXME: Rename and handle conversion of other evaluatable things
   // to bool.
   llvm::APSInt Int;
   if (!Cond->EvaluateAsInt(Int, getContext()))
-    return false;  // Not foldable, not integer or not fully evaluatable.
+    return false; // Not foldable, not integer or not fully evaluatable.
 
   if (!AllowLabels && CodeGenFunction::ContainsLabel(Cond))
-    return false;  // Contains a label.
+    return false; // Contains a label.
 
   ResultInt = Int;
   return true;
 }
 
-
-
 /// EmitBranchOnBoolExpr - Emit a branch on a boolean condition (e.g. for an if
 /// statement) to the specified blocks.  Based on the condition, this might try
 /// to simplify the codegen of the conditional based on the branch.
 ///
 void CodeGenFunction::EmitBranchOnBoolExpr(const Expr *Cond,
                                            llvm::BasicBlock *TrueBlock,
                                            llvm::BasicBlock *FalseBlock,
                                            uint64_t TrueCount) {
void CodeGenFunction::EmitBranchOnBoolExpr(const Expr *Cond,
   }
 
   if (const CXXThrowExpr *Throw = dyn_cast<CXXThrowExpr>(Cond)) {
     // Conditional operator handling can give us a throw expression as a
     // condition for a case like:
     //   br(c ? throw x : y, t, f) -> br(c, br(throw x, t, f), br(y, t, f)
     // Fold this to:
     //   br(c, throw x, br(y, t, f))
-    EmitCXXThrowExpr(Throw, /*KeepInsertionPoint*/false);
+    EmitCXXThrowExpr(Throw, /*KeepInsertionPoint*/ false);
     return;
   }
 
   // If the branch has a condition wrapped by __builtin_unpredictable,
   // create metadata that specifies that the branch is unpredictable.
   // Don't bother if not optimizing because that metadata would not be used.
   llvm::MDNode *Unpredictable = nullptr;
   auto *Call = dyn_cast<CallExpr>(Cond);

 /// base element of the array
 /// \param sizeInChars - the total size of the VLA, in chars
 static void emitNonZeroVLAInit(CodeGenFunction &CGF, QualType baseType,
                                Address dest, Address src,
                                llvm::Value *sizeInChars) {
   CGBuilderTy &Builder = CGF.Builder;
 
   CharUnits baseSize = CGF.getContext().getTypeSizeInChars(baseType);
-  llvm::Value *baseSizeInChars
-    = llvm::ConstantInt::get(CGF.IntPtrTy, baseSize.getQuantity());
+  llvm::Value *baseSizeInChars =
+      llvm::ConstantInt::get(CGF.IntPtrTy, baseSize.getQuantity());
 
-  Address begin =
-    Builder.CreateElementBitCast(dest, CGF.Int8Ty, "vla.begin");
+  Address begin = Builder.CreateElementBitCast(dest, CGF.Int8Ty, "vla.begin");
   llvm::Value *end =
       Builder.CreateInBoundsGEP(begin.getPointer(), sizeInChars, "vla.end");
 
   llvm::BasicBlock *originBB = CGF.Builder.GetInsertBlock();
   llvm::BasicBlock *loopBB = CGF.createBasicBlock("vla-init.loop");
   llvm::BasicBlock *contBB = CGF.createBasicBlock("vla-init.cont");
 
   // Make a loop over the VLA.  C99 guarantees that the VLA element
   // count must be nonzero.
   CGF.EmitBlock(loopBB);
 
   llvm::PHINode *cur = Builder.CreatePHI(begin.getType(), 2, "vla.cur");
   cur->addIncoming(begin.getPointer(), originBB);
 
-  CharUnits curAlign =
-    dest.getAlignment().alignmentOfArrayElement(baseSize);
+  CharUnits curAlign = dest.getAlignment().alignmentOfArrayElement(baseSize);
 
   // memcpy the individual element bit-pattern.
   Builder.CreateMemCpy(Address(cur, curAlign), src, baseSizeInChars,
                        /*volatile*/ false);
 
   // Go to the next element.
   llvm::Value *next =
       Builder.CreateInBoundsGEP(CGF.Int8Ty, cur, baseSizeInChars, "vla.next");
 
   // Leave if that's the end of the VLA.
   llvm::Value *done = Builder.CreateICmpEQ(next, end, "vla-init.isdone");
   Builder.CreateCondBr(done, contBB, loopBB);
   cur->addIncoming(next, loopBB);
 
   CGF.EmitBlock(contBB);
 }
 
-void
-CodeGenFunction::EmitNullInitialization(Address DestPtr, QualType Ty) {
+void CodeGenFunction::EmitNullInitialization(Address DestPtr, QualType Ty) {
   // Ignore empty classes in C++.
   if (getLangOpts().CPlusPlus) {
     if (const RecordType *RT = Ty->getAs<RecordType>()) {
       if (cast<CXXRecordDecl>(RT->getDecl())->isEmpty())
         return;
     }
   }
 
   // Cast the dest ptr to the appropriate i8 pointer type.
   if (DestPtr.getElementType() != Int8Ty)
     DestPtr = Builder.CreateElementBitCast(DestPtr, Int8Ty);
 
   // Get size and alignment info for this aggregate.
   CharUnits size = getContext().getTypeSizeInChars(Ty);
 
   llvm::Value *SizeVal;
   const VariableArrayType *vla;
 
   // Don't bother emitting a zero-byte memset.
   if (size.isZero()) {
     // But note that getTypeInfo returns 0 for a VLA.
-    if (const VariableArrayType *vlaType =
-          dyn_cast_or_null<VariableArrayType>(
-                                          getContext().getAsArrayType(Ty))) {
+    if (const VariableArrayType *vlaType = dyn_cast_or_null<VariableArrayType>(
       QualType eltType;
       llvm::Value *numElts;
       std::tie(numElts, eltType) = getVLASize(vlaType);
 
       SizeVal = numElts;
       CharUnits eltSize = getContext().getTypeSizeInChars(eltType);
       if (!eltSize.isOne())
         SizeVal = Builder.CreateNUWMul(SizeVal, CGM.getSize(eltSize));
       vla = vlaType;
     } else {
       return;
     }
   } else {
     SizeVal = CGM.getSize(size);
     vla = nullptr;
   }
 
   // If the type contains a pointer to data member we can't memset it to zero.
   // Instead, create a null constant and copy it to the destination.
   // TODO: there are other patterns besides zero that we can usefully memset,
   // like -1, which happens to be the pattern used by member-pointers.
   if (!CGM.getTypes().isZeroInitializable(Ty)) {
     // For a VLA, emit a single element, then splat that over the VLA.
-    if (vla) Ty = getContext().getBaseElementType(vla);
+    if (vla)
+      Ty = getContext().getBaseElementType(vla);
 
     llvm::Constant *NullConstant = CGM.EmitNullConstant(Ty);
 
-    llvm::GlobalVariable *NullVariable =
-      new llvm::GlobalVariable(CGM.getModule(), NullConstant->getType(),
-                               /*isConstant=*/true,
-                               llvm::GlobalVariable::PrivateLinkage,
-                               NullConstant, Twine());
+    llvm::GlobalVariable *NullVariable = new llvm::GlobalVariable(
+        CGM.getModule(), NullConstant->getType(),
+        /*isConstant=*/true, llvm::GlobalVariable::PrivateLinkage, NullConstant,
+        Twine());
     CharUnits NullAlign = DestPtr.getAlignment();
     NullVariable->setAlignment(NullAlign.getQuantity());
     Address SrcPtr(Builder.CreateBitCast(NullVariable, Builder.getInt8PtrTy()),
                    NullAlign);
 
-    if (vla) return emitNonZeroVLAInit(*this, Ty, DestPtr, SrcPtr, SizeVal);
+    if (vla)
+      return emitNonZeroVLAInit(*this, Ty, DestPtr, SrcPtr, SizeVal);
 
     // Get and call the appropriate llvm.memcpy overload.
     Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, false);
     return;
   }
 
   // Otherwise, just memset the whole thing to zero.  This is legal
   // because in LLVM, all default initializers (other than the ones we just
llvm::BlockAddress *CodeGenFunction::GetAddrOfLabel(const LabelDecl *L) {
 
   // Make sure the indirect branch includes all of the address-taken blocks.
   IndirectBranch->addDestination(BB);
   return llvm::BlockAddress::get(CurFn, BB);
 }
 
 llvm::BasicBlock *CodeGenFunction::GetIndirectGotoBlock() {
   // If we already made the indirect branch for indirect goto, return its block.
-  if (IndirectBranch) return IndirectBranch->getParent();
+  if (IndirectBranch)
+    return IndirectBranch->getParent();
 
   CGBuilderTy TmpBuilder(*this, createBasicBlock("indirectgoto"));
 
   // Create the PHI node that indirect gotos will add entries to.
-  llvm::Value *DestVal = TmpBuilder.CreatePHI(Int8PtrTy, 0,
-                                              "indirect.goto.dest");
+  llvm::Value *DestVal =
+      TmpBuilder.CreatePHI(Int8PtrTy, 0, "indirect.goto.dest");
 
   // Create the indirect branch instruction.
   IndirectBranch = TmpBuilder.CreateIndirectBr(DestVal);
   return IndirectBranch->getParent();
 }
 
 /// Computes the length of an array in elements, as well as the base
 /// element type and a properly-typed first element pointer.
if (isa<VariableArrayType>(arrayType)) {
 
     // We get out here only if we find a constant array type
     // inside the VLA.
   }
 
   // We have some number of constant-length arrays, so addr should
   // have LLVM type [M x [N x [...]]]*.  Build a GEP that walks
   // down to the first element of addr.
-  SmallVector<llvm::Value*, 8> gepIndices;
+  SmallVector<llvm::Value *, 8> gepIndices;
 
   // GEP down to the array type.
   llvm::ConstantInt *zero = Builder.getInt32(0);
   gepIndices.push_back(zero);
 
   uint64_t countFromCLAs = 1;
   QualType eltType;
 
   llvm::ArrayType *llvmArrayType =
       dyn_cast<llvm::ArrayType>(addr.getElementType());
   while (llvmArrayType) {
     assert(isa<ConstantArrayType>(arrayType));
-    assert(cast<ConstantArrayType>(arrayType)->getSize().getZExtValue()
-             == llvmArrayType->getNumElements());
+    assert(cast<ConstantArrayType>(arrayType)->getSize().getZExtValue() ==
+           llvmArrayType->getNumElements());
 
     gepIndices.push_back(zero);
     countFromCLAs *= llvmArrayType->getNumElements();
     eltType = arrayType->getElementType();
 
-    llvmArrayType =
-      dyn_cast<llvm::ArrayType>(llvmArrayType->getElementType());
+    llvmArrayType = dyn_cast<llvm::ArrayType>(llvmArrayType->getElementType());
     arrayType = getContext().getAsArrayType(arrayType->getElementType());
     assert((!llvmArrayType || arrayType) &&
            "LLVM and Clang types are out-of-synch");
   }
 
   if (arrayType) {
     // From this point onwards, the Clang array type has been emitted
     // as some other type (probably a packed struct). Compute the array
     // size, and just emit the 'begin' expression as a bitcast.
     while (arrayType) {
       countFromCLAs *=
           cast<ConstantArrayType>(arrayType)->getSize().getZExtValue();
       eltType = arrayType->getElementType();
       arrayType = getContext().getAsArrayType(eltType);
     }
 
     llvm::Type *baseType = ConvertType(eltType);
     addr = Builder.CreateElementBitCast(addr, baseType, "array.begin");
   } else {
     // Create the actual GEP.
-    addr = Address(Builder.CreateInBoundsGEP(addr.getPointer(),
-                                             gepIndices, "array.begin"),
-                   addr.getAlignment());
+    addr = Address(
+        Builder.CreateInBoundsGEP(addr.getPointer(), gepIndices, "array.begin"),
   }
 
   baseType = eltType;
 
-  llvm::Value *numElements
-    = llvm::ConstantInt::get(SizeTy, countFromCLAs);
+  llvm::Value *numElements = llvm::ConstantInt::get(SizeTy, countFromCLAs);
 
   // If we had any VLA dimensions, factor them in.
   if (numVLAElements)
     numElements = Builder.CreateNUWMul(numVLAElements, numElements);
 
   return numElements;
 }
 
-std::pair<llvm::Value*, QualType>
-CodeGenFunction::getVLASize(QualType type) {
+std::pair<llvm::Value *, QualType> CodeGenFunction::getVLASize(QualType type) {
   const VariableArrayType *vla = getContext().getAsVariableArrayType(type);
   assert(vla && "type was not a variable array type!");
   return getVLASize(vla);
 }
 
-std::pair<llvm::Value*, QualType>
+std::pair<llvm::Value *, QualType>
 CodeGenFunction::getVLASize(const VariableArrayType *type) {
   // The number of elements so far; always size_t.
   llvm::Value *numElements = nullptr;
 
   QualType elementType;
   do {
     elementType = type->getElementType();
     llvm::Value *vlaSize = VLASizeMap[type->getSizeExpr()];
     assert(vlaSize && "no size for VLA!");
     assert(vlaSize->getType() == SizeTy);
 
     if (!numElements) {
       numElements = vlaSize;
     } else {
       // It's undefined behavior if this wraps around, so mark it that way.
       // FIXME: Teach -fsanitize=undefined to trap this.
       numElements = Builder.CreateNUWMul(numElements, vlaSize);
     }
   } while ((type = getContext().getAsVariableArrayType(elementType)));
 
-  return std::pair<llvm::Value*,QualType>(numElements, elementType);
+  return std::pair<llvm::Value *, QualType>(numElements, elementType);
 }
 
 void CodeGenFunction::EmitVariablyModifiedType(QualType type) {
   assert(type->isVariablyModifiedType() &&
          "Must pass variably modified type to EmitVLASizes!");
 
   EnsureInsertPoint();
 
case Type::VariableArray: {
           //   If the size is an expression that is not an integer constant
           //   expression [...] each time it is evaluated it shall have a value
           //   greater than zero.
           if (SanOpts.has(SanitizerKind::VLABound) &&
               size->getType()->isSignedIntegerType()) {
             SanitizerScope SanScope(this);
             llvm::Value *Zero = llvm::Constant::getNullValue(Size->getType());
             llvm::Constant *StaticArgs[] = {
                 EmitCheckSourceLocation(size->getLocStart()),
-              EmitCheckTypeDescriptor(size->getType())
-            };
+                EmitCheckTypeDescriptor(size->getType())};
             EmitCheck(std::make_pair(Builder.CreateICmpSGT(Size, Zero),
                                      SanitizerKind::VLABound),
                       "vla_bound_not_positive", StaticArgs, Size);
           }
 
           // Always zexting here would be wrong if it weren't
           // undefined behavior to have a negative bound.
           entry = Builder.CreateIntCast(Size, SizeTy, /*signed*/ false);
#include "clang/AST/TypeNodes.def"
 
     case Type::Pipe:
       type = cast<PipeType>(ty)->getElementType();
       break;
     }
   } while (type->isVariablyModifiedType());
 }
 
-Address CodeGenFunction::EmitVAListRef(const Expr* E) {
+Address CodeGenFunction::EmitVAListRef(const Expr *E) {
   if (getContext().getBuiltinVaListType()->isArrayType())
     return EmitPointerWithAlignment(E);
   return EmitLValue(E).getAddress();
 }
 
 Address CodeGenFunction::EmitMSVAListRef(const Expr *E) {
   return EmitLValue(E).getAddress();
 }
 
 void CodeGenFunction::EmitDeclRefExprDbgValue(const DeclRefExpr *E,
                                               llvm::Constant *Init) {
-  assert (Init && "Invalid DeclRefExpr initializer!");
+  assert(Init && "Invalid DeclRefExpr initializer!");
   if (CGDebugInfo *Dbg = getDebugInfo())
     if (CGM.getCodeGenOpts().getDebugInfo() >= codegenoptions::LimitedDebugInfo)
       Dbg->EmitGlobalVariable(E->getDecl(), Init);
 }
 
 CodeGenFunction::PeepholeProtection
 CodeGenFunction::protectFromPeepholes(RValue rvalue) {
   // At the moment, the only aggressive peephole we do in IR gen
   // is trunc(zext) folding, but if we add more, we can easily
   // extend this protection.
 
-  if (!rvalue.isScalar()) return PeepholeProtection();
+  if (!rvalue.isScalar())
+    return PeepholeProtection();
   llvm::Value *value = rvalue.getScalarVal();
-  if (!isa<llvm::ZExtInst>(value)) return PeepholeProtection();
+  if (!isa<llvm::ZExtInst>(value))
+    return PeepholeProtection();
 
   // Just make an extra bitcast.
   assert(HaveInsertPoint());
   llvm::Instruction *inst = new llvm::BitCastInst(value, value->getType(), "",
                                                   Builder.GetInsertBlock());
 
   PeepholeProtection protection;
   protection.Inst = inst;
   return protection;
 }
 
 void CodeGenFunction::unprotectFromPeepholes(PeepholeProtection protection) {
-  if (!protection.Inst) return;
+  if (!protection.Inst)
+    return;
 
   // In theory, we could try to duplicate the peepholes now, but whatever.
   protection.Inst->eraseFromParent();
 }
 
 llvm::Value *CodeGenFunction::EmitAnnotationCall(llvm::Value *AnnotationFn,
                                                  llvm::Value *AnnotatedVal,
                                                  StringRef AnnotationStr,
                                                  SourceLocation Location) {
   llvm::Value *Args[4] = {
       AnnotatedVal,
       Builder.CreateBitCast(CGM.EmitAnnotationString(AnnotationStr), Int8PtrTy),
       Builder.CreateBitCast(CGM.EmitAnnotationUnit(Location), Int8PtrTy),
-    CGM.EmitAnnotationLineNo(Location)
-  };
+      CGM.EmitAnnotationLineNo(Location)};
   return Builder.CreateCall(AnnotationFn, Args);
 }
 
 void CodeGenFunction::EmitVarAnnotations(const VarDecl *D, llvm::Value *V) {
   assert(D->hasAttr<AnnotateAttr>() && "no annotate attribute");
   // FIXME We create a new bitcast for every annotation because that's what
   // llvm-gcc was doing.
   for (const auto *I : D->specific_attrs<AnnotateAttr>())
     EmitAnnotationCall(CGM.getIntrinsic(llvm::Intrinsic::var_annotation),
                        Builder.CreateBitCast(V, CGM.Int8PtrTy, V->getName()),
                        I->getAnnotation(), D->getLocation());
 }
 
 Address CodeGenFunction::EmitFieldAnnotations(const FieldDecl *D,
                                               Address Addr) {
   assert(D->hasAttr<AnnotateAttr>() && "no annotate attribute");
   llvm::Value *V = Addr.getPointer();
   llvm::Type *VTy = V->getType();
-  llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::ptr_annotation,
-                                    CGM.Int8PtrTy);
+  llvm::Value *F =
+      CGM.getIntrinsic(llvm::Intrinsic::ptr_annotation, CGM.Int8PtrTy);
 
   for (const auto *I : D->specific_attrs<AnnotateAttr>()) {
     // FIXME Always emit the cast inst so we can differentiate between
     // annotation on the first field of a struct and annotation on the struct
     // itself.
     if (VTy != CGM.Int8PtrTy)
       V = Builder.Insert(new llvm::BitCastInst(V, CGM.Int8PtrTy));
     V = EmitAnnotationCall(F, V, I->getAnnotation(), D->getLocation());
     V = Builder.CreateBitCast(V, VTy);
   }
 
   return Address(V, Addr.getAlignment());
 }
 
-CodeGenFunction::CGCapturedStmtInfo::~CGCapturedStmtInfo() { }
+CodeGenFunction::CGCapturedStmtInfo::~CGCapturedStmtInfo() {}
 
 CodeGenFunction::SanitizerScope::SanitizerScope(CodeGenFunction *CGF)
     : CGF(CGF) {
   assert(!CGF->IsSanitizerScope);
   CGF->IsSanitizerScope = true;
 }
 
 CodeGenFunction::SanitizerScope::~SanitizerScope() {

lib/Driver/Tools.cpp

static bool addSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
   }
   // If there is a static runtime with no dynamic list, force all the symbols
   // to be dynamic to be sure we export sanitizer interface functions.
   if (AddExportDynamic)
     CmdArgs.push_back("-export-dynamic");
   return !StaticRuntimes.empty();
 }
 
+static bool addXRayRuntime(const ToolChain &TC, const ArgList &Args,
+                           ArgStringList &CmdArgs) {
+  if (Args.hasArg(options::OPT_fxray_instrument,
+                  options::OPT_fnoxray_instrument, false)) {
+    CmdArgs.push_back("-whole-archive");
+    CmdArgs.push_back(TC.getCompilerRTArgString(Args, "xray", false));
+    CmdArgs.push_back("-no-whole-archive");
+    return true;
+  }
+  return false;
+}
+
+static void linkXRayRuntimeDeps(const ToolChain &TC, ArgStringList &CmdArgs) {
+  CmdArgs.push_back("--no-as-needed");
+  CmdArgs.push_back("-lpthread");
+  CmdArgs.push_back("-lrt");
+  CmdArgs.push_back("-lm");
+  CmdArgs.push_back("-latomic");
+  if (TC.getTriple().getOS() != llvm::Triple::FreeBSD)
+    CmdArgs.push_back("-ldl");
+}
+
 static bool areOptimizationsEnabled(const ArgList &Args) {
   // Find the last -O arg and see if it is non-zero.
   if (Arg *A = Args.getLastArg(options::OPT_O_Group))
     return !A->getOption().matches(options::OPT_O0);
   // Defaults to -O0.
   return false;
 }
 
#endif
   }
 
   if (!Args.hasFlag(options::OPT_funique_section_names,
                     options::OPT_fno_unique_section_names, true))
     CmdArgs.push_back("-fno-unique-section-names");
 
   Args.AddAllArgs(CmdArgs, options::OPT_finstrument_functions);
 
+  if (Args.hasArg(options::OPT_fxray_instrument,
+                  options::OPT_fnoxray_instrument, false)) {
+    CmdArgs.push_back("-fxray-instrument");
+    if (Arg *A = Args.getLastArg(options::OPT_fxray_instruction_threshold_,
+                                 options::OPT_fxray_instruction_threshold_EQ)) {
+      CmdArgs.push_back("-fxray-instruction-threshold");
+      CmdArgs.push_back(A->getValue());
+    }
+  }
+
   addPGOAndCoverageFlags(C, D, Output, Args, CmdArgs);
 
   // Add runtime flag for PS4 when PGO or Coverage are enabled.
   if (getToolChain().getTriple().isPS4CPU())
     addPS4ProfileRTArgs(getToolChain(), Args, CmdArgs);
 
   // Pass options for controlling the default header search paths.
   if (Args.hasArg(options::OPT_nostdinc)) {
void gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
 
   if (D.isUsingLTO())
     AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
     CmdArgs.push_back("--no-demangle");
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
+  bool NeedsXRayDeps = addXRayRuntime(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
   // The profile runtime also needs access to system libraries.
   getToolChain().addProfileRTLibs(Args, CmdArgs);
 
   if (D.CCCIsCXX() &&
       !Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
     bool OnlyLibstdcxxStatic = Args.hasArg(options::OPT_static_libstdcxx) &&
                                !Args.hasArg(options::OPT_static);
void gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
   if (!Args.hasArg(options::OPT_nostdlib)) {
     if (!Args.hasArg(options::OPT_nodefaultlibs)) {
       if (Args.hasArg(options::OPT_static))
         CmdArgs.push_back("--start-group");
 
       if (NeedsSanitizerDeps)
         linkSanitizerRuntimeDeps(ToolChain, CmdArgs);
 
+      if (NeedsXRayDeps)
+        linkXRayRuntimeDeps(ToolChain, CmdArgs);
+
       bool WantPthread = Args.hasArg(options::OPT_pthread) ||
                          Args.hasArg(options::OPT_pthreads);
 
       if (Args.hasFlag(options::OPT_fopenmp, options::OPT_fopenmp_EQ,
                        options::OPT_fno_openmp, false)) {
         // OpenMP runtimes implies pthreads when using the GNU toolchain.
         // FIXME: Does this really make sense for all GNU toolchains?
         WantPthread = true;

lib/Frontend/CompilerInvocation.cpp

for (const auto &A : Args) {
         Opts.CmdArgs.insert(Opts.CmdArgs.end(), ArgStr.begin(), ArgStr.end());
         // using \00 to seperate each commandline options.
         Opts.CmdArgs.push_back('\0');
       }
     }
   }
 
   Opts.InstrumentFunctions = Args.hasArg(OPT_finstrument_functions);
+  Opts.XRayInstrumentFunctions = Args.hasArg(OPT_fxray_instrument);
+  Opts.XRayInstructionThreshold =
+      getLastArgIntValue(Args, OPT_fxray_instruction_threshold_, 200, Diags);
   Opts.InstrumentForProfiling = Args.hasArg(OPT_pg);
   Opts.EmitOpenCLArgMetadata = Args.hasArg(OPT_cl_kernel_arg_info);
   Opts.CompressDebugSections = Args.hasArg(OPT_compress_debug_sections);
   Opts.RelaxELFRelocations = Args.hasArg(OPT_mrelax_relocations);
   Opts.DebugCompilationDir = Args.getLastArgValue(OPT_fdebug_compilation_dir);
   for (auto A : Args.filtered(OPT_mlink_bitcode_file, OPT_mlink_cuda_bitcode)) {
     unsigned LinkFlags = llvm::Linker::Flags::None;
     if (A->getOption().matches(OPT_mlink_cuda_bitcode))

lib/Sema/SemaDeclAttr.cpp

static void ProcessDeclAttribute(Sema &S, Scope *scope, Decl *D,
 
   // Type safety attributes.
   case AttributeList::AT_ArgumentWithTypeTag:
     handleArgumentWithTypeTagAttr(S, D, Attr);
     break;
   case AttributeList::AT_TypeTagForDatatype:
     handleTypeTagForDatatypeAttr(S, D, Attr);
     break;
-
   case AttributeList::AT_RenderScriptKernel:
     handleSimpleAttribute<RenderScriptKernelAttr>(S, D, Attr);
     break;
+  // XRay attributes.
+  case AttributeList::AT_XRayInstrument:
+    handleSimpleAttribute<XRayInstrumentAttr>(S, D, Attr);
+    break;
   }
 }
 
 /// ProcessDeclAttributeList - Apply all the decl attributes in the specified
 /// attribute list to the specified decl, ignoring any type attributes.
 void Sema::ProcessDeclAttributeList(Scope *S, Decl *D,
                                     const AttributeList *AttrList,
                                     bool IncludeCXX11Attributes) {

test/CodeGen/xray-attributes-supported.cpp

+// RUN: %clang_cc1 %s -fxray-instrument -std=c++11 -x c++ -emit-llvm -o - -triple x86_64-unknown-linux-gnu | FileCheck %s
+
+// Make sure that the LLVM attribute for XRay-annotated functions do show up.
+[[clang::xray_always_instrument]] void foo() {
+// CHECK: define void @_Z3foov() #0
+};
+
+[[clang::xray_never_instrument]] void bar() {
+// CHECK: define void @_Z3barv() #1
+};
+
+// CHECK: #0 = {{.*}}"function-instrument"="xray-always"
+// CHECK: #1 = {{.*}}"function-instrument"="xray-never"

test/Sema/xray-always-instrument-attr.c

+// RUN: %clang_cc1 %s -verify -fsyntax-only -std=c11
+void foo() __attribute__((xray_always_instrument));
+
+struct __attribute__((xray_always_instrument)) a { int x; }; // expected-warning {{'xray_always_instrument' attribute only applies to functions and methods}}
+
+void bar() __attribute__((xray_always_instrument("not-supported"))); // expected-error {{'xray_always_instrument' attribute takes no arguments}}

test/Sema/xray-always-instrument-attr.cpp

+// RUN: %clang_cc1 %s -verify -fsyntax-only -std=c++11 -x c++
+void foo [[clang::xray_always_instrument]] ();
+
+struct [[clang::xray_always_instrument]] a { int x; }; // expected-warning {{'xray_always_instrument' attribute only applies to functions and methods}}
+
+class b {
+ void c [[clang::xray_always_instrument]] ();
+};
+
+void baz [[clang::xray_always_instrument("not-supported")]] (); // expected-error {{'xray_always_instrument' attribute takes no arguments}}