[clang] Adds `-ftime-trace` option to clang that produces Chrome `chrome://tracing` compatible JSON profiling output dumps

[anton-afanasyev]

Summary

This is the first part of time tracing system, I have splitted them cause this part is mostly written by Aras Pranckevicius except of several minor fixes concerning formatting. I'm to commit this in behalf of Aras, we have an arrangment with him.
The second part extends this option adding terminal output making no need for profiling visualization. I can also add xray support though this need is arguable.
The third part is for cleaning up previous attempts of such implementations (like https://reviews.llvm.org/D45619, https://reviews.llvm.org/D43578 and https://reviews.llvm.org/D47196).

This is taken from GitHub PR: https://github.com/aras-p/llvm-project-20170507/pull/2
Here is cfe maillist subject discussion: http://lists.llvm.org/pipermail/cfe-dev/2019-January/060836.html

[RKSimon]

Ideally I think you need more clang/frontend experts reviewers - many of us on the reviewers list tend to work mainly in llvm.

Test cases would be good as well - even if its just basic sanity tests for command line args etc.

clang/lib/Sema/Sema.cpp

113 ↗

(On Diff #188373)

remove braces

clang/lib/Sema/SemaTemplateInstantiate.cpp

24 ↗

(On Diff #188373)

nfc change?

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

23 ↗

(On Diff #188373)

nfc change?

clang/tools/driver/cc1_main.cpp

201 ↗

(On Diff #188373)

remove braces

[anton-afanasyev]

Ok, I'm to add tests and reviewers.

clang/lib/Sema/Sema.cpp

113 ↗

(On Diff #188373)

Ok

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

23 ↗

(On Diff #188373)

What do you mean? This change is not NFC.

clang/tools/driver/cc1_main.cpp

201 ↗

(On Diff #188373)

Ok

[RKSimon]

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

23 ↗

(On Diff #188373)

Moving the "clang/Sema/SemaInternal.h" include?

[anton-afanasyev]

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

23 ↗

(On Diff #188373)

Ok, I'm to move this line back. This was done by clang-format.

[anton-afanasyev]

Ping!
Should I add more FE guys to review this?

[RKSimon]

@rsmith Any suggestions for good reviewers for this please?

[zturner]

+reid and hans, as they might be interested in this.

[rnk]

Very cool! I'll take a look, I wasn't aware this had been rebased and uploaded, I was thinking about doing it myself yesterday as a side project.

As I think I've said elsewhere, I'm really excited to give users the tools they need to analyze why their code compiles slowly. If we just give them the tools, maybe they will restructure their code on their own and they won't come asking, "why is the compiler so slow on my (many transitive includes | template metaprogram)?".

I'll take a look and try to get back with some thoughts.

[rnk]

clang/lib/Parse/ParseAST.cpp

172 ↗

(On Diff #188881)

I think you may want to move this to clang::EmitBackendOutput, which is closer to real "backend-y" actions. I don't think there's any other heavy lifting that happens in HandleTranslationUnit, it looks like diagnostic teardown and setup for calling EmitBackendOutput.

clang/lib/Parse/ParseDeclCXX.cpp

3115–3118 ↗

(On Diff #188881)

For example, this would be simplified by a TimeTraceScope callable overload.

clang/lib/Parse/ParseTemplate.cpp

237–238 ↗

(On Diff #188881)

I guess this would be simplified as well with a callable.

clang/lib/Sema/Sema.cpp

98 ↗

(On Diff #188881)

This doesn't match the LLVM naming and auto usage conventions: https://llvm.org/docs/CodingStandards.html#use-auto-type-deduction-to-make-code-more-readable

Yes, there is an active debate about changing the way we name variables, but please just match what we have for now.

llvm/lib/IR/LegacyPassManager.cpp

1632–1634 ↗

(On Diff #188881)

Someone is going to have to figure out where the equivalent annotations should live in the new passmanager. I wasn't able to find it just by looking myself, so I won't ask you to do it. I guess it's in llvm/include/llvm/IR/PassManager.h.

llvm/lib/Support/TimeProfiler.cpp

28 ↗

(On Diff #188881)

Here and else where things should be named the LLVM way for consistency:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly
escapeString, Src, etc. Tedious, I know.

70 ↗

(On Diff #188881)

I'm tempted to micro-optimize this with StringRef and StringSaver, but I think it's unnecessary. Calling malloc may affect the profile, but probably not much.

80 ↗

(On Diff #188881)

Comments in this file need to be complete sentences with a leading capital and full stop.

llvm/lib/Support/TimeProfiler.h

53 ↗

(On Diff #188881)

It'd be nice if these took StringRefs, it would avoid a fair amount of .data() and .c_str(), but it messes up TIME_TRACE_OR_NULL. Another way to delay the work would be to have an llvm::function_ref<StringRef()> detail overload. The callable also allows the user to bind variables like this:

TimeTraceScope timeScope("ParseTag", [&]() {
  if (auto *TD = dyn_cast_or_null<TagDecl>(D))
    return TD->getNameAsStr();
  return "";
});

Instead of the isa<TagDecl>(D) ? cast<TagDecl>(D)->getNameAsStr() : "" pattern that I see, which repeats TagDecl.

[rnk]

llvm/lib/Support/TimeProfiler.h

1 ↗

(On Diff #188881)

I applied this patch locally to try it, and I noticed this header should be in llvm/include/llvm/Support, not llvm/lib/Support.

[rnk]

llvm/lib/IR/LegacyPassManager.cpp

1686 ↗

(On Diff #188881)

I think these OptModule and OptFunction labels may need some improvement. For a backend-heavy compilation like LLVM's X86ISelLowering.cpp, these labels aren't as good as they could be:

I think it's fine to leave that for later, though.

[anton-afanasyev]

clang/lib/Parse/ParseAST.cpp

172 ↗

(On Diff #188881)

Yes, thanks.

clang/lib/Sema/Sema.cpp

98 ↗

(On Diff #188881)

Ok

llvm/lib/IR/LegacyPassManager.cpp

1632–1634 ↗

(On Diff #188881)

Ok, I'm to look for it.

llvm/lib/Support/TimeProfiler.cpp

28 ↗

(On Diff #188881)

Ok

llvm/lib/Support/TimeProfiler.h

1 ↗

(On Diff #188881)

Oops, you're right, this is accidentally moved in patch! Thanks.

[anton-afanasyev]

llvm/lib/IR/LegacyPassManager.cpp

1632–1634 ↗

(On Diff #188881)

Looks like PassManager.h changing is enough to support new PassManager. But I prefer to make this change in subsequent commits.

1686 ↗

(On Diff #188881)

Yes, leaving this for later. The subsequent commits are planned.

llvm/lib/Support/TimeProfiler.cpp

70 ↗

(On Diff #188881)

Ok, I'm not changing this.

[anton-afanasyev]

Updated following @rnk review notes.

[rnk]

Other than the lifetime issue, I think this is basically ready.

clang/lib/Sema/SemaTemplateInstantiate.cpp

2014 ↗

(On Diff #191934)

Shoot, I think the callable should probably return std::string instead of StringRef, otherwise this looks like it will be a use-after-free. You allocate a temporary std::string, get a pointer to the characters, return, the string is destroyed, and then UAF.

With a std::string return type, you won't need .c_str().

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

4089 ↗

(On Diff #191934)

.c_str() is unnecessary

llvm/include/llvm/Support/TimeProfiler.h

54–58 ↗

(On Diff #191934)

I think you can replace const char * in these prototypes with StringRef to save a few .data() calls at some call sites.

As mentioned before, I think the callable needs to return std::string, though.

[takuto.ikuta]

llvm/lib/Support/TimeProfiler.cpp

28 ↗

(On Diff #191934)

you can pass StringRef here and remove .data() or .c_str() from caller.

33 ↗

(On Diff #191934)

Include escape for '/' too.
https://tools.ietf.org/html/rfc8259#section-7

44 ↗

(On Diff #191934)

I prefer to use isPrint here.
https://github.com/llvm/llvm-project/blob/2946cd701067404b99c39fb29dc9c74bd7193eb3/llvm/include/llvm/ADT/StringExtras.h#L105

72 ↗

(On Diff #191934)

I prefer to write

Stack.push_back(std::move(E));

or

Stack.emplace_back(steady_clock::now(), {}, Name, Detail);

103 ↗

(On Diff #191934)

Don't we want to use json utility for this?
https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm/Support/JSON.h

147 ↗

(On Diff #191934)

better to have one hash map so that we don't need to call 2 lookup in L92 and L93.
Also StringMap may be faster than unordered_map.

[anton-afanasyev]

llvm/lib/Support/TimeProfiler.cpp

33 ↗

(On Diff #191934)

Yes, thanks!

[takuto.ikuta]

clang/lib/CodeGen/BackendUtil.cpp

1352 ↗

(On Diff #192268)

We don't need explicit StringRef constructor call here. Just passing "" is enough.

clang/lib/Parse/ParseAST.cpp

154 ↗

(On Diff #192268)

Remove StringRef?

clang/lib/Serialization/GlobalModuleIndex.cpp

131 ↗

(On Diff #192268)

Remove StringRef?

745 ↗

(On Diff #192268)

Remove StringRef?

clang/tools/driver/cc1_main.cpp

224 ↗

(On Diff #192268)

Remove StringRef?

llvm/lib/Support/TimeProfiler.cpp

44 ↗

(On Diff #192268)

include StringExtras.h for this?

[anton-afanasyev]

clang/lib/Parse/ParseAST.cpp

154 ↗

(On Diff #192268)

I use StringRef("") to explicitly cast to one of overloaded TimeScope(StringRef, StringRef) and TimeScope(StringRef, fuction_ref(std::string())).

llvm/lib/Support/TimeProfiler.cpp

44 ↗

(On Diff #192268)

Should one do it? It's already implicitly included.

103 ↗

(On Diff #191934)

This implementation looks compact and fast. I've read proposal for this library https://docs.google.com/document/d/1OEF9IauWwNuSigZzvvbjc1cVS1uGHRyGTXaoy3DjqM4 , it's recent, so I'm not sure it's stable and speed optimized. Do you actually can advise it? I can do it in the next refactor commit as well.

[takuto.ikuta]

clang/lib/Parse/ParseAST.cpp

154 ↗

(On Diff #192268)

I think function_ref(std::string()) does not have constructor receiving StringRef, so I feel explicit cast is redundant.

llvm/lib/Support/TimeProfiler.cpp

44 ↗

(On Diff #192268)

I think it is better to do, because if someone removes the StringExtras.h include used for this file, they need to include header in this file at the same time. That may require to touch unrelated files in their change.

75 ↗

(On Diff #192268)

s/Begin/begin/

103 ↗

(On Diff #191934)

Hmm, I think using json library is preferred because we don't need to take care the correctness of json format.
Confirming correctness of json format with many escaped literals is bit hard and I'm afraid to miss json format error.

[anton-afanasyev]

clang/lib/Parse/ParseAST.cpp

154 ↗

(On Diff #192268)

The compiler tries to use function_ref<..>(Callable&&, ...) constructor and instantiates function_ref<std::string()>::function_ref<char const (&)[1]>, so one gets error like:

error: call to constructor of 'llvm::TimeTraceScope' is ambiguous
    llvm::TimeTraceScope TimeScope("Frontend", "");
                         ^         ~~~~~~~~~~~~~~
.../include/llvm/Support/TimeProfiler.h:54:3: note: candidate constructor
  TimeTraceScope(StringRef Name, StringRef Detail) {
  ^
.../include/llvm/Support/TimeProfiler.h:58:3: note: candidate constructor
  TimeTraceScope(StringRef Name, llvm::function_ref<std::string()> Detail) {
  ^

[rnk]

lgtm

I think this is ready. We can adjust the overloads after the fact. I'd like to get the feature in so we can make improvements independently.

clang/lib/Parse/ParseAST.cpp

154 ↗

(On Diff #192268)

Oh no. We are too clever for ourselves. :(

So, we'd need to make the enable_if logic in the function_ref constructor more clever to prevent it from being instantiated for non-callables.

[anton-afanasyev]

In D58675#1448112, @rnk wrote:

lgtm

I think this is ready. We can adjust the overloads after the fact. I'd like to get the feature in so we can make improvements independently.

Ok, so I'm to commit this and fix three points in the next commit(s):

1. Improve hashing (it's easy).
2. Use json utility.
3. Improve function_ref constructor and delete StringRef("...") explicit casting.

[thakis]

Looks like this landed without tests. Please add tests in a follow-up.

Also, it looks like the flag is currently a cc1 flag. This should probably be a CoreOption instead so that both the gcc-style and the cl-style drivers expose it. Right now, users can't really use this if I read the patch right (without using -Xclang, which is discouraged.)

[thakis]

ps: Hooray for landing this, and thanks for the cool feature!

[anton-afanasyev]

In D58675#1449051, @thakis wrote:

Looks like this landed without tests. Please add tests in a follow-up.

Also, it looks like the flag is currently a cc1 flag. This should probably be a CoreOption instead so that both the gcc-style and the cl-style drivers expose it. Right now, users can't really use this if I read the patch right (without using -Xclang, which is discouraged.)

Ok, thanks, I'm to add tests in a follow-up.

It works like clang -ftime-trace main.cpp now. Not sure I know where CoreOption is, but I'm to figure out it, thanks.

[anton-afanasyev]

In D58675#1449053, @thakis wrote:

ps: Hooray for landing this, and thanks for the cool feature!

Thanks! It's @aras_p accomplishment.

[anton-afanasyev]

Fix hashing update: https://reviews.llvm.org/D60404

[anton-afanasyev]

Use native llvm JSON library update: https://reviews.llvm.org/D60609

[lebedev.ri]

Some post-commit review (please submit a new review, don't replace this diff)
As usual, the incorrect license headers keep slipping through.

llvm/trunk/include/llvm/Support/TimeProfiler.h

5–6

OOOPS, wrong license.

53

Did you mean to explicitly prohibit all the default constructors / operator=?

llvm/trunk/lib/Support/TimeProfiler.cpp

5–6

Wrong license.

29–30

SmallString<32> ?
Also, it is safe to OS.reserve(Src.size())

59–60

SmallString<32>?

70–71

Why not either take StringRef arg, or at least std::move() it when creating Entry?

89

llvm::find_if(llvm::reverse(), )

99

Why pass std::unique_ptr ?
Just raw_pwrite_stream &OS

122

Elide {}

123

llvm::sort <- this is a correctness issue.

144–145

Would it make sense to make these SmallVector?

146–147

1. Eww, std::unordered_map, that will have *horrible* perf.
2. Eww, map with key = string. Use llvm::StringMap

162

Just raw_pwrite_stream &OS?

[lebedev.ri]

llvm/trunk/lib/Support/TimeProfiler.cpp

29–30

Also, you probably want to add raw_svector_ostream ontop, and << into it.

[anton-afanasyev]

In D58675#1465336, @lebedev.ri wrote:

Some post-commit review (please submit a new review, don't replace this diff)
As usual, the incorrect license headers keep slipping through.

Ok, I've made a separate review for this: https://reviews.llvm.org/D60663

llvm/trunk/include/llvm/Support/TimeProfiler.h

5–6

Yes, thanks.

53

Ok, I'm to add lines like TimeTraceScope() = delete; here.

llvm/trunk/lib/Support/TimeProfiler.cpp

5–6

Yes, I'm to fix it.

29–30

This function has been already dropped here https://reviews.llvm.org/D60609 . I'm to submit code without it if json library using are ok for performance.

59–60

Hmm, would it make a sense for Detail? Entrys are heap-allocated, the actual size is ranging from several bytes to several hundreds. Also, getQualifiedNameAsString() which is usually used for Detail returns std::string.

70–71

Do you mean std::move(Name)? Ok.

89

Hmm, one need not [rbegin(), rend()], but [rbegin()++,rend()], so have to explicitly specify begin and end, llvm::reverse(Stack) is inappropriate here.

99

Yes, thanks!
This was blindly copied from CompilerInstance::createOutputFile() return type.

122

Ok, thanks.

123

Yes, thanks!

144–145

Ok, I'm to change it to

SmallVector<Entry, 16> Stack;  
SmallVector<Entry, 128> Entries;

Stack size may be enough for small sources while Entries usually amounts many thousands of Entrys.

146–147

You are right, but this is already fixed and submitted in this follow-up: https://reviews.llvm.org/D60404

162

Yes, thanks.

[lebedev.ri]

In D58675#1465706, @anton-afanasyev wrote:

In D58675#1465336, @lebedev.ri wrote:

Some post-commit review (please submit a new review, don't replace this diff)
As usual, the incorrect license headers keep slipping through.

Ok, I've made a separate review for this: https://reviews.llvm.org/D60663

Thank you.

[anton-afanasyev]

Patch by @thakis for BE passes tracing: https://reviews.llvm.org/D60782

[lebedev.ri]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

This isn't covered by any timer; if you look in BackendUtil.cpp,
EmitAssemblyHelper actually has CodeGenerationTime("codegen", "Code Generation Time") timer.

llvm/trunk/lib/IR/LegacyPassManager.cpp

1686

I think this may be the wrong place for this.
This includes the entirety of the pipeline, including all of llvm back-end stuff.

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Thanks, I'm to add it.

llvm/trunk/lib/IR/LegacyPassManager.cpp

1686

Ok, I'm just to delete this line, this block is outputted by MPPassManager::runOnModule() as well. This is just a section to see module name rather than to cover proper part of pipeline.

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Hmm, I've figured out this isn't needed: such new timer mostly coincides with "Backend" timer (above). Legacy Timer CodeGenerationTime(...) is bad example of doing right timing.

[lebedev.ri]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

"Mostly coincides" may not be the best way to approach fine-grained timings, i think? :)

I have noticed this because when i looked at the produced time flame graph,
there's large section in the end that is covered only by "Backend" timer,
but nothing else. Now, i'm not going to say whether or not that extra section
should or should not be within "Backend" timer, but it certainly should *also*
be within "codegen" timer. Or is there no codegen time spent there?

VC5Decompressor.cpp.json335 KBDownload

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

"Mostly coincides" here means "identical" I believe, the difference is auxiliary stuff.
Please look at clang::EmitBackendOutput(), "Backend" timer is outer for "codegen" one.

[lebedev.ri]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Then we are talking about different things using the same name.
There are two distinct codegen steps:

1. clang AST -> LLVM IR codegen

(after that, all the opt passes run)

2. LLVM IR -> final assembly. This happens after all the opt middle-end passes.

Those are *different* codegen's.

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Yes, and step 1 is named as "CodeGen Function" whereas step 2 is named just "Backend".

[lebedev.ri]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Aha. So this is what i expected to see, apparently:

new.json263 KBDownload

diff --git a/clang/lib/CodeGen/BackendUtil.cpp b/clang/lib/CodeGen/BackendUtil.cpp
index 71ae8fd4fb0..d89d12612f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -910,6 +910,7 @@ void EmitAssemblyHelper::EmitAssembly(BackendAction Action,
 
   {
     PrettyStackTraceString CrashInfo("Code generation");
+    llvm::TimeTraceScope TimeScope("BackendCodegen", StringRef(""));
     CodeGenPasses.run(*TheModule);
   }

But that actually brings me to another question - what about PrettyStackTraceString CrashInfo("Per-function optimization");?
Should that be wrapped into some section, too?
I'm less certain here.

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Ok, you've talked about Timer CodeGenerationTime, which corresponds to Backend scope.
As for this BackendCodegen, adding this I would prefer to add adjacent "Per-function optimization" and "Per-module optimization passes" together.

[anton-afanasyev]

cfe/trunk/lib/CodeGen/BackendUtil.cpp

1426–1431

Changes are here: https://reviews.llvm.org/D68161

cfe/trunk/include/clang/Basic/CodeGenOptions.def

 CODEGENOPT(SanitizeStats     , 1, 0) ///< Collect statistics for sanitizers.
 CODEGENOPT(SimplifyLibCalls  , 1, 1) ///< Set when -fbuiltin is enabled.
 CODEGENOPT(SoftFloat         , 1, 0) ///< -soft-float.
 CODEGENOPT(SpeculativeLoadHardening, 1, 0) ///< Enable speculative load hardening.
 CODEGENOPT(FineGrainedBitfieldAccesses, 1, 0) ///< Enable fine-grained bitfield accesses.
 CODEGENOPT(StrictEnums       , 1, 0) ///< Optimize based on strict enum definition.
 CODEGENOPT(StrictVTablePointers, 1, 0) ///< Optimize based on the strict vtable pointers
 CODEGENOPT(TimePasses        , 1, 0) ///< Set when -ftime-report is enabled.
+CODEGENOPT(TimeTrace         , 1, 0) ///< Set when -ftime-trace is enabled.
 CODEGENOPT(UnrollLoops       , 1, 0) ///< Control whether loops are unrolled.
 CODEGENOPT(RerollLoops       , 1, 0) ///< Control whether loops are rerolled.
 CODEGENOPT(NoUseJumpTables   , 1, 0) ///< Set when -fno-jump-tables is enabled.
 CODEGENOPT(UnsafeFPMath      , 1, 0) ///< Allow unsafe floating point optzns.
 CODEGENOPT(UnwindTables      , 1, 0) ///< Emit unwind tables.
 CODEGENOPT(VectorizeLoop     , 1, 0) ///< Run loop vectorizer.
 CODEGENOPT(VectorizeSLP      , 1, 0) ///< Run SLP vectorizer.
 CODEGENOPT(ProfileSampleAccurate, 1, 0) ///< Sample profile is accurate.

cfe/trunk/include/clang/Driver/Options.td

 // Just silence warnings about -Wlarger-than for now.
 def Wlarger_than_EQ : Joined<["-"], "Wlarger-than=">, Group<clang_ignored_f_Group>;
 def Wlarger_than_ : Joined<["-"], "Wlarger-than-">, Alias<Wlarger_than_EQ>;
 def Wframe_larger_than_EQ : Joined<["-"], "Wframe-larger-than=">, Group<f_Group>, Flags<[DriverOption]>;
 
 def : Flag<["-"], "fterminated-vtables">, Alias<fapple_kext>;
 def fthreadsafe_statics : Flag<["-"], "fthreadsafe-statics">, Group<f_Group>;
 def ftime_report : Flag<["-"], "ftime-report">, Group<f_Group>, Flags<[CC1Option]>;
+def ftime_trace : Flag<["-"], "ftime-trace">, Group<f_Group>, Flags<[CC1Option]>;
 def ftlsmodel_EQ : Joined<["-"], "ftls-model=">, Group<f_Group>, Flags<[CC1Option]>;
 def ftrapv : Flag<["-"], "ftrapv">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Trap on integer overflow">;
 def ftrapv_handler_EQ : Joined<["-"], "ftrapv-handler=">, Group<f_Group>,
   MetaVarName<"<function name>">,
   HelpText<"Specify the function to be called on overflow">;
 def ftrapv_handler : Separate<["-"], "ftrapv-handler">, Group<f_Group>, Flags<[CC1Option]>;
 def ftrap_function_EQ : Joined<["-"], "ftrap-function=">, Group<f_Group>, Flags<[CC1Option]>,

cfe/trunk/include/clang/Frontend/FrontendOptions.h

public:
   unsigned ShowHelp : 1;
 
   /// Show frontend performance metrics and statistics.
   unsigned ShowStats : 1;
 
   /// Show timers for individual actions.
   unsigned ShowTimers : 1;
 
+  /// Output time trace profile.
+  unsigned TimeTrace : 1;
+
   /// Show the -version text.
   unsigned ShowVersion : 1;
 
   /// Apply fixes even if there are unfixable errors.
   unsigned FixWhatYouCan : 1;
 
   /// Apply fixes only for warnings.
   unsigned FixOnlyWarnings : 1;
public:
   std::string AuxTriple;
 
   /// Filename to write statistics to.
   std::string StatsFile;
 
 public:
   FrontendOptions()
       : DisableFree(false), RelocatablePCH(false), ShowHelp(false),
-        ShowStats(false), ShowTimers(false), ShowVersion(false),
-        FixWhatYouCan(false), FixOnlyWarnings(false), FixAndRecompile(false),
-        FixToTemporaries(false), ARCMTMigrateEmitARCErrors(false),
-        SkipFunctionBodies(false), UseGlobalModuleIndex(true),
-        GenerateGlobalModuleIndex(true), ASTDumpDecls(false),
-        ASTDumpLookups(false), BuildingImplicitModule(false),
-        ModulesEmbedAllFiles(false), IncludeTimestamps(true) {}
+        ShowStats(false), ShowTimers(false), TimeTrace(false),
+        ShowVersion(false), FixWhatYouCan(false), FixOnlyWarnings(false),
+        FixAndRecompile(false), FixToTemporaries(false),
+        ARCMTMigrateEmitARCErrors(false), SkipFunctionBodies(false),
+        UseGlobalModuleIndex(true), GenerateGlobalModuleIndex(true),
+        ASTDumpDecls(false), ASTDumpLookups(false),
+        BuildingImplicitModule(false), ModulesEmbedAllFiles(false),
+        IncludeTimestamps(true) {}
 
   /// getInputKindForExtension - Return the appropriate input kind for a file
   /// extension. For example, "c" would return InputKind::C.
   ///
   /// \return The input kind for the extension, or InputKind::Unknown if the
   /// extension is not recognized.
   static InputKind getInputKindForExtension(StringRef Extension);
 };
 
 } // namespace clang
 
 #endif // LLVM_CLANG_FRONTEND_FRONTENDOPTIONS_H

cfe/trunk/lib/CodeGen/BackendUtil.cpp

 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Passes/PassBuilder.h"
 #include "llvm/Passes/PassPlugin.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TimeProfiler.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Transforms/Coroutines.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/AlwaysInliner.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"

 void clang::EmitBackendOutput(DiagnosticsEngine &Diags,
                               const HeaderSearchOptions &HeaderOpts,
                               const CodeGenOptions &CGOpts,
                               const clang::TargetOptions &TOpts,
                               const LangOptions &LOpts,
                               const llvm::DataLayout &TDesc, Module *M,
                               BackendAction Action,
                               std::unique_ptr<raw_pwrite_stream> OS) {
+
+  llvm::TimeTraceScope TimeScope("Backend", StringRef(""));
+
   std::unique_ptr<llvm::Module> EmptyModule;
   if (!CGOpts.ThinLTOIndexFile.empty()) {
     // If we are performing a ThinLTO importing compile, load the function index
     // into memory and pass it into runThinLTOBackend, which will run the
     // function importer and invoke LTO passes.
     Expected<std::unique_ptr<ModuleSummaryIndex>> IndexOrErr =
         llvm::getModuleSummaryIndexForFile(CGOpts.ThinLTOIndexFile,
                                            /*IgnoreEmptyThinLTOIndexFile*/true);
if (CombinedIndex) {
       // the compilation as CombinedIndex does not contain all required
       // information.
       EmptyModule = llvm::make_unique<llvm::Module>("empty", M->getContext());
       EmptyModule->setTargetTriple(M->getTargetTriple());
       M = EmptyModule.get();
     }
   }
 
   EmitAssemblyHelper AsmHelper(Diags, HeaderOpts, CGOpts, TOpts, LOpts, M);
 
   if (CGOpts.ExperimentalNewPassManager)
     AsmHelper.EmitAssemblyWithNewPassManager(Action, std::move(OS));
   else
     AsmHelper.EmitAssembly(Action, std::move(OS));
 
   // Verify clang's TargetInfo DataLayout against the LLVM TargetMachine's
   // DataLayout.
   if (AsmHelper.TM) {
     std::string DLDesc = M->getDataLayout().getStringRepresentation();
     if (DLDesc != TDesc.getStringRepresentation()) {
       unsigned DiagID = Diags.getCustomDiagID(
           DiagnosticsEngine::Error, "backend data layout '%0' does not match "

cfe/trunk/lib/CodeGen/CodeGenModule.cpp

 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ProfileSummary.h"
 #include "llvm/ProfileData/InstrProfReader.h"
 #include "llvm/Support/CodeGen.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/MD5.h"
+#include "llvm/Support/TimeProfiler.h"
 
 using namespace clang;
 using namespace CodeGen;
 
 static llvm::cl::opt<bool> LimitedCoverage(
     "limited-coverage-experimental", llvm::cl::ZeroOrMore, llvm::cl::Hidden,
     llvm::cl::desc("Emit limited coverage mapping information (experimental)"),
     llvm::cl::init(false));
PrettyStackTraceDecl CrashInfo(const_cast<ValueDecl *>(D), D->getLocation(),
                                  "Generating code for declaration");
 
   if (const auto *FD = dyn_cast<FunctionDecl>(D)) {
     // At -O0, don't generate IR for functions with available_externally
     // linkage.
     if (!shouldEmitFunction(GD))
       return;
 
+    llvm::TimeTraceScope TimeScope(
+        "CodeGen Function", [&]() { return FD->getQualifiedNameAsString(); });
+
     if (const auto *Method = dyn_cast<CXXMethodDecl>(D)) {
       // Make sure to emit the definition(s) before we emit the thunks.
       // This is necessary for the generation of certain thunks.
       if (isa<CXXConstructorDecl>(Method) || isa<CXXDestructorDecl>(Method))
         ABI->emitCXXStructor(GD);
       else if (FD->isMultiVersion())
         EmitMultiVersionFunctionDefinition(GD, GV);
       else

cfe/trunk/lib/Driver/ToolChains/Clang.cpp

#endif
   if (Args.getLastArg(options::OPT_fapple_kext) ||
       (Args.hasArg(options::OPT_mkernel) && types::isCXX(InputType)))
     CmdArgs.push_back("-fapple-kext");
 
   Args.AddLastArg(CmdArgs, options::OPT_fobjc_sender_dependent_dispatch);
   Args.AddLastArg(CmdArgs, options::OPT_fdiagnostics_print_source_range_info);
   Args.AddLastArg(CmdArgs, options::OPT_fdiagnostics_parseable_fixits);
   Args.AddLastArg(CmdArgs, options::OPT_ftime_report);
+  Args.AddLastArg(CmdArgs, options::OPT_ftime_trace);
   Args.AddLastArg(CmdArgs, options::OPT_ftrapv);
   Args.AddLastArg(CmdArgs, options::OPT_malign_double);
 
   if (Arg *A = Args.getLastArg(options::OPT_ftrapv_handler_EQ)) {
     CmdArgs.push_back("-ftrapv-handler");
     CmdArgs.push_back(A->getValue());
   }
 

cfe/trunk/lib/Frontend/CompilerInstance.cpp

 #include "llvm/Support/Errc.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/LockFileManager.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/Signals.h"
+#include "llvm/Support/TimeProfiler.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include <sys/stat.h>
 #include <system_error>
 #include <time.h>
 #include <utility>
 
 using namespace clang;

 static bool
 compileModuleImpl(CompilerInstance &ImportingInstance, SourceLocation ImportLoc,
                   StringRef ModuleName, FrontendInputFile Input,
                   StringRef OriginalModuleMapFile, StringRef ModuleFileName,
                   llvm::function_ref<void(CompilerInstance &)> PreBuildStep =
                       [](CompilerInstance &) {},
                   llvm::function_ref<void(CompilerInstance &)> PostBuildStep =
                       [](CompilerInstance &) {}) {
+  llvm::TimeTraceScope TimeScope("Module Compile", ModuleName);
+
   // Construct a compiler invocation for creating this module.
   auto Invocation =
       std::make_shared<CompilerInvocation>(ImportingInstance.getInvocation());
 
   PreprocessorOptions &PPOpts = Invocation->getPreprocessorOpts();
 
   // For any options that aren't intended to affect how a module is built,
   // reset them to their default values.
if (Known != KnownModules.end()) {
     if (!ModuleManager)
       createModuleManager();
 
     llvm::Timer Timer;
     if (FrontendTimerGroup)
       Timer.init("loading." + ModuleFileName, "Loading " + ModuleFileName,
                  *FrontendTimerGroup);
     llvm::TimeRegion TimeLoading(FrontendTimerGroup ? &Timer : nullptr);
+    llvm::TimeTraceScope TimeScope("Module Load", ModuleName);
 
     // Try to load the module file. If we are not trying to load from the
     // module cache, we don't know how to rebuild modules.
     unsigned ARRFlags = Source == ModuleCache ?
                         ASTReader::ARR_OutOfDate | ASTReader::ARR_Missing :
                         Source == PrebuiltModulePath ?
                             0 :
                             ASTReader::ARR_ConfigurationMismatch;

cfe/trunk/lib/Frontend/CompilerInvocation.cpp

static InputKind ParseFrontendArgs(FrontendOptions &Opts, ArgList &Args,
   Opts.DisableFree = Args.hasArg(OPT_disable_free);
 
   Opts.OutputFile = Args.getLastArgValue(OPT_o);
   Opts.Plugins = Args.getAllArgValues(OPT_load);
   Opts.RelocatablePCH = Args.hasArg(OPT_relocatable_pch);
   Opts.ShowHelp = Args.hasArg(OPT_help);
   Opts.ShowStats = Args.hasArg(OPT_print_stats);
   Opts.ShowTimers = Args.hasArg(OPT_ftime_report);
+  Opts.TimeTrace = Args.hasArg(OPT_ftime_trace);
   Opts.ShowVersion = Args.hasArg(OPT_version);
   Opts.ASTMergeFiles = Args.getAllArgValues(OPT_ast_merge);
   Opts.LLVMArgs = Args.getAllArgValues(OPT_mllvm);
   Opts.FixWhatYouCan = Args.hasArg(OPT_fix_what_you_can);
   Opts.FixOnlyWarnings = Args.hasArg(OPT_fix_only_warnings);
   Opts.FixAndRecompile = Args.hasArg(OPT_fixit_recompile);
   Opts.FixToTemporaries = Args.hasArg(OPT_fixit_to_temp);
   Opts.ASTDumpDecls = Args.hasArg(OPT_ast_dump);

cfe/trunk/lib/Parse/ParseAST.cpp

 #include "clang/AST/Stmt.h"
 #include "clang/Parse/ParseDiagnostic.h"
 #include "clang/Parse/Parser.h"
 #include "clang/Sema/CodeCompleteConsumer.h"
 #include "clang/Sema/Sema.h"
 #include "clang/Sema/SemaConsumer.h"
 #include "clang/Sema/TemplateInstCallback.h"
 #include "llvm/Support/CrashRecoveryContext.h"
+#include "llvm/Support/TimeProfiler.h"
 #include <cstdio>
 #include <memory>
 
 using namespace clang;
 
 namespace {
 
 /// Resets LLVM's pretty stack state so that stack traces are printed correctly
if (External)
     External->StartTranslationUnit(Consumer);
 
   // If a PCH through header is specified that does not have an include in
   // the source, or a PCH is being created with #pragma hdrstop with nothing
   // after the pragma, there won't be any tokens or a Lexer.
   bool HaveLexer = S.getPreprocessor().getCurrentLexer();
 
   if (HaveLexer) {
+    llvm::TimeTraceScope TimeScope("Frontend", StringRef(""));
     P.Initialize();
     Parser::DeclGroupPtrTy ADecl;
     for (bool AtEOF = P.ParseFirstTopLevelDecl(ADecl); !AtEOF;
          AtEOF = P.ParseTopLevelDecl(ADecl)) {
       // If we got a null return and something *was* parsed, ignore it.  This
       // is due to a top-level semicolon, an action override, or a parse error
       // skipping something.
       if (ADecl && !Consumer->HandleTopLevelDecl(ADecl.get()))

cfe/trunk/lib/Parse/ParseDeclCXX.cpp

 #include "clang/Basic/OperatorKinds.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Parse/ParseDiagnostic.h"
 #include "clang/Parse/RAIIObjectsForParser.h"
 #include "clang/Sema/DeclSpec.h"
 #include "clang/Sema/ParsedTemplate.h"
 #include "clang/Sema/Scope.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/Support/TimeProfiler.h"
 
 using namespace clang;
 
 /// ParseNamespace - We know that the current token is a namespace keyword. This
 /// may either be a top level namespace or a block-level namespace alias. If
 /// there was an inline keyword, it has already been parsed.
 ///
 ///       namespace-definition: [C++: namespace.def]
void Parser::ParseCXXMemberSpecification(SourceLocation RecordLoc,
                                          SourceLocation AttrFixitLoc,
                                          ParsedAttributesWithRange &Attrs,
                                          unsigned TagType, Decl *TagDecl) {
   assert((TagType == DeclSpec::TST_struct ||
          TagType == DeclSpec::TST_interface ||
          TagType == DeclSpec::TST_union  ||
          TagType == DeclSpec::TST_class) && "Invalid TagType!");
 
+  llvm::TimeTraceScope TimeScope("ParseClass", [&]() {
+    if (auto *TD = dyn_cast_or_null<NamedDecl>(TagDecl))
+      return TD->getQualifiedNameAsString();
+    return std::string("<anonymous>");
+  });
+
   PrettyDeclStackTraceEntry CrashInfo(Actions.Context, TagDecl, RecordLoc,
                                       "parsing struct/union/class body");
 
   // Determine whether this is a non-nested class. Note that local
   // classes are *not* considered to be nested classes.
   bool NonNestedClass = true;
   if (!ClassStack.empty()) {
     for (const Scope *S = getCurScope(); S; S = S->getParent()) {

cfe/trunk/lib/Parse/ParseTemplate.cpp

 #include "clang/AST/ASTContext.h"
 #include "clang/AST/DeclTemplate.h"
 #include "clang/Parse/ParseDiagnostic.h"
 #include "clang/Parse/Parser.h"
 #include "clang/Parse/RAIIObjectsForParser.h"
 #include "clang/Sema/DeclSpec.h"
 #include "clang/Sema/ParsedTemplate.h"
 #include "clang/Sema/Scope.h"
+#include "llvm/Support/TimeProfiler.h"
 using namespace clang;
 
 /// Parse a template declaration, explicit instantiation, or
 /// explicit specialization.
 Decl *Parser::ParseDeclarationStartingWithTemplate(
     DeclaratorContext Context, SourceLocation &DeclEnd,
     ParsedAttributes &AccessAttrs, AccessSpecifier AS) {
   ObjCDeclContextSwitch ObjCDC(*this);
Decl *Parser::ParseSingleDeclarationAfterTemplate(
   if (!DeclaratorInfo.hasName()) {
     // If so, skip until the semi-colon or a }.
     SkipUntil(tok::r_brace, StopAtSemi | StopBeforeMatch);
     if (Tok.is(tok::semi))
       ConsumeToken();
     return nullptr;
   }
 
+  llvm::TimeTraceScope TimeScope("ParseTemplate", [&]() {
+    return DeclaratorInfo.getIdentifier() != nullptr
+               ? DeclaratorInfo.getIdentifier()->getName()
+               : "<unknown>";
+  });
+
   LateParsedAttrList LateParsedAttrs(true);
   if (DeclaratorInfo.isFunctionDeclarator())
     MaybeParseGNUAttributes(DeclaratorInfo, &LateParsedAttrs);
 
   if (DeclaratorInfo.isFunctionDeclarator() &&
       isStartOfFunctionDefinition(DeclaratorInfo)) {
 
     // Function definitions are only allowed at file scope and in C++ classes.

cfe/trunk/lib/Sema/Sema.cpp

 #include "clang/Sema/Scope.h"
 #include "clang/Sema/ScopeInfo.h"
 #include "clang/Sema/SemaConsumer.h"
 #include "clang/Sema/SemaInternal.h"
 #include "clang/Sema/TemplateDeduction.h"
 #include "clang/Sema/TemplateInstCallback.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallSet.h"
+#include "llvm/Support/TimeProfiler.h"
+
 using namespace clang;
 using namespace sema;
 
 SourceLocation Sema::getLocForEndOfToken(SourceLocation Loc, unsigned Offset) {
   return Lexer::getLocForEndOfToken(Loc, Offset, SourceMgr, LangOpts);
 }
 
 ModuleLoader &Sema::getModuleLoader() const { return PP.getModuleLoader(); }
virtual void FileChanged(SourceLocation Loc, FileChangeReason Reason,
                            FileID PrevFID) override {
     if (!S)
       return;
     switch (Reason) {
     case EnterFile: {
       SourceManager &SM = S->getSourceManager();
       SourceLocation IncludeLoc = SM.getIncludeLoc(SM.getFileID(Loc));
       if (IncludeLoc.isValid()) {
+        if (llvm::timeTraceProfilerEnabled()) {
+          const FileEntry *FE = SM.getFileEntryForID(SM.getFileID(Loc));
+          llvm::timeTraceProfilerBegin(
+              "Source", FE != nullptr ? FE->getName() : StringRef("<unknown>"));
+        }
+
         IncludeStack.push_back(IncludeLoc);
         S->DiagnoseNonDefaultPragmaPack(
             Sema::PragmaPackDiagnoseKind::NonDefaultStateAtInclude, IncludeLoc);
       }
       break;
     }
     case ExitFile:
-      if (!IncludeStack.empty())
+      if (!IncludeStack.empty()) {
+        if (llvm::timeTraceProfilerEnabled())
+          llvm::timeTraceProfilerEnd();
+
         S->DiagnoseNonDefaultPragmaPack(
             Sema::PragmaPackDiagnoseKind::ChangedStateAtExit,
             IncludeStack.pop_back_val());
+      }
       break;
     default:
       break;
     }
   }
 };
 
 } // end namespace sema
if (ExternalSource) {
       ExternalSource->ReadPendingInstantiations(Pending);
       for (auto PII : Pending)
         if (auto Func = dyn_cast<FunctionDecl>(PII.first))
           Func->setInstantiationIsPending(true);
       PendingInstantiations.insert(PendingInstantiations.begin(),
                                    Pending.begin(), Pending.end());
     }
 
+    {
+      llvm::TimeTraceScope TimeScope("PerformPendingInstantiations",
+                                     StringRef(""));
       PerformPendingInstantiations();
+    }
 
     assert(LateParsedInstantiations.empty() &&
            "end of TU template instantiation should not create more "
            "late-parsed templates");
 
     if (LateTemplateParserCleanup)
       LateTemplateParserCleanup(OpaqueParser);
 

cfe/trunk/lib/Sema/SemaTemplateInstantiate.cpp

 #include "clang/AST/PrettyDeclStackTrace.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Sema/DeclSpec.h"
 #include "clang/Sema/Initialization.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/Template.h"
 #include "clang/Sema/TemplateDeduction.h"
 #include "clang/Sema/TemplateInstCallback.h"
+#include "llvm/Support/TimeProfiler.h"
 
 using namespace clang;
 using namespace sema;
 
 //===----------------------------------------------------------------------===/
 // Template Instantiation Support
 //===----------------------------------------------------------------------===/
 
Sema::InstantiateClass(SourceLocation PointOfInstantiation,
                        TemplateSpecializationKind TSK,
                        bool Complain) {
   CXXRecordDecl *PatternDef
     = cast_or_null<CXXRecordDecl>(Pattern->getDefinition());
   if (DiagnoseUninstantiableTemplate(PointOfInstantiation, Instantiation,
                                 Instantiation->getInstantiatedFromMemberClass(),
                                      Pattern, PatternDef, TSK, Complain))
     return true;
+
+  llvm::TimeTraceScope TimeScope("InstantiateClass", [&]() {
+    return Instantiation->getQualifiedNameAsString();
+  });
+
   Pattern = PatternDef;
 
   // Record the point of instantiation.
   if (MemberSpecializationInfo *MSInfo
         = Instantiation->getMemberSpecializationInfo()) {
     MSInfo->setTemplateSpecializationKind(TSK);
     MSInfo->setPointOfInstantiation(PointOfInstantiation);
   } else if (ClassTemplateSpecializationDecl *Spec

cfe/trunk/lib/Sema/SemaTemplateInstantiateDecl.cpp

 #include "clang/AST/Expr.h"
 #include "clang/AST/ExprCXX.h"
 #include "clang/AST/PrettyDeclStackTrace.h"
 #include "clang/AST/TypeLoc.h"
 #include "clang/Sema/Initialization.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/Template.h"
 #include "clang/Sema/TemplateInstCallback.h"
+#include "llvm/Support/TimeProfiler.h"
 
 using namespace clang;
 
 static bool isDeclWithinFunction(const Decl *D) {
   const DeclContext *DC = D->getDeclContext();
   if (DC->isFunctionOrMethod())
     return true;
 
void Sema::InstantiateFunctionDefinition(SourceLocation PointOfInstantiation,
   if (PatternDecl->isLateTemplateParsed() &&
       !LateTemplateParser) {
     Function->setInstantiationIsPending(true);
     LateParsedInstantiations.push_back(
         std::make_pair(Function, PointOfInstantiation));
     return;
   }
 
+  llvm::TimeTraceScope TimeScope("InstantiateFunction", [&]() {
+    return Function->getQualifiedNameAsString();
+  });
+
   // If we're performing recursive template instantiation, create our own
   // queue of pending implicit instantiations that we will instantiate later,
   // while we're still within our own instantiation context.
   // This has to happen before LateTemplateParser below is called, so that
   // it marks vtables used in late parsed templates as used.
   GlobalEagerInstantiationScope GlobalInstantiations(*this,
                                                      /*Enabled=*/Recursive);
   LocalEagerInstantiationScope LocalInstantiations(*this);

cfe/trunk/lib/Serialization/GlobalModuleIndex.cpp

 //===--- GlobalModuleIndex.cpp - Global Module Index ------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the GlobalModuleIndex class.
 //
 //===----------------------------------------------------------------------===//
 
+
 #include "ASTReaderInternals.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Serialization/ASTBitCodes.h"
 #include "clang/Serialization/GlobalModuleIndex.h"
 #include "clang/Serialization/Module.h"
 #include "clang/Serialization/PCHContainerOperations.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Bitcode/BitstreamReader.h"
 #include "llvm/Bitcode/BitstreamWriter.h"
 #include "llvm/Support/DJB.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/LockFileManager.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/OnDiskHashTable.h"
 #include "llvm/Support/Path.h"
+#include "llvm/Support/TimeProfiler.h"
 #include <cstdio>
 using namespace clang;
 using namespace serialization;
 
 //----------------------------------------------------------------------------//
 // Shared constants
 //----------------------------------------------------------------------------//
 namespace {
typedef llvm::OnDiskIterableChainedHashTable<IdentifierIndexReaderTrait>
     IdentifierIndexTable;
 
 }
 
 GlobalModuleIndex::GlobalModuleIndex(std::unique_ptr<llvm::MemoryBuffer> Buffer,
                                      llvm::BitstreamCursor Cursor)
     : Buffer(std::move(Buffer)), IdentifierIndex(), NumIdentifierLookups(),
       NumIdentifierLookupHits() {
+  llvm::TimeTraceScope TimeScope("Module LoadIndex", StringRef(""));
   // Read the global index.
   bool InGlobalIndexBlock = false;
   bool Done = false;
   while (!Done) {
     llvm::BitstreamEntry Entry = Cursor.advance();
 
     switch (Entry.Kind) {
     case llvm::BitstreamEntry::Error:
if (getModuleFileInfo(File).Signature) {
         return true;
     } else if (Info.StoredSize != File->getSize() ||
                Info.StoredModTime != File->getModificationTime())
       // Verify Size and ModTime.
       return true;
   }
 
   using namespace llvm;
+  llvm::TimeTraceScope TimeScope("Module WriteIndex", StringRef(""));
 
   // Emit the file header.
   Stream.Emit((unsigned)'B', 8);
   Stream.Emit((unsigned)'C', 8);
   Stream.Emit((unsigned)'G', 8);
   Stream.Emit((unsigned)'I', 8);
 
   // Write the block-info block, which describes the records in this bitcode

cfe/trunk/tools/driver/cc1_main.cpp

 #include "llvm/LinkAllPasses.h"
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/OptTable.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/TimeProfiler.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cstdio>
 
 #ifdef CLANG_HAVE_RLIMITS
 #include <sys/resource.h>
 #endif
 
#endif
   // Buffer diagnostics from argument parsing so that we can output them using a
   // well formed diagnostic object.
   IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
   TextDiagnosticBuffer *DiagsBuffer = new TextDiagnosticBuffer;
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, DiagsBuffer);
   bool Success = CompilerInvocation::CreateFromArgs(
       Clang->getInvocation(), Argv.begin(), Argv.end(), Diags);
 
+  if (Clang->getFrontendOpts().TimeTrace)
+    llvm::timeTraceProfilerInitialize();
+
   // Infer the builtin include path if unspecified.
   if (Clang->getHeaderSearchOpts().UseBuiltinIncludes &&
       Clang->getHeaderSearchOpts().ResourceDir.empty())
     Clang->getHeaderSearchOpts().ResourceDir =
       CompilerInvocation::GetResourcesPath(Argv0, MainAddr);
 
   // Create the actual diagnostics engine.
   Clang->createDiagnostics();
   if (!Clang->hasDiagnostics())
     return 1;
 
   // Set an error handler, so that any LLVM backend diagnostics go through our
   // error handler.
   llvm::install_fatal_error_handler(LLVMErrorHandler,
                                   static_cast<void*>(&Clang->getDiagnostics()));
 
   DiagsBuffer->FlushDiagnostics(Clang->getDiagnostics());
   if (!Success)
     return 1;
 
   // Execute the frontend actions.
+  {
+    llvm::TimeTraceScope TimeScope("ExecuteCompiler", StringRef(""));
     Success = ExecuteCompilerInvocation(Clang.get());
+  }
 
   // If any timers were active but haven't been destroyed yet, print their
   // results now.  This happens in -disable-free mode.
   llvm::TimerGroup::printAll(llvm::errs());
 
+  if (llvm::timeTraceProfilerEnabled()) {
+    SmallString<128> Path(Clang->getFrontendOpts().OutputFile);
+    llvm::sys::path::replace_extension(Path, "json");
+    auto profilerOutput =
+        Clang->createOutputFile(Path.str(),
+                                /*Binary=*/false,
+                                /*RemoveFileOnSignal=*/false, "",
+                                /*Extension=*/"json",
+                                /*useTemporary=*/false);
+
+    llvm::timeTraceProfilerWrite(profilerOutput);
+    llvm::timeTraceProfilerCleanup();
+  }
+
   // Our error handler depends on the Diagnostics object, which we're
   // potentially about to delete. Uninstall the handler now so that any
   // later errors use the default handling behavior instead.
   llvm::remove_fatal_error_handler();
 
   // When running with -disable-free, don't do any destruction or shutdown.
   if (Clang->getFrontendOpts().DisableFree) {
     llvm::BuryPointer(std::move(Clang));
     return !Success;
   }
 
   return !Success;
 }

llvm/trunk/include/llvm/Support/TimeProfiler.h

+//===- llvm/Support/TimeProfiler.h - Hierarchical Time Profiler -*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_TIME_PROFILER_H
+#define LLVM_SUPPORT_TIME_PROFILER_H
+
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+
+struct TimeTraceProfiler;
+extern TimeTraceProfiler *TimeTraceProfilerInstance;
+
+/// Initialize the time trace profiler.
+/// This sets up the global \p TimeTraceProfilerInstance
+/// variable to be the profiler instance.
+void timeTraceProfilerInitialize();
+
+/// Cleanup the time trace profiler, if it was initialized.
+void timeTraceProfilerCleanup();
+
+/// Is the time trace profiler enabled, i.e. initialized?
+inline bool timeTraceProfilerEnabled() {
+  return TimeTraceProfilerInstance != nullptr;
+}
+
+/// Write profiling data to output file.
+/// Data produced is JSON, in Chrome "Trace Event" format, see
+/// https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview
+void timeTraceProfilerWrite(std::unique_ptr<raw_pwrite_stream> &OS);
+
+/// Manually begin a time section, with the given \p Name and \p Detail.
+/// Profiler copies the string data, so the pointers can be given into
+/// temporaries. Time sections can be hierarchical; every Begin must have a
+/// matching End pair but they can nest.
+void timeTraceProfilerBegin(StringRef Name, StringRef Detail);
+void timeTraceProfilerBegin(StringRef Name,
+                            llvm::function_ref<std::string()> Detail);
+
+/// Manually end the last time section.
+void timeTraceProfilerEnd();
+
+/// The TimeTraceScope is a helper class to call the begin and end functions
+/// of the time trace profiler.  When the object is constructed, it begins
+/// the section; and when it is destroyed, it stops it. If the time profiler
+/// is not initialized, the overhead is a single branch.
+struct TimeTraceScope {
+  TimeTraceScope(StringRef Name, StringRef Detail) {
+    if (TimeTraceProfilerInstance != nullptr)
+      timeTraceProfilerBegin(Name, Detail);
+  }
+  TimeTraceScope(StringRef Name, llvm::function_ref<std::string()> Detail) {
+    if (TimeTraceProfilerInstance != nullptr)
+      timeTraceProfilerBegin(Name, Detail);
+  }
+  ~TimeTraceScope() {
+    if (TimeTraceProfilerInstance != nullptr)
+      timeTraceProfilerEnd();
+  }
+};
+
+} // end namespace llvm
+
+#endif

llvm/trunk/lib/IR/LegacyPassManager.cpp

 #include "llvm/IR/PassTimingInfo.h"
 #include "llvm/Support/Chrono.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/Mutex.h"
+#include "llvm/Support/TimeProfiler.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <unordered_set>
 using namespace llvm;
 using namespace llvm::legacy;
 
 // See PassManagers.h for Pass Manager infrastructure overview.
bool FPPassManager::runOnFunction(Function &F) {
   StringMap<std::pair<unsigned, unsigned>> FunctionToInstrCount;
   bool EmitICRemark = M.shouldEmitInstrCountChangedRemark();
   // Collect the initial size of the module.
   if (EmitICRemark) {
     InstrCount = initSizeRemarkInfo(M, FunctionToInstrCount);
     FunctionSize = F.getInstructionCount();
   }
 
+  bool ProfileTime = llvm::timeTraceProfilerEnabled();
+  if (ProfileTime)
+    llvm::timeTraceProfilerBegin("OptFunction", F.getName());
+
   for (unsigned Index = 0; Index < getNumContainedPasses(); ++Index) {
     FunctionPass *FP = getContainedPass(Index);
     bool LocalChanged = false;
 
     dumpPassInfo(FP, EXECUTION_MSG, ON_FUNCTION_MSG, F.getName());
     dumpRequiredSet(FP);
 
     initializeAnalysisImpl(FP);
for (unsigned Index = 0; Index < getNumContainedPasses(); ++Index) {
     dumpPreservedSet(FP);
     dumpUsedSet(FP);
 
     verifyPreservedAnalysis(FP);
     removeNotPreservedAnalysis(FP);
     recordAvailableAnalysis(FP);
     removeDeadPasses(FP, F.getName(), ON_FUNCTION_MSG);
   }
+
+  if (ProfileTime)
+    llvm::timeTraceProfilerEnd();
+
   return Changed;
 }
 
 bool FPPassManager::runOnModule(Module &M) {
   bool Changed = false;
 
+  llvm::TimeTraceScope TimeScope("OptModule", M.getName());
   for (Function &F : M)
     Changed |= runOnFunction(F);
 
   return Changed;
 }
 
 bool FPPassManager::doInitialization(Module &M) {
   bool Changed = false;

 //===----------------------------------------------------------------------===//
 // MPPassManager implementation
 
 /// Execute all of the passes scheduled for execution by invoking
 /// runOnModule method.  Keep track of whether any of the passes modifies
 /// the module, and if so, return true.
 bool
 MPPassManager::runOnModule(Module &M) {
+  llvm::TimeTraceScope TimeScope("OptModule", M.getName());
+
   bool Changed = false;
 
   // Initialize on-the-fly passes
   for (auto &OnTheFlyManager : OnTheFlyManagers) {
     FunctionPassManagerImpl *FPP = OnTheFlyManager.second;
     Changed |= FPP->doInitialization(M);
   }
 

llvm/trunk/lib/Support/CMakeLists.txt

add_llvm_library(LLVMSupport
   StringPool.cpp
   StringSaver.cpp
   StringRef.cpp
   SymbolRemappingReader.cpp
   SystemUtils.cpp
   TarWriter.cpp
   TargetParser.cpp
   ThreadPool.cpp
+  TimeProfiler.cpp
   Timer.cpp
   ToolOutputFile.cpp
   TrigramIndex.cpp
   Triple.cpp
   Twine.cpp
   Unicode.cpp
   UnicodeCaseFold.cpp
   VersionTuple.cpp

llvm/trunk/lib/Support/TimeProfiler.cpp

+//===-- TimeProfiler.cpp - Hierarchical Time Profiler ---------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file Hierarchical time profiler implementation.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/TimeProfiler.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/FileSystem.h"
+#include <cassert>
+#include <chrono>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+using namespace std::chrono;
+
+namespace llvm {
+
+TimeTraceProfiler *TimeTraceProfilerInstance = nullptr;
+
+static std::string escapeString(StringRef Src) {
+  std::string OS;
+  for (const unsigned char &C : Src) {
+    switch (C) {
+    case '"':
+    case '/':
+    case '\\':
+    case '\b':
+    case '\f':
+    case '\n':
+    case '\r':
+    case '\t':
+      OS += '\\';
+      OS += C;
+      break;
+    default:
+      if (isPrint(C)) {
+        OS += C;
+      }
+    }
+  }
+  return OS;
+}
+
+typedef duration<steady_clock::rep, steady_clock::period> DurationType;
+typedef std::pair<std::string, DurationType> NameAndDuration;
+
+struct Entry {
+  time_point<steady_clock> Start;
+  DurationType Duration;
+  std::string Name;
+  std::string Detail;
+};
+
+struct TimeTraceProfiler {
+  TimeTraceProfiler() {
+    Stack.reserve(8);
+    Entries.reserve(128);
+    StartTime = steady_clock::now();
+  }
+
+  void begin(std::string Name, llvm::function_ref<std::string()> Detail) {
+    Entry E = {steady_clock::now(), {}, Name, Detail()};
+    Stack.push_back(std::move(E));
+  }
+
+  void end() {
+    assert(!Stack.empty() && "Must call begin() first");
+    auto &E = Stack.back();
+    E.Duration = steady_clock::now() - E.Start;
+
+    // Only include sections longer than 500us.
+    if (duration_cast<microseconds>(E.Duration).count() > 500)
+      Entries.emplace_back(E);
+
+    // Track total time taken by each "name", but only the topmost levels of
+    // them; e.g. if there's a template instantiation that instantiates other
+    // templates from within, we only want to add the topmost one. "topmost"
+    // happens to be the ones that don't have any currently open entries above
+    // itself.
+    if (std::find_if(++Stack.rbegin(), Stack.rend(), [&](const Entry &Val) {
+          return Val.Name == E.Name;
+        }) == Stack.rend()) {
+      TotalPerName[E.Name] += E.Duration;
+      CountPerName[E.Name]++;
+    }
+
+    Stack.pop_back();
+  }
+
+  void Write(std::unique_ptr<raw_pwrite_stream> &OS) {
+    assert(Stack.empty() &&
+           "All profiler sections should be ended when calling Write");
+
+    *OS << "{ \"traceEvents\": [\n";
+
+    // Emit all events for the main flame graph.
+    for (const auto &E : Entries) {
+      auto StartUs = duration_cast<microseconds>(E.Start - StartTime).count();
+      auto DurUs = duration_cast<microseconds>(E.Duration).count();
+      *OS << "{ \"pid\":1, \"tid\":0, \"ph\":\"X\", \"ts\":" << StartUs
+          << ", \"dur\":" << DurUs << ", \"name\":\"" << escapeString(E.Name)
+          << "\", \"args\":{ \"detail\":\"" << escapeString(E.Detail)
+          << "\"} },\n";
+    }
+
+    // Emit totals by section name as additional "thread" events, sorted from
+    // longest one.
+    int Tid = 1;
+    std::vector<NameAndDuration> SortedTotals;
+    SortedTotals.reserve(TotalPerName.size());
+    for (const auto &E : TotalPerName) {
+      SortedTotals.push_back(E);
+    }
+    std::sort(SortedTotals.begin(), SortedTotals.end(),
+              [](const NameAndDuration &A, const NameAndDuration &B) {
+                return A.second > B.second;
+              });
+    for (const auto &E : SortedTotals) {
+      auto DurUs = duration_cast<microseconds>(E.second).count();
+      *OS << "{ \"pid\":1, \"tid\":" << Tid << ", \"ph\":\"X\", \"ts\":" << 0
+          << ", \"dur\":" << DurUs << ", \"name\":\"Total "
+          << escapeString(E.first)
+          << "\", \"args\":{ \"count\":" << CountPerName[E.first]
+          << ", \"avg ms\":" << (DurUs / CountPerName[E.first] / 1000)
+          << "} },\n";
+      ++Tid;
+    }
+
+    // Emit metadata event with process name.
+    *OS << "{ \"cat\":\"\", \"pid\":1, \"tid\":0, \"ts\":0, \"ph\":\"M\", "
+           "\"name\":\"process_name\", \"args\":{ \"name\":\"clang\" } }\n";
+    *OS << "] }\n";
+  }
+
+  std::vector<Entry> Stack;
+  std::vector<Entry> Entries;
+  std::unordered_map<std::string, DurationType> TotalPerName;
+  std::unordered_map<std::string, size_t> CountPerName;
+  time_point<steady_clock> StartTime;
+};
+
+void timeTraceProfilerInitialize() {
+  assert(TimeTraceProfilerInstance == nullptr &&
+         "Profiler should not be initialized");
+  TimeTraceProfilerInstance = new TimeTraceProfiler();
+}
+
+void timeTraceProfilerCleanup() {
+  delete TimeTraceProfilerInstance;
+  TimeTraceProfilerInstance = nullptr;
+}
+
+void timeTraceProfilerWrite(std::unique_ptr<raw_pwrite_stream> &OS) {
+  assert(TimeTraceProfilerInstance != nullptr &&
+         "Profiler object can't be null");
+  TimeTraceProfilerInstance->Write(OS);
+}
+
+void timeTraceProfilerBegin(StringRef Name, StringRef Detail) {
+  if (TimeTraceProfilerInstance != nullptr)
+    TimeTraceProfilerInstance->begin(Name, [&]() { return Detail; });
+}
+
+void timeTraceProfilerBegin(StringRef Name,
+                            llvm::function_ref<std::string()> Detail) {
+  if (TimeTraceProfilerInstance != nullptr)
+    TimeTraceProfilerInstance->begin(Name, Detail);
+}
+
+void timeTraceProfilerEnd() {
+  if (TimeTraceProfilerInstance != nullptr)
+    TimeTraceProfilerInstance->end();
+}
+
+} // namespace llvm