This is an archive of the discontinued LLVM Phabricator instance.

[Driver] -ftime-trace: derive trace file names from -o and -dumpdir
ClosedPublic

Authored by MaskRay on May 10 2023, 11:10 AM.

Details

Summary

Inspired by D133662.
Close https://github.com/llvm/llvm-project/issues/57285

When -ftime-trace is specified and the driver performs both compilation and
linking phases, the trace files are currently placed in the temporary directory
(/tmp by default on *NIX). A more sensible behavior would be to derive the trace
file names from the -o option, similar to how GCC derives auxiliary and dump
file names. Use -dumpdir (D149193) to implement the -gsplit-dwarf like behavior.

The following script demonstrates the time trace filenames.

#!/bin/sh -e
PATH=/tmp/Rel/bin:$PATH                # adapt according to your build directory
mkdir -p d e f
echo 'int main() {}' > d/a.c
echo > d/b.c

a() { rm $1 || exit 1; }

clang -ftime-trace d/a.c d/b.c         # previously /tmp/[ab]-*.json
a a-a.json; a a-b.json
clang -ftime-trace d/a.c d/b.c -o e/x  # previously /tmp/[ab]-*.json
a e/x-a.json; a e/x-b.json
clang -ftime-trace d/a.c d/b.c -o e/x -dumpdir f/
a f/a.json; a f/b.json
clang -ftime-trace=f d/a.c d/b.c -o e/x
a f/a-*.json; a f/b-*.json

clang -c -ftime-trace d/a.c d/b.c
a a.json b.json
clang -c -ftime-trace=f d/a.c d/b.c
a f/a.json f/b.json

clang -c -ftime-trace d/a.c -o e/xa.o
a e/xa.json
clang -c -ftime-trace d/a.c -o e/xa.o -dumpdir f/g
a f/ga.json

The driver checks -ftime-trace and -ftime-trace=, infers the trace file
name, and passes -ftime-trace= to cc1. The -ftime-trace cc1 option is
removed.

With offloading, previously -ftime-trace is passed to all offloading
actions, causing the same trace file to be overwritten by host and
offloading actions. This patch doesn't attempt to support offloading (D133662),
but makes a sensible change (OffloadingPrefix.empty()) to ensure we don't
overwrite the trace file.

Minor behavior differences: the trace file is now a result file, which
will be removed upon an error. -ftime-trace-granularity=0, like
-ftime-trace, can now cause a -Wunused-command-line-argument warning.

Diff Detail

Event Timeline

MaskRay created this revision.May 10 2023, 11:10 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 10 2023, 11:10 AM
MaskRay requested review of this revision.May 10 2023, 11:10 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 10 2023, 11:10 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript

LGTM, but I don't feel like I have the experience to "formally" approve. I left a nice-to have suggestion too, but feel free to ignore, if you feel its out of scope.

clang/lib/Driver/Driver.cpp
5517

One thing D133662 had that this change doesn't is that --ftime-trace was reported as unused when there was no job that invoked clang (e.g. if using the driver to link object files only).

I am not sure its worth the effort, but it would be nice. IIRC in D133662 I did this by having a method (supportsTimeTrace) on Tools to query support.

I can also do this if you feel its out of scope here, and if there's no objection to it.

MaskRay marked an inline comment as done.EditedMay 11 2023, 11:11 AM

LGTM, but I don't feel like I have the experience to "formally" approve. I left a nice-to have suggestion too, but feel free to ignore, if you feel its out of scope.

Thanks. You can certainly give your review opinion as you've studied and discussed the feature a lot! (https://llvm.org/docs/Phabricator.html#:~:text=participated%20might)
I can leave this patch open for a while in case others want to chime in.

If we land a change like this patch, D133662 can be turned to address the offloading side issue.

It doesn't work without or with this patch; there is only one single JSON output file.
I think supporting offloading will need some extensive changes to GetNamedOutputPath, so I avoid doing that in this patch.
I am not an offloading user and don't understand its behavior well enough.

clang/lib/Driver/Driver.cpp
5517

Thanks! This is a worthy change. I'll add a test separately.
I think we can reuse canEmitIR instead of adding supportsTimeTrace.

Another frontend flang-new defines canEmitIR to true as well. It's a work-in-progress and can support -ftime-trace later.

MaskRay updated this revision to Diff 521379.May 11 2023, 11:13 AM
MaskRay marked an inline comment as done.
MaskRay edited the summary of this revision. (Show Details)

Add a warning test. Report -Wunused-command-line-argument warning for -ftime-trace-granularity=0 if unused as well.

Does GCC have the same -ftime-trace= option? It seems like it doesn't, as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92396 is still open?

If so, I am happy with unifying more of the dump/aux handling, and I imagine when/if GCC adds the option it will behave similarly.

clang/include/clang/Driver/Compilation.h
279

If this is overriding previous paths should it be called setTimeTraceFile?

clang/lib/Driver/Driver.cpp
5253

Why += instead of append? Do we just know the value of dumpdir is terminated with the path separator?

MaskRay updated this revision to Diff 521427.May 11 2023, 1:26 PM
MaskRay marked an inline comment as done.
MaskRay edited the summary of this revision. (Show Details)

add an assert to addTimeTraceFile.
improve a -dumpdir test.

Does GCC have the same -ftime-trace= option? It seems like it doesn't, as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92396 is still open?

You found it!

I am trying to collect other auxiliary output file features in Clang and add them to https://maskray.me/blog/2023-04-25-compiler-output-files

If so, I am happy with unifying more of the dump/aux handling, and I imagine when/if GCC adds the option it will behave similarly.

Thanks.

clang/include/clang/Driver/Compilation.h
279

The naming is to match add*File above.
Do you want an assert that the entry hasn't been inserted before?

clang/lib/Driver/Driver.cpp
5253

-dumpdir is a bit misnomer that it may or may not end with a path separator.

clang -c -ftime-trace d/a.c -o e/xa.o -dumpdir f/ is intended to create fa.json

I updated a test and the description to give an example.

scott.linder added inline comments.May 11 2023, 2:05 PM
clang/include/clang/Driver/Compilation.h
279

If you think it is useful; otherwise consistency with the other options seems like a good enough reason

clang/lib/Driver/Driver.cpp
5253

Thanks, I just forgot this point since the previous discussions! LGTM

Maetveis accepted this revision.May 11 2023, 3:23 PM

I don't see anything in here that would block later extension for supporting offloading, so LGTM from my POV.

This revision is now accepted and ready to land.May 11 2023, 3:23 PM
MaskRay marked 3 inline comments as done.May 12 2023, 10:37 AM
This revision was landed with ongoing or failed builds.May 12 2023, 10:46 AM
This revision was automatically updated to reflect the committed changes.
dyung added a subscriber: dyung.May 12 2023, 11:49 AM
dyung added inline comments.
clang/test/Driver/ftime-trace.cpp
54

This seems to be failing on Windows due to path separator issues:
https://lab.llvm.org/buildbot/#/builders/216/builds/21115/steps/7/logs/FAIL__Clang__ftime-trace_cpp

... "-ftime-trace=e\\a-b9537d.json" ...
MaskRay added inline comments.May 12 2023, 12:57 PM
clang/test/Driver/ftime-trace.cpp
54
thakis added a subscriber: thakis.May 13 2023, 11:40 AM

This breaks tests on windows: http://45.33.8.238/win/78516/step_7.txt

Please take a look and revert for now if it takes a while to fix.

Apologies for the noise, looks like my win bot hasn't built in a while. Looks like this is already fixed.

Hi, this fails on AIX. It looks like it still produces the /tmp/lit-tmp-*/[ab]-*.json format. Can you take a look please?

https://lab.llvm.org/buildbot/#/builders/214/builds/7429/steps/6/logs/FAIL__Clang__ftime-trace_cpp

Hi, this fails on AIX. It looks like it still produces the /tmp/lit-tmp-*/[ab]-*.json format. Can you take a look please?

https://lab.llvm.org/buildbot/#/builders/214/builds/7429/steps/6/logs/FAIL__Clang__ftime-trace_cpp

The test should have been fixed by 2f999327534f7cc660d2747ce294f50184dc1f97 .
This is a preexisting problem with clang -c -ftime-trace -fno-integrated-as -O1 a.c -o e/a.o: we get /tmp/a-*.json w/o or with the patch.