This is an archive of the discontinued LLVM Phabricator instance.

[lit] Avoid calling realpath() for every printed message
ClosedPublic

Authored by arichardson on Oct 10 2020, 6:44 AM.

Details

Summary

I did some profiling of lit while trying to optimize the libc++ test
startup for remote hosts and it turns out that there is a realpath() call
for every message printed and this shows up in the profile.
The inspect.getframeinfo() function calls realpath() internally and
moreover we don't need most of the other information returned from it.
This patch uses inspect.getsourcefile() and os.path.normpath to remove
../ from the path instead. Not resolving symlinks reduces the startup time
for running a single test with lit by about 50ms for me.

Diff Detail

Event Timeline

arichardson created this revision.Oct 10 2020, 6:44 AM
arichardson requested review of this revision.Oct 10 2020, 6:44 AM
yln accepted this revision.Oct 12 2020, 10:05 AM

Just making sure I understand:
This is an implementation detail only and the message format stays the same (and it works on all platforms).
The only difference in behavior is that we don't resolve symlinks anymore; and we don't care about that.

LGTM, if the above is true.

Thanks!

This revision is now accepted and ready to land.Oct 12 2020, 10:05 AM
ldionne requested changes to this revision.Oct 13 2020, 7:11 AM
ldionne added inline comments.
llvm/utils/lit/lit/LitConfig.py
168–171

I don't think you need to call normpath() after calling abspath(). From the Python docs:

os.path.abspath(path)
Return a normalized absolutized version of the pathname path. On most platforms, this is equivalent to calling the function normpath() as follows: normpath(join(os.getcwd(), path)).

This revision now requires changes to proceed.Oct 13 2020, 7:11 AM
arichardson added inline comments.Oct 13 2020, 7:14 AM
llvm/utils/lit/lit/LitConfig.py
168–171

Thanks I didn't read the docs and just assumed it would behave the same as pathlib Path.absolute():

>>> os.chdir("/usr/bin"); (os.getcwd(), os.path.abspath(".."), Path("..").absolute())
('/usr/bin', '/usr', PosixPath('/usr/bin/..'))
yln added inline comments.Oct 13 2020, 10:55 AM
llvm/utils/lit/lit/LitConfig.py
168–171

Good catch, thanks Louis!

drop unncessary normpath

ldionne accepted this revision.Oct 19 2020, 7:56 AM
This revision is now accepted and ready to land.Oct 19 2020, 7:56 AM