This is an archive of the discontinued LLVM Phabricator instance.

Handle "." in target.source-map in PathMapListing::FindFiles
ClosedPublic

Authored by jingham on Jan 31 2019, 3:55 PM.

Details

Summary

When lldb switched to normalizing paths in FileSpec's, the source-map code failed to handle the case where the initial path was relative (usually "."). Greg fixed the PathMapListing::RemapPath code in r327600 and added a test for breakpoint setting with remapped paths. But he didn't do the similar fix to PathMapListing::FindFiles. That meant that though we could set breakpoints in files specified this way, we would fail to find and display the actual source contents from the remapped location.

This patch adds the handling of "." in the source slot for that function, and adds a test for it. I also changed the input file so that it came from a build where the source file was in a subdirectory of the comp dir. Sometimes we were succeeding by accident in the simpler setup.

I also fixed a bug in the test case where we were calling getBuildArtifact with a complete path to the test directory, which just returned a path in the test directory. Fixing that meant the output file was now correctly going in the build directory and we didn't have to delete the file anymore - we no longer delete test files from the build directory.

Diff Detail

Repository
rL LLVM

Event Timeline

jingham created this revision.Jan 31 2019, 3:55 PM
Herald added a project: Restricted Project. · View Herald Transcript
labath added inline comments.Feb 1 2019, 4:44 AM
packages/Python/lldbsuite/test/functionalities/source-map/TestTargetSourceMap.py
32–34 ↗(On Diff #184640)

Is this intentional?

source/Target/PathMappingList.cpp
207 ↗(On Diff #184640)

!orig_path.empty()?

232 ↗(On Diff #184640)

The size() is actually unneeded as consume_front is trivially true for an empty StringRef.

ted added a subscriber: ted.EditedFeb 1 2019, 7:40 AM

FYI, @jingham , this is also an issue with DYLD plugins, if the file in the link map starts with ".". "image search-paths add . /path/to/my/libraries" fails in this case, because "./library.so" gets changed to "library.so".

jingham updated this revision to Diff 184784.Feb 1 2019, 10:10 AM

Fixed some inadvertently commented out test code (test still passes...)
Applied Pavel's suggestions (thanks!)

jingham marked 5 inline comments as done.Feb 1 2019, 10:12 AM
jingham added inline comments.
packages/Python/lldbsuite/test/functionalities/source-map/TestTargetSourceMap.py
32–34 ↗(On Diff #184640)

No. The test still passes with these uncommented!

source/Target/PathMappingList.cpp
207 ↗(On Diff #184640)

Doh!

jingham marked 2 inline comments as done.Feb 1 2019, 10:16 AM
In D57552#1380782, @ted wrote:

FYI, @jingham , this is also an issue with DYLD plugins, if the file in the link map starts with ".". "image search-paths add . /path/to/my/libraries" fails in this case, because "./library.so" gets changed to "library.so".

This should fix that issue as well, since image search-paths are also PathMappingLists. I don't see any tests for this, however. Do you have something handy you could turn into a test?

zturner added inline comments.Feb 1 2019, 10:25 AM
source/Target/PathMappingList.cpp
204 ↗(On Diff #184784)

Can we put some early returns in here?

new_spec.Clear();

if (m_pairs.empty())
  return;
207 ↗(On Diff #184784)
if (orig_path.empty())
  return;
208 ↗(On Diff #184784)

I think we can move this declaration inside of the for loop

212 ↗(On Diff #184784)

How about

for (const auto &pair : m_pairs) {
}
232 ↗(On Diff #184784)

After moving inside of the for loop, you can just write if (orig_ref.consume_front(prefix_ref))

ted added a comment.Feb 1 2019, 10:39 AM
In D57552#1380782, @ted wrote:

FYI, @jingham , this is also an issue with DYLD plugins, if the file in the link map starts with ".". "image search-paths add . /path/to/my/libraries" fails in this case, because "./library.so" gets changed to "library.so".

This should fix that issue as well, since image search-paths are also PathMappingLists. I don't see any tests for this, however. Do you have something handy you could turn into a test?

I don't think this will fix it, because the problem happens in the initial FileSpec, before we get to search-paths.

From source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp, DYLDRendezvous::ReadSOEntryFromMemory():

std::string file_path = ReadStringFromMemory(entry.path_addr);
entry.file_spec.SetFile(file_path, false, FileSpec::Style::native);

FileSpec::SetFile normalizes the path, which will remove a leading "./", so "image search-paths add . /path/to/my/lib", which would work with "./mylib.so", won't work with "mylib.so".

I think I can write a test that uses LD_LIBRARY_PATH and a subdirectory so the target can load "./mylib.so", but LLDB won't be able to see it without a search-path.

jingham updated this revision to Diff 184790.Feb 1 2019, 10:43 AM

Implemented Zachary's suggestions.
Also added the source file sub-directory so the test will pass on other people's machines...

jingham marked 5 inline comments as done.Feb 1 2019, 10:44 AM

Sure, that's clearer.

In D57552#1381094, @ted wrote:
In D57552#1380782, @ted wrote:

FYI, @jingham , this is also an issue with DYLD plugins, if the file in the link map starts with ".". "image search-paths add . /path/to/my/libraries" fails in this case, because "./library.so" gets changed to "library.so".

This should fix that issue as well, since image search-paths are also PathMappingLists. I don't see any tests for this, however. Do you have something handy you could turn into a test?

I don't think this will fix it, because the problem happens in the initial FileSpec, before we get to search-paths.

From source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp, DYLDRendezvous::ReadSOEntryFromMemory():

std::string file_path = ReadStringFromMemory(entry.path_addr);
entry.file_spec.SetFile(file_path, false, FileSpec::Style::native);

FileSpec::SetFile normalizes the path, which will remove a leading "./", so "image search-paths add . /path/to/my/lib", which would work with "./mylib.so", won't work with "mylib.so".

I think I can write a test that uses LD_LIBRARY_PATH and a subdirectory so the target can load "./mylib.so", but LLDB won't be able to see it without a search-path.

That sounds like it will have to be fixed somewhere else, and not part of this patch then. Can you file a bug to get that fixed?

ted added a comment.Feb 1 2019, 11:36 AM

That sounds like it will have to be fixed somewhere else, and not part of this patch then. Can you file a bug to get that fixed?

Yeah, it's on my todo list. Which keeps growing!

clayborg added inline comments.Feb 1 2019, 1:52 PM
source/Target/PathMappingList.cpp
223 ↗(On Diff #184790)

We are we finding a "." in any path now? I thought we normalized those all out? Can a PathMappingList contain non normalized paths? if so, can't we just normalize them as they go into the list?

jingham marked an inline comment as done.Feb 4 2019, 10:15 AM
jingham added inline comments.
source/Target/PathMappingList.cpp
223 ↗(On Diff #184790)

That's the incoming path from the path maps. You can't normalize just a "." from the path mappings or it would become "" and we don't really know what that means. These are currently left as ".". I think that's right.

labath accepted this revision.Feb 5 2019, 1:56 AM

If Greg is happy, then LGTM from me too.

source/Target/PathMappingList.cpp
223 ↗(On Diff #184790)

FileSpec::IsValid() considers "" to be an invalid path. Most utilities/APIs do the same. The llvm api are a bit conflicted here though as sometimes they do consider "" to be invalid, but otoh they will happilly normalize "." to "".

Bottom line: I also think that the current FileSpec behavior is correct.

This revision is now accepted and ready to land.Feb 5 2019, 1:56 AM
clayborg accepted this revision.Feb 5 2019, 1:04 PM

Thanks for the answers. LGTM

This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptFeb 5 2019, 3:48 PM