This is an archive of the discontinued LLVM Phabricator instance.

Use shell cat command as a workaround if ADB stat cannot lookup a file
ClosedPublic

Authored by ovyalov on Jul 6 2016, 7:01 PM.

Details

Summary

adbd may fail to access a file due security constraints - in such case sync:stat command returns file's mode as 0. If it's the case - use shell cat a workaround then.

Diff Detail

Event Timeline

ovyalov updated this revision to Diff 63011.Jul 6 2016, 7:01 PM
ovyalov retitled this revision from to Use shell cat command as a workaround if ADB stat cannot lookup a file.
ovyalov updated this object.
ovyalov added reviewers: tberghammer, labath.
ovyalov added a subscriber: lldb-commits.
labath accepted this revision.Jul 7 2016, 2:15 AM
labath edited edge metadata.

Looks good.

Please also run clang-format over the patch (also when I don't request it, i tend to forget about that).

source/Plugins/Platform/Android/AdbClient.cpp
449

We should close the file and make check for the error flags on the stream to make sure the write was successful.

This revision is now accepted and ready to land.Jul 7 2016, 2:15 AM
ldrumm added a subscriber: ldrumm.Jul 7 2016, 7:59 AM
ldrumm added inline comments.
source/Plugins/Platform/Android/PlatformAndroid.cpp
244

I have some issues with this approach.

If the filename contains the literal ' character, this command fail with unpredictable results. The solution would seem to be to pass a list of strings a'la POSIX exec i.e.:

- AdbClient::internalShell (const char* command, uint32_t timeout_ms, std::vector<char> &output_buf)
+ AdbClient::internalShell (std::vector<const char*> command, uint32_t timeout_ms, std::vector<char> &output_buf)

but on a little investigation, it does seem that the adb protocol is limited to escaping for the shell command, and does not propagate return status. If I run the following python command:

import subprocess
print(subprocess.check_output(['adb', 'shell', 'cat', "/data/'"]))

The TCP conversation looks like this:

0017host:transport:6D8E7152
OKAY
0012shell:cat /data/\'
OKAY

When I try and cat a file that doesn't exist:

0017host:transport:6D8E7152OKAY0016
shell:cat /nonexistent
OKAY/system/bin/sh: cat: /nonexistent: No such file or directory

So there's seemingly no way to know whether the file exists using adb shell cat $FILE, or whether you're getting the error message as contents.

However, if I use the adb pull command on the same file, the return status is propogated to the calling shell because the stat fails:

0017host:transport:6D8E7152
OKAY
0005sync:
OKAY
STAT..../nonexistent
STAT............

So it seems like the protocol itself does not handle escaping consistently but requires clients to do this on a per adb command basis and there is no way to know the difference between success and failure for the cat file approach.

Unfortunately, I don't actually have a better solution to offer here.

labath added a comment.Jul 7 2016, 8:24 AM

You raised good points here, Luke.

I've thought about the quoting issue, but as you have already noticed there is no way to pass arguments containing quotes to adb correctly. Thinking about it more, what we could do is detect this situation (look for ' in the file name) and abort the whole process even before issuing the "adb shell" command.

As for the return value, this should not be a big problem, as further down the line we will discover that this is not a real ELF file and ignore it, but perhaps we could help here by searching for the file name in the first 1K bytes of the result, and ignoring the file if we find it?

Oleksiy, what do you think?

ovyalov updated this revision to Diff 63173.Jul 7 2016, 6:39 PM
ovyalov edited edge metadata.

Addressed review comments:

  • Check whether shell command output starts with /system/bin/sh: in order to find out whether command failed
  • Fail-fast download if filename contains single-quotes
  • clang-format

You raised good points here, Luke.

I've thought about the quoting issue, but as you have already noticed there is no way to pass arguments containing quotes to adb correctly. Thinking about it more, what we could do is detect this situation (look for ' in the file name) and abort the whole process even before issuing the "adb shell" command.

As for the return value, this should not be a big problem, as further down the line we will discover that this is not a real ELF file and ignore it, but perhaps we could help here by searching for the file name in the first 1K bytes of the result, and ignoring the file if we find it?

Oleksiy, what do you think?

Initially I was thinking that we can pass any shell output to the caller so either module cache or ELF ObjectFile will verify its correctness - but since Platform::GetFile can be used to download an arbitrary file we should try to handle execution errors as much as we can. I added for "/system/bin/sh:" output prefix check - presumably it should cover majority of failures. As an alternative we can run a command like "$user_cmd || echo $rand_id" and then look whether output ends with $rand_id - but it might be an overkill.

labath added a comment.Jul 8 2016, 2:08 AM

Thanks for adding the additional checks. Looks good apart from the clang-format changes. This looks like the llvm style. I am not sure how you've run this but it does not seem to have picked up LLDB's .clang-format file.

Are you using git? You need to set clangFormat.binary setting to a sufficient new version (ideally the one you've just built), so it knows how to handle our AlwaysBreakAfterReturnType: All format. Then git clang-format HEAD^ should just work (assuming you have git-clang-format in your path).

ovyalov updated this revision to Diff 63255.Jul 8 2016, 10:35 AM

Reran with proper clang-format.

Thanks for adding the additional checks. Looks good apart from the clang-format changes. This looks like the llvm style. I am not sure how you've run this but it does not seem to have picked up LLDB's .clang-format file.

Are you using git? You need to set clangFormat.binary setting to a sufficient new version (ideally the one you've just built), so it knows how to handle our AlwaysBreakAfterReturnType: All format. Then git clang-format HEAD^ should just work (assuming you have git-clang-format in your path).

Yep, apparently I used llvm style formatting - by running clang/tools/clang-format/clang-format-diff.py. Thank you for pointers - I reran clang-format on my patch, it looks better now.

ovyalov closed this revision.Jul 8 2016, 10:53 AM

Files:

/lldb/trunk/source/Plugins/Platform/Android/AdbClient.cpp
/lldb/trunk/source/Plugins/Platform/Android/AdbClient.h
/lldb/trunk/source/Plugins/Platform/Android/PlatformAndroid.cpp

Users:

ovyalov (Author)

http://reviews.llvm.org/rL274895