Page MenuHomePhabricator

[PGO] Add support to enable testing multiple supported targets
ClosedPublic

Authored by davidxl on Feb 17 2016, 5:30 PM.

Diff Detail

Repository
rL LLVM

Event Timeline

davidxl updated this revision to Diff 48265.Feb 17 2016, 5:30 PM
davidxl retitled this revision from to [PGO] Add support to enable testing multiple supported targets .
davidxl updated this object.
davidxl added reviewers: vsk, samsonov.
davidxl added a subscriber: llvm-commits.
samsonov accepted this revision.Feb 18 2016, 11:01 AM
samsonov edited edge metadata.

LGTM

This revision is now accepted and ready to land.Feb 18 2016, 11:01 AM
This revision was automatically updated to reflect the committed changes.
vsk edited edge metadata.Feb 22 2016, 11:37 AM

Hi David,

This commit enables tests on 64-bit Darwin which have apparently been failing. I now see:

******************** TEST 'Profile-i386 :: instrprof-basic.c' FAILED ********************
Script:
--
/Users/vk/Desktop/llvm/Debug+Asserts/./bin/clang -arch i386  -fprofile-instr-generate -o /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp -O3 /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-basic.c
env LLVM_PROFILE_FILE=/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp.profraw  /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp
llvm-profdata merge -o /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp.profdata /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp.profraw
/Users/vk/Desktop/llvm/Debug+Asserts/./bin/clang -arch i386  -fprofile-instr-use=/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp.profdata -o - -S -emit-llvm /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-basic.c | FileCheck /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-basic.c
--
Exit Code: 139

Command Output (stderr):
--
/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.script: line 4: 20417 Segmentation fault: 11  env LLVM_PROFILE_FILE=/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp.profraw /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-basic.c.tmp

The segmentation fault is in __llvm_profile_gather_value_data(). Accessing I->Values dereferences 0x1:

(lldb) p *I
(__llvm_profile_data) $11 = {
  NameRef = 10864
  FuncHash = 48378511633216
  CounterPtr = 0x00000000
  FunctionPointer = 0x0000df1e
  Values = 0x00000001
  NumCounters = 0
  NumValueSites = ([0] = 13723)
}

There's a second issue with corrupt raw profiles:

********************
FAIL: Profile-i386 :: instrprof-without-libc.c (39 of 96)
******************** TEST 'Profile-i386 :: instrprof-without-libc.c' FAILED ********************
Script:
--
/Users/vk/Desktop/llvm/Debug+Asserts/./bin/clang -arch i386  -fprofile-instr-generate -DCHECK_SYMBOLS -O3 -o /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.symbols /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-without-libc.c
llvm-nm /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.symbols | FileCheck /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-without-libc.c --check-prefix=CHECK-SYMBOLS
/Users/vk/Desktop/llvm/Debug+Asserts/./bin/clang -arch i386  -fprofile-instr-generate -O3 -o /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-without-libc.c
/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.profraw
llvm-profdata merge -o /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.profdata /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.profraw
/Users/vk/Desktop/llvm/Debug+Asserts/./bin/clang -arch i386  -fprofile-instr-use=/Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.profdata -o - -S -emit-llvm /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-without-libc.c | FileCheck /Users/vk/Desktop/llvm/projects/compiler-rt/test/profile/instrprof-without-libc.c
--
Exit Code: 1

Command Output (stderr):
--
error: /Users/vk/Desktop/llvm/build/projects/compiler-rt/test/profile/Profile-i386/Output/instrprof-without-libc.c.tmp.profraw: Invalid instrumentation profile data (file header is corrupt)

This happens because ProfileSize = 34359738544 (pretty close to 32 GB, which seems wrong):

    frame #0: 0x00000001001fa851 llvm-profdata`llvm::RawInstrProfReader<unsigned int>::readHeader(this=0x0000000100b00000, Header=0x0000000100a00110) + 801 at InstrProfReader.cpp:337
   334 	
   335 	  auto *Start = reinterpret_cast<const char *>(&Header);
   336 	  if (Start + ProfileSize > DataBuffer->getBufferEnd())
-> 337 	    return error(instrprof_error::bad_header);

I don't know if this last issue is affected by this commit, but I'm not sure why these tests are unsupported (they don't seem Linux-specific):

UNSUPPORTED: Profile-x86_64 :: Linux/coverage_ctors.cpp (1 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/coverage_dtor.cpp (2 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/instrprof-basic.c (3 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/coverage_test.cpp (4 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/coverage_shared.test (5 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/instrprof-comdat.test (6 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/instrprof-dlopen.test (7 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/instrprof-dynamic-one-shared.test (8 of 32)
UNSUPPORTED: Profile-x86_64 :: Linux/instrprof-dynamic-two-shared.test (9 of 32)

For reference, I configure llvm with: -DCMAKE_CROSSCOMPILING=False -DLLVM_TARGET_ARCH="X86" -DLLVM_TARGETS_TO_BUILD="X86;AArch64;ARM" -DCMAKE_BUILD_TYPE:STRING=Debug -DLLVM_ENABLE_ASSERTIONS=True -DLLVM_INCLUDE_TESTS=True -DLLVM_ENABLE_WARNINGS=False -G 'Ninja' ../.

AFAICT (checking -###), the compiler is grabbing the correct copy of compiler-rt and it has the right slice:

$ ninja check-profile [...]
$ file "/Users/vk/Desktop/llvm/build/bin/../lib/clang/3.9.0/lib/darwin/libclang_rt.profile_osx.a"
/Users/vk/Desktop/llvm/build/bin/../lib/clang/3.9.0/lib/darwin/libclang_rt.profile_osx.a: Mach-O universal binary with 3 architectures: [i386: current ar archive random library] [x86_64: current ar archive random library] [x86_64h: current ar archive random library]
/Users/vk/Desktop/llvm/build/bin/../lib/clang/3.9.0/lib/darwin/libclang_rt.profile_osx.a (for architecture i386):	current ar archive random library
/Users/vk/Desktop/llvm/build/bin/../lib/clang/3.9.0/lib/darwin/libclang_rt.profile_osx.a (for architecture x86_64):	current ar archive random library
/Users/vk/Desktop/llvm/build/bin/../lib/clang/3.9.0/lib/darwin/libclang_rt.profile_osx.a (for architecture x86_64h):	current ar archive random library

I'll try to get to the bottom of this, but any help is appreciated.

Is the root cause of the issue understood now?

thanks,

David

Alexey, This seems to have regressed fairly recently. I just noticed that
check-profile only tests X86_64 target

This is because PROFILE_SUPPORTED_ARCH is not properly defined by
filter_available_targets(PROFILE_SUPPORTED_ARCH
${ALL_PROFILE_SUPPORTED_ARCH})

UBSAN_SUPPORTED_ARCH seems to have the same problem. When I do ninja
check-pubsan, only X86_64 target is tested.

Are you aware of any recent changes that are related?

thanks,

David