[buildbot, CUDA] Adjust GPU list for the tests on cuda-build-test-01

[tra]

Summary

Recent drivers changed the way GPUs are enumerated, so we've been running
the tests on the wrong GPU which took ~45minutes.

[jlebar]

You don't want to pass in the GPU-deadbeef form from nvidia-smi -L so we don't have this problem?

[tra]

I'm not confident that the GUIDS remain stable, either. I have no data to tell one way or another, though.
CUDA currently allows specifying enumeration order by PCI_ID or by 'fastest'.
If it becomes a problem I'll just force enumeration by PCI_ID, which should be somewhat more stable (though it may change on BIOS update or if I add/remove other PCIe devices).

Another option would be to not include PTX in the binaries, so we'll know right away if we attempt to run the tests on the wrong GPU.

[jlebar]

I'm not confident that the GUIDS remain stable, either. I have no data to tell one way or another, though.

That's a good point, but presumably if the GUIDs change, they're not going to *permute* and point to a different GPU. That is, the failure mode is noisy? Dunno if the same is true for pcid, but it's certainly not true for the integer identifiers.

[tra]

Use GUIDs to identify GPUs.

[tra]

OK. Updated the patch to use GUIDs.

[gkistanova]

LGTM

buildbot/osuosl/master/config/builders.py

return [
                      stage1_config='Release',
                      extra_cmake_args=[
                          '-DLLVM_ENABLE_ASSERTIONS=ON',
                          "-DCMAKE_C_COMPILER:FILEPATH=/usr/bin/clang",
                          "-DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/clang++"
                      ],
                      externals="/home/botanist/bots/externals",
                      gpu_arch_list=["sm_35", "sm_61"],
-                     gpu_devices=[2, 0], # K40c(sm_35), GTX1070(sm_61)
+                     gpu_devices=["GPU-af66efa4",  # K40c(sm_35),
+                                  "GPU-44fe2444"   # GTX1070(sm_61)
+                     ],
                      extra_ts_cmake_args=[],
                      enable_thrust_tests=False,
          ),
          'category' : 'clang'},
 
         # lldb builders
         {'name': "lldb-x86_64-darwin-13.4",
          'slavenames': ["lldb-x86_64-darwin-13.4"],