[OpenMP][Tool] Introducing the `llvm-omp-device-info` tool

[josemonsalve2]

Summary

This patch introduces the llvm-omp-device-info tool, which uses the
omptarget library and interface to query the device info from all the
available devices as seen by OpenMP. This is inspired by PGI's pgaccelinfo

Since omptarget usually requires a description structure with executable
kernels, I split the initialization of the RTLs and Devices to be able to
initialize all possible devices and query each of them.

This revision relies on the patch that introduces the print device info.

A limitation is that the order in which the devices are initialized, and the
corresponding device ID is not necesarily the one seen by OpenMP.

The changes are as follows:

1. Separate the RTL initialization that was performed in RegisterLib to its own initRTLonce function
2. Create an initAllRTLs method that initializes all available RTLs at runtime
3. Created the llvm-deviceinfo.cpp tool that uses omptarget to query each device and prints its information.

Example Output:

Device (0):
    print_device_info not implemented

Device (1):
    print_device_info not implemented

Device (2):
    print_device_info not implemented

Device (3):
    print_device_info not implemented

Device (4):
    CUDA Driver Version:                11000
    CUDA Device Number:                 0
    Device Name:                        Quadro P1000
    Global Memory Size:                 4236312576 bytes
    Number of Multiprocessors:          5
    Concurrent Copy and Execution:      Yes
    Total Constant Memory:              65536 bytes
    Max Shared Memory per Block:        49152 bytes
    Registers per Block:                65536
    Warp Size:                          32 Threads
    Maximum Threads per Block:          1024
    Maximum Block Dimensions:           1024, 1024, 64
    Maximum Grid Dimensions:            2147483647 x 65535 x 65535
    Maximum Memory Pitch:               2147483647 bytes
    Texture Alignment:                  512 bytes
    Clock Rate:                         1480500 kHz
    Execution Timeout:                  Yes
    Integrated Device:                  No
    Can Map Host Memory:                Yes
    Compute Mode:                       DEFAULT
    Concurrent Kernels:                 Yes
    ECC Enabled:                        No
    Memory Clock Rate:                  2505000 kHz
    Memory Bus Width:                   128 bits
    L2 Cache Size:                      1048576 bytes
    Max Threads Per SMP:                2048
    Async Engines:                      Yes (2)
    Unified Addressing:                 Yes
    Managed Memory:                     Yes
    Concurrent Managed Memory:          Yes
    Preemption Supported:               Yes
    Cooperative Launch:                 Yes
    Multi-Device Boars:                 No
    Compute Capabilities:               61

[jdoerfert]

Can we rename it to llvm-omp-deviceinfo everywhere please.

This looks reasonable to me, some nits. @tianshilei1992 @JonChesterfield comments?

openmp/libomptarget/tools/CMakeLists.txt

2

llvm license header, also other files.

openmp/libomptarget/tools/deviceinfo/llvm-deviceinfo.cpp

15 ↗

(On Diff #361471)

[tianshilei1992]

In D106752#2903129, @jdoerfert wrote:

Can we rename it to llvm-omp-deviceinfo everywhere please.

Since we already have -, what about llvm-omp-device-info? deviceinfo looks really odd.

This looks reasonable to me, some nits. @tianshilei1992 @JonChesterfield comments?

Generally LGTM.

openmp/libomptarget/src/rtl.cpp

320–324

332

openmp/libomptarget/tools/deviceinfo/llvm-deviceinfo.cpp

15 ↗

(On Diff #361471)

It's a C++ file. No reason to use stdio.h instead of C++ header.

[josemonsalve2]

I thought about the -omp- in the name too. But I remembered that the ACC folks wanted to use the same runtime. I like both llvm-omp-device-info and llvm-omp-deviceinfo. Or we could drop the llvm- as they do in mlir- tools

[jdoerfert]

In D106752#2903152, @josemonsalve2 wrote:

I thought about the -omp- in the name too. But I remembered that the ACC folks wanted to use the same runtime. I like both llvm-omp-device-info and llvm-omp-deviceinfo. Or we could drop the llvm- as they do in mlir- tools

llvm-omp-device-info

[jdoerfert]

openmp/libomptarget/tools/deviceinfo/llvm-deviceinfo.cpp

15 ↗

(On Diff #361471)

cstdio then. Why include iostream which will eventually include cstdio which will include stdio.h just for printf. There is a lot that can go wrong with streams if the setup is not proper and there is no reason I can think of to go down that road for printf. C standard libary headers are compatible with C++.

[JonChesterfield]

openmp/libomptarget/tools/deviceinfo/llvm-deviceinfo.cpp

15 ↗

(On Diff #361471)

+1 for avoiding iostream. Don't have strong preferences for stdio.h vs cstdio.

[josemonsalve2]

Changing name and adding cstdio instead of iostream. Adding license headers. Other minor changes

[tianshilei1992]

LGTM with some nits.

openmp/libomptarget/plugins/cuda/CMakeLists.txt

66 ↗

(On Diff #361878)

what is this for? Do we have to do similar thing for AMDGCN? If no, why is it here?

openmp/libomptarget/tools/CMakeLists.txt

14

^^^ here

openmp/libomptarget/tools/deviceinfo/CMakeLists.txt

17–30

openmp/libomptarget/tools/deviceinfo/llvm-omp-device-info.cpp

25–27

[jdoerfert]

adjust commit message in phab wrt. tool name please.

[josemonsalve2]

openmp/libomptarget/plugins/cuda/CMakeLists.txt

66 ↗

(On Diff #361878)

I originally had it because I was getting some linking errors, but I just tried it and it is not really necessary, apparently.

[josemonsalve2]

Fixing @tianshilei1992 comments.

[josemonsalve2]

Rebase to main

[josemonsalve2]

Removing branch dependency

openmp/libomptarget/CMakeLists.txt

 set(LIBOMPTARGET_OPENMP_HOST_RTL_FOLDER "${LIBOMP_LIBRARY_DIR}" CACHE STRING
   "Path to folder containing libomp.so, and libLLVMSupport.so with profiling enabled")
 
 
 # Build offloading plugins and device RTLs if they are available.
 add_subdirectory(plugins)
 add_subdirectory(deviceRTLs)
 add_subdirectory(DeviceRTL)
+add_subdirectory(tools)
 
 # Add tests.
 add_subdirectory(test)

openmp/libomptarget/include/omptarget.h

 void *llvm_omp_target_alloc_shared(size_t size, int device_num);
 
 /// add the clauses of the requires directives in a given file
 void __tgt_register_requires(int64_t flags);
 
 /// adds a target shared library to the target execution image
 void __tgt_register_lib(__tgt_bin_desc *desc);
 
+/// Initialize all RTLs at once
+void __tgt_init_all_rtls();
+
 /// removes a target shared library from the target execution image
 void __tgt_unregister_lib(__tgt_bin_desc *desc);
 
 // creates the host to target data mapping, stores it in the
 // libomptarget.so internal structure (an entry in a stack of data maps) and
 // passes the data to the device;
 void __tgt_target_data_begin(int64_t device_id, int32_t arg_num,
                              void **args_base, void **args, int64_t *arg_sizes,

openmp/libomptarget/plugins/cuda/src/rtl.cpp

 //===----------------------------------------------------------------------===//
 
 #include <cassert>
 #include <cstddef>
 #include <cuda.h>
 #include <list>
 #include <memory>
 #include <mutex>
+#include <string.h>
 #include <string>
 #include <unordered_map>
 #include <vector>
+#include <string.h>
 
 #include "Debug.h"
 #include "omptargetplugin.h"
 
 #define TARGET_NAME CUDA
 #define DEBUG_PREFIX "Target " GETNAME(TARGET_NAME) " RTL"
 
 #include "MemoryManager.h"

openmp/libomptarget/src/exports

 VERS1.0 {
   global:
     __tgt_register_requires;
     __tgt_register_lib;
     __tgt_unregister_lib;
+    __tgt_init_all_rtls;
     __tgt_target_data_begin;
     __tgt_target_data_end;
     __tgt_target_data_update;
     __tgt_target;
     __tgt_target_teams;
     __tgt_target_data_begin_nowait;
     __tgt_target_data_end_nowait;
     __tgt_target_data_update_nowait;

openmp/libomptarget/src/interface.cpp

if (RTL.register_lib) {
         DP("Could not register library with %s", RTL.RTLName.c_str());
       }
     }
   }
   PM->RTLs.RegisterLib(desc);
 }
 
 ////////////////////////////////////////////////////////////////////////////////
+/// Initialize all available devices without registering any image
+EXTERN void __tgt_init_all_rtls() { PM->RTLs.initAllRTLs(); }
+
+////////////////////////////////////////////////////////////////////////////////
 /// unloads a target shared library
 EXTERN void __tgt_unregister_lib(__tgt_bin_desc *desc) {
   TIMESCOPE();
   PM->RTLs.UnregisterLib(desc);
   for (auto &RTL : PM->RTLs.UsedRTLs) {
     if (RTL->unregister_lib) {
       if ((*RTL->unregister_lib)(desc) != OFFLOAD_SUCCESS) {
         DP("Could not register library with %s", RTL->RTLName.c_str());

openmp/libomptarget/src/rtl.h

struct RTLsTy {
 
   int64_t RequiresFlags = OMP_REQ_UNDEFINED;
 
   explicit RTLsTy() = default;
 
   // Register the clauses of the requires directive.
   void RegisterRequires(int64_t flags);
 
+  // Initialize RTL if it has not been initialized
+  void initRTLonce(RTLInfoTy &RTL);
+
+  // Initialize all RTLs
+  void initAllRTLs();
+
   // Register a shared library with all (compatible) RTLs.
   void RegisterLib(__tgt_bin_desc *desc);
 
   // Unregister a shared library from all RTLs.
   void UnregisterLib(__tgt_bin_desc *desc);
 
   // Mutex-like object to guarantee thread-safety and unique initialization
   // (i.e. the library attempts to load the RTLs (plugins) only once).

openmp/libomptarget/src/rtl.cpp

void RTLsTy::RegisterRequires(int64_t flags) {
   }
 
   // TODO: insert any other missing checks
 
   DP("New requires flags %" PRId64 " compatible with existing %" PRId64 "!\n",
      flags, RequiresFlags);
 }
 
-void RTLsTy::RegisterLib(__tgt_bin_desc *desc) {
-  PM->RTLsMtx.lock();
-  // Register the images with the RTLs that understand them, if any.
-  for (int32_t i = 0; i < desc->NumDeviceImages; ++i) {
-    // Obtain the image.
-    __tgt_device_image *img = &desc->DeviceImages[i];
-
-    RTLInfoTy *FoundRTL = NULL;
-
-    // Scan the RTLs that have associated images until we find one that supports
-    // the current image.
-    for (auto &R : AllRTLs) {
-      if (!R.is_valid_binary(img)) {
-        DP("Image " DPxMOD " is NOT compatible with RTL %s!\n",
-           DPxPTR(img->ImageStart), R.RTLName.c_str());
-        continue;
-      }
-
-      DP("Image " DPxMOD " is compatible with RTL %s!\n",
-         DPxPTR(img->ImageStart), R.RTLName.c_str());
-
-      // If this RTL is not already in use, initialize it.
-      if (!R.isUsed) {
-        // Initialize the device information for the RTL we are about to use.
-        DeviceTy device(&R);
-        size_t Start = PM->Devices.size();
-        PM->Devices.resize(Start + R.NumberOfDevices, device);
-        for (int32_t device_id = 0; device_id < R.NumberOfDevices;
-             device_id++) {
-          // global device ID
-          PM->Devices[Start + device_id].DeviceID = Start + device_id;
-          // RTL local device ID
-          PM->Devices[Start + device_id].RTLDeviceID = device_id;
-        }
+void RTLsTy::initRTLonce(RTLInfoTy &R) {
+  if (!R.isUsed && R.NumberOfDevices != 0) {
+    for (int32_t device_id = 0; device_id < R.NumberOfDevices; device_id++) {
 
-        // Initialize the index of this RTL and save it in the used RTLs.
-        R.Idx = (UsedRTLs.empty())
-                    ? 0
-                    : UsedRTLs.back()->Idx + UsedRTLs.back()->NumberOfDevices;
-        assert((size_t)R.Idx == Start &&
-               "RTL index should equal the number of devices used so far.");
-        R.isUsed = true;
-        UsedRTLs.push_back(&R);
 
-        DP("RTL " DPxMOD " has index %d!\n", DPxPTR(R.LibraryHandler), R.Idx);
-      }
+}
+
+void RTLsTy::initAllRTLs() {
+  for (auto &R : AllRTLs)
+    initRTLonce(R);
+}
+
+void RTLsTy::RegisterLib(__tgt_bin_desc *desc) {
+  PM->RTLsMtx.lock();
+  // Register the images with the RTLs that understand them, if any.
+  for (int32_t i = 0; i < desc->NumDeviceImages; ++i) {
+    // Obtain the image.
+    __tgt_device_image *img = &desc->DeviceImages[i];
+
+    RTLInfoTy *FoundRTL = nullptr;
+
+    // Scan the RTLs that have associated images until we find one that supports
+    // the current image.
+    for (auto &R : AllRTLs) {
+      if (!R.is_valid_binary(img)) {
+        DP("Image " DPxMOD " is NOT compatible with RTL %s!\n",
+           DPxPTR(img->ImageStart), R.RTLName.c_str());
+        continue;
+      }
+
+      DP("Image " DPxMOD " is compatible with RTL %s!\n",
+         DPxPTR(img->ImageStart), R.RTLName.c_str());
+
+      initRTLonce(R);
 
       // Initialize (if necessary) translation table for this library.
       PM->TrlTblMtx.lock();
       if (!PM->HostEntriesBeginToTransTable.count(desc->HostEntriesBegin)) {
         PM->HostEntriesBeginRegistrationOrder.push_back(desc->HostEntriesBegin);
         TranslationTable &TransTable =
             (PM->HostEntriesBeginToTransTable)[desc->HostEntriesBegin];
         TransTable.HostTable.EntriesBegin = desc->HostEntriesBegin;

openmp/libomptarget/tools/CMakeLists.txt

+##===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+##===----------------------------------------------------------------------===##
+#
+# Adding omptarget tools
+#
+##===----------------------------------------------------------------------===##
+
+add_subdirectory(deviceinfo)

openmp/libomptarget/tools/deviceinfo/CMakeLists.txt

+##===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+##===----------------------------------------------------------------------===##
+#
+# Build llvm-omp-device-info tool
+#
+##===----------------------------------------------------------------------===##
+
+libomptarget_say("Building the llvm-omp-device-info tool")
+libomptarget_say("llvm-omp-device-info using plugins ${LIBOMPTARGET_TESTED_PLUGINS}")
+
+add_llvm_tool(llvm-omp-device-info llvm-omp-device-info.cpp)
+
+llvm_update_compile_flags(llvm-omp-device-info)
+
+target_link_libraries(llvm-omp-device-info PRIVATE
+  omp
+  omptarget
+  ${LIBOMPTARGET_TESTED_PLUGINS}
+)

openmp/libomptarget/tools/deviceinfo/llvm-omp-device-info.cpp

+//===- llvm-omp-device-info.cpp - Obtain device info as seen from OpenMP --===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This is a command line utility that, by using Libomptarget, and the device
+// plugins, list devices information as seen from the OpenMP Runtime.
+//
+//===----------------------------------------------------------------------===//
+
+#include "omptarget.h"
+#include <cstdio>
+
+int main(int argc, char **argv) {
+  __tgt_bin_desc EmptyDesc = {0, nullptr, nullptr, nullptr};
+  __tgt_register_lib(&EmptyDesc);
+  __tgt_init_all_rtls();
+
+  for (int Dev = 0; Dev < omp_get_num_devices(); Dev++) {
+    printf("Device (%d):\n", Dev);
+    if (!__tgt_print_device_info(Dev))
+      printf("    print_device_info not implemented\n");
+    printf("\n");
+  }
+
+  __tgt_unregister_lib(&EmptyDesc);
+  return 0;
+}