[HLSL] Enable vector types for hlsl.

[python3kgae]

Summary

Vector types in hlsl is using clang ext_vector_type.
Declaration of vector types is in builtin header hlsl.h.
hlsl.h will be included by default for hlsl shader.

[Anastasia]

clang/lib/Basic/LangOptions.cpp

121

Is this header expected to be large? You might want to flag up in the description of the review and the comments in the header itself what content is expected to be there.

If the file is expected to be large it might make sense to add a flag that would disable this include. You can then for example use bare clang without this header for all the tests that don't require functionality from the header to reduce the testing time.

clang/test/CodeGenHLSL/basic_types.hlsl

2

Technically mapping into IR types might be target specific, so passing the triple is necessary for this test to work correctly. Alternatively you can switch to AST dump checking, however even that might be target specific in some cases.

[python3kgae]

clang/lib/Basic/LangOptions.cpp

121

It might be large when more things are added.
I'll add an option to disable the include.

clang/test/CodeGenHLSL/basic_types.hlsl

2

The option -T lib_6_7 decides the triple.

"-Tlib_6_7 -Fo -" will be translated into
"-cc1" "-triple" "dxil--shadermodel6.7-library" "-o" "-" "-x" "hlsl" for cc1.

[python3kgae]

Add hlsl-no-stdinc to not include the hlsl header.

[Anastasia]

clang/lib/Driver/ToolChains/Clang.cpp

3490

Sharing command line options is great, let's just update the help message straight away that this flag now also applies to HLSL: https://github.com/llvm/llvm-project/blob/main/clang/include/clang/Driver/Options.td#L6053

[python3kgae]

Update comment for finclude_default_header.

[python3kgae]

Add newline at end of file

[Anastasia]

LGTM! Thanks

Please make sure to reflect in your commit message that you are adding the clang internal headers too.

[thakis]

Looks like this breaks tests on Mac/arm: http://45.33.8.238/macm1/36297/step_7.txt

Please take a look and revert for now if it takes a while to fix. (Maybe the test just needs an explicit triple?)

[thakis]

Reverted in c4eb8035ed6647e58d4c5161f393e9220f7402cf for now.

[python3kgae]

In D125052#3546205, @thakis wrote:

Reverted in c4eb8035ed6647e58d4c5161f393e9220f7402cf for now.

Hi thakis,
The issue could be fixed by https://reviews.llvm.org/D125585
Could you take a look?
Thanks

[python3kgae]

Reopen for the Mac/arm test fail.

[python3kgae]

Recover after fix Mac/arm test issue.

[rastogishubham]

Hi, this patch causes an issue with the CMake Xcode build configuration, if I try to use xcode as the generator with CMake, using the build command:

xcrun cmake -G Xcode -DCMAKE_BUILD_TYPE:STRING=Debug -DLLVM_ENABLE_ASSERTIONS:BOOL=TRUE -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DLLVM_ENABLE_PROJECTS='clang;' -DCMAKE_IGNORE_PATH="/usr/lib;/usr/local/lib;/lib" -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64" ../llvm

I get:

CMake Error in /Users/shubham/Development/llvm-project/clang/lib/Headers/CMakeLists.txt:
  The custom command generating

    /Users/shubham/Development/llvm-project/build/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)/lib/clang/15.0.0/include/hlsl.h

  is attached to multiple targets:

    hlsl-resource-headers
    clang-resource-headers

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".


CMake Error in /Users/shubham/Development/llvm-project/third-party/benchmark/CMakeLists.txt:
  The custom command generating

    /Users/shubham/Development/llvm-project/build/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)/lib/clang/15.0.0/include/hlsl.h

  is attached to multiple targets:

    hlsl-resource-headers
    clang-resource-headers

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".


CMake Generate step failed.  Build files cannot be regenerated correctly.

This is similar to the issue that was found in this patch: https://reviews.llvm.org/D123498

[rastogishubham]

Hi Xiang,

I would be more than happy to test the fix for you :). Though it is 11:45 pm for me now and I am going to bed. But I can take a look tomorrow.

Thanks,

Shubham

clang/include/clang/Driver/Options.td

def fdefault_calling_conv_EQ : Joined<["-"], "fdefault-calling-conv=">,
   HelpText<"Set default calling convention">,
   Values<"cdecl,fastcall,stdcall,vectorcall,regcall">,
   NormalizedValuesScope<"LangOptions">,
   NormalizedValues<["DCC_CDecl", "DCC_FastCall", "DCC_StdCall", "DCC_VectorCall", "DCC_RegCall"]>,
   MarshallingInfoEnum<LangOpts<"DefaultCallingConv">, "DCC_None">;
 
 // These options cannot be marshalled, because they are used to set up the LangOptions defaults.
 def finclude_default_header : Flag<["-"], "finclude-default-header">,
-  HelpText<"Include default header file for OpenCL">;
+  HelpText<"Include default header file for OpenCL and HLSL">;
 def fdeclare_opencl_builtins : Flag<["-"], "fdeclare-opencl-builtins">,
   HelpText<"Add OpenCL builtin function declarations (experimental)">;
 
 def fpreserve_vec3_type : Flag<["-"], "fpreserve-vec3-type">,
   HelpText<"Preserve 3-component vector type">,
   MarshallingInfoFlag<CodeGenOpts<"PreserveVec3Type">>;
 def fwchar_type_EQ : Joined<["-"], "fwchar-type=">,
   HelpText<"Select underlying type for wchar_t">,

 class DXCFlag<string name> : Option<["/", "-"], name, KIND_FLAG>,
   Group<dxc_Group>, Flags<[DXCOption, NoXarchOption]>;
 class DXCJoinedOrSeparate<string name> : Option<["/", "-"], name,
   KIND_JOINED_OR_SEPARATE>, Group<dxc_Group>, Flags<[DXCOption, NoXarchOption]>;
 
 def dxc_help : Option<["/", "-", "--"], "help", KIND_JOINED>,
   Group<dxc_Group>, Flags<[DXCOption, NoXarchOption]>, Alias<help>,
   HelpText<"Display available options">;
+def dxc_no_stdinc : DXCFlag<"hlsl-no-stdinc">,
+  HelpText<"HLSL only. Disables all standard includes containing non-native compiler types and functions.">;
 def Fo : DXCJoinedOrSeparate<"Fo">, Alias<o>,
   HelpText<"Output object file">;
 def dxil_validator_version : Option<["/", "-"], "validator-version", KIND_SEPARATE>,
   Group<dxc_Group>, Flags<[DXCOption, NoXarchOption, CC1Option, HelpHidden]>,
   HelpText<"Override validator version for module. Format: <major.minor>;"
            "Default: DXIL.dll version or current internal version">,
   MarshallingInfoString<TargetOpts<"DxilValidatorVersion">>;
 def target_profile : DXCJoinedOrSeparate<"T">, MetaVarName<"<profile>">,

clang/lib/Basic/LangOptions.cpp

void LangOptions::setLangDefaults(LangOptions &Opts, Language Lang,
   Opts.CPlusPlus2b = Std.isCPlusPlus2b();
   Opts.GNUMode = Std.isGNUMode();
   Opts.GNUCVersion = 0;
   Opts.HexFloats = Std.hasHexFloats();
   Opts.WChar = Std.isCPlusPlus();
   Opts.Digraphs = Std.hasDigraphs();
 
   Opts.HLSL = Lang == Language::HLSL;
+  if (Opts.HLSL && Opts.IncludeDefaultHeader)
+    Includes.push_back("hlsl.h");
 
   // Set OpenCL Version.
   Opts.OpenCL = Std.isOpenCL();
   if (LangStd == LangStandard::lang_opencl10)
     Opts.OpenCLVersion = 100;
   else if (LangStd == LangStandard::lang_opencl11)
     Opts.OpenCLVersion = 110;
   else if (LangStd == LangStandard::lang_opencl12)

clang/lib/Driver/ToolChains/Clang.cpp

static void RenderHLSLOptions(const ArgList &Args, ArgStringList &CmdArgs,
                               types::ID InputType) {
   const unsigned ForwardedArguments[] = {options::OPT_dxil_validator_version,
                                          options::OPT_S, options::OPT_emit_llvm,
                                          options::OPT_disable_llvm_passes};
 
   for (const auto &Arg : ForwardedArguments)
     if (const auto *A = Args.getLastArg(Arg))
       A->renderAsInput(Args, CmdArgs);
+  // Add the default headers if dxc_no_stdinc is not set.
+  if (!Args.hasArg(options::OPT_dxc_no_stdinc))
+    CmdArgs.push_back("-finclude-default-header");
 }
 
 static void RenderARCMigrateToolOptions(const Driver &D, const ArgList &Args,
                                         ArgStringList &CmdArgs) {
   bool ARCMTEnabled = false;
   if (!Args.hasArg(options::OPT_fno_objc_arc, options::OPT_fobjc_arc)) {
     if (const Arg *A = Args.getLastArg(options::OPT_ccc_arcmt_check,
                                        options::OPT_ccc_arcmt_modify,

clang/lib/Frontend/CompilerInvocation.cpp

#undef PREPROCESSOR_OPTION_WITH_MARSHALLING
 
   for (const auto &I : Opts.Includes) {
     // Don't generate OpenCL includes. They are implied by other flags that are
     // generated elsewhere.
     if (LangOpts.OpenCL && LangOpts.IncludeDefaultHeader &&
         ((LangOpts.DeclareOpenCLBuiltins && I == "opencl-c-base.h") ||
          I == "opencl-c.h"))
       continue;
+    // Don't generate HLSL includes. They are implied by other flags that are
+    // generated elsewhere.
+    if (LangOpts.HLSL && I == "hlsl.h")
+      continue;
 
     GenerateArg(Args, OPT_include, I, SA);
   }
 
   for (const auto &CI : Opts.ChainedIncludes)
     GenerateArg(Args, OPT_chain_include, CI, SA);
 
   for (const auto &RF : Opts.RemappedFiles)

clang/lib/Frontend/InitPreprocessor.cpp

static void InitializeStandardPredefinedMacros(const TargetInfo &TI,
                                                const FrontendOptions &FEOpts,
                                                MacroBuilder &Builder) {
   if (LangOpts.HLSL) {
     Builder.defineMacro("__hlsl_clang");
     // HLSL Version
     Builder.defineMacro("__HLSL_VERSION",
                         Twine((unsigned)LangOpts.getHLSLVersion()));
 
+    if (LangOpts.NativeHalfType)
+      Builder.defineMacro("__HLSL_ENABLE_16_BIT",
+                          Twine((unsigned)LangOpts.getHLSLVersion()));
+
     // Shader target information
     // "enums" for shader stages
     Builder.defineMacro("__SHADER_STAGE_VERTEX",
                         Twine((uint32_t)ShaderStage::Vertex));
     Builder.defineMacro("__SHADER_STAGE_PIXEL",
                         Twine((uint32_t)ShaderStage::Pixel));
     Builder.defineMacro("__SHADER_STAGE_GEOMETRY",
                         Twine((uint32_t)ShaderStage::Geometry));

clang/lib/Headers/CMakeLists.txt

 
 set(hip_files
   __clang_hip_libdevice_declares.h
   __clang_hip_cmath.h
   __clang_hip_math.h
   __clang_hip_runtime_wrapper.h
   )
 
+set(hlsl_files
+  hlsl.h
+  hlsl/hlsl_basic_types.h
+  )
+
 set(mips_msa_files
   msa.h
   )
 
 set(opencl_files
   opencl-c.h
   opencl-c-base.h
   )

 set(files
   ${core_files}
   ${arm_common_files}
   ${arm_only_files}
   ${aarch64_only_files}
   ${cuda_files}
   ${hexagon_files}
   ${hip_files}
+  ${hlsl_files}
   ${mips_msa_files}
   ${opencl_files}
   ${ppc_files}
   ${ppc_htm_files}
   ${systemz_files}
   ${ve_files}
   ${x86_files}
   ${webassembly_files}

 add_header_target("ppc-htm-resource-headers" "${ppc_htm_files}")
 add_header_target("riscv-resource-headers" "${riscv_generated_files}")
 add_header_target("systemz-resource-headers" "${systemz_files}")
 add_header_target("ve-resource-headers" "${ve_files}")
 add_header_target("webassembly-resource-headers" "${webassembly_files}")
 add_header_target("x86-resource-headers" "${x86_files}")
 
 # Other header groupings
+add_header_target("hlsl-resource-headers" ${hlsl_files})
 add_header_target("opencl-resource-headers" ${opencl_files})
 add_header_target("openmp-resource-headers" ${openmp_wrapper_files})
 add_header_target("windows-resource-headers" ${windows_only_files})
 add_header_target("utility-resource-headers" ${utility_files})
 
 set(header_install_dir lib${LLVM_LIBDIR_SUFFIX}/clang/${CLANG_VERSION}/include)
 
 #############################################################

 
 install(
   FILES ${x86_files}
   DESTINATION ${header_install_dir}
   EXCLUDE_FROM_ALL
   COMPONENT x86-resource-headers)
 
 install(
+  FILES ${hlsl_files}
+  DESTINATION ${header_install_dir}
+  EXCLUDE_FROM_ALL
+  COMPONENT hlsl-resource-headers)
+
+install(
   FILES ${opencl_files}
   DESTINATION ${header_install_dir}
   EXCLUDE_FROM_ALL
   COMPONENT opencl-resource-headers)
 
 install(
   FILES ${openmp_wrapper_files}
   DESTINATION ${header_install_dir}/openmp_wrappers
add_llvm_install_targets(install-ve-resource-headers
                            COMPONENT ve-resource-headers)
   add_llvm_install_targets(install-x86-resource-headers
                            DEPENDS x86-resource-headers
                            COMPONENT x86-resource-headers)
   add_llvm_install_targets(install-webassembly-resource-headers
                            DEPENDS webassembly-resource-headers
                            COMPONENT webassembly-resource-headers)
 
+  add_llvm_install_targets(install-hlsl-resource-headers
+                           DEPENDS hlsl-resource-headers
+                           COMPONENT hlsl-resource-headers)
   add_llvm_install_targets(install-opencl-resource-headers
                            DEPENDS opencl-resource-headers
                            COMPONENT opencl-resource-headers)
   add_llvm_install_targets(install-openmp-resource-headers
                            DEPENDS openmp-resource-headers
                            COMPONENT openmp-resource-headers)
   add_llvm_install_targets(install-windows-resource-headers
                            DEPENDS windows-resource-headers
                            COMPONENT windows-resource-headers)
   add_llvm_install_targets(install-utility-resource-headers
                            DEPENDS utility-resource-headers
                            COMPONENT utility-resource-headers)
 endif()

clang/lib/Headers/hlsl.h

+//===----- hlsl.h - HLSL definitions --------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _HLSL_H_
+#define _HLSL_H_
+
+#include "hlsl/hlsl_basic_types.h"
+
+#endif //_HLSL_H_

clang/lib/Headers/hlsl/hlsl_basic_types.h

+//===----- hlsl_basic_types.h - HLSL definitions for basic types ----------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _HLSL_HLSL_BASIC_TYPES_H_
+#define _HLSL_HLSL_BASIC_TYPES_H_
+
+// built-in scalar data types:
+
+#ifdef __HLSL_ENABLE_16_BIT
+// 16-bit integer.
+typedef unsigned short uint16_t;
+typedef short int16_t;
+#endif
+
+// unsigned 32-bit integer.
+typedef unsigned int uint;
+
+// 64-bit integer.
+typedef unsigned long uint64_t;
+typedef long int64_t;
+
+// built-in vector data types:
+
+#ifdef __HLSL_ENABLE_16_BIT
+typedef int16_t int16_t2 __attribute__((ext_vector_type(2)));
+typedef int16_t int16_t3 __attribute__((ext_vector_type(3)));
+typedef int16_t int16_t4 __attribute__((ext_vector_type(4)));
+typedef uint16_t uint16_t2 __attribute__((ext_vector_type(2)));
+typedef uint16_t uint16_t3 __attribute__((ext_vector_type(3)));
+typedef uint16_t uint16_t4 __attribute__((ext_vector_type(4)));
+#endif
+
+typedef int int2 __attribute__((ext_vector_type(2)));
+typedef int int3 __attribute__((ext_vector_type(3)));
+typedef int int4 __attribute__((ext_vector_type(4)));
+typedef uint uint2 __attribute__((ext_vector_type(2)));
+typedef uint uint3 __attribute__((ext_vector_type(3)));
+typedef uint uint4 __attribute__((ext_vector_type(4)));
+typedef int64_t int64_t2 __attribute__((ext_vector_type(2)));
+typedef int64_t int64_t3 __attribute__((ext_vector_type(3)));
+typedef int64_t int64_t4 __attribute__((ext_vector_type(4)));
+typedef uint64_t uint64_t2 __attribute__((ext_vector_type(2)));
+typedef uint64_t uint64_t3 __attribute__((ext_vector_type(3)));
+typedef uint64_t uint64_t4 __attribute__((ext_vector_type(4)));
+
+#ifdef __HLSL_ENABLE_16_BIT
+typedef half half2 __attribute__((ext_vector_type(2)));
+typedef half half3 __attribute__((ext_vector_type(3)));
+typedef half half4 __attribute__((ext_vector_type(4)));
+#endif
+
+typedef float float2 __attribute__((ext_vector_type(2)));
+typedef float float3 __attribute__((ext_vector_type(3)));
+typedef float float4 __attribute__((ext_vector_type(4)));
+typedef double double2 __attribute__((ext_vector_type(2)));
+typedef double double3 __attribute__((ext_vector_type(3)));
+typedef double double4 __attribute__((ext_vector_type(4)));
+
+#endif //_HLSL_HLSL_BASIC_TYPES_H_

clang/test/CodeGenHLSL/basic_types.hlsl

+// RUN: %clang_dxc  -Tlib_6_7 -fcgl -Fo - %s | FileCheck %s
+
+// FIXME: check 16bit types once enable-16bit-types is ready.
+
+// CHECK:@uint_Val = global i32 0, align 4
+// CHECK:@uint64_t_Val = global i64 0, align 8
+// CHECK:@int64_t_Val = global i64 0, align 8
+// CHECK:@int2_Val = global <2 x i32> zeroinitializer, align 8
+// CHECK:@int3_Val = global <3 x i32> zeroinitializer, align 16
+// CHECK:@int4_Val = global <4 x i32> zeroinitializer, align 16
+// CHECK:@uint2_Val = global <2 x i32> zeroinitializer, align 8
+// CHECK:@uint3_Val = global <3 x i32> zeroinitializer, align 16
+// CHECK:@uint4_Val = global <4 x i32> zeroinitializer, align 16
+// CHECK:@int64_t2_Val = global <2 x i64> zeroinitializer, align 16
+// CHECK:@int64_t3_Val = global <3 x i64> zeroinitializer, align 32
+// CHECK:@int64_t4_Val = global <4 x i64> zeroinitializer, align 32
+// CHECK:@uint64_t2_Val = global <2 x i64> zeroinitializer, align 16
+// CHECK:@uint64_t3_Val = global <3 x i64> zeroinitializer, align 32
+// CHECK:@uint64_t4_Val = global <4 x i64> zeroinitializer, align 32
+// CHECK:@float2_Val = global <2 x float> zeroinitializer, align 8
+// CHECK:@float3_Val = global <3 x float> zeroinitializer, align 16
+// CHECK:@float4_Val = global <4 x float> zeroinitializer, align 16
+// CHECK:@double2_Val = global <2 x double> zeroinitializer, align 16
+// CHECK:@double3_Val = global <3 x double> zeroinitializer, align 32
+// CHECK:@double4_Val = global <4 x double> zeroinitializer, align 32
+
+#define TYPE_DECL(T)  T T##_Val
+
+#ifdef __HLSL_ENABLE_16_BIT
+TYPE_DECL(uint16_t);
+TYPE_DECL(int16_t);
+#endif
+
+// unsigned 32-bit integer.
+TYPE_DECL(uint);
+
+// 64-bit integer.
+TYPE_DECL(uint64_t);
+TYPE_DECL(int64_t);
+
+// built-in vector data types:
+
+#ifdef __HLSL_ENABLE_16_BIT
+TYPE_DECL(int16_t2   );
+TYPE_DECL(int16_t3   );
+TYPE_DECL(int16_t4   );
+TYPE_DECL( uint16_t2 );
+TYPE_DECL( uint16_t3 );
+TYPE_DECL( uint16_t4 );
+#endif
+
+TYPE_DECL( int2  );
+TYPE_DECL( int3  );
+TYPE_DECL( int4  );
+TYPE_DECL( uint2 );
+TYPE_DECL( uint3 );
+TYPE_DECL( uint4     );
+TYPE_DECL( int64_t2  );
+TYPE_DECL( int64_t3  );
+TYPE_DECL( int64_t4  );
+TYPE_DECL( uint64_t2 );
+TYPE_DECL( uint64_t3 );
+TYPE_DECL( uint64_t4 );
+
+#ifdef __HLSL_ENABLE_16_BIT
+TYPE_DECL(half2 );
+TYPE_DECL(half3 );
+TYPE_DECL(half4 );
+#endif
+
+TYPE_DECL( float2  );
+TYPE_DECL( float3  );
+TYPE_DECL( float4  );
+TYPE_DECL( double2 );
+TYPE_DECL( double3 );
+TYPE_DECL( double4 );

clang/test/Driver/hlsl_no_stdinc.hlsl

+// RUN: %clang_dxc  -Tlib_6_7 -fcgl -Fo - %s -### 2>&1 | FileCheck %s --check-prefix=STDINC
+// RUN: %clang_dxc  -Tlib_6_7 -hlsl-no-stdinc -fcgl -Fo - %s -### 2>&1 | FileCheck %s --check-prefix=NOSTDINC
+
+// RUN: %clang -cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -ast-dump -o - %s -verify
+
+// Make sure hlsl-no-stdinc is translated into finclude-default-header.
+// STDINC:"-finclude-default-header"
+// NOSTDINC-NOT:"-finclude-default-header"
+
+// Make sure uint not work when finclude-default-header is off.
+// expected-error@+1 {{unknown type name 'uint'}}
+uint a;