Add clang headers that fix machine-dependent definitions on FreeBSD 9.2 (2nd attempt)

[kutuzov.viktor.84]

Summary

This patch is an updated version of the (previously reverted) D3908 patch.

The patch differs from D3908 in two things:

• it only copies and installs machine/_types.h and machine/_stdint.h on platforms where it's necessary (FreeBSD 9.2, currently) and
• it installs the two files mentioned in lib/clang/x.y.z/include/machine and not lib/clang/x.y.z/include and thus does not break resulting installation.

[kutuzov.viktor.84]

Richard, answering your question:

I'm also not sure why this patch makes any sense: it seems you're trying to work around FreeBSD's broken /usr/include files? That doesn't seem like something we should be doing to me, but maybe I'm missing some background here?

Well, I'm not sure what you would propose in situation like that, but the point is FreeBSD 9.2 headers define some things incorrectly in 32-bit mode and that patch fix them so we can pass 32-bit sanitizer tests.

[kutuzov.viktor.84]

Richard, somehow your questions didn't pass to this page; I will quote them here.

OK, lots of code has bugs. In the vast majority of cases, we don't teach the compiler to work around those bugs. Why is this case different? Shouldn't we tell people to patch their system headers instead? If we ship replacement system headers in every case where we find a problem, we'll have a maintenance nightmare; clang should try very hard not to grow its own version of fixincludes.

Yes, I realize that LLVM developers generally do not like to fix/workaround third-party defects and try to keep things small and simple. Well, I don't mind we do not commit this patch and leave it to FreeBSD 9.2 users to alter their systems the ways they like in order to build LLVM/clang and enable sanitizers. However, doing so several points should be taken into account, I believe. Here's the (most likely incomplete) list:

• It is not actually a bug, but rather incomplete 32-bit support. v9.3 adds more complete 32-bit support and (presumably) has no problems with headers in 32-bit mode, but there's no an "official patch" for v9.2, nor it is expected;

• According to FreeBSD developers, v9.2 is a kind of LTS (long-term support) version and is still in wide use. The FreeBSD sanitizers buildbot is running v9.2;

• Altering headers in omission of an "official patch" is a system-wide change that potentially break other software that workaround the issues with 32-bit definitions. Note that patching clang headers does not break anything as the HEAD version is known not to be the system compiler on any OS;

• This work on porting sanitizers to v9.2 helps to fix and improve subsequent FreeBSD versions so they compile and run smoothly without too much effort from the user's side. OTOH, once v9.2 is out of date enough, the workaround can be eliminated form the clang code base without affecting current FreeBSD users.

Also, why does this affect the sanitizer tests? Are they including these headers somehow?

Yes, since the sanitizer tests build and run in both 64-bit and 32-bit modes.

Can we fix them by detecting and fixing the bad typedefs/macros in those tests?

Yes, technically. In practice, I suspect the sanitizers team would be pretty much against following this way as that would mean adding a lot of FreeBSD 9.2-specific code to most of the tests.

[kutuzov.viktor.84]

Just for the record: the Richard Smith's response from the llvm-commits mailing list reads:

I do not think it is our job to "fix" partially-implemented OS features. This patch would take us in a very troubling direction, and it's not clear that we would ever be able to remove this hack. Please try to find another way to get the result you desire. If the sanitizers + FreeBSD 9.2 needs some patch to be applied to the system headers, then that patch should be provided by FreeBSD or possibly by the sanitizers, and *not* by the compiler.

So, 32-bit sanitizers tests are "officially" not supported on FreeBSD 9.2 and so we will proceed with the (upgraded) FreeBSD 10.0 sanitizers buildbot.

Thanks.

[emaste]

So, 32-bit sanitizers tests are "officially" not supported on FreeBSD 9.2 and so we will proceed with the (upgraded) FreeBSD 10.0 sanitizers buildbot.

That is reasonable, as -m32 compilation is officially not supported by
FreeBSD 9.x anyway. I merged as much as I could in time for 9.3 on a
best-effort basis, although I'm not sure if I addressed this specific
case or not.

lib/Headers/CMakeLists.txt

-set(files
+set(common_files
   altivec.h
   ammintrin.h
   arm_acle.h
   avxintrin.h
   avx2intrin.h
   avx512fintrin.h
   avx512erintrin.h
   bmiintrin.h
set(common_files
   x86intrin.h
   xmmintrin.h
   xopintrin.h
   cpuid.h
   unwind.h
   module.modulemap
   )
 
+set(machine_files)
+if( CMAKE_SYSTEM MATCHES "FreeBSD-9.2-RELEASE" )
+  list(APPEND machine_files
+    machine/_stdint.h
+    machine/_types.h)
+endif()
+
 set(output_dir ${LLVM_LIBRARY_OUTPUT_INTDIR}/clang/${CLANG_VERSION}/include)
 
 # Generate arm_neon.h
 clang_tablegen(arm_neon.h -gen-arm-neon
   SOURCE ${CLANG_SOURCE_DIR}/include/clang/Basic/arm_neon.td)
 
 set(out_files)
-foreach( f ${files} )
+foreach( f ${common_files} ${machine_files} )
   set( src ${CMAKE_CURRENT_SOURCE_DIR}/${f} )
   set( dst ${output_dir}/${f} )
   add_custom_command(OUTPUT ${dst}
     DEPENDS ${src}
     COMMAND ${CMAKE_COMMAND} -E copy_if_different ${src} ${dst}
     COMMENT "Copying clang's ${f}...")
   list(APPEND out_files ${dst})
 endforeach( f )
 
 add_custom_command(OUTPUT ${output_dir}/arm_neon.h 
   DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/arm_neon.h
   COMMAND ${CMAKE_COMMAND} -E copy_if_different ${CMAKE_CURRENT_BINARY_DIR}/arm_neon.h ${output_dir}/arm_neon.h
   COMMENT "Copying clang's arm_neon.h...")
 list(APPEND out_files ${output_dir}/arm_neon.h)
 
 add_custom_target(clang-headers ALL DEPENDS ${out_files})
 set_target_properties(clang-headers PROPERTIES FOLDER "Misc")
 
+set(install_destination lib${LLVM_LIBDIR_SUFFIX}/clang/${CLANG_VERSION}/include)
+
+install(
+  FILES ${common_files} ${CMAKE_CURRENT_BINARY_DIR}/arm_neon.h
+  PERMISSIONS OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ
+  DESTINATION ${install_destination})
+
+if( machine_files )
   install(
-  FILES ${files} ${CMAKE_CURRENT_BINARY_DIR}/arm_neon.h
+    FILES ${machine_files}
     PERMISSIONS OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ
-  DESTINATION lib${LLVM_LIBDIR_SUFFIX}/clang/${CLANG_VERSION}/include)
+    DESTINATION ${install_destination}/machine)
+endif()

lib/Headers/machine/_stdint.h

+/*===---- _stdint.h - C99 type-related definitions on FreeBSD -------------===*\
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+\*===----------------------------------------------------------------------===*/
+
+#ifndef __MACHINE_XSTDINT_H
+#define __MACHINE_XSTDINT_H
+
+#include_next <machine/_stdint.h>
+
+/* Fix some definitions on x86-64 FreeBSD 9.2 in 32-bit mode. */
+#if defined(__FreeBSD__) && defined(__i386__)
+# include <osreldate.h>
+# if __FreeBSD_version <= 902001  // v9.2
+#  if !defined(__cplusplus) || defined(__STDC_CONSTANT_MACROS)
+#   undef INT64_C
+#   define INT64_C(c) (c ## LL)
+
+#   undef UINT64_C
+#   define UINT64_C(c) (c ## ULL)
+#  endif  /* !defined(__cplusplus) || defined(__STDC_CONSTANT_MACROS) */
+
+#  if !defined(__cplusplus) || defined(__STDC_LIMIT_MACROS)
+#   undef INT64_MIN
+#   define INT64_MIN (-0x7fffffffffffffffLL-1)
+
+#   undef INT64_MAX
+#   define INT64_MAX 0x7fffffffffffffffLL
+
+#   undef UINT64_MAX
+#   define UINT64_MAX 0xffffffffffffffffULL
+
+#   undef INTPTR_MIN
+#   define INTPTR_MIN INT32_MIN
+
+#   undef INTPTR_MAX
+#   define INTPTR_MAX INT32_MAX
+
+#   undef UINTPTR_MAX
+#   define UINTPTR_MAX UINT32_MAX
+
+#   undef PTRDIFF_MIN
+#   define PTRDIFF_MIN INT32_MIN
+
+#   undef PTRDIFF_MAX
+#   define PTRDIFF_MAX INT32_MAX
+
+#   undef SIZE_MAX
+#   define SIZE_MAX UINT32_MAX
+#  endif  /* !defined(__cplusplus) || defined(__STDC_LIMIT_MACROS) */
+# endif  /* __FreeBSD_version <= 902001 */
+#endif  /* defined(__FreeBSD__) && defined(__i386__) */
+
+#endif /* !__MACHINE_XSTDINT_H */

lib/Headers/machine/_types.h

+/*===---- _stdint.h - Machine-dependent type definitions on FreeBSD -------===*\
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+\*===----------------------------------------------------------------------===*/
+
+#ifndef __MACHINE_XTYPES_H
+#define __MACHINE_XTYPES_H
+
+/* Fix some definitions on x86-64 FreeBSD 9.2 in 32-bit mode. */
+#if defined(__FreeBSD__) && defined(__i386__)
+# include <osreldate.h>
+# if __FreeBSD_version <= 902001  // v9.2
+#  define __FIX_FREEBSD_9_2_DEFINITIONS
+# endif  /* __FreeBSD_version <= 902001 */
+#endif  /* defined(__FreeBSD__) && defined(__i386__) */
+
+#if defined(__FIX_FREEBSD_9_2_DEFINITIONS)
+#define __int64_t __broken_int64_t
+#define __uint64_t __broken_uint64_t
+#define __critical_t __broken_critical_t
+#define __intfptr_t __broken_intfptr_t
+#define __intptr_t __broken_intptr_t
+#define __intmax_t __broken_intmax_t
+#define __int_fast64_t __broken_int_fast64_t
+#define __int_least64_t __broken_int_least64_t
+#define __ptrdiff_t __broken_ptrdiff_t
+#define __register_t __broken_register_t
+#define __segsz_t __broken_segsz_t
+#define __size_t __broken_size_t
+#define __ssize_t __broken_ssize_t
+#define __time_t __broken_time_t
+#define __uintfptr_t __broken_uintfptr_t
+#define __uintptr_t __broken_uintptr_t
+#define __uintmax_t __broken_uintmax_t
+#define __uint_fast64_t __broken_uint_fast64_t
+#define __uint_least64_t __broken_uint_least64_t
+#define __u_register_t __broken_u_register_t
+#define __vm_offset_t __broken_vm_offset_t
+#define __vm_paddr_t __broken_vm_paddr_t
+#define __vm_size_t __broken_vm_size_t
+#define __vm_ooffset_t __broken_vm_ooffset_t
+#define __vm_pindex_t __broken_vm_pindex_t
+#endif  // defined(__FIX_FREEBSD_9_2_DEFINITIONS)
+
+#include_next <machine/_types.h>
+
+#if defined(__FIX_FREEBSD_9_2_DEFINITIONS)
+#undef __int64_t
+typedef long long __int64_t;
+
+#undef __uint64_t
+typedef unsigned long long __uint64_t;
+
+#undef __critical_t
+typedef __int32_t __critical_t;
+
+#undef __intfptr_t
+typedef __int32_t __intfptr_t;
+
+#undef __intptr_t
+typedef __int32_t __intptr_t;
+
+#undef __intmax_t
+typedef __int64_t __intmax_t;
+
+#undef __int_fast64_t
+typedef __int64_t __int_fast64_t;
+
+#undef __int_least64_t
+typedef __int64_t __int_least64_t;
+
+#undef __ptrdiff_t
+typedef __int32_t __ptrdiff_t;
+
+#undef __register_t
+typedef __int64_t __register_t;
+
+#undef __segsz_t
+typedef __int32_t __segsz_t;
+
+#undef __size_t
+typedef __uint32_t __size_t;
+
+#undef __ssize_t
+typedef __int32_t __ssize_t;
+
+#undef __time_t
+typedef __int32_t __time_t;
+
+#undef __uintfptr_t
+typedef __uint32_t __uintfptr_t;
+
+#undef __uintptr_t
+typedef __uint32_t __uintptr_t;
+
+#undef __uintmax_t
+typedef __uint64_t __uintmax_t;
+
+#undef __uint_fast64_t
+typedef __uint64_t __uint_fast64_t;
+
+#undef __uint_least64_t
+typedef __uint64_t __uint_least64_t;
+
+#undef __u_register_t
+typedef __uint32_t __u_register_t;
+
+#undef __vm_offset_t
+typedef __uint32_t __vm_offset_t;
+
+#undef __vm_paddr_t
+#ifdef PAE
+typedef __uint64_t __vm_paddr_t;
+#else
+typedef __uint32_t __vm_paddr_t;
+#endif
+
+#undef __vm_size_t
+typedef __uint32_t __vm_size_t;
+
+#undef __vm_ooffset_t
+typedef __int64_t __vm_ooffset_t;
+
+#undef __vm_pindex_t
+typedef __uint64_t __vm_pindex_t;
+#endif  // defined(__FIX_FREEBSD_9_2_DEFINITIONS)
+
+#endif  /* !__MACHINE_XTYPES_H */