[libcxx] <experimental/simd> Added internal storage type, constructors, subscript operators of class simd/simd_mask and related tests

[Joy12138]

Summary

[libcxx] <experimental/simd> Added internal storage type for class simd/simd_mask
[libcxx] <experimental/simd> Added all constructors of class simd/simd_mask and related tests
[libcxx] <experimental/simd> Added basic simd reference implementation, subscript operators of class simd/simd_mask and related tests

[Mordante]

I mainly look quickly over the patch, can you get the patch to apply properly so the CI can test it?

libcxx/include/experimental/__simd/utility.h

48

Will the compiler optimize this?
Since a bool has two values it might otherwise be better to have a constexpr function to generate the result at compile time.

[Mordante]

libcxx/include/experimental/__simd/utility.h

26

This change needs to be done at other functions too.

40

Let's make sure unsupported sizes are ill-formed instead of triggering UB.

[Joy12138]

Rebase.

Due to the large amount of content in this section, I will split them into two patches to submit.

In this patch, we mainly implements the internal storage type of the simd/simd_mask class and the basic part of its auxiliary class reference.

In the next patch we will implement relevant user interfaces, including constructors and subscript access operators, as well as related tests.

The status paper will be updated in the next patch.

[Joy12138]

Fix some comments

[Joy12138]

Fix some CI errors

[Joy12138]

Rebase

[Joy12138]

Fix CI

[philnik]

libcxx/include/experimental/__simd/reference.h

20

No need to call it out. If it's _Uglified it's library-internal.

20

__reference as a name is really generic.

46

It looks like you don't have any tests for the reference type currently. Please add some.

libcxx/include/experimental/__simd/scalar.h

30–31

Why make it so complicated? I'd just go for

// __get
return __data;

// __set
__data = __v;

Also, let's add a few _LIBCPP_ASSERT_UNCATEGORIZED (might need a rebase?) to make sure the index is never out of bounds.

39–40

That makes it a bit more obvious what you do when using it.

libcxx/include/experimental/__simd/utility.h

45

52–53

Isn't this just __v ? std::numeric_limits<_Up>::max() : 0?

libcxx/include/experimental/__simd/vec_ext.h

30–34

Why the conditional? Does this somehow help optimizations?

31

clang-tidy should actually complain here. I'll have to look into this.

42

libcxx/test/std/experimental/simd/test_utils.h

53–58

This isn't portable, and I don't really see the point. Nobody cares what the type is named internally.

[Joy12138]

Rebase and fix some comments

[Joy12138]

libcxx/include/experimental/__simd/vec_ext.h

30–34

libcxx CI - GCC 12 / C++latest gives errors with vector size that is not a power of two:

/home/libcxx-builder/.buildkite-agent/builds/google-libcxx-builder-8484195cd0ed-1/llvm-project/libcxx-ci/build/generic-gcc/include/c++/v1/experimental/__simd/vec_ext.h:30:7: error: number of vector components 3 not a power of two
   30 |   _Tp __data __attribute__((vector_size(sizeof(_Tp) * _Np)));
      |       ^~~~~~

It seems GCC 12 doesn't support non-power-of-two vector lengths. So I added the conditional.

libcxx/test/std/experimental/simd/test_utils.h

53–58

Then the type check of reference in the simd[_mask]_alias.pass.cpp is also unnecessary, right? Can I directly remove them?

[philnik]

libcxx/include/experimental/__simd/vec_ext.h

30–34

I would just always make it a power of two then. If there is no improvement when introducing the conditional, we can just remove it and make both compilers happy.

libcxx/test/std/experimental/simd/test_utils.h

53–58

Yes, they are unnecessary. You should add tests that check the properties of simd::reference and simd_mask::reference though.

[Joy12138]

Rebase and fix CI

[Joy12138]

Fix some comments

[Joy12138]

Fix CI

[Joy12138]

Add test for reference alias

[Joy12138]

Fix CI

[Joy12138]

Fix CI

[Joy12138]

Fix CI

[Joy12138]

Fix CI

[Joy12138]

Fix

[philnik]

LGTM with the remaining comments addressed.

libcxx/include/experimental/__simd/reference.h

34

To avoid ADL you don't have to fully qualify the name.

libcxx/include/experimental/__simd/scalar.h

31

Note: Once the data is actually reachable, this should get a test.

libcxx/include/experimental/__simd/utility.h

50

[Joy12138]

Fix comments

[Joy12138]

Rebase

[Joy12138]

Rebase

[Joy12138]

It seems that the long double is a type with a size of 12 bytes on MinGW (DLL, i686). This conflicts with the current implementation. Vector extension requires the input parameter of vector size to be a power of 2 and a multiple of the size of the element type. 12 bytes cannot meet both requirements simultaneously. Should we temporarily exclude long double on MinGW (DLL, i686)? I think it would be more appropriate to add support for it when implement platform specialization optimization. @philnik

[philnik]

In D144364#4636494, @Joy12138 wrote:

It seems that the long double is a type with a size of 12 bytes on MinGW (DLL, i686). This conflicts with the current implementation. Vector extension requires the input parameter of vector size to be a power of 2 and a multiple of the size of the element type. 12 bytes cannot meet both requirements simultaneously. Should we temporarily exclude long double on MinGW (DLL, i686)? I think it would be more appropriate to add support for it when implement platform specialization optimization. @philnik

Sounds fine to me with a TODO.

[Joy12138]

Excluded long double on MinGW (DLL, i686) temperarily

[Joy12138]

Fix CI

[Joy12138]

Rebase

libcxx/include/CMakeLists.txt

set(files
   exception
   execution
   expected
   experimental/__config
   experimental/__memory
   experimental/__simd/abi_tag.h
   experimental/__simd/aligned_tag.h
   experimental/__simd/declaration.h
+  experimental/__simd/internal_declaration.h
+  experimental/__simd/reference.h
   experimental/__simd/scalar.h
   experimental/__simd/simd.h
   experimental/__simd/simd_mask.h
   experimental/__simd/traits.h
   experimental/__simd/utility.h
   experimental/__simd/vec_ext.h
   experimental/deque
   experimental/forward_list

libcxx/include/experimental/__simd/internal_declaration.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H
-#define _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H
-
-#include <cstddef>
+#ifndef _LIBCPP_EXPERIMENTAL___SIMD_INTERNAL_DECLARATION_H
+#define _LIBCPP_EXPERIMENTAL___SIMD_INTERNAL_DECLARATION_H
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
-namespace simd_abi {
-struct __scalar {
-  static constexpr size_t __simd_size = 1;
-};
 
-} // namespace simd_abi
+template <class _Tp, class _Abi>
+struct __simd_storage;
+
+template <class _Tp, class _Abi>
+struct __mask_storage;
+
+template <class _Tp, class _Abi>
+struct __simd_operations;
+
+template <class _Tp, class _Abi>
+struct __mask_operations;
+
 } // namespace parallelism_v2
 _LIBCPP_END_NAMESPACE_EXPERIMENTAL
 
 #endif // _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
-#endif // _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H
+#endif // _LIBCPP_EXPERIMENTAL___SIMD_INTERNAL_DECLARATION_H

libcxx/include/experimental/__simd/reference.h

+// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP_EXPERIMENTAL___SIMD_REFERENCE_H
+#define _LIBCPP_EXPERIMENTAL___SIMD_REFERENCE_H
+
+#include <experimental/__simd/utility.h>
+
+#if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
+
+_LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
+inline namespace parallelism_v2 {
+template <class _Tp, class _Storage, class _Vp>
+class __reference { // exposition only
+  template <class, class>
+  friend class simd;
+  template <class, class>
+  friend class simd_mask;
+
+  _Storage& __s_;
+  size_t __idx_;
+
+  _LIBCPP_HIDE_FROM_ABI __reference(_Storage& __s, size_t __idx) : __s_(__s), __idx_(__idx) {}
+
+  _LIBCPP_HIDE_FROM_ABI _Vp __get() const { return __s_.__get(__idx_); }
+
+  _LIBCPP_HIDE_FROM_ABI void __set(_Vp __v) {
+    if constexpr (is_same_v<_Vp, bool>)
+      __s_.__set(__idx_, std::experimental::parallelism_v2::__set_all_bits<_Tp>(__v));
+    else
+      __s_.__set(__idx_, __v);
+  }
+
+public:
+  using value_type = _Vp;
+
+  __reference()                   = delete;
+  __reference(const __reference&) = delete;
+
+  _LIBCPP_HIDE_FROM_ABI operator value_type() const noexcept { return __get(); }
+};
+
+} // namespace parallelism_v2
+_LIBCPP_END_NAMESPACE_EXPERIMENTAL
+
+#endif // _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
+#endif // _LIBCPP_EXPERIMENTAL___SIMD_REFERENCE_H

libcxx/include/experimental/__simd/scalar.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H
 #define _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H
 
 #include <cstddef>
+#include <experimental/__simd/internal_declaration.h>
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
 namespace simd_abi {
 struct __scalar {
   static constexpr size_t __simd_size = 1;
 };
-
 } // namespace simd_abi
+
+template <class _Tp>
+struct __simd_storage<_Tp, simd_abi::__scalar> {
+  _Tp __data;
+
+  _Tp __get(size_t __idx) const noexcept { return (&__data)[__idx]; }
+  void __set(size_t __idx, _Tp __v) noexcept { (&__data)[__idx] = __v; }
+};
+
+template <class _Tp>
+struct __mask_storage<_Tp, simd_abi::__scalar> : __simd_storage<bool, simd_abi::__scalar> {};
+
+template <class _Tp>
+struct __simd_operations<_Tp, simd_abi::__scalar> {
+  using _Simd = __simd_storage<_Tp, simd_abi::__scalar>;
+  using _Mask = __mask_storage<_Tp, simd_abi::__scalar>;
+};
+
+template <class _Tp>
+struct __mask_operations<_Tp, simd_abi::__scalar> {
+  using _Mask = __mask_storage<_Tp, simd_abi::__scalar>;
+};
+
 } // namespace parallelism_v2
 _LIBCPP_END_NAMESPACE_EXPERIMENTAL
 
 #endif // _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 #endif // _LIBCPP_EXPERIMENTAL___SIMD_SCALAR_H

libcxx/include/experimental/__simd/simd.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP_EXPERIMENTAL___SIMD_SIMD_H
 #define _LIBCPP_EXPERIMENTAL___SIMD_SIMD_H
 
 #include <experimental/__simd/abi_tag.h>
 #include <experimental/__simd/declaration.h>
+#include <experimental/__simd/reference.h>
+#include <experimental/__simd/scalar.h>
 #include <experimental/__simd/traits.h>
+#include <experimental/__simd/vec_ext.h>
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
 
 // class template simd [simd.class]
 // TODO: implement simd class
 template <class _Tp, class _Abi>
 class simd {
+  using _Impl    = __simd_operations<_Tp, _Abi>;
+  using _Storage = typename _Impl::_Simd;
+
+  _Storage __s_;
+
 public:
   using value_type = _Tp;
+  using reference  = __reference<_Tp, _Storage, value_type>;
   using mask_type  = simd_mask<_Tp, _Abi>;
   using abi_type   = _Abi;
 
   static _LIBCPP_HIDE_FROM_ABI constexpr size_t size() noexcept { return simd_size_v<value_type, abi_type>; }
 };
 
 template <class _Tp>
 using native_simd = simd<_Tp, simd_abi::native<_Tp>>;

libcxx/include/experimental/__simd/simd_mask.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP_EXPERIMENTAL___SIMD_SIMD_MASK_H
 #define _LIBCPP_EXPERIMENTAL___SIMD_SIMD_MASK_H
 
 #include <experimental/__simd/abi_tag.h>
 #include <experimental/__simd/declaration.h>
+#include <experimental/__simd/reference.h>
+#include <experimental/__simd/scalar.h>
+#include <experimental/__simd/vec_ext.h>
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
 
 // class template simd_mask [simd.mask.class]
 // TODO: implement simd_mask class
 template <class _Tp, class _Abi>
 class simd_mask {
+  using _Impl    = __mask_operations<_Tp, _Abi>;
+  using _Storage = typename _Impl::_Mask;
+
+  _Storage __s_;
+
 public:
   using value_type = bool;
+  using reference  = __reference<_Tp, _Storage, value_type>;
   using simd_type  = simd<_Tp, _Abi>;
   using abi_type   = _Abi;
 
   static _LIBCPP_HIDE_FROM_ABI constexpr size_t size() noexcept { return simd_type::size(); }
 };
 
 template <class _Tp>
 using native_simd_mask = simd_mask<_Tp, simd_abi::native<_Tp>>;

libcxx/include/experimental/__simd/utility.h

 #ifndef _LIBCPP_EXPERIMENTAL___SIMD_UTILITY_H
 #define _LIBCPP_EXPERIMENTAL___SIMD_UTILITY_H
 
 #include <__type_traits/is_arithmetic.h>
 #include <__type_traits/is_const.h>
 #include <__type_traits/is_constant_evaluated.h>
 #include <__type_traits/is_same.h>
 #include <__type_traits/is_volatile.h>
+#include <cstdint>
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
 template <class _Tp>
 constexpr bool __is_vectorizable_v =
     is_arithmetic_v<_Tp> && !is_const_v<_Tp> && !is_volatile_v<_Tp> && !is_same_v<_Tp, bool>;
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI auto __choose_mask_type() {
+  if constexpr (sizeof(_Tp) == 1) {
+    return uint8_t{};
+  } else if constexpr (sizeof(_Tp) == 2) {
+    return uint16_t{};
+  } else if constexpr (sizeof(_Tp) == 4) {
+    return uint32_t{};
+  } else if constexpr (sizeof(_Tp) == 8) {
+    return uint64_t{};
+  }
+#  ifndef _LIBCPP_HAS_NO_INT128
+  else if constexpr (sizeof(_Tp) == 16) {
+    return __uint128_t{};
+  }
+#  endif
+  else
+    static_assert(sizeof(_Tp) == 0);
+}
+
+template <class _Tp>
+_LIBCPP_HIDE_FROM_ABI auto constexpr __set_all_bits(bool __v) {
+  using _Up = decltype(__choose_mask_type<_Tp>());
+  _Up __res = 0;
+  for (unsigned long __i = 0; __i < __CHAR_BIT__ * sizeof(_Tp); __i++)
+    __res |= static_cast<_Up>(__v) << __i;
+  return __res;
+}
+
 } // namespace parallelism_v2
 _LIBCPP_END_NAMESPACE_EXPERIMENTAL
 
 #endif // _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 #endif // _LIBCPP_EXPERIMENTAL___SIMD_UTILITY_H

libcxx/include/experimental/__simd/vec_ext.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP_EXPERIMENTAL___SIMD_VEC_EXT_H
 #define _LIBCPP_EXPERIMENTAL___SIMD_VEC_EXT_H
 
 #include <cstddef>
+#include <experimental/__simd/internal_declaration.h>
+#include <experimental/__simd/utility.h>
 
 #if _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 
 _LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
 inline namespace parallelism_v2 {
 namespace simd_abi {
 template <int _Np>
 struct __vec_ext {
   static constexpr size_t __simd_size = _Np;
 };
-
 } // namespace simd_abi
+
+template <class _Tp, int _Np>
+struct __simd_storage<_Tp, simd_abi::__vec_ext<_Np>> {
+#  if defined(_LIBCPP_COMPILER_CLANG_BASED)
+  _Tp __data __attribute__((vector_size(sizeof(_Tp) * _Np)));
+#  else
+  _Tp __data __attribute__((vector_size(std::__bit_ceil((sizeof(_Tp) * _Np)))));
+#  endif
+
+  _Tp __get(size_t __idx) const noexcept { return __data[__idx]; }
+  void __set(size_t __idx, _Tp __v) noexcept { __data[__idx] = __v; }
+};
+
+template <class _Tp, int _Np>
+struct __mask_storage<_Tp, simd_abi::__vec_ext<_Np>>
+    : __simd_storage<decltype(std::experimental::parallelism_v2::__choose_mask_type<_Tp>()), simd_abi::__vec_ext<_Np>> {
+};
+
+template <class _Tp, int _Np>
+struct __simd_operations<_Tp, simd_abi::__vec_ext<_Np>> {
+  using _Simd = __simd_storage<_Tp, simd_abi::__vec_ext<_Np>>;
+  using _Mask = __mask_storage<_Tp, simd_abi::__vec_ext<_Np>>;
+};
+
+template <class _Tp, int _Np>
+struct __mask_operations<_Tp, simd_abi::__vec_ext<_Np>> {
+  using _Mask = __mask_storage<_Tp, simd_abi::__vec_ext<_Np>>;
+};
+
 } // namespace parallelism_v2
 _LIBCPP_END_NAMESPACE_EXPERIMENTAL
 
 #endif // _LIBCPP_STD_VER >= 17 && defined(_LIBCPP_ENABLE_EXPERIMENTAL)
 #endif // _LIBCPP_EXPERIMENTAL___SIMD_VEC_EXT_H

libcxx/test/std/experimental/simd/simd.class/simd_alias.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11, c++14
 
 // Test the alias.
 // template <class T, class Abi> class simd {
 // public:
 //   using value_type = T;
+//   using reference = see below;
 //   using mask_type = simd_mask<T, Abi>;
 //   using abi_type = Abi;
 // };
 
 #include "../test_utils.h"
 #include <experimental/simd>
 
 namespace ex = std::experimental::parallelism_v2;
 
 template <class T, std::size_t>
 struct CheckSimdAlias {
   template <class SimdAbi>
   void operator()() {
     static_assert(std::is_same_v<typename ex::simd<T, SimdAbi>::value_type, T>);
+    static_assert(is_simd_reference<typename ex::simd<T, SimdAbi>::reference>::value);
     static_assert(std::is_same_v<typename ex::simd<T, SimdAbi>::mask_type, ex::simd_mask<T, SimdAbi>>);
     static_assert(std::is_same_v<typename ex::simd<T, SimdAbi>::abi_type, SimdAbi>);
   }
 };
 
 int main(int, char**) {
   test_all_simd_abi<CheckSimdAlias>();
   return 0;
 }

libcxx/test/std/experimental/simd/simd.mask.class/simd_mask_alias.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11, c++14
 
 // Test the alias.
 // template <class T, class Abi> class simd_mask {
 // public:
 //   using value_type = bool;
+//   using reference = see below;
 //   using simd_type = simd<T, Abi>;
 //   using abi_type = Abi;
 // };
 
 #include "../test_utils.h"
 #include <experimental/simd>
 
 namespace ex = std::experimental::parallelism_v2;
 
 template <class T, std::size_t>
 struct CheckSimdMaskAlias {
   template <class SimdAbi>
   void operator()() {
     static_assert(std::is_same_v<typename ex::simd_mask<T, SimdAbi>::value_type, bool>);
+    static_assert(is_simd_reference<typename ex::simd_mask<T, SimdAbi>::reference>::value);
     static_assert(std::is_same_v<typename ex::simd_mask<T, SimdAbi>::simd_type, ex::simd<T, SimdAbi>>);
     static_assert(std::is_same_v<typename ex::simd_mask<T, SimdAbi>::abi_type, SimdAbi>);
   }
 };
 
 int main(int, char**) {
   test_all_simd_abi<CheckSimdMaskAlias>();
   return 0;
 }

libcxx/test/std/experimental/simd/test_utils.h

 
 constexpr size_t bit_ceil(size_t val) {
   size_t pow = 1;
   while (pow < val)
     pow <<= 1;
   return pow;
 }
 
+template <class T>
+struct is_simd_reference : std::false_type {};
+
+template <class T, class S, class V>
+struct is_simd_reference<ex::__reference<T, S, V>> : std::true_type {};
+
 #endif // TEST_UTIL_H