[IR] Add TargetExtTypeClass

[nhaehnle]

Summary

This allows for extensibility in how the target type info is obtained,
serving two purposes:

1. Small step towards "Dialects in LLVM", in providing extensibility for users of LLVM. Users of LLVM can now define their own "target" extension types (now a somewhat misleading name) with custom layout and properties, to represent high-level concepts that are lowered away before code reaches the actual backend.

2. By storing a pointer to the type class in the type itself, we can avoid string comparisons when looking up type info.

[jsilvanus]

Is the patch missing changes in "llvm/IR/TargetExtType.h"?

[jsilvanus]

llvm/lib/IR/LLVMContext.cpp

409

Maybe add an assert that TypeClass is not yet registered?

[nhaehnle]

Major update, adding IR printing, parsing and bitcode writing/reading of the type info.

Also adds the ability to register a verifier for target types, which is used by aarch64.svcount.

[nhaehnle]

Remove two TODOs that have been done.

[arsenm]

llvm/docs/LangRef.rst

3810–3811

I don't think the LangRef should refer to doxygen for things like this and should explicitly list the properties

llvm/lib/AsmParser/LLParser.cpp

3259

Missing assembler tests for the various error cases

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

3992

Use tuple unbinding syntax instead of .first/.second

4009

Seems like it should be report_fatal_error

[nhaehnle]

llvm/docs/LangRef.rst

3810–3811

That text was pre-existing, but I'll replace it.

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

4009

This is the bitcode *writer*, so getting here is always a bug in the code, not a bug in any input (extending sdata::Value without updating this code here).

[nhaehnle]

• explicitly list properties in LangRef
• add negative parser/assembler tests
• use tuple binding syntax

[Flakebi]

llvm/test/Assembler/invalid-typeinfo.ll

52

What happens when adding a target type with the same name twice?

[nhaehnle]

llvm/test/Assembler/invalid-typeinfo.ll

52

This should be accepted as long as there's no conflict in the type info. I'm adding a test.

[nhaehnle]

Rebase, layout type defaults to void

[Flakebi]

A question inline, looks good to me otherwise.

llvm/docs/BitCodeFormat.rst

568

Is the XXXXYXXXXX string intended or accidental?

[nhaehnle]

llvm/docs/BitCodeFormat.rst

568

Whoops :) That is very much accidental. I meant to update that line but then managed to miss it on my final scan.

[nhaehnle]

Missed a spot...

[Flakebi]

Thanks! LGTM

[nikic]

Could you please add a test where a target type is first used and later type info for it is provided? I'd like to make sure this is an error.

[nikic]

llvm/include/llvm/Bitcode/LLVMBitCodes.h

738

What is SDATA_SYMBOL used for, in the context of this patch? It doesn't look like the corresponding IR handling in D150370 supports this.

[nhaehnle]

Could you please add a test where a target type is first used and later type info for it is provided? I'd like to make sure this is an error.

Added the conflicting-typeinfos3.ll subtest which cover this (if I understood you correctly).

[nhaehnle]

llvm/include/llvm/Bitcode/LLVMBitCodes.h

738

It isn't used yet. Symbols-as-values is something that I added when exploring how to put enums into metadata, and I didn't remove it here because it does seem useful.

[nikic]

In D147697#4435720, @nhaehnle wrote:

Could you please add a test where a target type is first used and later type info for it is provided? I'd like to make sure this is an error.

Added the conflicting-typeinfos3.ll subtest which cover this (if I understood you correctly).

Yes, that's what I had in mind. Thanks!

llvm/docs/LangRef.rst

3837

This could use some clarification on how the properties for target types with different arguments relate, if at all. If you define properties for target("mytype"), do those also apply to target("mytype", i32)?

Do you need to repeat the properties for every combination of arguments in the IR, or is there some inheritance?

llvm/include/llvm/Bitcode/LLVMBitCodes.h

738

In that case, please drop it until it is used for something (and thus testable).

[nhaehnle]

• Drop symbols-as-values for now
• LangRef clarifications

llvm/docs/LangRef.rst

3837

They don't relate and there is no inheritance, every target type is distinct and has its own properties. I did briefly consider the possibility of inheritance, but it seemed to quickly get very complex and not that useful. I'm going to try to clarify that.

[nikic]

llvm/docs/LangRef.rst

3837

I see. I think this point could use some input from people who are using target types, in particular @jcranmer-intel.

The way I originally expected this to work is that the target type properties are independent of the type arguments entirely, so that target("mytype") and all target("mytype", Ti, Ni) share the same properties.

I'm not familiar enough with how these types are used to say whether independent properties for each argument combination makes sense or not.

[nhaehnle]

Rebase

[jcranmer-intel]

llvm/docs/LangRef.rst

3837

All of the use cases I have had have the property that properties are based entirely on the string property. Indeed, one property that could be useful to check is the number of integer/type arguments!

That said, there are some cases where varying type parameters might change the properties of the type--suppose something like target("complex", float) versus target("complex", double) or target("decfp", 64) versus target("decfp", 32). Although for many of those types, I do question if they should be target extension types in the first place.

llvm/include/llvm/IR/TargetExtType.h

52–60

I question the need for these to be function pointers rather than instance properties. (The verifier function does need to be its own function).

[nhaehnle]

It seems this comes down to this question: Should the type properties be per type name or should they be per type?

But so far, it seems that nobody has very strong feelings in either direction. Personally, I'd err on the side of flexibility (hence the current shape of this change), but I don't feel strongly about it. How do we decide such things? :)

llvm/docs/LangRef.rst

3837

There's a question of how strongly you feel about this and how it would affect the runtime representation.

If we did change the properties to be per type name instead of per type, we should probably also store them per type name.

llvm/include/llvm/IR/TargetExtType.h

52–60

This is basically the same question as before. If we decide that these properties should be fixed on a per-type-name basis, then yes, we can just make them member variables here. Otherwise, they need to be fallbacks.

[nhaehnle]

llvm/include/llvm/IR/TargetExtType.h

52–60

s/fallbacks/callbacks/

llvm/docs/BitCodeFormat.rst

 
 * 16 --- `METADATA_ATTACHMENT`_ --- This contains records associating metadata
   with function instruction values.
 
 * 17 --- `TYPE_BLOCK`_ --- This describes all of the types in the module.
 
 * 23 --- `STRTAB_BLOCK`_ --- The bitcode file's string table.
 
+Structured Data
+^^^^^^^^^^^^^^^
+
+Structured data, i.e., sequences of (key, value) pairs, is embedded into some
+records by encoding it into a sequence of 64-bit integers.
+
+The encoding begins with the number of pairs, followed by the encoding of each
+pair. Each (key, value) pair is encoded as a structured data symbol encoding
+the key followed by a type discriminator indicating the type of the value,
+followed by an encoding of the value itself.
+
+The following types discriminators are defined:
+
+* TYPE (1) --- encoded as a single 64-bit integer --- XXXXYXXXXX ; if 0, indicates a missing
+  type (nullptr); otherwise, a 1-based index into the type table
+
+* SYMBOL (2) --- encoded as a pair of 64-bit integers, the first one being an
+  offset into the IR module's string table, and the second being the length of
+  the symbol string
+
+* INT (15+n) --- an n-bit integer, encoded as a little-endian sequence of
+  64-bit words
+
+The encoding of the integer type discriminator is chosen such that:
+
+* Encoding values 3 .. 15 are available for future extension
+* Type discriminators up to `i16` are encoded in unabbreviated records as a
+  single vbr6 chunk
+
 .. _MODULE_BLOCK:
 
 MODULE_BLOCK Contents
 ---------------------
 
 The ``MODULE_BLOCK`` block (id 8) is the top-level block for LLVM bitcode files,
 and each module in a bitcode file must contain exactly one. A bitcode file with
 multi-module bitcode is valid. In addition to records (described below)

 
 TYPE_CODE_X86_AMX Record
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 ``[X86_AMX]``
 
 The ``X86_AMX`` record (code 24) adds an ``x86_amx`` type to the type table.
 
+TYPE_CODE_TARGET_TYPE_OLD Record
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``[TARGET_TYPE_OLD, num_tys, ...ty_params..., ...int_params... ]``
+
+The ``TARGET_TYPE_OLD`` record (code 26) adds a target extension type to the
+type table, with a name defined by a previously encountered ``STRUCT_NAME``
+record. The operand fields are
+
+* *num_tys*: The number of parameters that are types (as opposed to integers)
+
+* *ty_params*: Type indices that represent type parameters
+
+* *int_params*: Numbers that correspond to the integer parameters.
+
 TYPE_CODE_TARGET_TYPE Record
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-``[TARGET_TYPE, num_tys, ...ty_params..., ...int_params... ]``
+``[TARGET_TYPE, num_tys, ...ty_params..., num_ints, ...int_params...,
+   structured data ]``
 
-The ``TARGET_TYPE`` record (code 26) adds a target extension type to the type
-table, with a name defined by a previously encountered ``STRUCT_NAME`` record.
-The operand fields are
+The ``TARGET_TYPE`` record (code 27) adds a target extension type to the
+type table, with a name defined by a previously encountered ``STRUCT_NAME``
+record. The operand fields are
 
 * *num_tys*: The number of parameters that are types (as opposed to integers)
 
 * *ty_params*: Type indices that represent type parameters
 
+* *num_ints*: The number of parameters that are integers
+
 * *int_params*: Numbers that correspond to the integer parameters.
 
+* The structured data is deserialized into target type info
+
 .. _CONSTANTS_BLOCK:
 
 CONSTANTS_BLOCK Contents
 ------------------------
 
 The ``CONSTANTS_BLOCK`` block (id 11) ...
 
 .. _FUNCTION_BLOCK:

llvm/docs/LangRef.rst

 nor is it legal to use them in :ref:`ptrtoint <i_ptrtoint>` or
 :ref:`inttoptr <i_inttoptr>` instructions. Similarly, they are not legal to use
 in an :ref:`icmp <i_icmp>` instruction.
 
 Target extension types have a name and optional type or integer parameters. The
 meanings of name and parameters are defined by the target. When being defined in
 LLVM IR, all of the type parameters must precede all of the integer parameters.
 
-Specific target extension types are registered with LLVM as having specific
-properties. These properties can be used to restrict the type from appearing in
-certain contexts, such as being the type of a global variable or having a
-``zeroinitializer`` constant be valid. A complete list of type properties may be
-found in the documentation for ``llvm::TargetExtType::Property`` (`doxygen
-<https://llvm.org/doxygen/classllvm_1_1TargetExtType.html>`_).
-
 :Syntax:
 
 .. code-block:: llvm
 
       target("label")
       target("label", void)
       target("label", void, i32)
       target("label", 0, 1, 2)
       target("label", void, i32, 0, 1, 2)
 
+Target extension types can be registered with LLVM as having specific
+properties. The following properties are defined:
+
+- ``layout``: The underlying type to be used to determine memory layouts, e.g.
+  store size. May be missing (null), in which case the type cannot be used in
+  a memory context (e.g., the type cannot be used with ``alloca``).
+- ``hasZeroInit``: Boolean property indicating whether the ``zeroinitializer``
+  constant is valid.
+- ``canBeGlobal``: Boolean property indicating whether global variables of this
+  type are valid.
+
+The registration of these properties may occur at runtime by defining an
+``llvm::TargetExtTypeClass`` object and registering it with the ``LLVMContext``.
+Some target extension types are registered in this way in all ``LLVMContext``s.
+
+The properties can also be set with a special ``type`` block of
+:ref:`structured data <structured_data>` at the beginning of the module file.
+
+:Syntax:
+
+::
+
+      type <target type> <structured data>
+
+:Example:
+
+.. code-block:: llvm
+  
+      type target("mytype") {
+        layout: type i8,
+        hasZeroInit: i1 true,
+      }
 
 .. _t_vector:
 
 Vector Type
 """""""""""
 
 :Overview:
 

llvm/include/llvm/AsmParser/LLParser.h

private:
     bool parseAnonStructType(Type *&Result, bool Packed);
     bool parseStructBody(SmallVectorImpl<Type *> &Body);
     bool parseStructDefinition(SMLoc TypeLoc, StringRef Name,
                                std::pair<Type *, LocTy> &Entry,
                                Type *&ResultTy);
 
     bool parseArrayVectorType(Type *&Result, bool IsVector);
     bool parseFunctionType(Type *&Result);
+    bool parseTargetExtTypeImpl(std::string &Name,
+                                SmallVectorImpl<Type *> &TypeParams,
+                                SmallVectorImpl<unsigned> &IntParams);
     bool parseTargetExtType(Type *&Result);
 
     // Function Semantic Analysis.
     class PerFunctionState {
       LLParser &P;
       Function &F;
       std::map<std::string, std::pair<Value*, LocTy> > ForwardRefVals;
       std::map<unsigned, std::pair<Value*, LocTy> > ForwardRefValIDs;
#include "llvm/IR/Metadata.def"
     int parseInsertValue(Instruction *&Inst, PerFunctionState &PFS);
     bool parseFreeze(Instruction *&I, PerFunctionState &PFS);
 
     // Use-list order directives.
     bool parseUseListOrder(PerFunctionState *PFS = nullptr);
     bool parseUseListOrderBB();
     bool parseUseListOrderIndexes(SmallVectorImpl<unsigned> &Indexes);
     bool sortUseListOrder(Value *V, ArrayRef<unsigned> Indexes, SMLoc Loc);
+
+    bool parseTypeInfo();
   };
 } // End llvm namespace
 
 #endif

llvm/include/llvm/Bitcode/LLVMBitCodes.h

enum TypeCodes {
 
   TYPE_CODE_TOKEN = 22, // TOKEN
 
   TYPE_CODE_BFLOAT = 23,  // BRAIN FLOATING POINT
   TYPE_CODE_X86_AMX = 24, // X86 AMX
 
   TYPE_CODE_OPAQUE_POINTER = 25, // OPAQUE_POINTER: [addrspace]
 
-  TYPE_CODE_TARGET_TYPE = 26, // TARGET_TYPE
+  TYPE_CODE_TARGET_TYPE_OLD = 26, // TARGET_TYPE without type info (old)
+
+  TYPE_CODE_TARGET_TYPE = 27, // TARGET_TYPE
 };
 
 enum OperandBundleTagCode {
   OPERAND_BUNDLE_TAG = 1, // TAG: [strchr x N]
 };
 
 enum SyncScopeNameCode {
   SYNC_SCOPE_NAME = 1,

 enum StrtabCodes {
   STRTAB_BLOB = 1,
 };
 
 enum SymtabCodes {
   SYMTAB_BLOB = 1,
 };
 
+enum StructuredDataCodes {
+  SDATA_TYPE = 1,
+  SDATA_SYMBOL = 2,
+  SDATA_INT_BASE = 15, // add the bit size
+};
+
 } // End bitc namespace
 } // End llvm namespace
 
 #endif

llvm/include/llvm/IR/DerivedTypes.h

 #include <cassert>
 #include <cstdint>
 
 namespace llvm {
 
 class Value;
 class APInt;
 class LLVMContext;
+class TargetExtTypeClass;
 
 /// Class to represent integer types. Note that this class is also used to
 /// represent the built-in integer types: Int1Ty, Int8Ty, Int16Ty, Int32Ty and
 /// Int64Ty.
 /// Integer representation type
 class IntegerType : public Type {
   friend class LLVMContextImpl;
 

 
 /// Class to represent target extensions types, which are generally
 /// unintrospectable from target-independent optimizations.
 ///
 /// Target extension types have a string name, and optionally have type and/or
 /// integer parameters. The exact meaning of any parameters is dependent on the
 /// target.
 class TargetExtType : public Type {
+  friend class TargetTypeInfoDeserialize;
+
   TargetExtType(LLVMContext &C, StringRef Name, ArrayRef<Type *> Types,
                 ArrayRef<unsigned> Ints);
+  static std::pair<TargetExtType *, bool> getInternal(LLVMContext &C,
+                                                      StringRef Name,
+                                                      ArrayRef<Type *> Types,
+                                                      ArrayRef<unsigned> Ints);
+  void initFromClass(const TargetExtTypeClass *Class);
 
+  Type *LayoutType = nullptr;
+  uint64_t Properties = 0;
   // These strings are ultimately owned by the context.
   StringRef Name;
   unsigned *IntParams;
 
 public:
   TargetExtType(const TargetExtType &) = delete;
   TargetExtType &operator=(const TargetExtType &) = delete;
 
   /// Return a target extension type having the specified name and optional
   /// type and integer parameters.
   static TargetExtType *get(LLVMContext &Context, StringRef Name,
                             ArrayRef<Type *> Types = std::nullopt,
                             ArrayRef<unsigned> Ints = std::nullopt);
 
+  /// Return a target extension type having the specified name and optional
+  /// type and integer parameters, but also run the type through the verifier
+  /// of the corresponding type class, if any is registered, and return null
+  /// if verification fails.
+  static TargetExtType *getChecked(LLVMContext &Context, StringRef Name,
+                                   ArrayRef<Type *> Types = std::nullopt,
+                                   ArrayRef<unsigned> Ints = std::nullopt,
+                                   raw_ostream &Errs = llvm::errs());
+
+  /// Return a target extension type of a known type class, having the specified
+  /// name and optional type and integer parameters.
+  static TargetExtType *get(LLVMContext &Context,
+                            const TargetExtTypeClass *Class, StringRef Name,
+                            ArrayRef<Type *> Types = std::nullopt,
+                            ArrayRef<unsigned> Ints = std::nullopt);
+
   /// Return the name for this target extension type. Two distinct target
   /// extension types may have the same name if their type or integer parameters
   /// differ.
   StringRef getName() const { return Name; }
 
   /// Return the type parameters for this particular target extension type. If
   /// there are no parameters, an empty array is returned.
   ArrayRef<Type *> type_params() const {
public:
   enum Property {
     /// zeroinitializer is valid for this target extension type.
     HasZeroInit = 1U << 0,
     /// This type may be used as the value type of a global variable.
     CanBeGlobal = 1U << 1,
   };
 
   /// Returns true if the target extension type contains the given property.
-  bool hasProperty(Property Prop) const;
+  bool hasProperty(Property Prop) const { return (Properties & Prop) == Prop; }
+
+  /// Returns the properties bit field.
+  uint64_t getProperties() const { return Properties; }
 
   /// Returns an underlying layout type for the target extension type. This
   /// type can be used to query size and alignment information, if it is
   /// appropriate (although note that the layout type may also be void). It is
   /// not legal to bitcast between this type and the layout type, however.
-  Type *getLayoutType() const;
+  Type *getLayoutType() const { return LayoutType; }
 
   /// Methods for support type inquiry through isa, cast, and dyn_cast.
   static bool classof(const Type *T) { return T->getTypeID() == TargetExtTyID; }
 };
 
 StringRef Type::getTargetExtName() const {
   return cast<TargetExtType>(this)->getName();
 }
 
 } // end namespace llvm
 
 #endif // LLVM_IR_DERIVEDTYPES_H

llvm/include/llvm/IR/LLVMContext.h

 class Function;
 class Instruction;
 class LLVMContextImpl;
 class Module;
 class OptPassGate;
 template <typename T> class SmallVectorImpl;
 template <typename T> class StringMapEntry;
 class StringRef;
+class TargetExtTypeClass;
 class Twine;
 class LLVMRemarkStreamer;
 
 namespace remarks {
 class RemarkStreamer;
 }
 
 namespace SyncScope {
#undef LLVM_FIXED_MD_KIND
   /// times, but only with the same value. Note that creating a pointer type or
   /// otherwise querying the opaque pointer mode performs an implicit set to
   /// the default value.
   void setOpaquePointers(bool Enable) const;
 
   /// Whether typed pointers are supported. If false, all pointers are opaque.
   bool supportsTypedPointers() const;
 
+  /// Register a custom extension type class.
+  void registerTargetExtTypeClass(const TargetExtTypeClass *TypeClass);
+
+  /// Find an extension type class.
+  const TargetExtTypeClass *findTargetExtTypeClass(StringRef Name) const;
+
 private:
   // Module needs access to the add/removeModule methods.
   friend class Module;
 
   /// addModule - Register a module as being instantiated in this context.  If
   /// the context is deleted, the module will be deleted as well.
   void addModule(Module*);
 

llvm/include/llvm/IR/TargetExtType.h

+//===- llvm/TargetExtType.h - Target extension types ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the declarations of classes that are needed to *define*
+// custom target extension types. Source files that only *use* target extension
+// types only need to include DerivedTypes.h.
+//
+// The implementation of these classes live in Type.cpp.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_IR_TARGETEXTTYPE_H
+#define LLVM_IR_TARGETEXTTYPE_H
+
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/StructuredData.h"
+
+namespace llvm {
+
+/// Description of a class of target extension types.
+///
+/// This allows users to define custom "target" extension types which are not
+/// actually recognized by any backend but are lowered away by LLVM IR-level
+/// transforms that are custom to the particular user of LLVM.
+///
+/// Type classes must be explicitly registered with the context(s) in which
+/// they are used, before target extension types are used for the first time.
+///
+/// Type classes have callback functions to verify extension types and fill
+/// in their type info. These callback functions operate on "partially
+/// initialized types", meaning that the type name and parameters are
+/// initialized, but the type info isn't. For example, this means that
+/// the callback functions must not use TargetExtType::hasProperty on the
+/// argument type. (However, any extension types that appear in the type's
+/// parameter list *are* fully initialized.)
+///
+/// The lifetime of a TargetExtTypeClass object must extend beyond the lifetime
+/// of any context in which it is registered.
+struct TargetExtTypeClass {
+  /// Name (or prefix of names) of the types in the class.
+  std::string Name;
+
+  /// Whether Name is only a prefix. Prefixes must end in '.'
+  bool NameIsPrefix;
+
+  /// Given a partially initialized type, return the layout type (defaults to
+  /// void, indicating that the type can't be laid out in memory).
+  using GetLayoutTypeFn = Type *(TargetExtType *T);
+  GetLayoutTypeFn *GetLayoutType = nullptr;
+
+  /// Given a partially initialized type, return the type properties (defaults
+  /// to 0).
+  using GetPropertiesFn = uint64_t(TargetExtType *T);
+  GetPropertiesFn *GetProperties = nullptr;
+
+  /// Given a partially initialized type, check the type for validity and return
+  /// true if valid.
+  using VerifierFn = bool(TargetExtType *T, raw_ostream &Errs);
+  VerifierFn *Verifier = nullptr;
+
+  TargetExtTypeClass(StringRef Name, bool NameIsPrefix = false)
+      : Name(Name), NameIsPrefix(NameIsPrefix) {}
+
+  TargetExtTypeClass &setGetLayoutType(GetLayoutTypeFn *Fn) {
+    GetLayoutType = Fn;
+    return *this;
+  }
+  TargetExtTypeClass &setGetProperties(GetPropertiesFn *Fn) {
+    GetProperties = Fn;
+    return *this;
+  }
+  TargetExtTypeClass &setVerifier(VerifierFn *Fn) {
+    Verifier = Fn;
+    return *this;
+  }
+};
+
+class TargetTypeInfoDeserialize {
+  LLVMContext &Ctx;
+  std::string Name;
+  SmallVector<Type *> Types;
+  SmallVector<unsigned> Ints;
+
+  Type *LayoutType = nullptr;
+  uint64_t Properties = 0;
+
+public:
+  TargetTypeInfoDeserialize(LLVMContext &Ctx, StringRef Name,
+                            ArrayRef<Type *> Types, ArrayRef<unsigned> Ints);
+
+  Error parseField(sdata::Symbol K, sdata::Value V);
+
+  Expected<TargetExtType *> finish();
+
+  static void registerSymbols();
+};
+
+/// Serialize the target type info into structured data.
+///
+/// If UseSchema is true, fields are generated according to a fixed schema.
+SmallVector<std::pair<sdata::Symbol, sdata::Value>>
+serializeTargetTypeInfo(TargetExtType *Ty, bool UseSchema);
+
+} // namespace llvm
+
+#endif // LLVM_IR_TARGETEXTTYPE_H

llvm/lib/AsmParser/LLParser.cpp

 //===-- LLParser.cpp - Parser Class ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file defines the parser class for .ll files.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/AsmParser/LLParser.h"
 #include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/DenseMap.h"
-#include "llvm/ADT/ScopeExit.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/ScopeExit.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/AsmParser/LLToken.h"
 #include "llvm/AsmParser/SlotMapping.h"
 #include "llvm/BinaryFormat/Dwarf.h"
 #include "llvm/IR/Argument.h"
 #include "llvm/IR/AutoUpgrade.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/CallingConv.h"
 #include "llvm/IR/Comdat.h"
 #include "llvm/IR/ConstantRange.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalIFunc.h"
 #include "llvm/IR/GlobalObject.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Operator.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/IR/Value.h"
 #include "llvm/IR/ValueSymbolTable.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/ModRef.h"
 #include "llvm/Support/SaveAndRestore.h"
 #include "llvm/Support/raw_ostream.h"
while (true) {
     case lltok::kw_uselistorder:
       if (parseUseListOrder())
         return true;
       break;
     case lltok::kw_uselistorder_bb:
       if (parseUseListOrderBB())
         return true;
       break;
+    case lltok::kw_type:
+      if (parseTypeInfo())
+        return true;
+      break;
     }
   }
 }
 
 /// toplevelentity
 ///   ::= 'module' 'asm' STRINGCONSTANT
 bool LLParser::parseModuleAsm() {
   assert(Lex.getKind() == lltok::kw_module);
bool LLParser::parseArrayVectorType(Type *&Result, bool IsVector) {
   } else {
     if (!ArrayType::isValidElementType(EltTy))
       return error(TypeLoc, "invalid array element type");
     Result = ArrayType::get(EltTy, Size);
   }
   return false;
 }
 
-/// parseTargetExtType - handle target extension type syntax
+/// parseTargetExtTypeImpl - handle target extension type syntax
 ///   TargetExtType
-///     ::= 'target' '(' STRINGCONSTANT TargetExtTypeParams TargetExtIntParams ')'
+///     ::= 'target' '(' STRINGCONSTANT TargetExtTypeParams TargetExtIntParams
+///         ')'
 ///
 ///   TargetExtTypeParams
 ///     ::= /*empty*/
 ///     ::= ',' Type TargetExtTypeParams
 ///
 ///   TargetExtIntParams
 ///     ::= /*empty*/
 ///     ::= ',' uint32 TargetExtIntParams
-bool LLParser::parseTargetExtType(Type *&Result) {
+bool LLParser::parseTargetExtTypeImpl(std::string &Name,
+                                      SmallVectorImpl<Type *> &TypeParams,
+                                      SmallVectorImpl<unsigned> &IntParams) {
+  assert(Lex.getKind() == lltok::kw_target);
   Lex.Lex(); // Eat the 'target' keyword.
 
   // Get the mandatory type name.
-  std::string TypeName;
   if (parseToken(lltok::lparen, "expected '(' in target extension type") ||
-      parseStringConstant(TypeName))
+      parseStringConstant(Name))
     return true;
 
   // Parse all of the integer and type parameters at the same time; the use of
   // SeenInt will allow us to catch cases where type parameters follow integer
   // parameters.
-  SmallVector<Type *> TypeParams;
-  SmallVector<unsigned> IntParams;
   bool SeenInt = false;
   while (Lex.getKind() == lltok::comma) {
     Lex.Lex(); // Eat the comma.
 
     if (Lex.getKind() == lltok::APSInt) {
       SeenInt = true;
       unsigned IntVal;
       if (parseUInt32(IntVal))
if (Lex.getKind() == lltok::APSInt) {
         return true;
       TypeParams.push_back(TypeParam);
     }
   }
 
   if (parseToken(lltok::rparen, "expected ')' in target extension type"))
     return true;
 
-  Result = TargetExtType::get(Context, TypeName, TypeParams, IntParams);
+  return false;
+}
+
+bool LLParser::parseTargetExtType(Type *&Result) {
+  auto Loc = Lex.getLoc();
+  std::string TypeName;
+  SmallVector<Type *> TypeParams;
+  SmallVector<unsigned> IntParams;
+  if (parseTargetExtTypeImpl(TypeName, TypeParams, IntParams))
+    return true;
+
+  std::string ErrStr;
+  raw_string_ostream Err(ErrStr);
+  Result =
+      TargetExtType::getChecked(Context, TypeName, TypeParams, IntParams, Err);
+  if (!Result)
+    return error(Loc, Twine("target type failed validation:\n") + ErrStr);
   return false;
 }
 
 //===----------------------------------------------------------------------===//
 // Function Semantic Analysis.
 //===----------------------------------------------------------------------===//
 
 LLParser::PerFunctionState::PerFunctionState(LLParser &p, Function &f,
case lltok::Type: {
         case lltok::kw_true:
         case lltok::kw_false:
           if (IntTy->getBitWidth() != 1)
             return tokError("true/false can only be used with i1");
           V = sdata::Value(Lex.getKind() == lltok::kw_true);
           Lex.Lex();
           break;
         default:
-          return tokError("expected an integer value");
+          return tokError("expected an integer constant");
         }
 
         break;
       }
 
       return tokError("only integer types are supported in structured data");
     }
 
     default:
-      return tokError("expected structured data value");
+      return tokError("expected the type of a structured data value");
     }
 
     if (ParseField(KeyLoc, Key, ValueLoc, V))
       return true;
 
     if (Lex.getKind() == lltok::rbrace)
       break;
     if (parseToken(lltok::comma, "expected ',' or '}' here"))
for (auto I : IdToIndexMap) {
     }
   }
 
   if (parseToken(lltok::rparen, "expected ')' in callsites"))
     return true;
 
   return false;
 }
+
+bool LLParser::parseTypeInfo() {
+  LocTy KwLoc = Lex.getLoc();
+  assert(Lex.getKind() == lltok::kw_type);
+  Lex.Lex();
+
+  if (Lex.getKind() != lltok::kw_target)
+    return tokError("expected 'target' type");
+
+  std::string TypeName;
+  SmallVector<Type *> TypeParams;
+  SmallVector<unsigned> IntParams;
+  if (parseTargetExtTypeImpl(TypeName, TypeParams, IntParams))
+    return true;
+
+  TargetTypeInfoDeserialize D(Context, TypeName, TypeParams, IntParams);
+
+  if (parseStructuredData(
+          [&](LocTy KeyLoc, sdata::Symbol K, LocTy ValueLoc, sdata::Value V) {
+            if (Error Err = D.parseField(K, V))
+              return error(KeyLoc, toString(std::move(Err)));
+            return false;
+          }))
+    return true;
+
+  if (Error Err = D.finish().takeError())
+    return error(KwLoc, toString(std::move(Err)));
+  return false;
+}

llvm/lib/Bitcode/Reader/BitcodeReader.cpp

 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/IntrinsicsAArch64.h"
 #include "llvm/IR/IntrinsicsARM.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/Operator.h"
+#include "llvm/IR/StructuredData.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Debug.h"
public:
 
   const char *getOpcodeName() const {
     return Instruction::getOpcodeName(Opcode);
   }
 };
 
 class BitcodeReader : public BitcodeReaderBase, public GVMaterializer {
   LLVMContext &Context;
+  sdata::SymbolTableLockGuard SdataSymbols;
   Module *TheModule = nullptr;
   // Next offset to start scanning for lazy parsing of function bodies.
   uint64_t NextUnreadBit = 0;
   // Last function offset found in the VST.
   uint64_t LastFunctionBlockBit = 0;
   bool SeenValueSymbolTable = false;
   uint64_t VSTOffset = 0;
 
private:
   Error globalCleanup();
   Error resolveGlobalAndIndirectSymbolInits();
   Error parseUseLists();
   Error findFunctionInStream(
       Function *F,
       DenseMap<Function *, uint64_t>::iterator DeferredFunctionInfoIterator);
 
   SyncScope::ID getDecodedSyncScopeID(unsigned Val);
+
+  Error decodeStructuredData(
+      ArrayRef<uint64_t> &Tail,
+      function_ref<Error(sdata::Symbol, sdata::Value)> ParseField);
 };
 
 /// Class to manage reading and parsing function summary index bitcode
 /// files/sections.
 class ModuleSummaryIndexBitcodeReader : public BitcodeReaderBase {
   /// The module index built during parsing.
   ModuleSummaryIndex &TheIndex;
 
static void upgradeDLLImportExportLinkage(GlobalValue *GV, unsigned Val) {
   if (GV->hasLocalLinkage())
     return;
   switch (Val) {
   case 5: GV->setDLLStorageClass(GlobalValue::DLLImportStorageClass); break;
   case 6: GV->setDLLStorageClass(GlobalValue::DLLExportStorageClass); break;
   }
 }
 
+Error BitcodeReader::decodeStructuredData(
+    ArrayRef<uint64_t> &Tail,
+    function_ref<Error(sdata::Symbol, sdata::Value)> ParseField) {
+  if (Tail.empty())
+    return error("missing sdata field count");
+
+  uint64_t NumFields = Tail[0];
+  Tail = Tail.drop_front(1);
+
+  for (uint64_t i = 0; i != NumFields; ++i) {
+    if (Tail.size() < 4)
+      return error("incomplete sdata field");
+
+    auto K = SdataSymbols.getSymbol(Context,
+                                    Strtab.slice(Tail[0], Tail[0] + Tail[1]));
+    Tail = Tail.drop_front(2);
+
+    sdata::Value V;
+    if (Tail[0] > bitc::SDATA_INT_BASE) {
+      unsigned NumBits = Tail[0] - bitc::SDATA_INT_BASE;
+      unsigned NumWords = divideCeil(NumBits, 64);
+      if (Tail.size() <= NumWords)
+        return error("incomplete sdata int value");
+
+      V = APInt(NumBits, Tail.slice(1, NumWords));
+      Tail = Tail.drop_front(1 + NumWords);
+    } else if (Tail[0] == bitc::SDATA_TYPE) {
+      V = getTypeByID(Tail[1]);
+      Tail = Tail.drop_front(2);
+    } else if (Tail[0] == bitc::SDATA_SYMBOL) {
+      if (Tail.size() < 3)
+        return error("incomplete sdata symbol value");
+      V = SdataSymbols.getSymbol(Context,
+                                 Strtab.slice(Tail[1], Tail[1] + Tail[2]));
+      Tail = Tail.drop_front(3);
+    } else {
+      return error("bad sdata value type: " + Twine(Tail[0]));
+    }
+
+    if (Error Err = ParseField(K, V))
+      return Err;
+  }
+
+  return Error::success();
+}
+
 Type *BitcodeReader::getTypeByID(unsigned ID) {
   // The type table size is always specified correctly.
   if (ID >= TypeList.size())
     return nullptr;
 
   if (Type *Ty = TypeList[ID])
     return Ty;
 
case bitc::TYPE_CODE_OPAQUE: {       // OPAQUE: []
         Res->setName(TypeName);
         TypeList[NumRecords] = nullptr;
       } else  // Otherwise, create a new struct with no body.
         Res = createIdentifiedStructType(Context, TypeName);
       TypeName.clear();
       ResultTy = Res;
       break;
     }
+    case bitc::TYPE_CODE_TARGET_TYPE_OLD:
     case bitc::TYPE_CODE_TARGET_TYPE: { // TARGET_TYPE: [NumTy, Tys..., Ints...]
+      bool IsOld = MaybeRecord.get() == bitc::TYPE_CODE_TARGET_TYPE_OLD;
+
       if (Record.size() < 1)
         return error("Invalid target extension type record");
 
       if (NumRecords >= TypeList.size())
         return error("Invalid TYPE table");
 
       if (Record[0] >= Record.size())
         return error("Too many type parameters");
 
       unsigned NumTys = Record[0];
       SmallVector<Type *, 4> TypeParams;
       SmallVector<unsigned, 8> IntParams;
       for (unsigned i = 0; i < NumTys; i++) {
         if (Type *T = getTypeByID(Record[i + 1]))
           TypeParams.push_back(T);
         else
           return error("Invalid type");
       }
 
-      for (unsigned i = NumTys + 1, e = Record.size(); i < e; i++) {
-        if (Record[i] > UINT_MAX)
+      ArrayRef<uint64_t> Tail = ArrayRef(Record).drop_front(NumTys + 1);
+      unsigned NumInts;
+
+      if (IsOld) {
+        NumInts = Tail.size();
+      } else {
+        NumInts = Tail[0];
+        Tail = Tail.drop_front(1);
+      }
+
+      for (uint64_t Val : Tail.take_front(NumInts)) {
+        if (Val > UINT_MAX)
           return error("Integer parameter too large");
-        IntParams.push_back(Record[i]);
+        IntParams.push_back(Val);
       }
-      ResultTy = TargetExtType::get(Context, TypeName, TypeParams, IntParams);
+
+      Tail = Tail.drop_front(NumInts);
+
+      TargetTypeInfoDeserialize D(Context, TypeName, TypeParams, IntParams);
+
+      if (!IsOld) {
+        if (Error Err = decodeStructuredData(
+                Tail, [&](sdata::Symbol K, sdata::Value V) {
+                  return D.parseField(K, V);
+                }))
+          return Err;
+      }
+
+      auto Result = D.finish();
+      if (Error Err = Result.takeError())
+        return Err;
+
+      ResultTy = Result.get();
       TypeName.clear();
       break;
     }
     case bitc::TYPE_CODE_ARRAY:     // ARRAY: [numelts, eltty]
       if (Record.size() < 2)
         return error("Invalid array type record");
       ResultTy = getTypeByID(Record[1]);
       if (!ResultTy || !ArrayType::isValidElementType(ResultTy))

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

 #include "llvm/IR/InstrTypes.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/Operator.h"
+#include "llvm/IR/StructuredData.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/UseListOrder.h"
 #include "llvm/IR/Value.h"
 #include "llvm/IR/ValueSymbolTable.h"
 #include "llvm/MC/StringTableBuilder.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Object/IRSymtab.h"
 #include "llvm/Support/AtomicOrdering.h"
for (const auto &GUIDSummaryLists : *Index)
           for (auto &CallEdge : FS->calls())
             if (!CallEdge.first.haveGVs() || !CallEdge.first.getValue())
               assignValueId(CallEdge.first.getGUID());
   }
 
 protected:
   void writePerModuleGlobalValueSummary();
 
+  void encodeSymbol(SmallVectorImpl<uint64_t> &Vals, sdata::Symbol S);
+  void
+  encodeStructuredData(SmallVectorImpl<uint64_t> &Vals,
+                       ArrayRef<std::pair<sdata::Symbol, sdata::Value>> Fields);
+
 private:
   void writePerModuleFunctionSummaryRecord(
       SmallVector<uint64_t, 64> &NameVals, GlobalValueSummary *Summary,
       unsigned ValueID, unsigned FSCallsAbbrev, unsigned FSCallsProfileAbbrev,
       unsigned CallsiteAbbrev, unsigned AllocAbbrev, const Function &F);
   void writeModuleLevelReferences(const GlobalVariable &V,
                                   SmallVector<uint64_t, 64> &NameVals,
                                   unsigned FSModRefsAbbrev,
for (Type *T : TypeList) {
     case Type::TargetExtTyID: {
       TargetExtType *TET = cast<TargetExtType>(T);
       Code = bitc::TYPE_CODE_TARGET_TYPE;
       writeStringRecord(Stream, bitc::TYPE_CODE_STRUCT_NAME, TET->getName(),
                         StructNameAbbrev);
       TypeVals.push_back(TET->getNumTypeParameters());
       for (Type *InnerTy : TET->type_params())
         TypeVals.push_back(VE.getTypeID(InnerTy));
+      TypeVals.push_back(TET->getNumIntParameters());
       for (unsigned IntParam : TET->int_params())
         TypeVals.push_back(IntParam);
+
+      auto Fields = serializeTargetTypeInfo(TET, /*UseSchema=*/false);
+      encodeStructuredData(TypeVals, Fields);
       break;
     }
     case Type::TypedPointerTyID:
       llvm_unreachable("Typed pointers cannot be added to IR modules");
     }
 
     // Emit the finished record.
     Stream.EmitRecord(Code, TypeVals, AbbrevToUse);
if (!PerModule) {
         Record.push_back(V);
     }
     Stream.EmitRecord(PerModule ? bitc::FS_PERMODULE_ALLOC_INFO
                                 : bitc::FS_COMBINED_ALLOC_INFO,
                       Record, AllocAbbrev);
   }
 }
 
+void ModuleBitcodeWriterBase::encodeSymbol(SmallVectorImpl<uint64_t> &Vals,
+                                           sdata::Symbol S) {
+  StringRef Str = S.getAsString();
+  Vals.push_back(StrtabBuilder.add(Str));
+  Vals.push_back(Str.size());
+}
+
+void ModuleBitcodeWriterBase::encodeStructuredData(
+    SmallVectorImpl<uint64_t> &Vals,
+    ArrayRef<std::pair<sdata::Symbol, sdata::Value>> Fields) {
+  Vals.push_back(Fields.size());
+  for (const auto &[Key, Value] : Fields) {
+    encodeSymbol(Vals, Key);
+
+    if (Value.isAPInt()) {
+      const APInt &V = Value.getAPInt();
+      Vals.push_back(bitc::SDATA_INT_BASE + V.getBitWidth());
+
+      unsigned NumWords = divideCeil(V.getBitWidth(), 64);
+      const uint64_t *RawData = V.getRawData();
+      Vals.insert(Vals.end(), RawData, RawData + NumWords);
+    } else if (Value.isType()) {
+      Type *T = Value.getType();
+      Vals.push_back(bitc::SDATA_TYPE);
+      Vals.push_back(VE.getTypeID(T));
+    } else if (Value.isSymbol()) {
+      sdata::Symbol S = Value.getSymbol();
+      Vals.push_back(bitc::SDATA_SYMBOL);
+      encodeSymbol(Vals, S);
+    } else {
+      llvm_unreachable("sdata value type not implemented");
+    }
+  }
+}
+
 // Helper to emit a single function summary record.
 void ModuleBitcodeWriterBase::writePerModuleFunctionSummaryRecord(
     SmallVector<uint64_t, 64> &NameVals, GlobalValueSummary *Summary,
     unsigned ValueID, unsigned FSCallsAbbrev, unsigned FSCallsProfileAbbrev,
     unsigned CallsiteAbbrev, unsigned AllocAbbrev, const Function &F) {
   NameVals.push_back(ValueID);
 
   FunctionSummary *FS = cast<FunctionSummary>(Summary);

llvm/lib/Bitcode/Writer/ValueEnumerator.cpp

if (StructType *STy = dyn_cast<StructType>(Ty))
     if (!STy->isLiteral())
       *TypeID = ~0U;
 
   // Enumerate all of the subtypes before we enumerate this type.  This ensures
   // that the type will be enumerated in an order that can be directly built.
   for (Type *SubTy : Ty->subtypes())
     EnumerateType(SubTy);
 
+  if (auto *ExtTy = dyn_cast<TargetExtType>(Ty))
+    EnumerateType(ExtTy->getLayoutType());
+
   // Refresh the TypeID pointer in case the table rehashed.
   TypeID = &TypeMap[Ty];
 
   // Check to see if we got the pointer another way.  This can happen when
   // enumerating recursive types that hit the base case deeper than they start.
   //
   // If this is actually a struct that we are treating as forward ref'able,
   // then emit the definition now that all of its contents are available.

llvm/lib/IR/AsmWriter.cpp

 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSlotTracker.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/Operator.h"
 #include "llvm/IR/StructuredData.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/TypeFinder.h"
 #include "llvm/IR/TypedPointerType.h"
 #include "llvm/IR/Use.h"
 #include "llvm/IR/User.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/Casting.h"
for (const auto &P : Type2Number) {
     assert(!NumberedTypes[P.second] && "Didn't get a unique numbering?");
     NumberedTypes[P.second] = P.first;
   }
   return NumberedTypes;
 }
 
 bool TypePrinting::empty() {
   incorporateTypes();
-  return TF.structs_empty() && Type2Number.empty();
+  return TF.structs_empty() && Type2Number.empty() && TF.target_exts_empty();
 }
 
 void TypePrinting::incorporateTypes() {
   if (!DeferredM)
     return;
 
   TF.run(*DeferredM, false);
   DeferredM = nullptr;
for (StructType *NamedType : TF.structs()) {
     PrintLLVMName(Out, NamedType->getName(), LocalPrefix);
     Out << " = type ";
 
     // Make sure we print out at least one level of the type structure, so
     // that we do not get %FILE = type %FILE
     TypePrinter.printStructBody(NamedType, Out);
     Out << '\n';
   }
+
+  for (TargetExtType *TTy : TF.target_exts()) {
+    auto Fields = serializeTargetTypeInfo(TTy, /*UseSchema=*/false);
+    if (Fields.empty())
+      continue;
+
+    Out << "type ";
+    TypePrinter.print(TTy, Out);
+    Out << ' ';
+
+    printStructuredData(Fields);
+  }
 }
 
 /// printFunction - Print all aspects of a function.
 void AssemblyWriter::printFunction(const Function *F) {
   if (AnnotationWriter) AnnotationWriter->emitFunctionAnnot(F, Out);
 
   if (F->isMaterializable())
     Out << "; Materializable\n";

llvm/lib/IR/LLVMContext.cpp

 #include "LLVMContextImpl.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/LLVMRemarkStreamer.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/Remarks/RemarkStreamer.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 #include <cstdlib>
 #include <string>
 #include <utility>
assert(SingleThreadSSID == SyncScope::SingleThread &&
          "singlethread synchronization scope ID drifted!");
   (void)SingleThreadSSID;
 
   SyncScope::ID SystemSSID =
       pImpl->getOrInsertSyncScopeID("");
   assert(SystemSSID == SyncScope::System &&
          "system synchronization scope ID drifted!");
   (void)SystemSSID;
+
+  TargetTypeInfoDeserialize::registerSymbols();
+
+  static const auto SpirvTypes =
+      TargetExtTypeClass("spirv.", true)
+          .setGetLayoutType([](TargetExtType *T) -> Type * {
+            return Type::getInt8PtrTy(T->getContext(), 0);
+          })
+          .setGetProperties([](TargetExtType *T) -> uint64_t {
+            return TargetExtType::HasZeroInit | TargetExtType::CanBeGlobal;
+          });
+  registerTargetExtTypeClass(&SpirvTypes);
+
+  static const auto Aarch64SVCount =
+      TargetExtTypeClass("aarch64.svcount")
+          .setGetLayoutType([](TargetExtType *T) -> Type * {
+            return ScalableVectorType::get(Type::getInt1Ty(T->getContext()),
+                                           16);
+          })
+          .setVerifier([](TargetExtType *T, raw_ostream &Errs) -> bool {
+            if (T->getNumTypeParameters() || T->getNumIntParameters()) {
+              Errs << "aarch64.svcount cannot have parameters\n";
+              return false;
+            }
+            return true;
+          });
+  registerTargetExtTypeClass(&Aarch64SVCount);
 }
 
 LLVMContext::~LLVMContext() { delete pImpl; }
 
 void LLVMContext::addModule(Module *M) {
   pImpl->OwnedModules.insert(M);
 }
 

 
 void LLVMContext::setOpaquePointers(bool Enable) const {
   pImpl->setOpaquePointers(Enable);
 }
 
 bool LLVMContext::supportsTypedPointers() const {
   return !pImpl->getOpaquePointers();
 }
+
+void LLVMContext::registerTargetExtTypeClass(
+    const TargetExtTypeClass *TypeClass) {
+  assert(!pImpl->TargetExtTypeClassesFrozen);
+  assert(!TypeClass->Name.empty());
+  assert(TypeClass->NameIsPrefix == StringRef(TypeClass->Name).ends_with("."));
+
+  pImpl->TargetExtTypeClasses.push_back(TypeClass);
+}
+
+const TargetExtTypeClass *
+LLVMContext::findTargetExtTypeClass(StringRef Name) const {
+  pImpl->TargetExtTypeClassesFrozen = true;
+
+  for (const TargetExtTypeClass *Class : pImpl->TargetExtTypeClasses) {
+    if ((Class->NameIsPrefix && Name.starts_with(Class->Name)) ||
+        (!Class->NameIsPrefix && Name == Class->Name)) {
+      return Class;
+    }
+  }
+
+  return nullptr;
+}

llvm/lib/IR/LLVMContextImpl.h

#include "llvm/IR/Metadata.def"
 
   /// Maintain the GC name for each function.
   ///
   /// This saves allocating an additional word in Function for programs which
   /// do not use GC (i.e., most programs) at the cost of increased overhead for
   /// clients which do use GC.
   DenseMap<const Function *, std::string> GCNames;
 
+  SmallVector<const TargetExtTypeClass *> TargetExtTypeClasses;
+  bool TargetExtTypeClassesFrozen = false;
+
   /// Flag to indicate if Value (other than GlobalValue) retains their name or
   /// not.
   bool DiscardValueNames = false;
 
   LLVMContextImpl(LLVMContext &C);
   ~LLVMContextImpl();
 
   /// Destroy the ConstantArrays if they are not used.

llvm/lib/IR/Type.cpp

 #include "llvm/ADT/APInt.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/Constant.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/TargetExtType.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/TypeSize.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 #include <utility>
 
 using namespace llvm;

 bool PointerType::isLoadableOrStorableType(Type *ElemTy) {
   return isValidElementType(ElemTy) && !ElemTy->isFunctionTy();
 }
 
 //===----------------------------------------------------------------------===//
 //                       TargetExtType Implementation
 //===----------------------------------------------------------------------===//
 
+namespace {
+
+struct TargetTypeInfoKeys {
+  sdata::RegisterSymbol Layout{"layout"};
+  sdata::RegisterSymbol HasZeroInit{"hasZeroInit"};
+  sdata::RegisterSymbol CanBeGlobal{"canBeGlobal"};
+
+  static const TargetTypeInfoKeys &get() {
+    static const TargetTypeInfoKeys TTIK;
+    return TTIK;
+  }
+};
+
+} // anonymous namespace
+
+TargetTypeInfoDeserialize::TargetTypeInfoDeserialize(LLVMContext &Ctx,
+                                                     StringRef Name,
+                                                     ArrayRef<Type *> Types,
+                                                     ArrayRef<unsigned> Ints)
+    : Ctx(Ctx), Name(Name), Types(Types), Ints(Ints) {
+  LayoutType = Type::getVoidTy(Ctx);
+}
+
+void TargetTypeInfoDeserialize::registerSymbols() {
+  (void)TargetTypeInfoKeys::get();
+}
+
+Error TargetTypeInfoDeserialize::parseField(sdata::Symbol K, sdata::Value V) {
+  const auto &TTIK = TargetTypeInfoKeys::get();
+
+  if (K == TTIK.Layout) {
+    if (!V.isType())
+      return sdata::makeDeserializeError("expected a type");
+    LayoutType = V.getType();
+    return Error::success();
+  }
+
+  struct {
+    const sdata::RegisterSymbol &Symbol;
+    uint64_t Value;
+  } BoolFields[] = {
+      {TTIK.CanBeGlobal, TargetExtType::CanBeGlobal},
+      {TTIK.HasZeroInit, TargetExtType::HasZeroInit},
+  };
+
+  for (const auto &Field : BoolFields) {
+    if (K == Field.Symbol) {
+      if (!V.isBool())
+        return sdata::makeDeserializeError("expected a boolean");
+      if (V.getBool())
+        Properties |= Field.Value;
+      else
+        Properties &= Field.Value;
+      return Error::success();
+    }
+  }
+
+  return sdata::makeDeserializeError(
+      "expected 'layout', 'canBeGlobal', or 'hasZeroInit'");
+}
+
+Expected<TargetExtType *> TargetTypeInfoDeserialize::finish() {
+  auto GetResult = TargetExtType::getInternal(Ctx, Name, Types, Ints);
+  TargetExtType *T = GetResult.first;
+
+  if (GetResult.second) {
+    auto *Class = Ctx.findTargetExtTypeClass(Name);
+    if (Class) {
+      if (Class->Verifier) {
+        std::string ErrStr;
+        raw_string_ostream Errs(ErrStr);
+        if (!Class->Verifier(T, Errs))
+          return sdata::makeDeserializeError(Twine("invalid target type\n:") +
+                                             ErrStr);
+      }
+      GetResult.first->initFromClass(Class);
+    } else {
+      T->LayoutType = LayoutType;
+      T->Properties = Properties;
+    }
+  }
+
+  if (T->LayoutType != LayoutType)
+    return sdata::makeDeserializeError("target type has wrong layout type");
+  if (T->Properties != Properties)
+    return sdata::makeDeserializeError("target type has wrong properties");
+
+  return T;
+}
+
+SmallVector<std::pair<sdata::Symbol, sdata::Value>>
+llvm::serializeTargetTypeInfo(TargetExtType *Ty, bool UseSchema) {
+  const auto &TTIK = TargetTypeInfoKeys::get();
+  SmallVector<std::pair<sdata::Symbol, sdata::Value>> Fields;
+
+  if (UseSchema || !Ty->getLayoutType()->isVoidTy())
+    Fields.emplace_back(TTIK.Layout, Ty->getLayoutType());
+
+  struct {
+    const sdata::RegisterSymbol &Symbol;
+    TargetExtType::Property Value;
+  } BoolFields[] = {
+      {TTIK.CanBeGlobal, TargetExtType::CanBeGlobal},
+      {TTIK.HasZeroInit, TargetExtType::HasZeroInit},
+  };
+
+  for (const auto &Field : BoolFields) {
+    if (UseSchema || Ty->hasProperty(Field.Value))
+      Fields.emplace_back(Field.Symbol, Ty->hasProperty(Field.Value));
+  }
+
+  return Fields;
+}
+
 TargetExtType::TargetExtType(LLVMContext &C, StringRef Name,
                              ArrayRef<Type *> Types, ArrayRef<unsigned> Ints)
     : Type(C, TargetExtTyID), Name(C.pImpl->Saver.save(Name)) {
   NumContainedTys = Types.size();
 
   // Parameter storage immediately follows the class in allocation.
   Type **Params = reinterpret_cast<Type **>(this + 1);
   ContainedTys = Params;
   for (Type *T : Types)
     *Params++ = T;
 
   setSubclassData(Ints.size());
   unsigned *IntParamSpace = reinterpret_cast<unsigned *>(Params);
   IntParams = IntParamSpace;
   for (unsigned IntParam : Ints)
     *IntParamSpace++ = IntParam;
 }
 
 TargetExtType *TargetExtType::get(LLVMContext &C, StringRef Name,
                                   ArrayRef<Type *> Types,
                                   ArrayRef<unsigned> Ints) {
+  auto Result = getInternal(C, Name, Types, Ints);
+  if (Result.second) {
+    const auto *Class = C.findTargetExtTypeClass(Name);
+    if (Class)
+      Result.first->initFromClass(Class);
+    else
+      Result.first->LayoutType = Type::getVoidTy(C);
+  }
+  return Result.first;
+}
+
+TargetExtType *TargetExtType::getChecked(LLVMContext &C, StringRef Name,
+                                         ArrayRef<Type *> Types,
+                                         ArrayRef<unsigned> Ints,
+                                         raw_ostream &Errs) {
+  auto Result = getInternal(C, Name, Types, Ints);
+  if (Result.second) {
+    const auto *Class = C.findTargetExtTypeClass(Name);
+    if (Class) {
+      if (Class->Verifier) {
+        if (!Class->Verifier(Result.first, Errs))
+          return nullptr;
+      }
+      Result.first->initFromClass(Class);
+    } else {
+      Result.first->LayoutType = Type::getVoidTy(C);
+    }
+  }
+  return Result.first;
+}
+
+TargetExtType *TargetExtType::get(LLVMContext &C,
+                                  const TargetExtTypeClass *Class,
+                                  StringRef Name, ArrayRef<Type *> Types,
+                                  ArrayRef<unsigned> Ints) {
+  assert(Name.starts_with(Class->Name));
+  assert(Class->NameIsPrefix || Class->Name.size() == Name.size());
+
+  auto Result = getInternal(C, Name, Types, Ints);
+  if (Result.second)
+    Result.first->initFromClass(Class);
+  return Result.first;
+}
+
+void TargetExtType::initFromClass(const TargetExtTypeClass *Class) {
+  assert(!Class->Verifier || Class->Verifier(this, llvm::errs()));
+  if (Class->GetLayoutType)
+    LayoutType = Class->GetLayoutType(this);
+  else
+    LayoutType = Type::getVoidTy(getContext());
+  if (Class->GetProperties)
+    Properties = Class->GetProperties(this);
+}
+
+std::pair<TargetExtType *, bool>
+TargetExtType::getInternal(LLVMContext &C, StringRef Name,
+                           ArrayRef<Type *> Types, ArrayRef<unsigned> Ints) {
+  assert(!Name.ends_with("."));
   const TargetExtTypeKeyInfo::KeyTy Key(Name, Types, Ints);
-  TargetExtType *TT;
+
   // Since we only want to allocate a fresh target type in case none is found
   // and we don't want to perform two lookups (one for checking if existent and
   // one for inserting the newly allocated one), here we instead lookup based on
   // Key and update the reference to the target type in-place to a newly
   // allocated one if not found.
   auto Insertion = C.pImpl->TargetExtTypes.insert_as(nullptr, Key);
-  if (Insertion.second) {
-    // The target type was not found. Allocate one and update TargetExtTypes
-    // in-place.
-    TT = (TargetExtType *)C.pImpl->Alloc.Allocate(
-        sizeof(TargetExtType) + sizeof(Type *) * Types.size() +
-            sizeof(unsigned) * Ints.size(),
-        alignof(TargetExtType));
-    new (TT) TargetExtType(C, Name, Types, Ints);
-    *Insertion.first = TT;
-  } else {
-    // The target type was found. Just return it.
-    TT = *Insertion.first;
-  }
-  return TT;
-}
-
-namespace {
-struct TargetTypeInfo {
-  Type *LayoutType;
-  uint64_t Properties;
-
-  template <typename... ArgTys>
-  TargetTypeInfo(Type *LayoutType, ArgTys... Properties)
-      : LayoutType(LayoutType), Properties((0 | ... | Properties)) {}
-};
-} // anonymous namespace
-
-static TargetTypeInfo getTargetTypeInfo(const TargetExtType *Ty) {
-  LLVMContext &C = Ty->getContext();
-  StringRef Name = Ty->getName();
-  if (Name.startswith("spirv."))
-    return TargetTypeInfo(Type::getInt8PtrTy(C, 0), TargetExtType::HasZeroInit,
-                          TargetExtType::CanBeGlobal);
-
-  // Opaque types in the AArch64 name space.
-  if (Name == "aarch64.svcount")
-    return TargetTypeInfo(ScalableVectorType::get(Type::getInt1Ty(C), 16));
-
-  return TargetTypeInfo(Type::getVoidTy(C));
-}
-
-Type *TargetExtType::getLayoutType() const {
-  return getTargetTypeInfo(this).LayoutType;
-}
-
-bool TargetExtType::hasProperty(Property Prop) const {
-  uint64_t Properties = getTargetTypeInfo(this).Properties;
-  return (Properties & Prop) == Prop;
+  if (!Insertion.second)
+    return {*Insertion.first, false};
+
+  auto *TT = (TargetExtType *)C.pImpl->Alloc.Allocate(
+  return {TT, true};
 }

llvm/test/Assembler/invalid-typeinfo.ll

+; RUN: split-file %s %t
+; RUN: not llvm-as < %t/not-target.ll -disable-output 2>&1 | FileCheck %t/not-target.ll
+; RUN: not llvm-as < %t/bad-target.ll -disable-output 2>&1 | FileCheck %t/bad-target.ll
+; RUN: not llvm-as < %t/eof1.ll -disable-output 2>&1 | FileCheck %t/eof1.ll
+; RUN: not llvm-as < %t/eof2.ll -disable-output 2>&1 | FileCheck %t/eof2.ll
+; RUN: not llvm-as < %t/unknown-field.ll -disable-output 2>&1 | FileCheck %t/unknown-field.ll
+; RUN: not llvm-as < %t/bad-value.ll -disable-output 2>&1 | FileCheck %t/bad-value.ll
+; RUN: not llvm-as < %t/layout-bad-type.ll -disable-output 2>&1 | FileCheck %t/layout-bad-type.ll
+; RUN: not llvm-as < %t/bad-bool.ll -disable-output 2>&1 | FileCheck %t/bad-bool.ll
+; RUN: not llvm-as < %t/missing-comma.ll -disable-output 2>&1 | FileCheck %t/missing-comma.ll
+; RUN: not llvm-as < %t/extra-comma.ll -disable-output 2>&1 | FileCheck %t/extra-comma.ll
+; RUN: not llvm-as < %t/missing-type.ll -disable-output 2>&1 | FileCheck %t/missing-type.ll
+; RUN: not llvm-as < %t/conflicting-typeinfos1.ll -disable-output 2>&1 | FileCheck %t/conflicting-typeinfos1.ll
+; RUN: not llvm-as < %t/conflicting-typeinfos2.ll -disable-output 2>&1 | FileCheck %t/conflicting-typeinfos2.ll
+
+;--- not-target.ll
+; CHECK: [[@LINE+1]]:6: error: expected 'target' type
+type i32 {}
+
+;--- bad-target.ll
+; CHECK: [[@LINE+1]]:20: error: expected type
+type target("foo", "bar") {}
+
+;--- eof1.ll
+; CHECK: [[@LINE+3]]:1: error: expected '{' here
+type target("foo")
+
+;--- eof2.ll
+; CHECK: [[@LINE+3]]:1: error: expected '}' or field label here
+type target("foo") {
+
+;--- unknown-field.ll
+; CHECK: [[@LINE+1]]:22: error: expected 'layout', 'canBeGlobal', or 'hasZeroInit'
+type target("foo") { unknown: i1 true, }
+
+;--- bad-value.ll
+; CHECK: [[@LINE+1]]:30: error: expected the type of a structured data value
+type target("foo") { layout: ###, }
+
+;--- layout-bad-type.ll
+; CHECK: [[@LINE+1]]:22: error: expected a type
+type target("foo") { layout: i1 true, }
+
+;--- bad-bool.ll
+; CHECK: [[@LINE+1]]:22: error: expected a boolean
+type target("foo") { canBeGlobal: i8 5 }
+
+;--- missing-comma.ll
+; CHECK: [[@LINE+1]]:39: error: expected ',' or '}' here
+type target("foo") { layout: type i32 canBeGlobal: i1 true }
+
+;--- extra-comma.ll
+; CHECK: [[@LINE+1]]:40: error: expected '}' or field label here
+type target("foo") { layout: type i32, , canBeGlobal: true }
+
+;--- missing-type.ll
+; CHECK: [[@LINE+1]]:35: error: expected the type of a structured data value
+type target("foo") { canBeGlobal: true }
+
+;--- conflicting-typeinfos1.ll
+; CHECK: [[@LINE+2]]:1: error: target type has wrong layout type
+type target("foo") { canBeGlobal: i1 true }
+type target("foo") { layout: type i32 }
+
+;--- conflicting-typeinfos2.ll
+; CHECK: [[@LINE+2]]:1: error: target type has wrong properties
+type target("foo") { canBeGlobal: i1 true }
+type target("foo") { }

llvm/test/Assembler/target-type-custom.ll

+; RUN: llvm-as < %s | llvm-dis | FileCheck %s
+
+; CHECK: type target("b", 1) {
+; CHECK:   canBeGlobal: i1 true,
+; CHECK:   hasZeroInit: i1 true,
+; CHECK: }
+; CHECK: type target("a") {
+; CHECK:   layout: type i32,
+; CHECK: }
+
+type target("a") {
+  layout: type i32,
+}
+type target("a") { ; duplicate type infos are accepted
+  layout: type i32,
+}
+type target("b", 0) {
+  hasZeroInit: i1 false,
+}
+type target("b", 1) {
+  hasZeroInit: i1 true,
+  canBeGlobal: i1 true,
+}
+type target("b", 2) {}
+
+; CHECK: @global = external global target("b", 1)
+@global = external global target("b", 1)
+
+; CHECK: declare void @callee(target("b", 1))
+declare void @callee(target("b", 1))
+
+; CHECK: define void @test1() {
+; CHECK:   %p = alloca target("a")
+; CHECK:   call void @callee(target("b", 1) zeroinitializer)
+; CHECK:   ret void
+; CHECK: }
+define void @test1() {
+  %p = alloca target("a")
+  call void @callee(target("b", 1) zeroinitializer)
+  ret void
+}

llvm/test/Bitcode/compatibility.ll

 
 target triple = "x86_64-apple-macosx10.10.0"
 ; CHECK: target triple = "x86_64-apple-macosx10.10.0"
 
 ;; Module-level assembly
 module asm "beep boop"
 ; CHECK: module asm "beep boop"
 
+type target("foo", i32, 5) {
+  layout: type i32,
+  canBeGlobal: i1 true,
+  hasZeroInit: i1 true,
+}
+; CHECK: type target("foo", i32, 5) {
+; CHECK:   layout: type i32,
+; CHECK:   canBeGlobal: i1 true,
+; CHECK:   hasZeroInit: i1 true,
+; CHECK: }
+
 ;; Comdats
 $comdat.any = comdat any
 ; CHECK: $comdat.any = comdat any
 $comdat.exactmatch = comdat exactmatch
 ; CHECK: $comdat.exactmatch = comdat exactmatch
 $comdat.largest = comdat largest
 ; CHECK: $comdat.largest = comdat largest
 $comdat.noduplicates = comdat nodeduplicate
define void @typesystem() {
   ; CHECK: %t10 = alloca <vscale x 4 x i32>
 
   ret void
 }
 
 declare void @llvm.token(token)
 ; CHECK: declare void @llvm.token(token)
 
+declare target("foo", i32, 5) @target_ext_type(target("bar"))
+; CHECK: declare target("foo", i32, 5) @target_ext_type(target("bar"))
+
 ;; Inline Assembler Expressions
 define void @inlineasm(i32 %arg) {
   call i32 asm "bswap $0", "=r,r"(i32 %arg)
   ; CHECK: call i32 asm "bswap $0", "=r,r"(i32 %arg)
   call i32 asm sideeffect "blt $1, $2, $3", "=r,r,rm"(i32 %arg, i32 %arg)
   ; CHECK: call i32 asm sideeffect "blt $1, $2, $3", "=r,r,rm"(i32 %arg, i32 %arg)
   ret void
 }

llvm/test/Verifier/target-type.ll

+; RUN: split-file %s %t
+; RUN: not llvm-as -disable-output %t/aarch64-svcount-type-parameter.ll 2>&1 | FileCheck --check-prefixes=CHECK %t/aarch64-svcount-type-parameter.ll
+; RUN: not llvm-as -disable-output %t/aarch64-svcount-int-parameter.ll 2>&1 | FileCheck --check-prefixes=CHECK %t/aarch64-svcount-int-parameter.ll
+
+;--- aarch64-svcount-type-parameter.ll
+
+; CHECK: [[@LINE+2]]:20: error: target type failed validation:
+; CHECK:   aarch64.svcount cannot have parameters
+declare void @test(target("aarch64.svcount", i32))
+
+;--- aarch64-svcount-int-parameter.ll
+
+; CHECK: [[@LINE+2]]:20: error: target type failed validation:
+; CHECK:   aarch64.svcount cannot have parameters
+declare void @test(target("aarch64.svcount", 5))