[mlir][llvm] Import pointer data layout specification.

[gysit]

Summary

The revision moves the data layout parsing into a separate file
and extends it to support pointer data layout specifications.
Additionally, it also produces more precise warnings and error
messages.

[definelicht]

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

44

Wouldn't it be useful to have error messages on what failed parsing?

83–88

It would be useful to show examples of alignment strings in a comment to understand why this code looks the way it does

99–107

Again, it would be great with some examples of what this string can look like and why 🙂

140–141

192–193

This should be a constexpr defined somewhere very visible, perhaps even in the header file? Is there a test that would flag if this ever changed?

200

mlir/lib/Target/LLVMIR/DataLayoutImporter.h

42

[Dinistro]

I dropped some comments. As elaborated in one of the comments, I'm uncertain if it makes sense to add an instance of the DataLayoutImporter to the ModuleImport class and keep it around.

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

37

Ultra nit: I would prefer {} here.

44

I guess it's unclear where this error message should be attached to. But I agree, error messages would be nice to have.

46

NIT: Can you not just pass the isalpha function to the take_while or is there some type inference issue?

55

I'm not sure about the coding conventions, but this initialisation is not necessary.

255

Is there a good reason why this function expects a Type instead of just a LLVMPointerType?

267

NIT: Are you sure that key will not be treated like an unused variable by all compilers. I recall some issues related to this, but that might be a false alarm.

269

As above

271

NIT: No need to store this in a local variable.

mlir/lib/Target/LLVMIR/DataLayoutImporter.h

58

Should that really return a copy of the SmallVector? The only usage would work with an ArrayRef<StringRef>

mlir/lib/Target/LLVMIR/ModuleImport.cpp

703

Given that this is the only usage of the dataLayoutImporter outside of the ModuleImport::convertDataLayout, it seems a bit excessive to keep it around over the full lifetime of a ModuleImporter.
This function could still take the context as an argument and be decoupled from the DataLayoutImporter class. In that case, the imported could be allocated on the stack of the ModuleImport::convertDataLayout function.

[gysit]

Rebase and address comments.

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

44

Producing precise error messages is hard since there are many places where parsing may fail. There is also no real error handling infrastructure here. The good thing is the user we get the token that failed parsing (e.g. something like i32:32:32). That should be precise enough especially since this data layout strings are mostly not user facing and we do not expect any parsing errors (produced by LLVM and consumed by MLIR).

192–193

It would be nice to get the string directly from LLVM but I did not find a good way to achieve this since LLVM returns an empty string for the default layout. I added a test that verifies all elements of the default.

[Dinistro]

LGTM module the UB comment.

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

112

Careful here ;)

[gysit]

Improve test.

[Dinistro]

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

112

NVM, I mixed something up here.

[definelicht]

LGTM modulo nits! It's very nice to have the comments describing possible layouts, it's a big service to anyone who ever needs to look at this again 😉

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

103

mlir/lib/Target/LLVMIR/DataLayoutImporter.h

68–71

I would also still be in favor of exposing this publicly, either by making it public or moving it out of the class. God forbid anyone will need it, but it's better than it can be retrieved like this instead of copy-pasting it 😉

[gysit]

Address comments.

[gysit]

Address last comment.

[gysit]

mlir/lib/Target/LLVMIR/DataLayoutImporter.h

68–71

I prefer keeping it private for now since I do not want others to use it. Instead users should query the imported data layout. We may still change this in the future if there happens to be a use case.

[gysit]

Move the string constant to the cpp.

mlir/include/mlir/Target/LLVMIR/ModuleImport.h

 class Instruction;
 class Value;
 } // namespace llvm
 
 namespace mlir {
 namespace LLVM {
 
 namespace detail {
+class DataLayoutImporter;
 class DebugImporter;
 class LoopAnnotationImporter;
 } // namespace detail
 
 /// Module import implementation class that provides methods to import globals
 /// and functions from an LLVM module into an MLIR module. It holds mappings
 /// between the original and translated globals, basic blocks, and values used
 /// during the translation. Additionally, it keeps track of the current constant
public:
   LogicalResult initializeImportInterface() { return iface.initializeImport(); }
 
   /// Converts all functions of the LLVM module to MLIR functions.
   LogicalResult convertFunctions();
 
   /// Converts all global variables of the LLVM module to MLIR global variables.
   LogicalResult convertGlobals();
 
+  /// Converts the data layout of the LLVM module to an MLIR data layout
+  /// specification.
+  LogicalResult convertDataLayout();
+
   /// Stores the mapping between an LLVM value and its MLIR counterpart.
   void mapValue(llvm::Value *llvm, Value mlir) { mapValue(llvm) = mlir; }
 
   /// Provides write-once access to store the MLIR value corresponding to the
   /// given LLVM value.
   Value &mapValue(llvm::Value *value) {
     Value &mlir = valueMapping[value];
     assert(mlir == nullptr &&

mlir/lib/Target/LLVMIR/CMakeLists.txt

 add_subdirectory(Dialect)
 
 set(LLVM_OPTIONAL_SOURCES
   ConvertFromLLVMIR.cpp
   ConvertToLLVMIR.cpp
+  DataLayoutImporter.cpp
   DebugTranslation.cpp
   DebugImporter.cpp
   LoopAnnotationImporter.cpp
   LoopAnnotationTranslation.cpp
   ModuleTranslation.cpp
   ModuleImport.cpp
   TypeToLLVM.cpp
   TypeFromLLVM.cpp
add_mlir_translation_library(MLIRToLLVMIRTranslationRegistration
   MLIRLLVMToLLVMIRTranslation
   MLIRNVVMToLLVMIRTranslation
   MLIROpenACCToLLVMIRTranslation
   MLIROpenMPToLLVMIRTranslation
   MLIRROCDLToLLVMIRTranslation
   )
 
 add_mlir_translation_library(MLIRTargetLLVMIRImport
+  DataLayoutImporter.cpp
   DebugImporter.cpp
   LoopAnnotationImporter.cpp
   ModuleImport.cpp
   TypeFromLLVM.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Target/LLVMIR
 

mlir/lib/Target/LLVMIR/DataLayoutImporter.h

+//===- DataLayoutImporter.h - LLVM to MLIR data layout conversion -*- C++ -*-=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the translation between the LLVMIR data layout and the
+// corresponding MLIR representation.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef MLIR_LIB_TARGET_LLVMIR_DATALAYOUTIMPORTER_H_
+#define MLIR_LIB_TARGET_LLVMIR_DATALAYOUTIMPORTER_H_
+
+#include "mlir/Dialect/LLVMIR/LLVMTypes.h"
+#include "mlir/IR/BuiltinAttributes.h"
+#include "mlir/Interfaces/DataLayoutInterfaces.h"
+
+namespace llvm {
+class StringRef;
+class DataLayout;
+} // namespace llvm
+
+namespace mlir {
+class FloatType;
+class MLIRContext;
+class Operation;
+
+namespace LLVM {
+class LLVMFuncOp;
+
+namespace detail {
+
+/// Returns a supported MLIR floating point type of the given bit width or
+/// null if the bit width is not supported.
+FloatType getFloatType(MLIRContext *context, unsigned width);
+
+/// Helper class that translates an LLVM data layout to an MLIR data layout
+/// specification. Only integer, float, pointer, alloca memory space, and
+/// endianness entries are translated. The class also returns all entries from
+/// the default data layout specification found in the language reference
+/// (https://llvm.org/docs/LangRef.html#data-layout) if they are not overwritten
+/// by the provided data layout.
+class DataLayoutImporter {
+public:
+  DataLayoutImporter(MLIRContext *context,
+                     const llvm::DataLayout &llvmDataLayout)
+      : context(context) {
+    translateDataLayout(llvmDataLayout);
+  }
+
+  /// Returns the MLIR data layout specification translated from the LLVM
+  /// data layout.
+  DataLayoutSpecInterface getDataLayout() const { return dataLayout; }
+
+  /// Returns the last data layout token that has been processed before
+  /// the data layout translation failed.
+  StringRef getLastToken() const { return lastToken; }
+
+  /// Returns the data layout tokens that have not been handled during the
+  /// data layout translation.
+  ArrayRef<StringRef> getUnhandledTokens() const { return unhandledTokens; }
+
+private:
+  /// Translates the LLVM `dataLayout` to an MLIR data layout specification.
+  void translateDataLayout(const llvm::DataLayout &llvmDataLayout);
+
+  /// Tries to parse the letter only prefix that identifies the specification
+  /// and removes the consumed characters from the beginning of the string.
+  FailureOr<StringRef> tryToParseAlphaPrefix(StringRef &token) const;
+
+  /// Tries to parse an integer parameter and removes the integer from the
+  /// beginning of the string.
+  FailureOr<unsigned> tryToParseInt(StringRef &token) const;
+
+  /// Tries to parse an integer parameter array.
+  FailureOr<SmallVector<unsigned>> tryToParseIntList(StringRef token) const;
+
+  /// Tries to parse the parameters of a type alignment entry.
+  FailureOr<DenseIntElementsAttr> tryToParseAlignment(StringRef token) const;
+
+  /// Tries to parse the parameters of a pointer alignment entry.
+  FailureOr<DenseIntElementsAttr>
+  tryToParsePointerAlignment(StringRef token) const;
+
+  /// Adds a type alignment entry if there is none yet.
+  LogicalResult tryToEmplaceAlignmentEntry(Type type, StringRef token);
+
+  /// Adds a pointer alignment entry if there is none yet.
+  LogicalResult tryToEmplacePointerAlignmentEntry(LLVMPointerType type,
+                                                  StringRef token);
+
+  /// Adds an endianness entry if there is none yet.
+  LogicalResult tryToEmplaceEndiannessEntry(StringRef endianness,
+                                            StringRef token);
+
+  /// Adds an alloca address space entry if there is none yet.
+  LogicalResult tryToEmplaceAllocaAddrSpaceEntry(StringRef token);
+
+  std::string layoutStr = {};
+  StringRef lastToken = {};
+  SmallVector<StringRef> unhandledTokens;
+  DenseMap<StringAttr, DataLayoutEntryInterface> keyEntries;
+  DenseMap<TypeAttr, DataLayoutEntryInterface> typeEntries;
+  MLIRContext *context;
+  DataLayoutSpecInterface dataLayout;
+};
+
+} // namespace detail
+} // namespace LLVM
+} // namespace mlir
+
+#endif // MLIR_LIB_TARGET_LLVMIR_DATALAYOUTIMPORTER_H_

mlir/lib/Target/LLVMIR/DataLayoutImporter.cpp

+//===- DataLayoutImporter.cpp - LLVM to MLIR data layout conversion -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "DataLayoutImporter.h"
+#include "mlir/Dialect/DLTI/DLTI.h"
+#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
+#include "mlir/IR/Builders.h"
+#include "mlir/IR/BuiltinAttributes.h"
+#include "mlir/IR/BuiltinTypes.h"
+#include "mlir/Interfaces/DataLayoutInterfaces.h"
+#include "mlir/Target/LLVMIR/Import.h"
+#include "llvm/IR/DataLayout.h"
+
+using namespace mlir;
+using namespace mlir::LLVM;
+using namespace mlir::LLVM::detail;
+
+/// The default data layout used during the translation.
+static constexpr StringRef kDefaultDataLayout =
+    "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-"
+    "f16:16:16-f64:64:64-f128:128:128";
+
+FloatType mlir::LLVM::detail::getFloatType(MLIRContext *context,
+                                           unsigned width) {
+  switch (width) {
+  case 16:
+    return FloatType::getF16(context);
+  case 32:
+    return FloatType::getF32(context);
+  case 64:
+    return FloatType::getF64(context);
+  case 80:
+    return FloatType::getF80(context);
+  case 128:
+    return FloatType::getF128(context);
+  default:
+    return {};
+  }
+}
+
+FailureOr<StringRef>
+DataLayoutImporter::tryToParseAlphaPrefix(StringRef &token) const {
+  if (token.empty())
+    return failure();
+
+  StringRef prefix = token.take_while(isalpha);
+  if (prefix.empty())
+    return failure();
+
+  token.consume_front(prefix);
+  return prefix;
+}
+
+FailureOr<unsigned> DataLayoutImporter::tryToParseInt(StringRef &token) const {
+  unsigned parameter;
+  if (token.consumeInteger(/*Radix=*/10, parameter))
+    return failure();
+  return parameter;
+}
+
+FailureOr<SmallVector<unsigned>>
+DataLayoutImporter::tryToParseIntList(StringRef token) const {
+  SmallVector<StringRef> tokens;
+  token.consume_front(":");
+  token.split(tokens, ':');
+
+  // Parse an integer list.
+  SmallVector<unsigned> results(tokens.size());
+  for (auto [result, token] : llvm::zip(results, tokens))
+    if (token.getAsInteger(/*Radix=*/10, result))
+      return failure();
+  return results;
+}
+
+FailureOr<DenseIntElementsAttr>
+DataLayoutImporter::tryToParseAlignment(StringRef token) const {
+  FailureOr<SmallVector<unsigned>> alignment = tryToParseIntList(token);
+  if (failed(alignment))
+    return failure();
+  if (alignment->empty() || alignment->size() > 2)
+    return failure();
+
+  // Alignment specifications (such as 32 or 32:64) are of the
+  // form <abi>[:<pref>], where abi specifies the minimal alignment and pref the
+  // optional preferred alignment. The preferred alignment is set to the minimal
+  // alignment if not available.
+  unsigned minimal = (*alignment)[0];
+  unsigned preferred = alignment->size() == 1 ? minimal : (*alignment)[1];
+  return DenseIntElementsAttr::get(
+      VectorType::get({2}, IntegerType::get(context, 32)),
+      {minimal, preferred});
+}
+
+FailureOr<DenseIntElementsAttr>
+DataLayoutImporter::tryToParsePointerAlignment(StringRef token) const {
+  FailureOr<SmallVector<unsigned>> alignment = tryToParseIntList(token);
+  if (failed(alignment))
+    return failure();
+  if (alignment->size() < 2 || alignment->size() > 4)
+    return failure();
+
+  // Pointer alignment specifications (such as 64:32:64:32 or 32:32) are of
+  // the form <size>:<abi>[:<pref>][:<idx>], where size is the pointer size, abi
+  // specifies the minimal alignment, pref the optional preferred alignment, and
+  // idx the optional index computation bit width. The preferred alignment is
+  // set to the minimal alignment if not available and the index computation
+  // width is set to the pointer size if not available.
+  unsigned size = (*alignment)[0];
+  unsigned minimal = (*alignment)[1];
+  unsigned preferred = alignment->size() < 3 ? minimal : (*alignment)[2];
+  unsigned idx = alignment->size() < 4 ? size : (*alignment)[3];
+  return DenseIntElementsAttr::get(
+      VectorType::get({4}, IntegerType::get(context, 32)),
+      {size, minimal, preferred, idx});
+}
+
+LogicalResult DataLayoutImporter::tryToEmplaceAlignmentEntry(Type type,
+                                                             StringRef token) {
+  auto key = TypeAttr::get(type);
+  if (typeEntries.count(key))
+    return success();
+
+  FailureOr<DenseIntElementsAttr> params = tryToParseAlignment(token);
+  if (failed(params))
+    return failure();
+
+  typeEntries.try_emplace(key, DataLayoutEntryAttr::get(type, *params));
+  return success();
+}
+
+LogicalResult
+DataLayoutImporter::tryToEmplacePointerAlignmentEntry(LLVMPointerType type,
+                                                      StringRef token) {
+  auto key = TypeAttr::get(type);
+  if (typeEntries.count(key))
+    return success();
+
+  FailureOr<DenseIntElementsAttr> params = tryToParsePointerAlignment(token);
+  if (failed(params))
+    return failure();
+
+  typeEntries.try_emplace(key, DataLayoutEntryAttr::get(type, *params));
+  return success();
+}
+
+LogicalResult
+DataLayoutImporter::tryToEmplaceEndiannessEntry(StringRef endianness,
+                                                StringRef token) {
+  auto key = StringAttr::get(context, DLTIDialect::kDataLayoutEndiannessKey);
+  if (keyEntries.count(key))
+    return success();
+
+  if (!token.empty())
+    return failure();
+
+  keyEntries.try_emplace(
+      key, DataLayoutEntryAttr::get(key, StringAttr::get(context, endianness)));
+  return success();
+}
+
+LogicalResult
+DataLayoutImporter::tryToEmplaceAllocaAddrSpaceEntry(StringRef token) {
+  auto key =
+      StringAttr::get(context, DLTIDialect::kDataLayoutAllocaMemorySpaceKey);
+  if (keyEntries.count(key))
+    return success();
+
+  FailureOr<unsigned> space = tryToParseInt(token);
+  if (failed(space))
+    return failure();
+
+  // Only store the address space if it has a non-default value.
+  if (*space == 0)
+    return success();
+  OpBuilder builder(context);
+  keyEntries.try_emplace(
+      key, DataLayoutEntryAttr::get(key, builder.getUI32IntegerAttr(*space)));
+  return success();
+}
+
+void DataLayoutImporter::translateDataLayout(
+    const llvm::DataLayout &llvmDataLayout) {
+  dataLayout = {};
+
+  // Transform the data layout to its string representation and append the
+  // default data layout string specified in the language reference
+  // (https://llvm.org/docs/LangRef.html#data-layout). The translation then
+  // parses the string and ignores the default value if a specific kind occurs
+  // in both strings. Additionally, the following default values exist:
+  // - non-default address space pointer specifications default to the default
+  //   address space pointer specification
+  // - the alloca address space defaults to the default address space.
+  layoutStr = llvmDataLayout.getStringRepresentation();
+  if (!layoutStr.empty())
+    layoutStr += "-";
+  layoutStr += kDefaultDataLayout;
+  StringRef layout(layoutStr);
+
+  // Split the data layout string into tokens separated by a dash.
+  SmallVector<StringRef> tokens;
+  layout.split(tokens, '-');
+
+  for (StringRef token : tokens) {
+    lastToken = token;
+    FailureOr<StringRef> prefix = tryToParseAlphaPrefix(token);
+    if (failed(prefix))
+      return;
+
+    // Parse the endianness.
+    if (*prefix == "e") {
+      if (failed(tryToEmplaceEndiannessEntry(
+              DLTIDialect::kDataLayoutEndiannessLittle, token)))
+        return;
+      continue;
+    }
+    if (*prefix == "E") {
+      if (failed(tryToEmplaceEndiannessEntry(
+              DLTIDialect::kDataLayoutEndiannessBig, token)))
+        return;
+      continue;
+    }
+    // Parse the alloca address space.
+    if (*prefix == "A") {
+      if (failed(tryToEmplaceAllocaAddrSpaceEntry(token)))
+        return;
+      continue;
+    }
+    // Parse integer alignment specifications.
+    if (*prefix == "i") {
+      FailureOr<unsigned> width = tryToParseInt(token);
+      if (failed(width))
+        return;
+
+      Type type = IntegerType::get(context, *width);
+      if (failed(tryToEmplaceAlignmentEntry(type, token)))
+        return;
+      continue;
+    }
+    // Parse float alignment specifications.
+    if (*prefix == "f") {
+      FailureOr<unsigned> width = tryToParseInt(token);
+      if (failed(width))
+        return;
+
+      Type type = getFloatType(context, *width);
+      if (failed(tryToEmplaceAlignmentEntry(type, token)))
+        return;
+      continue;
+    }
+    // Parse pointer alignment specifications.
+    if (*prefix == "p") {
+      FailureOr<unsigned> space =
+          token.starts_with(":") ? 0 : tryToParseInt(token);
+      if (failed(space))
+        return;
+
+      auto type = LLVMPointerType::get(context, *space);
+      if (failed(tryToEmplacePointerAlignmentEntry(type, token)))
+        return;
+      continue;
+    }
+
+    // Store all tokens that have not been handled.
+    unhandledTokens.push_back(lastToken);
+  }
+
+  // Assemble all entries to a data layout specification.
+  SmallVector<DataLayoutEntryInterface> entries;
+  entries.reserve(typeEntries.size() + keyEntries.size());
+  for (const auto &it : typeEntries)
+    entries.push_back(it.second);
+  for (const auto &it : keyEntries)
+    entries.push_back(it.second);
+  dataLayout = DataLayoutSpecAttr::get(context, entries);
+}
+
+DataLayoutSpecInterface
+mlir::translateDataLayout(const llvm::DataLayout &dataLayout,
+                          MLIRContext *context) {
+  return DataLayoutImporter(context, dataLayout).getDataLayout();
+}

mlir/lib/Target/LLVMIR/DebugImporter.h

 //===- DebugImporter.h - LLVM to MLIR Debug conversion -------*- C++ -*----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the translation between LLVMIR debug information and
 // the corresponding MLIR representation.
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef MLIR_LIB_TARGET_LLVMIR_DEBUGIMPORT_H_
-#define MLIR_LIB_TARGET_LLVMIR_DEBUGIMPORT_H_
+#ifndef MLIR_LIB_TARGET_LLVMIR_DEBUGIMPORTER_H_
+#define MLIR_LIB_TARGET_LLVMIR_DEBUGIMPORTER_H_
 
 #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/MLIRContext.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 
 namespace mlir {
 class Operation;
private:
   MLIRContext *context;
   ModuleOp mlirModule;
 };
 
 } // namespace detail
 } // namespace LLVM
 } // namespace mlir
 
-#endif // MLIR_LIB_TARGET_LLVMIR_DEBUIMPORTN_H_
+#endif // MLIR_LIB_TARGET_LLVMIR_DEBUGIMPORTER_H_

mlir/lib/Target/LLVMIR/ModuleImport.cpp

 // module.
 //
 //===----------------------------------------------------------------------===//
 
 #include "mlir/Target/LLVMIR/ModuleImport.h"
 #include "mlir/Target/LLVMIR/Import.h"
 
 #include "AttrKindDetail.h"
+#include "DataLayoutImporter.h"
 #include "DebugImporter.h"
 #include "LoopAnnotationImporter.h"
 
 #include "mlir/Dialect/DLTI/DLTI.h"
 #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
 #include "mlir/IR/Builders.h"
 #include "mlir/IR/Matchers.h"
 #include "mlir/Interfaces/DataLayoutInterfaces.h"

 }
 
 /// Returns the symbol name for the module-level metadata operation. It must not
 /// conflict with the user namespace.
 static constexpr StringRef getGlobalMetadataOpName() {
   return "__llvm_global_metadata";
 }
 
-/// Returns a supported MLIR floating point type of the given bit width or null
-/// if the bit width is not supported.
-static FloatType getDLFloatType(MLIRContext &ctx, int32_t bitwidth) {
-  switch (bitwidth) {
-  case 16:
-    return FloatType::getF16(&ctx);
-  case 32:
-    return FloatType::getF32(&ctx);
-  case 64:
-    return FloatType::getF64(&ctx);
-  case 80:
-    return FloatType::getF80(&ctx);
-  case 128:
-    return FloatType::getF128(&ctx);
-  default:
-    return nullptr;
-  }
-}
-
 /// Converts the sync scope identifier of `inst` to the string representation
 /// necessary to build an atomic LLVM dialect operation. Returns the empty
 /// string if the operation has either no sync scope or the default system-level
 /// sync scope attached. The atomic operations only set their sync scope
 /// attribute if they have a non-default sync scope attached.
 static StringRef getLLVMSyncScope(llvm::Instruction *inst) {
   std::optional<llvm::SyncScope::ID> syncScopeID =
       llvm::getAtomicSyncScopeID(inst);
static LogicalResult convertInstructionImpl(OpBuilder &odsBuilder,
   SmallVector<llvm::Value *> operands(inst->operands());
   ArrayRef<llvm::Value *> llvmOperands(operands);
 
   // Convert all instructions that provide an MLIR builder.
 #include "mlir/Dialect/LLVMIR/LLVMOpFromLLVMIRConversions.inc"
   return failure();
 }
 
-/// Creates an attribute containing ABI and preferred alignment numbers parsed
-/// a string. The string may be either "abi:preferred" or just "abi". In the
-/// latter case, the preferred alignment is considered equal to ABI alignment.
-static DenseIntElementsAttr parseDataLayoutAlignment(MLIRContext &ctx,
-                                                     StringRef spec) {
-  auto i32 = IntegerType::get(&ctx, 32);
-
-  StringRef abiString, preferredString;
-  std::tie(abiString, preferredString) = spec.split(':');
-  int abi, preferred;
-  if (abiString.getAsInteger(/*Radix=*/10, abi))
-    return nullptr;
-
-  if (preferredString.empty())
-    preferred = abi;
-  else if (preferredString.getAsInteger(/*Radix=*/10, preferred))
-    return nullptr;
-
-  return DenseIntElementsAttr::get(VectorType::get({2}, i32), {abi, preferred});
-}
-
-/// Translate the given LLVM data layout into an MLIR equivalent using the DLTI
-/// dialect.
-DataLayoutSpecInterface
-mlir::translateDataLayout(const llvm::DataLayout &dataLayout,
-                          MLIRContext *context) {
-  assert(context && "expected MLIR context");
-  std::string layoutstr = dataLayout.getStringRepresentation();
-
-  // Remaining unhandled default layout defaults
-  // e (little endian if not set)
-  // p[n]:64:64:64 (non zero address spaces have 64-bit properties)
-  // Alloca address space defaults to 0.
-  std::string append =
-      "p:64:64:64-S0-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f16:16:16-f64:"
-      "64:64-f128:128:128-v64:64:64-v128:128:128-a:0:64-A0";
-  if (layoutstr.empty())
-    layoutstr = append;
-  else
-    layoutstr = layoutstr + "-" + append;
-
-  StringRef layout(layoutstr);
-
-  SmallVector<DataLayoutEntryInterface> entries;
-  StringSet<> seen;
-  while (!layout.empty()) {
-    // Split at '-'.
-    std::pair<StringRef, StringRef> split = layout.split('-');
-    StringRef current;
-    std::tie(current, layout) = split;
-
-    // Split at ':'.
-    StringRef kind, spec;
-    std::tie(kind, spec) = current.split(':');
-    if (seen.contains(kind))
-      continue;
-    seen.insert(kind);
-
-    char symbol = kind.front();
-    StringRef parameter = kind.substr(1);
-
-    if (symbol == 'i' || symbol == 'f') {
-      unsigned bitwidth;
-      if (parameter.getAsInteger(/*Radix=*/10, bitwidth))
-        return nullptr;
-      DenseIntElementsAttr params = parseDataLayoutAlignment(*context, spec);
-      if (!params)
-        return nullptr;
-      auto entry = DataLayoutEntryAttr::get(
-          symbol == 'i' ? static_cast<Type>(IntegerType::get(context, bitwidth))
-                        : getDLFloatType(*context, bitwidth),
-          params);
-      entries.emplace_back(entry);
-    } else if (symbol == 'e' || symbol == 'E') {
-      auto value = StringAttr::get(
-          context, symbol == 'e' ? DLTIDialect::kDataLayoutEndiannessLittle
-                                 : DLTIDialect::kDataLayoutEndiannessBig);
-      auto entry = DataLayoutEntryAttr::get(
-          StringAttr::get(context, DLTIDialect::kDataLayoutEndiannessKey),
-          value);
-      entries.emplace_back(entry);
-    } else if (symbol == 'A') {
-      unsigned addressSpace;
-      if (parameter.getAsInteger(/*Radix=*/10, addressSpace))
-        return nullptr;
-      // Skip storing if generic address space is defined.
-      if (addressSpace != 0) {
-        auto entry = DataLayoutEntryAttr::get(
-            StringAttr::get(context,
-                            DLTIDialect::kDataLayoutAllocaMemorySpaceKey),
-            mlir::Builder(context).getUI32IntegerAttr(addressSpace));
-        entries.emplace_back(entry);
-      }
-    }
-  }
-
-  return DataLayoutSpecAttr::get(context, entries);
-}
-
 /// Get a topologically sorted list of blocks for the given function.
 static SetVector<llvm::BasicBlock *>
 getTopologicallySortedBlocks(llvm::Function *func) {
   SetVector<llvm::BasicBlock *> blocks;
   for (llvm::BasicBlock &bb : *func) {
     if (blocks.count(&bb) == 0) {
       llvm::ReversePostOrderTraversal<llvm::BasicBlock *> traversal(&bb);
       blocks.insert(traversal.begin(), traversal.end());
for (llvm::GlobalVariable &globalVar : llvmModule->globals()) {
     if (failed(convertGlobal(&globalVar))) {
       return emitError(mlirModule.getLoc())
              << "unhandled global variable: " << diag(globalVar);
     }
   }
   return success();
 }
 
+LogicalResult ModuleImport::convertDataLayout() {
+  Location loc = mlirModule.getLoc();
+  DataLayoutImporter dataLayoutImporter(context, llvmModule->getDataLayout());
+  if (!dataLayoutImporter.getDataLayout())
+    return emitError(loc, "cannot translate data layout: ")
+           << dataLayoutImporter.getLastToken();
+
+  for (StringRef token : dataLayoutImporter.getUnhandledTokens())
+    emitWarning(loc, "unhandled data layout token: ") << token;
+
+  mlirModule->setAttr(DLTIDialect::kDataLayoutAttrName,
+                      dataLayoutImporter.getDataLayout());
+  return success();
+}
+
 LogicalResult ModuleImport::convertFunctions() {
   for (llvm::Function &func : llvmModule->functions())
     if (failed(processFunction(&func)))
       return failure();
   return success();
 }
 
 void ModuleImport::setNonDebugMetadataAttrs(llvm::Instruction *inst,
if (auto *c = dyn_cast<llvm::ConstantDataArray>(value))
     if (c->isString())
       return builder.getStringAttr(c->getAsString());
   if (auto *c = dyn_cast<llvm::ConstantFP>(value)) {
     llvm::Type *type = c->getType();
     FloatType floatTy;
     if (type->isBFloatTy())
       floatTy = FloatType::getBF16(context);
     else
-      floatTy = getDLFloatType(*context, type->getScalarSizeInBits());
+      floatTy = detail::getFloatType(context, type->getScalarSizeInBits());
     assert(floatTy && "unsupported floating point type");
     return builder.getFloatAttr(floatTy, c->getValueAPF());
   }
   if (auto *f = dyn_cast<llvm::Function>(value))
     return SymbolRefAttr::get(builder.getContext(), f->getName());
 
   // Convert constant data to a dense elements attribute.
   if (auto *cd = dyn_cast<llvm::ConstantDataSequential>(value)) {
mlir::translateLLVMIRToModule(std::unique_ptr<llvm::Module> llvmModule,
   assert(llvm::is_contained(context->getAvailableDialects(),
                             DLTIDialect::getDialectNamespace()));
   context->loadAllAvailableDialects();
 
   OwningOpRef<ModuleOp> module(ModuleOp::create(FileLineColLoc::get(
       StringAttr::get(context, llvmModule->getSourceFileName()), /*line=*/0,
       /*column=*/0)));
 
-  DataLayoutSpecInterface dlSpec =
-      translateDataLayout(llvmModule->getDataLayout(), context);
-  if (!dlSpec) {
-    emitError(UnknownLoc::get(context), "can't translate data layout");
-    return {};
-  }
-  module.get()->setAttr(DLTIDialect::kDataLayoutAttrName, dlSpec);
-
   ModuleImport moduleImport(module.get(), std::move(llvmModule));
   if (failed(moduleImport.initializeImportInterface()))
     return {};
+  if (failed(moduleImport.convertDataLayout()))
+    return {};
   if (failed(moduleImport.convertMetadata()))
     return {};
   if (failed(moduleImport.convertGlobals()))
     return {};
   if (failed(moduleImport.convertFunctions()))
     return {};
 
   return module;
 }

mlir/test/Dialect/LLVMIR/layout.mlir

 // RUN: mlir-opt --test-data-layout-query --split-input-file --verify-diagnostics %s | FileCheck %s
 
 module {
   // CHECK: @no_spec
   func.func @no_spec() {
+    // CHECK: alignment = 8
+    // CHECK: alloca_memory_space = 0
+    // CHECK: bitsize = 64
+    // CHECK: preferred = 8
+    // CHECK: size = 8
     "test.data_layout_query"() : () -> !llvm.ptr
     // CHECK: alignment = 8
     // CHECK: alloca_memory_space = 0
     // CHECK: bitsize = 64
     // CHECK: preferred = 8
     // CHECK: size = 8
     "test.data_layout_query"() : () -> !llvm.ptr<3>
     // CHECK: alignment = 8

mlir/test/Target/LLVMIR/Import/data-layout.ll

-; RUN: mlir-translate -import-llvm %s | FileCheck %s
+; RUN: mlir-translate -import-llvm -split-input-file %s | FileCheck %s
+
+; Test the default data layout import.
+
+; CHECK: dlti.dl_spec =
+; CHECK: #dlti.dl_spec<
+; CHECK-DAG:   #dlti.dl_entry<"dlti.endianness", "little">
+; CHECK-DAG:   #dlti.dl_entry<i1, dense<8> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<i8, dense<8> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<i16, dense<16> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<i32, dense<32> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<i64, dense<[32, 64]> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<!llvm.ptr, dense<64> : vector<4xi32>>
+; CHECK-DAG:   #dlti.dl_entry<f16, dense<16> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<f64, dense<64> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<f128, dense<128> : vector<2xi32>>
+; CHECK: >
+target datalayout = ""
+
+; // -----
+
+; CHECK: dlti.dl_spec =
+; CHECK: #dlti.dl_spec<
+; CHECK-DAG:   #dlti.dl_entry<"dlti.endianness", "little">
+; CHECK-DAG:   #dlti.dl_entry<i64, dense<64> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<f80, dense<128> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<i8, dense<8> : vector<2xi32>>
+; CHECK-DAG:   #dlti.dl_entry<!llvm.ptr<270>, dense<[32, 64, 64, 32]> : vector<4xi32>>
+; CHECK-DAG:   #dlti.dl_entry<!llvm.ptr<271>, dense<32> : vector<4xi32>>
+; CHECK-DAG:   #dlti.dl_entry<!llvm.ptr<272>, dense<64> : vector<4xi32>>
+target datalayout = "e-m:e-p270:32:64-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+
+; // -----
 
-; CHECK: dlti.dl_spec = 
+; CHECK: dlti.dl_spec =
 ; CHECK: #dlti.dl_spec<
-; CHECK:   #dlti.dl_entry<"dlti.endianness", "little">
-; CHECK:   #dlti.dl_entry<i64, dense<64> : vector<2xi32>>
-; CHECK:   #dlti.dl_entry<f80, dense<128> : vector<2xi32>>
-; CHECK:   #dlti.dl_entry<i8, dense<8> : vector<2xi32>>
-target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+; CHECK-DAG:   #dlti.dl_entry<"dlti.endianness", "big">
+; CHECK-DAG:   #dlti.dl_entry<!llvm.ptr<270>, dense<[16, 32, 64, 128]> : vector<4xi32>>
+; CHECK-DAG:   #dlti.dl_entry<"dlti.alloca_memory_space", 1 : ui32>
+; CHECK-DAG:   #dlti.dl_entry<i64, dense<[64, 128]> : vector<2xi32>>
+target datalayout = "E-p270:16:32:64:128-A1-i64:64:128"
 
-declare void @foo()
+; // -----
+
+; CHECK: dlti.dl_spec =
+; CHECK: #dlti.dl_spec<
+; CHECK-NOT:   #dlti.dl_entry<"dlti.alloca_memory_space"
+target datalayout = "E-A0"

mlir/test/Target/LLVMIR/Import/import-failure.ll

define void @unused_scope() {
   ret void
 }
 
 declare void @llvm.experimental.noalias.scope.decl(metadata)
 
 !0 = !{!1}
 !1 = !{!1, !2}
 !2 = distinct !{!2, !"The domain"}
+
+; // -----
+
+; CHECK:      import-failure.ll
+; CHECK-SAME: error: cannot translate data layout: i8:8:8:8
+target datalayout = "e-i8:8:8:8"
+
+; // -----
+
+; CHECK:      import-failure.ll
+; CHECK-SAME: warning: unhandled data layout token: ni:42
+target datalayout = "e-ni:42-i64:64"