This is an archive of the discontinued LLVM Phabricator instance.

[mlir][llvm] Delete LLVMIntrinsicGen.cpp.
AbandonedPublic

Authored by gysit on Mar 1 2023, 12:09 AM.

Download Raw Diff

Details

Reviewers

ftynse
frgossen
hgreving

Summary

LLVMIntrinsicGen.cpp translates the LLVM intrinsic definitions to MLIR
intrinsic definitions. This automatic translations does not seem to
be in use anymore. Instead, intrinsics are defined manually in
LLVMIntrinsicOps.td.

This revision deletes LLVMIntrinsicGen.cpp and the associated test.

Depends on D144965

Diff Detail

Repository: rG LLVM Github Monorepo

Event Timeline

gysit created this revision.Mar 1 2023, 12:09 AM

Herald added a reviewer: ftynse. · View Herald TranscriptMar 1 2023, 12:09 AM

Herald added a project: Restricted Project. · View Herald Transcript

Herald added subscribers: Moerafaat, zero9178, bzcheeseman and 23 others. · View Herald Transcript

gysit requested review of this revision.Mar 1 2023, 12:09 AM

Herald added a project: Restricted Project. · View Herald TranscriptMar 1 2023, 12:09 AM

Herald added subscribers: stephenneuendorffer, nicolasvasilache, jdoerfert. · View Herald Transcript

Let me know if this is still used somewhere? The upstream intrinsic definitions have quite a bit of additional structuring (e.g. base classes) by now and I do not think they can be generated from LLVM directly anymore.

Harbormaster completed remote builds in B216654: Diff 501393.Mar 1 2023, 12:33 AM

Is this compatible with dowstream targets defining their own intrinsics outside of Intrinics.td (in LLVM)?

Is this compatible with dowstream targets defining their own intrinsics outside of Intrinics.td (in LLVM)?

@hgreving Downstream projects can copy the script and continue using it? Admittedly, I do not have full context but I am happy to hear why copying is not possible?

At the moment, the script is unused upstream and detached from everything else. In particular, the script is not used to generate any intrinsics and therefore the intrinsics cannot be compiled and tested (export, import, roundtrip). One way forward may be to add a use case for the script, e.g. for one of the upstream targets (ArmNEON, ArmSVE, AMX, etc), and then implement the aforementioned tests. Otherwise, I favor moving the script to the downstream project where it can be properly maintained and tested.

Let me know if you have a good upstream use case or plan to work on one!

I don't see why this needs to be deleted. It was used to generate the initial ODS for some intrinsics and is still useful for adding new target-specific ones. It is not used in the build process because the complexity of adding a stage to the build process does not justify it.

This revision now requires changes to proceed.Mar 1 2023, 11:36 AM

Note that many other tools are not directly "used" upstream, mlir-reduce, mlir-parser-fuzzer, etc. They are nevertheless useless.

Ok, I did not think of the additional stage needed to test this. It would have been great to have such a test since the tool needs to be adapted if the intrinsic base class changes.

We can have test for this, but we probably don't want this to be on the _main_ build path.

I think something that can be called manually would also be great. E.g. to somehow test if the changes I did in the base revision really work. That only helps if one is aware of the tool though.

OpDSL has such an additional build step (the yaml is translated to tablegen during the normal build if I remember correctly). It is very complex though and very much overkill for this I guess.

Revision Contents

Path

Size

mlir/

test/

mlir-tblgen/

llvm-intrinsics.td

tools/

mlir-tblgen/

CMakeLists.txt

1 line

LLVMIRIntrinsicGen.cpp

Diff 501393

mlir/test/mlir-tblgen/llvm-intrinsics.td

This file was deleted.

	// This checks that we can consume LLVM's Intrinsic definitions from TableGen
	// files and produce ODS. Unlike MLIR, LLVM's main Intrinsics.td file that
	// contains the definition of the Intrinsic class also includes files for
	// platform-specific intrinsics, so we need to give it to TableGen instead of
	// writing a local test source. We filter out platform-specific intrinsic
	// includes from the main file to avoid unnecessary dependencies and decrease
	// the test cost. The command-line flags further ensure a specific intrinsic is
	// processed and we only check the output below.
	// We also verify emission of type specialization for overloadable intrinsics.
	//
	// RUN: cat %S/../../../llvm/include/llvm/IR/Intrinsics.td \
	// RUN: \| grep -v "llvm/IR/Intrinsics" \
	// RUN: \| mlir-tblgen -gen-llvmir-intrinsics -I %S/../../../llvm/include/ --llvmir-intrinsics-filter=ptrmask \
	// RUN: \| FileCheck %s

	// CHECK-LABEL: def LLVM_ptrmask
	// CHECK: LLVM_IntrOp<"ptrmask
	// The result of this intrinsic result is overloadable.
	// CHECK: [0]
	// The second operand is overloadable, but the first operand needs to
	// match the result type.
	// CHECK: [1]
	// It has no side effects.
	// CHECK: [NoMemoryEffect]
	// It has a result.
	// CHECK: 1,
	// It does not require an access group.
	// CHECK: 0,
	// It does not require alias scopes.
	// CHECK: 0>
	// CHECK: Arguments<(ins LLVM_Type, LLVM_Type

	//---------------------------------------------------------------------------//

	// This checks that we can define an op that takes in an access group metadata.
	//
	// RUN: cat %S/../../../llvm/include/llvm/IR/Intrinsics.td \
	// RUN: \| grep -v "llvm/IR/Intrinsics" \
	// RUN: \| mlir-tblgen -gen-llvmir-intrinsics -I %S/../../../llvm/include/ --llvmir-intrinsics-filter=ptrmask --llvmir-intrinsics-access-group-regexp=ptrmask \
	// RUN: \| FileCheck --check-prefix=GROUPS %s

	// GROUPS-LABEL: def LLVM_ptrmask
	// GROUPS: LLVM_IntrOp<"ptrmask
	// It has no side effects.
	// GROUPS: [NoMemoryEffect]
	// It has a result.
	// GROUPS: 1,
	// It requires generation of an access group LLVM metadata.
	// GROUPS: 1,
	// It does not require alias scopes.
	// GROUPS: 0>
	// It has an access group attribute.
	// GROUPS: OptionalAttr<SymbolRefArrayAttr>:$access_groups

	//---------------------------------------------------------------------------//

	// This checks that we can define an op that takes in alias scopes metadata.
	//
	// RUN: cat %S/../../../llvm/include/llvm/IR/Intrinsics.td \
	// RUN: \| grep -v "llvm/IR/Intrinsics" \
	// RUN: \| mlir-tblgen -gen-llvmir-intrinsics -I %S/../../../llvm/include/ --llvmir-intrinsics-filter=ptrmask --llvmir-intrinsics-alias-scopes-regexp=ptrmask \
	// RUN: \| FileCheck --check-prefix=ALIAS %s

	// ALIAS-LABEL: def LLVM_ptrmask
	// ALIAS: LLVM_IntrOp<"ptrmask
	// It has no side effects.
	// ALIAS: [NoMemoryEffect]
	// It has a result.
	// ALIAS: 1,
	// It does not require an access group.
	// ALIAS: 0,
	// It requires generation of alias scopes LLVM metadata.
	// ALIAS: 1>
	// It has alias scopes and noalias.
	// ALIAS: OptionalAttr<SymbolRefArrayAttr>:$alias_scopes
	// ALIAS: OptionalAttr<SymbolRefArrayAttr>:$noalias_scopes
	// ALIAS: OptionalAttr<SymbolRefArrayAttr>:$tbaa

	//---------------------------------------------------------------------------//

	// This checks that the ODS we produce can be consumed by MLIR tablegen. We only
	// make sure the entire process does not fail and produces some C++. The shape
	// of this C++ code is tested by ODS tests.

	// RUN: cat %S/../../../llvm/include/llvm/IR/Intrinsics.td \
	// RUN: \| grep -v "llvm/IR/Intrinsics" \
	// RUN: \| mlir-tblgen -gen-llvmir-intrinsics -I %S/../../../llvm/include/ --llvmir-intrinsics-filter=vastart \
	// RUN: \| mlir-tblgen -gen-op-decls -I %S/../../include \
	// RUN: \| FileCheck --check-prefix=ODS %s

	// ODS-LABEL: class vastart

	// RUN: cat %S/../../../llvm/include/llvm/IR/Intrinsics.td \
	// RUN: \| grep -v "llvm/IR/Intrinsics" \
	// RUN: \| mlir-tblgen -gen-llvmir-intrinsics -I %S/../../../llvm/include/ --llvmir-intrinsics-filter=ptrmask -dialect-opclass-base My_OpBase \
	// RUN: \| FileCheck %s --check-prefix=DIALECT-OPBASE

	// DIALECT-OPBASE-LABEL: def LLVM_ptrmask
	// DIALECT-OPBASE: My_OpBase<"ptrmask

mlir/tools/mlir-tblgen/CMakeLists.txt

	set(LLVM_LINK_COMPONENTS			set(LLVM_LINK_COMPONENTS
	Demangle			Demangle
	Support			Support
	TableGen			TableGen
	)			)

	add_tablegen(mlir-tblgen MLIR			add_tablegen(mlir-tblgen MLIR
	DESTINATION "${MLIR_TOOLS_INSTALL_DIR}"			DESTINATION "${MLIR_TOOLS_INSTALL_DIR}"
	EXPORT MLIR			EXPORT MLIR
	AttrOrTypeDefGen.cpp			AttrOrTypeDefGen.cpp
	AttrOrTypeFormatGen.cpp			AttrOrTypeFormatGen.cpp
	DialectGen.cpp			DialectGen.cpp
	DirectiveCommonGen.cpp			DirectiveCommonGen.cpp
	EnumsGen.cpp			EnumsGen.cpp
	FormatGen.cpp			FormatGen.cpp
	LLVMIRConversionGen.cpp			LLVMIRConversionGen.cpp
	LLVMIRIntrinsicGen.cpp
	mlir-tblgen.cpp			mlir-tblgen.cpp
	OpClass.cpp			OpClass.cpp
	OpDefinitionsGen.cpp			OpDefinitionsGen.cpp
	OpDocGen.cpp			OpDocGen.cpp
	OpFormatGen.cpp			OpFormatGen.cpp
	OpGenHelpers.cpp			OpGenHelpers.cpp
	OpInterfacesGen.cpp			OpInterfacesGen.cpp
	OpPythonBindingGen.cpp			OpPythonBindingGen.cpp
	Show All 14 Lines

mlir/tools/mlir-tblgen/LLVMIRIntrinsicGen.cpp

This file was deleted.

	//===- LLVMIntrinsicGen.cpp - TableGen utility for converting intrinsics --===//
	//
	// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
	// See https://llvm.org/LICENSE.txt for license information.
	// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
	//
	//===----------------------------------------------------------------------===//
	//
	// This is a TableGen generator that converts TableGen definitions for LLVM
	// intrinsics to TableGen definitions for MLIR operations.
	//
	//===----------------------------------------------------------------------===//

	#include "mlir/TableGen/GenInfo.h"

	#include "llvm/ADT/SmallBitVector.h"
	#include "llvm/Support/CommandLine.h"
	#include "llvm/Support/MachineValueType.h"
	#include "llvm/Support/PrettyStackTrace.h"
	#include "llvm/Support/Regex.h"
	#include "llvm/Support/Signals.h"
	#include "llvm/TableGen/Error.h"
	#include "llvm/TableGen/Main.h"
	#include "llvm/TableGen/Record.h"
	#include "llvm/TableGen/TableGenBackend.h"

	static llvm::cl::OptionCategory intrinsicGenCat("Intrinsics Generator Options");

	static llvm::cl::opt<std::string>
	nameFilter("llvmir-intrinsics-filter",
	llvm::cl::desc("Only keep the intrinsics with the specified "
	"substring in their record name"),
	llvm::cl::cat(intrinsicGenCat));

	static llvm::cl::opt<std::string>
	opBaseClass("dialect-opclass-base",
	llvm::cl::desc("The base class for the ops in the dialect we "
	"are planning to emit"),
	llvm::cl::init("LLVM_IntrOp"), llvm::cl::cat(intrinsicGenCat));

	static llvm::cl::opt<std::string> accessGroupRegexp(
	"llvmir-intrinsics-access-group-regexp",
	llvm::cl::desc("Mark intrinsics that match the specified "
	"regexp as taking an access group metadata"),
	llvm::cl::cat(intrinsicGenCat));

	static llvm::cl::opt<std::string> aliasScopesRegexp(
	"llvmir-intrinsics-alias-scopes-regexp",
	llvm::cl::desc("Mark intrinsics that match the specified "
	"regexp as taking alias.scopes and noalias metadata"),
	llvm::cl::cat(intrinsicGenCat));

	// Used to represent the indices of overloadable operands/results.
	using IndicesTy = llvm::SmallBitVector;

	/// Return a CodeGen value type entry from a type record.
	static llvm::MVT::SimpleValueType getValueType(const llvm::Record *rec) {
	return (llvm::MVT::SimpleValueType)rec->getValueAsDef("VT")->getValueAsInt(
	"Value");
	}

	/// Return the indices of the definitions in a list of definitions that
	/// represent overloadable types
	static IndicesTy getOverloadableTypeIdxs(const llvm::Record &record,
	const char *listName) {
	auto results = record.getValueAsListOfDefs(listName);
	IndicesTy overloadedOps(results.size());
	for (const auto &r : llvm::enumerate(results)) {
	llvm::MVT::SimpleValueType vt = getValueType(r.value());
	switch (vt) {
	case llvm::MVT::iAny:
	case llvm::MVT::fAny:
	case llvm::MVT::Any:
	case llvm::MVT::iPTRAny:
	case llvm::MVT::vAny:
	overloadedOps.set(r.index());
	break;
	default:
	continue;
	}
	}
	return overloadedOps;
	}

	namespace {
	/// A wrapper for LLVM's Tablegen class `Intrinsic` that provides accessors to
	/// the fields of the record.
	class LLVMIntrinsic {
	public:
	LLVMIntrinsic(const llvm::Record &record) : record(record) {}

	/// Get the name of the operation to be used in MLIR. Uses the appropriate
	/// field if not empty, constructs a name by replacing underscores with dots
	/// in the record name otherwise.
	std::string getOperationName() const {
	llvm::StringRef name = record.getValueAsString(fieldName);
	if (!name.empty())
	return name.str();

	name = record.getName();
	assert(name.startswith("int_") &&
	"LLVM intrinsic names are expected to start with 'int_'");
	name = name.drop_front(4);
	llvm::SmallVector<llvm::StringRef, 8> chunks;
	llvm::StringRef targetPrefix = record.getValueAsString("TargetPrefix");
	name.split(chunks, '_');
	auto *chunksBegin = chunks.begin();
	// Remove the target prefix from target specific intrinsics.
	if (!targetPrefix.empty()) {
	assert(targetPrefix == *chunksBegin &&
	"Intrinsic has TargetPrefix, but "
	"record name doesn't begin with it");
	assert(chunks.size() >= 2 &&
	"Intrinsic has TargetPrefix, but "
	"chunks has only one element meaning the intrinsic name is empty");
	++chunksBegin;
	}
	return llvm::join(chunksBegin, chunks.end(), ".");
	}

	/// Get the name of the record without the "intrinsic" prefix.
	llvm::StringRef getProperRecordName() const {
	llvm::StringRef name = record.getName();
	assert(name.startswith("int_") &&
	"LLVM intrinsic names are expected to start with 'int_'");
	return name.drop_front(4);
	}

	/// Get the number of operands.
	unsigned getNumOperands() const {
	auto operands = record.getValueAsListOfDefs(fieldOperands);
	assert(llvm::all_of(operands,
	[](const llvm::Record *r) {
	return r->isSubClassOf("LLVMType");
	}) &&
	"expected operands to be of LLVM type");
	return operands.size();
	}

	/// Get the number of results. Note that LLVM does not support multi-value
	/// operations so, in fact, multiple results will be returned as a value of
	/// structure type.
	unsigned getNumResults() const {
	auto results = record.getValueAsListOfDefs(fieldResults);
	for (const llvm::Record *r : results) {
	(void)r;
	assert(r->isSubClassOf("LLVMType") &&
	"expected operands to be of LLVM type");
	}
	return results.size();
	}

	/// Return true if the intrinsic may have side effects, i.e. does not have the
	/// `IntrNoMem` property.
	bool hasSideEffects() const {
	return llvm::none_of(
	record.getValueAsListOfDefs(fieldTraits),
	[](const llvm::Record *r) { return r->getName() == "IntrNoMem"; });
	}

	/// Return true if the intrinsic is commutative, i.e. has the respective
	/// property.
	bool isCommutative() const {
	return llvm::any_of(
	record.getValueAsListOfDefs(fieldTraits),
	[](const llvm::Record *r) { return r->getName() == "Commutative"; });
	}

	IndicesTy getOverloadableOperandsIdxs() const {
	return getOverloadableTypeIdxs(record, fieldOperands);
	}

	IndicesTy getOverloadableResultsIdxs() const {
	return getOverloadableTypeIdxs(record, fieldResults);
	}

	private:
	/// Names of the fields in the Intrinsic LLVM Tablegen class.
	const char *fieldName = "LLVMName";
	const char *fieldOperands = "ParamTypes";
	const char *fieldResults = "RetTypes";
	const char *fieldTraits = "IntrProperties";

	const llvm::Record &record;
	};
	} // namespace

	/// Prints the elements in "range" separated by commas and surrounded by "[]".
	template <typename Range>
	void printBracketedRange(const Range &range, llvm::raw_ostream &os) {
	os << '[';
	llvm::interleaveComma(range, os);
	os << ']';
	}

	/// Emits ODS (TableGen-based) code for `record` representing an LLVM intrinsic.
	/// Returns true on error, false on success.
	static bool emitIntrinsic(const llvm::Record &record, llvm::raw_ostream &os) {
	LLVMIntrinsic intr(record);

	llvm::Regex accessGroupMatcher(accessGroupRegexp);
	bool requiresAccessGroup =
	!accessGroupRegexp.empty() && accessGroupMatcher.match(record.getName());

	llvm::Regex aliasScopesMatcher(aliasScopesRegexp);
	bool requiresAliasAnalysis =
	!aliasScopesRegexp.empty() && aliasScopesMatcher.match(record.getName());

	// Prepare strings for traits, if any.
	llvm::SmallVector<llvm::StringRef, 2> traits;
	if (intr.isCommutative())
	traits.push_back("Commutative");
	if (!intr.hasSideEffects())
	traits.push_back("NoMemoryEffect");

	// Prepare strings for operands.
	llvm::SmallVector<llvm::StringRef, 8> operands(intr.getNumOperands(),
	"LLVM_Type");
	if (requiresAccessGroup)
	operands.push_back("OptionalAttr<SymbolRefArrayAttr>:$access_groups");
	if (requiresAliasAnalysis) {
	operands.push_back("OptionalAttr<SymbolRefArrayAttr>:$alias_scopes");
	operands.push_back("OptionalAttr<SymbolRefArrayAttr>:$noalias_scopes");
	operands.push_back("OptionalAttr<SymbolRefArrayAttr>:$tbaa");
	}

	// Emit the definition.
	os << "def LLVM_" << intr.getProperRecordName() << " : " << opBaseClass
	<< "<\"" << intr.getOperationName() << "\", ";
	printBracketedRange(intr.getOverloadableResultsIdxs().set_bits(), os);
	os << ", ";
	printBracketedRange(intr.getOverloadableOperandsIdxs().set_bits(), os);
	os << ", ";
	printBracketedRange(traits, os);
	os << ", " << intr.getNumResults() << ", "
	<< (requiresAccessGroup ? "1" : "0") << ", "
	<< (requiresAliasAnalysis ? "1" : "0") << ">, Arguments<(ins"
	<< (operands.empty() ? "" : " ");
	llvm::interleaveComma(operands, os);
	os << ")>;\n\n";

	return false;
	}

	/// Traverses the list of TableGen definitions derived from the "Intrinsic"
	/// class and generates MLIR ODS definitions for those intrinsics that have
	/// the name matching the filter.
	static bool emitIntrinsics(const llvm::RecordKeeper &records,
	llvm::raw_ostream &os) {
	llvm::emitSourceFileHeader("Operations for LLVM intrinsics", os);
	os << "include \"mlir/Dialect/LLVMIR/LLVMOpBase.td\"\n";
	os << "include \"mlir/Interfaces/SideEffectInterfaces.td\"\n\n";

	auto defs = records.getAllDerivedDefinitions("Intrinsic");
	for (const llvm::Record *r : defs) {
	if (!nameFilter.empty() && !r->getName().contains(nameFilter))
	continue;
	if (emitIntrinsic(*r, os))
	return true;
	}

	return false;
	}

	static mlir::GenRegistration genLLVMIRIntrinsics("gen-llvmir-intrinsics",
	"Generate LLVM IR intrinsics",
	emitIntrinsics);