[RISCV][llvm-mca] Use LMUL Instruments to provide more accurate reports on RISCV

[michaelmaitland]

Summary

On x86 and AArch, SIMD instructions encode all of the scheduling information in the instruction
itself. For example, VADD.I16 q0, q1, q2 is a neon instruction that operates on 16-bit integer
elements stored in 128-bit Q registers, which leads to eight 16-bit lanes in parallel. This kind
of information impacts how the instruction takes to execute and what dependencies this may cause.

On RISCV however, the data that impacts scheduling is encoded in CSR registers such as vtype or
vl, in addition with the instruction itself. But MCA does not track or use the data in these
registers. This patch fixes this problem.

Consider the following examples:

1. No vset{i}vl{i}:

vadd.vv v12, v12, v12

MCA is expectd to work on snippets of programs. This means it is okay for an llvm-mca user
to pass in a program without a vset{i}vl{i} as such. But the user may still know the LMUL
that this instruction was executed under. As a solution they can instrument the program
with this information as follows:

# LLVM-MCA-RISCV-LMUL M1
vadd.vv v12, v12, v12

It was considered as a design to use a vset{i}vl{i} instead of a comment. The problem with this is that
the vset{i}vl{i} may take physical cycles, and artifically inserting it would have tangible impact on
the performance analysis.

2. Program with vset{i}vl{i}

vsetvli zero, a0, e8, m1, tu, mu
vadd.vv v12, v12, v12

This program supplies the information to MCA about how the vtype register is being set,
but MCA is built to ignore the values of registers and immediates. Additionally, we're
not sure what vsetvli sets vl as since this is implementation specific.
As a result, I propose that a vset{i}vl{i} is considered in the same was as it was before
and we still rely on instruments placed to track the vtype state:

vsetvli zero, a0, e8, m1, tu, mu
# LLVM-MCA-RISCV-LMUL M1
vadd.vv v12, v12, v12

3. Multiple vset{i}vl{i}

vsetvli zero, a0, e8, m1, tu, mu
vadd.vv v12, v12, v12
vsetvli zero, a0, e8, m8, tu, mu
vadd.vv v12, v12, v12

I had considered using a command line option to set the relevant vtype register contents.
But the data in the register can change over the life of the program as demonstrated above.
Using instrument comments also solves this problem:

vsetvli zero, a0, e8, m1, tu, mu
# LLVM-MCA-RISCV-LMUL M1
vadd.vv v12, v12, v12
vsetvli zero, a0, e8, m8, tu, mu
# LLVM-MCA-RISCV-LMUL M8
vadd.vv v12, v12, v12

In this example we get a better understanding of the behavior of instrument regions: an instrument
is active until the end of the program, or until another instrument of the same type (in this case
RISCV-LMUL type). This means a programmer only needs to insert instruments after vset{i}vl{i}
instructions, or when a vset{i}vl{i} instruction above another instruction in the sequence was
not included in the program.

4. vsetvl

vsetvl rd, rs1, rs2
vadd.vv v12, v12, v12
vsetvl rd, rs1, rs2
vadd.vv v12, v12, v12

Example 2 proposed that we could use the values in the immediates instead of using instrument comments.
But vsetvl reads registers, not immediates, so we need to rely on instruments anyway.

vsetvl rd, rs1, rs2
# LLVM-MCA-RISCV-LMUL M1
vadd.vv v12, v12, v12
vsetvl rd, rs1, rs2
# LLVM-MCA-RISCV-LMUL M8
vadd.vv v12, v12, v12

• Replace CodeRegions with AnalysisRegions
• Add Instrument and InstrumentManager
• Add InstrumentRegions
• Add RISCV Instrument and InstrumentationManager
• Parse Instruments in driver
• Use instruments to override schedule class
• RISCV use lmul instrument to override schedule class
• Fix unit tests to pass empty instruments
• Add -ignore-im clopt to disable this change

[michaelmaitland]

Fix description of InstrumentManager

[andreadb]

Hi Michael,

Thanks a lot for this patch!

Two requests:

1. Please update the offical docs by adding a small paragraph (like we did for CustomBehaviour) describing this new feature and how to use it.
2. Please add some tests to verify that your new custom behaviour works as intended.

About 2.: you could reuse the code snippets from the summary of this review.
For each test, you can provide two (-instruction-tables) RUN lines:

• A RUN line with flag -disable-im.
• A RUN line without flag -disable-im.

That way, we can see how resource pressure changes when instruments are used to override the scheduling class IDs.

I also suggest to add a test where your special "instrument comments" are used in the presence of multiple nested code regions.
This last test is to ensure that lifetime of instruments is correctly handled, and that top-level instruments are correctly associated to nested regions too.

Thanks
-Andrea

llvm/include/llvm/MCA/CustomBehaviour.h

119

These fields should be private.

140–141

This comment should be improved.

This class allows targets to optionally customize the logic that resolves scheduling class IDs.
Targets can use information encoded in so-called Instrument objects to make more informed scheduling decisions.

Something like that... Essentially, a brief description of what this class does.

llvm/tools/llvm-mca/CodeRegion.h

29–55

Could you please also update the llvm-mca documentation?

llvm/tools/llvm-mca/llvm-mca.cpp

234–239

Please update the llvm-mca docs too.

[myhsu]

I read your patch descriptions with great interest and kudos on listing all cases. I generally agree with your instrument comment / analysis region. I think it's the most practical solution we have for now and also opens a door for creating other kinds of MCA extensions in the future.

On the technical side I dropped some drive-by comments, I'll have a deeper look later. As Andrea mentioned, please add some tests first. I think it's not hard in your case as you already had plenty of examples.

llvm/include/llvm/MCA/CustomBehaviour.h

121

you're already in llvm namespace so you can drop "llvm::". Ditto for rest of the changes.

130

Instrument() = default or even without one should suffice

llvm/lib/Target/RISCV/MCA/RISCVCustomBehaviour.cpp

50

nit: maybe replace this with StringSwitch?

58

okay these lines should definitely be replaced with StringSwitch

79

nit: use == instead? (unlike Java StringRef::operator== and StringRef::equals are the same)

llvm/tools/llvm-mca/CodeRegionGenerator.h

40

nit: add const qualifier

[michaelmaitland]

• Updated docs
• Made fields in Instrument private
• Improved classdoc for InstrumentManager
• Removed llvm namespace qualifier where it wasn't needed
• Simplified Instrument default constructor
• Use StringSwitch in RISCVCustomBehaviour
• Use == instead of .equals on StringRefs
• add const qualifier on MCACommentConsumer::hadErr

With respect to the request that tests are added:

There is no RISCV vector scheduler model that exists upstream. As a result, any tests I would add
will fail because processor does not support the vector instructions. I propose that we should create
a generic RISCV vector processor that exists upstream in a seperate patch. I think that this
would be beneficial for MCA so I can include those tests, but also for llvm contributors in areas
such as academia. What do you think about this?

[andreadb]

In D137440#3913468, @michaelmaitland wrote:

• Updated docs
• Made fields in Instrument private
• Improved classdoc for InstrumentManager
• Removed llvm namespace qualifier where it wasn't needed
• Simplified Instrument default constructor
• Use StringSwitch in RISCVCustomBehaviour
• Use == instead of .equals on StringRefs
• add const qualifier on MCACommentConsumer::hadErr

With respect to the request that tests are added:

There is no RISCV vector scheduler model that exists upstream. As a result, any tests I would add
will fail because processor does not support the vector instructions. I propose that we should create
a generic RISCV vector processor that exists upstream in a seperate patch. I think that this
would be beneficial for MCA so I can include those tests, but also for llvm contributors in areas
such as academia. What do you think about this?

Sounds like a good idea. Thanks!

I don't have other comments. Patch looks good to me. @myhsu what do you think?

[myhsu]

I only have minor comments. Otherwise LGTM. Cheers :-)

Also it will be great if you can mark inline comments as done upon addressing them, so that it's easier for both parties to navigate outstanding comments by simply clicking on "X comments" button at the top right corner. Another related tip: you can check those "done" boxes before updating a new diff. Phabricator will automatically submit them after uploading the diff.

llvm/lib/Target/RISCV/MCA/RISCVCustomBehaviour.cpp

99

this creates a copy for each element in IVec and brings some overhead due to extra bookkeeping upon copying a shared_ptr (and when I goes out of scope). I think you can auto &I : IVec (or even const auto& I : IVec) in this case.

106

llvm/lib/Target/RISCV/MCA/RISCVCustomBehaviour.h

33

remove "llvm::". ditto for other occurrences in this patch

llvm/tools/llvm-mca/CodeRegion.h

169

nit: const qualifier

174

nit: use struct instead and remove "public:". Ditto for InstrumentRegions

[reames]

Longer term, I really do think we need to be able to parse LMUL from vsetvli instructions, and have that influence scheduling of following instructions. I want to be able to take a program trace, and run it through llvm-mca without having to parse the trace and add annotations before doing so. Having said that, I have zero objection to the annotation mechanism being proposed here, and think it is useful. For the actual review, I'll defer to already active reviewers who are a lot more familiar with the internals of MCA that I am. :)

[michaelmaitland]

In D137440#3919848, @reames wrote:

Longer term, I really do think we need to be able to parse LMUL from vsetvli instructions, and have that influence scheduling of following instructions. I want to be able to take a program trace, and run it through llvm-mca without having to parse the trace and add annotations before doing so. Having said that, I have zero objection to the annotation mechanism being proposed here, and think it is useful. For the actual review, I'll defer to already active reviewers who are a lot more familiar with the internals of MCA that I am. :)

In the future, I am not opposed to adding support to use the immediate of the vsetvli instruction so annotations are not needed. But there are a few things we must consider:

• I plan on adding the ability to generate assembly that contains the annotations (using llc for example), so you wouldn't need to add annotations yourself on code that you generated anyway. This does not mean reading immediate of vsetvli can't happen. In fact, reading immediates would be helpful if you have assembly you did not generate yourself or didn't generate with the expectation to use it with llvm-mca.
• We would need to be more careful on programs that contained both vsetvli and vsetvl.
• We would need to consider the case where there was an annotation that had one LMUL value but the vsetvli instruction that was associated with the instrument had a different LMUL value. Of course maybe a user messed up writing instrument by hand, but the other alternative is that they originally emitted code that had one LMUL but they wanted to see what the impact is if they used a different LMUL. I think we would be able to control this behavior with a clopt though.

Overall, I think that this suggestion would make using llvm-mca an even better experience on RISCV vector programs, and I'm confident that all the concerns above can be solved.

[andreadb]

In D137440#3919869, @michaelmaitland wrote:

In D137440#3919848, @reames wrote:

Longer term, I really do think we need to be able to parse LMUL from vsetvli instructions, and have that influence scheduling of following instructions. I want to be able to take a program trace, and run it through llvm-mca without having to parse the trace and add annotations before doing so. Having said that, I have zero objection to the annotation mechanism being proposed here, and think it is useful. For the actual review, I'll defer to already active reviewers who are a lot more familiar with the internals of MCA that I am. :)

In the future, I am not opposed to adding support to use the immediate of the vsetvli instruction so annotations are not needed. But there are a few things we must consider:

• I plan on adding the ability to generate assembly that contains the annotations (using llc for example), so you wouldn't need to add annotations yourself on code that you generated anyway. This does not mean reading immediate of vsetvli can't happen. In fact, reading immediates would be helpful if you have assembly you did not generate yourself or didn't generate with the expectation to use it with llvm-mca.
• We would need to be more careful on programs that contained both vsetvli and vsetvl.
• We would need to consider the case where there was an annotation that had one LMUL value but the vsetvli instruction that was associated with the instrument had a different LMUL value. Of course maybe a user messed up writing instrument by hand, but the other alternative is that they originally emitted code that had one LMUL but they wanted to see what the impact is if they used a different LMUL. I think we would be able to control this behavior with a clopt though.

Overall, I think that this suggestion would make using llvm-mca an even better experience on RISCV vector programs, and I'm confident that all the concerns above can be solved.

Sounds like a good plan.

Thanks a lot for working on this.

Patch LGTM.

[michaelmaitland]

• drop llvm:: where possible
• iterate over reference instead of copy of IVec
• simplify nullptr check
• add const qualifier
• change public InstrumentRegion and AnalysisRegion classes to struct

[myhsu]

LGTM thanks

[thakis]

This doesn't build on Windows: http://45.33.8.238/win/70012/step_4.txt

Please take a look and revert for now if it takes a while to fix.

[thakis]

Doesn't build on mac either: http://45.33.8.238/macm1/48655/step_4.txt

[thakis]

Apparently doesn't build on linux either: http://45.33.8.238/linux/91480/step_4.txt

Maybe it's "doesn't' build with clang as host compiler"?

Anyways, thanks for the quick revert!

[michaelmaitland]

In D137440#3927809, @thakis wrote:

Apparently doesn't build on linux either: http://45.33.8.238/linux/91480/step_4.txt

Maybe it's "doesn't' build with clang as host compiler"?

Anyways, thanks for the quick revert!

It was because building with ninja all didn't build the x86 unit tests so I didn't catch it. I have fixed the issue on my local machine. What is standard procedure after reverting commit? Do I make an entirely new revision or modify this one?

[thakis]

I think most people modify the existing review.

[michaelmaitland]

Reopening so I can fix test cases.

[michaelmaitland]

Fix unit tests so it builds

[michaelmaitland]

clang-format

[andreadb]

In D137440#3927956, @michaelmaitland wrote:

Fix unit tests so it builds

LGTM

[michaelmaitland]

Reopening because there was still problems with the build. Not seeing these errors locally nor on the pre-commit checks. Going to investigate and will update once I resolve the issues.

[michaelmaitland]

llvm/include/llvm/MCA/CustomBehaviour.h

130

I cannot use this constructor form because N4659[dcl.init]p7.4), since:

• the default constructor for StringRef is not user-provided, following N4659[dcl.fct.def.default]p5

• the field Desc and Data are not const-default-constructible and has no default member initializer.

[michaelmaitland]

• Fix Instrument constructor
• SmallVector<SharedInstrument, 2> -> SmallVector<SharedInstrument>

[michaelmaitland]

I accidentally bled some other changes into here from an unrelated change. I will remove those.

[michaelmaitland]

Remove unrelated changes

[myhsu]

I think you'd addressed all errors from the buildbot failure.

llvm/include/llvm/MCA/CustomBehaviour.h

130

thanks for the investigation!

[RKSimon]

@michaelmaitland I've raised https://github.com/llvm/llvm-project/issues/59091 as we're seeing some MSVC warnings due to multiple inheritance

llvm/docs/CommandGuide/llvm-mca.rst

 
 .. option:: -disable-cb
 
   Force usage of the generic CustomBehaviour and InstrPostProcess classes rather
   than using the target specific implementation. The generic classes never
   detect any custom hazards or make any post processing modifications to
   instructions.
 
+.. option:: -disable-im
+
+  Force usage of the generic InstrumentManager rather than using the target
+  specific implementation. The generic class creates Instruments that provide
+  no extra information, and InstrumentManager never overrides the default
+  schedule class for a given instruction.
 
 EXIT STATUS
 -----------
 
 :program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed
 to standard error, and the tool returns 1.
 
 USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS
 ---------------------------------------------
 :program:`llvm-mca` allows for the optional usage of special code comments to
 mark regions of the assembly code to be analyzed.  A comment starting with
-substring ``LLVM-MCA-BEGIN`` marks the beginning of a code region. A comment
-starting with substring ``LLVM-MCA-END`` marks the end of a code region.  For
-example:
+substring ``LLVM-MCA-BEGIN`` marks the beginning of an analysis region. A
+comment starting with substring ``LLVM-MCA-END`` marks the end of a region.
+For example:
 
 .. code-block:: none
 
   # LLVM-MCA-BEGIN
     ...
   # LLVM-MCA-END
 
 If no user-defined region is specified, then :program:`llvm-mca` assumes a
 default region which contains every instruction in the input file.  Every region
 is analyzed in isolation, and the final performance report is the union of all
-the reports generated for every code region.
+the reports generated for every analysis region.
 
-Code regions can have names. For example:
+Analysis regions can have names. For example:
 
 .. code-block:: none
 
   # LLVM-MCA-BEGIN A simple example
     add %eax, %eax
   # LLVM-MCA-END
 
 The code from the example above defines a region named "A simple example" with a

 an impact on the code generated. This is because the ``__asm`` statements are
 seen as real code having important side effects, which limits how the code
 around them can be transformed. If users want to make use of inline assembly
 to emit markers, then the recommendation is to always verify that the output
 assembly is equivalent to the assembly generated in the absence of markers.
 The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_
 can also help in detecting missed optimizations.
 
+INSTRUMENT REGIONS
+------------------
+
+An InstrumentRegion describes a region of assembly code guarded by
+special LLVM-MCA comment directives.
+
+.. code-block:: none
+
+  # LLVM-MCA-<INSTRUMENT_TYPE> <data>
+    ...  ## asm
+
+where `INSTRUMENT_TYPE` is a type defined by the target and expects
+to use `data`.
+
+A comment starting with substring `LLVM-MCA-<INSTRUMENT_TYPE>`
+brings data into scope for llvm-mca to use in its analysis for
+all following instructions.
+
+If a comment with the same `INSTRUMENT_TYPE` is found later in the
+instruction list, then the original InstrumentRegion will be
+automatically ended, and a new InstrumentRegion will begin.
+
+If there are comments containing the different `INSTRUMENT_TYPE`,
+then both data sets remain available. In contrast with an AnalysisRegion,
+an InstrumentRegion does not need a comment to end the region.
+
+Comments that are prefixed with `LLVM-MCA-` but do not correspond to
+a valid `INSTRUMENT_TYPE` for the target cause an error, except for
+`BEGIN` and `END`, since those correspond to AnalysisRegions. Comments
+that do not start with `LLVM-MCA-` are ignored by :program `llvm-mca`.
+
+An instruction (a MCInst) is added to an InstrumentRegion R only
+if its location is in range [R.RangeStart, R.RangeEnd].
+
+On RISCV targets, vector instructions have different behaviour depending
+on the LMUL. Code can be instrumented with a comment that takes the
+following form:
+
+.. code-block:: none
+
+  # LLVM-MCA-RISCV-LMUL <M1|M2|M4|M8|MF2|MF4|MF8>
+
+The RISCV InstrumentManager will override the schedule class for vector
+instructions to use the scheduling behaviour of its pseudo-instruction
+which is LMUL dependent. It makes sense to place RISCV instrument
+comments directly after `vset{i}vl{i}` instructions, although
+they can be placed anywhere in the program.
+
+Example of program with no call to `vset{i}vl{i}`:
+
+.. code-block:: none
+
+  # LLVM-MCA-RISCV-LMUL M2
+  vadd.vv v2, v2, v2
+
+Example of program with call to `vset{i}vl{i}`:
+
+.. code-block:: none
+
+  vsetvli zero, a0, e8, m1, tu, mu
+  # LLVM-MCA-RISCV-LMUL M1
+  vadd.vv v2, v2, v2
+
+Example of program with multiple calls to `vset{i}vl{i}`:
+
+.. code-block:: none
+
+  vsetvli zero, a0, e8, m1, tu, mu
+  # LLVM-MCA-RISCV-LMUL M1
+  vadd.vv v2, v2, v2
+  vsetvli zero, a0, e8, m8, tu, mu
+  # LLVM-MCA-RISCV-LMUL M8
+  vadd.vv v2, v2, v2
+
+Example of program with call to `vsetvl`:
+
+.. code-block:: none
+
+ vsetvl rd, rs1, rs2
+ # LLVM-MCA-RISCV-LMUL M1
+ vadd.vv v12, v12, v12
+ vsetvl rd, rs1, rs2
+ # LLVM-MCA-RISCV-LMUL M4
+ vadd.vv v12, v12, v12
+
 HOW LLVM-MCA WORKS
 ------------------
 
 :program:`llvm-mca` takes assembly code as input. The assembly code is parsed
 into a sequence of MCInst with the help of the existing LLVM target assembly
 parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module
 to generate a performance report.
 

 that the current instruction must stall for (this can be an underestimate
 if you don't know the exact number and a value of 0 represents no stall).
 
 If you'd like to add a CustomBehaviour class for a target that doesn't
 already have one, refer to an existing implementation to see how to set it
 up. The classes are implemented within the target specific backend (for
 example `/llvm/lib/Target/AMDGPU/MCA/`) so that they can access backend symbols.
 
+Instrument Manager
+""""""""""""""""""""""""""""""""""""
+On certain architectures, scheduling information for certain instructions
+do not contain all of the information required to identify the most precise
+schedule class. For example, data that can have an impact on scheduling can
+be stored in CSR registers.
+
+One example of this is on RISCV, where values in registers such as `vtype`
+and `vl` change the scheduling behaviour of vector instructions. Since MCA
+does not keep track of the values in registers, instrument comments can
+be used to specify these values.
+
+InstrumentManager's main function is `getSchedClassID()` which has access
+to the MCInst and all of the instruments that are active for that MCInst.
+This function can use the instruments to override the schedule class of
+the MCInst.
+
+On RISCV, instrument comments containing LMUL information are used
+by `getSchedClassID()` to map a vector instruction and the active
+LMUL to the scheduling class of the pseudo-instruction that describes
+that base instruction and the active LMUL.
+
 Custom Views
 """"""""""""""""""""""""""""""""""""
 :program:`llvm-mca` comes with several Views such as the Timeline View and
 Summary View. These Views are generic and can work with most (if not all)
 targets. If you wish to add a new View to :program:`llvm-mca` and it does not
 require any backend functionality that is not already exposed through MC layer
 classes (MCSubtargetInfo, MCInstrInfo, etc.), please add it to the
 `/tools/llvm-mca/View/` directory. However, if your new View is target specific

llvm/include/llvm/MC/TargetRegistry.h

 class MCTargetOptions;
 class MCTargetStreamer;
 class raw_ostream;
 class TargetMachine;
 class TargetOptions;
 namespace mca {
 class CustomBehaviour;
 class InstrPostProcess;
+class InstrumentManager;
 struct SourceMgr;
 } // namespace mca
 
 MCStreamer *createNullStreamer(MCContext &Ctx);
 // Takes ownership of \p TAB and \p CE.
 
 /// Create a machine code streamer which will print out assembly for the native
 /// target, suitable for compiling with a native assembler.

 
 mca::CustomBehaviour *createCustomBehaviour(const MCSubtargetInfo &STI,
                                             const mca::SourceMgr &SrcMgr,
                                             const MCInstrInfo &MCII);
 
 mca::InstrPostProcess *createInstrPostProcess(const MCSubtargetInfo &STI,
                                               const MCInstrInfo &MCII);
 
+mca::InstrumentManager *createInstrumentManager(const MCSubtargetInfo &STI,
+                                                const MCInstrInfo &MCII);
+
 /// Target - Wrapper for Target specific information.
 ///
 /// For registration purposes, this is a POD type so that targets can be
 /// registered without the use of static constructors.
 ///
 /// Targets should implement a single global instance of this class (which
 /// will be zero initialized), and pass that instance to the TargetRegistry as
 /// part of their initialization.
using CustomBehaviourCtorTy =
       mca::CustomBehaviour *(*)(const MCSubtargetInfo &STI,
                                 const mca::SourceMgr &SrcMgr,
                                 const MCInstrInfo &MCII);
 
   using InstrPostProcessCtorTy =
       mca::InstrPostProcess *(*)(const MCSubtargetInfo &STI,
                                  const MCInstrInfo &MCII);
 
+  using InstrumentManagerCtorTy =
+      mca::InstrumentManager *(*)(const MCSubtargetInfo &STI,
+                                  const MCInstrInfo &MCII);
+
 private:
   /// Next - The next registered target in the linked list, maintained by the
   /// TargetRegistry.
   Target *Next;
 
   /// The target function for checking if an architecture is supported.
   ArchMatchFnTy ArchMatchFn;
 
private:
   /// CustomBehaviourCtorFn - Construction function for this target's
   /// CustomBehaviour, if registered (default = nullptr).
   CustomBehaviourCtorTy CustomBehaviourCtorFn = nullptr;
 
   /// InstrPostProcessCtorFn - Construction function for this target's
   /// InstrPostProcess, if registered (default = nullptr).
   InstrPostProcessCtorTy InstrPostProcessCtorFn = nullptr;
 
+  /// InstrumentManagerCtorFn - Construction function for this target's
+  /// InstrumentManager, if registered (default = nullptr).
+  InstrumentManagerCtorTy InstrumentManagerCtorFn = nullptr;
+
 public:
   Target() = default;
 
   /// @name Target Information
   /// @{
 
   // getNext - Return the next registered target.
   const Target *getNext() const { return Next; }
public:
   /// This class is used by llvm-mca and requires backend functionality.
   mca::InstrPostProcess *createInstrPostProcess(const MCSubtargetInfo &STI,
                                                 const MCInstrInfo &MCII) const {
     if (InstrPostProcessCtorFn)
       return InstrPostProcessCtorFn(STI, MCII);
     return nullptr;
   }
 
+  /// createInstrumentManager - Create a target specific
+  /// InstrumentManager. This class is used by llvm-mca and requires
+  /// backend functionality.
+  mca::InstrumentManager *
+  createInstrumentManager(const MCSubtargetInfo &STI,
+                          const MCInstrInfo &MCII) const {
+    if (InstrumentManagerCtorFn)
+      return InstrumentManagerCtorFn(STI, MCII);
+    return nullptr;
+  }
+
   /// @}
 };
 
 /// TargetRegistry - Generic interface to target specific features.
 struct TargetRegistry {
   // FIXME: Make this a namespace, probably just move all the Register*
   // functions into Target (currently they all just set members on the Target
   // anyway, and Target friends this class so those functions can...
struct TargetRegistry {
   ///
   /// @param T - The target being registered.
   /// @param Fn - A function to construct an InstrPostProcess for the target.
   static void RegisterInstrPostProcess(Target &T,
                                        Target::InstrPostProcessCtorTy Fn) {
     T.InstrPostProcessCtorFn = Fn;
   }
 
+  /// RegisterInstrumentManager - Register an InstrumentManager
+  /// implementation for the given target.
+  ///
+  /// Clients are responsible for ensuring that registration doesn't occur
+  /// while another thread is attempting to access the registry. Typically
+  /// this is done by initializing all targets at program startup.
+  ///
+  /// @param T - The target being registered.
+  /// @param Fn - A function to construct an InstrumentManager for the
+  /// target.
+  static void RegisterInstrumentManager(Target &T,
+                                        Target::InstrumentManagerCtorTy Fn) {
+    T.InstrumentManagerCtorFn = Fn;
+  }
+
   /// @}
 };
 
 //===--------------------------------------------------------------------===//
 
 /// RegisterTarget - Helper template for registering a target, for use in the
 /// target's initialization function. Usage:
 ///

llvm/include/llvm/MCA/CustomBehaviour.h

 /// within the llvm-mca pipeline simulation that llvm-mca isn't already capable
 /// of extracting from the Scheduling Models.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_MCA_CUSTOMBEHAVIOUR_H
 #define LLVM_MCA_CUSTOMBEHAVIOUR_H
 
+#include "llvm/ADT/SmallVector.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MCA/SourceMgr.h"
 #include "llvm/MCA/View.h"
 
 namespace llvm {
 namespace mca {
public:
   virtual std::vector<std::unique_ptr<View>>
   getPostInstrInfoViews(llvm::MCInstPrinter &IP,
                         llvm::ArrayRef<llvm::MCInst> Insts);
   /// Return a vector of Views that will be added after all other Views.
   virtual std::vector<std::unique_ptr<View>>
   getEndViews(llvm::MCInstPrinter &IP, llvm::ArrayRef<llvm::MCInst> Insts);
 };
 
+class Instrument {
+  /// The description of Instrument kind
+  const StringRef Desc;
+
+  /// The instrumentation data
+  const StringRef Data;
+
+public:
+  Instrument(StringRef Desc, StringRef Data) : Desc(Desc), Data(Data) {}
+
+  Instrument() : Instrument("", "") {}
+
+  virtual ~Instrument() = default;
+
+  StringRef getDesc() const { return Desc; }
+  StringRef getData() const { return Data; }
+};
+
+using SharedInstrument = std::shared_ptr<Instrument>;
+
+/// This class allows targets to optionally customize the logic that resolves
+/// scheduling class IDs. Targets can use information encoded in Instrument
+/// objects to make more informed scheduling decisions.
+class InstrumentManager {
+protected:
+  const MCSubtargetInfo &STI;
+  const MCInstrInfo &MCII;
+
+public:
+  InstrumentManager(const MCSubtargetInfo &STI, const MCInstrInfo &MCII)
+      : STI(STI), MCII(MCII) {}
+
+  virtual ~InstrumentManager() = default;
+
+  /// Returns true if llvm-mca should ignore instruments.
+  virtual bool shouldIgnoreInstruments() const { return true; }
+
+  // Returns true if this supports processing Instrument with
+  // Instrument.Desc equal to Type
+  virtual bool supportsInstrumentType(StringRef Type) const { return false; }
+
+  /// Allocate an Instrument, and return a shared pointer to it.
+  virtual SharedInstrument createInstrument(StringRef Desc, StringRef Data);
+
+  /// Given an MCInst and a vector of Instrument, a target can
+  /// return a SchedClassID. This can be used by a subtarget to return a
+  /// PseudoInstruction SchedClassID instead of the one that belongs to the
+  /// BaseInstruction This can be useful when a BaseInstruction does not convey
+  /// the correct scheduling information without additional data. By default,
+  /// it returns the SchedClassID that belongs to MCI.
+  virtual unsigned
+  getSchedClassID(const MCInstrInfo &MCII, const MCInst &MCI,
+                  const SmallVector<SharedInstrument> &IVec) const;
+};
+
 } // namespace mca
 } // namespace llvm
 
 #endif /* LLVM_MCA_CUSTOMBEHAVIOUR_H */

llvm/include/llvm/MCA/InstrBuilder.h

 #ifndef LLVM_MCA_INSTRBUILDER_H
 #define LLVM_MCA_INSTRBUILDER_H
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/MC/MCInstrAnalysis.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSubtargetInfo.h"
+#include "llvm/MCA/CustomBehaviour.h"
 #include "llvm/MCA/Instruction.h"
 #include "llvm/MCA/Support.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace mca {
 
 class RecycledInstErr : public ErrorInfo<RecycledInstErr> {

 /// descriptors (i.e. InstrDesc objects).
 /// Information from the machine scheduling model is used to identify processor
 /// resources that are consumed by an instruction.
 class InstrBuilder {
   const MCSubtargetInfo &STI;
   const MCInstrInfo &MCII;
   const MCRegisterInfo &MRI;
   const MCInstrAnalysis *MCIA;
+  const InstrumentManager &IM;
   SmallVector<uint64_t, 8> ProcResourceMasks;
 
-  DenseMap<unsigned short, std::unique_ptr<const InstrDesc>> Descriptors;
-  DenseMap<const MCInst *, std::unique_ptr<const InstrDesc>> VariantDescriptors;
+  // Key is the MCI.Opcode and SchedClassID the describe the value InstrDesc
+  DenseMap<std::pair<unsigned short, unsigned>,
+           std::unique_ptr<const InstrDesc>>
+      Descriptors;
+
+  // Key is the MCIInst and SchedClassID the describe the value InstrDesc
+  DenseMap<std::pair<const MCInst *, unsigned>,
+           std::unique_ptr<const InstrDesc>>
+      VariantDescriptors;
 
   bool FirstCallInst;
   bool FirstReturnInst;
 
   using InstRecycleCallback =
       llvm::function_ref<Instruction *(const InstrDesc &)>;
   InstRecycleCallback InstRecycleCB;
 
-  Expected<const InstrDesc &> createInstrDescImpl(const MCInst &MCI);
-  Expected<const InstrDesc &> getOrCreateInstrDesc(const MCInst &MCI);
+  Expected<const InstrDesc &>
+  createInstrDescImpl(const MCInst &MCI,
+                      const SmallVector<SharedInstrument> &IVec);
+  Expected<const InstrDesc &>
+  getOrCreateInstrDesc(const MCInst &MCI,
+                       const SmallVector<SharedInstrument> &IVec);
 
   InstrBuilder(const InstrBuilder &) = delete;
   InstrBuilder &operator=(const InstrBuilder &) = delete;
 
   void populateWrites(InstrDesc &ID, const MCInst &MCI, unsigned SchedClassID);
   void populateReads(InstrDesc &ID, const MCInst &MCI, unsigned SchedClassID);
   Error verifyInstrDesc(const InstrDesc &ID, const MCInst &MCI) const;
 
 public:
   InstrBuilder(const MCSubtargetInfo &STI, const MCInstrInfo &MCII,
-               const MCRegisterInfo &RI, const MCInstrAnalysis *IA);
+               const MCRegisterInfo &RI, const MCInstrAnalysis *IA,
+               const InstrumentManager &IM);
 
   void clear() {
     Descriptors.clear();
     VariantDescriptors.clear();
     FirstCallInst = true;
     FirstReturnInst = true;
   }
 
   /// Set a callback which is invoked to retrieve a recycled mca::Instruction
   /// or null if there isn't any.
   void setInstRecycleCallback(InstRecycleCallback CB) { InstRecycleCB = CB; }
 
-  Expected<std::unique_ptr<Instruction>> createInstruction(const MCInst &MCI);
+  Expected<std::unique_ptr<Instruction>>
+  createInstruction(const MCInst &MCI,
+                    const SmallVector<SharedInstrument> &IVec);
 };
 } // namespace mca
 } // namespace llvm
 
 #endif // LLVM_MCA_INSTRBUILDER_H

llvm/lib/MCA/CustomBehaviour.cpp

 }
 
 std::vector<std::unique_ptr<View>>
 CustomBehaviour::getEndViews(llvm::MCInstPrinter &IP,
                              llvm::ArrayRef<llvm::MCInst> Insts) {
   return std::vector<std::unique_ptr<View>>();
 }
 
+SharedInstrument InstrumentManager::createInstrument(llvm::StringRef Desc,
+                                                     llvm::StringRef Data) {
+  return std::make_shared<Instrument>(Desc, Data);
+}
+
+unsigned InstrumentManager::getSchedClassID(
+    const MCInstrInfo &MCII, const MCInst &MCI,
+    const llvm::SmallVector<SharedInstrument> &IVec) const {
+  return MCII.get(MCI.getOpcode()).getSchedClass();
+}
+
 } // namespace mca
 } // namespace llvm

llvm/lib/MCA/InstrBuilder.cpp

 namespace llvm {
 namespace mca {
 
 char RecycledInstErr::ID = 0;
 
 InstrBuilder::InstrBuilder(const llvm::MCSubtargetInfo &sti,
                            const llvm::MCInstrInfo &mcii,
                            const llvm::MCRegisterInfo &mri,
-                           const llvm::MCInstrAnalysis *mcia)
-    : STI(sti), MCII(mcii), MRI(mri), MCIA(mcia), FirstCallInst(true),
+                           const llvm::MCInstrAnalysis *mcia,
+                           const mca::InstrumentManager &im)
+    : STI(sti), MCII(mcii), MRI(mri), MCIA(mcia), IM(im), FirstCallInst(true),
       FirstReturnInst(true) {
   const MCSchedModel &SM = STI.getSchedModel();
   ProcResourceMasks.resize(SM.getNumProcResourceKinds());
   computeProcResourceMasks(STI.getSchedModel(), ProcResourceMasks);
 }
 
 static void initializeUsedResources(InstrDesc &ID,
                                     const MCSchedClassDesc &SCDesc,
Error InstrBuilder::verifyInstrDesc(const InstrDesc &ID,
   // FIXME: see PR44797. We should revisit these checks and possibly move them
   // in CodeGenSchedule.cpp.
   StringRef Message = "found an inconsistent instruction that decodes to zero "
                       "opcodes and that consumes scheduler resources.";
   return make_error<InstructionError<MCInst>>(std::string(Message), MCI);
 }
 
 Expected<const InstrDesc &>
-InstrBuilder::createInstrDescImpl(const MCInst &MCI) {
+InstrBuilder::createInstrDescImpl(const MCInst &MCI,
+                                  const SmallVector<SharedInstrument> &IVec) {
   assert(STI.getSchedModel().hasInstrSchedModel() &&
          "Itineraries are not yet supported!");
 
   // Obtain the instruction descriptor from the opcode.
   unsigned short Opcode = MCI.getOpcode();
   const MCInstrDesc &MCDesc = MCII.get(Opcode);
   const MCSchedModel &SM = STI.getSchedModel();
 
   // Then obtain the scheduling class information from the instruction.
-  unsigned SchedClassID = MCDesc.getSchedClass();
+  // Allow InstrumentManager to override and use a different SchedClassID
+  unsigned SchedClassID = IM.getSchedClassID(MCII, MCI, IVec);
   bool IsVariant = SM.getSchedClassDesc(SchedClassID)->isVariant();
 
   // Try to solve variant scheduling classes.
   if (IsVariant) {
     unsigned CPUID = SM.getProcessorID();
     while (SchedClassID && SM.getSchedClassDesc(SchedClassID)->isVariant())
       SchedClassID =
           STI.resolveVariantSchedClass(SchedClassID, &MCI, &MCII, CPUID);
InstrBuilder::createInstrDescImpl(const MCInst &MCI,
 
   // Validation check on the instruction descriptor.
   if (Error Err = verifyInstrDesc(*ID, MCI))
     return std::move(Err);
 
   // Now add the new descriptor.
   bool IsVariadic = MCDesc.isVariadic();
   if ((ID->IsRecyclable = !IsVariadic && !IsVariant)) {
-    Descriptors[MCI.getOpcode()] = std::move(ID);
-    return *Descriptors[MCI.getOpcode()];
+    auto DKey = std::make_pair(MCI.getOpcode(), SchedClassID);
+    Descriptors[DKey] = std::move(ID);
+    return *Descriptors[DKey];
   }
 
-  VariantDescriptors[&MCI] = std::move(ID);
-  return *VariantDescriptors[&MCI];
+  auto VDKey = std::make_pair(&MCI, SchedClassID);
+  VariantDescriptors[VDKey] = std::move(ID);
+  return *VariantDescriptors[VDKey];
 }
 
 Expected<const InstrDesc &>
-InstrBuilder::getOrCreateInstrDesc(const MCInst &MCI) {
-  if (Descriptors.find_as(MCI.getOpcode()) != Descriptors.end())
-    return *Descriptors[MCI.getOpcode()];
+InstrBuilder::getOrCreateInstrDesc(const MCInst &MCI,
+                                   const SmallVector<SharedInstrument> &IVec) {
+  // Cache lookup using SchedClassID from Instrumentation
+  unsigned SchedClassID = IM.getSchedClassID(MCII, MCI, IVec);
+
+  auto DKey = std::make_pair(MCI.getOpcode(), SchedClassID);
+  if (Descriptors.find_as(DKey) != Descriptors.end())
+    return *Descriptors[DKey];
+
+  unsigned CPUID = STI.getSchedModel().getProcessorID();
+  SchedClassID = STI.resolveVariantSchedClass(SchedClassID, &MCI, &MCII, CPUID);
+  auto VDKey = std::make_pair(&MCI, SchedClassID);
+  if (VariantDescriptors.find(VDKey) != VariantDescriptors.end())
+    return *VariantDescriptors[VDKey];
 
-  if (VariantDescriptors.find(&MCI) != VariantDescriptors.end())
-    return *VariantDescriptors[&MCI];
-
-  return createInstrDescImpl(MCI);
+  return createInstrDescImpl(MCI, IVec);
 }
 
 STATISTIC(NumVariantInst, "Number of MCInsts that doesn't have static Desc");
 
 Expected<std::unique_ptr<Instruction>>
-InstrBuilder::createInstruction(const MCInst &MCI) {
-  Expected<const InstrDesc &> DescOrErr = getOrCreateInstrDesc(MCI);
+InstrBuilder::createInstruction(const MCInst &MCI,
+                                const SmallVector<SharedInstrument> &IVec) {
+  Expected<const InstrDesc &> DescOrErr = getOrCreateInstrDesc(MCI, IVec);
   if (!DescOrErr)
     return DescOrErr.takeError();
   const InstrDesc &D = *DescOrErr;
   Instruction *NewIS = nullptr;
   std::unique_ptr<Instruction> CreatedIS;
   bool IsInstRecycled = false;
 
   if (!D.IsRecyclable)

llvm/lib/Target/RISCV/CMakeLists.txt

add_llvm_target(RISCVCodeGen
 
   ADD_TO_COMPONENT
   RISCV
   )
 
 add_subdirectory(AsmParser)
 add_subdirectory(Disassembler)
 add_subdirectory(MCTargetDesc)
+add_subdirectory(MCA)
 add_subdirectory(TargetInfo)

llvm/lib/Target/RISCV/MCA/CMakeLists.txt

+add_llvm_component_library(LLVMRISCVTargetMCA
+  RISCVCustomBehaviour.cpp
+
+  LINK_COMPONENTS
+  MC
+  MCParser
+  RISCVDesc
+  RISCVInfo
+  Support
+  MCA
+
+  ADD_TO_COMPONENT
+  RISCV
+  )

llvm/lib/Target/RISCV/MCA/RISCVCustomBehaviour.h

+//===-------------------- RISCVCustomBehaviour.h -----------------*-C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file defines the RISCVCustomBehaviour class which inherits from
+/// CustomBehaviour. This class is used by the tool llvm-mca to enforce
+/// target specific behaviour that is not expressed well enough in the
+/// scheduling model for mca to enforce it automatically.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_RISCV_MCA_RISCVCUSTOMBEHAVIOUR_H
+#define LLVM_LIB_TARGET_RISCV_MCA_RISCVCUSTOMBEHAVIOUR_H
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/MC/MCInst.h"
+#include "llvm/MC/MCInstrDesc.h"
+#include "llvm/MC/MCInstrInfo.h"
+#include "llvm/MCA/CustomBehaviour.h"
+
+namespace llvm {
+namespace mca {
+
+class RISCVLMULInstrument : public Instrument {
+public:
+  static const StringRef DESC_NAME;
+  static bool isDataValid(StringRef Data);
+
+  RISCVLMULInstrument(StringRef Data) : Instrument(DESC_NAME, Data) {}
+
+  ~RISCVLMULInstrument() = default;
+
+  uint8_t getLMUL() const;
+};
+
+class RISCVInstrumentManager : public InstrumentManager {
+public:
+  RISCVInstrumentManager(const MCSubtargetInfo &STI, const MCInstrInfo &MCII)
+      : InstrumentManager(STI, MCII) {}
+
+  bool shouldIgnoreInstruments() const override { return false; }
+  bool supportsInstrumentType(StringRef Type) const override;
+
+  /// Create a Instrument for RISCV target
+  SharedInstrument createInstrument(StringRef Desc, StringRef Data) override;
+
+  /// Using the Instrument, returns a SchedClassID to use instead of
+  /// the SchedClassID that belongs to the MCI or the original SchedClassID.
+  unsigned
+  getSchedClassID(const MCInstrInfo &MCII, const MCInst &MCI,
+                  const SmallVector<SharedInstrument> &IVec) const override;
+};
+
+} // namespace mca
+} // namespace llvm
+
+#endif

llvm/lib/Target/RISCV/MCA/RISCVCustomBehaviour.cpp

+//===------------------- RISCVCustomBehaviour.cpp ---------------*-C++ -* -===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file implements methods from the RISCVCustomBehaviour class.
+///
+//===----------------------------------------------------------------------===//
+
+#include "RISCVCustomBehaviour.h"
+#include "MCTargetDesc/RISCVMCTargetDesc.h"
+#include "RISCVInstrInfo.h"
+#include "TargetInfo/RISCVTargetInfo.h"
+#include "llvm/MC/TargetRegistry.h"
+#include "llvm/Support/Debug.h"
+
+#define DEBUG_TYPE "llvm-mca-riscv-custombehaviour"
+
+// This brings in a table with primary key of
+// base instruction opcode and lmul and maps
+// to the opcode of the pseudo instruction.
+namespace RISCVVInversePseudosTable {
+using namespace llvm;
+using namespace llvm::RISCV;
+
+struct PseudoInfo {
+  uint16_t Pseudo;
+  uint16_t BaseInstr;
+  uint8_t VLMul;
+};
+
+#define GET_RISCVVInversePseudosTable_IMPL
+#define GET_RISCVVInversePseudosTable_DECL
+#include "RISCVGenSearchableTables.inc"
+
+} // end namespace RISCVVInversePseudosTable
+
+namespace llvm {
+namespace mca {
+
+const llvm::StringRef RISCVLMULInstrument::DESC_NAME = "RISCV-LMUL";
+
+bool RISCVLMULInstrument::isDataValid(llvm::StringRef Data) {
+  // Return true if not one of the valid LMUL strings
+  return StringSwitch<bool>(Data)
+      .Cases("M1", "M2", "M4", "M8", "MF2", "MF4", "MF8", true)
+      .Default(false);
+}
+
+uint8_t RISCVLMULInstrument::getLMUL() const {
+  // assertion prevents us from needing llvm_unreachable in the StringSwitch
+  // below
+  assert(isDataValid(getData()) &&
+         "Cannot get LMUL because invalid Data value");
+  // These are the LMUL values that are used in RISCV tablegen
+  return StringSwitch<uint8_t>(getData())
+      .Case("M1", 0b000)
+      .Case("M2", 0b001)
+      .Case("M4", 0b010)
+      .Case("M8", 0b011)
+      .Case("MF2", 0b101)
+      .Case("MF4", 0b110)
+      .Case("MF8", 0b111);
+}
+
+bool RISCVInstrumentManager::supportsInstrumentType(
+    llvm::StringRef Type) const {
+  // Currently, only support for RISCVLMULInstrument type
+  return Type == RISCVLMULInstrument::DESC_NAME;
+}
+
+SharedInstrument
+RISCVInstrumentManager::createInstrument(llvm::StringRef Desc,
+                                         llvm::StringRef Data) {
+  if (Desc != RISCVLMULInstrument::DESC_NAME) {
+    LLVM_DEBUG(dbgs() << "RVCB: Unknown instrumentation Desc: " << Desc
+                      << '\n');
+    return nullptr;
+  }
+  if (RISCVLMULInstrument::isDataValid(Data)) {
+    LLVM_DEBUG(dbgs() << "RVCB: Bad data for instrument kind " << Desc << ": "
+                      << Data << '\n');
+    return nullptr;
+  }
+  return std::make_shared<RISCVLMULInstrument>(Data);
+}
+
+unsigned RISCVInstrumentManager::getSchedClassID(
+    const MCInstrInfo &MCII, const MCInst &MCI,
+    const llvm::SmallVector<SharedInstrument> &IVec) const {
+  unsigned short Opcode = MCI.getOpcode();
+  unsigned SchedClassID = MCII.get(Opcode).getSchedClass();
+
+  for (const auto &I : IVec) {
+    // Unknown Instrument kind
+    if (I->getDesc() == RISCVLMULInstrument::DESC_NAME) {
+      uint8_t LMUL = static_cast<RISCVLMULInstrument *>(I.get())->getLMUL();
+      const RISCVVInversePseudosTable::PseudoInfo *RVV =
+          RISCVVInversePseudosTable::getBaseInfo(Opcode, LMUL);
+      // Not a RVV instr
+      if (!RVV) {
+        LLVM_DEBUG(
+            dbgs()
+            << "RVCB: Could not find PseudoInstruction for Opcode "
+            << MCII.getName(Opcode) << ", LMUL=" << I->getData()
+            << ". Ignoring instrumentation and using original SchedClassID="
+            << SchedClassID << '\n');
+        return SchedClassID;
+      }
+
+      // Override using pseudo
+      LLVM_DEBUG(dbgs() << "RVCB: Found Pseudo Instruction for Opcode "
+                        << MCII.getName(Opcode) << ", LMUL=" << I->getData()
+                        << ". Overriding original SchedClassID=" << SchedClassID
+                        << " with " << MCII.getName(RVV->Pseudo) << '\n');
+      return MCII.get(RVV->Pseudo).getSchedClass();
+    }
+  }
+
+  // Unknown Instrument kind
+  LLVM_DEBUG(
+      dbgs() << "RVCB: Did not use instrumentation to override Opcode.\n");
+  return SchedClassID;
+}
+
+} // namespace mca
+} // namespace llvm
+
+using namespace llvm;
+using namespace mca;
+
+static InstrumentManager *
+createRISCVInstrumentManager(const MCSubtargetInfo &STI,
+                             const MCInstrInfo &MCII) {
+  return new RISCVInstrumentManager(STI, MCII);
+}
+
+/// Extern function to initialize the targets for the RISCV backend
+extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeRISCVTargetMCA() {
+  TargetRegistry::RegisterInstrumentManager(getTheRISCV32Target(),
+                                            createRISCVInstrumentManager);
+  TargetRegistry::RegisterInstrumentManager(getTheRISCV64Target(),
+                                            createRISCVInstrumentManager);
+}

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

def RISCVVPseudosTable : GenericTable {
   let FilterClass = "RISCVVPseudo";
   let CppTypeName = "PseudoInfo";
   let Fields = [ "Pseudo", "BaseInstr" ];
   let PrimaryKey = [ "Pseudo" ];
   let PrimaryKeyName = "getPseudoInfo";
   let PrimaryKeyEarlyOut = true;
 }
 
+def RISCVVInversePseudosTable : GenericTable {
+  let FilterClass = "RISCVVPseudo";
+  let CppTypeName = "PseudoInfo";
+  let Fields = [ "Pseudo", "BaseInstr", "VLMul" ];
+  let PrimaryKey = [ "BaseInstr", "VLMul" ];
+  let PrimaryKeyName = "getBaseInfo";
+  let PrimaryKeyEarlyOut = true;
+}
+
 def RISCVVIntrinsicsTable : GenericTable {
   let FilterClass = "RISCVVIntrinsic";
   let CppTypeName = "RISCVVIntrinsicInfo";
   let Fields = ["IntrinsicID", "ScalarOperand", "VLOperand"];
   let PrimaryKey = ["IntrinsicID"];
   let PrimaryKeyName = "getRISCVVIntrinsicInfo";
 }
 

llvm/tools/llvm-mca/CodeRegion.h

 //===-------------------------- CodeRegion.h -------------------*- C++ -* -===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 /// \file
 ///
-/// This file implements class CodeRegion and CodeRegions.
+/// This file implements class CodeRegion and CodeRegions, InstrumentRegion,
+/// AnalysisRegions, and InstrumentRegions.
 ///
 /// A CodeRegion describes a region of assembly code guarded by special LLVM-MCA
 /// comment directives.
 ///
 ///   # LLVM-MCA-BEGIN foo
 ///     ...  ## asm
 ///   # LLVM-MCA-END
 ///
 /// A comment starting with substring LLVM-MCA-BEGIN marks the beginning of a
 /// new region of code.
 /// A comment starting with substring LLVM-MCA-END marks the end of the
 /// last-seen region of code.
 ///
 /// Code regions are not allowed to overlap. Each region can have a optional
 /// description; internally, regions are described by a range of source
 /// locations (SMLoc objects).
 ///
-/// An instruction (a MCInst) is added to a region R only if its location is in
-/// range [R.RangeStart, R.RangeEnd].
+/// An instruction (a MCInst) is added to a CodeRegion R only if its
+/// location is in range [R.RangeStart, R.RangeEnd].
+///
+/// A InstrumentRegion describes a region of assembly code guarded by
+/// special LLVM-MCA comment directives.
+///
+///   # LLVM-MCA-<INSTRUMENTATION_TYPE> <data>
+///     ...  ## asm
+///
+/// where INSTRUMENTATION_TYPE is a type defined in llvm and expects to use
+/// data.
+///
+/// A comment starting with substring LLVM-MCA-<INSTRUMENTATION_TYPE>
+/// brings data into scope for llvm-mca to use in its analysis for
+/// all following instructions.
+///
+/// If the same INSTRUMENTATION_TYPE is found later in the instruction list,
+/// then the original InstrumentRegion will be automatically ended,
+/// and a new InstrumentRegion will begin.
+///
+/// If there are comments containing the different INSTRUMENTATION_TYPEs,
+/// then both data sets remain available. In contrast with a CodeRegion,
+/// an InstrumentRegion does not need a comment to end the region.
+//
+// An instruction (a MCInst) is added to an InstrumentRegion R only
+// if its location is in range [R.RangeStart, R.RangeEnd].
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_LLVM_MCA_CODEREGION_H
 #define LLVM_TOOLS_LLVM_MCA_CODEREGION_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/MC/MCInst.h"
+#include "llvm/MCA/CustomBehaviour.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/SMLoc.h"
 #include "llvm/Support/SourceMgr.h"
 #include <vector>
 
 namespace llvm {
 namespace mca {
 
public:
   bool empty() const { return Instructions.empty(); }
   bool isLocInRange(llvm::SMLoc Loc) const;
 
   llvm::ArrayRef<llvm::MCInst> getInstructions() const { return Instructions; }
 
   llvm::StringRef getDescription() const { return Description; }
 };
 
+/// Alias AnalysisRegion with CodeRegion since CodeRegionGenerator
+/// is absract and AnalysisRegionGenerator operates on AnalysisRegions
+using AnalysisRegion = CodeRegion;
+
+/// A CodeRegion that contains instrumentation that can be used
+/// in analysis of the region.
+class InstrumentRegion : public CodeRegion {
+  /// Instrument for this region.
+  SharedInstrument Instrument;
+
+public:
+  InstrumentRegion(llvm::StringRef Desc, llvm::SMLoc Start, SharedInstrument I)
+      : CodeRegion(Desc, Start), Instrument(I) {}
+
+public:
+  SharedInstrument getInstrument() const { return Instrument; }
+};
+
 class CodeRegionParseError final : public Error {};
 
 class CodeRegions {
+  CodeRegions(const CodeRegions &) = delete;
+  CodeRegions &operator=(const CodeRegions &) = delete;
+
+protected:
   // A source manager. Used by the tool to generate meaningful warnings.
   llvm::SourceMgr &SM;
 
   using UniqueCodeRegion = std::unique_ptr<CodeRegion>;
   std::vector<UniqueCodeRegion> Regions;
   llvm::StringMap<unsigned> ActiveRegions;
   bool FoundErrors;
 
-  CodeRegions(const CodeRegions &) = delete;
-  CodeRegions &operator=(const CodeRegions &) = delete;
-
 public:
-  CodeRegions(llvm::SourceMgr &S);
+  CodeRegions(llvm::SourceMgr &S) : SM(S), FoundErrors(false) {}
 
   typedef std::vector<UniqueCodeRegion>::iterator iterator;
   typedef std::vector<UniqueCodeRegion>::const_iterator const_iterator;
 
   iterator begin() { return Regions.begin(); }
   iterator end() { return Regions.end(); }
   const_iterator begin() const { return Regions.cbegin(); }
   const_iterator end() const { return Regions.cend(); }
 
-  void beginRegion(llvm::StringRef Description, llvm::SMLoc Loc);
-  void endRegion(llvm::StringRef Description, llvm::SMLoc Loc);
   void addInstruction(const llvm::MCInst &Instruction);
   llvm::SourceMgr &getSourceMgr() const { return SM; }
 
   llvm::ArrayRef<llvm::MCInst> getInstructionSequence(unsigned Idx) const {
     return Regions[Idx]->getInstructions();
   }
 
   bool empty() const {
     return llvm::all_of(Regions, [](const UniqueCodeRegion &Region) {
       return Region->empty();
     });
   }
 
   bool isValid() const { return !FoundErrors; }
+
+  bool isRegionActive(llvm::StringRef Description) const {
+    return ActiveRegions.find(Description) != ActiveRegions.end();
+  }
+};
+
+struct AnalysisRegions : public CodeRegions {
+  AnalysisRegions(llvm::SourceMgr &S);
+
+  void beginRegion(llvm::StringRef Description, llvm::SMLoc Loc);
+  void endRegion(llvm::StringRef Description, llvm::SMLoc Loc);
+};
+
+struct InstrumentRegions : public CodeRegions {
+  InstrumentRegions(llvm::SourceMgr &S);
+
+  void beginRegion(llvm::StringRef Description, llvm::SMLoc Loc,
+                   SharedInstrument Instrument);
+  void endRegion(llvm::StringRef Description, llvm::SMLoc Loc);
+
+  const SmallVector<SharedInstrument>
+  getActiveInstruments(llvm::SMLoc Loc) const;
 };
 
 } // namespace mca
 } // namespace llvm
 
 #endif

llvm/tools/llvm-mca/CodeRegion.cpp

 ///
 //===----------------------------------------------------------------------===//
 
 #include "CodeRegion.h"
 
 namespace llvm {
 namespace mca {
 
-CodeRegions::CodeRegions(llvm::SourceMgr &S) : SM(S), FoundErrors(false) {
-  // Create a default region for the input code sequence.
-  Regions.emplace_back(std::make_unique<CodeRegion>("", SMLoc()));
-}
-
 bool CodeRegion::isLocInRange(SMLoc Loc) const {
   if (RangeEnd.isValid() && Loc.getPointer() > RangeEnd.getPointer())
     return false;
   if (RangeStart.isValid() && Loc.getPointer() < RangeStart.getPointer())
     return false;
   return true;
 }
 
-void CodeRegions::beginRegion(StringRef Description, SMLoc Loc) {
+void CodeRegions::addInstruction(const MCInst &Instruction) {
+  SMLoc Loc = Instruction.getLoc();
+  for (UniqueCodeRegion &Region : Regions)
+    if (Region->isLocInRange(Loc))
+      Region->addInstruction(Instruction);
+}
+
+AnalysisRegions::AnalysisRegions(llvm::SourceMgr &S) : CodeRegions(S) {
+  // Create a default region for the input code sequence.
+  Regions.emplace_back(std::make_unique<CodeRegion>("", SMLoc()));
+}
+
+void AnalysisRegions::beginRegion(StringRef Description, SMLoc Loc) {
   if (ActiveRegions.empty()) {
     // Remove the default region if there is at least one user defined region.
     // By construction, only the default region has an invalid start location.
     if (Regions.size() == 1 && !Regions[0]->startLoc().isValid() &&
         !Regions[0]->endLoc().isValid()) {
       ActiveRegions[Description] = 0;
       Regions[0] = std::make_unique<CodeRegion>(Description, Loc);
       return;
     }
   } else {
     auto It = ActiveRegions.find(Description);
     if (It != ActiveRegions.end()) {
       const CodeRegion &R = *Regions[It->second];
       if (Description.empty()) {
-        SM.PrintMessage(Loc, SourceMgr::DK_Error,
+        SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
                         "found multiple overlapping anonymous regions");
-        SM.PrintMessage(R.startLoc(), SourceMgr::DK_Note,
+        SM.PrintMessage(R.startLoc(), llvm::SourceMgr::DK_Note,
                         "Previous anonymous region was defined here");
         FoundErrors = true;
         return;
       }
 
-      SM.PrintMessage(Loc, SourceMgr::DK_Error,
+      SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
                       "overlapping regions cannot have the same name");
-      SM.PrintMessage(R.startLoc(), SourceMgr::DK_Note,
+      SM.PrintMessage(R.startLoc(), llvm::SourceMgr::DK_Note,
                       "region " + Description + " was previously defined here");
       FoundErrors = true;
       return;
     }
   }
 
   ActiveRegions[Description] = Regions.size();
   Regions.emplace_back(std::make_unique<CodeRegion>(Description, Loc));
 }
 
-void CodeRegions::endRegion(StringRef Description, SMLoc Loc) {
+void AnalysisRegions::endRegion(StringRef Description, SMLoc Loc) {
   if (Description.empty()) {
     // Special case where there is only one user defined region,
     // and this LLVM-MCA-END directive doesn't provide a region name.
     // In this case, we assume that the user simply wanted to just terminate
     // the only active region.
     if (ActiveRegions.size() == 1) {
       auto It = ActiveRegions.begin();
       Regions[It->second]->setEndLocation(Loc);
void AnalysisRegions::endRegion(StringRef Description, SMLoc Loc) {
   auto It = ActiveRegions.find(Description);
   if (It != ActiveRegions.end()) {
     Regions[It->second]->setEndLocation(Loc);
     ActiveRegions.erase(It);
     return;
   }
 
   FoundErrors = true;
-  SM.PrintMessage(Loc, SourceMgr::DK_Error,
+  SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
                   "found an invalid region end directive");
   if (!Description.empty()) {
-    SM.PrintMessage(Loc, SourceMgr::DK_Note,
+    SM.PrintMessage(Loc, llvm::SourceMgr::DK_Note,
                     "unable to find an active region named " + Description);
   } else {
-    SM.PrintMessage(Loc, SourceMgr::DK_Note,
+    SM.PrintMessage(Loc, llvm::SourceMgr::DK_Note,
                     "unable to find an active anonymous region");
   }
 }
 
-void CodeRegions::addInstruction(const MCInst &Instruction) {
-  SMLoc Loc = Instruction.getLoc();
-  for (UniqueCodeRegion &Region : Regions)
-    if (Region->isLocInRange(Loc))
-      Region->addInstruction(Instruction);
+InstrumentRegions::InstrumentRegions(llvm::SourceMgr &S) : CodeRegions(S) {}
+
+void InstrumentRegions::beginRegion(StringRef Description, SMLoc Loc,
+                                    SharedInstrument I) {
+  if (Description.empty()) {
+    SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
+                    "anonymous instrumentation regions are not permitted");
+    FoundErrors = true;
+    return;
+  }
+
+  auto It = ActiveRegions.find(Description);
+  if (It != ActiveRegions.end()) {
+    const CodeRegion &R = *Regions[It->second];
+    SM.PrintMessage(
+        Loc, llvm::SourceMgr::DK_Error,
+        "overlapping instrumentation regions cannot be of the same kind");
+    SM.PrintMessage(R.startLoc(), llvm::SourceMgr::DK_Note,
+                    "instrumentation region " + Description +
+                        " was previously defined here");
+    FoundErrors = true;
+    return;
+  }
+
+  ActiveRegions[Description] = Regions.size();
+  Regions.emplace_back(std::make_unique<InstrumentRegion>(Description, Loc, I));
+}
+
+void InstrumentRegions::endRegion(StringRef Description, SMLoc Loc) {
+  auto It = ActiveRegions.find(Description);
+  if (It != ActiveRegions.end()) {
+    Regions[It->second]->setEndLocation(Loc);
+    ActiveRegions.erase(It);
+    return;
+  }
+
+  FoundErrors = true;
+  SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
+                  "found an invalid instrumentation region end directive");
+  if (!Description.empty()) {
+    SM.PrintMessage(Loc, llvm::SourceMgr::DK_Note,
+                    "unable to find an active instrumentation region named " +
+                        Description);
+  }
+}
+
+const SmallVector<SharedInstrument>
+InstrumentRegions::getActiveInstruments(SMLoc Loc) const {
+  SmallVector<SharedInstrument> AI;
+  for (auto &R : Regions) {
+    if (R->isLocInRange(Loc)) {
+      InstrumentRegion *IR = static_cast<InstrumentRegion *>(R.get());
+      AI.emplace_back(IR->getInstrument());
+    }
+  }
+  return AI;
 }
 
 } // namespace mca
 } // namespace llvm

llvm/tools/llvm-mca/CodeRegionGenerator.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TOOLS_LLVM_MCA_CODEREGION_GENERATOR_H
 #define LLVM_TOOLS_LLVM_MCA_CODEREGION_GENERATOR_H
 
 #include "CodeRegion.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
+#include "llvm/MC/MCParser/MCAsmLexer.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/TargetRegistry.h"
+#include "llvm/MCA/CustomBehaviour.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/SourceMgr.h"
 #include <memory>
 
 namespace llvm {
 namespace mca {
 
-/// This class is responsible for parsing the input given to the llvm-mca
-/// driver, and converting that into a CodeRegions instance.
+class MCACommentConsumer : public AsmCommentConsumer {
+protected:
+  bool FoundError;
+
+public:
+  MCACommentConsumer() : FoundError(false) {}
+
+  bool hadErr() const { return FoundError; }
+};
+
+/// A comment consumer that parses strings.  The only valid tokens are strings.
+class AnalysisRegionCommentConsumer : public MCACommentConsumer {
+  AnalysisRegions &Regions;
+
+public:
+  AnalysisRegionCommentConsumer(AnalysisRegions &R) : Regions(R) {}
+
+  /// Parses a comment. It begins a new region if it is of the form
+  /// LLVM-MCA-BEGIN. It ends a region if it is of the form LLVM-MCA-END.
+  /// Regions can be optionally named if they are of the form
+  /// LLVM-MCA-BEGIN <name> or LLVM-MCA-END <name>. Subregions are
+  /// permitted, but a region that begins while another region is active
+  /// must be ended before the outer region is ended. If thre is only one
+  /// active region, LLVM-MCA-END does not need to provide a name.
+  void HandleComment(SMLoc Loc, StringRef CommentText) override;
+};
+
+/// A comment consumer that parses strings to create InstrumentRegions.
+/// The only valid tokens are strings.
+class InstrumentRegionCommentConsumer : public MCACommentConsumer {
+  llvm::SourceMgr &SM;
+
+  InstrumentRegions &Regions;
+
+  InstrumentManager &IM;
+
+public:
+  InstrumentRegionCommentConsumer(llvm::SourceMgr &SM, InstrumentRegions &R,
+                                  InstrumentManager &IM)
+      : SM(SM), Regions(R), IM(IM) {}
+
+  /// Parses a comment. It begins a new region if it is of the form
+  /// LLVM-MCA-<INSTRUMENTATION_TYPE> <data> where INSTRUMENTATION_TYPE
+  /// is a valid InstrumentKind. If there is already an active
+  /// region of type INSTRUMENATION_TYPE, then it will end the active
+  /// one and begin a new one using the new data.
+  void HandleComment(SMLoc Loc, StringRef CommentText) override;
+};
+
+/// This abstract class is responsible for parsing the input given to
+/// the llvm-mca driver, and converting that into a CodeRegions instance.
 class CodeRegionGenerator {
 protected:
-  CodeRegions Regions;
   CodeRegionGenerator(const CodeRegionGenerator &) = delete;
   CodeRegionGenerator &operator=(const CodeRegionGenerator &) = delete;
+  virtual Expected<const CodeRegions &>
+  parseCodeRegions(const std::unique_ptr<MCInstPrinter> &IP) = 0;
 
 public:
-  CodeRegionGenerator(llvm::SourceMgr &SM) : Regions(SM) {}
+  CodeRegionGenerator() {}
   virtual ~CodeRegionGenerator();
-  virtual Expected<const CodeRegions &>
-  parseCodeRegions(const std::unique_ptr<MCInstPrinter> &IP) = 0;
 };
 
-/// This class is responsible for parsing input ASM and generating
-/// a CodeRegions instance.
-class AsmCodeRegionGenerator final : public CodeRegionGenerator {
+/// Abastract CodeRegionGenerator with AnalysisRegions member
+class AnalysisRegionGenerator : public virtual CodeRegionGenerator {
+protected:
+  AnalysisRegions Regions;
+
+public:
+  AnalysisRegionGenerator(llvm::SourceMgr &SM) : Regions(SM) {}
+
+  virtual Expected<const AnalysisRegions &>
+  parseAnalysisRegions(const std::unique_ptr<MCInstPrinter> &IP) = 0;
+};
+
+/// Abstract CodeRegionGenerator with InstrumentRegionsRegions member
+class InstrumentRegionGenerator : public virtual CodeRegionGenerator {
+protected:
+  InstrumentRegions Regions;
+
+public:
+  InstrumentRegionGenerator(llvm::SourceMgr &SM) : Regions(SM) {}
+
+  virtual Expected<const InstrumentRegions &>
+  parseInstrumentRegions(const std::unique_ptr<MCInstPrinter> &IP) = 0;
+};
+
+/// This abstract class is responsible for parsing input ASM and
+/// generating a CodeRegions instance.
+class AsmCodeRegionGenerator : public virtual CodeRegionGenerator {
   const Target &TheTarget;
   MCContext &Ctx;
   const MCAsmInfo &MAI;
   const MCSubtargetInfo &STI;
   const MCInstrInfo &MCII;
   unsigned AssemblerDialect; // This is set during parsing.
 
 public:
-  AsmCodeRegionGenerator(const Target &T, llvm::SourceMgr &SM, MCContext &C,
-                         const MCAsmInfo &A, const MCSubtargetInfo &S,
-                         const MCInstrInfo &I)
-      : CodeRegionGenerator(SM), TheTarget(T), Ctx(C), MAI(A), STI(S), MCII(I),
-        AssemblerDialect(0) {}
+  AsmCodeRegionGenerator(const Target &T, MCContext &C, const MCAsmInfo &A,
+                         const MCSubtargetInfo &S, const MCInstrInfo &I)
+      : TheTarget(T), Ctx(C), MAI(A), STI(S), MCII(I), AssemblerDialect(0) {}
+
+  virtual MCACommentConsumer *getCommentConsumer() = 0;
+  virtual CodeRegions &getRegions() = 0;
 
   unsigned getAssemblerDialect() const { return AssemblerDialect; }
   Expected<const CodeRegions &>
   parseCodeRegions(const std::unique_ptr<MCInstPrinter> &IP) override;
 };
 
+class AsmAnalysisRegionGenerator final : public AnalysisRegionGenerator,
+                                         public AsmCodeRegionGenerator {
+  AnalysisRegionCommentConsumer CC;
+
+public:
+  AsmAnalysisRegionGenerator(const Target &T, llvm::SourceMgr &SM, MCContext &C,
+                             const MCAsmInfo &A, const MCSubtargetInfo &S,
+                             const MCInstrInfo &I)
+      : AnalysisRegionGenerator(SM), AsmCodeRegionGenerator(T, C, A, S, I),
+        CC(Regions) {}
+
+  MCACommentConsumer *getCommentConsumer() override { return &CC; };
+  CodeRegions &getRegions() override { return Regions; };
+
+  Expected<const AnalysisRegions &>
+  parseAnalysisRegions(const std::unique_ptr<MCInstPrinter> &IP) override {
+    Expected<const CodeRegions &> RegionsOrErr = parseCodeRegions(IP);
+    if (!RegionsOrErr)
+      return RegionsOrErr.takeError();
+    else
+      return static_cast<const AnalysisRegions &>(*RegionsOrErr);
+  }
+};
+
+class AsmInstrumentRegionGenerator final : public InstrumentRegionGenerator,
+                                           public AsmCodeRegionGenerator {
+  InstrumentRegionCommentConsumer CC;
+
+public:
+  AsmInstrumentRegionGenerator(const Target &T, llvm::SourceMgr &SM,
+                               MCContext &C, const MCAsmInfo &A,
+                               const MCSubtargetInfo &S, const MCInstrInfo &I,
+                               InstrumentManager &IM)
+      : InstrumentRegionGenerator(SM), AsmCodeRegionGenerator(T, C, A, S, I),
+        CC(SM, Regions, IM) {}
+
+  MCACommentConsumer *getCommentConsumer() override { return &CC; };
+  CodeRegions &getRegions() override { return Regions; };
+
+  Expected<const InstrumentRegions &>
+  parseInstrumentRegions(const std::unique_ptr<MCInstPrinter> &IP) override {
+    Expected<const CodeRegions &> RegionsOrErr = parseCodeRegions(IP);
+    if (!RegionsOrErr)
+      return RegionsOrErr.takeError();
+    else
+      return static_cast<const InstrumentRegions &>(*RegionsOrErr);
+  }
+};
+
 } // namespace mca
 } // namespace llvm
 
 #endif // LLVM_TOOLS_LLVM_MCA_CODEREGION_GENERATOR_H

llvm/tools/llvm-mca/CodeRegionGenerator.cpp

 /// CodeRegions from various types of input. llvm-mca only analyzes CodeRegions,
 /// so the classes here provide the input-to-CodeRegions translation.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CodeRegionGenerator.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
-#include "llvm/MC/MCParser/MCAsmLexer.h"
 #include "llvm/MC/MCParser/MCTargetAsmParser.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/SMLoc.h"
 #include <memory>
 
 namespace llvm {
 namespace mca {
 
 // This virtual dtor serves as the anchor for the CodeRegionGenerator class.
 CodeRegionGenerator::~CodeRegionGenerator() {}
 
-// A comment consumer that parses strings.  The only valid tokens are strings.
-class MCACommentConsumer : public AsmCommentConsumer {
-public:
-  CodeRegions &Regions;
-
-  MCACommentConsumer(CodeRegions &R) : Regions(R) {}
-  void HandleComment(SMLoc Loc, StringRef CommentText) override;
-};
-
 // This class provides the callbacks that occur when parsing input assembly.
 class MCStreamerWrapper final : public MCStreamer {
   CodeRegions &Regions;
 
 public:
   MCStreamerWrapper(MCContext &Context, mca::CodeRegions &R)
       : MCStreamer(Context), Regions(R) {}
 
public:
   void emitCOFFSymbolType(int Type) override {}
   void endCOFFSymbolDef() override {}
 
   ArrayRef<MCInst> GetInstructionSequence(unsigned Index) const {
     return Regions.getInstructionSequence(Index);
   }
 };
 
-void MCACommentConsumer::HandleComment(SMLoc Loc, StringRef CommentText) {
+Expected<const CodeRegions &> AsmCodeRegionGenerator::parseCodeRegions(
+    const std::unique_ptr<MCInstPrinter> &IP) {
+  MCTargetOptions Opts;
+  Opts.PreserveAsmComments = false;
+  CodeRegions &Regions = getRegions();
+  MCStreamerWrapper Str(Ctx, Regions);
+
+  // Need to initialize an MCTargetStreamer otherwise
+  // certain asm directives will cause a segfault.
+  // Using nulls() so that anything emitted by the MCTargetStreamer
+  // doesn't show up in the llvm-mca output.
+  raw_ostream &OSRef = nulls();
+  formatted_raw_ostream FOSRef(OSRef);
+  TheTarget.createAsmTargetStreamer(Str, FOSRef, IP.get(),
+                                    /*IsVerboseAsm=*/true);
+
+  // Create a MCAsmParser and setup the lexer to recognize llvm-mca ASM
+  // comments.
+  std::unique_ptr<MCAsmParser> Parser(
+      createMCAsmParser(Regions.getSourceMgr(), Ctx, Str, MAI));
+  MCAsmLexer &Lexer = Parser->getLexer();
+  MCACommentConsumer *CCP = getCommentConsumer();
+  Lexer.setCommentConsumer(CCP);
+  // Enable support for MASM literal numbers (example: 05h, 101b).
+  Lexer.setLexMasmIntegers(true);
+
+  std::unique_ptr<MCTargetAsmParser> TAP(
+      TheTarget.createMCAsmParser(STI, *Parser, MCII, Opts));
+  if (!TAP)
+    return make_error<StringError>(
+        "This target does not support assembly parsing.",
+        inconvertibleErrorCode());
+  Parser->setTargetParser(*TAP);
+  Parser->Run(false);
+
+  if (CCP->hadErr())
+    return make_error<StringError>("There was an error parsing comments.",
+                                   inconvertibleErrorCode());
+
+  // Set the assembler dialect from the input. llvm-mca will use this as the
+  // default dialect when printing reports.
+  AssemblerDialect = Parser->getAssemblerDialect();
+  return Regions;
+}
+
+void AnalysisRegionCommentConsumer::HandleComment(SMLoc Loc,
+                                                  StringRef CommentText) {
   // Skip empty comments.
   StringRef Comment(CommentText);
   if (Comment.empty())
     return;
 
   // Skip spaces and tabs.
   unsigned Position = Comment.find_first_not_of(" \t");
   if (Position >= Comment.size())
void AnalysisRegionCommentConsumer::HandleComment(SMLoc Loc,
   // Skip spaces and tabs.
   Position = Comment.find_first_not_of(" \t");
   if (Position < Comment.size())
     Comment = Comment.drop_front(Position);
   // Use the rest of the string as a descriptor for this code snippet.
   Regions.beginRegion(Comment, Loc);
 }
 
-Expected<const CodeRegions &> AsmCodeRegionGenerator::parseCodeRegions(
-    const std::unique_ptr<MCInstPrinter> &IP) {
-  MCTargetOptions Opts;
-  Opts.PreserveAsmComments = false;
-  MCStreamerWrapper Str(Ctx, Regions);
+void InstrumentRegionCommentConsumer::HandleComment(SMLoc Loc,
+                                                    StringRef CommentText) {
+  // Skip empty comments.
+  StringRef Comment(CommentText);
+  if (Comment.empty())
+    return;
 
-  // Need to initialize an MCTargetStreamer otherwise
-  // certain asm directives will cause a segfault.
-  // Using nulls() so that anything emitted by the MCTargetStreamer
-  // doesn't show up in the llvm-mca output.
-  raw_ostream &OSRef = nulls();
-  formatted_raw_ostream FOSRef(OSRef);
-  TheTarget.createAsmTargetStreamer(Str, FOSRef, IP.get(),
-                                    /*IsVerboseAsm=*/true);
+  // Skip spaces and tabs.
+  unsigned Position = Comment.find_first_not_of(" \t");
+  if (Position >= Comment.size())
+    // We reached the end of the comment. Bail out.
+    return;
+  Comment = Comment.drop_front(Position);
 
-  // Create a MCAsmParser and setup the lexer to recognize llvm-mca ASM
-  // comments.
-  std::unique_ptr<MCAsmParser> Parser(
-      createMCAsmParser(Regions.getSourceMgr(), Ctx, Str, MAI));
-  MCAsmLexer &Lexer = Parser->getLexer();
-  MCACommentConsumer CC(Regions);
-  Lexer.setCommentConsumer(&CC);
-  // Enable support for MASM literal numbers (example: 05h, 101b).
-  Lexer.setLexMasmIntegers(true);
+  // Bail out if not an MCA style comment
+  if (!Comment.consume_front("LLVM-MCA-"))
+    return;
 
-  std::unique_ptr<MCTargetAsmParser> TAP(
-      TheTarget.createMCAsmParser(STI, *Parser, MCII, Opts));
-  if (!TAP)
-    return make_error<StringError>(
-        "This target does not support assembly parsing.",
-        inconvertibleErrorCode());
-  Parser->setTargetParser(*TAP);
-  Parser->Run(false);
+  // Skip AnalysisRegion comments
+  if (Comment.consume_front("BEGIN") || Comment.consume_front("END"))
+    return;
 
-  // Set the assembler dialect from the input. llvm-mca will use this as the
-  // default dialect when printing reports.
-  AssemblerDialect = Parser->getAssemblerDialect();
-  return Regions;
+  if (IM.shouldIgnoreInstruments())
+    return;
+
+  auto [InstrumentKind, Data] = Comment.split(" ");
+
+  // An error if not of the form LLVM-MCA-TARGET-KIND
+  if (!IM.supportsInstrumentType(InstrumentKind)) {
+    if (InstrumentKind.empty())
+      SM.PrintMessage(
+          Loc, llvm::SourceMgr::DK_Error,
+          "No instrumentation kind was provided in LLVM-MCA comment");
+    else
+      SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
+                      "Unknown instrumentation type in LLVM-MCA comment: " +
+                          InstrumentKind);
+    FoundError = true;
+    return;
+  }
+
+  SharedInstrument I = IM.createInstrument(InstrumentKind, Data);
+  if (!I) {
+    if (Data.empty())
+      SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
+                      "Failed to create " + InstrumentKind +
+                          " instrument with no data");
+    else
+      SM.PrintMessage(Loc, llvm::SourceMgr::DK_Error,
+                      "Failed to create " + InstrumentKind +
+                          " instrument with data: " + Data);
+    FoundError = true;
+    return;
+  }
+
+  // End InstrumentType region if one is open
+  if (Regions.isRegionActive(InstrumentKind))
+    Regions.endRegion(InstrumentKind, Loc);
+  // Start new instrumentation region
+  Regions.beginRegion(InstrumentKind, Loc, I);
 }
 
 } // namespace mca
 } // namespace llvm

llvm/tools/llvm-mca/llvm-mca.cpp

static cl::opt<bool> ShowBarriers(
     cl::cat(ViewOptions), cl::init(false));
 
 static cl::opt<bool> DisableCustomBehaviour(
     "disable-cb",
     cl::desc(
         "Disable custom behaviour (use the default class which does nothing)."),
     cl::cat(ViewOptions), cl::init(false));
 
+static cl::opt<bool> DisableInstrumentManager(
+    "disable-im",
+    cl::desc("Disable instrumentation manager (use the default class which "
+             "ignores instruments.)."),
+    cl::cat(ViewOptions), cl::init(false));
+
 namespace {
 
 const Target *getTarget(const char *ProgName) {
   if (TripleName.empty())
     TripleName = Triple::normalize(sys::getDefaultTargetTriple());
   Triple TheTriple(TripleName);
 
   // Get the target specific parser.
int main(int argc, char **argv) {
   std::unique_ptr<MCInstrInfo> MCII(TheTarget->createMCInstrInfo());
   assert(MCII && "Unable to create instruction info!");
 
   std::unique_ptr<MCInstrAnalysis> MCIA(
       TheTarget->createMCInstrAnalysis(MCII.get()));
 
   // Need to initialize an MCInstPrinter as it is
   // required for initializing the MCTargetStreamer
-  // which needs to happen within the CRG.parseCodeRegions() call below.
+  // which needs to happen within the CRG.parseAnalysisRegions() call below.
   // Without an MCTargetStreamer, certain assembly directives can trigger a
   // segfault. (For example, the .cv_fpo_proc directive on x86 will segfault if
   // we don't initialize the MCTargetStreamer.)
   unsigned IPtempOutputAsmVariant =
       OutputAsmVariant == -1 ? 0 : OutputAsmVariant;
   std::unique_ptr<MCInstPrinter> IPtemp(TheTarget->createMCInstPrinter(
       Triple(TripleName), IPtempOutputAsmVariant, *MAI, *MCII, *MRI));
   if (!IPtemp) {
     WithColor::error()
         << "unable to create instruction printer for target triple '"
         << TheTriple.normalize() << "' with assembly variant "
         << IPtempOutputAsmVariant << ".\n";
     return 1;
   }
 
   // Parse the input and create CodeRegions that llvm-mca can analyze.
-  mca::AsmCodeRegionGenerator CRG(*TheTarget, SrcMgr, Ctx, *MAI, *STI, *MCII);
-  Expected<const mca::CodeRegions &> RegionsOrErr =
-      CRG.parseCodeRegions(std::move(IPtemp));
+  mca::AsmAnalysisRegionGenerator CRG(*TheTarget, SrcMgr, Ctx, *MAI, *STI,
+                                      *MCII);
+  Expected<const mca::AnalysisRegions &> RegionsOrErr =
+      CRG.parseAnalysisRegions(std::move(IPtemp));
   if (!RegionsOrErr) {
     if (auto Err =
             handleErrors(RegionsOrErr.takeError(), [](const StringError &E) {
               WithColor::error() << E.getMessage() << '\n';
             })) {
       // Default case.
       WithColor::error() << toString(std::move(Err)) << '\n';
     }
     return 1;
   }
-  const mca::CodeRegions &Regions = *RegionsOrErr;
+  const mca::AnalysisRegions &Regions = *RegionsOrErr;
 
   // Early exit if errors were found by the code region parsing logic.
   if (!Regions.isValid())
     return 1;
 
   if (Regions.empty()) {
     WithColor::error() << "no assembly instructions found.\n";
     return 1;
   }
 
+  std::unique_ptr<mca::InstrumentManager> IM;
+  if (!DisableInstrumentManager) {
+    IM = std::unique_ptr<mca::InstrumentManager>(
+        TheTarget->createInstrumentManager(*STI, *MCII));
+  }
+  if (!IM) {
+    // If the target doesn't have its own IM implemented (or the -disable-cb
+    // flag is set) then we use the base class (which does nothing).
+    IM = std::make_unique<mca::InstrumentManager>(*STI, *MCII);
+  }
+
+  // Parse the input and create InstrumentRegion that llvm-mca
+  // can use to improve analysis.
+  mca::AsmInstrumentRegionGenerator IRG(*TheTarget, SrcMgr, Ctx, *MAI, *STI,
+                                        *MCII, *IM);
+  Expected<const mca::InstrumentRegions &> InstrumentRegionsOrErr =
+      IRG.parseInstrumentRegions(std::move(IPtemp));
+  if (!InstrumentRegionsOrErr) {
+    if (auto Err = handleErrors(InstrumentRegionsOrErr.takeError(),
+                                [](const StringError &E) {
+                                  WithColor::error() << E.getMessage() << '\n';
+                                })) {
+      // Default case.
+      WithColor::error() << toString(std::move(Err)) << '\n';
+    }
+    return 1;
+  }
+  const mca::InstrumentRegions &InstrumentRegions = *InstrumentRegionsOrErr;
+
+  // Early exit if errors were found by the instrumentation parsing logic.
+  if (!InstrumentRegions.isValid())
+    return 1;
+
   // Now initialize the output file.
   auto OF = getOutputStream();
   if (std::error_code EC = OF.getError()) {
     WithColor::error() << EC.message() << '\n';
     return 1;
   }
 
   unsigned AssemblerDialect = CRG.getAssemblerDialect();
int main(int argc, char **argv) {
   }
   if (!IPP) {
     // If the target doesn't have its own IPP implemented (or the -disable-cb
     // flag is set) then we use the base class (which does nothing).
     IPP = std::make_unique<mca::InstrPostProcess>(*STI, *MCII);
   }
 
   // Create an instruction builder.
-  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get());
+  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get(), *IM);
 
   // Create a context to control ownership of the pipeline hardware.
   mca::Context MCA(*MRI, *STI);
 
   mca::PipelineOptions PO(MicroOpQueue, DecoderThroughput, DispatchWidth,
                           RegisterFileSize, LoadQueueSize, StoreQueueSize,
                           AssumeNoAlias, EnableBottleneckAnalysis);
 
   // Number each region in the sequence.
   unsigned RegionIdx = 0;
 
   std::unique_ptr<MCCodeEmitter> MCE(
       TheTarget->createMCCodeEmitter(*MCII, Ctx));
   assert(MCE && "Unable to create code emitter!");
 
   std::unique_ptr<MCAsmBackend> MAB(TheTarget->createMCAsmBackend(
       *STI, *MRI, mc::InitMCTargetOptionsFromFlags()));
   assert(MAB && "Unable to create asm backend!");
 
   json::Object JSONOutput;
-  for (const std::unique_ptr<mca::CodeRegion> &Region : Regions) {
+  for (const std::unique_ptr<mca::AnalysisRegion> &Region : Regions) {
     // Skip empty code regions.
     if (Region->empty())
       continue;
 
     IB.clear();
 
     // Lower the MCInst sequence into an mca::Instruction sequence.
     ArrayRef<MCInst> Insts = Region->getInstructions();
     mca::CodeEmitter CE(*STI, *MAB, *MCE, Insts);
 
     IPP->resetState();
 
     SmallVector<std::unique_ptr<mca::Instruction>> LoweredSequence;
     for (const MCInst &MCI : Insts) {
+      SMLoc Loc = MCI.getLoc();
+      const SmallVector<mca::SharedInstrument> Instruments =
+          InstrumentRegions.getActiveInstruments(Loc);
+
       Expected<std::unique_ptr<mca::Instruction>> Inst =
-          IB.createInstruction(MCI);
+          IB.createInstruction(MCI, Instruments);
       if (!Inst) {
         if (auto NewE = handleErrors(
                 Inst.takeError(),
                 [&IP, &STI](const mca::InstructionError<MCInst> &IE) {
                   std::string InstructionStr;
                   raw_string_ostream SS(InstructionStr);
                   WithColor::error() << IE.Message << '\n';
                   IP->printInst(&IE.Inst, 0, "", *STI, SS);

llvm/unittests/tools/llvm-mca/MCATestBase.cpp

void MCATestBase::SetUp() {
   ASSERT_TRUE(IP);
 }
 
 Error MCATestBase::runBaselineMCA(json::Object &Result, ArrayRef<MCInst> Insts,
                                   ArrayRef<mca::View *> Views,
                                   const mca::PipelineOptions *PO) {
   mca::Context MCA(*MRI, *STI);
 
-  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get());
+  // Default InstrumentManager
+  auto IM = std::make_unique<mca::InstrumentManager>(*STI, *MCII);
+  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get(), *IM);
 
+  const SmallVector<mca::SharedInstrument> Instruments;
   SmallVector<std::unique_ptr<mca::Instruction>> LoweredInsts;
   for (const auto &MCI : Insts) {
     Expected<std::unique_ptr<mca::Instruction>> Inst =
-        IB.createInstruction(MCI);
+        IB.createInstruction(MCI, Instruments);
     if (!Inst) {
       if (auto NewE =
               handleErrors(Inst.takeError(),
                            [this](const mca::InstructionError<MCInst> &IE) {
                              std::string InstructionStr;
                              raw_string_ostream SS(InstructionStr);
                              WithColor::error() << IE.Message << '\n';
                              IP->printInst(&IE.Inst, 0, "", *STI, SS);

llvm/unittests/tools/llvm-mca/X86/TestIncrementalMCA.cpp

TEST_F(X86TestBase, TestResumablePipeline) {
   SmallVector<MCInst> MCIs;
   getSimpleInsts(MCIs, /*Repeats=*/100);
 
   // Add views.
   auto SV = std::make_unique<SummaryView>(STI->getSchedModel(), MCIs,
                                           PO.DispatchWidth);
   P->addEventListener(SV.get());
 
-  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get());
+  auto IM = std::make_unique<mca::InstrumentManager>(*STI, *MCII);
+  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get(), *IM);
 
+  const SmallVector<mca::SharedInstrument> Instruments;
   // Tile size = 7
   for (unsigned i = 0U, E = MCIs.size(); i < E;) {
     for (unsigned TE = i + 7; i < TE && i < E; ++i) {
       Expected<std::unique_ptr<mca::Instruction>> InstOrErr =
-          IB.createInstruction(MCIs[i]);
+          IB.createInstruction(MCIs[i], Instruments);
       ASSERT_TRUE(bool(InstOrErr));
       ISM.addInst(std::move(InstOrErr.get()));
     }
 
     // Run the pipeline.
     Expected<unsigned> Cycles = P->run();
     if (!Cycles) {
       // Should be a stream pause error.
TEST_F(X86TestBase, TestInstructionRecycling) {
   SmallVector<MCInst> MCIs;
   getSimpleInsts(MCIs, /*Repeats=*/100);
 
   // Add views.
   auto SV = std::make_unique<SummaryView>(STI->getSchedModel(), MCIs,
                                           PO.DispatchWidth);
   P->addEventListener(SV.get());
 
-  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get());
+  // Default InstrumentManager
+  auto IM = std::make_unique<mca::InstrumentManager>(*STI, *MCII);
+
+  mca::InstrBuilder IB(*STI, *MCII, *MRI, MCIA.get(), *IM);
   IB.setInstRecycleCallback(GetRecycledInst);
 
+  const SmallVector<mca::SharedInstrument> Instruments;
   // Tile size = 7
   for (unsigned i = 0U, E = MCIs.size(); i < E;) {
     for (unsigned TE = i + 7; i < TE && i < E; ++i) {
       Expected<std::unique_ptr<mca::Instruction>> InstOrErr =
-          IB.createInstruction(MCIs[i]);
+          IB.createInstruction(MCIs[i], Instruments);
 
       if (!InstOrErr) {
         mca::Instruction *RecycledInst = nullptr;
         // Check if the returned instruction is a recycled
         // one.
         auto RemainingE = handleErrors(InstOrErr.takeError(),
                                        [&](const mca::RecycledInstErr &RC) {
                                          RecycledInst = RC.getInst();