[docs][NewPM] Add docs for writing NPM passes

[aeubanks]

Summary

As to not conflict with the legacy PM example passes under
llvm/lib/Transforms/Hello, this is under HelloNew. This makes the
CMakeLists.txt and general directory structure less confusing for people
following the example.

Much of the doc structure was taken from WritinAnLLVMPass.rst.

This adds a HelloWorld pass which simply prints out each function name.

More will follow after this, e.g. passes over different units of IR, analyses.
https://llvm.org/docs/WritingAnLLVMPass.html contains a lot more.

[aeubanks]

format

[aeubanks]

more updates

[ychen]

Thanks for getting this started.

llvm/docs/WritingAnLLVMNewPMPass.rst

20

Not an English grammar expert. But it feels to me the was should is?

22

As mentioned in the comments on top of PassManager.h, I think we could say there is no interface for pass since the NPM is adopting concept-based polymorphism where not using inheritance is a goal. Maybe add a reference to https://sean-parent.stlab.cc/papers-and-presentations/#value-semantics-and-concept-based-polymorphism

48

Do we need this "(just `opt` is necessary for now)"? If I'm new to LLVM, I would find this a little bit distracting. It does not hurt anything though.

91

Could we delete this?

98

#endif // LLVM_TRANSFORMS_HELLO_HELLOWORLD_H

100

Extra comma. Or if we delete the ctor, we could also delete the wording here?

114

Is the ... here intentional?

Looking at https://llvm.org/docs/WritingAnLLVMPass.html makes me think it was a typo.

133

ditto

146

ditto

llvm/include/llvm/Transforms/HelloNew/HelloWorld.h

10

LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H

[aeubanks]

address review comments

llvm/docs/WritingAnLLVMNewPMPass.rst

20

I was trying to go for the angle of "the legacy PM is in the past", but I guess that's not true yet, so "is" makes more sense.

22

Clarified a bit, mentioned comments in PassManager.h rather than link to that since there's more that we're missing anyway.

48

Done. Also linked to GettingStarted.

114

I think it's meant to show that the comment is a continuation of before the code snippet which makes sense to me.
I changed a bit of the wording in a couple places, let me know if it's still weird.

[ychen]

One more comment. Please let another reviewer ack.

llvm/docs/UserGuides.rst

57

This needs a :doc: entry below.

[aeubanks]

One more place in UserGuides.rst

[asbirlea]

Thank you adding this!

llvm/docs/WritingAnLLVMNewPMPass.rst

78

s/should/should be
or
s/containing/contain

[aeubanks]

rebase
containing -> contain

[dcaballe]

This commit seems to be breaking the build with -DBUILD_SHARED_LIBS=TRUE. Could you please have a quick look? Thanks!

[aeubanks]

In D86979#2272354, @dcaballe wrote:

This commit seems to be breaking the build with -DBUILD_SHARED_LIBS=TRUE. Could you please have a quick look? Thanks!

Taking a look

[aeubanks]

In D86979#2272399, @aeubanks wrote:

In D86979#2272354, @dcaballe wrote:

This commit seems to be breaking the build with -DBUILD_SHARED_LIBS=TRUE. Could you please have a quick look? Thanks!

Taking a look

Reverted, will fix and push again

llvm/docs/UserGuides.rst

.. toctree::
    Remarks
    SourceLevelDebugging
    StackSafetyAnalysis
    SupportLibrary
    TableGen/index
    TableGenFundamentals
    Vectorizers
    WritingAnLLVMPass
+   WritingAnLLVMNewPMPass
    WritingAnLLVMBackend
    yaml2obj
 
 Clang
 -----
 
 :doc:`HowToBuildOnARM`
    Notes on building and testing LLVM/Clang on ARM.
:doc:`AdvancedBuilds`
    This document describes more advanced build configurations.
 
 Optimizations
 -------------
 
 :doc:`WritingAnLLVMPass`
    Information on how to write LLVM transformations and analyses.
 
+:doc:`WritingAnLLVMNewPMPass`
+   Information on how to write LLVM transformations under the new pass
+   manager.
+
 :doc:`Passes`
    A list of optimizations and analyses implemented in LLVM.
 
 :doc:`StackSafetyAnalysis`
   This document describes the design of the stack safety analysis of local
   variables.
 
 :doc:`MergeFunctions`

llvm/docs/WritingAnLLVMNewPMPass.rst

+====================
+Writing an LLVM Pass
+====================
+
+.. program:: opt
+
+.. contents::
+    :local:
+
+Introduction --- What is a pass?
+================================
+
+The LLVM pass framework is an important part of the LLVM system, because LLVM
+passes are where most of the interesting parts of the compiler exist. Passes
+perform the transformations and optimizations that make up the compiler, they
+build the analysis results that are used by these transformations, and they
+are, above all, a structuring technique for compiler code.
+
+Unlike passes under the legacy pass manager where the pass interface is
+defined via inheritance, passes under the new pass manager rely on
+concept-based polymorphism, meaning there is no explicit interface (see
+comments in ``PassManager.h`` for more details). All LLVM passes inherit from
+the CRTP mix-in ``PassInfoMixin<PassT>``. The pass should have a ``run()``
+method which returns a ``PreservedAnalyses`` and takes in some unit of IR
+along with an analysis manager. For example, a function pass would have a
+``PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);`` method.
+
+We start by showing you how to construct a pass, from setting up the build,
+creating the pass, to executing and testing it. Looking at existing passes is
+always a great way to learn details.
+
+Quick Start --- Writing hello world
+===================================
+
+Here we describe how to write the "hello world" of passes. The "HelloWorld"
+pass is designed to simply print out the name of non-external functions that
+exist in the program being compiled. It does not modify the program at all,
+it just inspects it.
+
+The code below already exists; feel free to create a pass with a different
+name alongside the HelloWorld source files.
+
+.. _writing-an-llvm-npm-pass-build:
+
+Setting up the build
+--------------------
+
+First, configure and build LLVM as described in :doc:`GettingStarted`.
+
+Next, we will reuse an existing directory (creating a new directory involves
+modifying more ``CMakeLists.txt``s and ``LLVMBuild.txt``s than we want). For
+this example, we'll use ``llvm/lib/Transforms/HelloNew/HelloWorld.cpp``,
+which has already been created. If you'd like to create your own pass, add a
+new source file into ``llvm/lib/Transforms/HelloNew/CMakeLists.txt`` under
+``HelloWorld.cpp``:
+
+.. code-block:: cmake
+
+  add_llvm_component_library(LLVMHelloWorld
+    HelloWorld.cpp
+
+    DEPENDS
+    intrinsics_gen
+    )
+
+Now that we have the build set up for a new pass, we need to write the code
+for the pass itself.
+
+.. _writing-an-llvm-npm-pass-basiccode:
+
+Basic code required
+-------------------
+
+Now that the build is setup for a new pass, we just have to write it.
+
+First we need to define the pass in a header file. We'll create
+``llvm/include/llvm/Transforms/HelloNew/HelloWorld.h``. The file should
+contain the following boilerplate:
+
+.. code-block:: c++
+
+  #ifndef LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H
+  #define LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H
+
+  #include "llvm/IR/PassManager.h"
+
+  namespace llvm {
+
+  class HelloWorldPass : public PassInfoMixin<HelloWorldPass> {
+  public:
+    PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);
+  };
+
+  } // namespace llvm
+
+  #endif // LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H
+
+This creates the class for the pass with a declaration of the ``run()``
+method which actually runs the pass. Inheriting from ``PassInfoMixin<PassT>``
+sets up some more boilerplate so that we don't have to write it ourselves.
+
+Our class is in the ``llvm`` namespace so that we don't pollute the global
+namespace.
+
+Next we'll create ``llvm/lib/Transforms/HelloNew/HelloWorld.cpp``, starting
+with
+
+.. code-block:: c++
+
+  #include "llvm/Transforms/HelloNew/HelloWorld.h"
+
+... to include the header file we just created.
+
+.. code-block:: c++
+
+  using namespace llvm;
+
+... is required because the functions from the include files live in the llvm
+namespace. This should only be done in non-header files.
+
+Next we have the pass's ``run()`` definition:
+
+.. code-block:: c++
+
+  PreservedAnalyses HelloWorldPass::run(Function &F,
+                                        FunctionAnalysisManager &AM) {
+    errs() << F.getName() << "\n";
+    return PreservedAnalyses::all();
+  }
+
+... which simply prints out the name of the function to stderr. The pass
+manager will ensure that the pass will be run on every function in a module.
+The ``PreservedAnalyses`` return value says that all analyses (e.g. dominator
+tree) are still valid after this pass since we didn't modify any functions.
+
+That's it for the pass itself. Now in order to "register" the pass, we need
+to add it to a couple places. Add the following to
+``llvm\lib\Passes\PassRegistry.def`` in the ``FUNCTION_PASS`` section
+
+.. code-block:: c++
+
+  FUNCTION_PASS("helloworld", HelloWorldPass())
+
+... which adds the pass under the name "helloworld".
+
+``llvm\lib\Passes\PassRegistry.def`` is #include'd into
+``llvm\lib\Passes\PassBuilder.cpp`` multiple times for various reasons. Since
+it constructs our pass, we need to also add the proper #include in
+``llvm\lib\Passes\PassBuilder.cpp``:
+
+.. code-block:: c++
+
+  #include "llvm/Transforms/HelloNew/HelloWorld.h"
+
+This should be all the code necessary for our pass, now it's time to compile
+and run it.
+
+Running a pass with ``opt``
+---------------------------
+
+Now that you have a brand new shiny pass, we can build :program:`opt` and use
+it to run some LLVM IR through the pass.
+
+.. code-block:: console
+
+  $ ninja -C build/ opt
+  # or whatever build system/build directory you are using
+
+  $ cat /tmp/a.ll
+  define i32 @foo() {
+    %a = add i32 2, 3
+    ret i32 %a
+  }
+
+  define void @bar() {
+    ret void
+  }
+
+  $ build/bin/opt -disable-output /tmp/a.ll -passes=helloworld
+  foo
+  bar
+
+Our pass ran and printed the names of functions as expected!
+
+Testing a pass
+--------------
+
+Testing our pass is important to prevent future regressions. We'll add a lit
+test at ``llvm/test/Transforms/HelloNew/helloworld.ll``. See
+:doc:`TestingGuide` for more information on testing.
+
+.. code-block:: llvm
+
+  $ cat llvm/test/Transforms/HelloNew/helloworld.ll
+  ; RUN: opt -disable-output -passes=helloworld %s 2>&1 | FileCheck %s
+
+  ; CHECK: {{^}}foo{{$}}
+  define i32 @foo() {
+    %a = add i32 2, 3
+    ret i32 %a
+  }
+
+  ; CHECK-NEXT: {{^}}bar{{$}}
+  define void @bar() {
+    ret void
+  }
+
+  $ ninja -C build check-llvm
+  # runs our new test alongside all other llvm lit tests

llvm/docs/WritingAnLLVMPass.rst

 passes.  One of the main features of the LLVM Pass Framework is that it
 schedules passes to run in an efficient way based on the constraints that your
 pass meets (which are indicated by which class they derive from).
 
 We start by showing you how to construct a pass, everything from setting up the
 code, to compiling, loading, and executing it.  After the basics are down, more
 advanced features are discussed.
 
+This document deals with the legacy pass manager. LLVM is transitioning to
+the new pass manager, which has its own way of defining passes. For more
+details, see :doc:`WritingAnLLVMNewPMPass`.
+
 Quick Start --- Writing hello world
 ===================================
 
 Here we describe how to write the "hello world" of passes.  The "Hello" pass is
 designed to simply print out the name of non-external functions that exist in
 the program being compiled.  It does not modify the program at all, it just
 inspects it.  The source code and files for this pass are available in the LLVM
 source tree in the ``lib/Transforms/Hello`` directory.

llvm/include/llvm/Transforms/HelloNew/HelloWorld.h

+//===-- HelloWorld.h - Example Transformations ------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H
+#define LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+class HelloWorldPass : public PassInfoMixin<HelloWorldPass> {
+public:
+  PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);
+};
+
+} // namespace llvm
+
+#endif // LLVM_TRANSFORMS_HELLONEW_HELLOWORLD_H

llvm/lib/Passes/LLVMBuild.txt

 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [component_0]
 type = Library
 name = Passes
 parent = Libraries
-required_libraries = AggressiveInstCombine Analysis Core Coroutines IPO InstCombine ObjCARC Scalar Support Target TransformUtils Vectorize Instrumentation
+required_libraries = AggressiveInstCombine Analysis Core Coroutines HelloNew IPO InstCombine ObjCARC Scalar Support Target TransformUtils Vectorize Instrumentation

llvm/lib/Passes/PassBuilder.cpp

 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/Regex.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/AggressiveInstCombine/AggressiveInstCombine.h"
 #include "llvm/Transforms/Coroutines/CoroCleanup.h"
 #include "llvm/Transforms/Coroutines/CoroEarly.h"
 #include "llvm/Transforms/Coroutines/CoroElide.h"
 #include "llvm/Transforms/Coroutines/CoroSplit.h"
+#include "llvm/Transforms/HelloNew/HelloWorld.h"
 #include "llvm/Transforms/IPO/AlwaysInliner.h"
 #include "llvm/Transforms/IPO/ArgumentPromotion.h"
 #include "llvm/Transforms/IPO/Attributor.h"
 #include "llvm/Transforms/IPO/CalledValuePropagation.h"
 #include "llvm/Transforms/IPO/ConstantMerge.h"
 #include "llvm/Transforms/IPO/CrossDSOCFI.h"
 #include "llvm/Transforms/IPO/DeadArgumentElimination.h"
 #include "llvm/Transforms/IPO/ElimAvailExtern.h"

llvm/lib/Passes/PassRegistry.def

 FUNCTION_PASS("dot-cfg", CFGPrinterPass())
 FUNCTION_PASS("dot-cfg-only", CFGOnlyPrinterPass())
 FUNCTION_PASS("early-cse", EarlyCSEPass(/*UseMemorySSA=*/false))
 FUNCTION_PASS("early-cse-memssa", EarlyCSEPass(/*UseMemorySSA=*/true))
 FUNCTION_PASS("ee-instrument", EntryExitInstrumenterPass(/*PostInlining=*/false))
 FUNCTION_PASS("make-guards-explicit", MakeGuardsExplicitPass())
 FUNCTION_PASS("post-inline-ee-instrument", EntryExitInstrumenterPass(/*PostInlining=*/true))
 FUNCTION_PASS("gvn-hoist", GVNHoistPass())
+FUNCTION_PASS("helloworld", HelloWorldPass())
 FUNCTION_PASS("instcombine", InstCombinePass())
 FUNCTION_PASS("instcount", InstCountPass())
 FUNCTION_PASS("instsimplify", InstSimplifyPass())
 FUNCTION_PASS("invalidate<all>", InvalidateAllAnalysesPass())
 FUNCTION_PASS("irce", IRCEPass())
 FUNCTION_PASS("float2int", Float2IntPass())
 FUNCTION_PASS("no-op-function", NoOpFunctionPass())
 FUNCTION_PASS("libcalls-shrinkwrap", LibCallsShrinkWrapPass())

llvm/lib/Transforms/CMakeLists.txt

 add_subdirectory(Utils)
 add_subdirectory(Instrumentation)
 add_subdirectory(AggressiveInstCombine)
 add_subdirectory(InstCombine)
 add_subdirectory(Scalar)
 add_subdirectory(IPO)
 add_subdirectory(Vectorize)
 add_subdirectory(Hello)
+add_subdirectory(HelloNew)
 add_subdirectory(ObjCARC)
 add_subdirectory(Coroutines)
 add_subdirectory(CFGuard)

llvm/lib/Transforms/HelloNew/CMakeLists.txt

+add_llvm_component_library(LLVMHelloNew
+  HelloWorld.cpp
+
+  DEPENDS
+  intrinsics_gen
+  )

llvm/lib/Transforms/HelloNew/HelloWorld.cpp

+//===-- HelloWorld.cpp - Example Transformations --------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/HelloNew/HelloWorld.h"
+
+using namespace llvm;
+
+PreservedAnalyses HelloWorldPass::run(Function &F,
+                                      FunctionAnalysisManager &AM) {
+  errs() << F.getName() << "\n";
+  return PreservedAnalyses::all();
+}

llvm/lib/Transforms/HelloNew/LLVMBuild.txt

-;===- ./lib/Transforms/LLVMBuild.txt ---------------------------*- Conf -*--===;
+;===- ./lib/Transforms/HelloNew/LLVMBuild.txt ------------------*- Conf -*--===;
 ;
 ; Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 ; See https://llvm.org/LICENSE.txt for license information.
 ; SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 ;
 ;===------------------------------------------------------------------------===;
 ;
 ; This is an LLVMBuild description file for the components in this subdirectory.
 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
-[common]
-subdirectories = AggressiveInstCombine Coroutines IPO InstCombine Instrumentation Scalar Utils Vectorize ObjCARC CFGuard
-
 [component_0]
-type = Group
-name = Transforms
-parent = Libraries
+type = Library
+name = HelloNew
+parent = Transforms
+library_name = HelloNew
+required_libraries = Core

llvm/lib/Transforms/LLVMBuild.txt

 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [common]
-subdirectories = AggressiveInstCombine Coroutines IPO InstCombine Instrumentation Scalar Utils Vectorize ObjCARC CFGuard
+subdirectories = AggressiveInstCombine Coroutines HelloNew IPO InstCombine Instrumentation Scalar Utils Vectorize ObjCARC CFGuard
 
 [component_0]
 type = Group
 name = Transforms
 parent = Libraries

llvm/test/Transforms/HelloNew/helloworld.ll

+; RUN: opt -disable-output -passes=helloworld %s 2>&1 | FileCheck %s
+
+; CHECK: {{^}}foo{{$}}
+define i32 @foo() {
+  %a = add i32 2, 3
+  ret i32 %a
+}
+
+; CHECK-NEXT: {{^}}bar{{$}}
+define void @bar() {
+  ret void
+}

llvm/utils/gn/secondary/llvm/lib/Passes/BUILD.gn

 static_library("Passes") {
   output_name = "LLVMPasses"
   deps = [
     "//llvm/lib/Analysis",
     "//llvm/lib/CodeGen",
     "//llvm/lib/IR",
     "//llvm/lib/Support",
     "//llvm/lib/Target",
     "//llvm/lib/Transforms/AggressiveInstCombine",
     "//llvm/lib/Transforms/Coroutines",
+    "//llvm/lib/Transforms/HelloNew",
     "//llvm/lib/Transforms/IPO",
     "//llvm/lib/Transforms/InstCombine",
     "//llvm/lib/Transforms/Instrumentation",
     "//llvm/lib/Transforms/ObjCARC",
     "//llvm/lib/Transforms/Scalar",
     "//llvm/lib/Transforms/Utils",
     "//llvm/lib/Transforms/Vectorize",
   ]
   sources = [
     "PassBuilder.cpp",
     "PassPlugin.cpp",
     "StandardInstrumentations.cpp",
   ]
 }

llvm/utils/gn/secondary/llvm/lib/Transforms/HelloNew/BUILD.gn

+static_library("HelloNew") {
+  output_name = "LLVMHelloNew"
+  deps = [
+    "//llvm/lib/Analysis",
+    "//llvm/lib/IR",
+    "//llvm/lib/Support",
+  ]
+  sources = [ "HelloWorld.cpp" ]
+}