[mlir][sparse] add more unittest cases to sparse dialect merger

[bixia]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

79–98

Nice! Shall we combine BIN_PATTERN and BIN_EXP, to avoid repeating this list?

[Peiming]

Merge BIN_PATTERN and BIN_EXP

[Peiming]

rollback change by mistake

[aartbik]

Good work, I like the extension of our testing a lot!
A few nits on style.

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

64

why is this empty line gone? I like vertical space around the section-separating comments

(attached comments usually apply to code below directly)

79–98

+1, or use a FOR construct that Wren introduced in our library header (search for FOREVERY_ .... for inspiration)

111

add comment on new parameter

244

Make this a section separation, i.e.

/
/ Tests with all sparse inputs.
///

or, if this applies to the class, then attach, but add comment to MergerTest4T1L

335–336

remove this period, since sentence continues

[Peiming]

This comment was removed by Peiming.

[Peiming]

use FOREVERY_ marco to remove repeated code + improve comments

[wrengr]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

43–48

I would rename this to FOREVERY_BINOP, since this is just for the binops not for Ops with other arities.

44–55

You should pass the second arguments as Kind::kFooT rather than just as kFooT. That way all the users of this x-macro can simply use the KIND parameter they're given rather than needing to say Kind::KIND. Any sort of code repetition like needing to say Kind::KIND over and over is a source of potential bugs from typos, and so should be avoided.

57

Macro parameters should also be uppercase. Ditto for all the other macros you define below

[Peiming]

Use uppercase for macro parameters

[Peiming]

Fix typo

[wrengr]

Getting there! :) I have a lot of duplicate comments, but wanted to make sure you got them all before I do a final pass of reviewing

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

57

In C/C++ jargon, this isn't a "declaration" but rather a "definition"; so even though your macro's name makes sense colloquially, you might want to rename it to avoid confusion. In other places we use DECL_FOO for macros that declare something, and IMPL_FOO for macros that implement/define something.

...Though for this one I'm not sure IMPL_PATTERN is a terribly good name either. Perhaps IMPL_BINOP_PATTERN?

82

Same issue re "declare". Perhaps use IMPL_BINOP_ADDEXP?

83

I'm not terribly comfortable using the OP directly here since it seems likely to cause name clashing. Perhaps use OP##Expr instead?

(You could use OP##Exp to remain closer to the style in the rest of this file, though personally I'm not a fan of using "exp" to abbreviate "expression", since there's too much potential for confusion with "exp" meaning "exponential". Then again I've worked on a lot of compilers that do a lot of work with exponentials/logarithms.)

342–365

Should capitalize the macro parameter here too

349–364

I think it'd clean up this code considerably to float out variable bindings for auto p0 = tensorPattern(t0) and ditto for t1

367–371

To avoid repeating things here, You should break up the FOREVERY_BINOP x-macro into two separate macros: FOREVERY_DISJ_BINOP and FOREVERY_CONJ_BINOP. Then you can just call FOREVERY_DISJ_BINOP(MERGER_TEST_DISJ), where you update the MERGER_TEST_DISJ macro to take the extra KIND parameter and just ignore it.

For other places where you really do want to iterate over all the binops, you can define the old combined x-macro in terms of the two new ones: #define FOREVERY_BINOP(DO) FOREVERY_DISJ_BINOP(DO) FOREVERY_CONJ_BINOP(DO)

You can of course repeat the idea in order to split FOREVERY_DISJ_BINOP up into FOREVERY_COMMUTATIVE_DISJ_BINOP and FOREVERY_NONCOMMUTATIVE_DISJ_BINOP, in order to distinguish subtraction from addition/or as mentioned in your todo below. Or if commutativity isn't the most logical way to split things up, then you could instead do a three-way split into FOREVERY_ADD_BINOP, FOREVERY_OR_BINOP, and FOREVERY_SUB_BINOP.

386–647

Capitalize the OP parameter here too

420

capitalize macro arguments here too

473

ditto

482

ditto

538

ditto. Also, for consistency with the other tests you may want to rename them to CONJ1 and CONJ2 (or CONJ0 and CONJ1)

547

ditto

581

ditto

581

"opted" is a confusing and unnecessary abbreviation, just spell out "optimized"

588

ditto, though it's less of an issue here than in the previous few tests

623

ditto

623

ditto

[Peiming]

Reduce code repetition, address comments from Wren

[aartbik]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

44

This is very nitpicky (but this is how we are ;-), but I would like to see a consistent order on the type suffix.
We always use the order F,C,I

152

This too needs to make the bits/simple distinction clear in the doc, like L104

327

I think I have a slight preference for putting all FOR macros together, since you already define one up, since readers are typically searching for #defines at the top of the file. Perhaps you can even start this file with

Iteration macros.
//

#define FOR1 ...

Note that I only mean moving the FOR loops up. The actual IMPL macros and applying FOR to them should of course remain the in proper section, since it expands to code at that place

#define FOR2 ...

etc.

342–365

So, just to be clear, this define + FOR applied to it, then undefine, remains here.

[aartbik]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

327

The online tool messed up my suggested code. Let me try with formatting

//
// Iteration macros.
//

#define FOR1 ...
#define FOR2 ...
etc.

//
// Next section
//

[wrengr]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

82

I like the name IMPL_BINOP_EXPR :)

327

+1 for grouping all the FOREVERY x-macros together in one spot, and ideally all at the top of the file.

Remember, x-macros are defining groups of things that we always want to handle together. So it's important to give all the x-macros together in one spot so that readers can look at them and say "ah yes, these are the natural groupings of things" (e.g., conjunctive vs disjunctive things), so they know how to think about them before getting into the various places where the x-macros are called.

330–344

These two x-macros are redundant with the ones which take (TEST, OP). Remember, just because you pass OP to TEST doesn't mean that TEST actually needs to use it.

In fact, I'd reduce things further by combining the (TEST, OP) variants with the FOREVERY_BINOP xmacro. That is, you should only need to define:

#define FOREVERY_BINOP(DO, ...) \
    DO(mulf, Kind::kMulF, __VA_ARGS__) \
    DO(mulc, Kind::kMulC, __VA_ARGS__) \
    and so on and so forth

And then you can call it à la:

#define X(OP, KIND, FOO, BAR) \
   whatever you want to do
FOREVERY_BINOP(X, foo, bar)
#undef X

Notably, FOREVERY_BINOP only provides the OP and KIND arguments of DO, and whatever else you pass into FOREVERY_BINOP will simply be passed along to DO. Which keeps FOREVERY_BINOP generic and not forced to know anything about the extra arguments that DO might want. On the other side, X isn't restricted to only taking in the OP and KIND parameters, but can take in whatever other parameters it might like. Moreover, by making the xmacro variadic like this, you can nest calls to xmacros (e.g., to handle the tests which have two or more ops); though it takes a little bit of boilerplate to do that.

https://en.wikipedia.org/wiki/X_Macro
https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html

[Peiming]

Address Aart's comments

[Peiming]

Further reduce code

[Peiming]

Adding missing period..

[Peiming]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

44

haha, will do!

If you look closer, you might find that they are already ordered as F,C,I (there is a kAndI in between of kMulI and kAddF), I will reordered it so that kAddI appears at the end of the list

327

Good Suggestions!

330–344

I am not sure whether I understand your point, since the (TEST, OP) macro is used to generate the production of two set (pairs of conj and disj, etc).

However, I do find that __VA_ARGS__ can help to remove those marcos, if it is not what you are referring to, just let me know!

367–371

I renamed them to FOREVERY_COMMON_XX_BINOP as they do not require special handling when doing pattern matching, etc. But I am not sure whether it is a good name.

Commutativity is not the logical way to split them up as matrix multiplication is also noncommunitive but it does not require special handling here.

Let me know if you have better name suggestion!

[aartbik]

I am okay with the revision, but please wait for Wren's approval too, since she had a lot of feedback, and may have some more.

[wrengr]

The x-macros could still be cleaned up a bit, but I think it'd be easier to do myself than try to explain what I mean, so let's do that in a separate CL.

Other than that it's looking good! Once you remove the inline keyword, ship it! :)

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

104

All method definitions given within a class declaration are implicitly inline, so you don't need the keyword and should remove it.

Also, it's not clear to me why this is supposed to help improve readability of tests; but since this method (and comment) was there from before, we can address that in a separate CL.

[Peiming]

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

104

Oh, good to know!

[Peiming]

remove inline

[Peiming]

use full diff

[brooksmoses]

I'm seeing test warning/errors from this:

external/llvm-project/mlir/unittests/Dialect/SparseTensor/MergerTest.cpp:112:16: error: unused function 'subfPattern' [-Werror,-Wunused-function]
FOREVERY_BINOP(IMPL_BINOP_PATTERN)

^

external/llvm-project/mlir/unittests/Dialect/SparseTensor/MergerTest.cpp:112:16: error: unused function 'subcPattern' [-Werror,-Wunused-function]
external/llvm-project/mlir/unittests/Dialect/SparseTensor/MergerTest.cpp:112:16: error: unused function 'subiPattern' [-Werror,-Wunused-function]
3 errors generated.

[wrengr]

Should be able to fix that by adding LLVM_ATTRIBUTE_UNUSED from llvm/Support/Compiler.h. I'll send out a differential to do so

[wrengr]

@brooksmoses Just posted D129027. (Am currently building it to see if any CMake changes are necessary)

[brooksmoses]

@wrengr : Looks fixed; thanks!

mlir/unittests/Dialect/SparseTensor/MergerTest.cpp

 /// Readable Pattern builder functions.
 /// These should be preferred over the actual constructors.
 ///
 
 static std::shared_ptr<Pattern> tensorPattern(unsigned tensorNum) {
   return std::make_shared<Pattern>(tensorNum);
 }
 
-static std::shared_ptr<Pattern>
-addfPattern(const std::shared_ptr<Pattern> &e0,
-            const std::shared_ptr<Pattern> &e1) {
-  return std::make_shared<Pattern>(Kind::kAddF, e0, e1);
-}
+#define BIN_PATTERN(op, kind)                                                  \
+  static std::shared_ptr<Pattern> op##Pattern(                                 \
+      const std::shared_ptr<Pattern> &e0,                                      \
+      const std::shared_ptr<Pattern> &e1) {                                    \
+    return std::make_shared<Pattern>(Kind::kind, e0, e1);                      \
+  }
+
+BIN_PATTERN(mulf, kMulF)
+BIN_PATTERN(mulc, kMulC)
+BIN_PATTERN(muli, kMulF)
+BIN_PATTERN(andi, kAndI)
+BIN_PATTERN(addf, kAddF)
+BIN_PATTERN(addc, kAddC)
+BIN_PATTERN(addi, kAddI)
+BIN_PATTERN(subf, kSubF)
+BIN_PATTERN(subc, kSubC)
+BIN_PATTERN(subi, kSubI)
+BIN_PATTERN(ori, kOrI)
+BIN_PATTERN(xori, kXorI)
 
-static std::shared_ptr<Pattern>
-mulfPattern(const std::shared_ptr<Pattern> &e0,
-            const std::shared_ptr<Pattern> &e1) {
-  return std::make_shared<Pattern>(Kind::kMulF, e0, e1);
-}
+#undef BIN_PATTERN
 
 class MergerTestBase : public ::testing::Test {
 protected:
   MergerTestBase(unsigned numTensors, unsigned numLoops)
       : numTensors(numTensors), numLoops(numLoops),
         merger(numTensors, numLoops) {}
 
   ///
   /// Expression construction helpers.
   ///
 
   unsigned tensor(unsigned tensor) {
     return merger.addExp(Kind::kTensor, tensor);
   }
 
-  unsigned addf(unsigned e0, unsigned e1) {
-    return merger.addExp(Kind::kAddF, e0, e1);
-  }
+#define BIN_EXP(op, kind)                                                      \
+  unsigned op(unsigned e0, unsigned e1) {                                      \
+    return merger.addExp(Kind::kind, e0, e1);                                  \
+  }
+
+  BIN_EXP(mulf, kMulF)
+  BIN_EXP(mulc, kMulC)
+  BIN_EXP(muli, kMulF)
+  BIN_EXP(andi, kAndI)
+  BIN_EXP(addf, kAddF)
+  BIN_EXP(addc, kAddC)
+  BIN_EXP(addi, kAddI)
+  BIN_EXP(subf, kSubF)
+  BIN_EXP(subc, kSubC)
+  BIN_EXP(subi, kSubI)
+  BIN_EXP(ori, kOrI)
+  BIN_EXP(xori, kXorI)
 
-  unsigned mulf(unsigned e0, unsigned e1) {
-    return merger.addExp(Kind::kMulF, e0, e1);
-  }
+#undef BIN_EXP
 
   ///
   /// Comparison helpers.
   ///
 
   /// For readability of tests.
   unsigned lat(unsigned lat) { return lat; }
 
   /// Returns true if a lattice point with an expression matching the given
   /// pattern and bits matching the given bits is present in lattice points
   /// [p, p+n) of lattice set s. This is useful for testing partial ordering
   /// constraints between lattice points. We generally know how contiguous
   /// groups of lattice points should be ordered with respect to other groups,
   /// but there is no required ordering within groups.
   bool latPointWithinRange(unsigned s, unsigned p, unsigned n,
                            const std::shared_ptr<Pattern> &pattern,
-                           const BitVector &bits) {
+                           const BitVector &bits, bool simple) {
     for (unsigned i = p; i < p + n; ++i) {
       if (compareExpression(merger.lat(merger.set(s)[i]).exp, pattern) &&
-          compareBits(s, i, bits))
+          compareBits(s, i, bits, simple))
         return true;
     }
     return false;
   }
 
   /// Wrapper over latPointWithinRange for readability of tests.
+  /// if simple is true compare the lat.simple field instead to test the
+  /// result after optimization
   void expectLatPointWithinRange(unsigned s, unsigned p, unsigned n,
                                  const std::shared_ptr<Pattern> &pattern,
-                                 const BitVector &bits) {
-    EXPECT_TRUE(latPointWithinRange(s, p, n, pattern, bits));
+                                 const BitVector &bits, bool simple = false) {
+    EXPECT_TRUE(latPointWithinRange(s, p, n, pattern, bits, simple));
   }
 
   /// Wrapper over expectLatPointWithinRange for a single lat point.
   void expectLatPoint(unsigned s, unsigned p,
                       const std::shared_ptr<Pattern> &pattern,
-                      const BitVector &bits) {
-    EXPECT_TRUE(latPointWithinRange(s, p, 1, pattern, bits));
+                      const BitVector &bits, bool simple = false) {
+    EXPECT_TRUE(latPointWithinRange(s, p, 1, pattern, bits, simple));
   }
 
   /// Converts a vector of (loop, tensor) pairs to a bitvector with the
   /// corresponding bits set.
   BitVector
   loopsToBits(const std::vector<std::pair<unsigned, unsigned>> &loops) {
     BitVector testBits = BitVector(numTensors + 1, false);
     for (auto l : loops) {
       auto loop = std::get<0>(l);
       auto tensor = std::get<1>(l);
       testBits.set(numTensors * loop + tensor);
     }
     return testBits;
   }
 
   /// Returns true if the bits of lattice point p in set s match the given bits.
-  bool compareBits(unsigned s, unsigned p, const BitVector &bits) {
+  bool compareBits(unsigned s, unsigned p, const BitVector &bits, bool simple) {
+    if (simple)
+      return merger.lat(merger.set(s)[p]).simple == bits;
     return merger.lat(merger.set(s)[p]).bits == bits;
   }
 
   /// Check that there are n lattice points in set s.
   void expectNumLatPoints(unsigned s, unsigned n) {
     EXPECT_THAT(merger.set(s).size(), n);
   }
 
bool compareExpression(unsigned e, const std::shared_ptr<Pattern> &pattern) {
     llvm_unreachable("unexpected kind");
   }
 
   unsigned numTensors;
   unsigned numLoops;
   Merger merger;
 };
 
+///
+/// Tests with all sparse inputs.
+///
+
 class MergerTest3T1L : public MergerTestBase {
 protected:
   // Our three tensors (two inputs, one output).
   const unsigned t0 = 0, t1 = 1, t2 = 2;
 
   // Our single loop.
   const unsigned l0 = 0;
 
   MergerTest3T1L() : MergerTestBase(3, 1) {
     // Tensor 0: sparse input vector.
     merger.addExp(Kind::kTensor, t0, -1u);
     merger.setDim(t0, l0, Dim::kSparse);
 
     // Tensor 1: sparse input vector.
     merger.addExp(Kind::kTensor, t1, -1u);
     merger.setDim(t1, l0, Dim::kSparse);
 
     // Tensor 2: dense output vector.
     merger.addExp(Kind::kTensor, t2, -1u);
     merger.setDim(t2, l0, Dim::kDense);
   }
 };
 
+class MergerTest4T1L : public MergerTestBase {
+protected:
+  // Our four tensors (three inputs, one output).
+  const unsigned t0 = 0, t1 = 1, t2 = 2, t3 = 3;
+
+  // Our single loop.
+  const unsigned l0 = 0;
+
+  MergerTest4T1L() : MergerTestBase(4, 1) {
+    // Tensor 0: sparse input vector.
+    merger.addExp(Kind::kTensor, t0, -1u);
+    merger.setDim(t0, l0, Dim::kSparse);
+
+    // Tensor 1: sparse input vector.
+    merger.addExp(Kind::kTensor, t1, -1u);
+    merger.setDim(t1, l0, Dim::kSparse);
+
+    // Tensor 2: sparse input vector
+    merger.addExp(Kind::kTensor, t2, -1u);
+    merger.setDim(t2, l0, Dim::kSparse);
+
+    // Tensor 3: dense output vector
+    merger.addExp(Kind::kTensor, t3, -1u);
+    merger.setDim(t3, l0, Dim::kDense);
+  }
+};
+
+///
+/// Tests with both sparse and dense input.
+///
+
+class MergerTest3T1LD : public MergerTestBase {
+protected:
+  // Our three tensors (two inputs, one output).
+  const unsigned t0 = 0, t1 = 1, t2 = 2;
+
+  // Our single loop.
+  const unsigned l0 = 0;
+
+  MergerTest3T1LD() : MergerTestBase(3, 1) {
+    // Tensor 0: sparse input vector.
+    merger.addExp(Kind::kTensor, t0, -1u);
+    merger.setDim(t0, l0, Dim::kSparse);
+
+    // Tensor 1: dense input vector.
+    merger.addExp(Kind::kTensor, t1, -1u);
+    merger.setDim(t1, l0, Dim::kDense);
+
+    // Tensor 2: dense output vector.
+    merger.addExp(Kind::kTensor, t2, -1u);
+    merger.setDim(t2, l0, Dim::kDense);
+  }
+};
+
 } // namespace
 
-/// Vector addition of 2 vectors, i.e.:
+/// Vector addition (disjunction) of 2 vectors. i.e.;
 ///   a(i) = b(i) + c(i)
 /// which should form the 3 lattice points
 /// {
 ///   lat( i_00 i_01 / (tensor_0 + tensor_1) )
 ///   lat( i_00 / tensor_0 )
 ///   lat( i_01 / tensor_1 )
 /// }
-/// and after optimization, will reduce to the 2 lattice points
+/// and after optimization, the lattice points do not change (as there is no
+/// duplicated point and all input vectors are sparse vector).
 /// {
 ///   lat( i_00 i_01 / (tensor_0 + tensor_1) )
 ///   lat( i_00 / tensor_0 )
+///   lat( i_01 / tensor_1 )
 /// }
-TEST_F(MergerTest3T1L, VectorAdd2) {
-  // Construct expression.
-  auto e = addf(tensor(t0), tensor(t1));
-
-  // Build lattices and check.
-  auto s = merger.buildLattices(e, l0);
-  expectNumLatPoints(s, 3);
-  expectLatPoint(s, lat(0), addfPattern(tensorPattern(t0), tensorPattern(t1)),
-                 loopsToBits({{l0, t0}, {l0, t1}}));
-  expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t0),
-                            loopsToBits({{l0, t0}}));
-  expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t1),
-                            loopsToBits({{l0, t1}}));
-
-  // Optimize lattices and check.
-  s = merger.optimizeSet(s);
-  expectNumLatPoints(s, 3);
-  expectLatPoint(s, lat(0), addfPattern(tensorPattern(t0), tensorPattern(t1)),
-                 loopsToBits({{l0, t0}, {l0, t1}}));
-  expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t0),
-                            loopsToBits({{l0, t0}}));
-  expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t1),
-                            loopsToBits({{l0, t1}}));
+#define MERGER_TEST_DISJ(op)                                                   \
+  TEST_F(MergerTest3T1L, vector_##op) {                                        \
+    auto e = op(tensor(t0), tensor(t1));                                       \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 3);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}));                         \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t0),                 \
+                              loopsToBits({{l0, t0}}));                        \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t1),                 \
+                              loopsToBits({{l0, t1}}));                        \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 3);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}), true);                   \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t0),                 \
+                              loopsToBits({{l0, t0}}), true);                  \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t1),                 \
+                              loopsToBits({{l0, t1}}), true);                  \
   }
+
+MERGER_TEST_DISJ(addf)
+MERGER_TEST_DISJ(addc)
+MERGER_TEST_DISJ(addi)
+MERGER_TEST_DISJ(ori)
+MERGER_TEST_DISJ(xori)
+
+#undef MERGER_TEST_DISJ
+// TODO :: pattern for substraction is different as it is mapped to negate
+// operation.
+// MERGER_TEST_DISJ(subf)
+// MERGER_TEST_DISJ(subc)
+// MERGER_TEST_DISJ(subi)
 
-/// Vector multiplication of 2 vectors, i.e.:
+/// Vector multiplication (conjunction) of 2 vectors, i.e.;
 ///   a(i) = b(i) * c(i)
 /// which should form the single lattice point
 /// {
 ///   lat( i_00 i_01 / (tensor_0 * tensor_1) )
 /// }
-TEST_F(MergerTest3T1L, VectorMul2) {
-  // Construct expression.
-  auto e = mulf(t0, t1);
-
-  // Build lattices and check.
-  auto s = merger.buildLattices(e, l0);
-  expectNumLatPoints(s, 1);
-  expectLatPoint(s, lat(0), mulfPattern(tensorPattern(t0), tensorPattern(t1)),
-                 loopsToBits({{l0, t0}, {l0, t1}}));
-
-  // Optimize lattices and check.
-  s = merger.optimizeSet(s);
-  expectNumLatPoints(s, 1);
-  expectLatPoint(s, lat(0), mulfPattern(tensorPattern(t0), tensorPattern(t1)),
-                 loopsToBits({{l0, t0}, {l0, t1}}));
+#define MERGER_TEST_CONJ(op)                                                   \
+  TEST_F(MergerTest3T1L, vector_##op) {                                        \
+    auto e = op(t0, t1);                                                       \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}));                         \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}), true);                   \
   }
+
+// TODO: div are not tested for now as it need a constant non-zero dividend.
+MERGER_TEST_CONJ(mulf)
+MERGER_TEST_CONJ(mulc)
+MERGER_TEST_CONJ(muli)
+MERGER_TEST_CONJ(andi)
+
+#undef MERGER_TEST_CONJ
+
+/// Vector multiplication (conjunction) then addition (disjunction), i.e.;
+///   a(i) = b(i) * c(i) + d(i);
+/// which should form
+/// {
+///    lat( i_00 i_01 i_02 / (tensor_0 * tensor_1) + tensor_2 )
+///    lat( i_00 i_01 / tensor_0 * tensor_1
+///    lat( i_02 / tensor_2 )
+/// }
+
+#define MERGER_TEST_CONJ_DISJ(conj, disj)                                      \
+  TEST_F(MergerTest4T1L, vector_##conj##_##disj) {                             \
+    auto em = conj(t0, t1);                                                    \
+    auto e = disj(em, t2);                                                     \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 3);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        disj##Pattern(conj##Pattern(tensorPattern(t0), tensorPattern(t1)),     \
+                      tensorPattern(t2)),                                      \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}));                          \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 2, conj##Pattern(tensorPattern(t0), tensorPattern(t1)),     \
+        loopsToBits({{l0, t0}, {l0, t1}}));                                    \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t2),                 \
+                              loopsToBits({{l0, t2}}));                        \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 3);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        disj##Pattern(conj##Pattern(tensorPattern(t0), tensorPattern(t1)),     \
+                      tensorPattern(t2)),                                      \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}));                          \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 2, conj##Pattern(tensorPattern(t0), tensorPattern(t1)),     \
+        loopsToBits({{l0, t0}, {l0, t1}}));                                    \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t2),                 \
+                              loopsToBits({{l0, t2}}));                        \
+  }
+
+MERGER_TEST_CONJ_DISJ(mulf, addf)
+MERGER_TEST_CONJ_DISJ(andi, addf)
+MERGER_TEST_CONJ_DISJ(mulf, ori)
+MERGER_TEST_CONJ_DISJ(andi, ori)
+MERGER_TEST_CONJ_DISJ(mulf, xori)
+MERGER_TEST_CONJ_DISJ(andi, xori)
+
+#undef MERGER_TEST_CONJ_DISJ
+
+/// Vector addition (disjunction) then addition (disjunction), i.e.;
+///   a(i) = b(i) + c(i) + d(i)
+/// which should form
+/// {
+///   lat( i_00 i_01 i_02 / (tensor_0 + tensor_1) + tensor_2 )
+///   lat( i_02 i_01 / tensor_2 + tensor_1 )
+///   lat( i_02 i_00 / tensor_2 + tensor_0 )
+///   lat( i_01 i_00 / tensor_1 + tensor_0 )
+///   lat( i_02 / tensor_2 )
+///   lat( i_01 / tensor_1 )
+///   lat( i_00 / tensor_0 )
+/// }
+#define MERGER_TEST_DISJ_DISJ(op1, op2)                                        \
+  TEST_F(MergerTest4T1L, Vector_##op1##_##op2) {                               \
+    auto em = op1(t0, t1);                                                     \
+    auto e = op2(em, t2);                                                      \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 7);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        op2##Pattern(op1##Pattern(tensorPattern(t0), tensorPattern(t1)),       \
+                     tensorPattern(t2)),                                       \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}));                          \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op2##Pattern(tensorPattern(t1), tensorPattern(t2)),      \
+        loopsToBits({{l0, t1}, {l0, t2}}));                                    \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op2##Pattern(tensorPattern(t0), tensorPattern(t2)),      \
+        loopsToBits({{l0, t0}, {l0, t2}}));                                    \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op1##Pattern(tensorPattern(t0), tensorPattern(t1)),      \
+        loopsToBits({{l0, t0}, {l0, t1}}));                                    \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t2),                 \
+                              loopsToBits({{l0, t2}}));                        \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t1),                 \
+                              loopsToBits({{l0, t1}}));                        \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t0),                 \
+                              loopsToBits({{l0, t0}}));                        \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 7);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        op2##Pattern(op1##Pattern(tensorPattern(t0), tensorPattern(t1)),       \
+                     tensorPattern(t2)),                                       \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}));                          \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op2##Pattern(tensorPattern(t1), tensorPattern(t2)),      \
+        loopsToBits({{l0, t1}, {l0, t2}}));                                    \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op2##Pattern(tensorPattern(t0), tensorPattern(t2)),      \
+        loopsToBits({{l0, t0}, {l0, t2}}));                                    \
+    expectLatPointWithinRange(                                                 \
+        s, lat(1), 6, op1##Pattern(tensorPattern(t0), tensorPattern(t1)),      \
+        loopsToBits({{l0, t0}, {l0, t1}}));                                    \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t2),                 \
+                              loopsToBits({{l0, t2}}));                        \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t1),                 \
+                              loopsToBits({{l0, t1}}));                        \
+    expectLatPointWithinRange(s, lat(1), 6, tensorPattern(t0),                 \
+                              loopsToBits({{l0, t0}}));                        \
+  }
+
+MERGER_TEST_DISJ_DISJ(addf, addf)
+MERGER_TEST_DISJ_DISJ(ori, ori)
+MERGER_TEST_DISJ_DISJ(xori, xori)
+MERGER_TEST_DISJ_DISJ(ori, xori)
+
+#undef MERGER_TEST_DISJ_DISJ
+
+/// Vector multiplication (conjunction) then multiplication (conjunction), i.e.;
+///   a(i) = b(i) * c(i) * d(i);
+/// which should form
+/// {
+///    lat( i_00 i_01 i_02 / tensor_0 * tensor_1 * tensor_2 )
+/// }
+#define MERGER_TEST_CONJ_CONJ(op1, op2)                                        \
+  TEST_F(MergerTest4T1L, vector_##op1##_##op2) {                               \
+    auto em = op1(t0, t1);                                                     \
+    auto e = op2(em, t2);                                                      \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        op2##Pattern(op1##Pattern(tensorPattern(t0), tensorPattern(t1)),       \
+                     tensorPattern(t2)),                                       \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}));                          \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(                                                            \
+        s, lat(0),                                                             \
+        op2##Pattern(op1##Pattern(tensorPattern(t0), tensorPattern(t1)),       \
+                     tensorPattern(t2)),                                       \
+        loopsToBits({{l0, t0}, {l0, t1}, {l0, t2}}), true);                    \
+  }
+
+MERGER_TEST_CONJ_CONJ(mulf, mulf)
+MERGER_TEST_CONJ_CONJ(mulf, andi)
+MERGER_TEST_CONJ_CONJ(andi, andi)
+
+#undef MERGER_TEST_CONJ_CONJ
+
+/// Vector addition (disjunction) of 2 vectors, i.e.;
+///   a(i) = b(i) + c(i)
+/// which should form the 3 lattice points
+/// {
+///   lat( i_00 i_01 / (sparse_tensor_0 + dense_tensor_1) )
+///   lat( i_00 / sparse_tensor_0 )
+///   lat( i_01 / dense_tensor_1 )
+/// }
+/// which should be optimized to
+/// {
+///   lat( i_00 i_01 / (sparse_tensor_0 + dense_tensor_1) ) (not singleton)
+///   lat( i_01 / dense_tensor_0 ) (no sparse dimension)
+/// }
+///
+/// lat( i_00 / sparse_tensor_0 ) should be opted out as it only has dense diff
+/// with lat( i_00 i_01 / (sparse_tensor_0 + dense_tensor_1) ).
+#define MERGER_TEST_OPTED_DISJ(op)                                             \
+  TEST_F(MergerTest3T1LD, vector_opted_##op) {                                 \
+    auto e = op(tensor(t0), tensor(t1));                                       \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 3);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}));                         \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t0),                 \
+                              loopsToBits({{l0, t0}}));                        \
+    expectLatPointWithinRange(s, lat(1), 2, tensorPattern(t1),                 \
+                              loopsToBits({{l0, t1}}));                        \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 2);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}), true);                   \
+    expectLatPoint(s, lat(1), tensorPattern(t1), loopsToBits({{l0, t1}}),      \
+                   true);                                                      \
+  }
+
+MERGER_TEST_OPTED_DISJ(addf)
+MERGER_TEST_OPTED_DISJ(addc)
+MERGER_TEST_OPTED_DISJ(addi)
+MERGER_TEST_OPTED_DISJ(ori)
+MERGER_TEST_OPTED_DISJ(xori)
+
+#undef MERGER_TEST_OPTED_DISJ
+
+/// Vector multiplication (conjunction) of 2 vectors, i.e.:
+///   a(i) = b(i) * c(i)
+/// which should form the single lattice point
+/// {
+///   lat( i_00 i_01 / (sparse_tensor_0 * dense_tensor_1) )
+/// }
+/// it should be optimized to
+/// {
+///   lat( i_00 / (sparse_tensor_0 * dense_tensor_1) )
+/// }
+/// since i_01 is a dense dimension.
+#define MERGER_TEST_OPTED_CONJ(op)                                             \
+  TEST_F(MergerTest3T1LD, vector_opted_##op) {                                 \
+    auto e = op(t0, t1);                                                       \
+                                                                               \
+    auto s = merger.buildLattices(e, l0);                                      \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}, {l0, t1}}));                         \
+                                                                               \
+    s = merger.optimizeSet(s);                                                 \
+    expectNumLatPoints(s, 1);                                                  \
+    expectLatPoint(s, lat(0),                                                  \
+                   op##Pattern(tensorPattern(t0), tensorPattern(t1)),          \
+                   loopsToBits({{l0, t0}}), true);                             \
+  }
+
+MERGER_TEST_OPTED_CONJ(mulf)
+MERGER_TEST_OPTED_CONJ(mulc)
+MERGER_TEST_OPTED_CONJ(muli)
+MERGER_TEST_OPTED_CONJ(andi)
+
+#undef MERGER_TEST_OPTED_CONJ
+
+// TODO: mult-dim tests