[lit] Extend DEFINE/REDEFINE with function substitutions

[jdenny]

Summary

For example, in a test file, the following directives define a
%{check} substitution with two formal parameters and then use it
with varying actual arguments:

// DEFINE: %{check}( CFLAGS %, FC_PREFIX %) =                   \
// DEFINE:   %clang_cc1 %{CFLAGS} -emit-llvm -o - %s |          \
// DEFINE:              FileCheck -check-prefix=%{FC_PREFIX} %s

// RUN: %{check}( -foo %, FOO %)
// RUN: %{check}( -bar %, BAR %)

[jdenny]

Update for changes to D132513.

[jdenny]

Update for changes to D132513.

[jdenny]

• Fixed examples in the doc: consistently use FCFLAGS not FC_PREFIX as a parameter.
• Updated the examples in the test suite to match the ones in the documentation.

[jdenny]

Update for changes to D132513.

[jdenny]

Update for changes to D132513.

[jdenny]

Update for changes to D132513.

[jdenny]

Update for changes to D132513.

[jdenny]

Updated for changes to D132513.

[jdenny]

Updated for changes to D132513.

[jhenderson]

I'll be honest, I don't like the syntax of this, so much so that I think I'd prefer people use the existing method of providing function arguments for these substitutions (i.e. DEFINE then REDEFINE them as needed) versus this. The complex regex required in the lit scripts to make this format work also doesn't help this solution's case.

Is the following format at all viable?

%{func(arg1, arg2)}

This would require lit to treat these kinds of substitutions specially, but the format including ( and ) in the name means this would at least be fairly easy to detect.

[jdenny]

In D133172#3807800, @jhenderson wrote:

Thanks for taking a look.

I'll be honest, I don't like the syntax of this,

I get it. It's not pretty. However, it was the simplest syntax I came up with to address the use cases I had in mind. I figured there would be a syntax debate, and I am open to suggestions. (For reference, @MaskRay and I also discussed the syntax some starting at https://discourse.llvm.org/t/lit-run-a-run-line-multiple-times-with-different-replacements/64932/14.)

so much so that I think I'd prefer people use the existing method of providing function arguments for these substitutions (i.e. DEFINE then REDEFINE them as needed) versus this.

For some uses cases, the existing DEFINE/REDEFINE functionality is sufficient and preferable. However, as the new TestingGuide.rst entry in this patch discusses, that syntax is not sufficient for some other uses cases.

The complex regex required in the lit scripts to make this format work also doesn't help this solution's case.

I'm not sure what you mean by "the lit scripts". Are you referring to TestRunner.py? Or are you thinking it will show up again somewhere else?

Is the following format at all viable?

%{func(arg1, arg2)}

I tried something similar originally:

%{func}(arg1, arg2)

However, I quickly ran into many cases where I needed , and ) in my actual arguments. For example, an argument might be a comma-delimited list of FileCheck/-verify prefixes. An argument might contain expected output strings that contained parenthesized elements, such as (null). The above suggestion has the same problems.

Ultimately, I settled on %, and %) as delimiters. That way, there’s no limit on what can go in an actual argument. Moreover, they stick with % for introducing new special sequences. The %if syntax ultimately chose to stick with % as well.

This would require lit to treat these kinds of substitutions specially,

I'm not sure I know what you mean.

Do you mean that, where we currently use python's re.sub and a regex to apply substitutions, we would, for the case of function substitutions, extend python with a hand-written parser for function argument lists? Given the previous negative reaction to making lit more complex for %if, I figured that expressing the argument list syntax in a regex that can be used within lit's existing implementation for applying substitutions would be more acceptable. I'm not sure that regex is actually more complicated than a hand-written parser would be. I'm pretty sure the regex is at least more concise. In any case, I'm open to rewriting if that's the direction the majority want. Eventually, we'll reinvent m4.

Also, I don't think the possibility of implementing with a hand-written parser vs. a regex is specific to your proposed syntax. Either is also possible for the syntax I proposed in this review.

But maybe I misunderstood your point.

but the format including ( and ) in the name means this would at least be fairly easy to detect.

I don't know what you mean. Can you explain more?

[jhenderson]

In D133172#3809343, @jdenny wrote:

In D133172#3807800, @jhenderson wrote:

Thanks for taking a look.

I'll be honest, I don't like the syntax of this,

I get it. It's not pretty. However, it was the simplest syntax I came up with to address the use cases I had in mind. I figured there would be a syntax debate, and I am open to suggestions. (For reference, @MaskRay and I also discussed the syntax some starting at https://discourse.llvm.org/t/lit-run-a-run-line-multiple-times-with-different-replacements/64932/14.)

so much so that I think I'd prefer people use the existing method of providing function arguments for these substitutions (i.e. DEFINE then REDEFINE them as needed) versus this.

For some uses cases, the existing DEFINE/REDEFINE functionality is sufficient and preferable. However, as the new TestingGuide.rst entry in this patch discusses, that syntax is not sufficient for some other uses cases.

I see it. I didn't register the need on my first skim through this, but I think I get it on second look.

The complex regex required in the lit scripts to make this format work also doesn't help this solution's case.

I'm not sure what you mean by "the lit scripts". Are you referring to TestRunner.py? Or are you thinking it will show up again somewhere else?

No, TestRunner.py was what I was referring to.

Is the following format at all viable?

%{func(arg1, arg2)}

I tried something similar originally:

%{func}(arg1, arg2)

However, I quickly ran into many cases where I needed , and ) in my actual arguments. For example, an argument might be a comma-delimited list of FileCheck/-verify prefixes. An argument might contain expected output strings that contained parenthesized elements, such as (null). The above suggestion has the same problems.

Ultimately, I settled on %, and %) as delimiters. That way, there’s no limit on what can go in an actual argument. Moreover, they stick with % for introducing new special sequences. The %if syntax ultimately chose to stick with % as well.

The %if syntax is slightly different, because there's no way of "grouping" the whole thing into one "name", I think? (I haven't looked back at the syntax for it, so might be misremembering it). For the DEFINE/REDEFINE case, a format with %{foo(arg1, arg2)} has the closing } as a convenient terminator for the "pattern", so anything (e.g. , and )) in between could be considered special (just don't try to use a } in the arg list!).

One of my main concerns is that the argument portion of the proposed syntax doesn't necessarily look like it really is a set of arguments to the untrained eye. The %) maybe just about binds itself to the (, but none of it really binds itself to the %{name} part, so it just doesn't feel like the two halves of the whole expression have anything to do with each other.

This would require lit to treat these kinds of substitutions specially,

I'm not sure I know what you mean.

Do you mean that, where we currently use python's re.sub and a regex to apply substitutions, we would, for the case of function substitutions, extend python with a hand-written parser for function argument lists? Given the previous negative reaction to making lit more complex for %if, I figured that expressing the argument list syntax in a regex that can be used within lit's existing implementation for applying substitutions would be more acceptable. I'm not sure that regex is actually more complicated than a hand-written parser would be. I'm pretty sure the regex is at least more concise. In any case, I'm open to rewriting if that's the direction the majority want. Eventually, we'll reinvent m4.

I think I might have misremembered how these substitutions are handled in practice, but in essence, I imagined doing (effectively) re.search to find a substitution in the input string, and swapping it in, repeating the process for each substitution. As the %{name(arg1, arg2)} syntax wouldn't work for that, you'd have to do something different or extra for these cases. Perhaps it would be re.search for the %{name part and then find the next } manually to form the "matching" block. It may not be less complex in terms of overall code, but it seems like it would require a much simpler regex, therefore making the code perhaps easier to understand?

Also, I don't think the possibility of implementing with a hand-written parser vs. a regex is specific to your proposed syntax. Either is also possible for the syntax I proposed in this review.

But maybe I misunderstood your point.

but the format including ( and ) in the name means this would at least be fairly easy to detect.

I don't know what you mean. Can you explain more?

I was thinking that you'd be able to distinguish the arg-based substitutions from regular ones, by identifying the presence of ( in the pattern, and then could decide to do a different thing for that substitution.

[jdenny]

In D133172#3810988, @jhenderson wrote:

The %if syntax is slightly different, because there's no way of "grouping" the whole thing into one "name", I think? (I haven't looked back at the syntax for it, so might be misremembering it).

At one point, %if used a single regex. There were two issues, as I recall:

1. The match had to be processed to evaluate the condition and select the then or else. As a result, re.sub wasn't used. Instead, re.search was used followed by post-processing. (Actually, I believe the post-processing could be have been handled by providing a python function for the replacement in re.sub.)

2. People wanted to nest %if in the then and else. That requires a parser stack, so a simple regex wasn't sufficient, and a recursive-descent parser was implemented. My hunch is that this part of the implementation led to the negative comments about complexity.

Function substitutions as proposed in this patch also have problem 2 (requires parser stack): you might want a function call to appear in an argument of another function call. Because I never found a reason to do that in my test suites, I suspected it wouldn't be common. I also complicated the regex a bit to be sure recursiveExpansionLimit could make such cases work. So, for now, I went with a regex applied via re.sub instead of a recursive-descent parser.

I've read that recursive regexes will be possible in python one day. I don't know the current status for sure, but I got the impression it's a bleeding-edge feature requiring a different regex library than re. That feature will probably enable addressing 2 elegantly for both %if and function substitutions. One day.

For the DEFINE/REDEFINE case, a format with %{foo(arg1, arg2)} has the closing } as a convenient terminator for the "pattern", so anything (e.g. , and )) in between could be considered special (just don't try to use a } in the arg list!).

OK, I see the idea now. However, that adds a third character that cannot be used in arguments.

One of my main concerns is that the argument portion of the proposed syntax doesn't necessarily look like it really is a set of arguments to the untrained eye. The %) maybe just about binds itself to the (, but none of it really binds itself to the %{name} part, so it just doesn't feel like the two halves of the whole expression have anything to do with each other.

I understand. I suffer from the lack of objectivity that comes from a "trained eye". That is, I stared at my syntax too long as it evolved into its current form.

To help ( and %) seem to bind better, we could change them to %( and %). To rationalize how the argument list then binds to %{name}, compare name(args) in a C-like programming language to %{name}%(args%). That is, name -> %{name}, ( -> %(, and ) -> %). Notice that % is used to introduce every special sequence.

I considered %( and %) early on, but it seemed more cluttered than technically necessary, so I decided to think of %{name}( as a single token instead. I don't care that much though, so I'm fine to switch to %( if people like it better.

I think I might have misremembered how these substitutions are handled in practice, but in essence, I imagined doing (effectively) re.search to find a substitution in the input string, and swapping it in, repeating the process for each substitution. As the %{name(arg1, arg2)} syntax wouldn't work for that, you'd have to do something different or extra for these cases. Perhaps it would be re.search for the %{name part and then find the next } manually to form the "matching" block.

It sound like you're trying to handle the call-in-args part of this, which requires a parser stack or recursiveExpansionLimit, as discussed above. In that regard, I see no difference between the syntax proposed in the current patch and the %{name(arg1, arg2)} syntax.

It may not be less complex in terms of overall code, but it seems like it would require a much simpler regex, therefore making the code perhaps easier to understand?

I guess that's in the eye of the beholder. Where possible, I tend to prefer formal definitions of syntax (like regexes) over ad-hoc implementations (like recursive-descent parsing), and I'm looking forward to a more powerful regex library in python. However, I'm open to reimplementing this as a recursive-descent parser now if that's more acceptable to everyone. Or maybe I should spend more time researching that new regex library to see if it's usable now somehow.

but the format including ( and ) in the name means this would at least be fairly easy to detect.

I don't know what you mean. Can you explain more?

I was thinking that you'd be able to distinguish the arg-based substitutions from regular ones, by identifying the presence of ( in the pattern, and then could decide to do a different thing for that substitution.

I might be misunderstanding your point, but I don't think that's an actual problem inherent in the syntax proposed in this patch.

The general substitution algorithm now is to iterate the list of substitutions and apply one at a time to the current RUN line. When arriving at a function substitution %{name} in the list, if we were to use recursive-descent parsing, I imagine it would then search for %{name}( in the RUN line. Upon finding it, it would then look for the closing %) while recursively balancing anything like %{looks-like-another-func}(args%) in the argument list. Regardless of whether %{looks-like-another-func}( is actually a function substitution, the matching %) binds to it for the sake of processing the arguments of %{name}. With your proposed syntax, I think the behavior would be analogous: regardless of whether %{looks-like-another-func( is actually a function substitution, the matching )} binds to it.

[jdenny]

In D133172#3811800, @jdenny wrote:

I guess that's in the eye of the beholder. Where possible, I tend to prefer formal definitions of syntax (like regexes) over ad-hoc implementations (like recursive-descent parsing), and I'm looking forward to a more powerful regex library in python. However, I'm open to reimplementing this as a recursive-descent parser now if that's more acceptable to everyone. Or maybe I should spend more time researching that new regex library to see if it's usable now somehow.

Arguing the other side, recursive-descent parsing can help to improve the diagnostics. That's relevant here: where a function substitution is used with unbalanced parentheses or the wrong number of arguments, the current patch just doesn't perform the substitution. The lit user experience would be better if lit instead terminated with a diagnostic.

In D133172#3811800, @jdenny wrote:

The general substitution algorithm now is to iterate the list of substitutions and apply one at a time to the current RUN line. When arriving at a function substitution %{name} in the list, if we were to use recursive-descent parsing, I imagine it would then search for %{name}( in the RUN line. Upon finding it, it would then look for the closing %) while recursively balancing anything like %{looks-like-another-func}(args%) in the argument list. Regardless of whether %{looks-like-another-func}( is actually a function substitution, the matching %) binds to it for the sake of processing the arguments of %{name}. With your proposed syntax, I think the behavior would be analogous: regardless of whether %{looks-like-another-func( is actually a function substitution, the matching )} binds to it.

I just thought of a potentially problematic case, which is perhaps what you were after:

%{real-func}( %{not-a-func}(expr) %)

%{not-a-func} might be a substitution but not a function substitution, and there happens to be a following parenthesized expression that would remain after expansion. %{not-a-func}( can prevent %{real-func}(...%) from expanding because the one %) matches %{not-a-func}(.

That's not a big deal with the current patch because you just increase recursiveExpansionLimit, and %{real-func)(...%) will be sure to expand after %{not-a-func}. In general, the current patch can require increasing recursiveExpansionLimit when something that looks like a call appears in an argument.

However, if we move to recursive-descent parsing and diagnose unbalanced parentheses, then this case can fail on the first attempt at expanding %{real-func}(...%). I noticed your proposed syntax suffers from the same problem, but I think it's only for cases that are probably less apt to occur in practice:

%{real-func( %{not-a-func(foo)-bar} )}

With either syntax, one way to solve this problem is to, upon each potential function use like %{not-a-func}(, do a substitution lookup to see if it's a function substitution.

A simpler way is to move to the other syntax I mentioned in my last comment:

%{real-func}%( %{not-a-func}(expr) %)

That is, %(...%) would always indicate a function substitution use. If you don't want it to, escape it using %%. Lit could even diagnose leftover %(...%) that never matched a function substitution.

[jhenderson]

In D133172#3811800, @jdenny wrote:

OK, I see the idea now. However, that adds a third character that cannot be used in arguments.

This is true. However, it seems like any solution has a restriction along these lines, unless you provide some form of escaping. In the current solution, you've effectively done this escaping via the use of %). I don't know whether this is a good idea, but in my suggestion, you could potentially use % to escape the special characters to actually appear as literals in the input arguments. However, I'm also inclined to say we don't need to try to solve this problem up-front necessarily.

To help ( and %) seem to bind better, we could change them to %( and %). To rationalize how the argument list then binds to %{name}, compare name(args) in a C-like programming language to %{name}%(args%). That is, name -> %{name}, ( -> %(, and ) -> %). Notice that % is used to introduce every special sequence.

I considered %( and %) early on, but it seemed more cluttered than technically necessary, so I decided to think of %{name}( as a single token instead. I don't care that much though, so I'm fine to switch to %( if people like it better.

I'm not sure that would help the situation. The full syntax would look like this: %{name}%(arg1, arg2%). By squinting, I can get the %( and %) to marry up, but I don't see the name half pairing up with the arg part intuitively. I don't know if this is necessarily a good idea overall, but I'd almost find it easier if the syntax were %{name}(arg1, arg2)% with the two % characters providing a "paired grouping" much like parentheses already do in typical computer programming. I think my problem is that the % characters in the current syntax aren't symmetrical about the midpoint of the syntax.

It sound like you're trying to handle the call-in-args part of this, which requires a parser stack or recursiveExpansionLimit, as discussed above. In that regard, I see no difference between the syntax proposed in the current patch and the %{name(arg1, arg2)} syntax.

Actually, I wasn't looking for nested functions: I think they are likely to be far less useful than function arguments, which I think are already likely to be relatively rare. I also think there's an argument for not adding too much complexity to the lit command language, because if you make the test script too complex, then it becomes impossible to figure out what the test is testing. I don't think regular function-like subsitutions fall into this situation, but if we start doing function calls in arg lists, then I think it might start becoming rather unintuitive.

It may not be less complex in terms of overall code, but it seems like it would require a much simpler regex, therefore making the code perhaps easier to understand?

I guess that's in the eye of the beholder. Where possible, I tend to prefer formal definitions of syntax (like regexes) over ad-hoc implementations (like recursive-descent parsing), and I'm looking forward to a more powerful regex library in python. However, I'm open to reimplementing this as a recursive-descent parser now if that's more acceptable to everyone. Or maybe I should spend more time researching that new regex library to see if it's usable now somehow.

I'm just one opinion, but I believe that one of the problems of regexes is that they require a decent knowledge of that particular language's regex language, since every variant is somewhat different, especially when getting into the more complicated functionality. Contrast that to a (hopefully simple) parser, if someone is reading this code, it's likely they already know python, so it should be fairly simple to follow.

[jdenny]

In D133172#3814345, @jhenderson wrote:

In D133172#3811800, @jdenny wrote:

OK, I see the idea now. However, that adds a third character that cannot be used in arguments.

This is true. However, it seems like any solution has a restriction along these lines, unless you provide some form of escaping. In the current solution, you've effectively done this escaping via the use of %).

Right, and that can be escaped as %%), reusing the existing %% substitution.

I don't know whether this is a good idea, but in my suggestion, you could potentially use % to escape the special characters to actually appear as literals in the input arguments. However, I'm also inclined to say we don't need to try to solve this problem up-front necessarily.

I disagree about the last sentence. In my downstream work, I often needed commas because they often appear in command-line arguments, and I sometimes need parens there too.

I think this is an example of what you're proposing:

DEFINE: %{check( CFLAGS, FCFLAGS )} = %clang %{CFLAGS} -o %t %s && %t | FileCheck %{FCFLAGS} %s
DEFINE: %{fcflags-common} = -DPTR='%(null%)'
RUN: %{check( -Wl%,-q, -check-prefixes=CHECK%,QUIET %{fcflags-common} )}

which would expand to:

RUN: %clang -Wl,-q -o %t %s && %t | FileCheck -check-prefixes=CHECK,QUIET -DPTR='(null)' %s

Here's why I find that proposal complicated:

• Parens, comma, and }, which are normally inert text, would be special only within uses of function substitution argument lists. (I understand that, depending on how it would be parsed, } might not need to be special before the closing ), but the main point still stands.)
• We need to introduce several additional lit substitutions to allow those characters to appear literally in arguments : %,, %(, %), and %}.
• Where do we define those new substitutions? I argue they should be special substitutions that, similar to %%, expand after all other substitutions have been expanded recursiveExpansionLimit times. That way, substitutions like %{fcflags-common} can be reused outside argument lists in other RUN lines. Moreover, the test author doesn't have to worry the new substitutions will expand too early (before emerging in an argument list that's ready to be parsed) because he didn't think through substitution order, function uses hidden behind other substitutions, function uses within other functions' args, and how all that interacts with recursiveExpansionLimit.
• Test authors have to adjust all existing substitution values that contain commas or parens if they start to also use those substitutions in the args of functions. For example, maybe %{fcflags-common} was actually defined already in a lit configuration file.

It seems to me that it's simpler for the test author and the lit developer if we just say % is meaningful to lit, and other text is inert, as in the following example:

DEFINE: %{check}%( CFLAGS %, FCFLAGS %) = %clang %{CFLAGS} -o %t %s && %t | FileCheck %{FCFLAGS} %s
DEFINE: %{fcflags-common} = -DPTR='(null)'
RUN: %{check}%( -Wl,-q %, -check-prefixes=CHECK,QUIET %{fcflags-common} %)

I'm not sure that would help the situation. The full syntax would look like this: %{name}%(arg1, arg2%). By squinting, I can get the %( and %) to marry up, but I don't see the name half pairing up with the arg part intuitively. I don't know if this is necessarily a good idea overall, but I'd almost find it easier if the syntax were %{name}(arg1, arg2)% with the two % characters providing a "paired grouping" much like parentheses already do in typical computer programming. I think my problem is that the % characters in the current syntax aren't symmetrical about the midpoint of the syntax.

For the record, the %if syntax already uses %{ and %}. That is, % consistently starts a special sequence.

I'm just one opinion, but I believe that one of the problems of regexes is that they require a decent knowledge of that particular language's regex language, since every variant is somewhat different, especially when getting into the more complicated functionality. Contrast that to a (hopefully simple) parser, if someone is reading this code, it's likely they already know python, so it should be fairly simple to follow.

OK, if no one else objects, let's agree to go with a recursive-descent parser instead of a single regex. That should address several issues with the current patch:

• Some lit developers might find it easier to understand. If that's true for the majority of lit developers, then my original motivation for the single regex is defeated.

• For invalid uses of function substitutions (e.g., wrong number of arguments, unclosed argument list), it can provide helpful diagnostics instead of just quietly failing to substitute.

• It can handle function calls within function arguments without requiring recursiveExpansionLimit.

• Function substitution entries in the substitution list (that is, config.substitutions) will not have the form (pattern, replacement) but instead will list parameter names so a recursive-descent parser can parse function substitution uses. For example, we might decide that any entry with more than two items would be interpreted as (func_name, param1, param2, ..., replacement). On the one hand, the substitution list will be more complicated. On the other hand, it means lit configuration files would immediately have a way to define function substitutions. The current patch doesn't provide one.

But let's converge on the previous syntax questions before I pursue the rewrite.

[jhenderson]

Sorry for the delayed response, and even now I'm not going to respond directly to your latest comments, as I have to get on with some high priority work that I need to finish ahead of a week off next week. I'll try to look back at this when I'm back.

It might be worth raising some of these points on the discourse thread and see if there's any further thoughts on them, as I don't feel confident the two of us represent a real consensus given the interest shown elsewhere!

[jdenny]

In D133172#3822847, @jhenderson wrote:

Sorry for the delayed response, and even now I'm not going to respond directly to your latest comments, as I have to get on with some high priority work that I need to finish ahead of a week off next week. I'll try to look back at this when I'm back.

No worries. Thanks for the conversation so far. I should also focus on other things for the moment.

It might be worth raising some of these points on the discourse thread and see if there's any further thoughts on them, as I don't feel confident the two of us represent a real consensus given the interest shown elsewhere!

That sounds like a good plan. I'll try to pursue it soon.

llvm/docs/TestingGuide.rst

- **Line continuations**: If the last non-whitespace character on the line after
   (e.g., ``DEFINE:``) , and it is an error if there is no additional directive.
   That directive serves as a continuation.  That is, before following the rules
   above to parse the text after ``:`` in either directive, lit joins that text
   together to form a single directive, replaces the ``\`` with a single space,
   and removes any other whitespace that is now adjacent to that space.  A
   continuation can be continued in the same manner.  A continuation containing
   only whitespace after its ``:`` is an error.
 
+**Function substitutions:**
+
+In the example from the previous section, notice that there are
+multiple blocks, one per triple, and each has a pair of uses of
+``%{check}``, one with and one without ``-fopenmp-simd``.  The test
+would be easier to read and maintain if that pair were encapsulated in
+another substitution.  However, that pair contains ``REDEFINE:``
+directives, and there is no way to place ``DEFINE:`` or ``REDEFINE:``
+directives in the value of a substitution.
+
+Lit supports function substitutions to address this use case.  For example, if
+we define ``%{check}`` as a function substitution, we can define a
+``%{check-triple}`` substitution that uses it multiple times with different
+parameters.  Unless we are using ``recursiveExpansionLimit``, we must be careful
+to define ``%{check}`` before ``%{check-triple}`` in order for both to expand as
+expected:
+
+.. code-block:: llvm
+
+    ; DEFINE: %{triple} =
+
+    ; DEFINE: %{check}( CFLAGS %, FC_PREFIX %) =                          \
+    ; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS} \
+    ; DEFINE:              -triple %{triple} -emit-llvm -o - %s |         \
+    ; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s
+
+    ; DEFINE: %{check-triple} =                                           \
+    ; DEFINE:   %{check}(               %, CHECK %) &&                    \
+    ; DEFINE:   %{check}( -fopenmp-simd %, SIMD  %)
+
+    ; REDEFINE: %{triple} = x86_64-apple-darwin10.6.0
+    ; RUN: %{check-triple}
+
+    ; REDEFINE: %{triple} = x86_64-unknown-linux-gnu
+    ; RUN: %{check-triple}
+
+Notice that ``%{check}`` has two kinds of parameters now:
+``%{CFLAGS}`` and ``%{FC_PREFIX}`` are function parameters, but
+``%{triple}`` is still a standalone parameter substitution that must
+be initially defined before ``%{check}`` unless we are using
+``recursiveExpansionLimit``.
+
+Because function substitutions are usually more composable, you might
+decide that any parameterized substitution is best written as a
+function substitution.  For example, ``%{check-triple}`` above could
+be converted to a function substitution as well:
+
+.. code-block:: llvm
+
+    ; DEFINE: %{check}( TRIPLE %, CFLAGS %, FC_PREFIX %) =                \
+    ; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS} \
+    ; DEFINE:              -triple %{TRIPLE} -emit-llvm -o - %s |         \
+    ; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s
+
+    ; DEFINE: %{check-triple}( TRIPLE %) =                                \
+    ; DEFINE:   %{check}( %{TRIPLE} %,               %, CHECK %) &&       \
+    ; DEFINE:   %{check}( %{TRIPLE} %, -fopenmp-simd %, SIMD  %)
+
+    ; RUN: %{check-triple}( x86_64-apple-darwin10.6.0 %)
+    ; RUN: %{check-triple}( x86_64-unknown-linux-gnu  %)
+
+For a complex substitution with a long list of parameters, it is
+sometimes more readable if at least some of its parameters remain as
+standalone substitutions.  As in the original example from the
+previous section, actual arguments are then optional, and they are
+recognized by their formal parameter names instead of by their
+positions in a long sequence of actual arguments.
+
+If you end up with too many levels of substitutions, debugging test
+failures can be challenging because the only location information lit
+reports in the test output is for the ``RUN:`` line.  ``%(line)``
+substitutions can improve this scenario.  For example, because
+``%(line)`` expands immediately within a ``DEFINE:``, ``REDEFINE:``,
+or ``RUN:`` directive, it can be used to gather location information
+at every level of the expansion, and that information can then be
+reported with a command like ``echo``:
+
+.. code-block:: llvm
+
+    ; DEFINE: %{check}( LOC %, TRIPLE %, CFLAGS %, FC_PREFIX %) =         \
+    ; DEFINE:   echo 'At test lines %{LOC}, %(line)' &&                   \
+    ; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS} \
+    ; DEFINE:              -triple %{TRIPLE} -emit-llvm -o - %s |         \
+    ; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s
+
+    ; DEFINE: %{check-triple}( LOC %, TRIPLE %) =                                      \
+    ; DEFINE:   %{check}( %{LOC}, %(line) %, %{TRIPLE} %,               %, CHECK %) && \
+    ; DEFINE:   %{check}( %{LOC}, %(line) %, %{TRIPLE} %, -fopenmp-simd %, SIMD  %)
+
+    ; RUN: %{check-triple}( %(line) %, x86_64-apple-darwin10.6.0 %)
+    ; RUN: %{check-triple}( %(line) %, x86_64-unknown-linux-gnu  %)
+
+If your test suite is configured to use lit's internal shell, you can
+portably replace ``echo`` with the ``:`` command so that the location
+information is reported only when lit prints the command and not again
+when lit executes the command.
+
+As a modification of the rules presented in the previous section for
+``DEFINE:`` and ``REDEFINE:`` directives, the following properties
+apply in the case of function substitutions:
+
+- **Structural syntax**: In the directive, ``%{name}`` must be followed
+  immediately by ``(`` without intervening whitespace.  That is, ``%{name}(`` is
+  a single token.  The same is true in uses of the substitution.  Formal
+  parameter names in the directive appear after ``%{name}(`` and must be
+  separated by ``%,`` and terminated by ``%)``.  The same is true for actual
+  arguments in uses.  In the directive, there must be at least one formal
+  parameter.  If either ``%,`` or ``%)`` must occur in an actual argument, it
+  can be escaped as ``%%,`` or ``%%)``.  However, ``,`` and ``)`` need not be
+  otherwise escaped.  For example, an actual argument can easily contain a
+  comma-delimited list of FileCheck prefixes, or it can contain parenthesized
+  expressions.
+- **Parameter names**: Each parameter name must be one or more alphanumeric
+  characters and underscores not starting with a digit.  Parameter names do not
+  have the same set of constraints as substitution names because parameter names
+  must be valid python identifiers in order to be used in the substitution's
+  pattern.  In the formal parameter list in the directive, parameters are
+  spelled without the enclosing ``%{`` and ``}`` for the sake of conciseness and
+  readability.  However, they must include those when referenced in the
+  substitution's value.
+- **Whitespace**: In the directive, whitespace before the ``%{name}(`` and
+  between the ``%)`` and ``=`` is optional and discarded.  Leading and trailing
+  whitespace around a formal parameter name in the directive or around an actual
+  argument in a use is optional and discarded.  If only whitespace is provided
+  for an actual argument, the argument is the empty string.  This behavior
+  facilitates formatting parameter lists and argument lists for readability.
+- **Line continuations**: Line continuations are permitted in both formal
+  parameter lists and actual argument lists.  This is because the text in a
+  ``RUN:``, ``DEFINE:``, or ``REDEFINE:`` directive is not parsed until all line
+  continuations have been appended.
+- **Invalid syntax**: Lit diagnoses invalid parameter list syntax in the
+  directive.  However, lit does not diagnose invalid actual argument list syntax
+  in uses.  Instead, if the syntax for a use is not followed, the substitution's
+  pattern simply does not match, and so the substitution does not expand, just
+  like any other substitution.
+- **Required actual arguments**: All arguments must appear in a use, or the
+  substitution does not expand.  This behavior facilitates debugging expansions
+  by making it clear which substitution needs to be investigated.  In contrast,
+  imagine the behavior if trailing arguments were optional and you accidentally
+  type ``,`` instead of ``%,``.  All remaining actual arguments would map to the
+  wrong formal parameters, and the only clue as to the problem would likely be a
+  corrupt ``RUN:`` line.
+- **Substitution name conflicts**: As discussed in the previous section,
+  ``DEFINE:`` and ``REDEFINE:`` are careful to make sure ``%{name}`` doesn't
+  appear in any other substitution's pattern.  For function substitutions, the
+  parameter list is not included in this check.  That is, the ``%{name}`` alone
+  is enough to create a conflict.  As a result, overloading functions is not
+  possible.  This restriction facilitates debugging in the same way as requiring
+  all arguments.
+- **Parameter name conflicts**: When ``REDEFINE:`` searches the substitution
+  list for the substitution to actually redefine, it does check the parameters.
+  It will fail even if a parameter name changes.  That is, the parameter names
+  are part of the substitution's pattern, and the pattern must be identical.
+- **Function substitution use in argument**: A use of a function substitution,
+  such as ``%{outer}``, will initially fail to expand if an argument contains
+  text, such as ``%{inner}(``, that lexically looks like the start of a function
+  substitution use.  The problem is python's ``re`` package is not powerful
+  enough to balance parentheses in a single pattern match.  Thus, a function
+  substitution's pattern is designed to require the use of ``%{inner}`` to
+  expand first, and then the use of ``%{outer}`` can expand later, achieving
+  parentheses balancing via multiple pattern matches.  However, by default, this
+  means the ``DEFINE:`` directive order must be ``%{outer}`` before
+  ``%{inner}``.  That might be hard to remember, so it is probably best to use
+  ``recursiveExpansionLimit`` if you plan to use one function substitution in
+  the argument of another.  A more powerful regular expression engine for python
+  that is capable of recursion appears to be in development, so this limitation
+  might one day be eliminated.
+
 Options
 -------
 
 The llvm lit configuration allows to customize some things with user options:
 
 ``llc``, ``opt``, ...
     Substitute the respective llvm tool name with a custom command line. This
     allows to specify custom paths and default arguments for these tools.

llvm/utils/lit/lit/TestRunner.py

class CommandDirective(ScriptDirective):
     def needs_continuation(self):
         return self.command[-1] == '\\'
 
 class SubstDirective(ScriptDirective):
     def __init__(self, start_line_number, end_line_number, keyword, line):
         super().__init__(start_line_number, end_line_number, keyword)
         self.body = line
         self.name = None
+        self.params = None
         self.value = None
         self._parse_body()
 
     def add_continuation(self, line_number, keyword, line):
         if keyword != self.keyword or not self.needs_continuation():
             return False
         if not line.strip():
             raise ValueError("Substitution's continuation is empty")
def _parse_body(self):
         if self.needs_continuation():
             return
 
         # Extract the left-hand side and value, and discard any whitespace
         # enclosing each.
         parts = self.body.split('=', 1)
         if len(parts) == 1:
             raise ValueError("Substitution's definition does not contain '='")
-        self.name = parts[0].strip()
+        lhs = parts[0].strip()
         self.value = parts[1].strip()
 
         # Check the substitution's name.
         #
         # Do not extend this to permit '.' or any sequence that's special in a
         # python pattern.  We could escape that automatically for
         # DEFINE/REDEFINE directives in test files.  However, lit configuration
         # file authors would still have to remember to escape them manually in
         # substitution names but not in values.  Moreover, the manually chosen
         # and automatically chosen escape sequences would have to be consistent
         # (e.g., '\.' vs. '[.]') in order for REDEFINE to successfully redefine
         # a substitution previously defined by a lit configuration file.  All
         # this seems too error prone and confusing to be worthwhile.  If you
         # want your name to express structure, use ':' instead of '.'.
         #
         # Actually, '{' and '}' are special if they contain only digits possibly
         # separated by a comma.  Requiring a leading letter avoids that.
-        if not re.fullmatch(r'%{[_a-zA-Z][-_:0-9a-zA-Z]*}', self.name):
+        name_match = re.match(r'^%{[_a-zA-Z][-_:0-9a-zA-Z]*}', lhs)
+        if not name_match:
             raise ValueError(
                 "Substitution name '{name}' is malformed as it must start with "
                 "'%{{', it must end with '}}', and the rest must start with a "
                 "letter or underscore and contain only alphanumeric "
                 "characters, hyphens, underscores, and colons"
-                .format(name=self.name))
+                .format(name=lhs))
+        self.name = name_match.group(0)
+        param_list = lhs[name_match.end():]
 
+        # Parse any parameter list.
+        if not param_list:
+            return
+        if param_list[0] != '(':
+            raise ValueError(
+                "Substitution name '{name}' must be followed by '=' (possibly "
+                "with intervening whitespace) or by '(' (without intervening "
+                "whitespace)".format(name=self.name))
+        if param_list[-2:] != '%)':
+            raise ValueError("Parameter list of substitution '{name}' must end "
+                             "with '%)'".format(name=self.name))
+        self.params = param_list[1:-2].strip()
+        if not self.params:
+            raise ValueError(
+                "Substitution '{name}' must be defined with no parameter "
+                "list or at least one parameter".format(name=self.name))
+        self.params = self.params.split('%,')
+        for i, param in enumerate(self.params):
+            self.params[i] = param = param.strip()
+            notes = ''
+            # Parameter names must be valid python identifiers so we can use
+            # '(?P<param_name>...)' to match them.
+            if not re.fullmatch('[_a-zA-Z][_0-9a-zA-Z]*', param):
+                if re.search('[,\s]', param):
+                    notes += "\nParameter names must be separated by '%,'"
+                raise ValueError(
+                    "Parameter {i} of substitution '{name}' is malformed as it "
+                    "must be one or more alphanumeric characters and "
+                    "underscores not starting with a digit\n"
+                    "Parameter {i} is: {param}{notes}"
+                    .format(i=i+1, name=self.name, param=param, notes=notes))
     def adjust_substitutions(self, substitutions):
+        call_re = self.name
         value_repl = self.value.replace('\\', '\\\\')
+        if self.params:
+            params_re = ''
+            for param in self.params:
+                # Trailing empty actual arguments are *not* optional for a
+                # reason.  Imagine the behavior if they were optional and you
+                # accidentally typed ',' instead of '%,'.  That and the next
+                # intended actual argument would become part of the previous
+                # actual argument, and any additional actual arguments would map
+                # to the wrong formal parameters.  The result would be a
+                # confusing expansion.  However, because actual arguments are
+                # not optional, the effect instead is that the substitution
+                # doesn't expand, making it clearer that you simply botched the
+                # substitution syntax (or the expansion order is insufficient).
+                # The minor inconvenience of extra '%,' seems worth the improved
+                # debugability.
+                #
+                # Don't expand the substitution if an argument contains a call
+                # to another function-like substitution.  The problem is
+                # python's re is not powerful enough to balance parentheses, so
+                # we have no way to prevent the outer call from terminating at
+                # the end of the inner call or vice-versa.  Instead, we just
+                # prevent arguments from containing either the start or end of a
+                # call, and thus the inner call must expand first.  Substitution
+                # order or recursiveExpansionLimit might have to be adjusted in
+                # order for the outer call to then expand.
+                if params_re:
+                    params_re += '%,'
+                params_re += '\s*' # trim leading ws to facilitate formatting
+                params_re += '(?!\s)' # can't be followed by ws, or might have
+                                      # catastrophic backtracking
+                params_re += '(?P<' + param + '>' # start actual arg
+                params_re +=   '(?:'
+                params_re +=     '(?:'
+                params_re +=       '[^%]' # char that doesn't start an escape
+                params_re +=       '|%[^{,)]' # escape not special in arg list
+                params_re +=       '|%{(?![^}]+}\()' # '%{' but not '%{...}('
+                params_re +=     ')+' # require non-empty arg
+                params_re +=     '(?<!\s)' # non-empty arg can't end in ws, or
+                                           # might have catastrophic
+                                           # backtracking
+                params_re +=   ')?' # if no non-empty arg, skip previous assert,
+                                    # or it's impossible for arg to be only ws
+                params_re += ')'
+                params_re += '\s*' # trim trailing ws to facilitate formatting
+                value_repl = value_repl.replace('%{' + param + '}',
+                                                '\\g<' + param + '>')
+            call_re += '\(' + params_re + '%\)'
         existing = [i for i, subst in enumerate(substitutions)
                     if self.name in subst[0]]
         existing_res = ''.join("\nExisting pattern: " + substitutions[i][0]
                                for i in existing)
         if self.keyword == 'DEFINE:':
             if existing:
                 raise ValueError(
                     "Substitution whose pattern contains '{name}' is already "
                     "defined before '{key}' directive {loc}"
                     "{existing}"
                     .format(name=self.name, key=self.keyword,
                             loc=self.get_location(), existing=existing_res))
-            substitutions.insert(0, (self.name, value_repl))
+            substitutions.insert(0, (call_re, value_repl))
             return
         assert self.keyword == 'REDEFINE:'
         if len(existing) > 1:
             raise ValueError(
                 "Multiple substitutions whose patterns contain '{name}' are "
                 "defined before '{key}' directive {loc}"
                 "{existing}"
                 .format(name=self.name, key=self.keyword,
                         loc=self.get_location(), existing=existing_res))
         if not existing:
             raise ValueError(
                 "No substitution for '{name}' is defined before '{key}' "
                 "directive {loc}"
                 .format(name=self.name, key=self.keyword,
                         loc=self.get_location()))
-        if substitutions[existing[0]][0] != self.name:
+        if substitutions[existing[0]][0] != call_re:
             raise ValueError(
                 "Existing substitution whose pattern contains '{name}' does "
                 "not have the pattern specified by '{key}' directive "
                 "{loc}\n"
                 "Expected pattern: {expected}"
                 "{existing}"
                 .format(name=self.name, key=self.keyword,
-                        loc=self.get_location(), expected=self.name,
+                        loc=self.get_location(), expected=call_re,
                         existing=existing_res))
-        substitutions[existing[0]] = (self.name, value_repl)
+        substitutions[existing[0]] = (call_re, value_repl)
 
 
 def applySubstitutions(script, substitutions, conditions={},
                        recursion_limit=None):
     """
     Apply substitutions to the script.  Allow full regular expression syntax.
     Replace each matching occurrence of regular expression pattern a with
     substitution b in line ln.

llvm/utils/lit/tests/Inputs/shtest-define/actual-arg-catastrophic-backtracking.txt

+# Make sure two few arguments don't lead to catastrophic backtracking in
+# python's regex engine.  This can occur because it's tempting to write the
+# regex for an actual argument and its leading/trailing whitespace as
+# '\s*(?P<param>...)\s*' where '...' is a regex that accepts any series of
+# leading/trailing whitespace as well.
+#
+# No substitution expansion should actually occur because there are two few
+# arguments.  White space is collapsed in CHECK directives here just because
+# we're not using FileCheck's -strict-whitespace.
+
+# DEFINE: %{f}(A%, B%, C%, D%, E%) = hello world
+
+# Check when each arg is only whitespace.
+#
+# RUN: echo '%{f}(\
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                                                      %)'
+#
+# CHECK: %{f}( %, %, %, %)
+
+# Check when each arg has non-whitespace with leading and trailing whitespace.
+#
+# RUN: echo '%{f}(\
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                    a                                    \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                    b                                    \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                    c                                    \
+# RUN:                                                                         \
+# RUN:                                                                      %, \
+# RUN:                                                                         \
+# RUN:                                                                         \
+# RUN:                                    d                                    \
+# RUN:                                                                         \
+# RUN:                                                                      %)'
+#
+# CHECK: %{f}( a %, b %, c %, d %)
+
+# CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/actual-arg-count.txt

+# Check that expansion only happens with the correct number of actual arguments
+# and that empty arguments are fine.
+
+# DEFINE: %{f1}( p0             %) = f1('%{p0}')
+# DEFINE: %{f2}( p0 %, p1       %) = f2('%{p0}', '%{p1}')
+# DEFINE: %{f3}( p0 %, p1 %, p2 %) = f3('%{p0}', '%{p1}', '%{p2}')
+
+# RUN: echo "%{f1}(%)"
+# RUN: echo "%{f1}(f%)"
+# RUN: echo "%{f1}(fgh%)"
+# RUN: echo "%{f1}(%,)"
+# RUN: echo "%{f1}(f%,)"
+# RUN: echo "%{f1}(f%,g)"
+# RUN: echo "%{f1}(f%,g%,h)"
+# CHECK: f1('')
+# CHECK: f1('f')
+# CHECK: f1('fgh')
+# CHECK: %{f1}(%,)
+# CHECK: %{f1}(f%,)
+# CHECK: %{f1}(f%,g)
+# CHECK: %{f1}(f%,g%,h)
+
+# RUN: echo "%{f2}(%)"
+# RUN: echo "%{f2}(f%)"
+# RUN: echo "%{f2}(%,%)"
+# RUN: echo "%{f2}(f%,%)"
+# RUN: echo "%{f2}(%,g%)"
+# RUN: echo "%{f2}(f%,g%)"
+# RUN: echo "%{f2}(fgh%,ghi%)"
+# RUN: echo "%{f2}(%,%,%)"
+# RUN: echo "%{f2}(f%,%,%)"
+# RUN: echo "%{f2}(f%,g%,%)"
+# RUN: echo "%{f2}(f%,g%,h%)"
+# RUN: echo "%{f2}(f%,g%,h%,i%)"
+# CHECK: %{f2}(%)
+# CHECK: %{f2}(f%)
+# CHECK: f2('', '')
+# CHECK: f2('f', '')
+# CHECK: f2('', 'g')
+# CHECK: f2('f', 'g')
+# CHECK: f2('fgh', 'ghi')
+# CHECK: %{f2}(%,%,%)
+# CHECK: %{f2}(f%,%,%)
+# CHECK: %{f2}(f%,g%,%)
+# CHECK: %{f2}(f%,g%,h%)
+# CHECK: %{f2}(f%,g%,h%,i%)
+
+# RUN: echo "%{f3}(%)"
+# RUN: echo "%{f3}(%,%)"
+# RUN: echo "%{f3}(f%,g%)"
+# RUN: echo "%{f3}(%,%,%)"
+# RUN: echo "%{f3}(%,%,%)"
+# RUN: echo "%{f3}(f%,%,%)"
+# RUN: echo "%{f3}(%,g%,%)"
+# RUN: echo "%{f3}(%,%,h%)"
+# RUN: echo "%{f3}(f%,g%,%)"
+# RUN: echo "%{f3}(f%,%,h%)"
+# RUN: echo "%{f3}(%,g%,h%)"
+# RUN: echo "%{f3}(f%,g%,h%)"
+# RUN: echo "%{f3}(fgh%,ghi%,hij%)"
+# RUN: echo "%{f3}(%,%,%,%)"
+# RUN: echo "%{f3}(f%,%,%,%)"
+# RUN: echo "%{f3}(f%,g%,%,%)"
+# RUN: echo "%{f3}(f%,g%,h%,%)"
+# RUN: echo "%{f3}(f%,g%,h%,i%)"
+# CHECK: %{f3}(%)
+# CHECK: %{f3}(%,%)
+# CHECK: %{f3}(f%,g%)
+# CHECK: f3('', '', '')
+# CHECK: f3('f', '', '')
+# CHECK: f3('', 'g', '')
+# CHECK: f3('', '', 'h')
+# CHECK: f3('f', 'g', '')
+# CHECK: f3('f', '', 'h')
+# CHECK: f3('', 'g', 'h')
+# CHECK: f3('f', 'g', 'h')
+# CHECK: f3('fgh', 'ghi', 'hij')
+# CHECK: %{f3}(%,%,%,%)
+# CHECK: %{f3}(f%,%,%,%)
+# CHECK: %{f3}(f%,g%,%,%)
+# CHECK: %{f3}(f%,g%,h%,%)
+# CHECK: %{f3}(f%,g%,h%,i%)
+
+# CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/actual-arg-special-chars.txt

+# Check that various sequences that are or look relevant to function
+# substitutions behave correctly within actual arguments:
+#
+# - {name}(
+# - (
+# - ,
+# - )
+# - %{name}(
+# - %(
+# - %,
+# - %)
+# - %%{name}(
+# - %%(
+# - %%,
+# - %%)
+
+# %{fn}, %{call-a-b}, and substitutions defined before and after them.
+#
+# DEFINE: %{expand-last-fn}(p0 %, p1%) = expand-last-fn(%{p0},%{p1})
+# DEFINE: %{expand-last-str} = Goodbye world
+# DEFINE: %{fn}(p0 %, p1 %, p2%)=fn('%{p0}', '%{p1}', '%{p2}')
+# DEFINE: %{call-a-b}( FUNC %) = %{FUNC}( a %, b %)
+# DEFINE: %{expand-first-fn}(p0 %, p1%) = expand-first-fn(%{p0},%{p1})
+# DEFINE: %{expand-first-str} = Hello world
+
+# Try each of these sequences in args: {name}( ( , )
+#
+# RUN: echo "%{fn}( {expand-first-fn}(foo, bar)%, (foo,%, ,bar)%)"
+# CHECK: fn('{expand-first-fn}(foo, bar)', '(foo,', ',bar)')
+
+# Make sure non-function substitutions are always fine in an arg.  However,
+# sorry, before they expand, they prevent expansion of the actual function use
+# if they lexically look like function uses too: %{name}(
+#
+# RUN: echo "%{fn}( %{expand-first-str}() %, %{expand-first-str}( %, %) %, %)"
+# RUN: echo "%{fn}( %{expand-last-str}() %, %{expand-last-str}( %, %) %, %)"
+#           CHECK: fn('Hello world()', 'Hello world(', '') %, %)
+# CHECK-NON-RECUR: %{fn}( Goodbye world() %, Goodbye world( %, %) %, %)
+#     CHECK-RECUR: fn('Goodbye world()', 'Goodbye world(', '') %, %)
+
+# '%(' is not special.  '%{name}%(' does not look like a function use.
+#
+# RUN: echo "%{fn}( %{expand-first-str}%() %, %{expand-first-str}%( %, %) %, %)"
+# RUN: echo "%{fn}( %{expand-last-str}%() %, %{expand-last-str}%( %, %) %, %)"
+# CHECK: fn('Hello world%()', 'Hello world%(', '') %, %)
+# CHECK: fn('Goodbye world%()', 'Goodbye world%(', '') %, %)
+
+# Try each of these sequences in args: %%{name}( %%( %%, %%)
+# Can we use those to fully escape a function substitution use within an arg?
+#
+# RUN: echo "%{fn}( %%{expand-first-fn}(a%%,b%%) %, %%{expand-first-fn}%%(a%%,b%%) %, %)"
+# RUN: echo "%{fn}( %%{expand-last-fn}(a%%,b%%) %, %%{expand-last-fn}%%(a%%,b%%) %, %)"
+# CHECK: fn('%{expand-first-fn}(a%,b%)', '%{expand-first-fn}%(a%,b%)', '')
+# CHECK: fn('%{expand-last-fn}(a%,b%)', '%{expand-last-fn}%(a%,b%)', '')
+
+# In an arg of %{fn}, try to use %{expand-first-fn}.  Both expand.
+#
+# RUN: echo "%{fn}( %, %{expand-first-fn}(a%,b%) %, %)"
+# CHECK: fn('', 'expand-first-fn(a,b)', '')
+
+# In an arg of %{fn}, try to use %{expand-last-fn}.  First, the use of %{fn}
+# fails to expand because of the appearance of '%{expand-last-fn}(' in its arg.
+# That's good because python's re package cannot balance parentheses, and we
+# don't want to match the wrong '%)' after '%{fn}(' (without the restriction
+# against a function use in the arg, that match would have been possible at the
+# first '%)' because the correct number of remaining '%,' for %{fn} appear
+# before it).  Next, the use of %{expand-last-fn} expands.  If
+# recursiveExpansionLimit were >=2, the use of %{fn} would then expand.
+#
+# RUN: echo "%{fn}( %, %{expand-last-fn}(a%,b%) %, %)"
+# CHECK-NON-RECUR: %{fn}( %, expand-last-fn(a,b) %, %)
+#     CHECK-RECUR: fn('', 'expand-last-fn(a,b)', '')
+
+# What happens when a function substitution is used in its own arg?  Again, we
+# must make sure the inner use expands first.  Otherwise, the outer use would
+# close at the inner uses's '%)'.  We need recursiveExpansionLimit>=2 to expand
+# the outer use.
+#
+# RUN: echo "%{fn}( %{fn}(a %, b %, c %) %, %, %)"
+# CHECK-NON-RECUR: %{fn}( fn('a', 'b', 'c') %, %, %)
+#     CHECK-RECUR: fn('fn('a', 'b', 'c')', '', '')
+
+# Make sure we always match the first '%)'.  In this case, the second arg in the
+# use of %{expand-first-fn} would extend to the '%)' that pairs with '%{fn}(',
+# but it doesn't because we don't permit '%)' in an argument.
+#
+# RUN: echo "%{fn}( %, %, %{expand-first-fn}(a%,b%) %)"
+# RUN: echo "%{fn}( %, %, %{expand-last-fn}(a%,b%) %)"
+#           CHECK: fn('', '', 'expand-first-fn(a,b)')
+# CHECK-NON-RECUR: %{fn}( %, %, expand-last-fn(a,b) %)
+#     CHECK-RECUR: fn('', '', 'expand-last-fn(a,b)')
+
+# In an arg, can we use a function substitution's name without arguments?  The
+# latter shouldn't expand.
+#
+# RUN: echo "%{fn}(%{expand-first-fn} %, %{expand-last-fn} %, %{fn} %)"
+# CHECK: fn('%{expand-first-fn}', '%{expand-last-fn}', '%{fn}')
+
+# That means we can effectively pass around function pointers.
+#
+# RUN: echo "%{call-a-b}( %{expand-last-fn} %)"
+# RUN: echo "%{call-a-b}( %{expand-first-fn} %)"
+#           CHECK: expand-last-fn(a,b)
+# CHECK-NON-RECUR: %{expand-first-fn}( a %, b %)
+#     CHECK-RECUR: expand-first-fn(a,b)
+
+# CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/bare-comma.txt

+# DEFINE: %{name}( PARAM1%, PARAM2, PARAM3%, PARAM4 %) = value
+# RUN: echo %{name}
+
+#      CHECK: Parameter 2 of substitution '%{name}' is malformed as {{.*}}
+# CHECK-NEXT: Parameter 2 is: PARAM2, PARAM3
+# CHECK-NEXT: Parameter names must be separated by '%,'
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-6]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/bare-rparen.txt

+# DEFINE: %{name}(PARAM1) = value
+# RUN: echo %{name}
+
+#      CHECK: Parameter list of substitution '%{name}' must end with '%)'
+# CHECK-NEXT: in {{DEFINE}}: directive on test line 1
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/between-name-equals.txt

-# ':' is treated like part of the name, so the name is bad.
-
 # DEFINE: %{name}:=value
 # RUN: echo %{name}
 
-#      CHECK: Substitution name '%{name}:' is malformed {{.*}}
+#      CHECK: Substitution name '%{name}' must be followed by '=' (possibly with intervening whitespace) or by '(' (without intervening whitespace)
 # CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-4]]
 
 # CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/between-name-lparen.txt

+# DEFINE: %{name} ( PARAM %) = value
+# RUN: echo %{name}
+
+#      CHECK: Substitution name '%{name}' must be followed by '=' (possibly with intervening whitespace) or by '(' (without intervening whitespace)
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-4]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/no-comma.txt

+# DEFINE: %{name}( PARAM1 PARAM2 %) = value
+# RUN: echo %{name}
+
+#      CHECK: Parameter 1 of substitution '%{name}' is malformed as {{.*}}
+# CHECK-NEXT: Parameter 1 is: PARAM1 PARAM2
+# CHECK-NEXT: Parameter names must be separated by '%,'
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-6]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/no-rparen.txt

+# DEFINE: %{name}( PARAM1 = value
+# RUN: echo %{name}
+
+#      CHECK: Parameter list of substitution '%{name}' must end with '%)'
+# CHECK-NEXT: in {{DEFINE}}: directive on test line 1
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/param-list-is-empty-string.txt

+# DEFINE: %{name}(%) = value
+# RUN: echo %{name}
+
+#      CHECK: Substitution '%{name}' must be defined with no parameter list or at least one parameter
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-4]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/param-list-is-ws.txt

+# DEFINE: %{name}( %) = value
+# RUN: echo %{name}
+
+#      CHECK: Substitution '%{name}' must be defined with no parameter list or at least one parameter
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-4]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/param-name-leading-digit.txt

+# A parameter name must be a valid python identifier for the function
+# substitution's pattern to work, and leading digits are not permitted.
+
+# DEFINE: %{foo}( 0PARAM %) = %{0PARAM}
+# RUN: echo %{foo}
+
+#      CHECK: Parameter 1 of substitution '%{foo}' is malformed as {{.*}}
+# CHECK-NEXT: Parameter 1 is: 0PARAM
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-5]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/param-name-with-braces.txt

+# For conciseness, parameter names are unbraced in the formal parameter list.
+
+# DEFINE: %{foo}( %{PARAM} %) = %{PARAM}
+# RUN: echo %{foo}
+
+#      CHECK: Parameter 1 of substitution '%{foo}' is malformed as {{.*}}
+# CHECK-NEXT: Parameter 1 is: %{PARAM}
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-5]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/assignment/param-name-with-dash.txt

+# A parameter name must be a valid python identifier for the function
+# substitution's pattern to work, and dashes are not permitted.
+
+# DEFINE: %{foo}( MY-PARAM %) = %{MY-PARAM}
+# RUN: echo %{foo}
+
+#      CHECK: Parameter 1 of substitution '%{foo}' is malformed as {{.*}}
+# CHECK-NEXT: Parameter 1 is: MY-PARAM
+# CHECK-NEXT: in {{DEFINE}}: directive on test line [[#@LINE-5]]
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/define-func-already-diff-param-count.txt

+# DEFINE: %{foo}( PARAM0 %) = %{PARAM0}
+# DEFINE: %{foo}( PARAM0 %, PARAM1 %) = %{PARAM0}, %{PARAM1}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Substitution whose pattern contains '%{foo}' is already defined before '{{DEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/define-func-already-diff-param-name.txt

+# DEFINE: %{foo}( PARAM0 %) = %{PARAM0}
+# DEFINE: %{foo}( PARAM1 %) = %{PARAM1}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Substitution whose pattern contains '%{foo}' is already defined before '{{DEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/define-func-already-non-func.txt

+# DEFINE: %{foo} = bar
+# DEFINE: %{foo}( PARAM %) = %{PARAM}
+# RUN: echo %{foo}(bar)
+
+#      CHECK: ValueError: Substitution whose pattern contains '%{foo}' is already defined before '{{DEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Existing pattern: %{foo}
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/define-func-already.txt

+# DEFINE: %{foo}( PARAM %) = %{PARAM}
+# DEFINE: %{foo}( PARAM %) = %{PARAM}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Substitution whose pattern contains '%{foo}' is already defined before '{{DEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/define-non-func-already-func.txt

+# DEFINE: %{foo}( PARAM %) = %{PARAM}
+# DEFINE: %{foo} = bar
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Substitution whose pattern contains '%{foo}' is already defined before '{{DEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/redefine-func-already-diff-param-count.txt

+# DEFINE: %{foo}( PARAM0 %) = %{PARAM0}
+# REDEFINE: %{foo}( PARAM0 %, PARAM1 %) = %{PARAM0}, %{PARAM1}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Existing substitution whose pattern contains '%{foo}' does not have the pattern specified by '{{REDEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Expected pattern: %{foo}\({{.*PARAM0.*PARAM1.*}}\)
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*PARAM0.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/redefine-func-already-diff-param-name.txt

+# DEFINE: %{foo}( PARAM0 %) = %{PARAM0}
+# REDEFINE: %{foo}( PARAM1 %) = %{PARAM1}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Existing substitution whose pattern contains '%{foo}' does not have the pattern specified by '{{REDEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Expected pattern: %{foo}\({{.*PARAM1.*}}\)
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*PARAM0.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/redefine-func-already-non-func.txt

+# DEFINE: %{foo} = bar
+# REDEFINE: %{foo}( PARAM %) = %{PARAM}
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Existing substitution whose pattern contains '%{foo}' does not have the pattern specified by '{{REDEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Expected pattern: %{foo}\({{.*}}\)
+# CHECK-NEXT: Existing pattern: %{foo}
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/errors/defined-check/redefine-non-func-already-func.txt

+# DEFINE: %{foo}( PARAM %) = %{PARAM}
+# REDEFINE: %{foo} = bar
+# RUN: echo %{foo}
+
+#      CHECK: ValueError: Existing substitution whose pattern contains '%{foo}' does not have the pattern specified by '{{REDEFINE}}:' directive at line [[#@LINE-3]]
+# CHECK-NEXT: Expected pattern: %{foo}
+# CHECK-NEXT: Existing pattern: %{foo}\({{.*}}\)
+#  CHECK-NOT: Existing pattern:
+
+# CHECK: Unresolved: 1

llvm/utils/lit/tests/Inputs/shtest-define/examples/function-subst-with-locs.txt

+; This example originally appeared in TestingGuide.rst except here we've added
+; echo to the clang/FileCheck command line to be executed.
+
+; DEFINE: %{check}( LOC %, TRIPLE %, CFLAGS %, FC_PREFIX %) =                  \
+; DEFINE:   echo 'At test lines %{LOC}, %(line)' &&                            \
+; DEFINE:   echo '                                                             \
+; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS}          \
+; DEFINE:              -triple %{TRIPLE} -emit-llvm -o - %s |                  \
+; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s                          \
+; DEFINE:   '
+
+; DEFINE: %{check-triple}( LOC %, TRIPLE %) =                                      \
+; DEFINE:   %{check}( %{LOC}, %(line) %, %{TRIPLE} %,               %, CHECK %) && \
+; DEFINE:   %{check}( %{LOC}, %(line) %, %{TRIPLE} %, -fopenmp-simd %, SIMD  %)
+
+; RUN: %{check-triple}( %(line) %, x86_64-apple-darwin10.6.0 %)
+; CHECK: At test lines [[#@LINE-1]], [[#@LINE-4]], [[#@LINE-12]]
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: At test lines [[#@LINE-3]], [[#@LINE-5]], [[#@LINE-14]]
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; RUN: %{check-triple}( %(line) %, x86_64-unknown-linux-gnu  %)
+; CHECK: At test lines [[#@LINE-1]], [[#@LINE-10]], [[#@LINE-18]]
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: At test lines [[#@LINE-3]], [[#@LINE-11]], [[#@LINE-20]]
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/examples/function-subst.txt

+; This example originally appeared in TestingGuide.rst except here we've added
+; echo to the clang/FileCheck command line to be executed.
+
+; DEFINE: %{check}( TRIPLE %, CFLAGS %, FC_PREFIX %) =                         \
+; DEFINE:   echo '                                                             \
+; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS}          \
+; DEFINE:              -triple %{TRIPLE} -emit-llvm -o - %s |                  \
+; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s                          \
+; DEFINE:   '
+
+; DEFINE: %{check-triple}( TRIPLE %) =                                         \
+; DEFINE:   %{check}( %{TRIPLE} %,               %, CHECK %) &&                \
+; DEFINE:   %{check}( %{TRIPLE} %, -fopenmp-simd %, SIMD  %)
+
+; RUN: %{check-triple}( x86_64-apple-darwin10.6.0 %)
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; RUN: %{check-triple}( x86_64-unknown-linux-gnu  %)
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/examples/param-and-function-subst.txt

+; This example originally appeared in TestingGuide.rst except here we've added
+; echo to the clang/FileCheck command line to be executed.
+
+; DEFINE: %{triple} =
+; DEFINE: %{check}( CFLAGS %, FC_PREFIX %) =                                   \
+; DEFINE:   echo '                                                             \
+; DEFINE:   %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{CFLAGS}          \
+; DEFINE:              -triple %{triple} -emit-llvm -o - %s |                  \
+; DEFINE:     FileCheck -check-prefix=%{FC_PREFIX} %s                          \
+; DEFINE:   '
+
+; DEFINE: %{check-triple} =                                                    \
+; DEFINE:   %{check}(               %, CHECK %) &&                             \
+; DEFINE:   %{check}( -fopenmp-simd %, SIMD  %)
+
+; REDEFINE: %{triple} = x86_64-apple-darwin10.6.0
+; RUN: %{check-triple}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-apple-darwin10.6.0 -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; REDEFINE: %{triple} = x86_64-unknown-linux-gnu
+; RUN: %{check-triple}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
+; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+; CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/examples/param-subst.txt

 ; REDEFINE: %{fc-prefix} = CHECK
 ; RUN: %{check}
 ; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=CHECK {{.*}}
 ;
 ; REDEFINE: %{cflags} = -fopenmp-simd
 ; REDEFINE: %{fc-prefix} = SIMD
 ; RUN: %{check}
 ; CHECK: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -fopenmp-simd -triple x86_64-unknown-linux-gnu -emit-llvm -o - {{.*}} | FileCheck -check-prefix=SIMD {{.*}}
+
+# CHECK: Passed: 1

llvm/utils/lit/tests/Inputs/shtest-define/ws-and-continuations.txt

 # Empty values are permitted and reasonable, especially when just establishing
 # expansion order.
 #
 # DEFINE: %{empty}=
 # RUN: echo "'%{empty}'"
 # CHECK:''
+#
+# REDEFINE: %{empty}=
+# RUN: echo "'%{empty}'"
+# CHECK:''
+
+# A value consisting only of whitespace is trimmed to the empty string.
+#
+#               v~~ intentional whitespace
+# DEFINE: %{ws}=   
+# RUN: echo "'%{ws}'"
+# CHECK:''
+#
+#                 v intentional whitespace
+# REDEFINE: %{ws}= 
+# RUN: echo "'%{ws}'"
+# CHECK:''
 
 # White space is not required around the name or value.
 #
-# DEFINE:%{no-whitespace}=abc
-# RUN: echo "'%{no-whitespace}'"
+# DEFINE:%{non-ws}=abc
+# RUN: echo "'%{non-ws}'"
 # CHECK:'abc'
 #
-# REDEFINE:%{no-whitespace}=HelloWorld
-# RUN: echo "'%{no-whitespace}'"
+# REDEFINE:%{non-ws}=HelloWorld
+# RUN: echo "'%{non-ws}'"
 # CHECK:'HelloWorld'
 
-# Exact white space is preserved within the value, but whitespace enclosing the
-# name or value is discarded.  ('%{' and '}' are part of the name, and
-# whitespace in between isn't permitted.)
+# White space is not required around formal parameters or arguments.
+#
+# DEFINE:%{non-ws-fn}(P0%,P1%,P2%)=non-ws-fn(%{P0},%{P1},%{P2})
+# RUN: echo "'%{non-ws-fn}(a%,b%,c%)'"
+# CHECK:'non-ws-fn(a,b,c)'
+#
+# REDEFINE:%{non-ws-fn}(P0%,P1%,P2%)=non-ws-fn(%{P2},%{P1},%{P0})
+# RUN: echo "'%{non-ws-fn}(a%,b%,c%)'"
+# CHECK:'non-ws-fn(c,b,a)'
+
+# Exact white space is preserved within the value or an actual arg, but
+# whitespace enclosing the name, the value, a formal parameter, or an actual arg
+# is discarded.  ('%{' and '}' are part of the name, and whitespace in between
+# isn't permitted.)
 #
-#                                       v~~ intentional whitespace
-# DEFINE:   %{whitespace}  =  abc    def   
-# RUN: echo "'%{whitespace}'"
+#                                      v~~ intentional whitespace
+# DEFINE:   %{ws-non-ws}  =  abc    def   
+#                                                                v~ intentional whitespace
+# DEFINE:   %{ws-non-ws-fn}(  P0 %, P1  %)  =  <%{P0}>    <%{P1}>  
+# RUN: echo "'%{ws-non-ws}'"
+# RUN: echo "'%{ws-non-ws-fn}(      abc  def     %,    ghi     jkl         %)'"
 # CHECK:'abc    def'
-#                                      v intentional whitespace
-# REDEFINE: %{whitespace} = Hello World 
-# RUN: echo "'%{whitespace}'"
+# CHECK:'<abc  def>    <ghi     jkl>'
+#                                     v intentional whitespace
+# REDEFINE: %{ws-non-ws} = Hello World 
+#                                                         v intentional whitespace
+# REDEFINE: %{ws-non-ws-fn}( P0 %, P1 %) = <%{P0}> <%{P1}> 
+# RUN: echo "'%{ws-non-ws}'"
+# RUN: echo "'%{ws-non-ws-fn}( a b c %, d e %)'"
 # CHECK:'Hello World'
+# CHECK:'<a b c> <d e>'
 
 # Line continuations in the value are permitted and collapse whitespace.
 #
-# DEFINE: %{continue} = abc\
+# DEFINE: %{value-continue} = abc\
 # DEFINE:def \
 # DEFINE:ghi\
 # DEFINE: jkl \
 # DEFINE: mno  \
 # DEFINE:  pqr 
 #             ^ intentional whitespace
-# RUN: echo "'%{continue}'"
+# RUN: echo "'%{value-continue}'"
 # CHECK:'abc def ghi jkl mno pqr'
 #
-# REDEFINE: %{continue} =  abc  \
+# REDEFINE: %{value-continue} =  abc  \
 # REDEFINE: def
-# RUN: echo "'%{continue}'"
+# RUN: echo "'%{value-continue}'"
 # CHECK:'abc def'
 
+# Line continuations in formal parameters and actual arguments are permitted.
+#
+# DEFINE: %{params-continue}(                                                  \
+# DEFINE:                     PARAM0 %, PARAM1 %, PARAM2  %, PARAM3  %,        \
+# DEFINE:                     PARAM4 %, PARAM5 %, PARAM6  %, PARAM7  %,        \
+# DEFINE:                     PARAM8 %, PARAM9 %, PARAM10 %, PARAM11 %) =      \
+# DEFINE:   %{PARAM0} %{PARAM1} %{PARAM2} %{PARAM3}                            \
+# DEFINE:   %{PARAM4} %{PARAM5} %{PARAM6} %{PARAM7}                            \
+# DEFINE:   %{PARAM8} %{PARAM9} %{PARAM10} %{PARAM11}
+# RUN: echo "'%{params-continue}(          \
+# RUN:                           a %, b %, \
+# RUN:                           c %, d %, \
+# RUN:                           e %, f %, \
+# RUN:                           g %, h %, \
+# RUN:                           i %, j %, \
+# RUN:                           k %, l %)'"
+# CHECK:'a b c d e f g h i j k l'
+
 # Whitespace at the end of the line after a '\' is ignored, and it's treated as
 # a line continuation.  Otherwise, the behavior would be hard to understand
 # because it looks like a line continuation.
 #
 #                                   v~~~~~~~~~~~ intentional whitespace
 # DEFINE: %{ws-after-continue}=foo \            
 # DEFINE: bar
 # RUN: echo "'%{ws-after-continue}'"

llvm/utils/lit/tests/shtest-define.py

 # DEFINE: %{fc-args} =
 # DEFINE: %{record-test} =                                                     \
 # DEFINE:   echo '  shtest-define :: %{test}' >> %t.tests.actual.txt
 # DEFINE: %{run-test} =                                                        \
 # DEFINE:   %{lit-pre} %{lit} -va  %{lit-args} %{my-inputs}/%{test} 2>&1 |     \
 # DEFINE:     FileCheck -match-full-lines %{fc-args} %{my-inputs}/%{test}
 # DEFINE: %{run-and-record-test} = %{run-test} && %{record-test}
 
+# REDEFINE: %{test} = actual-arg-catastrophic-backtracking.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = actual-arg-count.txt
+# RUN: %{run-and-record-test}
+
+# REDEFINE: %{test} = actual-arg-special-chars.txt
+#
+# REDEFINE: %{fc-args} = -check-prefixes=CHECK,CHECK-NON-RECUR
+# RUN: %{run-test}
+#
+# REDEFINE: %{lit-args} = -Drecur=2
+# REDEFINE: %{fc-args} = -check-prefixes=CHECK,CHECK-RECUR
+# RUN: %{run-test}
+#
+# RUN: %{record-test}
+# REDEFINE: %{lit-args} =
+# REDEFINE: %{fc-args} =
+
 # REDEFINE: %{lit-pre} = not
 #
+# REDEFINE: %{test} = errors/assignment/bare-comma.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/bare-rparen.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/before-name.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/between-name-equals.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/between-name-lparen.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-empty.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-with-dot.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-with-equals.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-with-newline.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-with-number.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/braces-with-ws.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/empty.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/no-comma.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/no-equals.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/no-name.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/no-rparen.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/param-list-is-empty-string.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/param-list-is-ws.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/param-name-leading-digit.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/param-name-with-braces.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/assignment/param-name-with-dash.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/assignment/ws-only.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/continuation/empty.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/continuation/end-in-double-backslash.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/continuation/unterminated-define-bad-redefine.txt
 # RUN: %{run-and-record-test}

 # REDEFINE: %{test} = errors/continuation/unterminated-run-redefine.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/continuation/ws-only.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-already-by-config.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-already-by-test.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/define-func-already-diff-param-count.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/define-func-already-diff-param-name.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/define-func-already-non-func.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/define-func-already.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-inside-pattern.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-multiple-exact.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-multiple-once-exact.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/define-non-func-already-func.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-prefixes-pattern.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/define-suffixes-pattern.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/redefine-func-already-diff-param-count.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/redefine-func-already-diff-param-name.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/redefine-func-already-non-func.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-inside-pattern.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-multiple-exact.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-multiple-once-exact.txt
 # RUN: %{run-and-record-test}
+# REDEFINE: %{test} = errors/defined-check/redefine-non-func-already-func.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-none.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-prefixes-pattern.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/defined-check/redefine-suffixes-pattern.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/location-range.txt
 # RUN: %{run-and-record-test}
 # REDEFINE: %{test} = errors/no-run.txt
 # RUN: %{run-and-record-test}
 #
 # REDEFINE: %{lit-pre} =
 
+# REDEFINE: %{test} = examples/function-subst-with-locs.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = examples/function-subst.txt
+# RUN: %{run-and-record-test}
+# REDEFINE: %{test} = examples/param-and-function-subst.txt
+# RUN: %{run-and-record-test}
 # REDEFINE: %{test} = examples/param-subst.txt
 # RUN: %{run-and-record-test}
 
 # REDEFINE: %{test} = expansion-order.txt
 # RUN: %{run-and-record-test}
 
 # REDEFINE: %{test} = line-number-substitutions.txt
 # RUN: %{run-and-record-test}