Add [some] template argument-extraction calls to libclang & cindex

[rspringer]

Summary

Adds the following calls to libclang/cindex:

• clang_Cursor_getNumTemplateArgument: Retrieves the number of template arguments specified in a FunctionDecl
• clang_Cursor_getTemplateArgumentKind: Retrieves a CXTemplateArgumentKind (defined in this change, corresponds to clang::TemplateArgument::ArgKind) the number of template arguments specified in a FunctionDecl
• clang_Cursor_getTemplateArgumentType: Retrieves a CXType describing the n'th [template type] argument of a FunctionDecl
• clang_Cursor_getTemplateArgumentValue: Retrives, as a [signed] long long, the integral value of the n'th argument of a FunctionDecl
• clang_Cursor_getTemplateArgumentUnsignedValue: Retrives, as an unsigned long long, the integral value of the n'th argument of a FunctionDecl

[eliben]

bindings/python/tests/cindex/test_cursor.py

260

Since you're really only looking for one cursor for 'foo' here, maybe this loop can be simplified away? I.g. no need for 'found' etc. The test and its kin can be made shorter

264

Just:

assert foo.get_num_template_arguments() == 3

?

287

Why the ctypes casts here?

include/clang-c/Index.h

2947

Since this is all only for function decls that are specializations, IMHO the comment should mention this. A small code sample could help too

tools/libclang/CXCursor.cpp

1091

Why 0 and not -1 here?

[rspringer]

Updated patch coming shortly.

bindings/python/tests/cindex/test_cursor.py

260

That makes sense - it took me a few tries to get my test case code written correctly, so "found" was verifying, for me, that a template was present.
However, since the FUNCTION_DECL is returned after the FUNCTION_TEMPLATE, I can eliminate the loops.

264

Yep. Done.

287

get_template_argument_value and get_template_argument_unsigned_value return the values as 64-bit signed and unsigned integers, respectively, but the values as declared in the template are ints and bools, respectively, so I want to cast them to 32-bit values to make sure they're extracted correctly as those values.

It's really not necessary for the bools (obviously), but I suppose I'm a fan of symmetry. I'm happy to change that, though, if you think they're just clutter.

include/clang-c/Index.h

2947

Comment updated and examples added. It may be a touch verbose now, but I'll let you be the judge.

tools/libclang/CXCursor.cpp

1091

Oversight. Fixed & thanks!

[rspringer]

Updated patch, based on last round of review comments.

[rspringer]

bindings/python/tests/cindex/test_cursor.py

287

After further thought, I realized this was only necessary b/c I was defining get_template_argument_value in terms of get_template_argument_unsigned_value (which zero-extends, rather than sign-extends, its argument). When going to fix that, I realized that a large body of the code in that area could be refactored, which I've now done.
It's a lot better now; PTAL.

[rspringer]

Factored out common code between clang_Cursor_getTemplateArgumentType, clang_Cursor_getTemplateArgumentValue, and clang_Cursor_getTemplateArgumentUnsignedValue, and simplified test cases.

[eliben]

This looks good; just a couple of nits

bindings/python/tests/cindex/test_cursor.py

275

One empty line here for consistency?

include/clang-c/Index.h

3016

I think an example with an actual unsigned would be more useful here :)

[akyrtzi]

Hi Rob,

Thank you for the enhancements in the libclang API!
Looks good, some nitpicks:

+ * If the argument cursor cannot be converted into a template function
+ * declaration, -1 is returned.
<..>
+CINDEX_LINKAGE int clang_Cursor_getNumTemplateArguments(CXCursor C);

+ * If the argument CXCursor does not represent a FunctionDecl whose I'th
+ * template argument has a kind of ArgKind::Integral, an invalid type is\
+ * returned (in debug mode, this will assert).
<..>
+CINDEX_LINKAGE CXType clang_Cursor_getTemplateArgumentType(CXCursor C,
+ unsigned I);

The convention in the libclang API is that if the function can return an unambiguous invalid value, we do not assert. If you document the behavior of the function ("-1 is returned") you should allow the client to depend on it for both debug and release.
I suggest removing the assertions in clang_Cursor_getNumTemplateArguments and clang_Cursor_getTemplateArgumentType.

+ * If the argument CXCursor does not represent a FunctionDecl whose I'th
+ * template argument has a kind of ArgKind::Integral, an undefined value will be
+ * returned (in debug mode, this call will fail an assert).
<..>
+ * If called with I = 1 or 2, -7 or true will be returned, respectively.
+ * For I == 0, an invalid value will be returned or an assert will be triggered.
+ */
+CINDEX_LINKAGE long long clang_Cursor_getTemplateArgumentValue(CXCursor C,
+ unsigned I);

I suggest to keep the assertion in code, but don't document it and be so specific, say "it is undefined to call this function on a CXCursor that does not represent a FunctionDecl or the I'th template argument is not an integral value"
Same for clang_Cursor_getTemplateArgumentUnsignedValue.

+ if (Cursor.kind == CXCursor_FunctionDecl) {
+ // Collect the template parameter kinds from the base template.
+ unsigned NumTemplateArgs = clang_Cursor_getNumTemplateArguments(Cursor);
+ enum CXCursorKind *TemplateParamKinds = (enum CXCursorKind*)
+ malloc(NumTemplateArgs * sizeof(enum CXCursorKind));
+ VisitTemplateParamsData VTPD;
+ VTPD.TemplateParamKinds = TemplateParamKinds;
+ VTPD.CurrentIndex = 0;
+ clang_visitChildren(SpecializationOf, visitTemplateParams,
+ &VTPD);
+
+ for (unsigned I = 0; I < NumTemplateArgs; I++) {
+ if (TemplateParamKinds[I] == CXCursor_TemplateTypeParameter) {
+ CXType T = clang_Cursor_getTemplateArgumentType(Cursor, I);
+ CXString S = clang_getTypeSpelling(T);
+ printf(" [Template arg %d: type: %s]", I, clang_getCString(S));
+ clang_disposeString(S);
+ } else if (TemplateParamKinds[I] ==
+ CXCursor_NonTypeTemplateParameter) {
+ printf(" [Template arg %d: intval: %lld]",
+ I, clang_Cursor_getTemplateArgumentValue(Cursor, I));
+ }
+ }
+ free(TemplateParamKinds);
+ }

}

This indicates that we are missing another function to make using the others convenient. We are missing a function to get the kind of the I'th template argument, maybe have a function return if the I'th argument is a type or value ?

Also could you please add some tests for this new functionality, e.g. enhance 'test/Index/index-templates.cpp' ?

[rspringer]

Hi Argyrios,
Thanks a ton for the review! I think the patch is much better now, for it.

Responses:
+ The convention in the libclang API is that if the function can return an unambiguous invalid value, we do not + assert. If you document the behavior of the function ("-1 is returned") you should allow the client to depend on it for both debug and release.
+ I suggest removing the assertions in clang_Cursor_getNumTemplateArguments and clang_Cursor_getTemplateArgumentType.

Done.
One question: ...getNumTemplateArguments and other functions depend on the helper clang_Cursor_getTemplateArgument, which also has an unambiguous "bad" return ("false"), so I've removed the asserts there, as well (or else getNumTemplateArguments would still throw them). It seems like a shame, though, to lose the debugging information that was captured in the assert message ("Invalid template argument index", for example, in the prior patch revision). Is there a convention to preserve that information somehow, or is that not standard Clang practice?

+ This indicates that we are missing another function to make using the others convenient. We are missing a function to get the kind of the I'th template argument, maybe have a function return if the I'th argument is a type or value ?
+ Also could you please add some tests for this new functionality, e.g. enhance 'test/Index/index-templates.cpp' ?

That's a good call - you're absolutely right. I've added clang_Cursor_getTemplateArgumentKind and appropriate unit tests. Give them a once-over when you get a chance.

While porting this change to cindex.py, I realized that there was a lot of duplicated code for enumeration representation. I may have bitten off more than I can chew, but I at least took a stab at unifying all that somewhat. It definitely needs to be re-reviewed. :)

Again - thank you both for the reviews (updated patch coming shortly)!

Updated tests are still passing.

bindings/python/tests/cindex/test_cursor.py

275

Done.

include/clang-c/Index.h

3016

That's...a good point. Done.

[rspringer]

I overlooked the request to update test/Index/index-templates.cpp; I'll get right on that.

[rspringer]

Added some tests to index-templates.cpp; I can add more if necessary/desired (...but I was having some difficulty getting more TemplateArgument::ArgKinds to be expressed).

[rspringer]

Updated based on out-of-thread comments:

• Added enum to encode failures of clang_Cursor_getTemplateArgument
• Corrected clang_Cursor_getTemplateArgumentType comment.

[rspringer]

Updated a comment with an updated type name (ArgKind::Integral -> CXTemplateArgKind_Integral).

[eliben]

Committed in r219529.

bindings/python/clang/cindex.py

         for i in range(0, num_args):
             yield conf.lib.clang_Cursor_getArgument(self, i)
 
+    def get_num_template_arguments(self):
+        """Returns the number of template args associated with this cursor."""
+        return conf.lib.clang_Cursor_getNumTemplateArguments(self)
+
+    def get_template_argument_type(self, num):
+        """Returns the CXType for the indicated template argument."""
+        return conf.lib.clang_Cursor_getTemplateArgumentType(self, num)
+
+    def get_template_argument_value(self, num):
+        """Returns the value of the indicated arg as a signed 64b integer."""
+        return conf.lib.clang_Cursor_getTemplateArgumentValue(self, num)
+
+    def get_template_argument_unsigned_value(self, num):
+        """Returns the value of the indicated arg as an unsigned 64b integer."""
+        return conf.lib.clang_Cursor_getTemplateArgumentUnsignedValue(self, num)
+
     def get_children(self):
         """Return an iterator for accessing the children of this cursor."""
 

    Cursor,
    Cursor.from_result),
 
+  ("clang_Cursor_getNumTemplateArguments",
+   [Cursor],
+   c_int),
+
+  ("clang_Cursor_getTemplateArgumentType",
+   [Cursor, c_uint],
+   Type,
+   Type.from_result),
+
+  ("clang_Cursor_getTemplateArgumentValue",
+   [Cursor, c_uint],
+   c_longlong),
+
+  ("clang_Cursor_getTemplateArgumentUnsignedValue",
+   [Cursor, c_uint],
+   c_ulonglong),
+
   ("clang_Cursor_isBitField",
    [Cursor],
    bool),

bindings/python/tests/cindex/test_cursor.py

+import ctypes
 import gc
 
 from clang.cindex import CursorKind

     assert arguments[0].spelling == "i"
     assert arguments[1].spelling == "j"
 
+kTemplateArgTest = """\
+        template <int kInt, typename T, bool kBool>
+        void foo();
+
+        template<>
+        void foo<-7, float, true>();
+    """
+
+def test_get_num_template_arguments():
+    tu = get_tu(kTemplateArgTest, lang='cpp')
+    foos = get_cursors(tu, 'foo')
+
+    assert foos[1].get_num_template_arguments() == 3
+
+def test_get_template_argument_type():
+    tu = get_tu(kTemplateArgTest, lang='cpp')
+    foos = get_cursors(tu, 'foo')
+
+    assert foos[1].get_template_argument_type(1).kind == TypeKind.FLOAT
+
+def test_get_template_argument_value():
+    tu = get_tu(kTemplateArgTest, lang='cpp')
+    foos = get_cursors(tu, 'foo')
+
+    assert foos[1].get_template_argument_value(0) == -7
+    assert foos[1].get_template_argument_value(2) == True
+
+
+def test_get_template_argument_unsigned_value():
+    tu = get_tu(kTemplateArgTest, lang='cpp')
+    foos = get_cursors(tu, 'foo')
+
+    assert foos[1].get_template_argument_unsigned_value(0) == 2 ** 32 - 7
+    assert foos[1].get_template_argument_unsigned_value(2) == True
+
 def test_referenced():
     tu = get_tu('void foo(); void bar() { foo(); }')
     foo = get_cursor(tu, 'foo')

include/clang-c/Index.h

 CINDEX_LINKAGE CXCursor clang_Cursor_getArgument(CXCursor C, unsigned i);
 
 /**
+ *\brief Returns the number of template args of a function decl representing a
+ * template specialization.
+ *
+ * If the argument cursor cannot be converted into a template function
+ * declaration, -1 is returned.
+ *
+ * For example, for the following declaration and specialization:
+ *   template <typename T, int kInt, bool kBool>
+ *   void foo() { ... }
+ *
+ *   template <>
+ *   void foo<float, -7, true>();
+ *
+ * When this is called, 3 will be returned.
+ */
+CINDEX_LINKAGE int clang_Cursor_getNumTemplateArguments(CXCursor C);
+
+/**
+ * \brief Retrieve a CXType representing the type of a TemplateArgument of a
+ *  function decl representing a template specialization.
+ *
+ * If the argument CXCursor does not represent a FunctionDecl whose I'th
+ * template argument has a kind of ArgKind::Integral, an invalid type is\
+ * returned (in debug mode, this will assert).
+ *
+ * For example, for the following declaration and specialization:
+ *   template <typename T, int kInt, bool kBool>
+ *   void foo() { ... }
+ *
+ *   template <>
+ *   void foo<float, -7, true>();
+ *
+ * If called with I = 0, "float", will be returned.
+ * Invalid types will be returned for I == 1 or 2.
+ */
+CINDEX_LINKAGE CXType clang_Cursor_getTemplateArgumentType(CXCursor C,
+                                                           unsigned I);
+
+/**
+ * \brief Retrieve the value of an Integral TemplateArgument (of a function
+ *  decl representing a template specialization) as a signed long long.
+ *
+ * If the argument CXCursor does not represent a FunctionDecl whose I'th
+ * template argument has a kind of ArgKind::Integral, an undefined value will be
+ * returned (in debug mode, this call will fail an assert).
+ *
+ * For example, for the following declaration and specialization:
+ *   template <typename T, int kInt, bool kBool>
+ *   void foo() { ... }
+ *
+ *   template <>
+ *   void foo<float, -7, true>();
+ *
+ * If called with I = 1 or 2, -7 or true will be returned, respectively.
+ * For I == 0, an invalid value will be returned or an assert will be triggered.
+ */
+CINDEX_LINKAGE long long clang_Cursor_getTemplateArgumentValue(CXCursor C,
+                                                               unsigned I);
+
+/**
+ * \brief Retrieve the value of an Integral TemplateArgument (of a function
+ *  decl representing a template specialization) as an unsigned long long.
+ *
+ * If the argument CXCursor does not represent an TemplateArgument with a kind
+ * ArgKind::Integral, an undefined value will be returned (in debug mode, this
+ * call will fail an assert).
+ *
+ * For example, for the following declaration and specialization:
+ *   template <typename T, int kInt, bool kBool>
+ *   void foo() { ... }
+ *
+ *   template <>
+ *   void foo<float, -7, true>();
+ *
+ * If called with I = 1 or 2, 2^32 - 7 or true will be returned, respectively.
+ * For I == 0, an invalid value will be returned or an assert will be triggered.
+ */
+CINDEX_LINKAGE unsigned long long clang_Cursor_getTemplateArgumentUnsignedValue(
+    CXCursor C, unsigned I);
+
+/**
  * \brief Determine whether two CXTypes represent the same type.
  *
  * \returns non-zero if the CXTypes represent the same type and

test/Index/preamble_macro_template.cpp

 
 // RUN: c-index-test -write-pch %t.pch -fno-delayed-template-parsing -x c++-header %S/Inputs/preamble_macro_template.h
 // RUN: env CINDEXTEST_EDITING=1 c-index-test -test-load-source-reparse 5 local -fno-delayed-template-parsing -I %S/Inputs -include %t %s 2>&1 | tee %t.check.txt | FileCheck %s
-// CHECK: preamble_macro_template.h:4:6: FunctionDecl=foo:4:6 (Definition) [Specialization of foo:4:6] Extent=[4:1 - 6:2]
+// CHECK: preamble_macro_template.h:4:6: FunctionDecl=foo:4:6 (Definition) [Specialization of foo:4:6] [Template arg 0: type: int] Extent=[4:1 - 6:2]
 // CHECK: preamble_macro_template.h:4:13: ParmDecl=p:4:13 (Definition) Extent=[4:10 - 4:14]
 // CHECK: preamble_macro_template.h:4:16: CompoundStmt= Extent=[4:16 - 6:2]
 // CHECK: preamble_macro_template.h:5:3: CStyleCastExpr= Extent=[5:3 - 5:27]

tools/c-index-test/c-index-test.c

   return (int)lhs->col - (int)rhs->col;
 }
 
+typedef struct {
+  enum CXCursorKind *TemplateParamKinds;
+  unsigned CurrentIndex;
+} VisitTemplateParamsData;
+
+enum CXChildVisitResult visitTemplateParams(CXCursor Cursor, CXCursor Parent,
+                                            CXClientData ClientData) {
+  VisitTemplateParamsData *VTPD = (VisitTemplateParamsData*) ClientData;
+  VTPD->TemplateParamKinds[VTPD->CurrentIndex++] = clang_getCursorKind(Cursor);
+  return CXChildVisit_Continue;
+}
+
 static void PrintCursor(CXCursor Cursor, const char *CommentSchemaFile) {
   CXTranslationUnit TU = clang_Cursor_getTranslationUnit(Cursor);
   if (clang_isInvalid(Cursor.kind)) {

       printf(" [access=%s isVirtual=%s]", accessStr,
              isVirtual ? "true" : "false");
     }
-    
     SpecializationOf = clang_getSpecializedCursorTemplate(Cursor);
     if (!clang_equalCursors(SpecializationOf, clang_getNullCursor())) {
       CXSourceLocation Loc = clang_getCursorLocation(SpecializationOf);
       CXString Name = clang_getCursorSpelling(SpecializationOf);
       clang_getSpellingLocation(Loc, 0, &line, &column, 0);
-      printf(" [Specialization of %s:%d:%d]", 
+      printf(" [Specialization of %s:%d:%d]",
              clang_getCString(Name), line, column);
       clang_disposeString(Name);
+
+      if (Cursor.kind == CXCursor_FunctionDecl) {
+        // Collect the template parameter kinds from the base template.
+        unsigned NumTemplateArgs = clang_Cursor_getNumTemplateArguments(Cursor);
+        enum CXCursorKind *TemplateParamKinds = (enum CXCursorKind*)
+            malloc(NumTemplateArgs * sizeof(enum CXCursorKind));
+        VisitTemplateParamsData VTPD;
+        VTPD.TemplateParamKinds = TemplateParamKinds;
+        VTPD.CurrentIndex = 0;
+        clang_visitChildren(SpecializationOf, visitTemplateParams,
+                            &VTPD);
+
+        for (unsigned I = 0; I < NumTemplateArgs; I++) {
+          if (TemplateParamKinds[I] == CXCursor_TemplateTypeParameter) {
+            CXType T = clang_Cursor_getTemplateArgumentType(Cursor, I);
+            CXString S = clang_getTypeSpelling(T);
+            printf(" [Template arg %d: type: %s]", I, clang_getCString(S));
+            clang_disposeString(S);
+          } else if (TemplateParamKinds[I] ==
+                     CXCursor_NonTypeTemplateParameter) {
+            printf(" [Template arg %d: intval: %lld]",
+                   I, clang_Cursor_getTemplateArgumentValue(Cursor, I));
+          }
+        }
+        free(TemplateParamKinds);
+      }
     }
 
     clang_getOverriddenCursors(Cursor, &overridden, &num_overridden);

tools/libclang/CXCursor.cpp

   return clang_getNullCursor();
 }
 
+int clang_Cursor_getNumTemplateArguments(CXCursor C) {
+  if (clang_getCursorKind(C) != CXCursor_FunctionDecl) {
+    assert(0 && "Cursor is not a FunctionDecl");
+    return -1;
+  }
+
+  const FunctionDecl *FD = llvm::dyn_cast_or_null<clang::FunctionDecl>(
+      getCursorDecl(C));
+  if (!FD) {
+    assert(0 && "Cursor decl not dyn_cast-able to FunctionDecl");
+    return -1;
+  }
+
+  const FunctionTemplateSpecializationInfo* SpecInfo =
+      FD->getTemplateSpecializationInfo();
+  if (!SpecInfo) {
+    assert(0 && "NULL FunctionTemplateSpecializationInfo retrieved");
+    return -1;
+  }
+
+  return SpecInfo->TemplateArguments->size();
+}
+
+static bool clang_Cursor_getTemplateArgument(
+    CXCursor C, unsigned I, TemplateArgument *TA) {
+  if (clang_getCursorKind(C) != CXCursor_FunctionDecl) {
+    assert(0 && "Cursor is not a FunctionDecl");
+    return false;
+  }
+
+  const FunctionDecl *FD = llvm::dyn_cast_or_null<clang::FunctionDecl>(
+      getCursorDecl(C));
+  if (!FD) {
+    assert(0 && "Cursor decl not dyn_cast-able to FunctionDecl");
+    return false;
+  }
+
+  const FunctionTemplateSpecializationInfo* SpecInfo =
+      FD->getTemplateSpecializationInfo();
+  if (!SpecInfo) {
+    assert(0 && "NULL FunctionTemplateSpecializationInfo retrieved");
+    return false;
+  }
+
+  if (I >= SpecInfo->TemplateArguments->size()) {
+    assert(0 && "Invalid template argument index");
+    return false;
+  }
+
+  *TA = SpecInfo->TemplateArguments->get(I);
+  return true;
+}
+
+CXType clang_Cursor_getTemplateArgumentType(CXCursor C, unsigned I) {
+  TemplateArgument TA;
+  if (!clang_Cursor_getTemplateArgument(C, I, &TA)) {
+    return cxtype::MakeCXType(QualType(), getCursorTU(C));
+  }
+
+  if (TA.getKind() != TemplateArgument::Type) {
+    assert(0 && "Passed template argument is not a Type");
+    return cxtype::MakeCXType(QualType(), getCursorTU(C));
+  }
+  return cxtype::MakeCXType(TA.getAsType(), getCursorTU(C));
+}
+
+long long clang_Cursor_getTemplateArgumentValue(CXCursor C, unsigned I) {
+  TemplateArgument TA;
+  if (!clang_Cursor_getTemplateArgument(C, I, &TA)) {
+    return 0;
+  }
+
+  if (TA.getKind() != TemplateArgument::Integral) {
+    assert(0 && "Passed template argument is not Integral");
+    return 0;
+  }
+
+  return TA.getAsIntegral().getSExtValue();
+}
+
+unsigned long long clang_Cursor_getTemplateArgumentUnsignedValue(CXCursor C,
+                                                                 unsigned I) {
+  TemplateArgument TA;
+  if (!clang_Cursor_getTemplateArgument(C, I, &TA)) {
+    return 0;
+  }
+
+  if (TA.getKind() != TemplateArgument::Integral) {
+    assert(0 && "Passed template argument is not Integral");
+    return 0;
+  }
+
+  return TA.getAsIntegral().getZExtValue();
+}
+
 } // end: extern "C"
 
 //===----------------------------------------------------------------------===//

tools/libclang/libclang.exports

 clang_CXXMethod_isStatic
 clang_CXXMethod_isVirtual
 clang_Cursor_getArgument
+clang_Cursor_getNumTemplateArguments
+clang_Cursor_getTemplateArgumentType
+clang_Cursor_getTemplateArgumentValue
+clang_Cursor_getTemplateArgumentUnsignedValue
 clang_Cursor_getBriefCommentText
 clang_Cursor_getCommentRange
 clang_Cursor_getMangling