Config file support for clang-format, part 2.

[alexfh]

Summary

Adds actual config file reading to the clang-format utility.
Configuration file name is .clang-format. It is looked up for each input file
in its parent directories starting from immediate one. First found .clang-format
file is used. When using standard input, .clang-format is searched starting from
the current directory.
Added -dump-config option to easily create configuration files.

[jordan_rose]

tools/clang-format/ClangFormat.cpp

78–79

Not really following this, but (a) why not SmallString, and (b) why not llvm::sys::path::append?

[kimgr]

I like where this is going, found a few nits.

tools/clang-format/ClangFormat.cpp

79

Is there a path-separator-agnostic way of doing this? "/" should be OK on Windows, but not prefered.

You could run it through llvm::sys::path::native or llvm::sys::path::append.

80–81

Initialize IsFile or handle error return from is_regular_file. IsFile will be uninitialized if the function fails.

89

I think that basing the YAML parser on text content will cause diagnostics not to contain the filename.

Unfortunately it looks like the smart stuff in YAMLTraits can only work based on content, not a buffer. YAMLParser's Stream class takes a MemoryBuffer and pulls filename for diagnostics from its buffer identifier, usually a filename. You may want to provoke a diagnostic by feeding the parser malformed YAML, and see what the context is. I think the default is "YAML: ".

93

Remove this from stderr? Wasn't there a debug stream added to clang-format at some point?

[alexfh]

Use llvm::sys::path::append + SmallString. Print .clang-format file name on error. Output BasedOnStyle in -dump-config.

[kimgr]

Looks good!

Did you try provoking a syntax error? Unless it's been short-circuited, I think the YAML parsing framework will print a diagnostic before your manual llvm::errs() output, but I don't have time to rebuild and check right now.

lib/Format/Format.cpp

65–73

Maybe move this into clang::format::isPredefinedStyle(const FormatStyle& Style)?

That way you don't have to re-define the predefined style names here too.

70

IIRC, the YAML parser doesn't take "#" comments, only JavaScript-style "//", but I'm not 100% sure. Did you try roundtripping a dumped configuration?

[alexfh]

I've tried, of course. And my message goes after YAML I/O's, but this way
it's still informative enough for diagnostic purposes. And I thought we can
come back to this once YAML I/O has better support for this.

[klimek]

lib/Format/Format.cpp

49

Any reason to put an extra newline here?

67

Use llvm-style loop?

tools/clang-format/ClangFormat.cpp

83

Any reason to ignore the error?

[alexfh]

Addressed comments.

[klimek]

LG

tools/clang-format/ClangFormat.cpp

85

querying

[alexfh]

Now I have an answer about reasons to ignore errors from is_regular_file ;)

It should be enough to know that we can't read the file. In particular, we don't want "No such file or directory" to be treated as an error:

Error quering /.../work/llvm/tools/clang/.clang-format: No such file or directory
Error quering /.../work/llvm/tools/.clang-format: No such file or directory
YAML:1:1: error: not a mapping
akljsdflj
^~~~~~~~~
Error reading /.../work/llvm/.clang-format: Invalid argument
Error quering /.../work/.clang-format: No such file or directory
Error quering /.../.clang-format: No such file or directory
...
Error quering /.clang-format: No such file or directory
Can't find usable .clang-format, using LLVM style

So I removed error handling again, now output for a file with invalid contents looks better:

YAML:1:1: error: not a mapping
akljsdflj
^~~~~~~~~
Error reading /.../work/llvm/.clang-format: Invalid argument
Can't find usable .clang-format, using LLVM style

[alexfh]

lib/Format/Format.cpp

70

There's a round-trip test, and it passes for me.

include/clang/Format/Format.h

struct FormatStyle {
 
   /// \brief Add a space in front of an Objective-C protocol list, i.e. use
   /// Foo <Protocol> instead of Foo<Protocol>.
   bool ObjCSpaceBeforeProtocolList;
 
   /// \brief If \c true, aligns escaped newlines as far left as possible.
   /// Otherwise puts them into the right-most column.
   bool AlignEscapedNewlinesLeft;
+
+  bool operator==(const FormatStyle &R) const {
+    return AccessModifierOffset == R.AccessModifierOffset &&
+           AlignEscapedNewlinesLeft == R.AlignEscapedNewlinesLeft &&
+           AllowAllParametersOfDeclarationOnNextLine ==
+               R.AllowAllParametersOfDeclarationOnNextLine &&
+           AllowShortIfStatementsOnASingleLine ==
+               R.AllowShortIfStatementsOnASingleLine &&
+           BinPackParameters == R.BinPackParameters &&
+           ColumnLimit == R.ColumnLimit &&
+           ConstructorInitializerAllOnOneLineOrOnePerLine ==
+               R.ConstructorInitializerAllOnOneLineOrOnePerLine &&
+           DerivePointerBinding == R.DerivePointerBinding &&
+           IndentCaseLabels == R.IndentCaseLabels &&
+           MaxEmptyLinesToKeep == R.MaxEmptyLinesToKeep &&
+           ObjCSpaceBeforeProtocolList == R.ObjCSpaceBeforeProtocolList &&
+           PenaltyExcessCharacter == R.PenaltyExcessCharacter &&
+           PenaltyReturnTypeOnItsOwnLine == R.PenaltyReturnTypeOnItsOwnLine &&
+           PointerBindsToType == R.PointerBindsToType &&
+           SpacesBeforeTrailingComments == R.SpacesBeforeTrailingComments &&
+           Standard == R.Standard;
+  }
+
 };
 
 /// \brief Returns a format style complying with the LLVM coding standards:
 /// http://llvm.org/docs/CodingStandards.html.
 FormatStyle getLLVMStyle();
 
 /// \brief Returns a format style complying with Google's C++ style guide:
 /// http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml.

lib/Format/Format.cpp

static void enumeration(IO &io,
     io.enumCase(value, "C++03", clang::format::FormatStyle::LS_Cpp03);
     io.enumCase(value, "C++11", clang::format::FormatStyle::LS_Cpp11);
     io.enumCase(value, "Auto", clang::format::FormatStyle::LS_Auto);
   }
 };
 
 template <> struct MappingTraits<clang::format::FormatStyle> {
   static void mapping(llvm::yaml::IO &IO, clang::format::FormatStyle &Style) {
-    if (!IO.outputting()) {
+    if (IO.outputting()) {
+      StringRef StylesArray[] = { "LLVM", "Google", "Chromium", "Mozilla" };
+      ArrayRef<StringRef> Styles(StylesArray);
+      for (size_t i = 0, e = Styles.size(); i < e; ++i) {
+        StringRef StyleName(Styles[i]);
+        if (Style == clang::format::getPredefinedStyle(StyleName)) {
+          IO.mapOptional("# BasedOnStyle", StyleName);
+          break;
+        }
+      }
+    } else {
       StringRef BasedOnStyle;
       IO.mapOptional("BasedOnStyle", BasedOnStyle);
-
       if (!BasedOnStyle.empty())
         Style = clang::format::getPredefinedStyle(BasedOnStyle);
     }
 
     IO.mapOptional("AccessModifierOffset", Style.AccessModifierOffset);
     IO.mapOptional("AlignEscapedNewlinesLeft", Style.AlignEscapedNewlinesLeft);
     IO.mapOptional("AllowAllParametersOfDeclarationOnNextLine",
                    Style.AllowAllParametersOfDeclarationOnNextLine);
     IO.mapOptional("AllowShortIfStatementsOnASingleLine",
                    Style.AllowShortIfStatementsOnASingleLine);
     IO.mapOptional("BinPackParameters", Style.BinPackParameters);
     IO.mapOptional("ColumnLimit", Style.ColumnLimit);
     IO.mapOptional("ConstructorInitializerAllOnOneLineOrOnePerLine",
                    Style.ConstructorInitializerAllOnOneLineOrOnePerLine);
     IO.mapOptional("DerivePointerBinding", Style.DerivePointerBinding);
     IO.mapOptional("IndentCaseLabels", Style.IndentCaseLabels);
     IO.mapOptional("MaxEmptyLinesToKeep", Style.MaxEmptyLinesToKeep);
     IO.mapOptional("ObjCSpaceBeforeProtocolList",
                    Style.ObjCSpaceBeforeProtocolList);
     IO.mapOptional("PenaltyExcessCharacter", Style.PenaltyExcessCharacter);
unsigned format(const AnnotatedLine *NextLine) {
 
     // Find best solution in solution space.
     return analyzeSolutionSpace(State);
   }
 
 private:
   void DebugTokenState(const AnnotatedToken &AnnotatedTok) {
     const Token &Tok = AnnotatedTok.FormatTok.Tok;
-    llvm::errs() << StringRef(SourceMgr.getCharacterData(Tok.getLocation()),
+    llvm::dbgs() << StringRef(SourceMgr.getCharacterData(Tok.getLocation()),
                               Tok.getLength());
-    llvm::errs();
+    llvm::dbgs();
   }
 
   struct ParenState {
     ParenState(unsigned Indent, unsigned LastSpace, bool AvoidBinPacking,
                bool NoLineBreak)
         : Indent(Indent), LastSpace(LastSpace), FirstLessLess(0),
           BreakBeforeClosingBrace(false), QuestionColumn(0),
           AvoidBinPacking(AvoidBinPacking), BreakBeforeParameter(false),
unsigned analyzeSolutionSpace(LineState &InitialState) {
     Queue.push(QueueItem(OrderedPenalty(0, Count), Node));
     ++Count;
 
     // While not empty, take first element and follow edges.
     while (!Queue.empty()) {
       unsigned Penalty = Queue.top().first.first;
       StateNode *Node = Queue.top().second;
       if (Node->State.NextToken == NULL) {
-        DEBUG(llvm::errs() << "\n---\nPenalty for line: " << Penalty << "\n");
+        DEBUG(llvm::dbgs() << "\n---\nPenalty for line: " << Penalty << "\n");
         break;
       }
       Queue.pop();
 
       if (!Seen.insert(Node->State).second)
         // State already examined with lower penalty.
         continue;
 
       addNextStateToQueue(Penalty, Node, /*NewLine=*/ false);
       addNextStateToQueue(Penalty, Node, /*NewLine=*/ true);
     }
 
     if (Queue.empty())
       // We were unable to find a solution, do nothing.
       // FIXME: Add diagnostic?
       return 0;
 
     // Reconstruct the solution.
     reconstructPath(InitialState, Queue.top().second);
-    DEBUG(llvm::errs() << "Total number of analyzed states: " << Count << "\n");
-    DEBUG(llvm::errs() << "---\n");
+    DEBUG(llvm::dbgs() << "Total number of analyzed states: " << Count << "\n");
+    DEBUG(llvm::dbgs() << "---\n");
 
     // Return the column after the last token of the solution.
     return Queue.top().second->State.Column;
   }
 
   void reconstructPath(LineState &State, StateNode *Current) {
     // FIXME: This recursive implementation limits the possible number
     // of tokens per line if compiled into a binary with small stack space.
     // To become more independent of stack frame limitations we would need
     // to also change the TokenAnnotator.
     if (Current->Previous == NULL)
       return;
     reconstructPath(State, Current->Previous);
     DEBUG({
       if (Current->NewLine) {
-        llvm::errs()
+        llvm::dbgs()
             << "Penalty for splitting before "
             << Current->Previous->State.NextToken->FormatTok.Tok.getName()
             << ": " << Current->Previous->State.NextToken->SplitPenalty << "\n";
       }
     });
     addTokenToState(Current->NewLine, false, State);
   }
 

tools/clang-format/ClangFormat.cpp

 
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticOptions.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Format/Format.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Rewrite/Core/Rewriter.h"
+#include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Signals.h"
 
 using namespace llvm;
 
 static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden);
 
 static cl::list<unsigned>
 Offsets("offset", cl::desc("Format a range starting at this file offset. Can "
                            "only be used with one input file."));
 static cl::list<unsigned>
 Lengths("length", cl::desc("Format a range of this length. "
                            "When it's not specified, end of file is used. "
                            "Can only be used with one input file."));
 static cl::opt<std::string> Style(
     "style",
-    cl::desc("Coding style, currently supports: LLVM, Google, Chromium, Mozilla."),
+    cl::desc(
+        "Coding style, currently supports: LLVM, Google, Chromium, Mozilla. "
+        "Use '-style file' to load style configuration from .clang-format file "
+        "located in one of the parent directories of the source file (or "
+        "current directory for stdin)."),
     cl::init("LLVM"));
 static cl::opt<bool> Inplace("i",
                              cl::desc("Inplace edit <file>s, if specified."));
 
 static cl::opt<bool> OutputXML(
     "output-replacements-xml", cl::desc("Output replacements as XML."));
+static cl::opt<bool>
+    DumpConfig("dump-config",
+               cl::desc("Dump configuration options to stdout and exit. Can be used with -style option."));
 
 static cl::list<std::string> FileNames(cl::Positional,
                                        cl::desc("[<file> ...]"));
 
 namespace clang {
 namespace format {
 
 static FileID createInMemoryFile(StringRef FileName, const MemoryBuffer *Source,
                                  SourceManager &Sources, FileManager &Files) {
   const FileEntry *Entry = Files.getVirtualFile(FileName == "-" ? "<stdin>" :
                                                     FileName,
                                                 Source->getBufferSize(), 0);
   Sources.overrideFileContents(Entry, Source, true);
   return Sources.createFileID(Entry, SourceLocation(), SrcMgr::C_User);
 }
 
-static FormatStyle getStyle() {
-  FormatStyle TheStyle = getGoogleStyle();
-  if (Style == "LLVM")
-    TheStyle = getLLVMStyle();
-  else if (Style == "Chromium")
-    TheStyle = getChromiumStyle();
-  else if (Style == "Mozilla")
-    TheStyle = getMozillaStyle();
-  else if (Style != "Google")
-    llvm::errs() << "Unknown style " << Style << ", using Google style.\n";
-
-  return TheStyle;
+FormatStyle getStyle(StringRef StyleName, StringRef FileName) {
+  if (!StyleName.equals_lower("file"))
+    return getPredefinedStyle(StyleName);
+
+  SmallString<128> Path(FileName);
+  llvm::sys::fs::make_absolute(Path);
+  for (StringRef Directory = llvm::sys::path::parent_path(Path);
+       !Directory.empty();
+       Directory = llvm::sys::path::parent_path(Directory)) {
+    SmallString<128> ConfigFile(Directory);
+    llvm::sys::path::append(ConfigFile, ".clang-format");
+    DEBUG(llvm::dbgs() << "Trying " << ConfigFile << "...\n");
+    bool IsFile = false;
+    llvm::sys::fs::is_regular_file(Twine(ConfigFile), IsFile);
+    if (IsFile) {
+      OwningPtr<MemoryBuffer> Text;
+      if (error_code ec = MemoryBuffer::getFile(ConfigFile, Text)) {
+        llvm::errs() << ec.message() << "\n";
+        continue;
+      }
+      FormatStyle Style;
+      if (error_code ec = parseConfiguration(Text->getBuffer(), &Style)) {
+        llvm::errs() << "Error reading " << ConfigFile << ": " << ec.message()
+                     << "\n";
+        continue;
+      }
+      DEBUG(llvm::dbgs() << "Using configuration file " << ConfigFile << "\n");
+      return Style;
+    }
+  }
+  llvm::errs() << "Can't find usable .clang-format, using LLVM style\n";
+  return getLLVMStyle();
 }
 
 // Returns true on error.
 static bool format(std::string FileName) {
   FileManager Files((FileSystemOptions()));
   DiagnosticsEngine Diagnostics(
       IntrusiveRefCntPtr<DiagnosticIDs>(new DiagnosticIDs),
       new DiagnosticOptions);
if (i < Lengths.size()) {
         return true;
       }
       End = Start.getLocWithOffset(Lengths[i]);
     } else {
       End = Sources.getLocForEndOfFile(ID);
     }
     Ranges.push_back(CharSourceRange::getCharRange(Start, End));
   }
-  tooling::Replacements Replaces = reformat(getStyle(), Lex, Sources, Ranges);
+  tooling::Replacements Replaces =
+      reformat(getStyle(Style, FileName), Lex, Sources, Ranges);
   if (OutputXML) {
     llvm::outs()
         << "<?xml version='1.0'?>\n<replacements xml:space='preserve'>\n";
     for (tooling::Replacements::const_iterator I = Replaces.begin(),
                                                E = Replaces.end();
          I != E; ++I) {
       llvm::outs() << "<replacement "
                    << "offset='" << I->getOffset() << "' "
cl::ParseCommandLineOptions(
       "and writes the result to the standard output.\n"
       "If <file>s are given, it reformats the files. If -i is specified \n"
       "together with <file>s, the files are edited in-place. Otherwise, the \n"
       "result is written to the standard output.\n");
 
   if (Help)
     cl::PrintHelpMessage();
 
+  if (DumpConfig) {
+    std::string Config = clang::format::configurationAsText(
+        clang::format::getStyle(Style, FileNames.empty() ? "-" : FileNames[0]));
+    llvm::outs() << Config << "\n";
+    return 0;
+  }
+
   bool Error = false;
   switch (FileNames.size()) {
   case 0:
     Error = clang::format::format("-");
     break;
   case 1:
     Error = clang::format::format(FileNames[0]);
     break;

unittests/Format/FormatTest.cpp

verifyFormat("if (foo)\n"
                "test();");
   verifyFormat("int a[] = { void forgot_closing_brace()\n"
                "{\n"
                "  f();\n"
                "  g();\n"
                "}");
 }
 
-bool operator==(const FormatStyle &L, const FormatStyle &R) {
-  return L.AccessModifierOffset == R.AccessModifierOffset &&
-         L.AlignEscapedNewlinesLeft == R.AlignEscapedNewlinesLeft &&
-         L.AllowAllParametersOfDeclarationOnNextLine ==
-             R.AllowAllParametersOfDeclarationOnNextLine &&
-         L.AllowShortIfStatementsOnASingleLine ==
-             R.AllowShortIfStatementsOnASingleLine &&
-         L.BinPackParameters == R.BinPackParameters &&
-         L.ColumnLimit == R.ColumnLimit &&
-         L.ConstructorInitializerAllOnOneLineOrOnePerLine ==
-             R.ConstructorInitializerAllOnOneLineOrOnePerLine &&
-         L.DerivePointerBinding == R.DerivePointerBinding &&
-         L.IndentCaseLabels == R.IndentCaseLabels &&
-         L.MaxEmptyLinesToKeep == R.MaxEmptyLinesToKeep &&
-         L.ObjCSpaceBeforeProtocolList == R.ObjCSpaceBeforeProtocolList &&
-         L.PenaltyExcessCharacter == R.PenaltyExcessCharacter &&
-         L.PenaltyReturnTypeOnItsOwnLine == R.PenaltyReturnTypeOnItsOwnLine &&
-         L.PointerBindsToType == R.PointerBindsToType &&
-         L.SpacesBeforeTrailingComments == R.SpacesBeforeTrailingComments &&
-         L.Standard == R.Standard;
-}
-
 bool allStylesEqual(ArrayRef<FormatStyle> Styles) {
   for (size_t i = 1; i < Styles.size(); ++i)
     if (!(Styles[0] == Styles[i]))
       return false;
   return true;
 }
 
 TEST_F(FormatTest, GetsPredefinedStyleByName) {