[clangd] Add support for different file URI schemas.

[ioeric]

Summary

I will replace the existing URI struct in Protocol.h with the new URI and rename FileURI to URI in a followup patch.

[sammccall]

Lots of quibbles here, some things we can certainly defer. Happy to chat offline if you're not sure...

clangd/URI.cpp

17 ↗

(On Diff #129438)

This isn't correct, we need to escape some characters, turn \ to /, etc.

It's not quite obvious what to do here in general, because URI escaping rules are defined in terms of the full generic URI syntax.

I'd suggest:

1. instead of just splitting scheme/path, split scheme/authority/path if it starts with //, authority is the part up to the next /. Otherwise empty.

Splitting authority/scheme/path allows "file" schema to just care about the path part, not the // prefix, without mandating that all schemas have authorities.

2. when deserializing authority/scheme, decode all percent-encoding. when serializing, percent-encode characters that are neither reserved nor unreserved.

(The reserved characters basically have no business in filenames, apart from / which you don't want escaped. This allows custom URI schemes to be flexible)

35 ↗

(On Diff #129438)

this error needs to be handled, it shouldn't be an assert

clangd/URI.h

19 ↗

(On Diff #129438)

Why allow an invalid state?
I think I'd lean towards more type safety here: having a few factory functions that always produce valid URIs (like fromPath) and some that return Expected<URI> (like parse), but require URI itself to be valid.

This would probably mean making it a class and having schema/path be readonly...

19 ↗

(On Diff #129438)

This doesn't really describe what it's for. Consider:

/// A URI describes the location of a source file.
/// In the simplest case, this is a file:// URI that directly encodes the absolute path to a file.
/// More abstract cases are possible: a shared index service might expose repo:// URIs that
/// are relative to the source control root.

21–22 ↗

(On Diff #129438)

nit: scheme, not schema (they're synonyms, but URIs always talk about scheme)

24 ↗

(On Diff #129438)

I think calling this "path" is confusing, because there's ambiguity between the URI path component, and the file path that you're ultimately trying to transform it into.

Consider "body", without changing semantics.

31 ↗

(On Diff #129438)

(if you keep this: isValid(), or just valid(), or operator bool())

38 ↗

(On Diff #129438)

fromURI is confusing, because the input is a String and the output is a URI.
parse()?

45 ↗

(On Diff #129438)

Hmm, it manages file paths in exactly one schema :-)
"manages" is also very vague. What about

/ \brief URIScheme is an extension point for teaching clangd to recognize a custom URI scheme.
A scheme's job is to translate between URIs and file paths.

53 ↗

(On Diff #129438)

nit: move this to the registry?

64 ↗

(On Diff #129438)

Uri not in schema: why would the framework even call getAbsolutePath in that case?
Easier just to make that inexpressible by just having the framework pass the path/body as a stringref, I think.

We should have a way to handle errors though, e.g. file:// URIs that have relative paths. Return Expected rather than use the empty string for signaling?

65 ↗

(On Diff #129438)

make currentfile the last param and optional? Uri is the primary param

69 ↗

(On Diff #129438)

again:

• return std::string to avoid the possibility of returning a FileURI with the wrong scheme
• maybe provide a way to signal errors. In this case, optional<string> might be right, as we don't know in advance which scheme to use so failures are expected.

74 ↗

(On Diff #129438)

this should be defined in the cpp file

89 ↗

(On Diff #129438)

There are other possible failure cases.
Please propagate errors here rather than magic empty strings.

90 ↗

(On Diff #129438)

this is part of the "user" API, not the extension API, move up above?
in fact, should this maybe be a method on URI?

90 ↗

(On Diff #129438)

I think you want a public function to turn an absolute path into a file:// URI.
When communicating with the editor, we don't want to send custom URI schemes.

[ioeric]

• Addressed review comments.

[ioeric]

Thanks for the comments! I've refactored the code according to your suggestions. PTAL

[sammccall]

Few more interface nits, sorry for picking here! Will get through the impl in the morning.

clangd/URI.cpp

43 ↗

(On Diff #130201)

Don't you want the opposite here, convert from unix to native?

50 ↗

(On Diff #130201)

this error can be returned by the framework already (it's valid for all paths)

57 ↗

(On Diff #130201)

conversely, here you want native (which is the default)

58 ↗

(On Diff #130201)

hmm, file:///foo/bar is more widely used than file:/foo/bar.
You way want to emit that instead?

clangd/URI.h

24 ↗

(On Diff #130201)

You probably also want to describe roughly the subset of URI parsing we do.

e.g.
Clangd handles URIs of the form <scheme>:[//<authority>]<body>.
It doesn't further split the authority or body into constituent parts (e.g. query strings is included in the body).

27 ↗

(On Diff #130201)

e.g. "https"

29 ↗

(On Diff #130201)

e.g. "//reviews.llvm.org"

31 ↗

(On Diff #130201)

e.g. "/D41946"

43 ↗

(On Diff #130201)

I think "the first matching scheme" is a bit confusing - I guess here "scheme" refers to URIScheme, but in practice these should be 1:1 with schemes.
I'd just drop that, and instead add a sentence that errors may occur if the scheme isn't understood or the URI isn't valid in the scheme.

46 ↗

(On Diff #130201)

this is the "more public" API, so the semantics of CurrentFile should probably be defined here, and merely referred to below.
(See below comment for questions about semantics)

63 ↗

(On Diff #130201)

Hmm - what does "users" mean :-)
I can't think of a good way to rephrase. It might be clear enough without this sentence.

65 ↗

(On Diff #130201)

nit: I'm not sure this longer comment would really help someone use this API.
I'd suggest either making the example concrete and focusing around that, or dropping it entirely - it's OK to be a little sparse here, since we don't really expect unfamiliar users.

76 ↗

(On Diff #130201)

I think "the file from which the request is issued" is overly-specified (and a little unclear).

For instance, consider this workflow:
vim $projectroot
go to symbol -> "MyClass" (from global index)
Now we're trying to navigate to "monorepo://proj/myclass.h", but we don't have a "current file" to use as a hint. $projectroot would probably do nicely, and is available to clangd.

So maybe llvm::StringRef HintPath- "A related path, such as the current file or working directory, which can help disambiguate when the same file exists in many workspaces".

80 ↗

(On Diff #130201)

You should probably pass authority here too, or we're committing all schemes to ignore authority

82 ↗

(On Diff #130201)

similarly, this should return (authority, body), not just body.
a pair seems fine for such an internal API

90 ↗

(On Diff #130201)

this seems easy enough to test via uri::create rather than exposing it publicly, but up to you.

97 ↗

(On Diff #130201)

nit: you probably don't want anything between URIScheme and this - if you keep percent*, move them below?

[ioeric]

• Address review comments

clangd/URI.cpp

43 ↗

(On Diff #130201)

Ah, right!

57 ↗

(On Diff #130201)

Don't we still need to convert \ to / on windows?

clangd/URI.h

29 ↗

(On Diff #130201)

I think authority doesn't include leading "//"?

76 ↗

(On Diff #130201)

Makes sense. Thanks for the suggestion!

90 ↗

(On Diff #130201)

I think this would also be useful for other scheme extensions.

[sammccall]

clangd/URI.cpp

94 ↗

(On Diff #130201)

Performance doesn't really matter here, but it looks like this is intended to save time by precomputing, and I don't think it's any faster than the direct version of percentEncode which might be easier to read:

for (unsigned char C : Content)
  if (shouldEscape(C))
    OS << '%' << format_hex_no_prefix(C, 2);
  else
    OS << C;

where shouldEscape just does some range checks.
(the "unsigned" is important!)

95 ↗

(On Diff #130201)

128 should be 256, right?

123 ↗

(On Diff #130201)

most implementations seem to just treat treat the % as literal in this case, e.g. "%41%" -> "A%", which makes this function unable to fail, which simplifies parse() a bit

125 ↗

(On Diff #130201)

alternatively, using more llvm and less std:

if (*I == '%' && I + 2 < Content.end()
  && isHexDigit(*(I+1)) && isHexDigit(*(I+2)) {
  Result.push_back(hexFromNibbles(*(I+1), *(I+2)));
  I += 2;
} else
  Result.push_back(*I);

136 ↗

(On Diff #130201)

nit: i'd probably prefer to keep "uri" referring to the whole URI, and call the one you mutate something else

138 ↗

(On Diff #130201)

consider StringRef::split which avoids some index arithmetic:

pair<StringRef, StringRef> SchemeSplit = Uri.split(':');
if (SchemeSplit.second == "")
  return error
U.Scheme = percentDecode(SchemeSplit.first);
// then SchemeSplit.second contains the rest

149 ↗

(On Diff #130201)

this is e.g. http://llvm.org
This is valid: scheme = 'http', authority = 'llvm.org', path = ''.
(Wikipedia is wrong, but the RFC covers this)

155 ↗

(On Diff #130201)

this is e.g. file:///foo/bar, which is definitely valid!
scheme = 'file', authority = '', path = '/foo/bar'

(we should have a test for this one!)

168 ↗

(On Diff #130201)

body may be empty. (http://llvm.org again)

unittests/clangd/URITests.cpp

115 ↗

(On Diff #130201)

please include some typical examples:

"file:///etc/passwd"
"file:///c:/windows/system32/"
"http://llvm.org"
"urn:isbn:0451450523"

117 ↗

(On Diff #130201)

the body should be "//x/y/z" here - the / that terminates the authority is part of the path.
(This is a valid URI, but not a typical example)

143 ↗

(On Diff #130201)

this is the same as the next one

144 ↗

(On Diff #130201)

these are both valid

148 ↗

(On Diff #130201)

this is valid, authority may be empty

[sammccall]

Sorry, our comments crossed...

clangd/URI.cpp

57 ↗

(On Diff #130201)

yes. convert_to_slash(path, style) assumes that path is in style.

convert_to_slash("c:\\foo.txt", windows) --> "c:/foo.txt"
convert_to_slash("c:\\foo.txt", posix) --> "c:\\foo.txt" (because that's a perfectly valid filename on a posix system! well, apart from the colon)
convert_to_slash("c:\\foo.txt", native) --> "c:/foo.txt" if the host is windows, "c:\\foo.txt" if the host is windows (this is what you want)

clangd/URI.h

29 ↗

(On Diff #130201)

You're right! my mistake.

[ioeric]

• Addressed review comments.
• Check windows vs unit paths in tests.
• Properly check absolute paths.

[ioeric]

clangd/URI.cpp

57 ↗

(On Diff #130201)

I see. Thanks for the clarification!

138 ↗

(On Diff #130201)

split is neat. But here we want to distinguish between http: vs http while it's not trivial wth split. Similar for / following the authority if we want to keep body '/', say, in http://llvm.org/.

155 ↗

(On Diff #130201)

Right! Realized file:////foo/bar was weird while preparing the previous diff ;)

[sammccall]

LG with some nits addressed

clangd/URI.cpp

46 ↗

(On Diff #130766)

should this be native(Path, posix)?
native(Path) seems to be a no-op

89 ↗

(On Diff #130766)

nit: reserved
nit: you're not unescaping it, you're just not escaping it

and I don't think "for readability" is accurate.
Reserved characters are only reserved in certain contexts, escaping a slash in a path is (I think) incorrect.

138 ↗

(On Diff #130201)

I don't understand what you mean by distinguishing http: vs http - "http://foo" will split into {"http", "foo"}. "httpfoo" will not split at all, and "http::foo" will split into "http" and "://foo".

Agreed regarding authority/body.

clangd/URI.h

31 ↗

(On Diff #130766)

nit: prefer to omit \brief unless you want the brief comment to be something other than the first sentence.

(I know we have this in lots of places - it adds noise, and I finally got around to looking it up in the style guide!)

91 ↗

(On Diff #130766)

nit: unreserved

82 ↗

(On Diff #130201)

Still confused about this API, but let's chat offline?

90 ↗

(On Diff #130201)

Hmm, I think if individual schemes need to do encoding/decoding, then our API is wrong :-)

unittests/clangd/URITests.cpp

1 ↗

(On Diff #130766)

you may want a test that round-trips getVirtualTestFilePath("x") through a file URI and back again.
This should exercise the platform-specific code path - that path is under C:\ on windows, and /tmp/ on linux. The buildbots should give us platform coverage.

[ioeric]

• Addressed review comments.

[ioeric]

clangd/URI.h

31 ↗

(On Diff #130766)

Good to know. Thanks!

unittests/clangd/URITests.cpp

1 ↗

(On Diff #130766)

Good idea! Didn't know such a nice function exists.

clang-tools-extra/trunk/clangd/CMakeLists.txt

add_clang_library(clangDaemon
   GlobalCompilationDatabase.cpp
   JSONExpr.cpp
   JSONRPCDispatcher.cpp
   Logger.cpp
   Protocol.cpp
   ProtocolHandlers.cpp
   SourceCode.cpp
   Trace.cpp
+  URI.cpp
   XRefs.cpp
   index/FileIndex.cpp
   index/Index.cpp
   index/MemIndex.cpp
   index/Merge.cpp
   index/SymbolCollector.cpp
   index/SymbolYAML.cpp
 

clang-tools-extra/trunk/clangd/URI.h

+//===--- URI.h - File URIs with schemes --------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_PATHURI_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PATHURI_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Registry.h"
+
+namespace clang {
+namespace clangd {
+
+/// A URI describes the location of a source file.
+/// In the simplest case, this is a "file" URI that directly encodes the
+/// absolute path to a file. More abstract cases are possible: a shared index
+/// service might expose repo:// URIs that are relative to the source control
+/// root.
+///
+/// Clangd handles URIs of the form <scheme>:[//<authority>]<body>. It doesn't
+/// further split the authority or body into constituent parts (e.g. query
+/// strings is included in the body).
+class FileURI {
+public:
+  /// Returns decoded scheme e.g. "https"
+  llvm::StringRef scheme() const { return Scheme; }
+  /// Returns decoded authority e.g. "reviews.lvm.org"
+  llvm::StringRef authority() const { return Authority; }
+  /// Returns decoded body e.g. "/D41946"
+  llvm::StringRef body() const { return Body; }
+
+  /// Returns a string URI with all components percent-encoded.
+  std::string toString() const;
+
+  /// Create a FileURI from unescaped scheme+authority+body.
+  static llvm::Expected<FileURI> create(llvm::StringRef Scheme,
+                                        llvm::StringRef Authority,
+                                        llvm::StringRef Body);
+
+  /// Creates a FileURI for a file in the given scheme. \p Scheme must be
+  /// registered. The URI is percent-encoded.
+  static llvm::Expected<FileURI> create(llvm::StringRef AbsolutePath,
+                                        llvm::StringRef Scheme = "file");
+
+  /// Parse a URI string "<scheme>:[//<authority>/]<path>". Percent-encoded
+  /// characters in the URI will be decoded.
+  static llvm::Expected<FileURI> parse(llvm::StringRef Uri);
+
+  /// Resolves the absolute path of \p U. If there is no matching scheme, or the
+  /// URI is invalid in the scheme, this returns an error.
+  ///
+  /// \p HintPath A related path, such as the current file or working directory,
+  /// which can help disambiguate when the same file exists in many workspaces.
+  static llvm::Expected<std::string> resolve(const FileURI &U,
+                                             llvm::StringRef HintPath = "");
+
+  friend bool operator==(const FileURI &LHS, const FileURI &RHS) {
+    return std::tie(LHS.Scheme, LHS.Authority, LHS.Body) ==
+           std::tie(RHS.Scheme, RHS.Authority, RHS.Body);
+  }
+
+private:
+  FileURI() = default;
+
+  std::string Scheme;
+  std::string Authority;
+  std::string Body;
+};
+
+/// URIScheme is an extension point for teaching clangd to recognize a custom
+/// URI scheme. This is expected to be implemented and exposed via the
+/// URISchemeRegistry.
+class URIScheme {
+public:
+  virtual ~URIScheme() = default;
+
+  /// Returns the absolute path of the file corresponding to the URI
+  /// authority+body in the file system. See FileURI::resolve for semantics of
+  /// \p HintPath.
+  virtual llvm::Expected<std::string>
+  getAbsolutePath(llvm::StringRef Authority, llvm::StringRef Body,
+                  llvm::StringRef HintPath) const = 0;
+
+  virtual llvm::Expected<FileURI>
+  uriFromAbsolutePath(llvm::StringRef AbsolutePath) const = 0;
+};
+
+/// By default, a "file" scheme is supported where URI paths are always absolute
+/// in the file system.
+typedef llvm::Registry<URIScheme> URISchemeRegistry;
+
+} // namespace clangd
+} // namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_PATHURI_H

clang-tools-extra/trunk/clangd/URI.cpp

+//===---- URI.h - File URIs with schemes -------------------------*- C++-*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "URI.h"
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Format.h"
+#include "llvm/Support/Path.h"
+#include <iomanip>
+#include <sstream>
+
+LLVM_INSTANTIATE_REGISTRY(clang::clangd::URISchemeRegistry)
+
+namespace clang {
+namespace clangd {
+namespace {
+
+inline llvm::Error make_string_error(const llvm::Twine &Message) {
+  return llvm::make_error<llvm::StringError>(Message,
+                                             llvm::inconvertibleErrorCode());
+}
+
+/// \brief This manages file paths in the file system. All paths in the scheme
+/// are absolute (with leading '/').
+class FileSystemScheme : public URIScheme {
+public:
+  static const char *Scheme;
+
+  llvm::Expected<std::string>
+  getAbsolutePath(llvm::StringRef /*Authority*/, llvm::StringRef Body,
+                  llvm::StringRef /*HintPath*/) const override {
+    if (!Body.startswith("/"))
+      return make_string_error("File scheme: expect body to be an absolute "
+                               "path starting with '/': " +
+                               Body);
+    // For Windows paths e.g. /X:
+    if (Body.size() > 2 && Body[0] == '/' && Body[2] == ':')
+      Body.consume_front("/");
+    llvm::SmallVector<char, 16> Path(Body.begin(), Body.end());
+    llvm::sys::path::native(Path);
+    return std::string(Path.begin(), Path.end());
+  }
+
+  llvm::Expected<FileURI>
+  uriFromAbsolutePath(llvm::StringRef AbsolutePath) const override {
+    using namespace llvm::sys;
+
+    std::string Body;
+    // For Windows paths e.g. X:
+    if (AbsolutePath.size() > 1 && AbsolutePath[1] == ':')
+      Body = "/";
+    Body += path::convert_to_slash(AbsolutePath);
+    return FileURI::create(Scheme, /*Authority=*/"", Body);
+  }
+};
+
+const char *FileSystemScheme::Scheme = "file";
+
+static URISchemeRegistry::Add<FileSystemScheme>
+    X(FileSystemScheme::Scheme,
+      "URI scheme for absolute paths in the file system.");
+
+llvm::Expected<std::unique_ptr<URIScheme>>
+findSchemeByName(llvm::StringRef Scheme) {
+  for (auto I = URISchemeRegistry::begin(), E = URISchemeRegistry::end();
+       I != E; ++I) {
+    if (I->getName() != Scheme)
+      continue;
+    return I->instantiate();
+  }
+  return make_string_error("Can't find scheme: " + Scheme);
+}
+
+bool shouldEscape(unsigned char C) {
+  // Unreserved characters.
+  if ((C >= 'a' && C <= 'z') || (C >= 'A' && C <= 'Z'))
+    return false;
+  switch (C) {
+  case '-':
+  case '_':
+  case '.':
+  case '~':
+  case '/': // '/' is only reserved when parsing.
+    return false;
+  }
+  return true;
+}
+
+/// Encodes a string according to percent-encoding.
+/// - Unreserved characters are not escaped.
+/// - Reserved characters always escaped with exceptions like '/'.
+/// - All other characters are escaped.
+std::string percentEncode(llvm::StringRef Content) {
+  std::string Result;
+  llvm::raw_string_ostream OS(Result);
+  for (unsigned char C : Content)
+    if (shouldEscape(C))
+      OS << '%' << llvm::format_hex_no_prefix(C, 2);
+    else
+      OS << C;
+
+  OS.flush();
+  return Result;
+}
+
+/// Decodes a string according to percent-encoding.
+std::string percentDecode(llvm::StringRef Content) {
+  std::string Result;
+  for (auto I = Content.begin(), E = Content.end(); I != E; ++I) {
+    if (*I != '%') {
+      Result += *I;
+      continue;
+    }
+    if (*I == '%' && I + 2 < Content.end() && llvm::isHexDigit(*(I + 1)) &&
+        llvm::isHexDigit(*(I + 2))) {
+      Result.push_back(llvm::hexFromNibbles(*(I + 1), *(I + 2)));
+      I += 2;
+    } else
+      Result.push_back(*I);
+  }
+  return Result;
+}
+
+} // namespace
+
+llvm::Expected<FileURI> FileURI::create(llvm::StringRef Scheme,
+                                        llvm::StringRef Authority,
+                                        llvm::StringRef Body) {
+  if (Scheme.empty())
+    return make_string_error("Scheme must be specified in a URI.");
+  if (!Authority.empty() && !Body.startswith("/"))
+    return make_string_error(
+        "URI body must start with '/' when authority is present.");
+  FileURI U;
+  U.Scheme = Scheme;
+  U.Authority = Authority;
+  U.Body = Body;
+  return U;
+}
+
+std::string FileURI::toString() const {
+  std::string Result;
+  llvm::raw_string_ostream OS(Result);
+  OS << percentEncode(Scheme) << ":";
+  if (Authority.empty() && Body.empty())
+    return OS.str();
+  // If authority if empty, we only print body if it starts with "/"; otherwise,
+  // the URI is invalid.
+  if (!Authority.empty() || llvm::StringRef(Body).startswith("/"))
+    OS << "//" << percentEncode(Authority);
+  OS << percentEncode(Body);
+  OS.flush();
+  return Result;
+}
+
+llvm::Expected<FileURI> FileURI::parse(llvm::StringRef OrigUri) {
+  FileURI U;
+  llvm::StringRef Uri = OrigUri;
+
+  auto Pos = Uri.find(':');
+  if (Pos == 0 || Pos == llvm::StringRef::npos)
+    return make_string_error("Scheme must be provided in URI: " + OrigUri);
+  U.Scheme = percentDecode(Uri.substr(0, Pos));
+  Uri = Uri.substr(Pos + 1);
+  if (Uri.consume_front("//")) {
+    Pos = Uri.find('/');
+    U.Authority = percentDecode(Uri.substr(0, Pos));
+    Uri = Uri.substr(Pos);
+  }
+  U.Body = percentDecode(Uri);
+  return U;
+}
+
+llvm::Expected<FileURI> FileURI::create(llvm::StringRef AbsolutePath,
+                                        llvm::StringRef Scheme) {
+  if (!llvm::sys::path::is_absolute(AbsolutePath))
+    return make_string_error("Not a valid absolute path: " + AbsolutePath);
+  auto S = findSchemeByName(Scheme);
+  if (!S)
+    return S.takeError();
+  return S->get()->uriFromAbsolutePath(AbsolutePath);
+}
+
+llvm::Expected<std::string> FileURI::resolve(const FileURI &Uri,
+                                             llvm::StringRef HintPath) {
+  auto S = findSchemeByName(Uri.Scheme);
+  if (!S)
+    return S.takeError();
+  return S->get()->getAbsolutePath(Uri.Authority, Uri.Body, HintPath);
+}
+
+} // namespace clangd
+} // namespace clang

clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt

add_extra_unittest(ClangdTests
   ClangdTests.cpp
   CodeCompleteTests.cpp
   CodeCompletionStringsTests.cpp
   ContextTests.cpp
   FileIndexTests.cpp
   FuzzyMatchTests.cpp
   IndexTests.cpp
   JSONExprTests.cpp
+  URITests.cpp
   TestFS.cpp
   TraceTests.cpp
   SourceCodeTests.cpp
   SymbolCollectorTests.cpp
   XRefsTests.cpp
   )
 
 target_link_libraries(ClangdTests

clang-tools-extra/trunk/unittests/clangd/URITests.cpp

+//===-- URITests.cpp  ---------------------------------*- C++ -*-----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "TestFS.h"
+#include "URI.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+namespace clang {
+namespace clangd {
+namespace {
+
+using ::testing::AllOf;
+
+MATCHER_P(Scheme, S, "") { return arg.scheme() == S; }
+MATCHER_P(Authority, A, "") { return arg.authority() == A; }
+MATCHER_P(Body, B, "") { return arg.body() == B; }
+
+// Assume all files in the schema have a "test-root/" root directory, and the
+// schema path is the relative path to the root directory.
+// So the schema of "/some-dir/test-root/x/y/z" is "test:x/y/z".
+class TestScheme : public URIScheme {
+public:
+  static const char *Scheme;
+
+  static const char *TestRoot;
+
+  llvm::Expected<std::string>
+  getAbsolutePath(llvm::StringRef /*Authority*/, llvm::StringRef Body,
+                  llvm::StringRef HintPath) const override {
+    auto Pos = HintPath.find(TestRoot);
+    assert(Pos != llvm::StringRef::npos);
+    return (HintPath.substr(0, Pos + llvm::StringRef(TestRoot).size()) + Body)
+        .str();
+  }
+
+  llvm::Expected<FileURI>
+  uriFromAbsolutePath(llvm::StringRef AbsolutePath) const override {
+    auto Pos = AbsolutePath.find(TestRoot);
+    assert(Pos != llvm::StringRef::npos);
+    return FileURI::create(
+        Scheme, /*Authority=*/"",
+        AbsolutePath.substr(Pos + llvm::StringRef(TestRoot).size()));
+  }
+};
+
+const char *TestScheme::Scheme = "test";
+const char *TestScheme::TestRoot = "/test-root/";
+
+static URISchemeRegistry::Add<TestScheme> X(TestScheme::Scheme, "Test schema");
+
+std::string createOrDie(llvm::StringRef AbsolutePath,
+                        llvm::StringRef Scheme = "file") {
+  auto Uri = FileURI::create(AbsolutePath, Scheme);
+  if (!Uri)
+    llvm_unreachable(llvm::toString(Uri.takeError()).c_str());
+  return Uri->toString();
+}
+
+std::string createOrDie(llvm::StringRef Scheme, llvm::StringRef Authority,
+                        llvm::StringRef Body) {
+  auto Uri = FileURI::create(Scheme, Authority, Body);
+  if (!Uri)
+    llvm_unreachable(llvm::toString(Uri.takeError()).c_str());
+  return Uri->toString();
+}
+
+FileURI parseOrDie(llvm::StringRef Uri) {
+  auto U = FileURI::parse(Uri);
+  if (!U)
+    llvm_unreachable(llvm::toString(U.takeError()).c_str());
+  return *U;
+}
+
+TEST(PercentEncodingTest, Encode) {
+  EXPECT_EQ(createOrDie("x", /*Authority=*/"", "a/b/c"), "x:a/b/c");
+  EXPECT_EQ(createOrDie("x", /*Authority=*/"", "a!b;c~"), "x:a%21b%3bc~");
+}
+
+TEST(PercentEncodingTest, Decode) {
+  EXPECT_EQ(parseOrDie("x:a/b/c").body(), "a/b/c");
+
+  EXPECT_EQ(parseOrDie("%3a://%3a/%3").scheme(), ":");
+  EXPECT_EQ(parseOrDie("%3a://%3a/%3").authority(), ":");
+  EXPECT_EQ(parseOrDie("%3a://%3a/%3").body(), "/%3");
+
+  EXPECT_EQ(parseOrDie("x:a%21b%3ac~").body(), "a!b:c~");
+}
+
+std::string resolveOrDie(const FileURI &U, llvm::StringRef HintPath = "") {
+  auto Path = FileURI::resolve(U, HintPath);
+  if (!Path)
+    llvm_unreachable(llvm::toString(Path.takeError()).c_str());
+  return *Path;
+}
+
+TEST(URITest, Create) {
+#ifdef LLVM_ON_WIN32
+  EXPECT_THAT(createOrDie("c:\\x\\y\\z"), "file:///c:/x/y/z");
+#else
+  EXPECT_THAT(createOrDie("/x/y/z"), "file:///x/y/z");
+  EXPECT_THAT(createOrDie("/(x)/y/\\ z"), "file:///%28x%29/y/%5c%20z");
+#endif
+}
+
+TEST(URITest, FailedCreate) {
+  auto Fail = [](llvm::Expected<FileURI> U) {
+    if (!U) {
+      llvm::consumeError(U.takeError());
+      return true;
+    }
+    return false;
+  };
+  // Create from scheme+authority+body:
+  //
+  // Scheme must be provided.
+  EXPECT_TRUE(Fail(FileURI::create("", "auth", "/a")));
+  // Body must start with '/' if authority is present.
+  EXPECT_TRUE(Fail(FileURI::create("scheme", "auth", "x/y/z")));
+
+  // Create from scheme registry:
+  //
+  EXPECT_TRUE(Fail(FileURI::create("/x/y/z", "no")));
+  // Path has to be absolute.
+  EXPECT_TRUE(Fail(FileURI::create("x/y/z")));
+}
+
+TEST(URITest, Parse) {
+  EXPECT_THAT(parseOrDie("file://auth/x/y/z"),
+              AllOf(Scheme("file"), Authority("auth"), Body("/x/y/z")));
+
+  EXPECT_THAT(parseOrDie("file://au%3dth/%28x%29/y/%5c%20z"),
+              AllOf(Scheme("file"), Authority("au=th"), Body("/(x)/y/\\ z")));
+
+  EXPECT_THAT(parseOrDie("file:///%28x%29/y/%5c%20z"),
+              AllOf(Scheme("file"), Authority(""), Body("/(x)/y/\\ z")));
+  EXPECT_THAT(parseOrDie("file:///x/y/z"),
+              AllOf(Scheme("file"), Authority(""), Body("/x/y/z")));
+  EXPECT_THAT(parseOrDie("file:"),
+              AllOf(Scheme("file"), Authority(""), Body("")));
+  EXPECT_THAT(parseOrDie("file:///x/y/z%2"),
+              AllOf(Scheme("file"), Authority(""), Body("/x/y/z%2")));
+  EXPECT_THAT(parseOrDie("http://llvm.org"),
+              AllOf(Scheme("http"), Authority("llvm.org"), Body("")));
+  EXPECT_THAT(parseOrDie("http://llvm.org/"),
+              AllOf(Scheme("http"), Authority("llvm.org"), Body("/")));
+  EXPECT_THAT(parseOrDie("http://llvm.org/D"),
+              AllOf(Scheme("http"), Authority("llvm.org"), Body("/D")));
+  EXPECT_THAT(parseOrDie("http:/"),
+              AllOf(Scheme("http"), Authority(""), Body("/")));
+  EXPECT_THAT(parseOrDie("urn:isbn:0451450523"),
+              AllOf(Scheme("urn"), Authority(""), Body("isbn:0451450523")));
+  EXPECT_THAT(
+      parseOrDie("file:///c:/windows/system32/"),
+      AllOf(Scheme("file"), Authority(""), Body("/c:/windows/system32/")));
+}
+
+TEST(URITest, ParseFailed) {
+  auto FailedParse = [](llvm::StringRef U) {
+    auto URI = FileURI::parse(U);
+    if (!URI) {
+      llvm::consumeError(URI.takeError());
+      return true;
+    }
+    return false;
+  };
+
+  // Expect ':' in URI.
+  EXPECT_TRUE(FailedParse("file//x/y/z"));
+  // Empty.
+  EXPECT_TRUE(FailedParse(""));
+  EXPECT_TRUE(FailedParse(":/a/b/c"));
+}
+
+TEST(URITest, Resolve) {
+#ifdef LLVM_ON_WIN32
+  EXPECT_THAT(resolveOrDie(parseOrDie("file:///c:/x/y/z")), "c:\\x\\y\\z");
+#else
+  EXPECT_EQ(resolveOrDie(parseOrDie("file:/a/b/c")), "/a/b/c");
+  EXPECT_EQ(resolveOrDie(parseOrDie("file://auth/a/b/c")), "/a/b/c");
+  EXPECT_EQ(resolveOrDie(parseOrDie("test:a/b/c"), "/dir/test-root/x/y/z"),
+            "/dir/test-root/a/b/c");
+  EXPECT_THAT(resolveOrDie(parseOrDie("file://au%3dth/%28x%29/y/%20z")),
+              "/(x)/y/ z");
+  EXPECT_THAT(resolveOrDie(parseOrDie("file:///c:/x/y/z")), "c:/x/y/z");
+#endif
+}
+
+TEST(URITest, Platform) {
+  auto Path = getVirtualTestFilePath("x");
+  auto U = FileURI::create(Path, "file");
+  EXPECT_TRUE(static_cast<bool>(U));
+  EXPECT_THAT(resolveOrDie(*U), Path.str());
+}
+
+TEST(URITest, ResolveFailed) {
+  auto FailedResolve = [](llvm::StringRef Uri) {
+    auto Path = FileURI::resolve(parseOrDie(Uri));
+    if (!Path) {
+      llvm::consumeError(Path.takeError());
+      return true;
+    }
+    return false;
+  };
+
+  // Invalid scheme.
+  EXPECT_TRUE(FailedResolve("no:/a/b/c"));
+  // File path needs to be absolute.
+  EXPECT_TRUE(FailedResolve("file:a/b/c"));
+}
+
+} // namespace
+} // namespace clangd
+} // namespace clang