In D86990#2252592, @jkorous wrote:

@akyrtzi IIUC you are saying we should try to push the char-range semantics from libclang API as deep as possible to see if we could avoid this annoying conversion. I think that's a reasonable long-term strategy.
But I also think we'll inevitably run into some clang APIs that use SourceRange for valid reasons and we won't be able to change those. Some other clang APIs might require significant amount of work to be enhanced.

Ultimately, given this conversion isn't part of public libclang API and has explicit semantics, I would keep it - I'd just add a warning to its doc comment.

On practical level - I need this conversion for my other work and I can't go just now and refactor the Rewriter interface.
https://reviews.llvm.org/D86992

SourceRange cxloc::translateCharRangeToTokenRange(CXTranslationUnit TU, CXSourceRange R) is an expensive operation and I'm concerned it will be easy to start calling at places and introduce performance degradation.

We may have places in the code where SourceRange is used as a pair of locations, and those locations are character locations instead of token ones, so essentially the information of whether the range is token-based or character-based gets lost, and we get into trouble when passing such a SourceRange to APIs that assume token-based.

In D86840#2248463, @jkorous wrote:

Thanks a lot for this clarification! So, does that mean the issue is actually here?

static inline SourceRange translateCXSourceRange(CXSourceRange R) {
  return SourceRange(SourceLocation::getFromRawEncoding(R.begin_int_data),
                     SourceLocation::getFromRawEncoding(R.end_int_data));
}

AFAIK SourceRange is supposed to always represent a token range (begin loc points to beginning of first token, end loc points to beginning of last token). For representing a character range, CharSourceRange should be used, though IMO its IsTokenRange member was a mistake, CharSourceRange should have only being used to represent half-open character-based range. This distinction is thankfully more clear on the Swift side.

I agree it makes sense to disable CLANG_ENABLE_ARCMT if CLANG_ENABLE_STATIC_ANALYZER is also disabled.

Are you sure the difference is not random noise? It's not clear to me under which conditions you would encounter the same region in NewParsedRegions within the same TU. I assume this would be the case if the same file is parsed multiple times, but I'd expect this to be uncommon within the same TU.
I'd recommend to add some counters of when NewParsedRegions.count(region) is true and when ParsedRegions.count(region) is true, and print them at the end.

In D65846#1619752, @bruno wrote:

clang -fmodules -fmodules-cache-path=... is supposed to create the directory for the cache path, including the parent directories, AFAIK. If this doesn't happen it is a behavior change (and undesirable IMO).

Is c-index-test invoking clang or do we just have a similar interface? Perhaps it's not doing it right (haven't seen this problem happening directly while invoking clang). Should -fallow-pch-with-compiler-errors be considered somehow here?

In D65846#1619645, @jfb wrote:

My current guess is that this part of the test:

c-index-test -write-pch %t.h.pch %s -fmodules -fmodules-cache-path=%t.mcp -Xclang -triple -Xclang x86_64-apple-darwin

Is expected to generate the unknown type name error, but when than happens it ignores -fmodules-cache-path=%t.mcp and doesn't create the directory. The next line expects the directory to exist, which is why it can't create the lock file (because the directory it's trying to create it in doesn't exist).

Sorry, it's not clear to me what is the issue.

In D64384#1596521, @jkorous wrote:

For now I went with IndexWhileBuilding name but I'm open to discussion.

Thanks for reviewing Kristina & Aaron, much appreciated!

• Change #cmakedefine01 to #cmakedefine for consistency.
• Add comments to getLastAccessedTime()/ getLastModificationTime() APIs to make sure people are aware that time resolution can differ among file systems.
• Improve commit summary.

I think that should be documented on the public API part with a mention that the resolution is expected to differ depending on the file system and OS. WDYT?

That is a good idea, the way I consider file_status is that it provides a cross-platform way to get at the file properties. It provides times in nanoseconds but leaves the precision as implementation detail of the underlying file system, which makes sense, people should not expect anything more than that.
I'll add some comments.

In D54826#1307402, @aaron.ballman wrote:

Is it going to be an issue that the Windows side of things has a more wild idea of file timestamp resolution? NTFS has a theoretical max precision of 100ns intervals, though according to MSDN, the access time on NTFS has a resolution of 1 hour, which is better than FAT file systems, where the resolution is 1 day. It seems odd to rely on ns resolution for access time, as that seems like it's going to be highly platform and filesystem dependent.

I can't speak for the Windows side of things but what you are pointing out is a general question about applications taking into account that file properties may differ across platforms. This is orthogonal to file_status itself reporting file properties accurately.
If current file_status has 1 hour resolution on NTFS for access time, this patch is not going to make it any better or worse, nor affect behavior of applications that are querying files from NTFS.

I think this is the most important bit of information here. It should have been in the description of the differential.
If the current precision was 1ms (or even 1us), then yes, going all the way down to 1ns may not be important/worthwhile.
But 1 sec precision is clearly laughably not great, and should be improved.

Ah good point, I didn't consider that people are not broadly aware that current file_status has only 'up to second' precision.

On second look, if you are concerned with adding additional fields in basic_file_status, I can replace:

time_t fs_st_atime = 0;
time_t fs_st_mtime = 0;

with uint64_t's which are adequate to include nanosecond precision, while these time_t's are 64bits as well but waste space in the structure.

if there's no consumer for such API (pending review) there is no point in adding it.

To make sure it's clear, this is not a new API. getLastModificationTime() already reports a a timepoint in nanoseconds, but it is always in second intervals. This patch allows it to include the nanoseconds that are part of the underlying file status.

The general changes to the support libraries are usually facilitated because of missing functionality that is required somewhere else and it being the most logical place to add it. If so please attach related revisions to this one (through "Edit Related Revisions") so it's more clear why this was added

There have been enhancements to file_status that added more info without the requirement you stated (attach related revisions to this one), like:
https://reviews.llvm.org/D31110
https://reviews.llvm.org/D18456

Moved the toTimePoint conversion function to llvm/Support/Chrono.h so it can be broadly available.

LGTM!

Good enough, thanks for looking into this!

CXIndexDataConsumer.cpp uses ASTNode.OrigD, that code is exercised when you run c-index-test -index-file <...>. I'd recommend to at least check if that command would produce a different output for the test case that you detected originally.

Is it possible to add a regression test case ? I assume this is fixing some issue, we should make sure we don't regress again.

We could leave disableSourceFileDiagnostics off until someone finds a use case for it.