PR37189 Fix incorrect end source location and spelling for a split '>>' token.
When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
llvm-svn: 331155
diff --git a/clang/lib/Lex/Preprocessor.cpp b/clang/lib/Lex/Preprocessor.cpp
index 4370bc1..94490bd 100644
--- a/clang/lib/Lex/Preprocessor.cpp
+++ b/clang/lib/Lex/Preprocessor.cpp
@@ -487,6 +487,22 @@
Tok.setLiteralData(DestPtr);
}
+SourceLocation Preprocessor::SplitToken(SourceLocation Loc, unsigned Length) {
+ auto &SM = getSourceManager();
+ SourceLocation SpellingLoc = SM.getSpellingLoc(Loc);
+ std::pair<FileID, unsigned> LocInfo = SM.getDecomposedLoc(SpellingLoc);
+ bool Invalid = false;
+ StringRef Buffer = SM.getBufferData(LocInfo.first, &Invalid);
+ if (Invalid)
+ return SourceLocation();
+
+ // FIXME: We could consider re-using spelling for tokens we see repeatedly.
+ const char *DestPtr;
+ SourceLocation Spelling =
+ ScratchBuf->getToken(Buffer.data() + LocInfo.second, Length, DestPtr);
+ return SM.createTokenSplitLoc(Spelling, Loc, Loc.getLocWithOffset(Length));
+}
+
Module *Preprocessor::getCurrentModule() {
if (!getLangOpts().isCompilingModule())
return nullptr;