Implement serialization and lazy deserialization of the preprocessing
record (which includes all macro instantiations and definitions). As
with all lay deserialization, this introduces a new external source
(here, an external preprocessing record source) that loads all of the
preprocessed entities prior to iterating over the entities.

The preprocessing record is an optional part of the precompiled header
that is disabled by default (enabled with
-detailed-preprocessing-record). When the preprocessor given to the
PCH writer has a preprocessing record, that record is written into the
PCH file. When the PCH reader is given a PCH file that contains a
preprocessing record, it will be lazily loaded (which, effectively,
implicitly adds -detailed-preprocessing-record). This is the first
case where we have sections of the precompiled header that are
added/removed based on a compilation flag, which is
unfortunate. However, this data consumes ~550k in the PCH file for
Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering
this detailed preprocessing information, so it's too expensive to turn
on by default. In the future, we should investigate a better encoding
of this information.



git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@99002 91177308-0d34-0410-b5e6-96231b3b80d8
diff --git a/tools/CIndex/CIndex.cpp b/tools/CIndex/CIndex.cpp
index f20595e..7d63d26 100644
--- a/tools/CIndex/CIndex.cpp
+++ b/tools/CIndex/CIndex.cpp
@@ -158,31 +158,12 @@
 CXSourceRange cxloc::translateSourceRange(const SourceManager &SM,
                                           const LangOptions &LangOpts,
                                           SourceRange R) {
-  // FIXME: This is largely copy-paste from
-  // TextDiagnosticPrinter::HighlightRange.  When it is clear that this is what
-  // we want the two routines should be refactored.
-
   // We want the last character in this location, so we will adjust the
-  // instantiation location accordingly.
-
-  // If the location is from a macro instantiation, get the end of the
-  // instantiation range.
+  // location accordingly.
+  // FIXME: How do do this with a macro instantiation location?
   SourceLocation EndLoc = R.getEnd();
-  SourceLocation InstLoc = SM.getInstantiationLoc(EndLoc);
-  if (EndLoc.isMacroID())
-    InstLoc = SM.getInstantiationRange(EndLoc).second;
-
-  // Measure the length token we're pointing at, so we can adjust the physical
-  // location in the file to point at the last character.
-  //
-  // FIXME: This won't cope with trigraphs or escaped newlines well. For that,
-  // we actually need a preprocessor, which isn't currently available
-  // here. Eventually, we'll switch the pointer data of
-  // CXSourceLocation/CXSourceRange to a translation unit (CXXUnit), so that the
-  // preprocessor will be available here. At that point, we can use
-  // Preprocessor::getLocForEndOfToken().
-  if (InstLoc.isValid()) {
-    unsigned Length = Lexer::MeasureTokenLength(InstLoc, SM, LangOpts);
+  if (!EndLoc.isInvalid() && EndLoc.isFileID()) {
+    unsigned Length = Lexer::MeasureTokenLength(EndLoc, SM, LangOpts);
     EndLoc = EndLoc.getFileLocWithOffset(Length);
   }
 
@@ -434,7 +415,11 @@
                        = CXXUnit->getPreprocessor().getPreprocessingRecord()) {
       // FIXME: Once we have the ability to deserialize a preprocessing record,
       // do so.
-      for (PreprocessingRecord::iterator E = PPRec->begin(),EEnd = PPRec->end();
+      bool OnlyLocalDecls
+        = !CXXUnit->isMainFileAST() && CXXUnit->getOnlyLocalDecls();
+      for (PreprocessingRecord::iterator 
+             E = PPRec->begin(OnlyLocalDecls), 
+             EEnd = PPRec->end(OnlyLocalDecls);
            E != EEnd; ++E) {
         if (MacroInstantiation *MI = dyn_cast<MacroInstantiation>(*E)) {
           if (Visit(MakeMacroInstantiationCursor(MI, CXXUnit)))