Makes whitespace management more consistent.

Instead of selectively storing some changes and directly generating
replacements for others, we now notify the WhitespaceManager of the
whitespace before every token (and optionally with more changes inside
tokens).

Then, we run over all whitespace in the very end in original source
order, where we have all information available to correctly align
comments and escaped newlines.

The future direction is to pull more of the comment alignment
implementation that is now in the BreakableToken into the
WhitespaceManager.

This fixes a bug when aligning comments or escaped newlines in unwrapped
lines that are handled out of order:
  #define A \
    f({     \
      g();  \
    });
... now gets correctly layouted.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182467 91177308-0d34-0410-b5e6-96231b3b80d8
diff --git a/lib/Format/BreakableToken.cpp b/lib/Format/BreakableToken.cpp
index 3e2e0ce..320913c 100644
--- a/lib/Format/BreakableToken.cpp
+++ b/lib/Format/BreakableToken.cpp
@@ -56,13 +56,10 @@
     AdditionalPrefix = "";
   }
 
-  unsigned WhitespaceStartColumn =
-      getContentStartColumn(LineIndex, TailOffset) + Split.first;
   unsigned BreakOffset = Text.data() - TokenText.data() + Split.first;
   unsigned CharsToRemove = Split.second;
   Whitespaces.breakToken(Tok, BreakOffset, CharsToRemove, "", AdditionalPrefix,
-                         InPPDirective, IndentAtLineBreak,
-                         WhitespaceStartColumn);
+                         InPPDirective, IndentAtLineBreak);
 }
 
 BreakableBlockComment::BreakableBlockComment(const SourceManager &SourceMgr,
@@ -144,16 +141,25 @@
   if (LineIndex == Lines.size() - 1)
     return;
   StringRef Text = Lines[LineIndex].substr(TailOffset);
+
+  // FIXME: The algorithm for trimming a line should naturally yield a
+  // non-change if there is nothing to trim; removing this line breaks the
+  // algorithm; investigate the root cause, and make sure to either document
+  // why exactly this is needed for remove it.
   if (!Text.endswith(" ") && !InPPDirective)
     return;
 
   StringRef TrimmedLine = Text.rtrim();
-  unsigned WhitespaceStartColumn =
-      getLineLengthAfterSplit(LineIndex, TailOffset);
   unsigned BreakOffset = TrimmedLine.end() - TokenText.data();
   unsigned CharsToRemove = Text.size() - TrimmedLine.size() + 1;
+  // FIXME: It seems like we're misusing the call to breakToken to remove
+  // whitespace instead of breaking a token. We should make this an explicit
+  // call option to the WhitespaceManager, or handle trimming and alignment
+  // of comments completely within in the WhitespaceManger. Passing '0' here
+  // and relying on this not breaking assumptions of the WhitespaceManager seems
+  // like a bad idea.
   Whitespaces.breakToken(Tok, BreakOffset, CharsToRemove, "", "", InPPDirective,
-                         0, WhitespaceStartColumn);
+                         0);
 }
 
 BreakableLineComment::BreakableLineComment(const SourceManager &SourceMgr,