Unified token breaking logic: support for line comments.

Summary:
Added BreakableLineComment, moved common code from
BreakableBlockComment to newly added BreakableComment. As a side-effect of the
rewrite, found another problem with escaped newlines and had to change
code which removes trailing whitespace from line comments not to break after
this patch.

Reviewers: klimek, djasper

Reviewed By: klimek

CC: cfe-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D682

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@179693 91177308-0d34-0410-b5e6-96231b3b80d8
7 files changed