[mlir] Allow C-style escapes in Lexer
This patch passes the raw, unescaped value through to the rest of the stack. Partial escaping is a total pain to deal with, so we either need to implement escaping properly (ideally using a third party library like absl, I don't think LLVM has one that can handle the proper gamut of escape codes) or don't escape. I chose the latter for this patch.
PiperOrigin-RevId: 208608945
diff --git a/lib/Parser/Lexer.cpp b/lib/Parser/Lexer.cpp
index 91fa8ad..885608a 100644
--- a/lib/Parser/Lexer.cpp
+++ b/lib/Parser/Lexer.cpp
@@ -323,6 +323,12 @@
case '\v':
case '\f':
return emitError(curPtr-1, "expected '\"' in string literal");
+ case '\\':
+ // Handle explicitly \" -> ".
+ // TODO(someone): define more escaping rules.
+ if (*curPtr == '"')
+ ++curPtr;
+ continue;
default:
continue;
diff --git a/lib/Parser/Token.cpp b/lib/Parser/Token.cpp
index 9fc2b2f..2bce681 100644
--- a/lib/Parser/Token.cpp
+++ b/lib/Parser/Token.cpp
@@ -81,9 +81,8 @@
/// Given a 'string' token, return its value, including removing the quote
/// characters and unescaping the contents of the string.
std::string Token::getStringValue() const {
- // TODO: Handle escaping.
-
- // Just drop the quotes off for now.
+ // Start by dropping the quotes.
+ // TODO: Un-escape the string here instead of passing through the raw content.
return getSpelling().drop_front().drop_back().str();
}