blob: 72c82a3edebad2669c8addc1de60ae2886d4c26d [file] [log] [blame]
David Tolnay7aa1c9c2019-01-19 19:53:10 -08001//! A wrapper around the procedural macro API of the compiler's [`proc_macro`]
2//! crate. This library serves three purposes:
Alex Crichtonbabc99e2017-07-05 18:00:29 -07003//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -08004//! [`proc_macro`]: https://doc.rust-lang.org/proc_macro/
Alex Crichtonbabc99e2017-07-05 18:00:29 -07005//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -08006//! - **Bring proc-macro-like functionality to other contexts like build.rs and
7//! main.rs.** Types from `proc_macro` are entirely specific to procedural
8//! macros and cannot ever exist in code outside of a procedural macro.
9//! Meanwhile `proc_macro2` types may exist anywhere including non-macro code.
10//! By developing foundational libraries like [syn] and [quote] against
11//! `proc_macro2` rather than `proc_macro`, the procedural macro ecosystem
12//! becomes easily applicable to many other use cases and we avoid
13//! reimplementing non-macro equivalents of those libraries.
Alex Crichtonbabc99e2017-07-05 18:00:29 -070014//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080015//! - **Make procedural macros unit testable.** As a consequence of being
16//! specific to procedural macros, nothing that uses `proc_macro` can be
17//! executed from a unit test. In order for helper libraries or components of
18//! a macro to be testable in isolation, they must be implemented using
19//! `proc_macro2`.
Alex Crichton1fd0e8a2018-02-04 21:29:13 -080020//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080021//! - **Provide the latest and greatest APIs across all compiler versions.**
22//! Procedural macros were first introduced to Rust in 1.15.0 with an
23//! extremely minimal interface. Since then, many improvements have landed to
24//! make macros more flexible and easier to write. This library tracks the
25//! procedural macro API of the most recent stable compiler but employs a
26//! polyfill to provide that API consistently across any compiler since
27//! 1.15.0.
David Tolnay6b46deb2018-04-25 21:22:46 -070028//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080029//! [syn]: https://github.com/dtolnay/syn
30//! [quote]: https://github.com/dtolnay/quote
David Tolnay6b46deb2018-04-25 21:22:46 -070031//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080032//! # Usage
33//!
34//! The skeleton of a typical procedural macro typically looks like this:
35//!
36//! ```edition2018
37//! extern crate proc_macro;
38//!
39//! # const IGNORE: &str = stringify! {
40//! #[proc_macro_derive(MyDerive)]
41//! # };
42//! pub fn my_derive(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
43//! let input = proc_macro2::TokenStream::from(input);
44//!
45//! let output: proc_macro2::TokenStream = {
46//! /* transform input */
47//! # input
48//! };
49//!
50//! proc_macro::TokenStream::from(output)
51//! }
52//! ```
53//!
54//! If parsing with [Syn], you'll use [`parse_macro_input!`] instead to
55//! propagate parse errors correctly back to the compiler when parsing fails.
56//!
57//! [`parse_macro_input!`]: https://docs.rs/syn/0.15/syn/macro.parse_macro_input.html
58//!
59//! # Unstable features
60//!
61//! The default feature set of proc-macro2 tracks the most recent stable
62//! compiler API. Functionality in `proc_macro` that is not yet stable is not
63//! exposed by proc-macro2 by default.
64//!
65//! To opt into the additional APIs available in the most recent nightly
66//! compiler, the `procmacro2_semver_exempt` config flag must be passed to
67//! rustc. As usual, we will polyfill those nightly-only APIs all the way back
68//! to Rust 1.15.0. As these are unstable APIs that track the nightly compiler,
69//! minor versions of proc-macro2 may make breaking changes to them at any time.
David Tolnay6b46deb2018-04-25 21:22:46 -070070//!
71//! ```sh
72//! RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo build
73//! ```
74//!
75//! Note that this must not only be done for your crate, but for any crate that
76//! depends on your crate. This infectious nature is intentional, as it serves
77//! as a reminder that you are outside of the normal semver guarantees.
78//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080079//! Semver exempt methods are marked as such in the proc-macro2 documentation.
Michael Bryan43339962019-06-19 09:04:22 +080080//!
81//! # Thread-Safety
82//!
83//! Most types in this crate are `!Sync` because the underlying compiler
84//! types make use of thread-local memory, meaning they cannot be accessed from
85//! a different thread.
Alex Crichtonbabc99e2017-07-05 18:00:29 -070086
David Tolnay15cc4982018-01-08 08:03:27 -080087// Proc-macro2 types in rustdoc of other crates get linked to here.
David Tolnayf5149212019-05-08 13:39:08 -070088#![doc(html_root_url = "https://docs.rs/proc-macro2/0.4.30")]
David Tolnayb455dd72019-04-28 13:50:51 -070089#![cfg_attr(any(proc_macro_span, super_unstable), feature(proc_macro_span))]
David Tolnay3b1f7d22019-01-28 12:22:11 -080090#![cfg_attr(super_unstable, feature(proc_macro_raw_ident, proc_macro_def_site))]
Alex Crichtoncbec8ec2017-06-02 13:19:33 -070091
Alex Crichton53548482018-08-11 21:54:05 -070092#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -070093extern crate proc_macro;
David Tolnayb1032662017-05-31 15:52:28 -070094extern crate unicode_xid;
Alex Crichton44bffbc2017-05-19 17:51:59 -070095
Alex Crichtonf3888432018-05-16 09:11:05 -070096use std::cmp::Ordering;
Alex Crichton44bffbc2017-05-19 17:51:59 -070097use std::fmt;
Alex Crichtonf3888432018-05-16 09:11:05 -070098use std::hash::{Hash, Hasher};
Alex Crichton44bffbc2017-05-19 17:51:59 -070099use std::iter::FromIterator;
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700100use std::marker;
David Tolnay9cd3b4c2018-11-11 16:47:32 -0800101#[cfg(procmacro2_semver_exempt)]
102use std::path::PathBuf;
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700103use std::rc::Rc;
104use std::str::FromStr;
Alex Crichton44bffbc2017-05-19 17:51:59 -0700105
David Tolnayb1032662017-05-31 15:52:28 -0700106#[macro_use]
David Tolnayb1032662017-05-31 15:52:28 -0700107mod strnom;
David Tolnayaef075b2019-01-16 16:29:18 -0800108mod fallback;
David Tolnayb1032662017-05-31 15:52:28 -0700109
Alex Crichtonce0904d2018-08-27 17:29:49 -0700110#[cfg(not(wrap_proc_macro))]
David Tolnayaef075b2019-01-16 16:29:18 -0800111use fallback as imp;
112#[path = "wrapper.rs"]
Alex Crichtonce0904d2018-08-27 17:29:49 -0700113#[cfg(wrap_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700114mod imp;
115
David Tolnay82ba02d2018-05-20 16:22:43 -0700116/// An abstract stream of tokens, or more concretely a sequence of token trees.
117///
118/// This type provides interfaces for iterating over token trees and for
119/// collecting token trees into one stream.
120///
121/// Token stream is both the input and output of `#[proc_macro]`,
122/// `#[proc_macro_attribute]` and `#[proc_macro_derive]` definitions.
David Tolnaycb1b85f2017-06-03 16:40:35 -0700123#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700124pub struct TokenStream {
125 inner: imp::TokenStream,
126 _marker: marker::PhantomData<Rc<()>>,
127}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700128
David Tolnay82ba02d2018-05-20 16:22:43 -0700129/// Error returned from `TokenStream::from_str`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700130pub struct LexError {
131 inner: imp::LexError,
132 _marker: marker::PhantomData<Rc<()>>,
133}
134
135impl TokenStream {
136 fn _new(inner: imp::TokenStream) -> TokenStream {
137 TokenStream {
David Tolnayf0814122019-07-19 11:53:55 -0700138 inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700139 _marker: marker::PhantomData,
140 }
141 }
142
David Tolnayaef075b2019-01-16 16:29:18 -0800143 fn _new_stable(inner: fallback::TokenStream) -> TokenStream {
Alex Crichton30a4e9e2018-04-27 17:02:19 -0700144 TokenStream {
145 inner: inner.into(),
146 _marker: marker::PhantomData,
147 }
148 }
149
David Tolnay82ba02d2018-05-20 16:22:43 -0700150 /// Returns an empty `TokenStream` containing no token trees.
David Tolnayc3bb4592018-05-28 20:09:44 -0700151 pub fn new() -> TokenStream {
152 TokenStream::_new(imp::TokenStream::new())
153 }
154
155 #[deprecated(since = "0.4.4", note = "please use TokenStream::new")]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700156 pub fn empty() -> TokenStream {
David Tolnayc3bb4592018-05-28 20:09:44 -0700157 TokenStream::new()
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700158 }
159
David Tolnay82ba02d2018-05-20 16:22:43 -0700160 /// Checks if this `TokenStream` is empty.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700161 pub fn is_empty(&self) -> bool {
162 self.inner.is_empty()
163 }
164}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700165
Árpád Goretity4f74b682018-07-14 00:47:51 +0200166/// `TokenStream::default()` returns an empty stream,
167/// i.e. this is equivalent with `TokenStream::new()`.
168impl Default for TokenStream {
169 fn default() -> Self {
170 TokenStream::new()
171 }
172}
173
David Tolnay82ba02d2018-05-20 16:22:43 -0700174/// Attempts to break the string into tokens and parse those tokens into a token
175/// stream.
176///
177/// May fail for a number of reasons, for example, if the string contains
178/// unbalanced delimiters or characters not existing in the language.
179///
180/// NOTE: Some errors may cause panics instead of returning `LexError`. We
181/// reserve the right to change these errors into `LexError`s later.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700182impl FromStr for TokenStream {
183 type Err = LexError;
184
185 fn from_str(src: &str) -> Result<TokenStream, LexError> {
David Tolnayb28f38a2018-03-31 22:02:29 +0200186 let e = src.parse().map_err(|e| LexError {
187 inner: e,
188 _marker: marker::PhantomData,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700189 })?;
190 Ok(TokenStream::_new(e))
Alex Crichton44bffbc2017-05-19 17:51:59 -0700191 }
192}
193
Alex Crichton53548482018-08-11 21:54:05 -0700194#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700195impl From<proc_macro::TokenStream> for TokenStream {
196 fn from(inner: proc_macro::TokenStream) -> TokenStream {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700197 TokenStream::_new(inner.into())
Alex Crichton44bffbc2017-05-19 17:51:59 -0700198 }
199}
200
Alex Crichton53548482018-08-11 21:54:05 -0700201#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700202impl From<TokenStream> for proc_macro::TokenStream {
203 fn from(inner: TokenStream) -> proc_macro::TokenStream {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700204 inner.inner.into()
Alex Crichton44bffbc2017-05-19 17:51:59 -0700205 }
206}
207
Alex Crichtonf3888432018-05-16 09:11:05 -0700208impl Extend<TokenTree> for TokenStream {
209 fn extend<I: IntoIterator<Item = TokenTree>>(&mut self, streams: I) {
210 self.inner.extend(streams)
211 }
212}
213
David Tolnay5c58c532018-08-13 11:33:51 -0700214impl Extend<TokenStream> for TokenStream {
215 fn extend<I: IntoIterator<Item = TokenStream>>(&mut self, streams: I) {
216 self.inner
217 .extend(streams.into_iter().map(|stream| stream.inner))
218 }
219}
220
David Tolnay82ba02d2018-05-20 16:22:43 -0700221/// Collects a number of token trees into a single stream.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700222impl FromIterator<TokenTree> for TokenStream {
223 fn from_iter<I: IntoIterator<Item = TokenTree>>(streams: I) -> Self {
224 TokenStream::_new(streams.into_iter().collect())
Alex Crichton44bffbc2017-05-19 17:51:59 -0700225 }
226}
Alex Crichton53b00672018-09-06 17:16:10 -0700227impl FromIterator<TokenStream> for TokenStream {
228 fn from_iter<I: IntoIterator<Item = TokenStream>>(streams: I) -> Self {
229 TokenStream::_new(streams.into_iter().map(|i| i.inner).collect())
230 }
231}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700232
David Tolnay82ba02d2018-05-20 16:22:43 -0700233/// Prints the token stream as a string that is supposed to be losslessly
234/// convertible back into the same token stream (modulo spans), except for
235/// possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative
236/// numeric literals.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700237impl fmt::Display for TokenStream {
238 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
239 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700240 }
241}
242
David Tolnay82ba02d2018-05-20 16:22:43 -0700243/// Prints token in a form convenient for debugging.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700244impl fmt::Debug for TokenStream {
245 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
246 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700247 }
248}
249
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700250impl fmt::Debug for LexError {
251 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
252 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700253 }
254}
255
David Tolnay82ba02d2018-05-20 16:22:43 -0700256/// The source file of a given `Span`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700257///
258/// This type is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800259#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500260#[derive(Clone, PartialEq, Eq)]
David Tolnay7e654a82018-11-11 13:33:18 -0800261pub struct SourceFile {
262 inner: imp::SourceFile,
263 _marker: marker::PhantomData<Rc<()>>,
264}
Nika Layzellf8d5f212017-12-11 14:07:02 -0500265
David Tolnay1ebe3972018-01-02 20:14:20 -0800266#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500267impl SourceFile {
David Tolnay7e654a82018-11-11 13:33:18 -0800268 fn _new(inner: imp::SourceFile) -> Self {
269 SourceFile {
David Tolnayf0814122019-07-19 11:53:55 -0700270 inner,
David Tolnay7e654a82018-11-11 13:33:18 -0800271 _marker: marker::PhantomData,
272 }
273 }
274
David Tolnay82ba02d2018-05-20 16:22:43 -0700275 /// Get the path to this source file.
276 ///
277 /// ### Note
278 ///
279 /// If the code span associated with this `SourceFile` was generated by an
280 /// external macro, this may not be an actual path on the filesystem. Use
281 /// [`is_real`] to check.
282 ///
283 /// Also note that even if `is_real` returns `true`, if
284 /// `--remap-path-prefix` was passed on the command line, the path as given
285 /// may not actually be valid.
286 ///
287 /// [`is_real`]: #method.is_real
David Tolnay9cd3b4c2018-11-11 16:47:32 -0800288 pub fn path(&self) -> PathBuf {
David Tolnay7e654a82018-11-11 13:33:18 -0800289 self.inner.path()
Nika Layzellf8d5f212017-12-11 14:07:02 -0500290 }
291
David Tolnay82ba02d2018-05-20 16:22:43 -0700292 /// Returns `true` if this source file is a real source file, and not
293 /// generated by an external macro's expansion.
Nika Layzellf8d5f212017-12-11 14:07:02 -0500294 pub fn is_real(&self) -> bool {
David Tolnay7e654a82018-11-11 13:33:18 -0800295 self.inner.is_real()
Nika Layzellf8d5f212017-12-11 14:07:02 -0500296 }
297}
298
David Tolnay1ebe3972018-01-02 20:14:20 -0800299#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500300impl fmt::Debug for SourceFile {
301 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
David Tolnay7e654a82018-11-11 13:33:18 -0800302 self.inner.fmt(f)
Nika Layzellf8d5f212017-12-11 14:07:02 -0500303 }
304}
305
David Tolnay82ba02d2018-05-20 16:22:43 -0700306/// A line-column pair representing the start or end of a `Span`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700307///
308/// This type is semver exempt and not exposed by default.
David Tolnay3b1f7d22019-01-28 12:22:11 -0800309#[cfg(span_locations)]
David Tolnay9d4fb442019-04-22 16:42:41 -0700310#[derive(Copy, Clone, Debug, PartialEq, Eq)]
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500311pub struct LineColumn {
David Tolnay82ba02d2018-05-20 16:22:43 -0700312 /// The 1-indexed line in the source file on which the span starts or ends
313 /// (inclusive).
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500314 pub line: usize,
David Tolnay82ba02d2018-05-20 16:22:43 -0700315 /// The 0-indexed column (in UTF-8 characters) in the source file on which
316 /// the span starts or ends (inclusive).
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500317 pub column: usize,
318}
Nika Layzellf8d5f212017-12-11 14:07:02 -0500319
David Tolnay82ba02d2018-05-20 16:22:43 -0700320/// A region of source code, along with macro expansion information.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700321#[derive(Copy, Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700322pub struct Span {
323 inner: imp::Span,
324 _marker: marker::PhantomData<Rc<()>>,
325}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700326
Alex Crichton44bffbc2017-05-19 17:51:59 -0700327impl Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700328 fn _new(inner: imp::Span) -> Span {
329 Span {
David Tolnayf0814122019-07-19 11:53:55 -0700330 inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700331 _marker: marker::PhantomData,
332 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700333 }
Alex Crichtone6085b72017-11-21 07:24:25 -0800334
David Tolnayaef075b2019-01-16 16:29:18 -0800335 fn _new_stable(inner: fallback::Span) -> Span {
Alex Crichton30a4e9e2018-04-27 17:02:19 -0700336 Span {
337 inner: inner.into(),
338 _marker: marker::PhantomData,
339 }
340 }
341
David Tolnay82ba02d2018-05-20 16:22:43 -0700342 /// The span of the invocation of the current procedural macro.
343 ///
344 /// Identifiers created with this span will be resolved as if they were
345 /// written directly at the macro call location (call-site hygiene) and
346 /// other code at the macro call site will be able to refer to them as well.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700347 pub fn call_site() -> Span {
348 Span::_new(imp::Span::call_site())
349 }
350
David Tolnay82ba02d2018-05-20 16:22:43 -0700351 /// A span that resolves at the macro definition site.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700352 ///
353 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700354 #[cfg(procmacro2_semver_exempt)]
Alex Crichtone6085b72017-11-21 07:24:25 -0800355 pub fn def_site() -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700356 Span::_new(imp::Span::def_site())
Alex Crichtone6085b72017-11-21 07:24:25 -0800357 }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500358
David Tolnay4e8e3972018-01-05 18:10:22 -0800359 /// Creates a new span with the same line/column information as `self` but
360 /// that resolves symbols as though it were at `other`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700361 ///
362 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700363 #[cfg(procmacro2_semver_exempt)]
David Tolnay4e8e3972018-01-05 18:10:22 -0800364 pub fn resolved_at(&self, other: Span) -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700365 Span::_new(self.inner.resolved_at(other.inner))
David Tolnay4e8e3972018-01-05 18:10:22 -0800366 }
367
368 /// Creates a new span with the same name resolution behavior as `self` but
369 /// with the line/column information of `other`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700370 ///
371 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700372 #[cfg(procmacro2_semver_exempt)]
David Tolnay4e8e3972018-01-05 18:10:22 -0800373 pub fn located_at(&self, other: Span) -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700374 Span::_new(self.inner.located_at(other.inner))
David Tolnay4e8e3972018-01-05 18:10:22 -0800375 }
376
David Tolnay17eb0702019-01-05 12:23:17 -0800377 /// Convert `proc_macro2::Span` to `proc_macro::Span`.
378 ///
379 /// This method is available when building with a nightly compiler, or when
380 /// building with rustc 1.29+ *without* semver exempt features.
David Tolnayc0425cd2019-01-16 12:13:15 -0800381 ///
382 /// # Panics
383 ///
384 /// Panics if called from outside of a procedural macro. Unlike
385 /// `proc_macro2::Span`, the `proc_macro::Span` type can only exist within
386 /// the context of a procedural macro invocation.
David Tolnay17eb0702019-01-05 12:23:17 -0800387 #[cfg(wrap_proc_macro)]
David Tolnay40bbb1c2019-01-19 19:43:55 -0800388 pub fn unwrap(self) -> proc_macro::Span {
389 self.inner.unwrap()
390 }
391
392 // Soft deprecated. Please use Span::unwrap.
393 #[cfg(wrap_proc_macro)]
394 #[doc(hidden)]
David Tolnay16a17202017-12-31 10:47:24 -0500395 pub fn unstable(self) -> proc_macro::Span {
David Tolnay40bbb1c2019-01-19 19:43:55 -0800396 self.unwrap()
David Tolnay16a17202017-12-31 10:47:24 -0500397 }
398
David Tolnay82ba02d2018-05-20 16:22:43 -0700399 /// The original source file into which this span points.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700400 ///
401 /// This method is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800402 #[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500403 pub fn source_file(&self) -> SourceFile {
David Tolnay7e654a82018-11-11 13:33:18 -0800404 SourceFile::_new(self.inner.source_file())
Nika Layzellf8d5f212017-12-11 14:07:02 -0500405 }
406
David Tolnay82ba02d2018-05-20 16:22:43 -0700407 /// Get the starting line/column in the source file for this span.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700408 ///
David Tolnay3b1f7d22019-01-28 12:22:11 -0800409 /// This method requires the `"span-locations"` feature to be enabled.
410 #[cfg(span_locations)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500411 pub fn start(&self) -> LineColumn {
David Tolnayb28f38a2018-03-31 22:02:29 +0200412 let imp::LineColumn { line, column } = self.inner.start();
David Tolnayf0814122019-07-19 11:53:55 -0700413 LineColumn { line, column }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500414 }
415
David Tolnay82ba02d2018-05-20 16:22:43 -0700416 /// Get the ending line/column in the source file for this span.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700417 ///
David Tolnay3b1f7d22019-01-28 12:22:11 -0800418 /// This method requires the `"span-locations"` feature to be enabled.
419 #[cfg(span_locations)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500420 pub fn end(&self) -> LineColumn {
David Tolnayb28f38a2018-03-31 22:02:29 +0200421 let imp::LineColumn { line, column } = self.inner.end();
David Tolnayf0814122019-07-19 11:53:55 -0700422 LineColumn { line, column }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500423 }
424
David Tolnay82ba02d2018-05-20 16:22:43 -0700425 /// Create a new span encompassing `self` and `other`.
426 ///
427 /// Returns `None` if `self` and `other` are from different files.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700428 ///
429 /// This method is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800430 #[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500431 pub fn join(&self, other: Span) -> Option<Span> {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700432 self.inner.join(other.inner).map(Span::_new)
433 }
Alex Crichtonb2c94622018-04-04 07:36:41 -0700434
David Tolnay0360cf82019-06-30 13:09:49 -0700435 /// Compares two spans to see if they're equal.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700436 ///
437 /// This method is semver exempt and not exposed by default.
Alex Crichtonb2c94622018-04-04 07:36:41 -0700438 #[cfg(procmacro2_semver_exempt)]
439 pub fn eq(&self, other: &Span) -> bool {
440 self.inner.eq(&other.inner)
441 }
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700442}
443
David Tolnay82ba02d2018-05-20 16:22:43 -0700444/// Prints a span in a form convenient for debugging.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700445impl fmt::Debug for Span {
446 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
447 self.inner.fmt(f)
Nika Layzellf8d5f212017-12-11 14:07:02 -0500448 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700449}
450
David Tolnay82ba02d2018-05-20 16:22:43 -0700451/// A single token or a delimited sequence of token trees (e.g. `[1, (), ..]`).
David Tolnay034205f2018-04-22 16:45:28 -0700452#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700453pub enum TokenTree {
David Tolnay82ba02d2018-05-20 16:22:43 -0700454 /// A token stream surrounded by bracket delimiters.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700455 Group(Group),
David Tolnay82ba02d2018-05-20 16:22:43 -0700456 /// An identifier.
Alex Crichtonf3888432018-05-16 09:11:05 -0700457 Ident(Ident),
David Tolnay82ba02d2018-05-20 16:22:43 -0700458 /// A single punctuation character (`+`, `,`, `$`, etc.).
Alex Crichtonf3888432018-05-16 09:11:05 -0700459 Punct(Punct),
David Tolnay82ba02d2018-05-20 16:22:43 -0700460 /// A literal character (`'a'`), string (`"hello"`), number (`2.3`), etc.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700461 Literal(Literal),
Alex Crichton1a7f7622017-07-05 17:47:15 -0700462}
463
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700464impl TokenTree {
David Tolnay82ba02d2018-05-20 16:22:43 -0700465 /// Returns the span of this tree, delegating to the `span` method of
466 /// the contained token or a delimited stream.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700467 pub fn span(&self) -> Span {
468 match *self {
469 TokenTree::Group(ref t) => t.span(),
Alex Crichtonf3888432018-05-16 09:11:05 -0700470 TokenTree::Ident(ref t) => t.span(),
471 TokenTree::Punct(ref t) => t.span(),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700472 TokenTree::Literal(ref t) => t.span(),
473 }
474 }
475
David Tolnay82ba02d2018-05-20 16:22:43 -0700476 /// Configures the span for *only this token*.
477 ///
478 /// Note that if this token is a `Group` then this method will not configure
479 /// the span of each of the internal tokens, this will simply delegate to
480 /// the `set_span` method of each variant.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700481 pub fn set_span(&mut self, span: Span) {
482 match *self {
483 TokenTree::Group(ref mut t) => t.set_span(span),
Alex Crichtonf3888432018-05-16 09:11:05 -0700484 TokenTree::Ident(ref mut t) => t.set_span(span),
485 TokenTree::Punct(ref mut t) => t.set_span(span),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700486 TokenTree::Literal(ref mut t) => t.set_span(span),
487 }
488 }
489}
490
491impl From<Group> for TokenTree {
492 fn from(g: Group) -> TokenTree {
493 TokenTree::Group(g)
494 }
495}
496
Alex Crichtonf3888432018-05-16 09:11:05 -0700497impl From<Ident> for TokenTree {
498 fn from(g: Ident) -> TokenTree {
499 TokenTree::Ident(g)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700500 }
501}
502
Alex Crichtonf3888432018-05-16 09:11:05 -0700503impl From<Punct> for TokenTree {
504 fn from(g: Punct) -> TokenTree {
505 TokenTree::Punct(g)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700506 }
507}
508
509impl From<Literal> for TokenTree {
510 fn from(g: Literal) -> TokenTree {
511 TokenTree::Literal(g)
Alex Crichton1a7f7622017-07-05 17:47:15 -0700512 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700513}
514
David Tolnay82ba02d2018-05-20 16:22:43 -0700515/// Prints the token tree as a string that is supposed to be losslessly
516/// convertible back into the same token tree (modulo spans), except for
517/// possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative
518/// numeric literals.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700519impl fmt::Display for TokenTree {
520 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700521 match *self {
522 TokenTree::Group(ref t) => t.fmt(f),
Alex Crichtonf3888432018-05-16 09:11:05 -0700523 TokenTree::Ident(ref t) => t.fmt(f),
524 TokenTree::Punct(ref t) => t.fmt(f),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700525 TokenTree::Literal(ref t) => t.fmt(f),
526 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700527 }
528}
529
David Tolnay82ba02d2018-05-20 16:22:43 -0700530/// Prints token tree in a form convenient for debugging.
David Tolnay034205f2018-04-22 16:45:28 -0700531impl fmt::Debug for TokenTree {
532 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
533 // Each of these has the name in the struct type in the derived debug,
534 // so don't bother with an extra layer of indirection
535 match *self {
536 TokenTree::Group(ref t) => t.fmt(f),
David Tolnayd8fcdb82018-06-02 15:43:53 -0700537 TokenTree::Ident(ref t) => {
538 let mut debug = f.debug_struct("Ident");
539 debug.field("sym", &format_args!("{}", t));
David Tolnayfd8cdc82019-01-19 19:23:59 -0800540 imp::debug_span_field_if_nontrivial(&mut debug, t.span().inner);
David Tolnayd8fcdb82018-06-02 15:43:53 -0700541 debug.finish()
542 }
Alex Crichtonf3888432018-05-16 09:11:05 -0700543 TokenTree::Punct(ref t) => t.fmt(f),
David Tolnay034205f2018-04-22 16:45:28 -0700544 TokenTree::Literal(ref t) => t.fmt(f),
545 }
546 }
547}
548
David Tolnay82ba02d2018-05-20 16:22:43 -0700549/// A delimited token stream.
550///
551/// A `Group` internally contains a `TokenStream` which is surrounded by
552/// `Delimiter`s.
David Tolnay034205f2018-04-22 16:45:28 -0700553#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700554pub struct Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700555 inner: imp::Group,
Alex Crichton44bffbc2017-05-19 17:51:59 -0700556}
557
David Tolnay82ba02d2018-05-20 16:22:43 -0700558/// Describes how a sequence of token trees is delimited.
Michael Layzell5372f4b2017-06-02 10:29:31 -0400559#[derive(Copy, Clone, Debug, Eq, PartialEq)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700560pub enum Delimiter {
David Tolnay82ba02d2018-05-20 16:22:43 -0700561 /// `( ... )`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700562 Parenthesis,
David Tolnay82ba02d2018-05-20 16:22:43 -0700563 /// `{ ... }`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700564 Brace,
David Tolnay82ba02d2018-05-20 16:22:43 -0700565 /// `[ ... ]`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700566 Bracket,
David Tolnay82ba02d2018-05-20 16:22:43 -0700567 /// `Ø ... Ø`
568 ///
569 /// An implicit delimiter, that may, for example, appear around tokens
570 /// coming from a "macro variable" `$var`. It is important to preserve
571 /// operator priorities in cases like `$var * 3` where `$var` is `1 + 2`.
572 /// Implicit delimiters may not survive roundtrip of a token stream through
573 /// a string.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700574 None,
575}
576
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700577impl Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700578 fn _new(inner: imp::Group) -> Self {
David Tolnayf0814122019-07-19 11:53:55 -0700579 Group { inner }
David Tolnayf14813f2018-09-08 17:14:07 -0700580 }
581
David Tolnayaef075b2019-01-16 16:29:18 -0800582 fn _new_stable(inner: fallback::Group) -> Self {
David Tolnayf14813f2018-09-08 17:14:07 -0700583 Group {
584 inner: inner.into(),
585 }
586 }
587
David Tolnay82ba02d2018-05-20 16:22:43 -0700588 /// Creates a new `Group` with the given delimiter and token stream.
589 ///
590 /// This constructor will set the span for this group to
591 /// `Span::call_site()`. To change the span you can use the `set_span`
592 /// method below.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700593 pub fn new(delimiter: Delimiter, stream: TokenStream) -> Group {
594 Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700595 inner: imp::Group::new(delimiter, stream.inner),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700596 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700597 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700598
David Tolnay82ba02d2018-05-20 16:22:43 -0700599 /// Returns the delimiter of this `Group`
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700600 pub fn delimiter(&self) -> Delimiter {
David Tolnayf14813f2018-09-08 17:14:07 -0700601 self.inner.delimiter()
Alex Crichton44bffbc2017-05-19 17:51:59 -0700602 }
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700603
David Tolnay82ba02d2018-05-20 16:22:43 -0700604 /// Returns the `TokenStream` of tokens that are delimited in this `Group`.
605 ///
606 /// Note that the returned token stream does not include the delimiter
607 /// returned above.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700608 pub fn stream(&self) -> TokenStream {
David Tolnayf14813f2018-09-08 17:14:07 -0700609 TokenStream::_new(self.inner.stream())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700610 }
611
David Tolnay82ba02d2018-05-20 16:22:43 -0700612 /// Returns the span for the delimiters of this token stream, spanning the
613 /// entire `Group`.
David Tolnayf14813f2018-09-08 17:14:07 -0700614 ///
615 /// ```text
616 /// pub fn span(&self) -> Span {
617 /// ^^^^^^^
618 /// ```
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700619 pub fn span(&self) -> Span {
David Tolnayf14813f2018-09-08 17:14:07 -0700620 Span::_new(self.inner.span())
621 }
622
623 /// Returns the span pointing to the opening delimiter of this group.
624 ///
625 /// ```text
626 /// pub fn span_open(&self) -> Span {
627 /// ^
628 /// ```
629 #[cfg(procmacro2_semver_exempt)]
630 pub fn span_open(&self) -> Span {
631 Span::_new(self.inner.span_open())
632 }
633
634 /// Returns the span pointing to the closing delimiter of this group.
635 ///
636 /// ```text
637 /// pub fn span_close(&self) -> Span {
638 /// ^
639 /// ```
640 #[cfg(procmacro2_semver_exempt)]
641 pub fn span_close(&self) -> Span {
642 Span::_new(self.inner.span_close())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700643 }
644
David Tolnay82ba02d2018-05-20 16:22:43 -0700645 /// Configures the span for this `Group`'s delimiters, but not its internal
646 /// tokens.
647 ///
648 /// This method will **not** set the span of all the internal tokens spanned
649 /// by this group, but rather it will only set the span of the delimiter
650 /// tokens at the level of the `Group`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700651 pub fn set_span(&mut self, span: Span) {
David Tolnayf14813f2018-09-08 17:14:07 -0700652 self.inner.set_span(span.inner)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700653 }
654}
655
David Tolnay82ba02d2018-05-20 16:22:43 -0700656/// Prints the group as a string that should be losslessly convertible back
657/// into the same group (modulo spans), except for possibly `TokenTree::Group`s
658/// with `Delimiter::None` delimiters.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700659impl fmt::Display for Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700660 fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
661 fmt::Display::fmt(&self.inner, formatter)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700662 }
663}
664
David Tolnay034205f2018-04-22 16:45:28 -0700665impl fmt::Debug for Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700666 fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
667 fmt::Debug::fmt(&self.inner, formatter)
David Tolnay034205f2018-04-22 16:45:28 -0700668 }
669}
670
David Tolnay82ba02d2018-05-20 16:22:43 -0700671/// An `Punct` is an single punctuation character like `+`, `-` or `#`.
672///
673/// Multicharacter operators like `+=` are represented as two instances of
674/// `Punct` with different forms of `Spacing` returned.
Alex Crichtonf3888432018-05-16 09:11:05 -0700675#[derive(Clone)]
676pub struct Punct {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700677 op: char,
678 spacing: Spacing,
679 span: Span,
Alex Crichton44bffbc2017-05-19 17:51:59 -0700680}
681
David Tolnay82ba02d2018-05-20 16:22:43 -0700682/// Whether an `Punct` is followed immediately by another `Punct` or followed by
683/// another token or whitespace.
Lukas Kalbertodteb3f9302017-08-20 18:58:41 +0200684#[derive(Copy, Clone, Debug, Eq, PartialEq)]
Alex Crichton1a7f7622017-07-05 17:47:15 -0700685pub enum Spacing {
David Tolnay82ba02d2018-05-20 16:22:43 -0700686 /// E.g. `+` is `Alone` in `+ =`, `+ident` or `+()`.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700687 Alone,
David Tolnay82ba02d2018-05-20 16:22:43 -0700688 /// E.g. `+` is `Joint` in `+=` or `'#`.
689 ///
690 /// Additionally, single quote `'` can join with identifiers to form
691 /// lifetimes `'ident`.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700692 Joint,
693}
694
Alex Crichtonf3888432018-05-16 09:11:05 -0700695impl Punct {
David Tolnay82ba02d2018-05-20 16:22:43 -0700696 /// Creates a new `Punct` from the given character and spacing.
697 ///
698 /// The `ch` argument must be a valid punctuation character permitted by the
699 /// language, otherwise the function will panic.
700 ///
701 /// The returned `Punct` will have the default span of `Span::call_site()`
702 /// which can be further configured with the `set_span` method below.
Alex Crichtonf3888432018-05-16 09:11:05 -0700703 pub fn new(op: char, spacing: Spacing) -> Punct {
704 Punct {
David Tolnayf0814122019-07-19 11:53:55 -0700705 op,
706 spacing,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700707 span: Span::call_site(),
708 }
709 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700710
David Tolnay82ba02d2018-05-20 16:22:43 -0700711 /// Returns the value of this punctuation character as `char`.
Alex Crichtonf3888432018-05-16 09:11:05 -0700712 pub fn as_char(&self) -> char {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700713 self.op
714 }
715
David Tolnay82ba02d2018-05-20 16:22:43 -0700716 /// Returns the spacing of this punctuation character, indicating whether
717 /// it's immediately followed by another `Punct` in the token stream, so
718 /// they can potentially be combined into a multicharacter operator
719 /// (`Joint`), or it's followed by some other token or whitespace (`Alone`)
720 /// so the operator has certainly ended.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700721 pub fn spacing(&self) -> Spacing {
722 self.spacing
723 }
724
David Tolnay82ba02d2018-05-20 16:22:43 -0700725 /// Returns the span for this punctuation character.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700726 pub fn span(&self) -> Span {
727 self.span
728 }
729
David Tolnay82ba02d2018-05-20 16:22:43 -0700730 /// Configure the span for this punctuation character.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700731 pub fn set_span(&mut self, span: Span) {
732 self.span = span;
733 }
734}
735
David Tolnay82ba02d2018-05-20 16:22:43 -0700736/// Prints the punctuation character as a string that should be losslessly
737/// convertible back into the same character.
Alex Crichtonf3888432018-05-16 09:11:05 -0700738impl fmt::Display for Punct {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700739 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
740 self.op.fmt(f)
741 }
742}
743
Alex Crichtonf3888432018-05-16 09:11:05 -0700744impl fmt::Debug for Punct {
David Tolnay034205f2018-04-22 16:45:28 -0700745 fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
Alex Crichtonf3888432018-05-16 09:11:05 -0700746 let mut debug = fmt.debug_struct("Punct");
David Tolnay034205f2018-04-22 16:45:28 -0700747 debug.field("op", &self.op);
748 debug.field("spacing", &self.spacing);
David Tolnayfd8cdc82019-01-19 19:23:59 -0800749 imp::debug_span_field_if_nontrivial(&mut debug, self.span.inner);
David Tolnay034205f2018-04-22 16:45:28 -0700750 debug.finish()
751 }
752}
753
David Tolnay8b71dac2018-05-20 17:07:47 -0700754/// A word of Rust code, which may be a keyword or legal variable name.
755///
756/// An identifier consists of at least one Unicode code point, the first of
757/// which has the XID_Start property and the rest of which have the XID_Continue
758/// property.
759///
760/// - The empty string is not an identifier. Use `Option<Ident>`.
761/// - A lifetime is not an identifier. Use `syn::Lifetime` instead.
762///
763/// An identifier constructed with `Ident::new` is permitted to be a Rust
David Tolnayc239f032018-11-11 12:57:09 -0800764/// keyword, though parsing one through its [`Parse`] implementation rejects
765/// Rust keywords. Use `input.call(Ident::parse_any)` when parsing to match the
David Tolnay8b71dac2018-05-20 17:07:47 -0700766/// behaviour of `Ident::new`.
767///
David Tolnayc239f032018-11-11 12:57:09 -0800768/// [`Parse`]: https://docs.rs/syn/0.15/syn/parse/trait.Parse.html
David Tolnay8b71dac2018-05-20 17:07:47 -0700769///
770/// # Examples
771///
772/// A new ident can be created from a string using the `Ident::new` function.
773/// A span must be provided explicitly which governs the name resolution
774/// behavior of the resulting identifier.
775///
David Tolnay7a964732019-01-19 19:03:20 -0800776/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700777/// use proc_macro2::{Ident, Span};
778///
779/// fn main() {
780/// let call_ident = Ident::new("calligraphy", Span::call_site());
781///
782/// println!("{}", call_ident);
783/// }
784/// ```
785///
786/// An ident can be interpolated into a token stream using the `quote!` macro.
787///
David Tolnay7a964732019-01-19 19:03:20 -0800788/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700789/// use proc_macro2::{Ident, Span};
David Tolnay7a964732019-01-19 19:03:20 -0800790/// use quote::quote;
David Tolnay8b71dac2018-05-20 17:07:47 -0700791///
792/// fn main() {
793/// let ident = Ident::new("demo", Span::call_site());
794///
795/// // Create a variable binding whose name is this ident.
796/// let expanded = quote! { let #ident = 10; };
797///
798/// // Create a variable binding with a slightly different name.
799/// let temp_ident = Ident::new(&format!("new_{}", ident), Span::call_site());
800/// let expanded = quote! { let #temp_ident = 10; };
801/// }
802/// ```
803///
804/// A string representation of the ident is available through the `to_string()`
805/// method.
806///
David Tolnay7a964732019-01-19 19:03:20 -0800807/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700808/// # use proc_macro2::{Ident, Span};
809/// #
810/// # let ident = Ident::new("another_identifier", Span::call_site());
811/// #
812/// // Examine the ident as a string.
813/// let ident_string = ident.to_string();
814/// if ident_string.len() > 60 {
815/// println!("Very long identifier: {}", ident_string)
816/// }
817/// ```
Alex Crichtonf3888432018-05-16 09:11:05 -0700818#[derive(Clone)]
819pub struct Ident {
820 inner: imp::Ident,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700821 _marker: marker::PhantomData<Rc<()>>,
822}
823
Alex Crichtonf3888432018-05-16 09:11:05 -0700824impl Ident {
825 fn _new(inner: imp::Ident) -> Ident {
826 Ident {
David Tolnayf0814122019-07-19 11:53:55 -0700827 inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700828 _marker: marker::PhantomData,
829 }
830 }
831
David Tolnay82ba02d2018-05-20 16:22:43 -0700832 /// Creates a new `Ident` with the given `string` as well as the specified
833 /// `span`.
834 ///
835 /// The `string` argument must be a valid identifier permitted by the
836 /// language, otherwise the function will panic.
837 ///
838 /// Note that `span`, currently in rustc, configures the hygiene information
839 /// for this identifier.
840 ///
841 /// As of this time `Span::call_site()` explicitly opts-in to "call-site"
842 /// hygiene meaning that identifiers created with this span will be resolved
843 /// as if they were written directly at the location of the macro call, and
844 /// other code at the macro call site will be able to refer to them as well.
845 ///
846 /// Later spans like `Span::def_site()` will allow to opt-in to
847 /// "definition-site" hygiene meaning that identifiers created with this
848 /// span will be resolved at the location of the macro definition and other
849 /// code at the macro call site will not be able to refer to them.
850 ///
851 /// Due to the current importance of hygiene this constructor, unlike other
852 /// tokens, requires a `Span` to be specified at construction.
David Tolnay8b71dac2018-05-20 17:07:47 -0700853 ///
854 /// # Panics
855 ///
856 /// Panics if the input string is neither a keyword nor a legal variable
David Tolnay219b1d32019-04-22 16:04:11 -0700857 /// name. If you are not sure whether the string contains an identifier and
858 /// need to handle an error case, use
859 /// <a href="https://docs.rs/syn/0.15/syn/fn.parse_str.html"><code
860 /// style="padding-right:0;">syn::parse_str</code></a><code
861 /// style="padding-left:0;">::&lt;Ident&gt;</code>
862 /// rather than `Ident::new`.
Alex Crichtonf3888432018-05-16 09:11:05 -0700863 pub fn new(string: &str, span: Span) -> Ident {
864 Ident::_new(imp::Ident::new(string, span.inner))
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700865 }
866
David Tolnay82ba02d2018-05-20 16:22:43 -0700867 /// Same as `Ident::new`, but creates a raw identifier (`r#ident`).
David Tolnaya01ca8e2018-06-04 00:55:28 -0700868 ///
869 /// This method is semver exempt and not exposed by default.
Alex Crichtonf3888432018-05-16 09:11:05 -0700870 #[cfg(procmacro2_semver_exempt)]
871 pub fn new_raw(string: &str, span: Span) -> Ident {
872 Ident::_new_raw(string, span)
873 }
874
875 fn _new_raw(string: &str, span: Span) -> Ident {
876 Ident::_new(imp::Ident::new_raw(string, span.inner))
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700877 }
878
David Tolnay82ba02d2018-05-20 16:22:43 -0700879 /// Returns the span of this `Ident`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700880 pub fn span(&self) -> Span {
Alex Crichtonb2c94622018-04-04 07:36:41 -0700881 Span::_new(self.inner.span())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700882 }
883
David Tolnay82ba02d2018-05-20 16:22:43 -0700884 /// Configures the span of this `Ident`, possibly changing its hygiene
885 /// context.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700886 pub fn set_span(&mut self, span: Span) {
Alex Crichtonb2c94622018-04-04 07:36:41 -0700887 self.inner.set_span(span.inner);
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700888 }
889}
890
Alex Crichtonf3888432018-05-16 09:11:05 -0700891impl PartialEq for Ident {
David Tolnay3d9d6ad2018-05-18 10:51:55 -0700892 fn eq(&self, other: &Ident) -> bool {
David Tolnayc0b0f2e2018-09-02 17:56:08 -0700893 self.inner == other.inner
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700894 }
895}
896
David Tolnayc0bbcc52018-05-18 10:51:04 -0700897impl<T> PartialEq<T> for Ident
898where
899 T: ?Sized + AsRef<str>,
900{
901 fn eq(&self, other: &T) -> bool {
David Tolnayc0b0f2e2018-09-02 17:56:08 -0700902 self.inner == other
David Tolnayc0bbcc52018-05-18 10:51:04 -0700903 }
904}
905
David Tolnay3d9d6ad2018-05-18 10:51:55 -0700906impl Eq for Ident {}
Alex Crichtonf3888432018-05-16 09:11:05 -0700907
908impl PartialOrd for Ident {
909 fn partial_cmp(&self, other: &Ident) -> Option<Ordering> {
910 Some(self.cmp(other))
911 }
912}
913
914impl Ord for Ident {
915 fn cmp(&self, other: &Ident) -> Ordering {
916 self.to_string().cmp(&other.to_string())
917 }
918}
919
920impl Hash for Ident {
921 fn hash<H: Hasher>(&self, hasher: &mut H) {
922 self.to_string().hash(hasher)
923 }
924}
925
David Tolnay82ba02d2018-05-20 16:22:43 -0700926/// Prints the identifier as a string that should be losslessly convertible back
927/// into the same identifier.
Alex Crichtonf3888432018-05-16 09:11:05 -0700928impl fmt::Display for Ident {
929 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
930 self.inner.fmt(f)
931 }
932}
933
934impl fmt::Debug for Ident {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700935 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
936 self.inner.fmt(f)
937 }
938}
939
David Tolnay82ba02d2018-05-20 16:22:43 -0700940/// A literal string (`"hello"`), byte string (`b"hello"`), character (`'a'`),
941/// byte character (`b'a'`), an integer or floating point number with or without
942/// a suffix (`1`, `1u8`, `2.3`, `2.3f32`).
943///
944/// Boolean literals like `true` and `false` do not belong here, they are
945/// `Ident`s.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700946#[derive(Clone)]
947pub struct Literal {
948 inner: imp::Literal,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700949 _marker: marker::PhantomData<Rc<()>>,
950}
951
David Tolnay82ba02d2018-05-20 16:22:43 -0700952macro_rules! suffixed_int_literals {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700953 ($($name:ident => $kind:ident,)*) => ($(
David Tolnay82ba02d2018-05-20 16:22:43 -0700954 /// Creates a new suffixed integer literal with the specified value.
955 ///
956 /// This function will create an integer like `1u32` where the integer
957 /// value specified is the first part of the token and the integral is
958 /// also suffixed at the end. Literals created from negative numbers may
959 /// not survive rountrips through `TokenStream` or strings and may be
960 /// broken into two tokens (`-` and positive literal).
961 ///
962 /// Literals created through this method have the `Span::call_site()`
963 /// span by default, which can be configured with the `set_span` method
964 /// below.
965 pub fn $name(n: $kind) -> Literal {
966 Literal::_new(imp::Literal::$name(n))
967 }
968 )*)
969}
970
971macro_rules! unsuffixed_int_literals {
972 ($($name:ident => $kind:ident,)*) => ($(
973 /// Creates a new unsuffixed integer literal with the specified value.
974 ///
975 /// This function will create an integer like `1` where the integer
976 /// value specified is the first part of the token. No suffix is
977 /// specified on this token, meaning that invocations like
978 /// `Literal::i8_unsuffixed(1)` are equivalent to
979 /// `Literal::u32_unsuffixed(1)`. Literals created from negative numbers
980 /// may not survive rountrips through `TokenStream` or strings and may
981 /// be broken into two tokens (`-` and positive literal).
982 ///
983 /// Literals created through this method have the `Span::call_site()`
984 /// span by default, which can be configured with the `set_span` method
985 /// below.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700986 pub fn $name(n: $kind) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -0700987 Literal::_new(imp::Literal::$name(n))
Alex Crichton1a7f7622017-07-05 17:47:15 -0700988 }
989 )*)
990}
991
Alex Crichton852d53d2017-05-19 19:25:08 -0700992impl Literal {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700993 fn _new(inner: imp::Literal) -> Literal {
994 Literal {
David Tolnayf0814122019-07-19 11:53:55 -0700995 inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700996 _marker: marker::PhantomData,
997 }
Alex Crichton1a7f7622017-07-05 17:47:15 -0700998 }
999
David Tolnayaef075b2019-01-16 16:29:18 -08001000 fn _new_stable(inner: fallback::Literal) -> Literal {
Alex Crichton30a4e9e2018-04-27 17:02:19 -07001001 Literal {
1002 inner: inner.into(),
1003 _marker: marker::PhantomData,
1004 }
1005 }
1006
David Tolnay82ba02d2018-05-20 16:22:43 -07001007 suffixed_int_literals! {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001008 u8_suffixed => u8,
1009 u16_suffixed => u16,
1010 u32_suffixed => u32,
1011 u64_suffixed => u64,
David Tolnay1596a8c2019-07-19 11:45:26 -07001012 u128_suffixed => u128,
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001013 usize_suffixed => usize,
1014 i8_suffixed => i8,
1015 i16_suffixed => i16,
1016 i32_suffixed => i32,
1017 i64_suffixed => i64,
Alex Crichton69385662018-11-08 06:30:04 -08001018 i128_suffixed => i128,
David Tolnay1596a8c2019-07-19 11:45:26 -07001019 isize_suffixed => isize,
Alex Crichton69385662018-11-08 06:30:04 -08001020 }
1021
David Tolnay82ba02d2018-05-20 16:22:43 -07001022 unsuffixed_int_literals! {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001023 u8_unsuffixed => u8,
1024 u16_unsuffixed => u16,
1025 u32_unsuffixed => u32,
1026 u64_unsuffixed => u64,
David Tolnay1596a8c2019-07-19 11:45:26 -07001027 u128_unsuffixed => u128,
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001028 usize_unsuffixed => usize,
1029 i8_unsuffixed => i8,
1030 i16_unsuffixed => i16,
1031 i32_unsuffixed => i32,
1032 i64_unsuffixed => i64,
Alex Crichton69385662018-11-08 06:30:04 -08001033 i128_unsuffixed => i128,
David Tolnay1596a8c2019-07-19 11:45:26 -07001034 isize_unsuffixed => isize,
Alex Crichton69385662018-11-08 06:30:04 -08001035 }
1036
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001037 /// Creates a new unsuffixed floating-point literal.
1038 ///
1039 /// This constructor is similar to those like `Literal::i8_unsuffixed` where
1040 /// the float's value is emitted directly into the token but no suffix is
1041 /// used, so it may be inferred to be a `f64` later in the compiler.
1042 /// Literals created from negative numbers may not survive rountrips through
1043 /// `TokenStream` or strings and may be broken into two tokens (`-` and
1044 /// positive literal).
1045 ///
1046 /// # Panics
1047 ///
1048 /// This function requires that the specified float is finite, for example
1049 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001050 pub fn f64_unsuffixed(f: f64) -> Literal {
1051 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001052 Literal::_new(imp::Literal::f64_unsuffixed(f))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001053 }
1054
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001055 /// Creates a new suffixed floating-point literal.
1056 ///
1057 /// This constructor will create a literal like `1.0f64` where the value
1058 /// specified is the preceding part of the token and `f64` is the suffix of
1059 /// the token. This token will always be inferred to be an `f64` in the
1060 /// compiler. Literals created from negative numbers may not survive
1061 /// rountrips through `TokenStream` or strings and may be broken into two
1062 /// tokens (`-` and positive literal).
1063 ///
1064 /// # Panics
1065 ///
1066 /// This function requires that the specified float is finite, for example
1067 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001068 pub fn f64_suffixed(f: f64) -> Literal {
1069 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001070 Literal::_new(imp::Literal::f64_suffixed(f))
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001071 }
1072
David Tolnay82ba02d2018-05-20 16:22:43 -07001073 /// Creates a new unsuffixed floating-point literal.
1074 ///
1075 /// This constructor is similar to those like `Literal::i8_unsuffixed` where
1076 /// the float's value is emitted directly into the token but no suffix is
1077 /// used, so it may be inferred to be a `f64` later in the compiler.
1078 /// Literals created from negative numbers may not survive rountrips through
1079 /// `TokenStream` or strings and may be broken into two tokens (`-` and
1080 /// positive literal).
1081 ///
1082 /// # Panics
1083 ///
1084 /// This function requires that the specified float is finite, for example
1085 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001086 pub fn f32_unsuffixed(f: f32) -> Literal {
1087 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001088 Literal::_new(imp::Literal::f32_unsuffixed(f))
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001089 }
1090
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001091 /// Creates a new suffixed floating-point literal.
1092 ///
1093 /// This constructor will create a literal like `1.0f32` where the value
1094 /// specified is the preceding part of the token and `f32` is the suffix of
1095 /// the token. This token will always be inferred to be an `f32` in the
1096 /// compiler. Literals created from negative numbers may not survive
1097 /// rountrips through `TokenStream` or strings and may be broken into two
1098 /// tokens (`-` and positive literal).
1099 ///
1100 /// # Panics
1101 ///
1102 /// This function requires that the specified float is finite, for example
1103 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001104 pub fn f32_suffixed(f: f32) -> Literal {
1105 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001106 Literal::_new(imp::Literal::f32_suffixed(f))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001107 }
1108
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001109 /// String literal.
Alex Crichton1a7f7622017-07-05 17:47:15 -07001110 pub fn string(string: &str) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -07001111 Literal::_new(imp::Literal::string(string))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001112 }
1113
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001114 /// Character literal.
Alex Crichton1a7f7622017-07-05 17:47:15 -07001115 pub fn character(ch: char) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -07001116 Literal::_new(imp::Literal::character(ch))
Alex Crichton76a5cc82017-05-23 07:01:44 -07001117 }
1118
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001119 /// Byte string literal.
Alex Crichton9c2fb0a2017-05-26 08:49:31 -07001120 pub fn byte_string(s: &[u8]) -> Literal {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001121 Literal::_new(imp::Literal::byte_string(s))
Alex Crichton852d53d2017-05-19 19:25:08 -07001122 }
Alex Crichton76a5cc82017-05-23 07:01:44 -07001123
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001124 /// Returns the span encompassing this literal.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001125 pub fn span(&self) -> Span {
Alex Crichtonb2c94622018-04-04 07:36:41 -07001126 Span::_new(self.inner.span())
Alex Crichton1a7f7622017-07-05 17:47:15 -07001127 }
1128
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001129 /// Configures the span associated for this literal.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001130 pub fn set_span(&mut self, span: Span) {
Alex Crichtonb2c94622018-04-04 07:36:41 -07001131 self.inner.set_span(span.inner);
Alex Crichton31316622017-05-26 12:54:47 -07001132 }
Alex Crichton852d53d2017-05-19 19:25:08 -07001133}
1134
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001135impl fmt::Debug for Literal {
1136 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1137 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -07001138 }
1139}
David Tolnaycb1b85f2017-06-03 16:40:35 -07001140
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001141impl fmt::Display for Literal {
1142 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1143 self.inner.fmt(f)
1144 }
1145}
1146
David Tolnay82ba02d2018-05-20 16:22:43 -07001147/// Public implementation details for the `TokenStream` type, such as iterators.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001148pub mod token_stream {
1149 use std::fmt;
1150 use std::marker;
1151 use std::rc::Rc;
1152
David Tolnay48ea5042018-04-23 19:17:35 -07001153 use imp;
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001154 pub use TokenStream;
David Tolnayb28f38a2018-03-31 22:02:29 +02001155 use TokenTree;
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001156
David Tolnay82ba02d2018-05-20 16:22:43 -07001157 /// An iterator over `TokenStream`'s `TokenTree`s.
1158 ///
1159 /// The iteration is "shallow", e.g. the iterator doesn't recurse into
1160 /// delimited groups, and returns whole groups as token trees.
Isaac van Bakelf6754d32019-05-08 20:21:06 +01001161 #[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001162 pub struct IntoIter {
1163 inner: imp::TokenTreeIter,
1164 _marker: marker::PhantomData<Rc<()>>,
1165 }
1166
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001167 impl Iterator for IntoIter {
1168 type Item = TokenTree;
1169
1170 fn next(&mut self) -> Option<TokenTree> {
1171 self.inner.next()
1172 }
1173 }
1174
1175 impl fmt::Debug for IntoIter {
1176 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1177 self.inner.fmt(f)
1178 }
1179 }
1180
1181 impl IntoIterator for TokenStream {
1182 type Item = TokenTree;
1183 type IntoIter = IntoIter;
1184
1185 fn into_iter(self) -> IntoIter {
1186 IntoIter {
1187 inner: self.inner.into_iter(),
1188 _marker: marker::PhantomData,
1189 }
1190 }
1191 }
1192}