blob: 24a4001f13b1faf18f80193823a4245b5f84454c [file] [log] [blame]
David Tolnay7aa1c9c2019-01-19 19:53:10 -08001//! A wrapper around the procedural macro API of the compiler's [`proc_macro`]
2//! crate. This library serves three purposes:
Alex Crichtonbabc99e2017-07-05 18:00:29 -07003//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -08004//! [`proc_macro`]: https://doc.rust-lang.org/proc_macro/
Alex Crichtonbabc99e2017-07-05 18:00:29 -07005//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -08006//! - **Bring proc-macro-like functionality to other contexts like build.rs and
7//! main.rs.** Types from `proc_macro` are entirely specific to procedural
8//! macros and cannot ever exist in code outside of a procedural macro.
9//! Meanwhile `proc_macro2` types may exist anywhere including non-macro code.
10//! By developing foundational libraries like [syn] and [quote] against
11//! `proc_macro2` rather than `proc_macro`, the procedural macro ecosystem
12//! becomes easily applicable to many other use cases and we avoid
13//! reimplementing non-macro equivalents of those libraries.
Alex Crichtonbabc99e2017-07-05 18:00:29 -070014//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080015//! - **Make procedural macros unit testable.** As a consequence of being
16//! specific to procedural macros, nothing that uses `proc_macro` can be
17//! executed from a unit test. In order for helper libraries or components of
18//! a macro to be testable in isolation, they must be implemented using
19//! `proc_macro2`.
Alex Crichton1fd0e8a2018-02-04 21:29:13 -080020//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080021//! - **Provide the latest and greatest APIs across all compiler versions.**
22//! Procedural macros were first introduced to Rust in 1.15.0 with an
23//! extremely minimal interface. Since then, many improvements have landed to
24//! make macros more flexible and easier to write. This library tracks the
25//! procedural macro API of the most recent stable compiler but employs a
26//! polyfill to provide that API consistently across any compiler since
27//! 1.15.0.
David Tolnay6b46deb2018-04-25 21:22:46 -070028//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080029//! [syn]: https://github.com/dtolnay/syn
30//! [quote]: https://github.com/dtolnay/quote
David Tolnay6b46deb2018-04-25 21:22:46 -070031//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080032//! # Usage
33//!
34//! The skeleton of a typical procedural macro typically looks like this:
35//!
36//! ```edition2018
37//! extern crate proc_macro;
38//!
39//! # const IGNORE: &str = stringify! {
40//! #[proc_macro_derive(MyDerive)]
41//! # };
42//! pub fn my_derive(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
43//! let input = proc_macro2::TokenStream::from(input);
44//!
45//! let output: proc_macro2::TokenStream = {
46//! /* transform input */
47//! # input
48//! };
49//!
50//! proc_macro::TokenStream::from(output)
51//! }
52//! ```
53//!
54//! If parsing with [Syn], you'll use [`parse_macro_input!`] instead to
55//! propagate parse errors correctly back to the compiler when parsing fails.
56//!
57//! [`parse_macro_input!`]: https://docs.rs/syn/0.15/syn/macro.parse_macro_input.html
58//!
59//! # Unstable features
60//!
61//! The default feature set of proc-macro2 tracks the most recent stable
62//! compiler API. Functionality in `proc_macro` that is not yet stable is not
63//! exposed by proc-macro2 by default.
64//!
65//! To opt into the additional APIs available in the most recent nightly
66//! compiler, the `procmacro2_semver_exempt` config flag must be passed to
67//! rustc. As usual, we will polyfill those nightly-only APIs all the way back
68//! to Rust 1.15.0. As these are unstable APIs that track the nightly compiler,
69//! minor versions of proc-macro2 may make breaking changes to them at any time.
David Tolnay6b46deb2018-04-25 21:22:46 -070070//!
71//! ```sh
72//! RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo build
73//! ```
74//!
75//! Note that this must not only be done for your crate, but for any crate that
76//! depends on your crate. This infectious nature is intentional, as it serves
77//! as a reminder that you are outside of the normal semver guarantees.
78//!
David Tolnay7aa1c9c2019-01-19 19:53:10 -080079//! Semver exempt methods are marked as such in the proc-macro2 documentation.
Michael Bryan43339962019-06-19 09:04:22 +080080//!
81//! # Thread-Safety
82//!
83//! Most types in this crate are `!Sync` because the underlying compiler
84//! types make use of thread-local memory, meaning they cannot be accessed from
85//! a different thread.
Alex Crichtonbabc99e2017-07-05 18:00:29 -070086
David Tolnay15cc4982018-01-08 08:03:27 -080087// Proc-macro2 types in rustdoc of other crates get linked to here.
David Tolnayf5149212019-05-08 13:39:08 -070088#![doc(html_root_url = "https://docs.rs/proc-macro2/0.4.30")]
David Tolnayb455dd72019-04-28 13:50:51 -070089#![cfg_attr(any(proc_macro_span, super_unstable), feature(proc_macro_span))]
David Tolnay3b1f7d22019-01-28 12:22:11 -080090#![cfg_attr(super_unstable, feature(proc_macro_raw_ident, proc_macro_def_site))]
David Tolnayff384092019-07-14 19:50:15 -070091#![allow(bare_trait_objects, ellipsis_inclusive_range_patterns, unknown_lints)]
Alex Crichtoncbec8ec2017-06-02 13:19:33 -070092
Alex Crichton53548482018-08-11 21:54:05 -070093#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -070094extern crate proc_macro;
David Tolnayb1032662017-05-31 15:52:28 -070095extern crate unicode_xid;
Alex Crichton44bffbc2017-05-19 17:51:59 -070096
Alex Crichtonf3888432018-05-16 09:11:05 -070097use std::cmp::Ordering;
Alex Crichton44bffbc2017-05-19 17:51:59 -070098use std::fmt;
Alex Crichtonf3888432018-05-16 09:11:05 -070099use std::hash::{Hash, Hasher};
Alex Crichton44bffbc2017-05-19 17:51:59 -0700100use std::iter::FromIterator;
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700101use std::marker;
David Tolnay9cd3b4c2018-11-11 16:47:32 -0800102#[cfg(procmacro2_semver_exempt)]
103use std::path::PathBuf;
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700104use std::rc::Rc;
105use std::str::FromStr;
Alex Crichton44bffbc2017-05-19 17:51:59 -0700106
David Tolnayb1032662017-05-31 15:52:28 -0700107#[macro_use]
David Tolnayb1032662017-05-31 15:52:28 -0700108mod strnom;
David Tolnayaef075b2019-01-16 16:29:18 -0800109mod fallback;
David Tolnayb1032662017-05-31 15:52:28 -0700110
Alex Crichtonce0904d2018-08-27 17:29:49 -0700111#[cfg(not(wrap_proc_macro))]
David Tolnayaef075b2019-01-16 16:29:18 -0800112use fallback as imp;
113#[path = "wrapper.rs"]
Alex Crichtonce0904d2018-08-27 17:29:49 -0700114#[cfg(wrap_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700115mod imp;
116
David Tolnay82ba02d2018-05-20 16:22:43 -0700117/// An abstract stream of tokens, or more concretely a sequence of token trees.
118///
119/// This type provides interfaces for iterating over token trees and for
120/// collecting token trees into one stream.
121///
122/// Token stream is both the input and output of `#[proc_macro]`,
123/// `#[proc_macro_attribute]` and `#[proc_macro_derive]` definitions.
David Tolnaycb1b85f2017-06-03 16:40:35 -0700124#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700125pub struct TokenStream {
126 inner: imp::TokenStream,
127 _marker: marker::PhantomData<Rc<()>>,
128}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700129
David Tolnay82ba02d2018-05-20 16:22:43 -0700130/// Error returned from `TokenStream::from_str`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700131pub struct LexError {
132 inner: imp::LexError,
133 _marker: marker::PhantomData<Rc<()>>,
134}
135
136impl TokenStream {
137 fn _new(inner: imp::TokenStream) -> TokenStream {
138 TokenStream {
139 inner: inner,
140 _marker: marker::PhantomData,
141 }
142 }
143
David Tolnayaef075b2019-01-16 16:29:18 -0800144 fn _new_stable(inner: fallback::TokenStream) -> TokenStream {
Alex Crichton30a4e9e2018-04-27 17:02:19 -0700145 TokenStream {
146 inner: inner.into(),
147 _marker: marker::PhantomData,
148 }
149 }
150
David Tolnay82ba02d2018-05-20 16:22:43 -0700151 /// Returns an empty `TokenStream` containing no token trees.
David Tolnayc3bb4592018-05-28 20:09:44 -0700152 pub fn new() -> TokenStream {
153 TokenStream::_new(imp::TokenStream::new())
154 }
155
156 #[deprecated(since = "0.4.4", note = "please use TokenStream::new")]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700157 pub fn empty() -> TokenStream {
David Tolnayc3bb4592018-05-28 20:09:44 -0700158 TokenStream::new()
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700159 }
160
David Tolnay82ba02d2018-05-20 16:22:43 -0700161 /// Checks if this `TokenStream` is empty.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700162 pub fn is_empty(&self) -> bool {
163 self.inner.is_empty()
164 }
165}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700166
Árpád Goretity4f74b682018-07-14 00:47:51 +0200167/// `TokenStream::default()` returns an empty stream,
168/// i.e. this is equivalent with `TokenStream::new()`.
169impl Default for TokenStream {
170 fn default() -> Self {
171 TokenStream::new()
172 }
173}
174
David Tolnay82ba02d2018-05-20 16:22:43 -0700175/// Attempts to break the string into tokens and parse those tokens into a token
176/// stream.
177///
178/// May fail for a number of reasons, for example, if the string contains
179/// unbalanced delimiters or characters not existing in the language.
180///
181/// NOTE: Some errors may cause panics instead of returning `LexError`. We
182/// reserve the right to change these errors into `LexError`s later.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700183impl FromStr for TokenStream {
184 type Err = LexError;
185
186 fn from_str(src: &str) -> Result<TokenStream, LexError> {
David Tolnayb28f38a2018-03-31 22:02:29 +0200187 let e = src.parse().map_err(|e| LexError {
188 inner: e,
189 _marker: marker::PhantomData,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700190 })?;
191 Ok(TokenStream::_new(e))
Alex Crichton44bffbc2017-05-19 17:51:59 -0700192 }
193}
194
Alex Crichton53548482018-08-11 21:54:05 -0700195#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700196impl From<proc_macro::TokenStream> for TokenStream {
197 fn from(inner: proc_macro::TokenStream) -> TokenStream {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700198 TokenStream::_new(inner.into())
Alex Crichton44bffbc2017-05-19 17:51:59 -0700199 }
200}
201
Alex Crichton53548482018-08-11 21:54:05 -0700202#[cfg(use_proc_macro)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700203impl From<TokenStream> for proc_macro::TokenStream {
204 fn from(inner: TokenStream) -> proc_macro::TokenStream {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700205 inner.inner.into()
Alex Crichton44bffbc2017-05-19 17:51:59 -0700206 }
207}
208
Alex Crichtonf3888432018-05-16 09:11:05 -0700209impl Extend<TokenTree> for TokenStream {
210 fn extend<I: IntoIterator<Item = TokenTree>>(&mut self, streams: I) {
211 self.inner.extend(streams)
212 }
213}
214
David Tolnay5c58c532018-08-13 11:33:51 -0700215impl Extend<TokenStream> for TokenStream {
216 fn extend<I: IntoIterator<Item = TokenStream>>(&mut self, streams: I) {
217 self.inner
218 .extend(streams.into_iter().map(|stream| stream.inner))
219 }
220}
221
David Tolnay82ba02d2018-05-20 16:22:43 -0700222/// Collects a number of token trees into a single stream.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700223impl FromIterator<TokenTree> for TokenStream {
224 fn from_iter<I: IntoIterator<Item = TokenTree>>(streams: I) -> Self {
225 TokenStream::_new(streams.into_iter().collect())
Alex Crichton44bffbc2017-05-19 17:51:59 -0700226 }
227}
Alex Crichton53b00672018-09-06 17:16:10 -0700228impl FromIterator<TokenStream> for TokenStream {
229 fn from_iter<I: IntoIterator<Item = TokenStream>>(streams: I) -> Self {
230 TokenStream::_new(streams.into_iter().map(|i| i.inner).collect())
231 }
232}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700233
David Tolnay82ba02d2018-05-20 16:22:43 -0700234/// Prints the token stream as a string that is supposed to be losslessly
235/// convertible back into the same token stream (modulo spans), except for
236/// possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative
237/// numeric literals.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700238impl fmt::Display for TokenStream {
239 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
240 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700241 }
242}
243
David Tolnay82ba02d2018-05-20 16:22:43 -0700244/// Prints token in a form convenient for debugging.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700245impl fmt::Debug for TokenStream {
246 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
247 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700248 }
249}
250
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700251impl fmt::Debug for LexError {
252 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
253 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -0700254 }
255}
256
David Tolnay82ba02d2018-05-20 16:22:43 -0700257/// The source file of a given `Span`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700258///
259/// This type is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800260#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500261#[derive(Clone, PartialEq, Eq)]
David Tolnay7e654a82018-11-11 13:33:18 -0800262pub struct SourceFile {
263 inner: imp::SourceFile,
264 _marker: marker::PhantomData<Rc<()>>,
265}
Nika Layzellf8d5f212017-12-11 14:07:02 -0500266
David Tolnay1ebe3972018-01-02 20:14:20 -0800267#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500268impl SourceFile {
David Tolnay7e654a82018-11-11 13:33:18 -0800269 fn _new(inner: imp::SourceFile) -> Self {
270 SourceFile {
271 inner: inner,
272 _marker: marker::PhantomData,
273 }
274 }
275
David Tolnay82ba02d2018-05-20 16:22:43 -0700276 /// Get the path to this source file.
277 ///
278 /// ### Note
279 ///
280 /// If the code span associated with this `SourceFile` was generated by an
281 /// external macro, this may not be an actual path on the filesystem. Use
282 /// [`is_real`] to check.
283 ///
284 /// Also note that even if `is_real` returns `true`, if
285 /// `--remap-path-prefix` was passed on the command line, the path as given
286 /// may not actually be valid.
287 ///
288 /// [`is_real`]: #method.is_real
David Tolnay9cd3b4c2018-11-11 16:47:32 -0800289 pub fn path(&self) -> PathBuf {
David Tolnay7e654a82018-11-11 13:33:18 -0800290 self.inner.path()
Nika Layzellf8d5f212017-12-11 14:07:02 -0500291 }
292
David Tolnay82ba02d2018-05-20 16:22:43 -0700293 /// Returns `true` if this source file is a real source file, and not
294 /// generated by an external macro's expansion.
Nika Layzellf8d5f212017-12-11 14:07:02 -0500295 pub fn is_real(&self) -> bool {
David Tolnay7e654a82018-11-11 13:33:18 -0800296 self.inner.is_real()
Nika Layzellf8d5f212017-12-11 14:07:02 -0500297 }
298}
299
David Tolnay1ebe3972018-01-02 20:14:20 -0800300#[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500301impl fmt::Debug for SourceFile {
302 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
David Tolnay7e654a82018-11-11 13:33:18 -0800303 self.inner.fmt(f)
Nika Layzellf8d5f212017-12-11 14:07:02 -0500304 }
305}
306
David Tolnay82ba02d2018-05-20 16:22:43 -0700307/// A line-column pair representing the start or end of a `Span`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700308///
309/// This type is semver exempt and not exposed by default.
David Tolnay3b1f7d22019-01-28 12:22:11 -0800310#[cfg(span_locations)]
David Tolnay9d4fb442019-04-22 16:42:41 -0700311#[derive(Copy, Clone, Debug, PartialEq, Eq)]
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500312pub struct LineColumn {
David Tolnay82ba02d2018-05-20 16:22:43 -0700313 /// The 1-indexed line in the source file on which the span starts or ends
314 /// (inclusive).
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500315 pub line: usize,
David Tolnay82ba02d2018-05-20 16:22:43 -0700316 /// The 0-indexed column (in UTF-8 characters) in the source file on which
317 /// the span starts or ends (inclusive).
Nika Layzell1ecb6ce2017-12-30 14:34:05 -0500318 pub column: usize,
319}
Nika Layzellf8d5f212017-12-11 14:07:02 -0500320
David Tolnay82ba02d2018-05-20 16:22:43 -0700321/// A region of source code, along with macro expansion information.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700322#[derive(Copy, Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700323pub struct Span {
324 inner: imp::Span,
325 _marker: marker::PhantomData<Rc<()>>,
326}
Alex Crichton44bffbc2017-05-19 17:51:59 -0700327
Alex Crichton44bffbc2017-05-19 17:51:59 -0700328impl Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700329 fn _new(inner: imp::Span) -> Span {
330 Span {
331 inner: inner,
332 _marker: marker::PhantomData,
333 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700334 }
Alex Crichtone6085b72017-11-21 07:24:25 -0800335
David Tolnayaef075b2019-01-16 16:29:18 -0800336 fn _new_stable(inner: fallback::Span) -> Span {
Alex Crichton30a4e9e2018-04-27 17:02:19 -0700337 Span {
338 inner: inner.into(),
339 _marker: marker::PhantomData,
340 }
341 }
342
David Tolnay82ba02d2018-05-20 16:22:43 -0700343 /// The span of the invocation of the current procedural macro.
344 ///
345 /// Identifiers created with this span will be resolved as if they were
346 /// written directly at the macro call location (call-site hygiene) and
347 /// other code at the macro call site will be able to refer to them as well.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700348 pub fn call_site() -> Span {
349 Span::_new(imp::Span::call_site())
350 }
351
David Tolnay82ba02d2018-05-20 16:22:43 -0700352 /// A span that resolves at the macro definition site.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700353 ///
354 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700355 #[cfg(procmacro2_semver_exempt)]
Alex Crichtone6085b72017-11-21 07:24:25 -0800356 pub fn def_site() -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700357 Span::_new(imp::Span::def_site())
Alex Crichtone6085b72017-11-21 07:24:25 -0800358 }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500359
David Tolnay4e8e3972018-01-05 18:10:22 -0800360 /// Creates a new span with the same line/column information as `self` but
361 /// that resolves symbols as though it were at `other`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700362 ///
363 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700364 #[cfg(procmacro2_semver_exempt)]
David Tolnay4e8e3972018-01-05 18:10:22 -0800365 pub fn resolved_at(&self, other: Span) -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700366 Span::_new(self.inner.resolved_at(other.inner))
David Tolnay4e8e3972018-01-05 18:10:22 -0800367 }
368
369 /// Creates a new span with the same name resolution behavior as `self` but
370 /// with the line/column information of `other`.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700371 ///
372 /// This method is semver exempt and not exposed by default.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700373 #[cfg(procmacro2_semver_exempt)]
David Tolnay4e8e3972018-01-05 18:10:22 -0800374 pub fn located_at(&self, other: Span) -> Span {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700375 Span::_new(self.inner.located_at(other.inner))
David Tolnay4e8e3972018-01-05 18:10:22 -0800376 }
377
David Tolnay17eb0702019-01-05 12:23:17 -0800378 /// Convert `proc_macro2::Span` to `proc_macro::Span`.
379 ///
380 /// This method is available when building with a nightly compiler, or when
381 /// building with rustc 1.29+ *without* semver exempt features.
David Tolnayc0425cd2019-01-16 12:13:15 -0800382 ///
383 /// # Panics
384 ///
385 /// Panics if called from outside of a procedural macro. Unlike
386 /// `proc_macro2::Span`, the `proc_macro::Span` type can only exist within
387 /// the context of a procedural macro invocation.
David Tolnay17eb0702019-01-05 12:23:17 -0800388 #[cfg(wrap_proc_macro)]
David Tolnay40bbb1c2019-01-19 19:43:55 -0800389 pub fn unwrap(self) -> proc_macro::Span {
390 self.inner.unwrap()
391 }
392
393 // Soft deprecated. Please use Span::unwrap.
394 #[cfg(wrap_proc_macro)]
395 #[doc(hidden)]
David Tolnay16a17202017-12-31 10:47:24 -0500396 pub fn unstable(self) -> proc_macro::Span {
David Tolnay40bbb1c2019-01-19 19:43:55 -0800397 self.unwrap()
David Tolnay16a17202017-12-31 10:47:24 -0500398 }
399
David Tolnay82ba02d2018-05-20 16:22:43 -0700400 /// The original source file into which this span points.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700401 ///
402 /// This method is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800403 #[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500404 pub fn source_file(&self) -> SourceFile {
David Tolnay7e654a82018-11-11 13:33:18 -0800405 SourceFile::_new(self.inner.source_file())
Nika Layzellf8d5f212017-12-11 14:07:02 -0500406 }
407
David Tolnay82ba02d2018-05-20 16:22:43 -0700408 /// Get the starting line/column in the source file for this span.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700409 ///
David Tolnay3b1f7d22019-01-28 12:22:11 -0800410 /// This method requires the `"span-locations"` feature to be enabled.
411 #[cfg(span_locations)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500412 pub fn start(&self) -> LineColumn {
David Tolnayb28f38a2018-03-31 22:02:29 +0200413 let imp::LineColumn { line, column } = self.inner.start();
414 LineColumn {
415 line: line,
416 column: column,
417 }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500418 }
419
David Tolnay82ba02d2018-05-20 16:22:43 -0700420 /// Get the ending line/column in the source file for this span.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700421 ///
David Tolnay3b1f7d22019-01-28 12:22:11 -0800422 /// This method requires the `"span-locations"` feature to be enabled.
423 #[cfg(span_locations)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500424 pub fn end(&self) -> LineColumn {
David Tolnayb28f38a2018-03-31 22:02:29 +0200425 let imp::LineColumn { line, column } = self.inner.end();
426 LineColumn {
427 line: line,
428 column: column,
429 }
Nika Layzellf8d5f212017-12-11 14:07:02 -0500430 }
431
David Tolnay82ba02d2018-05-20 16:22:43 -0700432 /// Create a new span encompassing `self` and `other`.
433 ///
434 /// Returns `None` if `self` and `other` are from different files.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700435 ///
436 /// This method is semver exempt and not exposed by default.
David Tolnay1ebe3972018-01-02 20:14:20 -0800437 #[cfg(procmacro2_semver_exempt)]
Nika Layzellf8d5f212017-12-11 14:07:02 -0500438 pub fn join(&self, other: Span) -> Option<Span> {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700439 self.inner.join(other.inner).map(Span::_new)
440 }
Alex Crichtonb2c94622018-04-04 07:36:41 -0700441
David Tolnay0360cf82019-06-30 13:09:49 -0700442 /// Compares two spans to see if they're equal.
David Tolnaya01ca8e2018-06-04 00:55:28 -0700443 ///
444 /// This method is semver exempt and not exposed by default.
Alex Crichtonb2c94622018-04-04 07:36:41 -0700445 #[cfg(procmacro2_semver_exempt)]
446 pub fn eq(&self, other: &Span) -> bool {
447 self.inner.eq(&other.inner)
448 }
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700449}
450
David Tolnay82ba02d2018-05-20 16:22:43 -0700451/// Prints a span in a form convenient for debugging.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700452impl fmt::Debug for Span {
453 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
454 self.inner.fmt(f)
Nika Layzellf8d5f212017-12-11 14:07:02 -0500455 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700456}
457
David Tolnay82ba02d2018-05-20 16:22:43 -0700458/// A single token or a delimited sequence of token trees (e.g. `[1, (), ..]`).
David Tolnay034205f2018-04-22 16:45:28 -0700459#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700460pub enum TokenTree {
David Tolnay82ba02d2018-05-20 16:22:43 -0700461 /// A token stream surrounded by bracket delimiters.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700462 Group(Group),
David Tolnay82ba02d2018-05-20 16:22:43 -0700463 /// An identifier.
Alex Crichtonf3888432018-05-16 09:11:05 -0700464 Ident(Ident),
David Tolnay82ba02d2018-05-20 16:22:43 -0700465 /// A single punctuation character (`+`, `,`, `$`, etc.).
Alex Crichtonf3888432018-05-16 09:11:05 -0700466 Punct(Punct),
David Tolnay82ba02d2018-05-20 16:22:43 -0700467 /// A literal character (`'a'`), string (`"hello"`), number (`2.3`), etc.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700468 Literal(Literal),
Alex Crichton1a7f7622017-07-05 17:47:15 -0700469}
470
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700471impl TokenTree {
David Tolnay82ba02d2018-05-20 16:22:43 -0700472 /// Returns the span of this tree, delegating to the `span` method of
473 /// the contained token or a delimited stream.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700474 pub fn span(&self) -> Span {
475 match *self {
476 TokenTree::Group(ref t) => t.span(),
Alex Crichtonf3888432018-05-16 09:11:05 -0700477 TokenTree::Ident(ref t) => t.span(),
478 TokenTree::Punct(ref t) => t.span(),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700479 TokenTree::Literal(ref t) => t.span(),
480 }
481 }
482
David Tolnay82ba02d2018-05-20 16:22:43 -0700483 /// Configures the span for *only this token*.
484 ///
485 /// Note that if this token is a `Group` then this method will not configure
486 /// the span of each of the internal tokens, this will simply delegate to
487 /// the `set_span` method of each variant.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700488 pub fn set_span(&mut self, span: Span) {
489 match *self {
490 TokenTree::Group(ref mut t) => t.set_span(span),
Alex Crichtonf3888432018-05-16 09:11:05 -0700491 TokenTree::Ident(ref mut t) => t.set_span(span),
492 TokenTree::Punct(ref mut t) => t.set_span(span),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700493 TokenTree::Literal(ref mut t) => t.set_span(span),
494 }
495 }
496}
497
498impl From<Group> for TokenTree {
499 fn from(g: Group) -> TokenTree {
500 TokenTree::Group(g)
501 }
502}
503
Alex Crichtonf3888432018-05-16 09:11:05 -0700504impl From<Ident> for TokenTree {
505 fn from(g: Ident) -> TokenTree {
506 TokenTree::Ident(g)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700507 }
508}
509
Alex Crichtonf3888432018-05-16 09:11:05 -0700510impl From<Punct> for TokenTree {
511 fn from(g: Punct) -> TokenTree {
512 TokenTree::Punct(g)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700513 }
514}
515
516impl From<Literal> for TokenTree {
517 fn from(g: Literal) -> TokenTree {
518 TokenTree::Literal(g)
Alex Crichton1a7f7622017-07-05 17:47:15 -0700519 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700520}
521
David Tolnay82ba02d2018-05-20 16:22:43 -0700522/// Prints the token tree as a string that is supposed to be losslessly
523/// convertible back into the same token tree (modulo spans), except for
524/// possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative
525/// numeric literals.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700526impl fmt::Display for TokenTree {
527 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700528 match *self {
529 TokenTree::Group(ref t) => t.fmt(f),
Alex Crichtonf3888432018-05-16 09:11:05 -0700530 TokenTree::Ident(ref t) => t.fmt(f),
531 TokenTree::Punct(ref t) => t.fmt(f),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700532 TokenTree::Literal(ref t) => t.fmt(f),
533 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700534 }
535}
536
David Tolnay82ba02d2018-05-20 16:22:43 -0700537/// Prints token tree in a form convenient for debugging.
David Tolnay034205f2018-04-22 16:45:28 -0700538impl fmt::Debug for TokenTree {
539 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
540 // Each of these has the name in the struct type in the derived debug,
541 // so don't bother with an extra layer of indirection
542 match *self {
543 TokenTree::Group(ref t) => t.fmt(f),
David Tolnayd8fcdb82018-06-02 15:43:53 -0700544 TokenTree::Ident(ref t) => {
545 let mut debug = f.debug_struct("Ident");
546 debug.field("sym", &format_args!("{}", t));
David Tolnayfd8cdc82019-01-19 19:23:59 -0800547 imp::debug_span_field_if_nontrivial(&mut debug, t.span().inner);
David Tolnayd8fcdb82018-06-02 15:43:53 -0700548 debug.finish()
549 }
Alex Crichtonf3888432018-05-16 09:11:05 -0700550 TokenTree::Punct(ref t) => t.fmt(f),
David Tolnay034205f2018-04-22 16:45:28 -0700551 TokenTree::Literal(ref t) => t.fmt(f),
552 }
553 }
554}
555
David Tolnay82ba02d2018-05-20 16:22:43 -0700556/// A delimited token stream.
557///
558/// A `Group` internally contains a `TokenStream` which is surrounded by
559/// `Delimiter`s.
David Tolnay034205f2018-04-22 16:45:28 -0700560#[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700561pub struct Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700562 inner: imp::Group,
Alex Crichton44bffbc2017-05-19 17:51:59 -0700563}
564
David Tolnay82ba02d2018-05-20 16:22:43 -0700565/// Describes how a sequence of token trees is delimited.
Michael Layzell5372f4b2017-06-02 10:29:31 -0400566#[derive(Copy, Clone, Debug, Eq, PartialEq)]
Alex Crichton44bffbc2017-05-19 17:51:59 -0700567pub enum Delimiter {
David Tolnay82ba02d2018-05-20 16:22:43 -0700568 /// `( ... )`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700569 Parenthesis,
David Tolnay82ba02d2018-05-20 16:22:43 -0700570 /// `{ ... }`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700571 Brace,
David Tolnay82ba02d2018-05-20 16:22:43 -0700572 /// `[ ... ]`
Alex Crichton44bffbc2017-05-19 17:51:59 -0700573 Bracket,
David Tolnay82ba02d2018-05-20 16:22:43 -0700574 /// `Ø ... Ø`
575 ///
576 /// An implicit delimiter, that may, for example, appear around tokens
577 /// coming from a "macro variable" `$var`. It is important to preserve
578 /// operator priorities in cases like `$var * 3` where `$var` is `1 + 2`.
579 /// Implicit delimiters may not survive roundtrip of a token stream through
580 /// a string.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700581 None,
582}
583
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700584impl Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700585 fn _new(inner: imp::Group) -> Self {
David Tolnay4453fcc2019-01-16 12:15:01 -0800586 Group { inner: inner }
David Tolnayf14813f2018-09-08 17:14:07 -0700587 }
588
David Tolnayaef075b2019-01-16 16:29:18 -0800589 fn _new_stable(inner: fallback::Group) -> Self {
David Tolnayf14813f2018-09-08 17:14:07 -0700590 Group {
591 inner: inner.into(),
592 }
593 }
594
David Tolnay82ba02d2018-05-20 16:22:43 -0700595 /// Creates a new `Group` with the given delimiter and token stream.
596 ///
597 /// This constructor will set the span for this group to
598 /// `Span::call_site()`. To change the span you can use the `set_span`
599 /// method below.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700600 pub fn new(delimiter: Delimiter, stream: TokenStream) -> Group {
601 Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700602 inner: imp::Group::new(delimiter, stream.inner),
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700603 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700604 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700605
David Tolnay82ba02d2018-05-20 16:22:43 -0700606 /// Returns the delimiter of this `Group`
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700607 pub fn delimiter(&self) -> Delimiter {
David Tolnayf14813f2018-09-08 17:14:07 -0700608 self.inner.delimiter()
Alex Crichton44bffbc2017-05-19 17:51:59 -0700609 }
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700610
David Tolnay82ba02d2018-05-20 16:22:43 -0700611 /// Returns the `TokenStream` of tokens that are delimited in this `Group`.
612 ///
613 /// Note that the returned token stream does not include the delimiter
614 /// returned above.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700615 pub fn stream(&self) -> TokenStream {
David Tolnayf14813f2018-09-08 17:14:07 -0700616 TokenStream::_new(self.inner.stream())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700617 }
618
David Tolnay82ba02d2018-05-20 16:22:43 -0700619 /// Returns the span for the delimiters of this token stream, spanning the
620 /// entire `Group`.
David Tolnayf14813f2018-09-08 17:14:07 -0700621 ///
622 /// ```text
623 /// pub fn span(&self) -> Span {
624 /// ^^^^^^^
625 /// ```
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700626 pub fn span(&self) -> Span {
David Tolnayf14813f2018-09-08 17:14:07 -0700627 Span::_new(self.inner.span())
628 }
629
630 /// Returns the span pointing to the opening delimiter of this group.
631 ///
632 /// ```text
633 /// pub fn span_open(&self) -> Span {
634 /// ^
635 /// ```
636 #[cfg(procmacro2_semver_exempt)]
637 pub fn span_open(&self) -> Span {
638 Span::_new(self.inner.span_open())
639 }
640
641 /// Returns the span pointing to the closing delimiter of this group.
642 ///
643 /// ```text
644 /// pub fn span_close(&self) -> Span {
645 /// ^
646 /// ```
647 #[cfg(procmacro2_semver_exempt)]
648 pub fn span_close(&self) -> Span {
649 Span::_new(self.inner.span_close())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700650 }
651
David Tolnay82ba02d2018-05-20 16:22:43 -0700652 /// Configures the span for this `Group`'s delimiters, but not its internal
653 /// tokens.
654 ///
655 /// This method will **not** set the span of all the internal tokens spanned
656 /// by this group, but rather it will only set the span of the delimiter
657 /// tokens at the level of the `Group`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700658 pub fn set_span(&mut self, span: Span) {
David Tolnayf14813f2018-09-08 17:14:07 -0700659 self.inner.set_span(span.inner)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700660 }
661}
662
David Tolnay82ba02d2018-05-20 16:22:43 -0700663/// Prints the group as a string that should be losslessly convertible back
664/// into the same group (modulo spans), except for possibly `TokenTree::Group`s
665/// with `Delimiter::None` delimiters.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700666impl fmt::Display for Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700667 fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
668 fmt::Display::fmt(&self.inner, formatter)
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700669 }
670}
671
David Tolnay034205f2018-04-22 16:45:28 -0700672impl fmt::Debug for Group {
David Tolnayf14813f2018-09-08 17:14:07 -0700673 fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
674 fmt::Debug::fmt(&self.inner, formatter)
David Tolnay034205f2018-04-22 16:45:28 -0700675 }
676}
677
David Tolnay82ba02d2018-05-20 16:22:43 -0700678/// An `Punct` is an single punctuation character like `+`, `-` or `#`.
679///
680/// Multicharacter operators like `+=` are represented as two instances of
681/// `Punct` with different forms of `Spacing` returned.
Alex Crichtonf3888432018-05-16 09:11:05 -0700682#[derive(Clone)]
683pub struct Punct {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700684 op: char,
685 spacing: Spacing,
686 span: Span,
Alex Crichton44bffbc2017-05-19 17:51:59 -0700687}
688
David Tolnay82ba02d2018-05-20 16:22:43 -0700689/// Whether an `Punct` is followed immediately by another `Punct` or followed by
690/// another token or whitespace.
Lukas Kalbertodteb3f9302017-08-20 18:58:41 +0200691#[derive(Copy, Clone, Debug, Eq, PartialEq)]
Alex Crichton1a7f7622017-07-05 17:47:15 -0700692pub enum Spacing {
David Tolnay82ba02d2018-05-20 16:22:43 -0700693 /// E.g. `+` is `Alone` in `+ =`, `+ident` or `+()`.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700694 Alone,
David Tolnay82ba02d2018-05-20 16:22:43 -0700695 /// E.g. `+` is `Joint` in `+=` or `'#`.
696 ///
697 /// Additionally, single quote `'` can join with identifiers to form
698 /// lifetimes `'ident`.
Alex Crichton44bffbc2017-05-19 17:51:59 -0700699 Joint,
700}
701
Alex Crichtonf3888432018-05-16 09:11:05 -0700702impl Punct {
David Tolnay82ba02d2018-05-20 16:22:43 -0700703 /// Creates a new `Punct` from the given character and spacing.
704 ///
705 /// The `ch` argument must be a valid punctuation character permitted by the
706 /// language, otherwise the function will panic.
707 ///
708 /// The returned `Punct` will have the default span of `Span::call_site()`
709 /// which can be further configured with the `set_span` method below.
Alex Crichtonf3888432018-05-16 09:11:05 -0700710 pub fn new(op: char, spacing: Spacing) -> Punct {
711 Punct {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700712 op: op,
713 spacing: spacing,
714 span: Span::call_site(),
715 }
716 }
Alex Crichton44bffbc2017-05-19 17:51:59 -0700717
David Tolnay82ba02d2018-05-20 16:22:43 -0700718 /// Returns the value of this punctuation character as `char`.
Alex Crichtonf3888432018-05-16 09:11:05 -0700719 pub fn as_char(&self) -> char {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700720 self.op
721 }
722
David Tolnay82ba02d2018-05-20 16:22:43 -0700723 /// Returns the spacing of this punctuation character, indicating whether
724 /// it's immediately followed by another `Punct` in the token stream, so
725 /// they can potentially be combined into a multicharacter operator
726 /// (`Joint`), or it's followed by some other token or whitespace (`Alone`)
727 /// so the operator has certainly ended.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700728 pub fn spacing(&self) -> Spacing {
729 self.spacing
730 }
731
David Tolnay82ba02d2018-05-20 16:22:43 -0700732 /// Returns the span for this punctuation character.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700733 pub fn span(&self) -> Span {
734 self.span
735 }
736
David Tolnay82ba02d2018-05-20 16:22:43 -0700737 /// Configure the span for this punctuation character.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700738 pub fn set_span(&mut self, span: Span) {
739 self.span = span;
740 }
741}
742
David Tolnay82ba02d2018-05-20 16:22:43 -0700743/// Prints the punctuation character as a string that should be losslessly
744/// convertible back into the same character.
Alex Crichtonf3888432018-05-16 09:11:05 -0700745impl fmt::Display for Punct {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700746 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
747 self.op.fmt(f)
748 }
749}
750
Alex Crichtonf3888432018-05-16 09:11:05 -0700751impl fmt::Debug for Punct {
David Tolnay034205f2018-04-22 16:45:28 -0700752 fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
Alex Crichtonf3888432018-05-16 09:11:05 -0700753 let mut debug = fmt.debug_struct("Punct");
David Tolnay034205f2018-04-22 16:45:28 -0700754 debug.field("op", &self.op);
755 debug.field("spacing", &self.spacing);
David Tolnayfd8cdc82019-01-19 19:23:59 -0800756 imp::debug_span_field_if_nontrivial(&mut debug, self.span.inner);
David Tolnay034205f2018-04-22 16:45:28 -0700757 debug.finish()
758 }
759}
760
David Tolnay8b71dac2018-05-20 17:07:47 -0700761/// A word of Rust code, which may be a keyword or legal variable name.
762///
763/// An identifier consists of at least one Unicode code point, the first of
764/// which has the XID_Start property and the rest of which have the XID_Continue
765/// property.
766///
767/// - The empty string is not an identifier. Use `Option<Ident>`.
768/// - A lifetime is not an identifier. Use `syn::Lifetime` instead.
769///
770/// An identifier constructed with `Ident::new` is permitted to be a Rust
David Tolnayc239f032018-11-11 12:57:09 -0800771/// keyword, though parsing one through its [`Parse`] implementation rejects
772/// Rust keywords. Use `input.call(Ident::parse_any)` when parsing to match the
David Tolnay8b71dac2018-05-20 17:07:47 -0700773/// behaviour of `Ident::new`.
774///
David Tolnayc239f032018-11-11 12:57:09 -0800775/// [`Parse`]: https://docs.rs/syn/0.15/syn/parse/trait.Parse.html
David Tolnay8b71dac2018-05-20 17:07:47 -0700776///
777/// # Examples
778///
779/// A new ident can be created from a string using the `Ident::new` function.
780/// A span must be provided explicitly which governs the name resolution
781/// behavior of the resulting identifier.
782///
David Tolnay7a964732019-01-19 19:03:20 -0800783/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700784/// use proc_macro2::{Ident, Span};
785///
786/// fn main() {
787/// let call_ident = Ident::new("calligraphy", Span::call_site());
788///
789/// println!("{}", call_ident);
790/// }
791/// ```
792///
793/// An ident can be interpolated into a token stream using the `quote!` macro.
794///
David Tolnay7a964732019-01-19 19:03:20 -0800795/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700796/// use proc_macro2::{Ident, Span};
David Tolnay7a964732019-01-19 19:03:20 -0800797/// use quote::quote;
David Tolnay8b71dac2018-05-20 17:07:47 -0700798///
799/// fn main() {
800/// let ident = Ident::new("demo", Span::call_site());
801///
802/// // Create a variable binding whose name is this ident.
803/// let expanded = quote! { let #ident = 10; };
804///
805/// // Create a variable binding with a slightly different name.
806/// let temp_ident = Ident::new(&format!("new_{}", ident), Span::call_site());
807/// let expanded = quote! { let #temp_ident = 10; };
808/// }
809/// ```
810///
811/// A string representation of the ident is available through the `to_string()`
812/// method.
813///
David Tolnay7a964732019-01-19 19:03:20 -0800814/// ```edition2018
David Tolnay8b71dac2018-05-20 17:07:47 -0700815/// # use proc_macro2::{Ident, Span};
816/// #
817/// # let ident = Ident::new("another_identifier", Span::call_site());
818/// #
819/// // Examine the ident as a string.
820/// let ident_string = ident.to_string();
821/// if ident_string.len() > 60 {
822/// println!("Very long identifier: {}", ident_string)
823/// }
824/// ```
Alex Crichtonf3888432018-05-16 09:11:05 -0700825#[derive(Clone)]
826pub struct Ident {
827 inner: imp::Ident,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700828 _marker: marker::PhantomData<Rc<()>>,
829}
830
Alex Crichtonf3888432018-05-16 09:11:05 -0700831impl Ident {
832 fn _new(inner: imp::Ident) -> Ident {
833 Ident {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700834 inner: inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700835 _marker: marker::PhantomData,
836 }
837 }
838
David Tolnay82ba02d2018-05-20 16:22:43 -0700839 /// Creates a new `Ident` with the given `string` as well as the specified
840 /// `span`.
841 ///
842 /// The `string` argument must be a valid identifier permitted by the
843 /// language, otherwise the function will panic.
844 ///
845 /// Note that `span`, currently in rustc, configures the hygiene information
846 /// for this identifier.
847 ///
848 /// As of this time `Span::call_site()` explicitly opts-in to "call-site"
849 /// hygiene meaning that identifiers created with this span will be resolved
850 /// as if they were written directly at the location of the macro call, and
851 /// other code at the macro call site will be able to refer to them as well.
852 ///
853 /// Later spans like `Span::def_site()` will allow to opt-in to
854 /// "definition-site" hygiene meaning that identifiers created with this
855 /// span will be resolved at the location of the macro definition and other
856 /// code at the macro call site will not be able to refer to them.
857 ///
858 /// Due to the current importance of hygiene this constructor, unlike other
859 /// tokens, requires a `Span` to be specified at construction.
David Tolnay8b71dac2018-05-20 17:07:47 -0700860 ///
861 /// # Panics
862 ///
863 /// Panics if the input string is neither a keyword nor a legal variable
David Tolnay219b1d32019-04-22 16:04:11 -0700864 /// name. If you are not sure whether the string contains an identifier and
865 /// need to handle an error case, use
866 /// <a href="https://docs.rs/syn/0.15/syn/fn.parse_str.html"><code
867 /// style="padding-right:0;">syn::parse_str</code></a><code
868 /// style="padding-left:0;">::&lt;Ident&gt;</code>
869 /// rather than `Ident::new`.
Alex Crichtonf3888432018-05-16 09:11:05 -0700870 pub fn new(string: &str, span: Span) -> Ident {
871 Ident::_new(imp::Ident::new(string, span.inner))
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700872 }
873
David Tolnay82ba02d2018-05-20 16:22:43 -0700874 /// Same as `Ident::new`, but creates a raw identifier (`r#ident`).
David Tolnaya01ca8e2018-06-04 00:55:28 -0700875 ///
876 /// This method is semver exempt and not exposed by default.
Alex Crichtonf3888432018-05-16 09:11:05 -0700877 #[cfg(procmacro2_semver_exempt)]
878 pub fn new_raw(string: &str, span: Span) -> Ident {
879 Ident::_new_raw(string, span)
880 }
881
882 fn _new_raw(string: &str, span: Span) -> Ident {
883 Ident::_new(imp::Ident::new_raw(string, span.inner))
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700884 }
885
David Tolnay82ba02d2018-05-20 16:22:43 -0700886 /// Returns the span of this `Ident`.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700887 pub fn span(&self) -> Span {
Alex Crichtonb2c94622018-04-04 07:36:41 -0700888 Span::_new(self.inner.span())
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700889 }
890
David Tolnay82ba02d2018-05-20 16:22:43 -0700891 /// Configures the span of this `Ident`, possibly changing its hygiene
892 /// context.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700893 pub fn set_span(&mut self, span: Span) {
Alex Crichtonb2c94622018-04-04 07:36:41 -0700894 self.inner.set_span(span.inner);
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700895 }
896}
897
Alex Crichtonf3888432018-05-16 09:11:05 -0700898impl PartialEq for Ident {
David Tolnay3d9d6ad2018-05-18 10:51:55 -0700899 fn eq(&self, other: &Ident) -> bool {
David Tolnayc0b0f2e2018-09-02 17:56:08 -0700900 self.inner == other.inner
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700901 }
902}
903
David Tolnayc0bbcc52018-05-18 10:51:04 -0700904impl<T> PartialEq<T> for Ident
905where
906 T: ?Sized + AsRef<str>,
907{
908 fn eq(&self, other: &T) -> bool {
David Tolnayc0b0f2e2018-09-02 17:56:08 -0700909 self.inner == other
David Tolnayc0bbcc52018-05-18 10:51:04 -0700910 }
911}
912
David Tolnay3d9d6ad2018-05-18 10:51:55 -0700913impl Eq for Ident {}
Alex Crichtonf3888432018-05-16 09:11:05 -0700914
915impl PartialOrd for Ident {
916 fn partial_cmp(&self, other: &Ident) -> Option<Ordering> {
917 Some(self.cmp(other))
918 }
919}
920
921impl Ord for Ident {
922 fn cmp(&self, other: &Ident) -> Ordering {
923 self.to_string().cmp(&other.to_string())
924 }
925}
926
927impl Hash for Ident {
928 fn hash<H: Hasher>(&self, hasher: &mut H) {
929 self.to_string().hash(hasher)
930 }
931}
932
David Tolnay82ba02d2018-05-20 16:22:43 -0700933/// Prints the identifier as a string that should be losslessly convertible back
934/// into the same identifier.
Alex Crichtonf3888432018-05-16 09:11:05 -0700935impl fmt::Display for Ident {
936 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
937 self.inner.fmt(f)
938 }
939}
940
941impl fmt::Debug for Ident {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700942 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
943 self.inner.fmt(f)
944 }
945}
946
David Tolnay82ba02d2018-05-20 16:22:43 -0700947/// A literal string (`"hello"`), byte string (`b"hello"`), character (`'a'`),
948/// byte character (`b'a'`), an integer or floating point number with or without
949/// a suffix (`1`, `1u8`, `2.3`, `2.3f32`).
950///
951/// Boolean literals like `true` and `false` do not belong here, they are
952/// `Ident`s.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700953#[derive(Clone)]
954pub struct Literal {
955 inner: imp::Literal,
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700956 _marker: marker::PhantomData<Rc<()>>,
957}
958
David Tolnay82ba02d2018-05-20 16:22:43 -0700959macro_rules! suffixed_int_literals {
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700960 ($($name:ident => $kind:ident,)*) => ($(
David Tolnay82ba02d2018-05-20 16:22:43 -0700961 /// Creates a new suffixed integer literal with the specified value.
962 ///
963 /// This function will create an integer like `1u32` where the integer
964 /// value specified is the first part of the token and the integral is
965 /// also suffixed at the end. Literals created from negative numbers may
966 /// not survive rountrips through `TokenStream` or strings and may be
967 /// broken into two tokens (`-` and positive literal).
968 ///
969 /// Literals created through this method have the `Span::call_site()`
970 /// span by default, which can be configured with the `set_span` method
971 /// below.
972 pub fn $name(n: $kind) -> Literal {
973 Literal::_new(imp::Literal::$name(n))
974 }
975 )*)
976}
977
978macro_rules! unsuffixed_int_literals {
979 ($($name:ident => $kind:ident,)*) => ($(
980 /// Creates a new unsuffixed integer literal with the specified value.
981 ///
982 /// This function will create an integer like `1` where the integer
983 /// value specified is the first part of the token. No suffix is
984 /// specified on this token, meaning that invocations like
985 /// `Literal::i8_unsuffixed(1)` are equivalent to
986 /// `Literal::u32_unsuffixed(1)`. Literals created from negative numbers
987 /// may not survive rountrips through `TokenStream` or strings and may
988 /// be broken into two tokens (`-` and positive literal).
989 ///
990 /// Literals created through this method have the `Span::call_site()`
991 /// span by default, which can be configured with the `set_span` method
992 /// below.
Alex Crichtonaf5bad42018-03-27 14:45:10 -0700993 pub fn $name(n: $kind) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -0700994 Literal::_new(imp::Literal::$name(n))
Alex Crichton1a7f7622017-07-05 17:47:15 -0700995 }
996 )*)
997}
998
Alex Crichton852d53d2017-05-19 19:25:08 -0700999impl Literal {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001000 fn _new(inner: imp::Literal) -> Literal {
1001 Literal {
1002 inner: inner,
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001003 _marker: marker::PhantomData,
1004 }
Alex Crichton1a7f7622017-07-05 17:47:15 -07001005 }
1006
David Tolnayaef075b2019-01-16 16:29:18 -08001007 fn _new_stable(inner: fallback::Literal) -> Literal {
Alex Crichton30a4e9e2018-04-27 17:02:19 -07001008 Literal {
1009 inner: inner.into(),
1010 _marker: marker::PhantomData,
1011 }
1012 }
1013
David Tolnay82ba02d2018-05-20 16:22:43 -07001014 suffixed_int_literals! {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001015 u8_suffixed => u8,
1016 u16_suffixed => u16,
1017 u32_suffixed => u32,
1018 u64_suffixed => u64,
1019 usize_suffixed => usize,
1020 i8_suffixed => i8,
1021 i16_suffixed => i16,
1022 i32_suffixed => i32,
1023 i64_suffixed => i64,
1024 isize_suffixed => isize,
David Tolnay82ba02d2018-05-20 16:22:43 -07001025 }
Alex Crichton1a7f7622017-07-05 17:47:15 -07001026
Alex Crichton69385662018-11-08 06:30:04 -08001027 #[cfg(u128)]
1028 suffixed_int_literals! {
1029 u128_suffixed => u128,
1030 i128_suffixed => i128,
1031 }
1032
David Tolnay82ba02d2018-05-20 16:22:43 -07001033 unsuffixed_int_literals! {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001034 u8_unsuffixed => u8,
1035 u16_unsuffixed => u16,
1036 u32_unsuffixed => u32,
1037 u64_unsuffixed => u64,
1038 usize_unsuffixed => usize,
1039 i8_unsuffixed => i8,
1040 i16_unsuffixed => i16,
1041 i32_unsuffixed => i32,
1042 i64_unsuffixed => i64,
1043 isize_unsuffixed => isize,
Alex Crichton1a7f7622017-07-05 17:47:15 -07001044 }
1045
Alex Crichton69385662018-11-08 06:30:04 -08001046 #[cfg(u128)]
1047 unsuffixed_int_literals! {
1048 u128_unsuffixed => u128,
1049 i128_unsuffixed => i128,
1050 }
1051
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001052 /// Creates a new unsuffixed floating-point literal.
1053 ///
1054 /// This constructor is similar to those like `Literal::i8_unsuffixed` where
1055 /// the float's value is emitted directly into the token but no suffix is
1056 /// used, so it may be inferred to be a `f64` later in the compiler.
1057 /// Literals created from negative numbers may not survive rountrips through
1058 /// `TokenStream` or strings and may be broken into two tokens (`-` and
1059 /// positive literal).
1060 ///
1061 /// # Panics
1062 ///
1063 /// This function requires that the specified float is finite, for example
1064 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001065 pub fn f64_unsuffixed(f: f64) -> Literal {
1066 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001067 Literal::_new(imp::Literal::f64_unsuffixed(f))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001068 }
1069
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001070 /// Creates a new suffixed floating-point literal.
1071 ///
1072 /// This constructor will create a literal like `1.0f64` where the value
1073 /// specified is the preceding part of the token and `f64` is the suffix of
1074 /// the token. This token will always be inferred to be an `f64` in the
1075 /// compiler. Literals created from negative numbers may not survive
1076 /// rountrips through `TokenStream` or strings and may be broken into two
1077 /// tokens (`-` and positive literal).
1078 ///
1079 /// # Panics
1080 ///
1081 /// This function requires that the specified float is finite, for example
1082 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001083 pub fn f64_suffixed(f: f64) -> Literal {
1084 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001085 Literal::_new(imp::Literal::f64_suffixed(f))
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001086 }
1087
David Tolnay82ba02d2018-05-20 16:22:43 -07001088 /// Creates a new unsuffixed floating-point literal.
1089 ///
1090 /// This constructor is similar to those like `Literal::i8_unsuffixed` where
1091 /// the float's value is emitted directly into the token but no suffix is
1092 /// used, so it may be inferred to be a `f64` later in the compiler.
1093 /// Literals created from negative numbers may not survive rountrips through
1094 /// `TokenStream` or strings and may be broken into two tokens (`-` and
1095 /// positive literal).
1096 ///
1097 /// # Panics
1098 ///
1099 /// This function requires that the specified float is finite, for example
1100 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001101 pub fn f32_unsuffixed(f: f32) -> Literal {
1102 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001103 Literal::_new(imp::Literal::f32_unsuffixed(f))
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001104 }
1105
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001106 /// Creates a new suffixed floating-point literal.
1107 ///
1108 /// This constructor will create a literal like `1.0f32` where the value
1109 /// specified is the preceding part of the token and `f32` is the suffix of
1110 /// the token. This token will always be inferred to be an `f32` in the
1111 /// compiler. Literals created from negative numbers may not survive
1112 /// rountrips through `TokenStream` or strings and may be broken into two
1113 /// tokens (`-` and positive literal).
1114 ///
1115 /// # Panics
1116 ///
1117 /// This function requires that the specified float is finite, for example
1118 /// if it is infinity or NaN this function will panic.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001119 pub fn f32_suffixed(f: f32) -> Literal {
1120 assert!(f.is_finite());
Alex Crichtona914a612018-04-04 07:48:44 -07001121 Literal::_new(imp::Literal::f32_suffixed(f))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001122 }
1123
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001124 /// String literal.
Alex Crichton1a7f7622017-07-05 17:47:15 -07001125 pub fn string(string: &str) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -07001126 Literal::_new(imp::Literal::string(string))
Alex Crichton1a7f7622017-07-05 17:47:15 -07001127 }
1128
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001129 /// Character literal.
Alex Crichton1a7f7622017-07-05 17:47:15 -07001130 pub fn character(ch: char) -> Literal {
Alex Crichtona914a612018-04-04 07:48:44 -07001131 Literal::_new(imp::Literal::character(ch))
Alex Crichton76a5cc82017-05-23 07:01:44 -07001132 }
1133
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001134 /// Byte string literal.
Alex Crichton9c2fb0a2017-05-26 08:49:31 -07001135 pub fn byte_string(s: &[u8]) -> Literal {
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001136 Literal::_new(imp::Literal::byte_string(s))
Alex Crichton852d53d2017-05-19 19:25:08 -07001137 }
Alex Crichton76a5cc82017-05-23 07:01:44 -07001138
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001139 /// Returns the span encompassing this literal.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001140 pub fn span(&self) -> Span {
Alex Crichtonb2c94622018-04-04 07:36:41 -07001141 Span::_new(self.inner.span())
Alex Crichton1a7f7622017-07-05 17:47:15 -07001142 }
1143
David Tolnaybc3bd3f2019-06-30 13:03:56 -07001144 /// Configures the span associated for this literal.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001145 pub fn set_span(&mut self, span: Span) {
Alex Crichtonb2c94622018-04-04 07:36:41 -07001146 self.inner.set_span(span.inner);
Alex Crichton31316622017-05-26 12:54:47 -07001147 }
Alex Crichton852d53d2017-05-19 19:25:08 -07001148}
1149
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001150impl fmt::Debug for Literal {
1151 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1152 self.inner.fmt(f)
Alex Crichton44bffbc2017-05-19 17:51:59 -07001153 }
1154}
David Tolnaycb1b85f2017-06-03 16:40:35 -07001155
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001156impl fmt::Display for Literal {
1157 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1158 self.inner.fmt(f)
1159 }
1160}
1161
David Tolnay82ba02d2018-05-20 16:22:43 -07001162/// Public implementation details for the `TokenStream` type, such as iterators.
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001163pub mod token_stream {
1164 use std::fmt;
1165 use std::marker;
1166 use std::rc::Rc;
1167
David Tolnay48ea5042018-04-23 19:17:35 -07001168 use imp;
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001169 pub use TokenStream;
David Tolnayb28f38a2018-03-31 22:02:29 +02001170 use TokenTree;
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001171
David Tolnay82ba02d2018-05-20 16:22:43 -07001172 /// An iterator over `TokenStream`'s `TokenTree`s.
1173 ///
1174 /// The iteration is "shallow", e.g. the iterator doesn't recurse into
1175 /// delimited groups, and returns whole groups as token trees.
Isaac van Bakelf6754d32019-05-08 20:21:06 +01001176 #[derive(Clone)]
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001177 pub struct IntoIter {
1178 inner: imp::TokenTreeIter,
1179 _marker: marker::PhantomData<Rc<()>>,
1180 }
1181
Alex Crichtonaf5bad42018-03-27 14:45:10 -07001182 impl Iterator for IntoIter {
1183 type Item = TokenTree;
1184
1185 fn next(&mut self) -> Option<TokenTree> {
1186 self.inner.next()
1187 }
1188 }
1189
1190 impl fmt::Debug for IntoIter {
1191 fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
1192 self.inner.fmt(f)
1193 }
1194 }
1195
1196 impl IntoIterator for TokenStream {
1197 type Item = TokenTree;
1198 type IntoIter = IntoIter;
1199
1200 fn into_iter(self) -> IntoIter {
1201 IntoIter {
1202 inner: self.inner.into_iter(),
1203 _marker: marker::PhantomData,
1204 }
1205 }
1206 }
1207}