blob: 59bc773102260ec428f5a5edd8205567cee2e7f9 [file] [log] [blame]
David Tolnay55535012018-01-05 16:39:23 -08001// Copyright 2018 Syn Developers
2//
3// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
4// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
5// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
6// option. This file may not be copied, modified, or distributed
7// except according to those terms.
8
David Tolnay5e84e972018-01-05 17:51:06 -08009//! Syn is a parsing library for parsing a stream of Rust tokens into a syntax
10//! tree of Rust source code.
11//!
12//! Currently this library is geared toward the [custom derive] use case but
13//! contains some APIs that may be useful for Rust procedural macros more
14//! generally.
15//!
16//! [custom derive]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
17//!
18//! - **Data structures** — Syn provides a complete syntax tree that can
19//! represent any valid Rust source code. The syntax tree is rooted at
20//! [`syn::File`] which represents a full source file, but there are other
21//! entry points that may be useful to procedural macros including
22//! [`syn::Item`], [`syn::Expr`] and [`syn::Type`].
23//!
24//! - **Custom derives** — Of particular interest to custom derives is
25//! [`syn::DeriveInput`] which is any of the three legal input items to a
26//! derive macro. An example below shows using this type in a library that can
27//! derive implementations of a trait of your own.
28//!
29//! - **Parser combinators** — Parsing in Syn is built on a suite of public
30//! parser combinator macros that you can use for parsing any token-based
31//! syntax you dream up within a `functionlike!(...)` procedural macro. Every
32//! syntax tree node defined by Syn is individually parsable and may be used
33//! as a building block for custom syntaxes, or you may do it all yourself
34//! working from the most primitive tokens.
35//!
36//! - **Location information** — Every token parsed by Syn is associated with a
37//! `Span` that tracks line and column information back to the source of that
38//! token. These spans allow a procedural macro to display detailed error
39//! messages pointing to all the right places in the user's code. There is an
40//! example of this below.
41//!
42//! - **Feature flags** — Functionality is aggressively feature gated so your
43//! procedural macros enable only what they need, and do not pay in compile
44//! time for all the rest.
45//!
46//! [`syn::File`]: struct.File.html
47//! [`syn::Item`]: enum.Item.html
48//! [`syn::Expr`]: enum.Expr.html
49//! [`syn::Type`]: enum.Type.html
50//! [`syn::DeriveInput`]: struct.DeriveInput.html
51//!
52//! *Version requirement: Syn supports any compiler version back to Rust's very
53//! first support for procedural macros in Rust 1.15.0. Some features especially
54//! around error reporting are only available in newer compilers or on the
55//! nightly channel.*
56//!
57//! ## Example of a custom derive
58//!
59//! The canonical custom derive using Syn looks like this. We write an ordinary
60//! Rust function tagged with a `proc_macro_derive` attribute and the name of
61//! the trait we are deriving. Any time that derive appears in the user's code,
62//! the Rust compiler passes their data structure as tokens into our macro. We
63//! get to execute arbitrary Rust code to figure out what to do with those
64//! tokens, then hand some tokens back to the compiler to compile into the
65//! user's crate.
66//!
67//! [`TokenStream`]: https://doc.rust-lang.org/proc_macro/struct.TokenStream.html
68//!
69//! ```toml
70//! [dependencies]
David Tolnay87003d02018-05-20 19:45:13 -070071//! syn = "0.14"
72//! quote = "0.6"
David Tolnay5e84e972018-01-05 17:51:06 -080073//!
74//! [lib]
75//! proc-macro = true
76//! ```
77//!
78//! ```rust
79//! extern crate proc_macro;
80//! extern crate syn;
81//!
82//! #[macro_use]
83//! extern crate quote;
84//!
85//! use proc_macro::TokenStream;
86//! use syn::DeriveInput;
87//!
88//! # const IGNORE_TOKENS: &str = stringify! {
89//! #[proc_macro_derive(MyMacro)]
90//! # };
91//! pub fn my_macro(input: TokenStream) -> TokenStream {
92//! // Parse the input tokens into a syntax tree
93//! let input: DeriveInput = syn::parse(input).unwrap();
94//!
95//! // Build the output, possibly using quasi-quotation
96//! let expanded = quote! {
97//! // ...
98//! };
99//!
100//! // Hand the output tokens back to the compiler
101//! expanded.into()
102//! }
103//! #
104//! # fn main() {}
105//! ```
106//!
107//! The [`heapsize`] example directory shows a complete working Macros 1.1
108//! implementation of a custom derive. It works on any Rust compiler \>=1.15.0.
109//! The example derives a `HeapSize` trait which computes an estimate of the
110//! amount of heap memory owned by a value.
111//!
112//! [`heapsize`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize
113//!
114//! ```rust
115//! pub trait HeapSize {
116//! /// Total number of bytes of heap memory owned by `self`.
117//! fn heap_size_of_children(&self) -> usize;
118//! }
119//! ```
120//!
121//! The custom derive allows users to write `#[derive(HeapSize)]` on data
122//! structures in their program.
123//!
124//! ```rust
125//! # const IGNORE_TOKENS: &str = stringify! {
126//! #[derive(HeapSize)]
127//! # };
128//! struct Demo<'a, T: ?Sized> {
129//! a: Box<T>,
130//! b: u8,
131//! c: &'a str,
132//! d: String,
133//! }
134//! ```
135//!
136//! ## Spans and error reporting
137//!
138//! The [`heapsize2`] example directory is an extension of the `heapsize`
139//! example that demonstrates some of the hygiene and error reporting properties
140//! of Macros 2.0. This example currently requires a nightly Rust compiler
141//! \>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
142//!
143//! [`heapsize2`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize2
144//!
145//! The token-based procedural macro API provides great control over where the
146//! compiler's error messages are displayed in user code. Consider the error the
147//! user sees if one of their field types does not implement `HeapSize`.
148//!
149//! ```rust
150//! # const IGNORE_TOKENS: &str = stringify! {
151//! #[derive(HeapSize)]
152//! # };
153//! struct Broken {
154//! ok: String,
155//! bad: std::thread::Thread,
156//! }
157//! ```
158//!
159//! In the Macros 1.1 string-based procedural macro world, the resulting error
160//! would point unhelpfully to the invocation of the derive macro and not to the
161//! actual problematic field.
162//!
163//! ```text
164//! error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
165//! --> src/main.rs:4:10
166//! |
167//! 4 | #[derive(HeapSize)]
168//! | ^^^^^^^^
169//! ```
170//!
171//! By tracking span information all the way through the expansion of a
172//! procedural macro as shown in the `heapsize2` example, token-based macros in
173//! Syn are able to trigger errors that directly pinpoint the source of the
174//! problem.
175//!
176//! ```text
177//! error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
178//! --> src/main.rs:7:5
179//! |
180//! 7 | bad: std::thread::Thread,
David Tolnayefff2ff2018-01-07 11:49:52 -0800181//! | ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
David Tolnay5e84e972018-01-05 17:51:06 -0800182//! ```
183//!
184//! ## Parsing a custom syntax using combinators
185//!
186//! The [`lazy-static`] example directory shows the implementation of a
187//! `functionlike!(...)` procedural macro in which the input tokens are parsed
188//! using [`nom`]-style parser combinators.
189//!
190//! [`lazy-static`]: https://github.com/dtolnay/syn/tree/master/examples/lazy-static
191//! [`nom`]: https://github.com/Geal/nom
192//!
193//! The example reimplements the popular `lazy_static` crate from crates.io as a
194//! procedural macro.
195//!
196//! ```
197//! # macro_rules! lazy_static {
198//! # ($($tt:tt)*) => {}
199//! # }
200//! #
201//! lazy_static! {
202//! static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
203//! }
204//! ```
205//!
206//! The implementation shows how to trigger custom warnings and error messages
207//! on the macro input.
208//!
209//! ```text
210//! warning: come on, pick a more creative name
211//! --> src/main.rs:10:16
212//! |
213//! 10 | static ref FOO: String = "lazy_static".to_owned();
214//! | ^^^
215//! ```
216//!
217//! ## Debugging
218//!
219//! When developing a procedural macro it can be helpful to look at what the
220//! generated code looks like. Use `cargo rustc -- -Zunstable-options
221//! --pretty=expanded` or the [`cargo expand`] subcommand.
222//!
David Tolnay324db2d2018-01-07 11:51:09 -0800223//! [`cargo expand`]: https://github.com/dtolnay/cargo-expand
David Tolnay5e84e972018-01-05 17:51:06 -0800224//!
225//! To show the expanded code for some crate that uses your procedural macro,
226//! run `cargo expand` from that crate. To show the expanded code for one of
227//! your own test cases, run `cargo expand --test the_test_case` where the last
228//! argument is the name of the test file without the `.rs` extension.
229//!
230//! This write-up by Brandon W Maister discusses debugging in more detail:
231//! [Debugging Rust's new Custom Derive system][debugging].
232//!
233//! [debugging]: https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/
234//!
235//! ## Optional features
236//!
237//! Syn puts a lot of functionality behind optional features in order to
238//! optimize compile time for the most common use cases. The following features
239//! are available.
240//!
241//! - **`derive`** *(enabled by default)* — Data structures for representing the
242//! possible input to a custom derive, including structs and enums and types.
243//! - **`full`** — Data structures for representing the syntax tree of all valid
244//! Rust source code, including items and expressions.
245//! - **`parsing`** *(enabled by default)* — Ability to parse input tokens into
246//! a syntax tree node of a chosen type.
247//! - **`printing`** *(enabled by default)* — Ability to print a syntax tree
248//! node as tokens of Rust source code.
249//! - **`visit`** — Trait for traversing a syntax tree.
David Tolnay34981cf2018-01-06 16:22:35 -0800250//! - **`visit-mut`** — Trait for traversing and mutating in place a syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800251//! tree.
252//! - **`fold`** — Trait for transforming an owned syntax tree.
253//! - **`clone-impls`** *(enabled by default)* — Clone impls for all syntax tree
254//! types.
255//! - **`extra-traits`** — Debug, Eq, PartialEq, Hash impls for all syntax tree
256//! types.
hcpl4b72a382018-04-04 14:50:24 +0300257//! - **`proc-macro`** *(enabled by default)* — Runtime dependency on the
258//! dynamic library libproc_macro from rustc toolchain.
David Tolnay5e84e972018-01-05 17:51:06 -0800259
David Tolnay4cf7db82018-01-07 15:22:01 -0800260// Syn types in rustdoc of other crates get linked to here.
David Tolnaye6ec9e02018-08-21 21:32:03 -0400261#![doc(html_root_url = "https://docs.rs/syn/0.14.9")]
David Tolnay34071ba2018-05-20 20:00:41 -0700262#![cfg_attr(feature = "cargo-clippy", deny(clippy, clippy_pedantic))]
David Tolnay34071ba2018-05-20 20:00:41 -0700263// Ignored clippy lints.
David Tolnay94d2b792018-04-29 12:26:10 -0700264#![cfg_attr(
265 feature = "cargo-clippy",
266 allow(
David Tolnay4831ac62018-08-30 21:04:16 -0700267 block_in_if_condition_stmt,
David Tolnay0cec3e62018-07-21 09:08:30 -0700268 const_static_lifetime,
David Tolnay24b079d2018-08-27 08:28:10 -0700269 cyclomatic_complexity,
David Tolnay0cec3e62018-07-21 09:08:30 -0700270 doc_markdown,
David Tolnay24b079d2018-08-27 08:28:10 -0700271 eval_order_dependence,
David Tolnay0cec3e62018-07-21 09:08:30 -0700272 large_enum_variant,
273 match_bool,
274 redundant_closure,
275 needless_pass_by_value,
276 redundant_field_names,
277 trivially_copy_pass_by_ref
David Tolnay94d2b792018-04-29 12:26:10 -0700278 )
279)]
David Tolnay34071ba2018-05-20 20:00:41 -0700280// Ignored clippy_pedantic lints.
281#![cfg_attr(
282 feature = "cargo-clippy",
283 allow(
David Tolnay0cec3e62018-07-21 09:08:30 -0700284 cast_possible_truncation,
285 cast_possible_wrap,
David Tolnayb1617752018-08-24 21:16:33 -0400286 empty_enum,
David Tolnay0cec3e62018-07-21 09:08:30 -0700287 if_not_else,
288 indexing_slicing,
289 items_after_statements,
David Tolnay151f92f2018-08-14 22:44:53 -0700290 shadow_unrelated,
David Tolnay0cec3e62018-07-21 09:08:30 -0700291 similar_names,
292 single_match_else,
293 stutter,
294 unseparated_literal_suffix,
295 use_self,
296 used_underscore_binding
David Tolnay34071ba2018-05-20 20:00:41 -0700297 )
298)]
David Tolnayad2836d2017-04-20 10:11:43 -0700299
David Tolnay278f9e32018-08-14 22:41:11 -0700300#[cfg(all(
301 not(all(target_arch = "wasm32", target_os = "unknown")),
302 feature = "proc-macro"
303))]
David Tolnay51382052017-12-27 13:46:21 -0500304extern crate proc_macro;
David Tolnay94d2b792018-04-29 12:26:10 -0700305extern crate proc_macro2;
David Tolnay570695e2017-06-03 16:15:13 -0700306extern crate unicode_xid;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700307
David Tolnay1cf80912017-12-31 18:35:12 -0500308#[cfg(feature = "printing")]
David Tolnay87d0b442016-09-04 11:52:12 -0700309extern crate quote;
310
Alex Crichton62a0a592017-05-22 13:58:53 -0700311#[macro_use]
312mod macros;
313
David Tolnay852bff72018-08-27 08:24:02 -0700314#[cfg(feature = "parsing")]
315#[macro_use]
316mod group;
317
David Tolnayc5ab8c62017-12-26 16:43:39 -0500318#[macro_use]
David Tolnay32954ef2017-12-26 22:43:16 -0500319pub mod token;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500320
David Tolnay4fb71232018-08-25 23:14:50 -0400321mod ident;
322pub use ident::Ident;
David Tolnaye303b7c2018-05-20 16:46:35 -0700323
David Tolnay3cfd1d32018-01-03 00:22:08 -0800324#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700325mod attr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800326#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayaaadd782018-01-06 22:58:13 -0800327pub use attr::{AttrStyle, Attribute, Meta, MetaList, MetaNameValue, NestedMeta};
David Tolnay35161ff2016-09-03 11:33:15 -0700328
David Tolnay3cfd1d32018-01-03 00:22:08 -0800329#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf38cdf62016-09-23 19:07:09 -0700330mod data;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800331#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700332pub use data::{
333 Field, Fields, FieldsNamed, FieldsUnnamed, Variant, VisCrate, VisPublic, VisRestricted,
334 Visibility,
335};
David Tolnayf38cdf62016-09-23 19:07:09 -0700336
David Tolnay3cfd1d32018-01-03 00:22:08 -0800337#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700338mod expr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800339#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700340pub use expr::{
David Tolnay02a9c6f2018-08-24 18:58:45 -0400341 Expr, ExprArray, ExprAssign, ExprAssignOp, ExprAsync, ExprBinary, ExprBlock, ExprBox,
342 ExprBreak, ExprCall, ExprCast, ExprClosure, ExprContinue, ExprField, ExprForLoop, ExprGroup,
343 ExprIf, ExprIfLet, ExprInPlace, ExprIndex, ExprLit, ExprLoop, ExprMacro, ExprMatch,
344 ExprMethodCall, ExprParen, ExprPath, ExprRange, ExprReference, ExprRepeat, ExprReturn,
345 ExprStruct, ExprTry, ExprTryBlock, ExprTuple, ExprType, ExprUnary, ExprUnsafe, ExprVerbatim,
346 ExprWhile, ExprWhileLet, ExprYield, Index, Member,
David Tolnayb57c8492018-05-05 00:32:04 -0700347};
Michael Layzell734adb42017-06-07 16:58:31 -0400348
349#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700350pub use expr::{
351 Arm, Block, FieldPat, FieldValue, GenericMethodArgument, Label, Local, MethodTurbofish, Pat,
352 PatBox, PatIdent, PatLit, PatMacro, PatPath, PatRange, PatRef, PatSlice, PatStruct, PatTuple,
353 PatTupleStruct, PatVerbatim, PatWild, RangeLimits, Stmt,
354};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700355
David Tolnay3cfd1d32018-01-03 00:22:08 -0800356#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700357mod generics;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800358#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700359pub use generics::{
360 BoundLifetimes, ConstParam, GenericParam, Generics, LifetimeDef, PredicateEq,
361 PredicateLifetime, PredicateType, TraitBound, TraitBoundModifier, TypeParam, TypeParamBound,
362 WhereClause, WherePredicate,
363};
David Tolnay278f9e32018-08-14 22:41:11 -0700364#[cfg(all(
365 any(feature = "full", feature = "derive"),
366 feature = "printing"
367))]
David Tolnayfd6bf5c2017-11-12 09:41:14 -0800368pub use generics::{ImplGenerics, Turbofish, TypeGenerics};
David Tolnay35161ff2016-09-03 11:33:15 -0700369
David Tolnayf38cdf62016-09-23 19:07:09 -0700370#[cfg(feature = "full")]
David Tolnayb79ee962016-09-04 09:39:20 -0700371mod item;
David Tolnayf38cdf62016-09-23 19:07:09 -0700372#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700373pub use item::{
David Tolnay435c1782018-08-24 16:15:44 -0400374 ArgCaptured, ArgSelf, ArgSelfRef, FnArg, FnDecl, ForeignItem, ForeignItemFn, ForeignItemMacro,
375 ForeignItemStatic, ForeignItemType, ForeignItemVerbatim, ImplItem, ImplItemConst,
David Tolnaybb82ef02018-08-24 20:15:45 -0400376 ImplItemExistential, ImplItemMacro, ImplItemMethod, ImplItemType, ImplItemVerbatim, Item,
377 ItemConst, ItemEnum, ItemExistential, ItemExternCrate, ItemFn, ItemForeignMod, ItemImpl,
378 ItemMacro, ItemMacro2, ItemMod, ItemStatic, ItemStruct, ItemTrait, ItemType, ItemUnion,
David Tolnay3779bb72018-08-26 18:46:07 -0700379 ItemUse, ItemVerbatim, MethodSig, TraitItem, TraitItemConst, TraitItemMacro, TraitItemMethod,
380 TraitItemType, TraitItemVerbatim, UseGlob, UseGroup, UseName, UsePath, UseRename, UseTree,
David Tolnayb57c8492018-05-05 00:32:04 -0700381};
David Tolnay35161ff2016-09-03 11:33:15 -0700382
David Tolnay631cb8c2016-11-10 17:16:41 -0800383#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700384mod file;
David Tolnay631cb8c2016-11-10 17:16:41 -0800385#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700386pub use file::File;
David Tolnay631cb8c2016-11-10 17:16:41 -0800387
David Tolnay3cfd1d32018-01-03 00:22:08 -0800388#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700389mod lifetime;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800390#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700391pub use lifetime::Lifetime;
392
David Tolnay3cfd1d32018-01-03 00:22:08 -0800393#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700394mod lit;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800395#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700396pub use lit::{
397 FloatSuffix, IntSuffix, Lit, LitBool, LitByte, LitByteStr, LitChar, LitFloat, LitInt, LitStr,
398 LitVerbatim, StrStyle,
399};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700400
David Tolnay3cfd1d32018-01-03 00:22:08 -0800401#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700402mod mac;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800403#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayab919512017-12-30 23:31:51 -0500404pub use mac::{Macro, MacroDelimiter};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700405
David Tolnay3cfd1d32018-01-03 00:22:08 -0800406#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay0e837402016-12-22 17:25:55 -0500407mod derive;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800408#[cfg(feature = "derive")]
David Tolnaye3d41b72017-12-31 15:24:00 -0500409pub use derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
David Tolnayf38cdf62016-09-23 19:07:09 -0700410
David Tolnay3cfd1d32018-01-03 00:22:08 -0800411#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700412mod op;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800413#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700414pub use op::{BinOp, UnOp};
415
David Tolnay3cfd1d32018-01-03 00:22:08 -0800416#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700417mod ty;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800418#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700419pub use ty::{
420 Abi, BareFnArg, BareFnArgName, ReturnType, Type, TypeArray, TypeBareFn, TypeGroup,
421 TypeImplTrait, TypeInfer, TypeMacro, TypeNever, TypeParen, TypePath, TypePtr, TypeReference,
422 TypeSlice, TypeTraitObject, TypeTuple, TypeVerbatim,
423};
David Tolnay056de302018-01-05 14:29:05 -0800424
425#[cfg(any(feature = "full", feature = "derive"))]
426mod path;
David Tolnay278f9e32018-08-14 22:41:11 -0700427#[cfg(all(
428 any(feature = "full", feature = "derive"),
429 feature = "printing"
430))]
David Tolnay94d2b792018-04-29 12:26:10 -0700431pub use path::PathTokens;
David Tolnay056de302018-01-05 14:29:05 -0800432#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700433pub use path::{
434 AngleBracketedGenericArguments, Binding, GenericArgument, ParenthesizedGenericArguments, Path,
435 PathArguments, PathSegment, QSelf,
436};
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700437
David Tolnay1b752fb2017-12-26 21:41:39 -0500438#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800439pub mod buffer;
David Tolnay94d2b792018-04-29 12:26:10 -0700440pub mod punctuated;
David Tolnay1b752fb2017-12-26 21:41:39 -0500441#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500442pub mod synom;
David Tolnay3779bb72018-08-26 18:46:07 -0700443#[cfg(all(
444 any(feature = "full", feature = "derive"),
445 feature = "extra-traits"
446))]
David Tolnaye0824032017-12-27 15:25:56 -0500447mod tt;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500448
David Tolnaye83ef5a2018-01-11 15:18:36 -0800449// Not public API except the `parse_quote!` macro.
David Tolnay491680a2018-01-23 00:34:40 -0800450#[cfg(feature = "parsing")]
David Tolnaye83ef5a2018-01-11 15:18:36 -0800451#[doc(hidden)]
452pub mod parse_quote;
453
David Tolnay4d942b42018-01-02 22:14:04 -0800454#[cfg(all(feature = "parsing", feature = "printing"))]
David Tolnayf790b612017-12-31 18:46:57 -0500455pub mod spanned;
456
Nika Layzella6f46c42017-10-26 15:26:16 -0400457mod gen {
David Tolnayded2d682018-01-06 18:53:53 -0800458 /// Syntax tree traversal to walk a shared borrow of a syntax tree.
459 ///
460 /// Each method of the [`Visit`] trait is a hook that can be overridden to
461 /// customize the behavior when visiting the corresponding type of node. By
462 /// default, every method recursively visits the substructure of the input
463 /// by invoking the right visitor method of each of its fields.
464 ///
465 /// [`Visit`]: trait.Visit.html
466 ///
467 /// ```rust
468 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
469 /// #
470 /// pub trait Visit<'ast> {
471 /// /* ... */
472 ///
473 /// fn visit_expr_binary(&mut self, node: &'ast ExprBinary) {
474 /// for attr in &node.attrs {
475 /// self.visit_attribute(attr);
476 /// }
477 /// self.visit_expr(&*node.left);
478 /// self.visit_bin_op(&node.op);
479 /// self.visit_expr(&*node.right);
480 /// }
481 ///
482 /// /* ... */
483 /// # fn visit_attribute(&mut self, node: &'ast Attribute);
484 /// # fn visit_expr(&mut self, node: &'ast Expr);
485 /// # fn visit_bin_op(&mut self, node: &'ast BinOp);
486 /// }
487 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800488 ///
489 /// *This module is available if Syn is built with the `"visit"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400490 #[cfg(feature = "visit")]
491 pub mod visit;
David Tolnay55337722016-09-11 12:58:56 -0700492
David Tolnayded2d682018-01-06 18:53:53 -0800493 /// Syntax tree traversal to mutate an exclusive borrow of a syntax tree in
494 /// place.
495 ///
496 /// Each method of the [`VisitMut`] trait is a hook that can be overridden
497 /// to customize the behavior when mutating the corresponding type of node.
498 /// By default, every method recursively visits the substructure of the
499 /// input by invoking the right visitor method of each of its fields.
500 ///
501 /// [`VisitMut`]: trait.VisitMut.html
502 ///
503 /// ```rust
504 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
505 /// #
506 /// pub trait VisitMut {
507 /// /* ... */
508 ///
509 /// fn visit_expr_binary_mut(&mut self, node: &mut ExprBinary) {
510 /// for attr in &mut node.attrs {
511 /// self.visit_attribute_mut(attr);
512 /// }
513 /// self.visit_expr_mut(&mut *node.left);
514 /// self.visit_bin_op_mut(&mut node.op);
515 /// self.visit_expr_mut(&mut *node.right);
516 /// }
517 ///
518 /// /* ... */
519 /// # fn visit_attribute_mut(&mut self, node: &mut Attribute);
520 /// # fn visit_expr_mut(&mut self, node: &mut Expr);
521 /// # fn visit_bin_op_mut(&mut self, node: &mut BinOp);
522 /// }
523 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800524 ///
525 /// *This module is available if Syn is built with the `"visit-mut"`
526 /// feature.*
David Tolnay9df02c42018-01-06 13:52:48 -0800527 #[cfg(feature = "visit-mut")]
Nika Layzella6f46c42017-10-26 15:26:16 -0400528 pub mod visit_mut;
Nika Layzell27726662017-10-24 23:16:35 -0400529
David Tolnayded2d682018-01-06 18:53:53 -0800530 /// Syntax tree traversal to transform the nodes of an owned syntax tree.
531 ///
532 /// Each method of the [`Fold`] trait is a hook that can be overridden to
533 /// customize the behavior when transforming the corresponding type of node.
534 /// By default, every method recursively visits the substructure of the
535 /// input by invoking the right visitor method of each of its fields.
536 ///
537 /// [`Fold`]: trait.Fold.html
538 ///
539 /// ```rust
540 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
541 /// #
542 /// pub trait Fold {
543 /// /* ... */
544 ///
545 /// fn fold_expr_binary(&mut self, node: ExprBinary) -> ExprBinary {
546 /// ExprBinary {
547 /// attrs: node.attrs
548 /// .into_iter()
549 /// .map(|attr| self.fold_attribute(attr))
550 /// .collect(),
551 /// left: Box::new(self.fold_expr(*node.left)),
552 /// op: self.fold_bin_op(node.op),
553 /// right: Box::new(self.fold_expr(*node.right)),
554 /// }
555 /// }
556 ///
557 /// /* ... */
558 /// # fn fold_attribute(&mut self, node: Attribute) -> Attribute;
559 /// # fn fold_expr(&mut self, node: Expr) -> Expr;
560 /// # fn fold_bin_op(&mut self, node: BinOp) -> BinOp;
561 /// }
562 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800563 ///
564 /// *This module is available if Syn is built with the `"fold"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400565 #[cfg(feature = "fold")]
566 pub mod fold;
David Tolnayf60f4262017-12-28 19:17:58 -0500567
David Tolnay0a0d78c2018-01-05 15:24:01 -0800568 #[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf60f4262017-12-28 19:17:58 -0500569 #[path = "../gen_helper.rs"]
570 mod helper;
Nika Layzella6f46c42017-10-26 15:26:16 -0400571}
572pub use gen::*;
gnzlbg9ae88d82017-01-26 20:45:17 +0100573
David Tolnay456c9822018-08-25 08:09:46 -0400574// Not public API.
575#[doc(hidden)]
576pub mod export;
577
David Tolnay776f8e02018-08-24 22:32:10 -0400578#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400579mod lookahead;
580
581#[cfg(feature = "parsing")]
582pub mod parse;
583
David Tolnay776f8e02018-08-24 22:32:10 -0400584mod span;
585
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700586////////////////////////////////////////////////////////////////////////////////
587
David Tolnay55337722016-09-11 12:58:56 -0700588#[cfg(feature = "parsing")]
David Tolnayd641ee32018-08-30 17:11:09 -0700589use synom::Parser;
Ted Driggs054abbb2017-05-01 12:20:52 -0700590
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700591#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400592mod error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500593#[cfg(feature = "parsing")]
David Tolnayad4b2472018-08-25 08:25:24 -0400594use error::Error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500595
David Tolnayccab0be2018-01-06 22:24:47 -0800596/// Parse tokens of source code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700597///
598/// This is preferred over parsing a string because tokens are able to preserve
599/// information about where in the user's code they were originally written (the
600/// "span" of the token), possibly allowing the compiler to produce better error
601/// messages.
602///
David Tolnayccab0be2018-01-06 22:24:47 -0800603/// This function parses a `proc_macro::TokenStream` which is the type used for
604/// interop with the compiler in a procedural macro. To parse a
605/// `proc_macro2::TokenStream`, use [`syn::parse2`] instead.
606///
607/// [`syn::parse2`]: fn.parse2.html
608///
hcpl4b72a382018-04-04 14:50:24 +0300609/// *This function is available if Syn is built with both the `"parsing"` and
610/// `"proc-macro"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800611///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700612/// # Examples
613///
David Tolnaybcf26022017-12-25 22:10:52 -0500614/// ```rust
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700615/// extern crate proc_macro;
616/// use proc_macro::TokenStream;
617///
618/// extern crate syn;
619///
620/// #[macro_use]
621/// extern crate quote;
622///
623/// use syn::DeriveInput;
624///
David Tolnaybcf26022017-12-25 22:10:52 -0500625/// # const IGNORE_TOKENS: &str = stringify! {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700626/// #[proc_macro_derive(MyMacro)]
David Tolnaybcf26022017-12-25 22:10:52 -0500627/// # };
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700628/// pub fn my_macro(input: TokenStream) -> TokenStream {
629/// // Parse the tokens into a syntax tree
630/// let ast: DeriveInput = syn::parse(input).unwrap();
631///
632/// // Build the output, possibly using quasi-quotation
633/// let expanded = quote! {
634/// /* ... */
635/// };
636///
David Tolnaybcf26022017-12-25 22:10:52 -0500637/// // Convert into a token stream and return it
638/// expanded.into()
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700639/// }
David Tolnaybcf26022017-12-25 22:10:52 -0500640/// #
641/// # fn main() {}
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700642/// ```
David Tolnay278f9e32018-08-14 22:41:11 -0700643#[cfg(all(
644 not(all(target_arch = "wasm32", target_os = "unknown")),
645 feature = "parsing",
646 feature = "proc-macro"
647))]
David Tolnaye82a2b12018-08-30 16:31:10 -0700648pub fn parse<T: parse::Parse>(tokens: proc_macro::TokenStream) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700649 T::parse.parse(tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700650}
651
David Tolnayccab0be2018-01-06 22:24:47 -0800652/// Parse a proc-macro2 token stream into the chosen syntax tree node.
653///
654/// This function parses a `proc_macro2::TokenStream` which is commonly useful
655/// when the input comes from a node of the Syn syntax tree, for example the tts
656/// of a [`Macro`] node. When in a procedural macro parsing the
657/// `proc_macro::TokenStream` provided by the compiler, use [`syn::parse`]
658/// instead.
659///
660/// [`Macro`]: struct.Macro.html
661/// [`syn::parse`]: fn.parse.html
David Tolnay461d98e2018-01-07 11:07:19 -0800662///
663/// *This function is available if Syn is built with the `"parsing"` feature.*
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700664#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700665pub fn parse2<T: parse::Parse>(tokens: proc_macro2::TokenStream) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700666 T::parse.parse2(tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700667}
Alex Crichton954046c2017-05-30 21:49:42 -0700668
David Tolnayccab0be2018-01-06 22:24:47 -0800669/// Parse a string of Rust code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700670///
David Tolnay461d98e2018-01-07 11:07:19 -0800671/// *This function is available if Syn is built with the `"parsing"` feature.*
672///
David Tolnay3f5b06f2018-01-11 21:02:00 -0800673/// # Hygiene
674///
675/// Every span in the resulting syntax tree will be set to resolve at the macro
676/// call site.
677///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700678/// # Examples
679///
680/// ```rust
681/// extern crate syn;
682/// #
David Tolnay9174b972017-11-09 22:27:50 -0800683/// #
684/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700685///
686/// use syn::Expr;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700687///
688/// fn run() -> Result<()> {
689/// let code = "assert_eq!(u8::max_value(), 255)";
690/// let expr = syn::parse_str::<Expr>(code)?;
691/// println!("{:#?}", expr);
692/// Ok(())
693/// }
694/// #
695/// # fn main() { run().unwrap() }
696/// ```
697#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700698pub fn parse_str<T: parse::Parse>(s: &str) -> Result<T, Error> {
David Tolnayd641ee32018-08-30 17:11:09 -0700699 T::parse.parse_str(s)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700700}
Alex Crichton954046c2017-05-30 21:49:42 -0700701
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700702// FIXME the name parse_file makes it sound like you might pass in a path to a
703// file, rather than the content.
704/// Parse the content of a file of Rust code.
705///
706/// This is different from `syn::parse_str::<File>(content)` in two ways:
707///
708/// - It discards a leading byte order mark `\u{FEFF}` if the file has one.
709/// - It preserves the shebang line of the file, such as `#!/usr/bin/env rustx`.
710///
711/// If present, either of these would be an error using `from_str`.
712///
Christopher Serr9727ef22018-02-03 16:42:12 +0100713/// *This function is available if Syn is built with the `"parsing"` and `"full"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800714///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700715/// # Examples
716///
717/// ```rust,no_run
718/// extern crate syn;
719/// #
David Tolnay9174b972017-11-09 22:27:50 -0800720/// #
721/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700722///
723/// use std::fs::File;
724/// use std::io::Read;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700725///
726/// fn run() -> Result<()> {
727/// let mut file = File::open("path/to/code.rs")?;
728/// let mut content = String::new();
729/// file.read_to_string(&mut content)?;
730///
731/// let ast = syn::parse_file(&content)?;
732/// if let Some(shebang) = ast.shebang {
733/// println!("{}", shebang);
734/// }
735/// println!("{} items", ast.items.len());
736///
737/// Ok(())
738/// }
739/// #
740/// # fn main() { run().unwrap() }
741/// ```
742#[cfg(all(feature = "parsing", feature = "full"))]
David Tolnayad4b2472018-08-25 08:25:24 -0400743pub fn parse_file(mut content: &str) -> Result<File, Error> {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700744 // Strip the BOM if it is present
745 const BOM: &'static str = "\u{feff}";
746 if content.starts_with(BOM) {
747 content = &content[BOM.len()..];
David Tolnay35161ff2016-09-03 11:33:15 -0700748 }
Michael Layzell5e107ff2017-01-24 19:58:39 -0500749
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700750 let mut shebang = None;
751 if content.starts_with("#!") && !content.starts_with("#![") {
752 if let Some(idx) = content.find('\n') {
753 shebang = Some(content[..idx].to_string());
754 content = &content[idx..];
755 } else {
756 shebang = Some(content.to_string());
757 content = "";
758 }
Alex Crichton954046c2017-05-30 21:49:42 -0700759 }
David Tolnay0a8972b2017-02-27 02:10:01 -0800760
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700761 let mut file: File = parse_str(content)?;
762 file.shebang = shebang;
763 Ok(file)
Michael Layzell5e107ff2017-01-24 19:58:39 -0500764}
Alex Crichton259ee532017-07-14 06:51:02 -0700765
David Tolnayd641ee32018-08-30 17:11:09 -0700766/// Parse the input TokenStream of a macro, triggering a compile error if the
767/// tokens fail to parse.
768///
769/// # Intended usage
770///
771/// ```rust
772/// # extern crate proc_macro;
773/// # extern crate syn;
774/// #
775/// use proc_macro::TokenStream;
776/// use syn::parse_macro_input;
777/// use syn::parse::{Parse, ParseStream, Result};
778///
779/// struct MyMacroInput {
780/// /* ... */
781/// }
782///
783/// impl Parse for MyMacroInput {
784/// fn parse(input: ParseStream) -> Result<Self> {
785/// /* ... */
786/// # Ok(MyMacroInput {})
787/// }
788/// }
789///
790/// # const IGNORE: &str = stringify! {
791/// #[proc_macro]
792/// # };
793/// pub fn my_macro(tokens: TokenStream) -> TokenStream {
794/// let input = parse_macro_input!(tokens as MyMacroInput);
795///
796/// /* ... */
797/// # "".parse().unwrap()
798/// }
799/// #
800/// # fn main() {}
801/// ```
802#[cfg(feature = "proc-macro")]
803#[macro_export]
804macro_rules! parse_macro_input {
805 ($tokenstream:ident as $ty:ty) => {
806 match $crate::next::parse::<$ty>($tokenstream) {
807 $crate::export::Ok(data) => data,
808 $crate::export::Err(err) => {
809 return $crate::export::TokenStream::from(err.into_compile_error());
810 }
811 };
812 };
813}
814
David Tolnay278f9e32018-08-14 22:41:11 -0700815#[cfg(all(
816 any(feature = "full", feature = "derive"),
817 feature = "printing"
818))]
Alex Crichton259ee532017-07-14 06:51:02 -0700819struct TokensOrDefault<'a, T: 'a>(&'a Option<T>);
820
David Tolnay278f9e32018-08-14 22:41:11 -0700821#[cfg(all(
822 any(feature = "full", feature = "derive"),
823 feature = "printing"
824))]
Alex Crichton259ee532017-07-14 06:51:02 -0700825impl<'a, T> quote::ToTokens for TokensOrDefault<'a, T>
David Tolnay51382052017-12-27 13:46:21 -0500826where
827 T: quote::ToTokens + Default,
Alex Crichton259ee532017-07-14 06:51:02 -0700828{
Alex Crichtona74a1c82018-05-16 10:20:44 -0700829 fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
Alex Crichton259ee532017-07-14 06:51:02 -0700830 match *self.0 {
831 Some(ref t) => t.to_tokens(tokens),
832 None => T::default().to_tokens(tokens),
833 }
834 }
835}