blob: 55d898aeb84b0d8c25450cc8e989e1f024350d3f [file] [log] [blame]
David Tolnay55535012018-01-05 16:39:23 -08001// Copyright 2018 Syn Developers
2//
3// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
4// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
5// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
6// option. This file may not be copied, modified, or distributed
7// except according to those terms.
8
David Tolnay5e84e972018-01-05 17:51:06 -08009//! Syn is a parsing library for parsing a stream of Rust tokens into a syntax
10//! tree of Rust source code.
11//!
12//! Currently this library is geared toward the [custom derive] use case but
13//! contains some APIs that may be useful for Rust procedural macros more
14//! generally.
15//!
16//! [custom derive]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
17//!
18//! - **Data structures** — Syn provides a complete syntax tree that can
19//! represent any valid Rust source code. The syntax tree is rooted at
20//! [`syn::File`] which represents a full source file, but there are other
21//! entry points that may be useful to procedural macros including
22//! [`syn::Item`], [`syn::Expr`] and [`syn::Type`].
23//!
24//! - **Custom derives** — Of particular interest to custom derives is
25//! [`syn::DeriveInput`] which is any of the three legal input items to a
26//! derive macro. An example below shows using this type in a library that can
27//! derive implementations of a trait of your own.
28//!
29//! - **Parser combinators** — Parsing in Syn is built on a suite of public
30//! parser combinator macros that you can use for parsing any token-based
31//! syntax you dream up within a `functionlike!(...)` procedural macro. Every
32//! syntax tree node defined by Syn is individually parsable and may be used
33//! as a building block for custom syntaxes, or you may do it all yourself
34//! working from the most primitive tokens.
35//!
36//! - **Location information** — Every token parsed by Syn is associated with a
37//! `Span` that tracks line and column information back to the source of that
38//! token. These spans allow a procedural macro to display detailed error
39//! messages pointing to all the right places in the user's code. There is an
40//! example of this below.
41//!
42//! - **Feature flags** — Functionality is aggressively feature gated so your
43//! procedural macros enable only what they need, and do not pay in compile
44//! time for all the rest.
45//!
46//! [`syn::File`]: struct.File.html
47//! [`syn::Item`]: enum.Item.html
48//! [`syn::Expr`]: enum.Expr.html
49//! [`syn::Type`]: enum.Type.html
50//! [`syn::DeriveInput`]: struct.DeriveInput.html
51//!
52//! *Version requirement: Syn supports any compiler version back to Rust's very
53//! first support for procedural macros in Rust 1.15.0. Some features especially
54//! around error reporting are only available in newer compilers or on the
55//! nightly channel.*
56//!
57//! ## Example of a custom derive
58//!
59//! The canonical custom derive using Syn looks like this. We write an ordinary
60//! Rust function tagged with a `proc_macro_derive` attribute and the name of
61//! the trait we are deriving. Any time that derive appears in the user's code,
62//! the Rust compiler passes their data structure as tokens into our macro. We
63//! get to execute arbitrary Rust code to figure out what to do with those
64//! tokens, then hand some tokens back to the compiler to compile into the
65//! user's crate.
66//!
67//! [`TokenStream`]: https://doc.rust-lang.org/proc_macro/struct.TokenStream.html
68//!
69//! ```toml
70//! [dependencies]
71//! syn = "0.12"
72//! quote = "0.4"
73//!
74//! [lib]
75//! proc-macro = true
76//! ```
77//!
78//! ```rust
79//! extern crate proc_macro;
80//! extern crate syn;
81//!
82//! #[macro_use]
83//! extern crate quote;
84//!
85//! use proc_macro::TokenStream;
86//! use syn::DeriveInput;
87//!
88//! # const IGNORE_TOKENS: &str = stringify! {
89//! #[proc_macro_derive(MyMacro)]
90//! # };
91//! pub fn my_macro(input: TokenStream) -> TokenStream {
92//! // Parse the input tokens into a syntax tree
93//! let input: DeriveInput = syn::parse(input).unwrap();
94//!
95//! // Build the output, possibly using quasi-quotation
96//! let expanded = quote! {
97//! // ...
98//! };
99//!
100//! // Hand the output tokens back to the compiler
101//! expanded.into()
102//! }
103//! #
104//! # fn main() {}
105//! ```
106//!
107//! The [`heapsize`] example directory shows a complete working Macros 1.1
108//! implementation of a custom derive. It works on any Rust compiler \>=1.15.0.
109//! The example derives a `HeapSize` trait which computes an estimate of the
110//! amount of heap memory owned by a value.
111//!
112//! [`heapsize`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize
113//!
114//! ```rust
115//! pub trait HeapSize {
116//! /// Total number of bytes of heap memory owned by `self`.
117//! fn heap_size_of_children(&self) -> usize;
118//! }
119//! ```
120//!
121//! The custom derive allows users to write `#[derive(HeapSize)]` on data
122//! structures in their program.
123//!
124//! ```rust
125//! # const IGNORE_TOKENS: &str = stringify! {
126//! #[derive(HeapSize)]
127//! # };
128//! struct Demo<'a, T: ?Sized> {
129//! a: Box<T>,
130//! b: u8,
131//! c: &'a str,
132//! d: String,
133//! }
134//! ```
135//!
136//! ## Spans and error reporting
137//!
138//! The [`heapsize2`] example directory is an extension of the `heapsize`
139//! example that demonstrates some of the hygiene and error reporting properties
140//! of Macros 2.0. This example currently requires a nightly Rust compiler
141//! \>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
142//!
143//! [`heapsize2`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize2
144//!
145//! The token-based procedural macro API provides great control over where the
146//! compiler's error messages are displayed in user code. Consider the error the
147//! user sees if one of their field types does not implement `HeapSize`.
148//!
149//! ```rust
150//! # const IGNORE_TOKENS: &str = stringify! {
151//! #[derive(HeapSize)]
152//! # };
153//! struct Broken {
154//! ok: String,
155//! bad: std::thread::Thread,
156//! }
157//! ```
158//!
159//! In the Macros 1.1 string-based procedural macro world, the resulting error
160//! would point unhelpfully to the invocation of the derive macro and not to the
161//! actual problematic field.
162//!
163//! ```text
164//! error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
165//! --> src/main.rs:4:10
166//! |
167//! 4 | #[derive(HeapSize)]
168//! | ^^^^^^^^
169//! ```
170//!
171//! By tracking span information all the way through the expansion of a
172//! procedural macro as shown in the `heapsize2` example, token-based macros in
173//! Syn are able to trigger errors that directly pinpoint the source of the
174//! problem.
175//!
176//! ```text
177//! error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
178//! --> src/main.rs:7:5
179//! |
180//! 7 | bad: std::thread::Thread,
David Tolnayefff2ff2018-01-07 11:49:52 -0800181//! | ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
David Tolnay5e84e972018-01-05 17:51:06 -0800182//! ```
183//!
184//! ## Parsing a custom syntax using combinators
185//!
186//! The [`lazy-static`] example directory shows the implementation of a
187//! `functionlike!(...)` procedural macro in which the input tokens are parsed
188//! using [`nom`]-style parser combinators.
189//!
190//! [`lazy-static`]: https://github.com/dtolnay/syn/tree/master/examples/lazy-static
191//! [`nom`]: https://github.com/Geal/nom
192//!
193//! The example reimplements the popular `lazy_static` crate from crates.io as a
194//! procedural macro.
195//!
196//! ```
197//! # macro_rules! lazy_static {
198//! # ($($tt:tt)*) => {}
199//! # }
200//! #
201//! lazy_static! {
202//! static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
203//! }
204//! ```
205//!
206//! The implementation shows how to trigger custom warnings and error messages
207//! on the macro input.
208//!
209//! ```text
210//! warning: come on, pick a more creative name
211//! --> src/main.rs:10:16
212//! |
213//! 10 | static ref FOO: String = "lazy_static".to_owned();
214//! | ^^^
215//! ```
216//!
217//! ## Debugging
218//!
219//! When developing a procedural macro it can be helpful to look at what the
220//! generated code looks like. Use `cargo rustc -- -Zunstable-options
221//! --pretty=expanded` or the [`cargo expand`] subcommand.
222//!
David Tolnay324db2d2018-01-07 11:51:09 -0800223//! [`cargo expand`]: https://github.com/dtolnay/cargo-expand
David Tolnay5e84e972018-01-05 17:51:06 -0800224//!
225//! To show the expanded code for some crate that uses your procedural macro,
226//! run `cargo expand` from that crate. To show the expanded code for one of
227//! your own test cases, run `cargo expand --test the_test_case` where the last
228//! argument is the name of the test file without the `.rs` extension.
229//!
230//! This write-up by Brandon W Maister discusses debugging in more detail:
231//! [Debugging Rust's new Custom Derive system][debugging].
232//!
233//! [debugging]: https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/
234//!
235//! ## Optional features
236//!
237//! Syn puts a lot of functionality behind optional features in order to
238//! optimize compile time for the most common use cases. The following features
239//! are available.
240//!
241//! - **`derive`** *(enabled by default)* — Data structures for representing the
242//! possible input to a custom derive, including structs and enums and types.
243//! - **`full`** — Data structures for representing the syntax tree of all valid
244//! Rust source code, including items and expressions.
245//! - **`parsing`** *(enabled by default)* — Ability to parse input tokens into
246//! a syntax tree node of a chosen type.
247//! - **`printing`** *(enabled by default)* — Ability to print a syntax tree
248//! node as tokens of Rust source code.
249//! - **`visit`** — Trait for traversing a syntax tree.
David Tolnay34981cf2018-01-06 16:22:35 -0800250//! - **`visit-mut`** — Trait for traversing and mutating in place a syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800251//! tree.
252//! - **`fold`** — Trait for transforming an owned syntax tree.
253//! - **`clone-impls`** *(enabled by default)* — Clone impls for all syntax tree
254//! types.
255//! - **`extra-traits`** — Debug, Eq, PartialEq, Hash impls for all syntax tree
256//! types.
257
David Tolnay4cf7db82018-01-07 15:22:01 -0800258// Syn types in rustdoc of other crates get linked to here.
259#![doc(html_root_url = "https://docs.rs/syn/0.12.0")]
David Tolnay51382052017-12-27 13:46:21 -0500260#![cfg_attr(feature = "cargo-clippy",
261 allow(const_static_lifetime, doc_markdown, large_enum_variant, match_bool,
David Tolnay76ebcdd2018-01-05 17:07:26 -0800262 redundant_closure, needless_pass_by_value))]
David Tolnayad2836d2017-04-20 10:11:43 -0700263
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700264extern crate proc_macro2;
David Tolnay51382052017-12-27 13:46:21 -0500265extern crate proc_macro;
David Tolnay570695e2017-06-03 16:15:13 -0700266extern crate unicode_xid;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700267
David Tolnay1cf80912017-12-31 18:35:12 -0500268#[cfg(feature = "printing")]
David Tolnay87d0b442016-09-04 11:52:12 -0700269extern crate quote;
270
David Tolnay1b752fb2017-12-26 21:41:39 -0500271#[cfg(feature = "parsing")]
David Tolnayf8db7ba2017-11-11 22:52:16 -0800272#[macro_use]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500273#[doc(hidden)]
274pub mod parsers;
David Tolnay35161ff2016-09-03 11:33:15 -0700275
Alex Crichton62a0a592017-05-22 13:58:53 -0700276#[macro_use]
277mod macros;
278
David Tolnayc5ab8c62017-12-26 16:43:39 -0500279#[macro_use]
David Tolnay32954ef2017-12-26 22:43:16 -0500280pub mod token;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500281
David Tolnay3cfd1d32018-01-03 00:22:08 -0800282#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700283mod attr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800284#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayaaadd782018-01-06 22:58:13 -0800285pub use attr::{AttrStyle, Attribute, Meta, MetaList, MetaNameValue, NestedMeta};
David Tolnay35161ff2016-09-03 11:33:15 -0700286
David Tolnay3cfd1d32018-01-03 00:22:08 -0800287#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf38cdf62016-09-23 19:07:09 -0700288mod data;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800289#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay61037c62018-01-05 16:21:03 -0800290pub use data::{Field, Fields, FieldsNamed, FieldsUnnamed, Variant, VisCrate, VisPublic,
291 VisRestricted, Visibility};
David Tolnayf38cdf62016-09-23 19:07:09 -0700292
David Tolnay3cfd1d32018-01-03 00:22:08 -0800293#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700294mod expr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800295#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay51382052017-12-27 13:46:21 -0500296pub use expr::{Expr, ExprAddrOf, ExprArray, ExprAssign, ExprAssignOp, ExprBinary, ExprBlock,
297 ExprBox, ExprBreak, ExprCall, ExprCast, ExprCatch, ExprClosure, ExprContinue,
298 ExprField, ExprForLoop, ExprGroup, ExprIf, ExprIfLet, ExprInPlace, ExprIndex,
David Tolnay61037c62018-01-05 16:21:03 -0800299 ExprLit, ExprLoop, ExprMacro, ExprMatch, ExprMethodCall, ExprParen, ExprPath,
300 ExprRange, ExprRepeat, ExprReturn, ExprStruct, ExprTry, ExprTuple, ExprType,
301 ExprUnary, ExprUnsafe, ExprVerbatim, ExprWhile, ExprWhileLet, ExprYield, Index,
302 Member};
Michael Layzell734adb42017-06-07 16:58:31 -0400303
304#[cfg(feature = "full")]
David Tolnaybcd498f2017-12-29 12:02:33 -0500305pub use expr::{Arm, Block, FieldPat, FieldValue, GenericMethodArgument, Label, Local,
David Tolnay61037c62018-01-05 16:21:03 -0800306 MethodTurbofish, Pat, PatBox, PatIdent, PatLit, PatMacro, PatPath, PatRange,
307 PatRef, PatSlice, PatStruct, PatTuple, PatTupleStruct, PatVerbatim, PatWild,
308 RangeLimits, Stmt};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700309
David Tolnay3cfd1d32018-01-03 00:22:08 -0800310#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700311mod generics;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800312#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay61037c62018-01-05 16:21:03 -0800313pub use generics::{BoundLifetimes, ConstParam, GenericParam, Generics, LifetimeDef, PredicateEq,
314 PredicateLifetime, PredicateType, TraitBound, TraitBoundModifier, TypeParam,
315 TypeParamBound, WhereClause, WherePredicate};
David Tolnay3cfd1d32018-01-03 00:22:08 -0800316#[cfg(all(any(feature = "full", feature = "derive"), feature = "printing"))]
David Tolnayfd6bf5c2017-11-12 09:41:14 -0800317pub use generics::{ImplGenerics, Turbofish, TypeGenerics};
David Tolnay35161ff2016-09-03 11:33:15 -0700318
David Tolnay55337722016-09-11 12:58:56 -0700319mod ident;
David Tolnaydaaf7742016-10-03 11:11:43 -0700320pub use ident::Ident;
David Tolnay55337722016-09-11 12:58:56 -0700321
David Tolnayf38cdf62016-09-23 19:07:09 -0700322#[cfg(feature = "full")]
David Tolnayb79ee962016-09-04 09:39:20 -0700323mod item;
David Tolnayf38cdf62016-09-23 19:07:09 -0700324#[cfg(feature = "full")]
David Tolnay61037c62018-01-05 16:21:03 -0800325pub use item::{ArgCaptured, ArgSelf, ArgSelfRef, FnArg, FnDecl, ForeignItem, ForeignItemFn,
326 ForeignItemStatic, ForeignItemType, ForeignItemVerbatim, ImplItem, ImplItemConst,
327 ImplItemMacro, ImplItemMethod, ImplItemType, ImplItemVerbatim, Item, ItemConst,
328 ItemEnum, ItemExternCrate, ItemFn, ItemForeignMod, ItemImpl, ItemMacro, ItemMacro2,
329 ItemMod, ItemStatic, ItemStruct, ItemTrait, ItemType, ItemUnion, ItemUse,
330 ItemVerbatim, MethodSig, TraitItem, TraitItemConst, TraitItemMacro,
331 TraitItemMethod, TraitItemType, TraitItemVerbatim, UseGlob, UseList, UsePath,
332 UseTree};
David Tolnay35161ff2016-09-03 11:33:15 -0700333
David Tolnay631cb8c2016-11-10 17:16:41 -0800334#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700335mod file;
David Tolnay631cb8c2016-11-10 17:16:41 -0800336#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700337pub use file::File;
David Tolnay631cb8c2016-11-10 17:16:41 -0800338
David Tolnay3cfd1d32018-01-03 00:22:08 -0800339#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700340mod lifetime;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800341#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay63e3dee2017-06-03 20:13:17 -0700342pub use lifetime::Lifetime;
343
David Tolnay3cfd1d32018-01-03 00:22:08 -0800344#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700345mod lit;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800346#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay61037c62018-01-05 16:21:03 -0800347pub use lit::{FloatSuffix, IntSuffix, Lit, LitBool, LitByte, LitByteStr, LitChar, LitFloat,
348 LitInt, LitStr, LitVerbatim, StrStyle};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700349
David Tolnay3cfd1d32018-01-03 00:22:08 -0800350#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700351mod mac;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800352#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayab919512017-12-30 23:31:51 -0500353pub use mac::{Macro, MacroDelimiter};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700354
David Tolnay3cfd1d32018-01-03 00:22:08 -0800355#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay0e837402016-12-22 17:25:55 -0500356mod derive;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800357#[cfg(feature = "derive")]
David Tolnaye3d41b72017-12-31 15:24:00 -0500358pub use derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
David Tolnayf38cdf62016-09-23 19:07:09 -0700359
David Tolnay3cfd1d32018-01-03 00:22:08 -0800360#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700361mod op;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800362#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700363pub use op::{BinOp, UnOp};
364
David Tolnay3cfd1d32018-01-03 00:22:08 -0800365#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700366mod ty;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800367#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay61037c62018-01-05 16:21:03 -0800368pub use ty::{Abi, BareFnArg, BareFnArgName, ReturnType, Type, TypeArray, TypeBareFn, TypeGroup,
369 TypeImplTrait, TypeInfer, TypeMacro, TypeNever, TypeParen, TypePath, TypePtr,
370 TypeReference, TypeSlice, TypeTraitObject, TypeTuple, TypeVerbatim};
David Tolnay056de302018-01-05 14:29:05 -0800371
372#[cfg(any(feature = "full", feature = "derive"))]
373mod path;
374#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay61037c62018-01-05 16:21:03 -0800375pub use path::{AngleBracketedGenericArguments, Binding, GenericArgument,
376 ParenthesizedGenericArguments, Path, PathArguments, PathSegment, QSelf};
David Tolnay3cfd1d32018-01-03 00:22:08 -0800377#[cfg(all(any(feature = "full", feature = "derive"), feature = "printing"))]
David Tolnay056de302018-01-05 14:29:05 -0800378pub use path::PathTokens;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700379
David Tolnay1b752fb2017-12-26 21:41:39 -0500380#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800381pub mod buffer;
David Tolnay1b752fb2017-12-26 21:41:39 -0500382#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500383pub mod synom;
David Tolnayf2cfd722017-12-31 18:02:51 -0500384pub mod punctuated;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800385#[cfg(all(any(feature = "full", feature = "derive"), feature = "parsing"))]
David Tolnaye0824032017-12-27 15:25:56 -0500386mod tt;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500387
David Tolnay4d942b42018-01-02 22:14:04 -0800388#[cfg(all(feature = "parsing", feature = "printing"))]
David Tolnayf790b612017-12-31 18:46:57 -0500389pub mod spanned;
390
Nika Layzella6f46c42017-10-26 15:26:16 -0400391mod gen {
David Tolnayded2d682018-01-06 18:53:53 -0800392 /// Syntax tree traversal to walk a shared borrow of a syntax tree.
393 ///
394 /// Each method of the [`Visit`] trait is a hook that can be overridden to
395 /// customize the behavior when visiting the corresponding type of node. By
396 /// default, every method recursively visits the substructure of the input
397 /// by invoking the right visitor method of each of its fields.
398 ///
399 /// [`Visit`]: trait.Visit.html
400 ///
401 /// ```rust
402 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
403 /// #
404 /// pub trait Visit<'ast> {
405 /// /* ... */
406 ///
407 /// fn visit_expr_binary(&mut self, node: &'ast ExprBinary) {
408 /// for attr in &node.attrs {
409 /// self.visit_attribute(attr);
410 /// }
411 /// self.visit_expr(&*node.left);
412 /// self.visit_bin_op(&node.op);
413 /// self.visit_expr(&*node.right);
414 /// }
415 ///
416 /// /* ... */
417 /// # fn visit_attribute(&mut self, node: &'ast Attribute);
418 /// # fn visit_expr(&mut self, node: &'ast Expr);
419 /// # fn visit_bin_op(&mut self, node: &'ast BinOp);
420 /// }
421 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800422 ///
423 /// *This module is available if Syn is built with the `"visit"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400424 #[cfg(feature = "visit")]
425 pub mod visit;
David Tolnay55337722016-09-11 12:58:56 -0700426
David Tolnayded2d682018-01-06 18:53:53 -0800427
428 /// Syntax tree traversal to mutate an exclusive borrow of a syntax tree in
429 /// place.
430 ///
431 /// Each method of the [`VisitMut`] trait is a hook that can be overridden
432 /// to customize the behavior when mutating the corresponding type of node.
433 /// By default, every method recursively visits the substructure of the
434 /// input by invoking the right visitor method of each of its fields.
435 ///
436 /// [`VisitMut`]: trait.VisitMut.html
437 ///
438 /// ```rust
439 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
440 /// #
441 /// pub trait VisitMut {
442 /// /* ... */
443 ///
444 /// fn visit_expr_binary_mut(&mut self, node: &mut ExprBinary) {
445 /// for attr in &mut node.attrs {
446 /// self.visit_attribute_mut(attr);
447 /// }
448 /// self.visit_expr_mut(&mut *node.left);
449 /// self.visit_bin_op_mut(&mut node.op);
450 /// self.visit_expr_mut(&mut *node.right);
451 /// }
452 ///
453 /// /* ... */
454 /// # fn visit_attribute_mut(&mut self, node: &mut Attribute);
455 /// # fn visit_expr_mut(&mut self, node: &mut Expr);
456 /// # fn visit_bin_op_mut(&mut self, node: &mut BinOp);
457 /// }
458 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800459 ///
460 /// *This module is available if Syn is built with the `"visit-mut"`
461 /// feature.*
David Tolnay9df02c42018-01-06 13:52:48 -0800462 #[cfg(feature = "visit-mut")]
Nika Layzella6f46c42017-10-26 15:26:16 -0400463 pub mod visit_mut;
Nika Layzell27726662017-10-24 23:16:35 -0400464
David Tolnayded2d682018-01-06 18:53:53 -0800465 /// Syntax tree traversal to transform the nodes of an owned syntax tree.
466 ///
467 /// Each method of the [`Fold`] trait is a hook that can be overridden to
468 /// customize the behavior when transforming the corresponding type of node.
469 /// By default, every method recursively visits the substructure of the
470 /// input by invoking the right visitor method of each of its fields.
471 ///
472 /// [`Fold`]: trait.Fold.html
473 ///
474 /// ```rust
475 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
476 /// #
477 /// pub trait Fold {
478 /// /* ... */
479 ///
480 /// fn fold_expr_binary(&mut self, node: ExprBinary) -> ExprBinary {
481 /// ExprBinary {
482 /// attrs: node.attrs
483 /// .into_iter()
484 /// .map(|attr| self.fold_attribute(attr))
485 /// .collect(),
486 /// left: Box::new(self.fold_expr(*node.left)),
487 /// op: self.fold_bin_op(node.op),
488 /// right: Box::new(self.fold_expr(*node.right)),
489 /// }
490 /// }
491 ///
492 /// /* ... */
493 /// # fn fold_attribute(&mut self, node: Attribute) -> Attribute;
494 /// # fn fold_expr(&mut self, node: Expr) -> Expr;
495 /// # fn fold_bin_op(&mut self, node: BinOp) -> BinOp;
496 /// }
497 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800498 ///
499 /// *This module is available if Syn is built with the `"fold"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400500 #[cfg(feature = "fold")]
501 pub mod fold;
David Tolnayf60f4262017-12-28 19:17:58 -0500502
David Tolnay0a0d78c2018-01-05 15:24:01 -0800503 #[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf60f4262017-12-28 19:17:58 -0500504 #[path = "../gen_helper.rs"]
505 mod helper;
Nika Layzella6f46c42017-10-26 15:26:16 -0400506}
507pub use gen::*;
gnzlbg9ae88d82017-01-26 20:45:17 +0100508
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700509////////////////////////////////////////////////////////////////////////////////
510
David Tolnay55337722016-09-11 12:58:56 -0700511#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500512use synom::Synom;
513#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800514use buffer::TokenBuffer;
Ted Driggs054abbb2017-05-01 12:20:52 -0700515
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700516#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500517mod error;
518#[cfg(feature = "parsing")]
David Tolnay203557a2017-12-27 23:59:33 -0500519use error::ParseError;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500520
521// Not public API.
David Tolnay1b752fb2017-12-26 21:41:39 -0500522#[cfg(feature = "parsing")]
David Tolnayc5ab8c62017-12-26 16:43:39 -0500523#[doc(hidden)]
524pub use error::parse_error;
Michael Layzell416724e2017-05-24 21:12:34 -0400525
David Tolnayccab0be2018-01-06 22:24:47 -0800526/// Parse tokens of source code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700527///
528/// This is preferred over parsing a string because tokens are able to preserve
529/// information about where in the user's code they were originally written (the
530/// "span" of the token), possibly allowing the compiler to produce better error
531/// messages.
532///
David Tolnayccab0be2018-01-06 22:24:47 -0800533/// This function parses a `proc_macro::TokenStream` which is the type used for
534/// interop with the compiler in a procedural macro. To parse a
535/// `proc_macro2::TokenStream`, use [`syn::parse2`] instead.
536///
537/// [`syn::parse2`]: fn.parse2.html
538///
David Tolnay461d98e2018-01-07 11:07:19 -0800539/// *This function is available if Syn is built with the `"parsing"` feature.*
540///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700541/// # Examples
542///
David Tolnaybcf26022017-12-25 22:10:52 -0500543/// ```rust
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700544/// extern crate proc_macro;
545/// use proc_macro::TokenStream;
546///
547/// extern crate syn;
548///
549/// #[macro_use]
550/// extern crate quote;
551///
552/// use syn::DeriveInput;
553///
David Tolnaybcf26022017-12-25 22:10:52 -0500554/// # const IGNORE_TOKENS: &str = stringify! {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700555/// #[proc_macro_derive(MyMacro)]
David Tolnaybcf26022017-12-25 22:10:52 -0500556/// # };
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700557/// pub fn my_macro(input: TokenStream) -> TokenStream {
558/// // Parse the tokens into a syntax tree
559/// let ast: DeriveInput = syn::parse(input).unwrap();
560///
561/// // Build the output, possibly using quasi-quotation
562/// let expanded = quote! {
563/// /* ... */
564/// };
565///
David Tolnaybcf26022017-12-25 22:10:52 -0500566/// // Convert into a token stream and return it
567/// expanded.into()
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700568/// }
David Tolnaybcf26022017-12-25 22:10:52 -0500569/// #
570/// # fn main() {}
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700571/// ```
572#[cfg(feature = "parsing")]
573pub fn parse<T>(tokens: proc_macro::TokenStream) -> Result<T, ParseError>
David Tolnay51382052017-12-27 13:46:21 -0500574where
575 T: Synom,
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700576{
David Tolnay04c5dca2018-01-05 20:12:21 -0800577 parse2(tokens.into())
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700578}
579
David Tolnayccab0be2018-01-06 22:24:47 -0800580/// Parse a proc-macro2 token stream into the chosen syntax tree node.
581///
582/// This function parses a `proc_macro2::TokenStream` which is commonly useful
583/// when the input comes from a node of the Syn syntax tree, for example the tts
584/// of a [`Macro`] node. When in a procedural macro parsing the
585/// `proc_macro::TokenStream` provided by the compiler, use [`syn::parse`]
586/// instead.
587///
588/// [`Macro`]: struct.Macro.html
589/// [`syn::parse`]: fn.parse.html
David Tolnay461d98e2018-01-07 11:07:19 -0800590///
591/// *This function is available if Syn is built with the `"parsing"` feature.*
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700592#[cfg(feature = "parsing")]
David Tolnay04c5dca2018-01-05 20:12:21 -0800593pub fn parse2<T>(tokens: proc_macro2::TokenStream) -> Result<T, ParseError>
David Tolnay51382052017-12-27 13:46:21 -0500594where
595 T: Synom,
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700596{
David Tolnay7c3e77d2018-01-06 17:42:53 -0800597 let buf = TokenBuffer::new2(tokens);
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700598 let result = T::parse(buf.begin());
599 let err = match result {
David Tolnayf4aa6b42017-12-31 16:40:33 -0500600 Ok((t, rest)) => {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700601 if rest.eof() {
602 return Ok(t);
603 } else if rest == buf.begin() {
604 // parsed nothing
605 ParseError::new("failed to parse anything")
606 } else {
607 ParseError::new("failed to parse all tokens")
David Tolnay55337722016-09-11 12:58:56 -0700608 }
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700609 }
610 Err(err) => err,
611 };
612 match T::description() {
Alex Crichtonc1b76f52017-07-06 15:04:24 -0700613 Some(s) => Err(ParseError::new(format!("failed to parse {}: {}", s, err))),
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700614 None => Err(err),
615 }
616}
Alex Crichton954046c2017-05-30 21:49:42 -0700617
David Tolnayccab0be2018-01-06 22:24:47 -0800618/// Parse a string of Rust code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700619///
David Tolnay461d98e2018-01-07 11:07:19 -0800620/// *This function is available if Syn is built with the `"parsing"` feature.*
621///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700622/// # Examples
623///
624/// ```rust
625/// extern crate syn;
626/// #
David Tolnay9174b972017-11-09 22:27:50 -0800627/// #
628/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700629///
630/// use syn::Expr;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700631///
632/// fn run() -> Result<()> {
633/// let code = "assert_eq!(u8::max_value(), 255)";
634/// let expr = syn::parse_str::<Expr>(code)?;
635/// println!("{:#?}", expr);
636/// Ok(())
637/// }
638/// #
639/// # fn main() { run().unwrap() }
640/// ```
641#[cfg(feature = "parsing")]
642pub fn parse_str<T: Synom>(s: &str) -> Result<T, ParseError> {
David Tolnayffb1f4d2017-12-28 00:07:59 -0500643 match s.parse() {
David Tolnay04c5dca2018-01-05 20:12:21 -0800644 Ok(tts) => parse2(tts),
David Tolnayffb1f4d2017-12-28 00:07:59 -0500645 Err(_) => Err(ParseError::new("error while lexing input string")),
646 }
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700647}
Alex Crichton954046c2017-05-30 21:49:42 -0700648
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700649// FIXME the name parse_file makes it sound like you might pass in a path to a
650// file, rather than the content.
651/// Parse the content of a file of Rust code.
652///
653/// This is different from `syn::parse_str::<File>(content)` in two ways:
654///
655/// - It discards a leading byte order mark `\u{FEFF}` if the file has one.
656/// - It preserves the shebang line of the file, such as `#!/usr/bin/env rustx`.
657///
658/// If present, either of these would be an error using `from_str`.
659///
David Tolnay461d98e2018-01-07 11:07:19 -0800660/// *This function is available if Syn is built with the `"parsing"` feature.*
661///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700662/// # Examples
663///
664/// ```rust,no_run
665/// extern crate syn;
666/// #
David Tolnay9174b972017-11-09 22:27:50 -0800667/// #
668/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700669///
670/// use std::fs::File;
671/// use std::io::Read;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700672///
673/// fn run() -> Result<()> {
674/// let mut file = File::open("path/to/code.rs")?;
675/// let mut content = String::new();
676/// file.read_to_string(&mut content)?;
677///
678/// let ast = syn::parse_file(&content)?;
679/// if let Some(shebang) = ast.shebang {
680/// println!("{}", shebang);
681/// }
682/// println!("{} items", ast.items.len());
683///
684/// Ok(())
685/// }
686/// #
687/// # fn main() { run().unwrap() }
688/// ```
689#[cfg(all(feature = "parsing", feature = "full"))]
690pub fn parse_file(mut content: &str) -> Result<File, ParseError> {
691 // Strip the BOM if it is present
692 const BOM: &'static str = "\u{feff}";
693 if content.starts_with(BOM) {
694 content = &content[BOM.len()..];
David Tolnay35161ff2016-09-03 11:33:15 -0700695 }
Michael Layzell5e107ff2017-01-24 19:58:39 -0500696
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700697 let mut shebang = None;
698 if content.starts_with("#!") && !content.starts_with("#![") {
699 if let Some(idx) = content.find('\n') {
700 shebang = Some(content[..idx].to_string());
701 content = &content[idx..];
702 } else {
703 shebang = Some(content.to_string());
704 content = "";
705 }
Alex Crichton954046c2017-05-30 21:49:42 -0700706 }
David Tolnay0a8972b2017-02-27 02:10:01 -0800707
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700708 let mut file: File = parse_str(content)?;
709 file.shebang = shebang;
710 Ok(file)
Michael Layzell5e107ff2017-01-24 19:58:39 -0500711}
Alex Crichton259ee532017-07-14 06:51:02 -0700712
David Tolnaydace3f12018-01-06 21:33:29 -0800713/// Quasi-quotation macro that accepts input like the [`quote!`] macro but uses
714/// type inference to figure out a return type for those tokens.
715///
716/// [`quote!`]: https://docs.rs/quote/0.4/quote/index.html
717///
718/// The return type can be any syntax tree node that implements the [`Synom`]
719/// trait.
720///
721/// [`Synom`]: synom/trait.Synom.html
722///
723/// ```
724/// #[macro_use]
725/// extern crate syn;
726///
727/// #[macro_use]
728/// extern crate quote;
729///
730/// use syn::Stmt;
731///
732/// fn main() {
733/// let name = quote!(v);
734/// let ty = quote!(u8);
735///
736/// let stmt: Stmt = parse_quote! {
737/// let #name: #ty = Default::default();
738/// };
739///
740/// println!("{:#?}", stmt);
741/// }
742/// ```
743///
David Tolnay461d98e2018-01-07 11:07:19 -0800744/// *This macro is available if Syn is built with both the `"parsing"` and
745/// `"printing"` features.*
746///
David Tolnaydace3f12018-01-06 21:33:29 -0800747/// # Example
748///
749/// The following helper function adds a bound `T: HeapSize` to every type
750/// parameter `T` in the input generics.
751///
752/// ```
753/// # #[macro_use]
754/// # extern crate syn;
755/// #
756/// # #[macro_use]
757/// # extern crate quote;
758/// #
759/// # use syn::{Generics, GenericParam};
760/// #
761/// // Add a bound `T: HeapSize` to every type parameter T.
762/// fn add_trait_bounds(mut generics: Generics) -> Generics {
763/// for param in &mut generics.params {
764/// if let GenericParam::Type(ref mut type_param) = *param {
765/// type_param.bounds.push(parse_quote!(HeapSize));
766/// }
767/// }
768/// generics
769/// }
770/// #
771/// # fn main() {}
772/// ```
773///
774/// # Panics
775///
776/// Panics if the tokens fail to parse as the expected syntax tree type. The
777/// caller is responsible for ensuring that the input tokens are syntactically
778/// valid.
David Tolnay01cc0202018-01-02 11:13:07 -0800779#[cfg(all(feature = "parsing", feature = "printing"))]
780#[macro_export]
781macro_rules! parse_quote {
782 ($($tt:tt)*) => {
783 ::std::result::Result::unwrap(
David Tolnaydace3f12018-01-06 21:33:29 -0800784 $crate::parse2(
David Tolnay01cc0202018-01-02 11:13:07 -0800785 ::std::convert::Into::into(
786 quote!($($tt)*))))
787 };
788}
789
David Tolnay3cfd1d32018-01-03 00:22:08 -0800790#[cfg(all(any(feature = "full", feature = "derive"), feature = "printing"))]
Alex Crichton259ee532017-07-14 06:51:02 -0700791struct TokensOrDefault<'a, T: 'a>(&'a Option<T>);
792
David Tolnay3cfd1d32018-01-03 00:22:08 -0800793#[cfg(all(any(feature = "full", feature = "derive"), feature = "printing"))]
Alex Crichton259ee532017-07-14 06:51:02 -0700794impl<'a, T> quote::ToTokens for TokensOrDefault<'a, T>
David Tolnay51382052017-12-27 13:46:21 -0500795where
796 T: quote::ToTokens + Default,
Alex Crichton259ee532017-07-14 06:51:02 -0700797{
798 fn to_tokens(&self, tokens: &mut quote::Tokens) {
799 match *self.0 {
800 Some(ref t) => t.to_tokens(tokens),
801 None => T::default().to_tokens(tokens),
802 }
803 }
804}