blob: 799f1c7019f8d4aeefc4681fcf26ba937f5ebee1 [file] [log] [blame]
David Tolnay55535012018-01-05 16:39:23 -08001// Copyright 2018 Syn Developers
2//
3// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
4// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
5// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
6// option. This file may not be copied, modified, or distributed
7// except according to those terms.
8
David Tolnay5e84e972018-01-05 17:51:06 -08009//! Syn is a parsing library for parsing a stream of Rust tokens into a syntax
10//! tree of Rust source code.
11//!
12//! Currently this library is geared toward the [custom derive] use case but
13//! contains some APIs that may be useful for Rust procedural macros more
14//! generally.
15//!
16//! [custom derive]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
17//!
18//! - **Data structures** — Syn provides a complete syntax tree that can
19//! represent any valid Rust source code. The syntax tree is rooted at
20//! [`syn::File`] which represents a full source file, but there are other
21//! entry points that may be useful to procedural macros including
22//! [`syn::Item`], [`syn::Expr`] and [`syn::Type`].
23//!
24//! - **Custom derives** — Of particular interest to custom derives is
25//! [`syn::DeriveInput`] which is any of the three legal input items to a
26//! derive macro. An example below shows using this type in a library that can
27//! derive implementations of a trait of your own.
28//!
29//! - **Parser combinators** — Parsing in Syn is built on a suite of public
30//! parser combinator macros that you can use for parsing any token-based
31//! syntax you dream up within a `functionlike!(...)` procedural macro. Every
32//! syntax tree node defined by Syn is individually parsable and may be used
33//! as a building block for custom syntaxes, or you may do it all yourself
34//! working from the most primitive tokens.
35//!
36//! - **Location information** — Every token parsed by Syn is associated with a
37//! `Span` that tracks line and column information back to the source of that
38//! token. These spans allow a procedural macro to display detailed error
39//! messages pointing to all the right places in the user's code. There is an
40//! example of this below.
41//!
42//! - **Feature flags** — Functionality is aggressively feature gated so your
43//! procedural macros enable only what they need, and do not pay in compile
44//! time for all the rest.
45//!
46//! [`syn::File`]: struct.File.html
47//! [`syn::Item`]: enum.Item.html
48//! [`syn::Expr`]: enum.Expr.html
49//! [`syn::Type`]: enum.Type.html
50//! [`syn::DeriveInput`]: struct.DeriveInput.html
51//!
52//! *Version requirement: Syn supports any compiler version back to Rust's very
53//! first support for procedural macros in Rust 1.15.0. Some features especially
54//! around error reporting are only available in newer compilers or on the
55//! nightly channel.*
56//!
57//! ## Example of a custom derive
58//!
59//! The canonical custom derive using Syn looks like this. We write an ordinary
60//! Rust function tagged with a `proc_macro_derive` attribute and the name of
61//! the trait we are deriving. Any time that derive appears in the user's code,
62//! the Rust compiler passes their data structure as tokens into our macro. We
63//! get to execute arbitrary Rust code to figure out what to do with those
64//! tokens, then hand some tokens back to the compiler to compile into the
65//! user's crate.
66//!
67//! [`TokenStream`]: https://doc.rust-lang.org/proc_macro/struct.TokenStream.html
68//!
69//! ```toml
70//! [dependencies]
David Tolnay87003d02018-05-20 19:45:13 -070071//! syn = "0.14"
72//! quote = "0.6"
David Tolnay5e84e972018-01-05 17:51:06 -080073//!
74//! [lib]
75//! proc-macro = true
76//! ```
77//!
78//! ```rust
79//! extern crate proc_macro;
80//! extern crate syn;
81//!
82//! #[macro_use]
83//! extern crate quote;
84//!
85//! use proc_macro::TokenStream;
86//! use syn::DeriveInput;
87//!
88//! # const IGNORE_TOKENS: &str = stringify! {
89//! #[proc_macro_derive(MyMacro)]
90//! # };
91//! pub fn my_macro(input: TokenStream) -> TokenStream {
92//! // Parse the input tokens into a syntax tree
93//! let input: DeriveInput = syn::parse(input).unwrap();
94//!
95//! // Build the output, possibly using quasi-quotation
96//! let expanded = quote! {
97//! // ...
98//! };
99//!
100//! // Hand the output tokens back to the compiler
101//! expanded.into()
102//! }
103//! #
104//! # fn main() {}
105//! ```
106//!
107//! The [`heapsize`] example directory shows a complete working Macros 1.1
108//! implementation of a custom derive. It works on any Rust compiler \>=1.15.0.
109//! The example derives a `HeapSize` trait which computes an estimate of the
110//! amount of heap memory owned by a value.
111//!
112//! [`heapsize`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize
113//!
114//! ```rust
115//! pub trait HeapSize {
116//! /// Total number of bytes of heap memory owned by `self`.
117//! fn heap_size_of_children(&self) -> usize;
118//! }
119//! ```
120//!
121//! The custom derive allows users to write `#[derive(HeapSize)]` on data
122//! structures in their program.
123//!
124//! ```rust
125//! # const IGNORE_TOKENS: &str = stringify! {
126//! #[derive(HeapSize)]
127//! # };
128//! struct Demo<'a, T: ?Sized> {
129//! a: Box<T>,
130//! b: u8,
131//! c: &'a str,
132//! d: String,
133//! }
134//! ```
135//!
136//! ## Spans and error reporting
137//!
138//! The [`heapsize2`] example directory is an extension of the `heapsize`
139//! example that demonstrates some of the hygiene and error reporting properties
140//! of Macros 2.0. This example currently requires a nightly Rust compiler
141//! \>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
142//!
143//! [`heapsize2`]: https://github.com/dtolnay/syn/tree/master/examples/heapsize2
144//!
145//! The token-based procedural macro API provides great control over where the
146//! compiler's error messages are displayed in user code. Consider the error the
147//! user sees if one of their field types does not implement `HeapSize`.
148//!
149//! ```rust
150//! # const IGNORE_TOKENS: &str = stringify! {
151//! #[derive(HeapSize)]
152//! # };
153//! struct Broken {
154//! ok: String,
155//! bad: std::thread::Thread,
156//! }
157//! ```
158//!
159//! In the Macros 1.1 string-based procedural macro world, the resulting error
160//! would point unhelpfully to the invocation of the derive macro and not to the
161//! actual problematic field.
162//!
163//! ```text
164//! error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
165//! --> src/main.rs:4:10
166//! |
167//! 4 | #[derive(HeapSize)]
168//! | ^^^^^^^^
169//! ```
170//!
171//! By tracking span information all the way through the expansion of a
172//! procedural macro as shown in the `heapsize2` example, token-based macros in
173//! Syn are able to trigger errors that directly pinpoint the source of the
174//! problem.
175//!
176//! ```text
177//! error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
178//! --> src/main.rs:7:5
179//! |
180//! 7 | bad: std::thread::Thread,
David Tolnayefff2ff2018-01-07 11:49:52 -0800181//! | ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
David Tolnay5e84e972018-01-05 17:51:06 -0800182//! ```
183//!
184//! ## Parsing a custom syntax using combinators
185//!
186//! The [`lazy-static`] example directory shows the implementation of a
187//! `functionlike!(...)` procedural macro in which the input tokens are parsed
188//! using [`nom`]-style parser combinators.
189//!
190//! [`lazy-static`]: https://github.com/dtolnay/syn/tree/master/examples/lazy-static
191//! [`nom`]: https://github.com/Geal/nom
192//!
193//! The example reimplements the popular `lazy_static` crate from crates.io as a
194//! procedural macro.
195//!
196//! ```
197//! # macro_rules! lazy_static {
198//! # ($($tt:tt)*) => {}
199//! # }
200//! #
201//! lazy_static! {
202//! static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
203//! }
204//! ```
205//!
206//! The implementation shows how to trigger custom warnings and error messages
207//! on the macro input.
208//!
209//! ```text
210//! warning: come on, pick a more creative name
211//! --> src/main.rs:10:16
212//! |
213//! 10 | static ref FOO: String = "lazy_static".to_owned();
214//! | ^^^
215//! ```
216//!
217//! ## Debugging
218//!
219//! When developing a procedural macro it can be helpful to look at what the
220//! generated code looks like. Use `cargo rustc -- -Zunstable-options
221//! --pretty=expanded` or the [`cargo expand`] subcommand.
222//!
David Tolnay324db2d2018-01-07 11:51:09 -0800223//! [`cargo expand`]: https://github.com/dtolnay/cargo-expand
David Tolnay5e84e972018-01-05 17:51:06 -0800224//!
225//! To show the expanded code for some crate that uses your procedural macro,
226//! run `cargo expand` from that crate. To show the expanded code for one of
227//! your own test cases, run `cargo expand --test the_test_case` where the last
228//! argument is the name of the test file without the `.rs` extension.
229//!
230//! This write-up by Brandon W Maister discusses debugging in more detail:
231//! [Debugging Rust's new Custom Derive system][debugging].
232//!
233//! [debugging]: https://quodlibetor.github.io/posts/debugging-rusts-new-custom-derive-system/
234//!
235//! ## Optional features
236//!
237//! Syn puts a lot of functionality behind optional features in order to
238//! optimize compile time for the most common use cases. The following features
239//! are available.
240//!
241//! - **`derive`** *(enabled by default)* — Data structures for representing the
242//! possible input to a custom derive, including structs and enums and types.
243//! - **`full`** — Data structures for representing the syntax tree of all valid
244//! Rust source code, including items and expressions.
245//! - **`parsing`** *(enabled by default)* — Ability to parse input tokens into
246//! a syntax tree node of a chosen type.
247//! - **`printing`** *(enabled by default)* — Ability to print a syntax tree
248//! node as tokens of Rust source code.
249//! - **`visit`** — Trait for traversing a syntax tree.
David Tolnay34981cf2018-01-06 16:22:35 -0800250//! - **`visit-mut`** — Trait for traversing and mutating in place a syntax
David Tolnay5e84e972018-01-05 17:51:06 -0800251//! tree.
252//! - **`fold`** — Trait for transforming an owned syntax tree.
253//! - **`clone-impls`** *(enabled by default)* — Clone impls for all syntax tree
254//! types.
255//! - **`extra-traits`** — Debug, Eq, PartialEq, Hash impls for all syntax tree
256//! types.
hcpl4b72a382018-04-04 14:50:24 +0300257//! - **`proc-macro`** *(enabled by default)* — Runtime dependency on the
258//! dynamic library libproc_macro from rustc toolchain.
David Tolnay5e84e972018-01-05 17:51:06 -0800259
David Tolnay4cf7db82018-01-07 15:22:01 -0800260// Syn types in rustdoc of other crates get linked to here.
David Tolnaye6ec9e02018-08-21 21:32:03 -0400261#![doc(html_root_url = "https://docs.rs/syn/0.14.9")]
David Tolnay34071ba2018-05-20 20:00:41 -0700262#![cfg_attr(feature = "cargo-clippy", deny(clippy, clippy_pedantic))]
David Tolnay34071ba2018-05-20 20:00:41 -0700263// Ignored clippy lints.
David Tolnay94d2b792018-04-29 12:26:10 -0700264#![cfg_attr(
265 feature = "cargo-clippy",
266 allow(
David Tolnay4831ac62018-08-30 21:04:16 -0700267 block_in_if_condition_stmt,
David Tolnay0cec3e62018-07-21 09:08:30 -0700268 const_static_lifetime,
David Tolnay24b079d2018-08-27 08:28:10 -0700269 cyclomatic_complexity,
David Tolnay0cec3e62018-07-21 09:08:30 -0700270 doc_markdown,
David Tolnay24b079d2018-08-27 08:28:10 -0700271 eval_order_dependence,
David Tolnay0cec3e62018-07-21 09:08:30 -0700272 large_enum_variant,
273 match_bool,
274 redundant_closure,
275 needless_pass_by_value,
276 redundant_field_names,
277 trivially_copy_pass_by_ref
David Tolnay94d2b792018-04-29 12:26:10 -0700278 )
279)]
David Tolnay34071ba2018-05-20 20:00:41 -0700280// Ignored clippy_pedantic lints.
281#![cfg_attr(
282 feature = "cargo-clippy",
283 allow(
David Tolnay0cec3e62018-07-21 09:08:30 -0700284 cast_possible_truncation,
285 cast_possible_wrap,
David Tolnayb1617752018-08-24 21:16:33 -0400286 empty_enum,
David Tolnay0cec3e62018-07-21 09:08:30 -0700287 if_not_else,
288 indexing_slicing,
289 items_after_statements,
David Tolnay151f92f2018-08-14 22:44:53 -0700290 shadow_unrelated,
David Tolnay0cec3e62018-07-21 09:08:30 -0700291 similar_names,
292 single_match_else,
293 stutter,
294 unseparated_literal_suffix,
295 use_self,
296 used_underscore_binding
David Tolnay34071ba2018-05-20 20:00:41 -0700297 )
298)]
David Tolnayad2836d2017-04-20 10:11:43 -0700299
David Tolnay278f9e32018-08-14 22:41:11 -0700300#[cfg(all(
301 not(all(target_arch = "wasm32", target_os = "unknown")),
302 feature = "proc-macro"
303))]
David Tolnay51382052017-12-27 13:46:21 -0500304extern crate proc_macro;
David Tolnay94d2b792018-04-29 12:26:10 -0700305extern crate proc_macro2;
David Tolnay570695e2017-06-03 16:15:13 -0700306extern crate unicode_xid;
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700307
David Tolnay1cf80912017-12-31 18:35:12 -0500308#[cfg(feature = "printing")]
David Tolnay87d0b442016-09-04 11:52:12 -0700309extern crate quote;
310
Alex Crichton62a0a592017-05-22 13:58:53 -0700311#[macro_use]
312mod macros;
313
David Tolnay852bff72018-08-27 08:24:02 -0700314#[cfg(feature = "parsing")]
315#[macro_use]
316mod group;
317
David Tolnayc5ab8c62017-12-26 16:43:39 -0500318#[macro_use]
David Tolnay32954ef2017-12-26 22:43:16 -0500319pub mod token;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500320
David Tolnay4fb71232018-08-25 23:14:50 -0400321mod ident;
322pub use ident::Ident;
David Tolnaye303b7c2018-05-20 16:46:35 -0700323
David Tolnay3cfd1d32018-01-03 00:22:08 -0800324#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700325mod attr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800326#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayaaadd782018-01-06 22:58:13 -0800327pub use attr::{AttrStyle, Attribute, Meta, MetaList, MetaNameValue, NestedMeta};
David Tolnay35161ff2016-09-03 11:33:15 -0700328
David Tolnay3cfd1d32018-01-03 00:22:08 -0800329#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf38cdf62016-09-23 19:07:09 -0700330mod data;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800331#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700332pub use data::{
333 Field, Fields, FieldsNamed, FieldsUnnamed, Variant, VisCrate, VisPublic, VisRestricted,
334 Visibility,
335};
David Tolnayf38cdf62016-09-23 19:07:09 -0700336
David Tolnay3cfd1d32018-01-03 00:22:08 -0800337#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700338mod expr;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800339#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700340pub use expr::{
David Tolnay02a9c6f2018-08-24 18:58:45 -0400341 Expr, ExprArray, ExprAssign, ExprAssignOp, ExprAsync, ExprBinary, ExprBlock, ExprBox,
342 ExprBreak, ExprCall, ExprCast, ExprClosure, ExprContinue, ExprField, ExprForLoop, ExprGroup,
343 ExprIf, ExprIfLet, ExprInPlace, ExprIndex, ExprLit, ExprLoop, ExprMacro, ExprMatch,
344 ExprMethodCall, ExprParen, ExprPath, ExprRange, ExprReference, ExprRepeat, ExprReturn,
345 ExprStruct, ExprTry, ExprTryBlock, ExprTuple, ExprType, ExprUnary, ExprUnsafe, ExprVerbatim,
346 ExprWhile, ExprWhileLet, ExprYield, Index, Member,
David Tolnayb57c8492018-05-05 00:32:04 -0700347};
Michael Layzell734adb42017-06-07 16:58:31 -0400348
349#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700350pub use expr::{
351 Arm, Block, FieldPat, FieldValue, GenericMethodArgument, Label, Local, MethodTurbofish, Pat,
352 PatBox, PatIdent, PatLit, PatMacro, PatPath, PatRange, PatRef, PatSlice, PatStruct, PatTuple,
353 PatTupleStruct, PatVerbatim, PatWild, RangeLimits, Stmt,
354};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700355
David Tolnay3cfd1d32018-01-03 00:22:08 -0800356#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700357mod generics;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800358#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700359pub use generics::{
360 BoundLifetimes, ConstParam, GenericParam, Generics, LifetimeDef, PredicateEq,
361 PredicateLifetime, PredicateType, TraitBound, TraitBoundModifier, TypeParam, TypeParamBound,
362 WhereClause, WherePredicate,
363};
David Tolnay278f9e32018-08-14 22:41:11 -0700364#[cfg(all(
365 any(feature = "full", feature = "derive"),
366 feature = "printing"
367))]
David Tolnayfd6bf5c2017-11-12 09:41:14 -0800368pub use generics::{ImplGenerics, Turbofish, TypeGenerics};
David Tolnay35161ff2016-09-03 11:33:15 -0700369
David Tolnayf38cdf62016-09-23 19:07:09 -0700370#[cfg(feature = "full")]
David Tolnayb79ee962016-09-04 09:39:20 -0700371mod item;
David Tolnayf38cdf62016-09-23 19:07:09 -0700372#[cfg(feature = "full")]
David Tolnayb57c8492018-05-05 00:32:04 -0700373pub use item::{
David Tolnay435c1782018-08-24 16:15:44 -0400374 ArgCaptured, ArgSelf, ArgSelfRef, FnArg, FnDecl, ForeignItem, ForeignItemFn, ForeignItemMacro,
375 ForeignItemStatic, ForeignItemType, ForeignItemVerbatim, ImplItem, ImplItemConst,
David Tolnaybb82ef02018-08-24 20:15:45 -0400376 ImplItemExistential, ImplItemMacro, ImplItemMethod, ImplItemType, ImplItemVerbatim, Item,
377 ItemConst, ItemEnum, ItemExistential, ItemExternCrate, ItemFn, ItemForeignMod, ItemImpl,
David Tolnayc6b04dd2018-08-30 23:22:51 -0700378 ItemMacro, ItemMacro2, ItemMod, ItemStatic, ItemStruct, ItemTrait, ItemTraitAlias, ItemType,
379 ItemUnion, ItemUse, ItemVerbatim, MethodSig, TraitItem, TraitItemConst, TraitItemMacro,
380 TraitItemMethod, TraitItemType, TraitItemVerbatim, UseGlob, UseGroup, UseName, UsePath,
381 UseRename, UseTree,
David Tolnayb57c8492018-05-05 00:32:04 -0700382};
David Tolnay35161ff2016-09-03 11:33:15 -0700383
David Tolnay631cb8c2016-11-10 17:16:41 -0800384#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700385mod file;
David Tolnay631cb8c2016-11-10 17:16:41 -0800386#[cfg(feature = "full")]
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700387pub use file::File;
David Tolnay631cb8c2016-11-10 17:16:41 -0800388
David Tolnay63e3dee2017-06-03 20:13:17 -0700389mod lifetime;
390pub use lifetime::Lifetime;
391
David Tolnay3cfd1d32018-01-03 00:22:08 -0800392#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700393mod lit;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800394#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700395pub use lit::{
396 FloatSuffix, IntSuffix, Lit, LitBool, LitByte, LitByteStr, LitChar, LitFloat, LitInt, LitStr,
397 LitVerbatim, StrStyle,
398};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700399
David Tolnay3cfd1d32018-01-03 00:22:08 -0800400#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf4bbbd92016-09-23 14:41:55 -0700401mod mac;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800402#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayab919512017-12-30 23:31:51 -0500403pub use mac::{Macro, MacroDelimiter};
David Tolnayf4bbbd92016-09-23 14:41:55 -0700404
David Tolnay3cfd1d32018-01-03 00:22:08 -0800405#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay0e837402016-12-22 17:25:55 -0500406mod derive;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800407#[cfg(feature = "derive")]
David Tolnaye3d41b72017-12-31 15:24:00 -0500408pub use derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
David Tolnayf38cdf62016-09-23 19:07:09 -0700409
David Tolnay3cfd1d32018-01-03 00:22:08 -0800410#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700411mod op;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800412#[cfg(any(feature = "full", feature = "derive"))]
David Tolnay3cb23a92016-10-07 23:02:21 -0700413pub use op::{BinOp, UnOp};
414
David Tolnay3cfd1d32018-01-03 00:22:08 -0800415#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb79ee962016-09-04 09:39:20 -0700416mod ty;
David Tolnay3cfd1d32018-01-03 00:22:08 -0800417#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700418pub use ty::{
419 Abi, BareFnArg, BareFnArgName, ReturnType, Type, TypeArray, TypeBareFn, TypeGroup,
420 TypeImplTrait, TypeInfer, TypeMacro, TypeNever, TypeParen, TypePath, TypePtr, TypeReference,
421 TypeSlice, TypeTraitObject, TypeTuple, TypeVerbatim,
422};
David Tolnay056de302018-01-05 14:29:05 -0800423
424#[cfg(any(feature = "full", feature = "derive"))]
425mod path;
David Tolnay278f9e32018-08-14 22:41:11 -0700426#[cfg(all(
427 any(feature = "full", feature = "derive"),
428 feature = "printing"
429))]
David Tolnay94d2b792018-04-29 12:26:10 -0700430pub use path::PathTokens;
David Tolnay056de302018-01-05 14:29:05 -0800431#[cfg(any(feature = "full", feature = "derive"))]
David Tolnayb57c8492018-05-05 00:32:04 -0700432pub use path::{
433 AngleBracketedGenericArguments, Binding, GenericArgument, ParenthesizedGenericArguments, Path,
434 PathArguments, PathSegment, QSelf,
435};
Alex Crichtonccbb45d2017-05-23 10:58:24 -0700436
David Tolnay1b752fb2017-12-26 21:41:39 -0500437#[cfg(feature = "parsing")]
David Tolnaydfc886b2018-01-06 08:03:09 -0800438pub mod buffer;
David Tolnay94d304f2018-08-30 23:43:53 -0700439#[cfg(feature = "parsing")]
440pub mod ext;
David Tolnay94d2b792018-04-29 12:26:10 -0700441pub mod punctuated;
David Tolnay3779bb72018-08-26 18:46:07 -0700442#[cfg(all(
443 any(feature = "full", feature = "derive"),
444 feature = "extra-traits"
445))]
David Tolnaye0824032017-12-27 15:25:56 -0500446mod tt;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500447
David Tolnaye83ef5a2018-01-11 15:18:36 -0800448// Not public API except the `parse_quote!` macro.
David Tolnay491680a2018-01-23 00:34:40 -0800449#[cfg(feature = "parsing")]
David Tolnaye83ef5a2018-01-11 15:18:36 -0800450#[doc(hidden)]
451pub mod parse_quote;
452
David Tolnay4d942b42018-01-02 22:14:04 -0800453#[cfg(all(feature = "parsing", feature = "printing"))]
David Tolnayf790b612017-12-31 18:46:57 -0500454pub mod spanned;
455
Nika Layzella6f46c42017-10-26 15:26:16 -0400456mod gen {
David Tolnayded2d682018-01-06 18:53:53 -0800457 /// Syntax tree traversal to walk a shared borrow of a syntax tree.
458 ///
459 /// Each method of the [`Visit`] trait is a hook that can be overridden to
460 /// customize the behavior when visiting the corresponding type of node. By
461 /// default, every method recursively visits the substructure of the input
462 /// by invoking the right visitor method of each of its fields.
463 ///
464 /// [`Visit`]: trait.Visit.html
465 ///
466 /// ```rust
467 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
468 /// #
469 /// pub trait Visit<'ast> {
470 /// /* ... */
471 ///
472 /// fn visit_expr_binary(&mut self, node: &'ast ExprBinary) {
473 /// for attr in &node.attrs {
474 /// self.visit_attribute(attr);
475 /// }
476 /// self.visit_expr(&*node.left);
477 /// self.visit_bin_op(&node.op);
478 /// self.visit_expr(&*node.right);
479 /// }
480 ///
481 /// /* ... */
482 /// # fn visit_attribute(&mut self, node: &'ast Attribute);
483 /// # fn visit_expr(&mut self, node: &'ast Expr);
484 /// # fn visit_bin_op(&mut self, node: &'ast BinOp);
485 /// }
486 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800487 ///
488 /// *This module is available if Syn is built with the `"visit"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400489 #[cfg(feature = "visit")]
490 pub mod visit;
David Tolnay55337722016-09-11 12:58:56 -0700491
David Tolnayded2d682018-01-06 18:53:53 -0800492 /// Syntax tree traversal to mutate an exclusive borrow of a syntax tree in
493 /// place.
494 ///
495 /// Each method of the [`VisitMut`] trait is a hook that can be overridden
496 /// to customize the behavior when mutating the corresponding type of node.
497 /// By default, every method recursively visits the substructure of the
498 /// input by invoking the right visitor method of each of its fields.
499 ///
500 /// [`VisitMut`]: trait.VisitMut.html
501 ///
502 /// ```rust
503 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
504 /// #
505 /// pub trait VisitMut {
506 /// /* ... */
507 ///
508 /// fn visit_expr_binary_mut(&mut self, node: &mut ExprBinary) {
509 /// for attr in &mut node.attrs {
510 /// self.visit_attribute_mut(attr);
511 /// }
512 /// self.visit_expr_mut(&mut *node.left);
513 /// self.visit_bin_op_mut(&mut node.op);
514 /// self.visit_expr_mut(&mut *node.right);
515 /// }
516 ///
517 /// /* ... */
518 /// # fn visit_attribute_mut(&mut self, node: &mut Attribute);
519 /// # fn visit_expr_mut(&mut self, node: &mut Expr);
520 /// # fn visit_bin_op_mut(&mut self, node: &mut BinOp);
521 /// }
522 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800523 ///
524 /// *This module is available if Syn is built with the `"visit-mut"`
525 /// feature.*
David Tolnay9df02c42018-01-06 13:52:48 -0800526 #[cfg(feature = "visit-mut")]
Nika Layzella6f46c42017-10-26 15:26:16 -0400527 pub mod visit_mut;
Nika Layzell27726662017-10-24 23:16:35 -0400528
David Tolnayded2d682018-01-06 18:53:53 -0800529 /// Syntax tree traversal to transform the nodes of an owned syntax tree.
530 ///
531 /// Each method of the [`Fold`] trait is a hook that can be overridden to
532 /// customize the behavior when transforming the corresponding type of node.
533 /// By default, every method recursively visits the substructure of the
534 /// input by invoking the right visitor method of each of its fields.
535 ///
536 /// [`Fold`]: trait.Fold.html
537 ///
538 /// ```rust
539 /// # use syn::{Attribute, BinOp, Expr, ExprBinary};
540 /// #
541 /// pub trait Fold {
542 /// /* ... */
543 ///
544 /// fn fold_expr_binary(&mut self, node: ExprBinary) -> ExprBinary {
545 /// ExprBinary {
546 /// attrs: node.attrs
547 /// .into_iter()
548 /// .map(|attr| self.fold_attribute(attr))
549 /// .collect(),
550 /// left: Box::new(self.fold_expr(*node.left)),
551 /// op: self.fold_bin_op(node.op),
552 /// right: Box::new(self.fold_expr(*node.right)),
553 /// }
554 /// }
555 ///
556 /// /* ... */
557 /// # fn fold_attribute(&mut self, node: Attribute) -> Attribute;
558 /// # fn fold_expr(&mut self, node: Expr) -> Expr;
559 /// # fn fold_bin_op(&mut self, node: BinOp) -> BinOp;
560 /// }
561 /// ```
David Tolnay461d98e2018-01-07 11:07:19 -0800562 ///
563 /// *This module is available if Syn is built with the `"fold"` feature.*
Nika Layzella6f46c42017-10-26 15:26:16 -0400564 #[cfg(feature = "fold")]
565 pub mod fold;
David Tolnayf60f4262017-12-28 19:17:58 -0500566
David Tolnay0a0d78c2018-01-05 15:24:01 -0800567 #[cfg(any(feature = "full", feature = "derive"))]
David Tolnayf60f4262017-12-28 19:17:58 -0500568 #[path = "../gen_helper.rs"]
569 mod helper;
Nika Layzella6f46c42017-10-26 15:26:16 -0400570}
571pub use gen::*;
gnzlbg9ae88d82017-01-26 20:45:17 +0100572
David Tolnay456c9822018-08-25 08:09:46 -0400573// Not public API.
574#[doc(hidden)]
575pub mod export;
576
David Tolnay776f8e02018-08-24 22:32:10 -0400577#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400578mod lookahead;
579
580#[cfg(feature = "parsing")]
581pub mod parse;
582
David Tolnay776f8e02018-08-24 22:32:10 -0400583mod span;
584
David Tolnay64023912018-08-31 09:51:12 -0700585#[cfg(all(
586 any(feature = "full", feature = "derive"),
587 feature = "printing"
588))]
589mod print;
590
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700591////////////////////////////////////////////////////////////////////////////////
592
David Tolnay55337722016-09-11 12:58:56 -0700593#[cfg(feature = "parsing")]
David Tolnayb6254182018-08-25 08:44:54 -0400594mod error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500595#[cfg(feature = "parsing")]
David Tolnayad4b2472018-08-25 08:25:24 -0400596use error::Error;
David Tolnayc5ab8c62017-12-26 16:43:39 -0500597
David Tolnayccab0be2018-01-06 22:24:47 -0800598/// Parse tokens of source code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700599///
600/// This is preferred over parsing a string because tokens are able to preserve
601/// information about where in the user's code they were originally written (the
602/// "span" of the token), possibly allowing the compiler to produce better error
603/// messages.
604///
David Tolnayccab0be2018-01-06 22:24:47 -0800605/// This function parses a `proc_macro::TokenStream` which is the type used for
606/// interop with the compiler in a procedural macro. To parse a
607/// `proc_macro2::TokenStream`, use [`syn::parse2`] instead.
608///
609/// [`syn::parse2`]: fn.parse2.html
610///
hcpl4b72a382018-04-04 14:50:24 +0300611/// *This function is available if Syn is built with both the `"parsing"` and
612/// `"proc-macro"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800613///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700614/// # Examples
615///
David Tolnaybcf26022017-12-25 22:10:52 -0500616/// ```rust
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700617/// extern crate proc_macro;
618/// use proc_macro::TokenStream;
619///
620/// extern crate syn;
621///
622/// #[macro_use]
623/// extern crate quote;
624///
625/// use syn::DeriveInput;
626///
David Tolnaybcf26022017-12-25 22:10:52 -0500627/// # const IGNORE_TOKENS: &str = stringify! {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700628/// #[proc_macro_derive(MyMacro)]
David Tolnaybcf26022017-12-25 22:10:52 -0500629/// # };
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700630/// pub fn my_macro(input: TokenStream) -> TokenStream {
631/// // Parse the tokens into a syntax tree
632/// let ast: DeriveInput = syn::parse(input).unwrap();
633///
634/// // Build the output, possibly using quasi-quotation
635/// let expanded = quote! {
636/// /* ... */
637/// };
638///
David Tolnaybcf26022017-12-25 22:10:52 -0500639/// // Convert into a token stream and return it
640/// expanded.into()
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700641/// }
David Tolnaybcf26022017-12-25 22:10:52 -0500642/// #
643/// # fn main() {}
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700644/// ```
David Tolnay278f9e32018-08-14 22:41:11 -0700645#[cfg(all(
646 not(all(target_arch = "wasm32", target_os = "unknown")),
647 feature = "parsing",
648 feature = "proc-macro"
649))]
David Tolnaye82a2b12018-08-30 16:31:10 -0700650pub fn parse<T: parse::Parse>(tokens: proc_macro::TokenStream) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700651 parse::Parser::parse(T::parse, tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700652}
653
David Tolnayccab0be2018-01-06 22:24:47 -0800654/// Parse a proc-macro2 token stream into the chosen syntax tree node.
655///
656/// This function parses a `proc_macro2::TokenStream` which is commonly useful
657/// when the input comes from a node of the Syn syntax tree, for example the tts
658/// of a [`Macro`] node. When in a procedural macro parsing the
659/// `proc_macro::TokenStream` provided by the compiler, use [`syn::parse`]
660/// instead.
661///
662/// [`Macro`]: struct.Macro.html
663/// [`syn::parse`]: fn.parse.html
David Tolnay461d98e2018-01-07 11:07:19 -0800664///
665/// *This function is available if Syn is built with the `"parsing"` feature.*
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700666#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700667pub fn parse2<T: parse::Parse>(tokens: proc_macro2::TokenStream) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700668 parse::Parser::parse2(T::parse, tokens)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700669}
Alex Crichton954046c2017-05-30 21:49:42 -0700670
David Tolnayccab0be2018-01-06 22:24:47 -0800671/// Parse a string of Rust code into the chosen syntax tree node.
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700672///
David Tolnay461d98e2018-01-07 11:07:19 -0800673/// *This function is available if Syn is built with the `"parsing"` feature.*
674///
David Tolnay3f5b06f2018-01-11 21:02:00 -0800675/// # Hygiene
676///
677/// Every span in the resulting syntax tree will be set to resolve at the macro
678/// call site.
679///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700680/// # Examples
681///
682/// ```rust
683/// extern crate syn;
684/// #
David Tolnay9174b972017-11-09 22:27:50 -0800685/// #
686/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700687///
688/// use syn::Expr;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700689///
690/// fn run() -> Result<()> {
691/// let code = "assert_eq!(u8::max_value(), 255)";
692/// let expr = syn::parse_str::<Expr>(code)?;
693/// println!("{:#?}", expr);
694/// Ok(())
695/// }
696/// #
697/// # fn main() { run().unwrap() }
698/// ```
699#[cfg(feature = "parsing")]
David Tolnaye82a2b12018-08-30 16:31:10 -0700700pub fn parse_str<T: parse::Parse>(s: &str) -> Result<T, Error> {
David Tolnay80a914f2018-08-30 23:49:53 -0700701 parse::Parser::parse_str(T::parse, s)
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700702}
Alex Crichton954046c2017-05-30 21:49:42 -0700703
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700704// FIXME the name parse_file makes it sound like you might pass in a path to a
705// file, rather than the content.
706/// Parse the content of a file of Rust code.
707///
708/// This is different from `syn::parse_str::<File>(content)` in two ways:
709///
710/// - It discards a leading byte order mark `\u{FEFF}` if the file has one.
711/// - It preserves the shebang line of the file, such as `#!/usr/bin/env rustx`.
712///
713/// If present, either of these would be an error using `from_str`.
714///
Christopher Serr9727ef22018-02-03 16:42:12 +0100715/// *This function is available if Syn is built with the `"parsing"` and `"full"` features.*
David Tolnay461d98e2018-01-07 11:07:19 -0800716///
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700717/// # Examples
718///
719/// ```rust,no_run
720/// extern crate syn;
721/// #
David Tolnay9174b972017-11-09 22:27:50 -0800722/// #
723/// # type Result<T> = std::result::Result<T, Box<std::error::Error>>;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700724///
725/// use std::fs::File;
726/// use std::io::Read;
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700727///
728/// fn run() -> Result<()> {
729/// let mut file = File::open("path/to/code.rs")?;
730/// let mut content = String::new();
731/// file.read_to_string(&mut content)?;
732///
733/// let ast = syn::parse_file(&content)?;
734/// if let Some(shebang) = ast.shebang {
735/// println!("{}", shebang);
736/// }
737/// println!("{} items", ast.items.len());
738///
739/// Ok(())
740/// }
741/// #
742/// # fn main() { run().unwrap() }
743/// ```
744#[cfg(all(feature = "parsing", feature = "full"))]
David Tolnayad4b2472018-08-25 08:25:24 -0400745pub fn parse_file(mut content: &str) -> Result<File, Error> {
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700746 // Strip the BOM if it is present
747 const BOM: &'static str = "\u{feff}";
748 if content.starts_with(BOM) {
749 content = &content[BOM.len()..];
David Tolnay35161ff2016-09-03 11:33:15 -0700750 }
Michael Layzell5e107ff2017-01-24 19:58:39 -0500751
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700752 let mut shebang = None;
753 if content.starts_with("#!") && !content.starts_with("#![") {
754 if let Some(idx) = content.find('\n') {
755 shebang = Some(content[..idx].to_string());
756 content = &content[idx..];
757 } else {
758 shebang = Some(content.to_string());
759 content = "";
760 }
Alex Crichton954046c2017-05-30 21:49:42 -0700761 }
David Tolnay0a8972b2017-02-27 02:10:01 -0800762
David Tolnayc7a5d3d2017-06-04 12:11:05 -0700763 let mut file: File = parse_str(content)?;
764 file.shebang = shebang;
765 Ok(file)
Michael Layzell5e107ff2017-01-24 19:58:39 -0500766}
Alex Crichton259ee532017-07-14 06:51:02 -0700767
David Tolnayd641ee32018-08-30 17:11:09 -0700768/// Parse the input TokenStream of a macro, triggering a compile error if the
769/// tokens fail to parse.
770///
771/// # Intended usage
772///
773/// ```rust
774/// # extern crate proc_macro;
775/// # extern crate syn;
776/// #
777/// use proc_macro::TokenStream;
778/// use syn::parse_macro_input;
779/// use syn::parse::{Parse, ParseStream, Result};
780///
781/// struct MyMacroInput {
782/// /* ... */
783/// }
784///
785/// impl Parse for MyMacroInput {
786/// fn parse(input: ParseStream) -> Result<Self> {
787/// /* ... */
788/// # Ok(MyMacroInput {})
789/// }
790/// }
791///
792/// # const IGNORE: &str = stringify! {
793/// #[proc_macro]
794/// # };
795/// pub fn my_macro(tokens: TokenStream) -> TokenStream {
796/// let input = parse_macro_input!(tokens as MyMacroInput);
797///
798/// /* ... */
799/// # "".parse().unwrap()
800/// }
801/// #
802/// # fn main() {}
803/// ```
804#[cfg(feature = "proc-macro")]
805#[macro_export]
806macro_rules! parse_macro_input {
807 ($tokenstream:ident as $ty:ty) => {
David Tolnay02465352018-08-30 23:38:00 -0700808 match $crate::parse::<$ty>($tokenstream) {
David Tolnayd641ee32018-08-30 17:11:09 -0700809 $crate::export::Ok(data) => data,
810 $crate::export::Err(err) => {
811 return $crate::export::TokenStream::from(err.into_compile_error());
812 }
813 };
814 };
815}